[==========] Running 8 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 6 tests from LeaderRebalancerTest
[ RUN      ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260709 12:08:12.239421 11318 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.11.13.190:33281
I20260709 12:08:12.241113 11318 env_posix.cc:2285] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260709 12:08:12.242357 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:08:12.263089 11325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:12.263121 11324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:12.266084 11318 server_base.cc:1061] running on GCE node
W20260709 12:08:12.263657 11327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:13.531853 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:08:13.532111 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:08:13.532375 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783598893532350 us; error 0 us; skew 500 ppm
I20260709 12:08:13.533308 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:08:13.539304 11318 webserver.cc:533] Webserver started at http://127.11.13.190:38659/ using document root <none> and password file <none>
I20260709 12:08:13.540289 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:08:13.540503 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:08:13.541105 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:08:13.547695 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598892056905-11318-0/minicluster-data/master-0-root/instance:
uuid: "eef8fe4a3fbf4f829af15fb71d14c041"
format_stamp: "Formatted at 2026-07-09 12:08:13 on dist-test-slave-zqnl"
I20260709 12:08:13.556684 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.008s	user 0.009s	sys 0.000s
I20260709 12:08:13.563292 11333 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:13.564666 11318 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 12:08:13.565097 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598892056905-11318-0/minicluster-data/master-0-root
uuid: "eef8fe4a3fbf4f829af15fb71d14c041"
format_stamp: "Formatted at 2026-07-09 12:08:13 on dist-test-slave-zqnl"
I20260709 12:08:13.565443 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598892056905-11318-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598892056905-11318-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598892056905-11318-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:08:13.641546 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:08:13.643677 11318 env_posix.cc:2285] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260709 12:08:13.644503 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:08:13.729035 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.190:33281
I20260709 12:08:13.729126 11384 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.190:33281 every 8 connection(s)
I20260709 12:08:13.738098 11385 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:13.763073 11385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041: Bootstrap starting.
I20260709 12:08:13.771341 11385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:13.773432 11385 log.cc:826] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041: Log is configured to *not* fsync() on all Append() calls
I20260709 12:08:13.778844 11385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041: No bootstrap required, opened a new log
I20260709 12:08:13.801939 11385 raft_consensus.cc:359] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef8fe4a3fbf4f829af15fb71d14c041" member_type: VOTER }
I20260709 12:08:13.802554 11385 raft_consensus.cc:385] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:13.802781 11385 raft_consensus.cc:740] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eef8fe4a3fbf4f829af15fb71d14c041, State: Initialized, Role: FOLLOWER
I20260709 12:08:13.803634 11385 consensus_queue.cc:260] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef8fe4a3fbf4f829af15fb71d14c041" member_type: VOTER }
I20260709 12:08:13.804172 11385 raft_consensus.cc:399] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:08:13.804661 11385 raft_consensus.cc:493] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:08:13.805001 11385 raft_consensus.cc:3060] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:13.810070 11385 raft_consensus.cc:515] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef8fe4a3fbf4f829af15fb71d14c041" member_type: VOTER }
I20260709 12:08:13.810879 11385 leader_election.cc:304] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eef8fe4a3fbf4f829af15fb71d14c041; no voters: 
I20260709 12:08:13.812817 11385 leader_election.cc:290] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:08:13.813133 11388 raft_consensus.cc:2804] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:13.815387 11388 raft_consensus.cc:697] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041 [term 1 LEADER]: Becoming Leader. State: Replica: eef8fe4a3fbf4f829af15fb71d14c041, State: Running, Role: LEADER
I20260709 12:08:13.816396 11388 consensus_queue.cc:237] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef8fe4a3fbf4f829af15fb71d14c041" member_type: VOTER }
I20260709 12:08:13.817494 11385 sys_catalog.cc:565] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:08:13.828377 11390 sys_catalog.cc:455] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eef8fe4a3fbf4f829af15fb71d14c041. Latest consensus state: current_term: 1 leader_uuid: "eef8fe4a3fbf4f829af15fb71d14c041" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef8fe4a3fbf4f829af15fb71d14c041" member_type: VOTER } }
I20260709 12:08:13.828284 11389 sys_catalog.cc:455] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eef8fe4a3fbf4f829af15fb71d14c041" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eef8fe4a3fbf4f829af15fb71d14c041" member_type: VOTER } }
I20260709 12:08:13.829154 11389 sys_catalog.cc:458] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041 [sys.catalog]: This master's current role is: LEADER
I20260709 12:08:13.829167 11390 sys_catalog.cc:458] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041 [sys.catalog]: This master's current role is: LEADER
I20260709 12:08:13.833261 11396 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:08:13.848184 11396 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:08:13.850850 11318 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:08:13.868261 11396 catalog_manager.cc:1383] Generated new cluster ID: 13c1ef4987f0484a98f4eb7d022a9995
I20260709 12:08:13.868635 11396 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:08:13.893575 11396 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:08:13.895877 11396 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:08:13.914525 11396 catalog_manager.cc:6070] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041: Generated new TSK 0
I20260709 12:08:13.916478 11396 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:08:13.982761 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:08:13.992508 11407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:13.994305 11408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:13.998667 11318 server_base.cc:1061] running on GCE node
W20260709 12:08:13.999456 11410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:14.000376 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:08:14.000576 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:08:14.000815 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783598894000772 us; error 0 us; skew 500 ppm
I20260709 12:08:14.001466 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:08:14.007270 11318 webserver.cc:533] Webserver started at http://127.11.13.129:35671/ using document root <none> and password file <none>
I20260709 12:08:14.007956 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:08:14.008180 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:08:14.008536 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:08:14.009986 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598892056905-11318-0/minicluster-data/ts-0-root/instance:
uuid: "e65f179f7b664838b6770533692da7aa"
format_stamp: "Formatted at 2026-07-09 12:08:14 on dist-test-slave-zqnl"
I20260709 12:08:14.016960 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.001s
I20260709 12:08:14.022485 11415 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:14.023495 11318 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.003s	sys 0.000s
I20260709 12:08:14.023826 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598892056905-11318-0/minicluster-data/ts-0-root
uuid: "e65f179f7b664838b6770533692da7aa"
format_stamp: "Formatted at 2026-07-09 12:08:14 on dist-test-slave-zqnl"
I20260709 12:08:14.024154 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598892056905-11318-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598892056905-11318-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598892056905-11318-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:08:14.055332 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:08:14.056792 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:08:14.059842 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:08:14.065079 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:08:14.065335 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:14.065699 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:08:14.065869 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:14.185412 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.129:35345
I20260709 12:08:14.185487 11477 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.129:35345 every 8 connection(s)
I20260709 12:08:14.194398 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:08:14.211041 11482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:14.214386 11483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:14.224699 11485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:14.227154 11318 server_base.cc:1061] running on GCE node
I20260709 12:08:14.228088 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:08:14.228336 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:08:14.228560 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783598894228540 us; error 0 us; skew 500 ppm
I20260709 12:08:14.229357 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:08:14.233294 11478 heartbeater.cc:344] Connected to a master server at 127.11.13.190:33281
I20260709 12:08:14.233844 11478 heartbeater.cc:461] Registering TS with master...
I20260709 12:08:14.234014 11318 webserver.cc:533] Webserver started at http://127.11.13.130:43743/ using document root <none> and password file <none>
I20260709 12:08:14.234757 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:08:14.235126 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:08:14.235268 11478 heartbeater.cc:507] Master 127.11.13.190:33281 requested a full tablet report, sending...
I20260709 12:08:14.235489 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:08:14.236900 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598892056905-11318-0/minicluster-data/ts-1-root/instance:
uuid: "9a8e2255342c492eb37a3b0daee59dfb"
format_stamp: "Formatted at 2026-07-09 12:08:14 on dist-test-slave-zqnl"
I20260709 12:08:14.239444 11350 ts_manager.cc:194] Registered new tserver with Master: e65f179f7b664838b6770533692da7aa (127.11.13.129:35345)
I20260709 12:08:14.243052 11350 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59550
I20260709 12:08:14.244976 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.005s	sys 0.004s
I20260709 12:08:14.251703 11490 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:14.252784 11318 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.002s	sys 0.002s
I20260709 12:08:14.253175 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598892056905-11318-0/minicluster-data/ts-1-root
uuid: "9a8e2255342c492eb37a3b0daee59dfb"
format_stamp: "Formatted at 2026-07-09 12:08:14 on dist-test-slave-zqnl"
I20260709 12:08:14.253582 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598892056905-11318-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598892056905-11318-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598892056905-11318-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:08:14.292932 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:08:14.294700 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:08:14.296444 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:08:14.300211 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:08:14.300483 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:14.300767 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:08:14.300977 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:14.393952 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.130:34537
I20260709 12:08:14.394317 11552 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.130:34537 every 8 connection(s)
I20260709 12:08:14.405022 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:08:14.414337 11556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:14.416241 11553 heartbeater.cc:344] Connected to a master server at 127.11.13.190:33281
I20260709 12:08:14.416702 11553 heartbeater.cc:461] Registering TS with master...
I20260709 12:08:14.417788 11553 heartbeater.cc:507] Master 127.11.13.190:33281 requested a full tablet report, sending...
W20260709 12:08:14.426210 11557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:14.427582 11318 server_base.cc:1061] running on GCE node
I20260709 12:08:14.428186 11350 ts_manager.cc:194] Registered new tserver with Master: 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537)
W20260709 12:08:14.429828 11559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:14.430069 11350 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59560
I20260709 12:08:14.431144 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:08:14.431488 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:08:14.431705 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783598894431689 us; error 0 us; skew 500 ppm
I20260709 12:08:14.432538 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:08:14.435701 11318 webserver.cc:533] Webserver started at http://127.11.13.131:45487/ using document root <none> and password file <none>
I20260709 12:08:14.436384 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:08:14.436666 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:08:14.437018 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:08:14.439033 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598892056905-11318-0/minicluster-data/ts-2-root/instance:
uuid: "31512bd2b8c74ab89e184a85332e6f9d"
format_stamp: "Formatted at 2026-07-09 12:08:14 on dist-test-slave-zqnl"
I20260709 12:08:14.445889 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:08:14.450585 11564 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:14.451541 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.001s	sys 0.003s
I20260709 12:08:14.451902 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598892056905-11318-0/minicluster-data/ts-2-root
uuid: "31512bd2b8c74ab89e184a85332e6f9d"
format_stamp: "Formatted at 2026-07-09 12:08:14 on dist-test-slave-zqnl"
I20260709 12:08:14.452334 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598892056905-11318-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598892056905-11318-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1783598892056905-11318-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:08:14.487432 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:08:14.489126 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:08:14.491473 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:08:14.494553 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:08:14.494815 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:14.495062 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:08:14.495250 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:14.594859 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.131:37547
I20260709 12:08:14.594947 11626 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.131:37547 every 8 connection(s)
I20260709 12:08:14.615252 11627 heartbeater.cc:344] Connected to a master server at 127.11.13.190:33281
I20260709 12:08:14.615667 11627 heartbeater.cc:461] Registering TS with master...
I20260709 12:08:14.616647 11627 heartbeater.cc:507] Master 127.11.13.190:33281 requested a full tablet report, sending...
I20260709 12:08:14.619071 11350 ts_manager.cc:194] Registered new tserver with Master: 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:14.619994 11318 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.017723488s
I20260709 12:08:14.620759 11350 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59572
I20260709 12:08:14.678354 11350 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59578:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:08:14.682720 11350 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:08:14.812198 11592 tablet_service.cc:1511] Processing CreateTablet for tablet f2fc2c80a37c43938dcb1dbaeacc2cc5 (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:08:14.812480 11590 tablet_service.cc:1511] Processing CreateTablet for tablet 44710cf3b4a748969c249dcbeccef11d (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:08:14.814545 11592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2fc2c80a37c43938dcb1dbaeacc2cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.814966 11585 tablet_service.cc:1511] Processing CreateTablet for tablet 7ee33eb902604730a36d8bfe692dc297 (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:08:14.815280 11589 tablet_service.cc:1511] Processing CreateTablet for tablet 7bbcb6cf7a2e4076b40fbde18996ddce (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:08:14.816061 11590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44710cf3b4a748969c249dcbeccef11d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.816834 11589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bbcb6cf7a2e4076b40fbde18996ddce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.819123 11591 tablet_service.cc:1511] Processing CreateTablet for tablet c7f41683de3849cfb6a5e4c93161ce46 (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:08:14.819553 11588 tablet_service.cc:1511] Processing CreateTablet for tablet 071aa019eff14096b27414993b8e9773 (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:08:14.820694 11591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7f41683de3849cfb6a5e4c93161ce46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.817668 11587 tablet_service.cc:1511] Processing CreateTablet for tablet af01620b86384cce826e034c1cf541fa (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:08:14.822069 11586 tablet_service.cc:1511] Processing CreateTablet for tablet 70c6dea7ace4433a91c235307de05d74 (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:08:14.826901 11584 tablet_service.cc:1511] Processing CreateTablet for tablet 53e6057698b843b18fff49462dc3679e (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:08:14.829566 11585 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ee33eb902604730a36d8bfe692dc297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.835038 11588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 071aa019eff14096b27414993b8e9773. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:08:14.838464 11398 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:08:14.839274 11586 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70c6dea7ace4433a91c235307de05d74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.840476 11584 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53e6057698b843b18fff49462dc3679e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.842514 11587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af01620b86384cce826e034c1cf541fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.865751 11518 tablet_service.cc:1511] Processing CreateTablet for tablet f2fc2c80a37c43938dcb1dbaeacc2cc5 (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:08:14.874668 11512 tablet_service.cc:1511] Processing CreateTablet for tablet 70c6dea7ace4433a91c235307de05d74 (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:08:14.874761 11516 tablet_service.cc:1511] Processing CreateTablet for tablet 44710cf3b4a748969c249dcbeccef11d (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:08:14.870107 11513 tablet_service.cc:1511] Processing CreateTablet for tablet af01620b86384cce826e034c1cf541fa (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:08:14.877786 11517 tablet_service.cc:1511] Processing CreateTablet for tablet c7f41683de3849cfb6a5e4c93161ce46 (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:08:14.879952 11514 tablet_service.cc:1511] Processing CreateTablet for tablet 071aa019eff14096b27414993b8e9773 (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:08:14.874790 11515 tablet_service.cc:1511] Processing CreateTablet for tablet 7bbcb6cf7a2e4076b40fbde18996ddce (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:08:14.883620 11511 tablet_service.cc:1511] Processing CreateTablet for tablet 7ee33eb902604730a36d8bfe692dc297 (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:08:14.889936 11509 tablet_service.cc:1511] Processing CreateTablet for tablet 53e6057698b843b18fff49462dc3679e (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:08:14.887894 11518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2fc2c80a37c43938dcb1dbaeacc2cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.894665 11516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44710cf3b4a748969c249dcbeccef11d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.896622 11515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bbcb6cf7a2e4076b40fbde18996ddce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.900729 11517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7f41683de3849cfb6a5e4c93161ce46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.901850 11511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ee33eb902604730a36d8bfe692dc297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.914093 11512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70c6dea7ace4433a91c235307de05d74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.911262 11441 tablet_service.cc:1511] Processing CreateTablet for tablet 44710cf3b4a748969c249dcbeccef11d (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:08:14.921799 11437 tablet_service.cc:1511] Processing CreateTablet for tablet 70c6dea7ace4433a91c235307de05d74 (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:08:14.926856 11442 tablet_service.cc:1511] Processing CreateTablet for tablet c7f41683de3849cfb6a5e4c93161ce46 (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:08:14.924288 11509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53e6057698b843b18fff49462dc3679e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.915373 11440 tablet_service.cc:1511] Processing CreateTablet for tablet 7bbcb6cf7a2e4076b40fbde18996ddce (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:08:14.923832 11443 tablet_service.cc:1511] Processing CreateTablet for tablet f2fc2c80a37c43938dcb1dbaeacc2cc5 (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:08:14.933058 11437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70c6dea7ace4433a91c235307de05d74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.941320 11435 tablet_service.cc:1511] Processing CreateTablet for tablet 53e6057698b843b18fff49462dc3679e (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:08:14.938313 11513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af01620b86384cce826e034c1cf541fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.929044 11438 tablet_service.cc:1511] Processing CreateTablet for tablet af01620b86384cce826e034c1cf541fa (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:08:14.938338 11439 tablet_service.cc:1511] Processing CreateTablet for tablet 071aa019eff14096b27414993b8e9773 (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:08:14.948385 11436 tablet_service.cc:1511] Processing CreateTablet for tablet 7ee33eb902604730a36d8bfe692dc297 (DEFAULT_TABLE table=test-workload [id=164198ea3923490eae517784f8312827]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:08:14.945425 11441 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44710cf3b4a748969c249dcbeccef11d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.951704 11435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53e6057698b843b18fff49462dc3679e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.952828 11514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 071aa019eff14096b27414993b8e9773. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.964913 11442 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c7f41683de3849cfb6a5e4c93161ce46. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.967404 11438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet af01620b86384cce826e034c1cf541fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.969547 11443 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f2fc2c80a37c43938dcb1dbaeacc2cc5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.982414 11436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7ee33eb902604730a36d8bfe692dc297. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:14.984246 11645 tablet_bootstrap.cc:492] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d: Bootstrap starting.
I20260709 12:08:15.026846 11440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7bbcb6cf7a2e4076b40fbde18996ddce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:15.040581 11439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 071aa019eff14096b27414993b8e9773. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:08:15.076829 11334 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.11.13.131:37547, user_credentials={real_user=slave}} blocked reactor thread for 79047.7us
I20260709 12:08:15.096755 11645 tablet_bootstrap.cc:654] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:15.177175 11645 tablet_bootstrap.cc:492] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d: No bootstrap required, opened a new log
I20260709 12:08:15.180128 11645 ts_tablet_manager.cc:1403] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d: Time spent bootstrapping tablet: real 0.202s	user 0.013s	sys 0.033s
I20260709 12:08:15.231980 11645 raft_consensus.cc:359] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } } peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } }
I20260709 12:08:15.233722 11645 raft_consensus.cc:385] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:15.234061 11645 raft_consensus.cc:740] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Initialized, Role: FOLLOWER
I20260709 12:08:15.236888 11648 tablet_bootstrap.cc:492] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb: Bootstrap starting.
I20260709 12:08:15.239853 11645 consensus_queue.cc:260] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } } peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } }
I20260709 12:08:15.248369 11478 heartbeater.cc:499] Master 127.11.13.190:33281 was elected leader, sending a full tablet report...
I20260709 12:08:15.250223 11627 heartbeater.cc:499] Master 127.11.13.190:33281 was elected leader, sending a full tablet report...
I20260709 12:08:15.250643 11647 tablet_bootstrap.cc:492] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa: Bootstrap starting.
I20260709 12:08:15.252640 11645 ts_tablet_manager.cc:1434] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d: Time spent starting tablet: real 0.050s	user 0.010s	sys 0.003s
I20260709 12:08:15.263952 11645 tablet_bootstrap.cc:492] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d: Bootstrap starting.
I20260709 12:08:15.266676 11648 tablet_bootstrap.cc:654] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:15.272802 11645 tablet_bootstrap.cc:654] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:15.280594 11647 tablet_bootstrap.cc:654] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:15.282303 11648 tablet_bootstrap.cc:492] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb: No bootstrap required, opened a new log
I20260709 12:08:15.283041 11648 ts_tablet_manager.cc:1403] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb: Time spent bootstrapping tablet: real 0.048s	user 0.011s	sys 0.012s
I20260709 12:08:15.284637 11645 tablet_bootstrap.cc:492] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d: No bootstrap required, opened a new log
I20260709 12:08:15.285224 11645 ts_tablet_manager.cc:1403] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.009s
I20260709 12:08:15.286345 11648 raft_consensus.cc:359] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.287475 11648 raft_consensus.cc:385] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:15.287853 11648 raft_consensus.cc:740] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Initialized, Role: FOLLOWER
I20260709 12:08:15.288790 11648 consensus_queue.cc:260] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.288971 11645 raft_consensus.cc:359] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.289778 11645 raft_consensus.cc:385] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:15.290199 11645 raft_consensus.cc:740] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Initialized, Role: FOLLOWER
I20260709 12:08:15.291159 11645 consensus_queue.cc:260] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.294585 11645 ts_tablet_manager.cc:1434] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:08:15.295964 11645 tablet_bootstrap.cc:492] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d: Bootstrap starting.
I20260709 12:08:15.305085 11553 heartbeater.cc:499] Master 127.11.13.190:33281 was elected leader, sending a full tablet report...
I20260709 12:08:15.306550 11647 tablet_bootstrap.cc:492] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa: No bootstrap required, opened a new log
I20260709 12:08:15.307507 11647 ts_tablet_manager.cc:1403] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa: Time spent bootstrapping tablet: real 0.058s	user 0.009s	sys 0.005s
I20260709 12:08:15.309815 11645 tablet_bootstrap.cc:654] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:15.311215 11647 raft_consensus.cc:359] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.311988 11647 raft_consensus.cc:385] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:15.312338 11647 raft_consensus.cc:740] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e65f179f7b664838b6770533692da7aa, State: Initialized, Role: FOLLOWER
I20260709 12:08:15.313122 11647 consensus_queue.cc:260] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.322273 11645 tablet_bootstrap.cc:492] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d: No bootstrap required, opened a new log
I20260709 12:08:15.322789 11645 ts_tablet_manager.cc:1403] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d: Time spent bootstrapping tablet: real 0.027s	user 0.015s	sys 0.006s
I20260709 12:08:15.323346 11647 ts_tablet_manager.cc:1434] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.000s
I20260709 12:08:15.325685 11645 raft_consensus.cc:359] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.326776 11645 raft_consensus.cc:385] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:15.327136 11648 ts_tablet_manager.cc:1434] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb: Time spent starting tablet: real 0.044s	user 0.013s	sys 0.007s
I20260709 12:08:15.327090 11645 raft_consensus.cc:740] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Initialized, Role: FOLLOWER
I20260709 12:08:15.327746 11647 tablet_bootstrap.cc:492] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa: Bootstrap starting.
I20260709 12:08:15.328382 11648 tablet_bootstrap.cc:492] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb: Bootstrap starting.
I20260709 12:08:15.327980 11645 consensus_queue.cc:260] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.330552 11645 ts_tablet_manager.cc:1434] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 12:08:15.331524 11645 tablet_bootstrap.cc:492] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d: Bootstrap starting.
I20260709 12:08:15.339224 11645 tablet_bootstrap.cc:654] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:15.342325 11648 tablet_bootstrap.cc:654] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:15.351491 11645 tablet_bootstrap.cc:492] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d: No bootstrap required, opened a new log
I20260709 12:08:15.352018 11645 ts_tablet_manager.cc:1403] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d: Time spent bootstrapping tablet: real 0.021s	user 0.014s	sys 0.004s
I20260709 12:08:15.350376 11647 tablet_bootstrap.cc:654] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa: Neither blocks nor log segments found. Creating new log.
W20260709 12:08:15.354605 11628 tablet.cc:2406] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:08:15.355263 11645 raft_consensus.cc:359] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.356348 11645 raft_consensus.cc:385] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:15.356745 11655 raft_consensus.cc:493] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:15.356815 11645 raft_consensus.cc:740] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Initialized, Role: FOLLOWER
I20260709 12:08:15.357264 11655 raft_consensus.cc:515] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.358104 11645 consensus_queue.cc:260] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.362212 11645 ts_tablet_manager.cc:1434] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.005s
I20260709 12:08:15.363372 11645 tablet_bootstrap.cc:492] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d: Bootstrap starting.
I20260709 12:08:15.364530 11648 tablet_bootstrap.cc:492] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb: No bootstrap required, opened a new log
I20260709 12:08:15.365072 11648 ts_tablet_manager.cc:1403] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb: Time spent bootstrapping tablet: real 0.037s	user 0.008s	sys 0.012s
I20260709 12:08:15.370191 11655 leader_election.cc:290] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:15.368453 11648 raft_consensus.cc:359] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.375526 11648 raft_consensus.cc:385] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:15.375936 11648 raft_consensus.cc:740] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Initialized, Role: FOLLOWER
I20260709 12:08:15.376658 11645 tablet_bootstrap.cc:654] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:15.377008 11648 consensus_queue.cc:260] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.381659 11647 tablet_bootstrap.cc:492] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa: No bootstrap required, opened a new log
I20260709 12:08:15.382267 11647 ts_tablet_manager.cc:1403] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa: Time spent bootstrapping tablet: real 0.055s	user 0.012s	sys 0.007s
I20260709 12:08:15.386766 11647 raft_consensus.cc:359] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.387753 11647 raft_consensus.cc:385] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:15.388147 11647 raft_consensus.cc:740] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e65f179f7b664838b6770533692da7aa, State: Initialized, Role: FOLLOWER
I20260709 12:08:15.388870 11648 ts_tablet_manager.cc:1434] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.000s
I20260709 12:08:15.389029 11647 consensus_queue.cc:260] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.391332 11645 tablet_bootstrap.cc:492] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d: No bootstrap required, opened a new log
I20260709 12:08:15.391862 11645 ts_tablet_manager.cc:1403] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d: Time spent bootstrapping tablet: real 0.029s	user 0.003s	sys 0.009s
I20260709 12:08:15.395648 11648 tablet_bootstrap.cc:492] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb: Bootstrap starting.
I20260709 12:08:15.399801 11649 raft_consensus.cc:493] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:15.400334 11649 raft_consensus.cc:515] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.402532 11647 ts_tablet_manager.cc:1434] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa: Time spent starting tablet: real 0.020s	user 0.009s	sys 0.000s
I20260709 12:08:15.403957 11649 leader_election.cc:290] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537)
I20260709 12:08:15.411877 11647 tablet_bootstrap.cc:492] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa: Bootstrap starting.
I20260709 12:08:15.413792 11645 raft_consensus.cc:359] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.414826 11645 raft_consensus.cc:385] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:15.415220 11645 raft_consensus.cc:740] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Initialized, Role: FOLLOWER
I20260709 12:08:15.416249 11645 consensus_queue.cc:260] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.421954 11647 tablet_bootstrap.cc:654] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:15.424124 11528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "071aa019eff14096b27414993b8e9773" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb" is_pre_election: true
W20260709 12:08:15.426864 11417 leader_election.cc:343] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:15.428853 11648 tablet_bootstrap.cc:654] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:15.431565 11645 ts_tablet_manager.cc:1434] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d: Time spent starting tablet: real 0.039s	user 0.010s	sys 0.000s
I20260709 12:08:15.432827 11645 tablet_bootstrap.cc:492] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d: Bootstrap starting.
I20260709 12:08:15.442085 11645 tablet_bootstrap.cc:654] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:15.462234 11647 tablet_bootstrap.cc:492] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa: No bootstrap required, opened a new log
I20260709 12:08:15.462787 11647 ts_tablet_manager.cc:1403] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa: Time spent bootstrapping tablet: real 0.051s	user 0.016s	sys 0.025s
I20260709 12:08:15.465695 11647 raft_consensus.cc:359] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.466950 11647 raft_consensus.cc:385] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:15.467499 11647 raft_consensus.cc:740] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e65f179f7b664838b6770533692da7aa, State: Initialized, Role: FOLLOWER
I20260709 12:08:15.468654 11648 tablet_bootstrap.cc:492] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb: No bootstrap required, opened a new log
I20260709 12:08:15.468416 11647 consensus_queue.cc:260] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.469291 11648 ts_tablet_manager.cc:1403] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb: Time spent bootstrapping tablet: real 0.074s	user 0.012s	sys 0.003s
I20260709 12:08:15.472060 11453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44710cf3b4a748969c249dcbeccef11d" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65f179f7b664838b6770533692da7aa" is_pre_election: true
I20260709 12:08:15.472425 11648 raft_consensus.cc:359] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.472920 11453 raft_consensus.cc:2468] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 0.
I20260709 12:08:15.473126 11648 raft_consensus.cc:385] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:15.473430 11648 raft_consensus.cc:740] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Initialized, Role: FOLLOWER
I20260709 12:08:15.474107 11647 ts_tablet_manager.cc:1434] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:08:15.474362 11648 consensus_queue.cc:260] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.475124 11647 tablet_bootstrap.cc:492] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa: Bootstrap starting.
I20260709 12:08:15.477506 11648 ts_tablet_manager.cc:1434] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:08:15.478652 11648 tablet_bootstrap.cc:492] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb: Bootstrap starting.
I20260709 12:08:15.486104 11648 tablet_bootstrap.cc:654] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:15.488936 11647 tablet_bootstrap.cc:654] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:15.501278 11565 leader_election.cc:304] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, e65f179f7b664838b6770533692da7aa; no voters: 
I20260709 12:08:15.501976 11647 tablet_bootstrap.cc:492] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa: No bootstrap required, opened a new log
I20260709 12:08:15.502507 11647 ts_tablet_manager.cc:1403] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa: Time spent bootstrapping tablet: real 0.028s	user 0.009s	sys 0.009s
I20260709 12:08:15.503090 11649 raft_consensus.cc:2804] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:15.503480 11649 raft_consensus.cc:493] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:15.504007 11649 raft_consensus.cc:3060] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:15.504462 11528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44710cf3b4a748969c249dcbeccef11d" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb" is_pre_election: true
I20260709 12:08:15.505146 11528 raft_consensus.cc:2468] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 0.
I20260709 12:08:15.506347 11647 raft_consensus.cc:359] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.507078 11647 raft_consensus.cc:385] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:15.507377 11647 raft_consensus.cc:740] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e65f179f7b664838b6770533692da7aa, State: Initialized, Role: FOLLOWER
I20260709 12:08:15.508092 11647 consensus_queue.cc:260] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.510048 11645 tablet_bootstrap.cc:492] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d: No bootstrap required, opened a new log
I20260709 12:08:15.510550 11645 ts_tablet_manager.cc:1403] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d: Time spent bootstrapping tablet: real 0.078s	user 0.023s	sys 0.011s
I20260709 12:08:15.514676 11602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "071aa019eff14096b27414993b8e9773" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d" is_pre_election: true
I20260709 12:08:15.515565 11602 raft_consensus.cc:2468] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e65f179f7b664838b6770533692da7aa in term 0.
I20260709 12:08:15.514057 11647 ts_tablet_manager.cc:1434] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:08:15.517426 11647 tablet_bootstrap.cc:492] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa: Bootstrap starting.
I20260709 12:08:15.520735 11416 leader_election.cc:304] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, e65f179f7b664838b6770533692da7aa; no voters: 9a8e2255342c492eb37a3b0daee59dfb
I20260709 12:08:15.514076 11645 raft_consensus.cc:359] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.521735 11655 raft_consensus.cc:2804] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:15.522185 11655 raft_consensus.cc:493] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:15.522575 11655 raft_consensus.cc:3060] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:15.521720 11645 raft_consensus.cc:385] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:15.528959 11645 raft_consensus.cc:740] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Initialized, Role: FOLLOWER
I20260709 12:08:15.530030 11645 consensus_queue.cc:260] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.536486 11648 tablet_bootstrap.cc:492] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb: No bootstrap required, opened a new log
I20260709 12:08:15.537164 11648 ts_tablet_manager.cc:1403] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb: Time spent bootstrapping tablet: real 0.059s	user 0.013s	sys 0.012s
I20260709 12:08:15.539917 11647 tablet_bootstrap.cc:654] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:15.539264 11649 raft_consensus.cc:515] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.537133 11655 raft_consensus.cc:515] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.543577 11673 raft_consensus.cc:493] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:15.548369 11674 raft_consensus.cc:493] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:15.545588 11649 leader_election.cc:290] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 election: Requested vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537)
I20260709 12:08:15.548583 11528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44710cf3b4a748969c249dcbeccef11d" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
I20260709 12:08:15.547587 11453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44710cf3b4a748969c249dcbeccef11d" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65f179f7b664838b6770533692da7aa"
I20260709 12:08:15.552191 11453 raft_consensus.cc:3060] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:15.551923 11528 raft_consensus.cc:3060] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:15.563882 11649 raft_consensus.cc:493] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:15.548962 11674 raft_consensus.cc:515] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.569461 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "071aa019eff14096b27414993b8e9773" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
I20260709 12:08:15.569772 11649 raft_consensus.cc:515] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.546233 11673 raft_consensus.cc:515] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
W20260709 12:08:15.574749 11417 leader_election.cc:343] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:15.579684 11673 leader_election.cc:290] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537)
I20260709 12:08:15.579725 11602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "071aa019eff14096b27414993b8e9773" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
I20260709 12:08:15.585171 11602 raft_consensus.cc:3060] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:15.585881 11645 ts_tablet_manager.cc:1434] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d: Time spent starting tablet: real 0.075s	user 0.006s	sys 0.008s
I20260709 12:08:15.588886 11645 tablet_bootstrap.cc:492] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d: Bootstrap starting.
I20260709 12:08:15.581899 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f41683de3849cfb6a5e4c93161ce46" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb" is_pre_election: true
I20260709 12:08:15.581813 11452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f41683de3849cfb6a5e4c93161ce46" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65f179f7b664838b6770533692da7aa" is_pre_election: true
I20260709 12:08:15.589815 11527 raft_consensus.cc:2468] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 0.
I20260709 12:08:15.578173 11655 leader_election.cc:290] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 election: Requested vote from peers 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
W20260709 12:08:15.597098 11565 leader_election.cc:343] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e65f179f7b664838b6770533692da7aa (127.11.13.129:35345): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:15.585175 11648 raft_consensus.cc:359] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.611378 11649 leader_election.cc:290] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537)
I20260709 12:08:15.615219 11648 raft_consensus.cc:385] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:15.615337 11674 leader_election.cc:290] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537)
I20260709 12:08:15.615612 11648 raft_consensus.cc:740] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Initialized, Role: FOLLOWER
I20260709 12:08:15.611117 11653 raft_consensus.cc:493] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:15.616421 11648 consensus_queue.cc:260] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.599071 11655 raft_consensus.cc:493] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:15.616475 11653 raft_consensus.cc:515] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.617209 11674 raft_consensus.cc:493] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:15.611730 11452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65f179f7b664838b6770533692da7aa" is_pre_election: true
I20260709 12:08:15.598341 11453 raft_consensus.cc:2468] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 1.
I20260709 12:08:15.598507 11566 leader_election.cc:304] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, 9a8e2255342c492eb37a3b0daee59dfb; no voters: e65f179f7b664838b6770533692da7aa
I20260709 12:08:15.628249 11602 raft_consensus.cc:2468] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65f179f7b664838b6770533692da7aa in term 1.
I20260709 12:08:15.631026 11416 leader_election.cc:304] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, e65f179f7b664838b6770533692da7aa; no voters: 9a8e2255342c492eb37a3b0daee59dfb
I20260709 12:08:15.632239 11677 raft_consensus.cc:2804] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:15.633912 11647 tablet_bootstrap.cc:492] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa: No bootstrap required, opened a new log
I20260709 12:08:15.635105 11647 ts_tablet_manager.cc:1403] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa: Time spent bootstrapping tablet: real 0.118s	user 0.030s	sys 0.012s
I20260709 12:08:15.618181 11655 raft_consensus.cc:515] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.639632 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb" is_pre_election: true
I20260709 12:08:15.641266 11452 raft_consensus.cc:2468] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 0.
I20260709 12:08:15.628150 11674 raft_consensus.cc:515] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } } peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } }
I20260709 12:08:15.632963 11677 raft_consensus.cc:493] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:15.640681 11526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ee33eb902604730a36d8bfe692dc297" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb" is_pre_election: true
I20260709 12:08:15.649984 11655 leader_election.cc:290] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:15.650849 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb" is_pre_election: true
W20260709 12:08:15.647202 11566 leader_election.cc:343] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:15.661818 11655 raft_consensus.cc:493] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:15.646530 11677 raft_consensus.cc:3060] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:08:15.663085 11417 leader_election.cc:343] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:15.662684 11655 raft_consensus.cc:515] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.657963 11602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d" is_pre_election: true
I20260709 12:08:15.669878 11648 ts_tablet_manager.cc:1434] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb: Time spent starting tablet: real 0.130s	user 0.018s	sys 0.025s
I20260709 12:08:15.671003 11648 tablet_bootstrap.cc:492] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb: Bootstrap starting.
I20260709 12:08:15.671929 11452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ee33eb902604730a36d8bfe692dc297" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65f179f7b664838b6770533692da7aa" is_pre_election: true
W20260709 12:08:15.665189 11566 leader_election.cc:343] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:15.667246 11653 leader_election.cc:290] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:15.677302 11565 leader_election.cc:304] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, e65f179f7b664838b6770533692da7aa; no voters: 
I20260709 12:08:15.678674 11674 leader_election.cc:290] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537), e65f179f7b664838b6770533692da7aa (127.11.13.129:35345)
I20260709 12:08:15.679200 11649 raft_consensus.cc:2804] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:15.679420 11565 leader_election.cc:304] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, e65f179f7b664838b6770533692da7aa; no voters: 9a8e2255342c492eb37a3b0daee59dfb
I20260709 12:08:15.679715 11649 raft_consensus.cc:697] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 1 LEADER]: Becoming Leader. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Running, Role: LEADER
W20260709 12:08:15.680622 11565 leader_election.cc:343] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e65f179f7b664838b6770533692da7aa (127.11.13.129:35345): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:15.680815 11645 tablet_bootstrap.cc:654] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:15.680574 11649 consensus_queue.cc:237] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.681102 11565 leader_election.cc:304] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d; no voters: 9a8e2255342c492eb37a3b0daee59dfb, e65f179f7b664838b6770533692da7aa
I20260709 12:08:15.676795 11647 raft_consensus.cc:359] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.672844 11602 raft_consensus.cc:2468] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e65f179f7b664838b6770533692da7aa in term 0.
I20260709 12:08:15.683063 11675 raft_consensus.cc:2749] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:15.684638 11673 raft_consensus.cc:2804] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:15.683408 11647 raft_consensus.cc:385] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:15.688067 11647 raft_consensus.cc:740] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e65f179f7b664838b6770533692da7aa, State: Initialized, Role: FOLLOWER
I20260709 12:08:15.689616 11647 consensus_queue.cc:260] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.695984 11647 ts_tablet_manager.cc:1434] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa: Time spent starting tablet: real 0.060s	user 0.009s	sys 0.000s
I20260709 12:08:15.695889 11452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fc2c80a37c43938dcb1dbaeacc2cc5" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65f179f7b664838b6770533692da7aa" is_pre_election: true
I20260709 12:08:15.697217 11647 tablet_bootstrap.cc:492] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa: Bootstrap starting.
I20260709 12:08:15.698555 11416 leader_election.cc:304] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, e65f179f7b664838b6770533692da7aa; no voters: 9a8e2255342c492eb37a3b0daee59dfb
I20260709 12:08:15.690800 11673 raft_consensus.cc:493] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:15.705094 11678 raft_consensus.cc:2804] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:15.707507 11526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bbcb6cf7a2e4076b40fbde18996ddce" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb" is_pre_election: true
I20260709 12:08:15.708242 11526 raft_consensus.cc:2468] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e65f179f7b664838b6770533692da7aa in term 0.
I20260709 12:08:15.705677 11678 raft_consensus.cc:493] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:15.710359 11678 raft_consensus.cc:3060] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:15.710747 11417 leader_election.cc:304] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a8e2255342c492eb37a3b0daee59dfb, e65f179f7b664838b6770533692da7aa; no voters: 
I20260709 12:08:15.702278 11673 raft_consensus.cc:3060] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:15.712299 11655 leader_election.cc:290] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:15.717135 11655 raft_consensus.cc:2804] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:15.717657 11655 raft_consensus.cc:493] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:15.704427 11528 raft_consensus.cc:2468] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 1.
W20260709 12:08:15.712960 11565 leader_election.cc:343] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e65f179f7b664838b6770533692da7aa (127.11.13.129:35345): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:15.719362 11655 raft_consensus.cc:3060] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:15.731980 11602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bbcb6cf7a2e4076b40fbde18996ddce" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d" is_pre_election: true
W20260709 12:08:15.740516 11416 leader_election.cc:343] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:15.728680 11680 raft_consensus.cc:2804] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:15.742581 11647 tablet_bootstrap.cc:654] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:15.746649 11349 catalog_manager.cc:5697] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d reported cstate change: term changed from 0 to 1, leader changed from <none> to 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131). New cstate: current_term: 1 leader_uuid: "31512bd2b8c74ab89e184a85332e6f9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:15.757342 11680 raft_consensus.cc:697] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 1 LEADER]: Becoming Leader. State: Replica: e65f179f7b664838b6770533692da7aa, State: Running, Role: LEADER
I20260709 12:08:15.758513 11680 consensus_queue.cc:237] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
W20260709 12:08:15.763523 11417 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:34537, user_credentials={real_user=slave}} blocked reactor thread for 52955.7us
I20260709 12:08:15.766002 11678 raft_consensus.cc:515] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.772377 11645 tablet_bootstrap.cc:492] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d: No bootstrap required, opened a new log
I20260709 12:08:15.772891 11645 ts_tablet_manager.cc:1403] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d: Time spent bootstrapping tablet: real 0.184s	user 0.023s	sys 0.025s
I20260709 12:08:15.776647 11645 raft_consensus.cc:359] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.777384 11645 raft_consensus.cc:385] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:15.777690 11648 tablet_bootstrap.cc:654] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:15.766331 11673 raft_consensus.cc:515] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.751981 11677 raft_consensus.cc:515] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.777684 11645 raft_consensus.cc:740] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Initialized, Role: FOLLOWER
I20260709 12:08:15.787524 11602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
I20260709 12:08:15.788074 11528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
I20260709 12:08:15.788591 11602 raft_consensus.cc:2393] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e65f179f7b664838b6770533692da7aa in current term 1: Already voted for candidate 31512bd2b8c74ab89e184a85332e6f9d in this term.
I20260709 12:08:15.783846 11678 leader_election.cc:290] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 election: Requested vote from peers 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:15.801489 11684 raft_consensus.cc:493] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:08:15.797119 11417 leader_election.cc:343] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:15.803718 11677 leader_election.cc:290] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 election: Requested vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537)
I20260709 12:08:15.808009 11452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f41683de3849cfb6a5e4c93161ce46" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65f179f7b664838b6770533692da7aa"
I20260709 12:08:15.801548 11655 raft_consensus.cc:515] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
W20260709 12:08:15.859503 11398 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:08:15.859723 11416 leader_election.cc:304] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e65f179f7b664838b6770533692da7aa; no voters: 31512bd2b8c74ab89e184a85332e6f9d, 9a8e2255342c492eb37a3b0daee59dfb
I20260709 12:08:15.860872 11680 raft_consensus.cc:2749] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:08:15.861074 11416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:37547, user_credentials={real_user=slave}} blocked reactor thread for 59358.6us
I20260709 12:08:15.862645 11526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bbcb6cf7a2e4076b40fbde18996ddce" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
I20260709 12:08:15.863255 11526 raft_consensus.cc:3060] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:15.861944 11655 leader_election.cc:290] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 election: Requested vote from peers 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:15.867450 11528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fc2c80a37c43938dcb1dbaeacc2cc5" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb" is_pre_election: true
W20260709 12:08:15.864224 11565 leader_election.cc:343] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e65f179f7b664838b6770533692da7aa (127.11.13.129:35345): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:15.802407 11684 raft_consensus.cc:515] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.886356 11452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65f179f7b664838b6770533692da7aa"
I20260709 12:08:15.889191 11452 raft_consensus.cc:2393] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 31512bd2b8c74ab89e184a85332e6f9d in current term 1: Already voted for candidate e65f179f7b664838b6770533692da7aa in this term.
I20260709 12:08:15.920292 11528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f41683de3849cfb6a5e4c93161ce46" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
I20260709 12:08:15.921231 11528 raft_consensus.cc:3060] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:15.921101 11653 raft_consensus.cc:493] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:15.922317 11653 raft_consensus.cc:515] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.924940 11653 leader_election.cc:290] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:15.927309 11602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bbcb6cf7a2e4076b40fbde18996ddce" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
W20260709 12:08:15.933360 11566 leader_election.cc:343] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:15.934069 11566 leader_election.cc:304] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d; no voters: 9a8e2255342c492eb37a3b0daee59dfb, e65f179f7b664838b6770533692da7aa
I20260709 12:08:15.935165 11677 raft_consensus.cc:2749] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:15.915699 11673 leader_election.cc:290] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 election: Requested vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537)
I20260709 12:08:15.947656 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
I20260709 12:08:15.947811 11452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bbcb6cf7a2e4076b40fbde18996ddce" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65f179f7b664838b6770533692da7aa" is_pre_election: true
I20260709 12:08:15.948686 11452 raft_consensus.cc:2393] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a8e2255342c492eb37a3b0daee59dfb in current term 1: Already voted for candidate e65f179f7b664838b6770533692da7aa in this term.
W20260709 12:08:15.940035 11416 leader_election.cc:343] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:15.942977 11645 consensus_queue.cc:260] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.950520 11526 raft_consensus.cc:2468] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65f179f7b664838b6770533692da7aa in term 1.
I20260709 12:08:15.950803 11453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "071aa019eff14096b27414993b8e9773" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65f179f7b664838b6770533692da7aa" is_pre_election: true
I20260709 12:08:15.951458 11684 leader_election.cc:290] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
W20260709 12:08:15.948966 11566 leader_election.cc:343] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:15.954872 11602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af01620b86384cce826e034c1cf541fa" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d" is_pre_election: true
I20260709 12:08:15.954957 11566 leader_election.cc:304] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d; no voters: 9a8e2255342c492eb37a3b0daee59dfb, e65f179f7b664838b6770533692da7aa
I20260709 12:08:15.952298 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af01620b86384cce826e034c1cf541fa" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb" is_pre_election: true
I20260709 12:08:15.956188 11673 raft_consensus.cc:2749] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:08:15.957885 11417 leader_election.cc:343] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:15.958809 11648 tablet_bootstrap.cc:492] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb: No bootstrap required, opened a new log
I20260709 12:08:15.958897 11417 leader_election.cc:304] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 9a8e2255342c492eb37a3b0daee59dfb, e65f179f7b664838b6770533692da7aa; no voters: 31512bd2b8c74ab89e184a85332e6f9d
I20260709 12:08:15.959318 11648 ts_tablet_manager.cc:1403] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb: Time spent bootstrapping tablet: real 0.289s	user 0.027s	sys 0.020s
I20260709 12:08:15.959784 11684 raft_consensus.cc:2804] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:15.960258 11684 raft_consensus.cc:697] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 1 LEADER]: Becoming Leader. State: Replica: e65f179f7b664838b6770533692da7aa, State: Running, Role: LEADER
I20260709 12:08:15.961109 11684 consensus_queue.cc:237] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.962790 11648 raft_consensus.cc:359] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.963182 11602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bbcb6cf7a2e4076b40fbde18996ddce" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d" is_pre_election: true
I20260709 12:08:15.963500 11648 raft_consensus.cc:385] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:08:15.963819 11416 leader_election.cc:343] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:15.964349 11416 leader_election.cc:304] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e65f179f7b664838b6770533692da7aa; no voters: 31512bd2b8c74ab89e184a85332e6f9d, 9a8e2255342c492eb37a3b0daee59dfb
I20260709 12:08:15.965085 11602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "071aa019eff14096b27414993b8e9773" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d" is_pre_election: true
I20260709 12:08:15.963820 11648 raft_consensus.cc:740] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Initialized, Role: FOLLOWER
I20260709 12:08:15.965659 11673 raft_consensus.cc:493] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:15.965840 11602 raft_consensus.cc:2393] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a8e2255342c492eb37a3b0daee59dfb in current term 1: Already voted for candidate e65f179f7b664838b6770533692da7aa in this term.
I20260709 12:08:15.966305 11655 raft_consensus.cc:2749] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:15.963354 11645 ts_tablet_manager.cc:1434] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d: Time spent starting tablet: real 0.190s	user 0.039s	sys 0.067s
I20260709 12:08:15.966321 11673 raft_consensus.cc:515] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.971203 11647 tablet_bootstrap.cc:492] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa: No bootstrap required, opened a new log
I20260709 12:08:15.971676 11647 ts_tablet_manager.cc:1403] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa: Time spent bootstrapping tablet: real 0.275s	user 0.019s	sys 0.008s
I20260709 12:08:15.971570 11673 leader_election.cc:290] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537)
I20260709 12:08:15.973996 11453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6057698b843b18fff49462dc3679e" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65f179f7b664838b6770533692da7aa" is_pre_election: true
W20260709 12:08:15.975057 11491 leader_election.cc:343] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:08:15.975462 11565 leader_election.cc:343] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e65f179f7b664838b6770533692da7aa (127.11.13.129:35345): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:15.975535 11491 leader_election.cc:304] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a8e2255342c492eb37a3b0daee59dfb; no voters: 31512bd2b8c74ab89e184a85332e6f9d, e65f179f7b664838b6770533692da7aa
I20260709 12:08:15.965190 11528 raft_consensus.cc:2468] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 1.
I20260709 12:08:15.977321 11653 raft_consensus.cc:2749] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:15.977039 11647 raft_consensus.cc:359] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } } peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } }
I20260709 12:08:15.977882 11647 raft_consensus.cc:385] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:15.978142 11645 tablet_bootstrap.cc:492] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d: Bootstrap starting.
I20260709 12:08:15.966619 11648 consensus_queue.cc:260] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.978263 11647 raft_consensus.cc:740] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e65f179f7b664838b6770533692da7aa, State: Initialized, Role: FOLLOWER
I20260709 12:08:15.979389 11647 consensus_queue.cc:260] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } } peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } }
I20260709 12:08:15.980324 11526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6057698b843b18fff49462dc3679e" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb" is_pre_election: true
I20260709 12:08:15.981544 11491 leader_election.cc:304] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a8e2255342c492eb37a3b0daee59dfb; no voters: 31512bd2b8c74ab89e184a85332e6f9d, e65f179f7b664838b6770533692da7aa
I20260709 12:08:15.982868 11647 ts_tablet_manager.cc:1434] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:08:15.984359 11647 tablet_bootstrap.cc:492] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa: Bootstrap starting.
I20260709 12:08:15.986502 11645 tablet_bootstrap.cc:654] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:15.991151 11679 raft_consensus.cc:3060] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:08:15.991767 11566 leader_election.cc:343] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:15.992499 11566 leader_election.cc:304] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d; no voters: 9a8e2255342c492eb37a3b0daee59dfb, e65f179f7b664838b6770533692da7aa
I20260709 12:08:15.993568 11673 raft_consensus.cc:2749] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:15.995772 11566 leader_election.cc:304] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, 9a8e2255342c492eb37a3b0daee59dfb; no voters: e65f179f7b664838b6770533692da7aa
I20260709 12:08:15.997069 11673 raft_consensus.cc:2804] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:15.997563 11673 raft_consensus.cc:697] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 LEADER]: Becoming Leader. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Running, Role: LEADER
I20260709 12:08:15.998430 11673 consensus_queue.cc:237] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:15.999434 11679 raft_consensus.cc:2749] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:16.010675 11648 ts_tablet_manager.cc:1434] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb: Time spent starting tablet: real 0.051s	user 0.005s	sys 0.005s
I20260709 12:08:16.012964 11647 tablet_bootstrap.cc:654] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:16.018460 11648 tablet_bootstrap.cc:492] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb: Bootstrap starting.
I20260709 12:08:16.027218 11645 tablet_bootstrap.cc:492] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d: No bootstrap required, opened a new log
I20260709 12:08:16.027572 11645 ts_tablet_manager.cc:1403] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d: Time spent bootstrapping tablet: real 0.051s	user 0.009s	sys 0.015s
I20260709 12:08:16.033227 11647 tablet_bootstrap.cc:492] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa: No bootstrap required, opened a new log
I20260709 12:08:16.034198 11647 ts_tablet_manager.cc:1403] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa: Time spent bootstrapping tablet: real 0.050s	user 0.008s	sys 0.011s
I20260709 12:08:16.035477 11645 raft_consensus.cc:359] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.036294 11645 raft_consensus.cc:385] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:16.036638 11645 raft_consensus.cc:740] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Initialized, Role: FOLLOWER
I20260709 12:08:16.037456 11645 consensus_queue.cc:260] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.039950 11645 ts_tablet_manager.cc:1434] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.004s
I20260709 12:08:16.040874 11645 tablet_bootstrap.cc:492] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d: Bootstrap starting.
I20260709 12:08:16.042055 11648 tablet_bootstrap.cc:654] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:16.042399 11350 catalog_manager.cc:5697] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa reported cstate change: term changed from 0 to 1, leader changed from <none> to e65f179f7b664838b6770533692da7aa (127.11.13.129). New cstate: current_term: 1 leader_uuid: "e65f179f7b664838b6770533692da7aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:16.044652 11647 raft_consensus.cc:359] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.045462 11647 raft_consensus.cc:385] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:16.045766 11647 raft_consensus.cc:740] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e65f179f7b664838b6770533692da7aa, State: Initialized, Role: FOLLOWER
I20260709 12:08:16.046517 11647 consensus_queue.cc:260] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.049639 11647 ts_tablet_manager.cc:1434] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa: Time spent starting tablet: real 0.015s	user 0.001s	sys 0.004s
I20260709 12:08:16.051007 11647 tablet_bootstrap.cc:492] T 7ee33eb902604730a36d8bfe692dc297 P e65f179f7b664838b6770533692da7aa: Bootstrap starting.
I20260709 12:08:16.051026 11645 tablet_bootstrap.cc:654] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:16.057978 11647 tablet_bootstrap.cc:654] T 7ee33eb902604730a36d8bfe692dc297 P e65f179f7b664838b6770533692da7aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:16.054702 11349 catalog_manager.cc:5697] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d reported cstate change: term changed from 0 to 1, leader changed from <none> to 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131). New cstate: current_term: 1 leader_uuid: "31512bd2b8c74ab89e184a85332e6f9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:16.070995 11647 tablet_bootstrap.cc:492] T 7ee33eb902604730a36d8bfe692dc297 P e65f179f7b664838b6770533692da7aa: No bootstrap required, opened a new log
I20260709 12:08:16.071542 11647 ts_tablet_manager.cc:1403] T 7ee33eb902604730a36d8bfe692dc297 P e65f179f7b664838b6770533692da7aa: Time spent bootstrapping tablet: real 0.021s	user 0.013s	sys 0.004s
I20260709 12:08:16.074112 11647 raft_consensus.cc:359] T 7ee33eb902604730a36d8bfe692dc297 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.074822 11647 raft_consensus.cc:385] T 7ee33eb902604730a36d8bfe692dc297 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:16.075099 11647 raft_consensus.cc:740] T 7ee33eb902604730a36d8bfe692dc297 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e65f179f7b664838b6770533692da7aa, State: Initialized, Role: FOLLOWER
I20260709 12:08:16.075819 11647 consensus_queue.cc:260] T 7ee33eb902604730a36d8bfe692dc297 P e65f179f7b664838b6770533692da7aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.077983 11647 ts_tablet_manager.cc:1434] T 7ee33eb902604730a36d8bfe692dc297 P e65f179f7b664838b6770533692da7aa: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:08:16.079057 11647 tablet_bootstrap.cc:492] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa: Bootstrap starting.
I20260709 12:08:16.081969 11350 catalog_manager.cc:5697] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa reported cstate change: term changed from 0 to 1, leader changed from <none> to e65f179f7b664838b6770533692da7aa (127.11.13.129). New cstate: current_term: 1 leader_uuid: "e65f179f7b664838b6770533692da7aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:16.088088 11647 tablet_bootstrap.cc:654] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:16.100414 11647 tablet_bootstrap.cc:492] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa: No bootstrap required, opened a new log
I20260709 12:08:16.100966 11647 ts_tablet_manager.cc:1403] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa: Time spent bootstrapping tablet: real 0.022s	user 0.005s	sys 0.011s
I20260709 12:08:16.104311 11647 raft_consensus.cc:359] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.105031 11647 raft_consensus.cc:385] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:16.105295 11647 raft_consensus.cc:740] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e65f179f7b664838b6770533692da7aa, State: Initialized, Role: FOLLOWER
I20260709 12:08:16.106230 11647 consensus_queue.cc:260] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.108651 11647 ts_tablet_manager.cc:1434] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.005s
I20260709 12:08:16.109928 11648 tablet_bootstrap.cc:492] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb: No bootstrap required, opened a new log
I20260709 12:08:16.110481 11648 ts_tablet_manager.cc:1403] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb: Time spent bootstrapping tablet: real 0.092s	user 0.026s	sys 0.005s
I20260709 12:08:16.111922 11645 tablet_bootstrap.cc:492] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d: No bootstrap required, opened a new log
I20260709 12:08:16.112399 11645 ts_tablet_manager.cc:1403] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d: Time spent bootstrapping tablet: real 0.072s	user 0.009s	sys 0.042s
I20260709 12:08:16.113605 11648 raft_consensus.cc:359] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.114424 11648 raft_consensus.cc:385] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:16.114728 11648 raft_consensus.cc:740] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Initialized, Role: FOLLOWER
I20260709 12:08:16.115168 11645 raft_consensus.cc:359] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.115900 11645 raft_consensus.cc:385] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:16.115516 11648 consensus_queue.cc:260] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.116194 11645 raft_consensus.cc:740] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Initialized, Role: FOLLOWER
I20260709 12:08:16.117161 11645 consensus_queue.cc:260] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.118325 11648 ts_tablet_manager.cc:1434] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:08:16.119392 11648 tablet_bootstrap.cc:492] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb: Bootstrap starting.
I20260709 12:08:16.119405 11645 ts_tablet_manager.cc:1434] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:08:16.128396 11679 raft_consensus.cc:493] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:16.128929 11679 raft_consensus.cc:515] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.131047 11679 leader_election.cc:290] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:16.133723 11673 consensus_queue.cc:1048] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:16.137168 11453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65f179f7b664838b6770533692da7aa" is_pre_election: true
I20260709 12:08:16.138070 11453 raft_consensus.cc:2393] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a8e2255342c492eb37a3b0daee59dfb in current term 1: Already voted for candidate e65f179f7b664838b6770533692da7aa in this term.
I20260709 12:08:16.166522 11602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d" is_pre_election: true
I20260709 12:08:16.167685 11602 raft_consensus.cc:2393] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a8e2255342c492eb37a3b0daee59dfb in current term 1: Already voted for candidate 31512bd2b8c74ab89e184a85332e6f9d in this term.
I20260709 12:08:16.167423 11684 consensus_queue.cc:1048] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:16.170506 11491 leader_election.cc:304] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a8e2255342c492eb37a3b0daee59dfb; no voters: 31512bd2b8c74ab89e184a85332e6f9d, e65f179f7b664838b6770533692da7aa
I20260709 12:08:16.171378 11679 raft_consensus.cc:3060] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:16.183987 11679 raft_consensus.cc:2749] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:16.191236 11648 tablet_bootstrap.cc:654] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:16.194811 11686 consensus_queue.cc:1048] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:16.208987 11648 tablet_bootstrap.cc:492] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb: No bootstrap required, opened a new log
I20260709 12:08:16.209547 11648 ts_tablet_manager.cc:1403] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb: Time spent bootstrapping tablet: real 0.090s	user 0.011s	sys 0.003s
I20260709 12:08:16.213109 11648 raft_consensus.cc:359] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.213929 11648 raft_consensus.cc:385] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:16.214372 11648 raft_consensus.cc:740] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Initialized, Role: FOLLOWER
I20260709 12:08:16.215171 11648 consensus_queue.cc:260] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.218411 11648 ts_tablet_manager.cc:1434] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb: Time spent starting tablet: real 0.009s	user 0.010s	sys 0.000s
I20260709 12:08:16.219812 11648 tablet_bootstrap.cc:492] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb: Bootstrap starting.
I20260709 12:08:16.230557 11648 tablet_bootstrap.cc:654] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:16.237092 11648 tablet_bootstrap.cc:492] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb: No bootstrap required, opened a new log
I20260709 12:08:16.238106 11648 ts_tablet_manager.cc:1403] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.004s
I20260709 12:08:16.239567 11673 raft_consensus.cc:493] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:16.240136 11673 raft_consensus.cc:515] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.240906 11648 raft_consensus.cc:359] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } } peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } }
I20260709 12:08:16.245713 11648 raft_consensus.cc:385] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:16.246033 11648 raft_consensus.cc:740] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Initialized, Role: FOLLOWER
I20260709 12:08:16.246351 11453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af01620b86384cce826e034c1cf541fa" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65f179f7b664838b6770533692da7aa" is_pre_election: true
I20260709 12:08:16.247021 11453 raft_consensus.cc:2468] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 0.
I20260709 12:08:16.246790 11648 consensus_queue.cc:260] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } } peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } }
I20260709 12:08:16.248517 11565 leader_election.cc:304] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, e65f179f7b664838b6770533692da7aa; no voters: 
I20260709 12:08:16.249174 11673 leader_election.cc:290] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537)
I20260709 12:08:16.244910 11677 consensus_queue.cc:1048] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:16.250509 11648 ts_tablet_manager.cc:1434] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:08:16.250765 11677 raft_consensus.cc:2804] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:16.250946 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af01620b86384cce826e034c1cf541fa" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb" is_pre_election: true
I20260709 12:08:16.251858 11648 tablet_bootstrap.cc:492] T 53e6057698b843b18fff49462dc3679e P 9a8e2255342c492eb37a3b0daee59dfb: Bootstrap starting.
I20260709 12:08:16.251794 11527 raft_consensus.cc:2468] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 0.
I20260709 12:08:16.251199 11677 raft_consensus.cc:493] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:16.254293 11677 raft_consensus.cc:3060] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:16.260020 11648 tablet_bootstrap.cc:654] T 53e6057698b843b18fff49462dc3679e P 9a8e2255342c492eb37a3b0daee59dfb: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:16.264549 11677 raft_consensus.cc:515] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.265405 11686 raft_consensus.cc:493] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:16.266160 11686 raft_consensus.cc:515] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.268216 11677 leader_election.cc:290] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 election: Requested vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537)
I20260709 12:08:16.270222 11686 leader_election.cc:290] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:16.270355 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6057698b843b18fff49462dc3679e" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb" is_pre_election: true
W20260709 12:08:16.272745 11417 leader_election.cc:343] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:16.274654 11452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af01620b86384cce826e034c1cf541fa" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65f179f7b664838b6770533692da7aa"
I20260709 12:08:16.275414 11452 raft_consensus.cc:3060] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:16.274544 11601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6057698b843b18fff49462dc3679e" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d" is_pre_election: true
I20260709 12:08:16.278461 11601 raft_consensus.cc:2468] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e65f179f7b664838b6770533692da7aa in term 0.
I20260709 12:08:16.279642 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af01620b86384cce826e034c1cf541fa" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
I20260709 12:08:16.280283 11527 raft_consensus.cc:3060] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:16.282778 11416 leader_election.cc:304] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, e65f179f7b664838b6770533692da7aa; no voters: 9a8e2255342c492eb37a3b0daee59dfb
I20260709 12:08:16.286502 11686 raft_consensus.cc:2804] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:16.297577 11686 raft_consensus.cc:493] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:16.298112 11686 raft_consensus.cc:3060] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:16.299785 11679 raft_consensus.cc:493] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:16.300269 11679 raft_consensus.cc:515] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.302426 11679 leader_election.cc:290] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:16.304723 11601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ee33eb902604730a36d8bfe692dc297" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d" is_pre_election: true
I20260709 12:08:16.305651 11601 raft_consensus.cc:2468] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a8e2255342c492eb37a3b0daee59dfb in term 0.
I20260709 12:08:16.307230 11491 leader_election.cc:304] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, 9a8e2255342c492eb37a3b0daee59dfb; no voters: 
I20260709 12:08:16.308411 11679 raft_consensus.cc:2804] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:16.308933 11679 raft_consensus.cc:493] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:16.309381 11679 raft_consensus.cc:3060] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:16.310564 11452 raft_consensus.cc:2468] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 1.
I20260709 12:08:16.311856 11565 leader_election.cc:304] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, e65f179f7b664838b6770533692da7aa; no voters: 
I20260709 12:08:16.312741 11677 raft_consensus.cc:2804] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:16.313221 11677 raft_consensus.cc:697] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 1 LEADER]: Becoming Leader. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Running, Role: LEADER
I20260709 12:08:16.314069 11677 consensus_queue.cc:237] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.315164 11453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ee33eb902604730a36d8bfe692dc297" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65f179f7b664838b6770533692da7aa" is_pre_election: true
I20260709 12:08:16.316107 11453 raft_consensus.cc:2468] T 7ee33eb902604730a36d8bfe692dc297 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9a8e2255342c492eb37a3b0daee59dfb in term 0.
I20260709 12:08:16.321709 11686 raft_consensus.cc:515] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.322810 11527 raft_consensus.cc:2468] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 1.
I20260709 12:08:16.324956 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6057698b843b18fff49462dc3679e" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
W20260709 12:08:16.326756 11417 leader_election.cc:343] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:16.326602 11679 raft_consensus.cc:515] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.328747 11648 tablet_bootstrap.cc:492] T 53e6057698b843b18fff49462dc3679e P 9a8e2255342c492eb37a3b0daee59dfb: No bootstrap required, opened a new log
I20260709 12:08:16.329226 11648 ts_tablet_manager.cc:1403] T 53e6057698b843b18fff49462dc3679e P 9a8e2255342c492eb37a3b0daee59dfb: Time spent bootstrapping tablet: real 0.078s	user 0.021s	sys 0.045s
I20260709 12:08:16.329820 11453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ee33eb902604730a36d8bfe692dc297" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65f179f7b664838b6770533692da7aa"
I20260709 12:08:16.330471 11453 raft_consensus.cc:3060] T 7ee33eb902604730a36d8bfe692dc297 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:16.331890 11648 raft_consensus.cc:359] T 53e6057698b843b18fff49462dc3679e P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.332422 11601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6057698b843b18fff49462dc3679e" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
I20260709 12:08:16.332757 11648 raft_consensus.cc:385] T 53e6057698b843b18fff49462dc3679e P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:16.333024 11601 raft_consensus.cc:3060] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:16.333058 11648 raft_consensus.cc:740] T 53e6057698b843b18fff49462dc3679e P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Initialized, Role: FOLLOWER
I20260709 12:08:16.334399 11648 consensus_queue.cc:260] T 53e6057698b843b18fff49462dc3679e P 9a8e2255342c492eb37a3b0daee59dfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.338038 11648 ts_tablet_manager.cc:1434] T 53e6057698b843b18fff49462dc3679e P 9a8e2255342c492eb37a3b0daee59dfb: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:08:16.341483 11453 raft_consensus.cc:2468] T 7ee33eb902604730a36d8bfe692dc297 P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8e2255342c492eb37a3b0daee59dfb in term 1.
I20260709 12:08:16.342856 11491 leader_election.cc:304] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a8e2255342c492eb37a3b0daee59dfb, e65f179f7b664838b6770533692da7aa; no voters: 
I20260709 12:08:16.343834 11653 raft_consensus.cc:2804] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:16.345235 11653 raft_consensus.cc:697] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb [term 1 LEADER]: Becoming Leader. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Running, Role: LEADER
I20260709 12:08:16.346182 11653 consensus_queue.cc:237] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.351383 11601 raft_consensus.cc:2468] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65f179f7b664838b6770533692da7aa in term 1.
I20260709 12:08:16.352613 11686 leader_election.cc:290] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 election: Requested vote from peers 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:16.353250 11679 leader_election.cc:290] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 1 election: Requested vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:16.353436 11602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7ee33eb902604730a36d8bfe692dc297" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
I20260709 12:08:16.352651 11350 catalog_manager.cc:5697] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d reported cstate change: term changed from 0 to 1, leader changed from <none> to 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131). New cstate: current_term: 1 leader_uuid: "31512bd2b8c74ab89e184a85332e6f9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:16.358172 11416 leader_election.cc:304] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, e65f179f7b664838b6770533692da7aa; no voters: 9a8e2255342c492eb37a3b0daee59dfb
I20260709 12:08:16.354074 11602 raft_consensus.cc:3060] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:16.360261 11655 raft_consensus.cc:493] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:16.360806 11655 raft_consensus.cc:515] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } } peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } }
I20260709 12:08:16.361706 11680 raft_consensus.cc:2804] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:16.362275 11680 raft_consensus.cc:697] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [term 1 LEADER]: Becoming Leader. State: Replica: e65f179f7b664838b6770533692da7aa, State: Running, Role: LEADER
I20260709 12:08:16.363008 11655 leader_election.cc:290] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:16.363121 11680 consensus_queue.cc:237] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.366519 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fc2c80a37c43938dcb1dbaeacc2cc5" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb" is_pre_election: true
I20260709 12:08:16.367203 11527 raft_consensus.cc:2468] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e65f179f7b664838b6770533692da7aa in term 0.
I20260709 12:08:16.368744 11417 leader_election.cc:304] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a8e2255342c492eb37a3b0daee59dfb, e65f179f7b664838b6770533692da7aa; no voters: 
I20260709 12:08:16.369688 11655 raft_consensus.cc:2804] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:16.370153 11655 raft_consensus.cc:493] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:16.370533 11655 raft_consensus.cc:3060] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:16.370594 11601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fc2c80a37c43938dcb1dbaeacc2cc5" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d" is_pre_election: true
I20260709 12:08:16.371363 11601 raft_consensus.cc:2468] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e65f179f7b664838b6770533692da7aa in term 0.
I20260709 12:08:16.369988 11602 raft_consensus.cc:2468] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8e2255342c492eb37a3b0daee59dfb in term 1.
I20260709 12:08:16.380508 11655 raft_consensus.cc:515] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } } peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } }
I20260709 12:08:16.385708 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fc2c80a37c43938dcb1dbaeacc2cc5" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
I20260709 12:08:16.386519 11527 raft_consensus.cc:3060] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:16.387341 11655 leader_election.cc:290] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 election: Requested vote from peers 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:16.394353 11602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fc2c80a37c43938dcb1dbaeacc2cc5" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
I20260709 12:08:16.395231 11602 raft_consensus.cc:3060] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:16.396831 11347 catalog_manager.cc:5697] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb reported cstate change: term changed from 0 to 1, leader changed from <none> to 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130). New cstate: current_term: 1 leader_uuid: "9a8e2255342c492eb37a3b0daee59dfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:16.403618 11527 raft_consensus.cc:2468] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65f179f7b664838b6770533692da7aa in term 1.
I20260709 12:08:16.404464 11349 catalog_manager.cc:5697] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa reported cstate change: term changed from 0 to 1, leader changed from <none> to e65f179f7b664838b6770533692da7aa (127.11.13.129). New cstate: current_term: 1 leader_uuid: "e65f179f7b664838b6770533692da7aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:16.405396 11417 leader_election.cc:304] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a8e2255342c492eb37a3b0daee59dfb, e65f179f7b664838b6770533692da7aa; no voters: 
I20260709 12:08:16.406402 11680 raft_consensus.cc:2804] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:16.407123 11680 raft_consensus.cc:697] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 1 LEADER]: Becoming Leader. State: Replica: e65f179f7b664838b6770533692da7aa, State: Running, Role: LEADER
I20260709 12:08:16.408028 11680 consensus_queue.cc:237] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } } peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } }
I20260709 12:08:16.413298 11677 consensus_queue.cc:1048] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:16.418519 11602 raft_consensus.cc:2468] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65f179f7b664838b6770533692da7aa in term 1.
I20260709 12:08:16.433188 11655 consensus_queue.cc:1048] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:16.447208 11453 raft_consensus.cc:3060] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:16.457574 11349 catalog_manager.cc:5697] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa reported cstate change: term changed from 0 to 1, leader changed from <none> to e65f179f7b664838b6770533692da7aa (127.11.13.129). New cstate: current_term: 1 leader_uuid: "e65f179f7b664838b6770533692da7aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:16.464283 11649 consensus_queue.cc:1048] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:16.475148 11649 raft_consensus.cc:493] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:16.475714 11649 raft_consensus.cc:515] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.478219 11649 leader_election.cc:290] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537)
I20260709 12:08:16.480286 11526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bbcb6cf7a2e4076b40fbde18996ddce" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb" is_pre_election: true
I20260709 12:08:16.480703 11452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bbcb6cf7a2e4076b40fbde18996ddce" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65f179f7b664838b6770533692da7aa" is_pre_election: true
I20260709 12:08:16.482458 11565 leader_election.cc:304] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d; no voters: 9a8e2255342c492eb37a3b0daee59dfb, e65f179f7b664838b6770533692da7aa
I20260709 12:08:16.483985 11649 raft_consensus.cc:2749] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:16.489473 11602 raft_consensus.cc:3060] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:16.502149 11684 consensus_queue.cc:1048] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:16.600139 11653 raft_consensus.cc:493] T 53e6057698b843b18fff49462dc3679e P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:16.600808 11653 raft_consensus.cc:515] T 53e6057698b843b18fff49462dc3679e P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:16.602591 11653 leader_election.cc:290] T 53e6057698b843b18fff49462dc3679e P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:16.603711 11453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6057698b843b18fff49462dc3679e" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65f179f7b664838b6770533692da7aa" is_pre_election: true
I20260709 12:08:16.603871 11602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6057698b843b18fff49462dc3679e" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d" is_pre_election: true
I20260709 12:08:16.604671 11602 raft_consensus.cc:2393] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 9a8e2255342c492eb37a3b0daee59dfb in current term 1: Already voted for candidate e65f179f7b664838b6770533692da7aa in this term.
I20260709 12:08:16.605785 11491 leader_election.cc:304] T 53e6057698b843b18fff49462dc3679e P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 9a8e2255342c492eb37a3b0daee59dfb; no voters: 31512bd2b8c74ab89e184a85332e6f9d, e65f179f7b664838b6770533692da7aa
I20260709 12:08:16.606716 11653 raft_consensus.cc:3060] T 53e6057698b843b18fff49462dc3679e P 9a8e2255342c492eb37a3b0daee59dfb [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:16.614889 11653 raft_consensus.cc:2749] T 53e6057698b843b18fff49462dc3679e P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:16.767457 11684 consensus_queue.cc:1048] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:16.778683 11655 consensus_queue.cc:1048] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:16.789193 11653 consensus_queue.cc:1048] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:16.795060 11649 consensus_queue.cc:1048] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:16.806654 11679 consensus_queue.cc:1048] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:16.836460 11677 consensus_queue.cc:1048] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:08:16.868971 11398 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:08:16.927888 11680 consensus_queue.cc:1048] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:16.940634 11686 consensus_queue.cc:1048] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:17.696015 11677 raft_consensus.cc:493] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:17.696468 11677 raft_consensus.cc:515] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:17.698963 11677 leader_election.cc:290] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537)
I20260709 12:08:17.699671 11452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65f179f7b664838b6770533692da7aa" is_pre_election: true
I20260709 12:08:17.699970 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb" is_pre_election: true
I20260709 12:08:17.700248 11452 raft_consensus.cc:2468] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 1.
I20260709 12:08:17.700556 11527 raft_consensus.cc:2468] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 1.
I20260709 12:08:17.701395 11565 leader_election.cc:304] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, e65f179f7b664838b6770533692da7aa; no voters: 
I20260709 12:08:17.702075 11677 raft_consensus.cc:2804] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:08:17.702432 11677 raft_consensus.cc:493] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:17.702775 11677 raft_consensus.cc:3060] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:17.778316 11677 raft_consensus.cc:515] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:17.781267 11677 leader_election.cc:290] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 2 election: Requested vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537)
I20260709 12:08:17.781510 11452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e65f179f7b664838b6770533692da7aa"
I20260709 12:08:17.782323 11452 raft_consensus.cc:3060] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:17.783916 11526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
I20260709 12:08:17.784538 11526 raft_consensus.cc:3060] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:08:17.874457 11398 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:08:17.885257 11526 raft_consensus.cc:2468] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 2.
I20260709 12:08:17.885285 11452 raft_consensus.cc:2468] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 2.
I20260709 12:08:17.886878 11566 leader_election.cc:304] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, 9a8e2255342c492eb37a3b0daee59dfb; no voters: 
I20260709 12:08:17.887784 11677 raft_consensus.cc:2804] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:17.888367 11677 raft_consensus.cc:697] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 2 LEADER]: Becoming Leader. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Running, Role: LEADER
I20260709 12:08:17.889382 11677 consensus_queue.cc:237] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:17.898741 11350 catalog_manager.cc:5697] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d reported cstate change: term changed from 0 to 2, leader changed from <none> to 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131). New cstate: current_term: 2 leader_uuid: "31512bd2b8c74ab89e184a85332e6f9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:18.277108 11526 raft_consensus.cc:1275] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Refusing update from remote peer 31512bd2b8c74ab89e184a85332e6f9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:08:18.278746 11677 consensus_queue.cc:1048] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:18.301327 11452 raft_consensus.cc:1275] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Refusing update from remote peer 31512bd2b8c74ab89e184a85332e6f9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:08:18.302764 11732 consensus_queue.cc:1048] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:18.310437 11602 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7f41683de3849cfb6a5e4c93161ce46"
dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
mode: GRACEFUL
new_leader_uuid: "e65f179f7b664838b6770533692da7aa"
 from {username='slave'} at 127.0.0.1:40052
I20260709 12:08:18.311223 11602 raft_consensus.cc:606] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 LEADER]: Received request to transfer leadership to TS e65f179f7b664838b6770533692da7aa
I20260709 12:08:18.320508 11602 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44710cf3b4a748969c249dcbeccef11d"
dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
mode: GRACEFUL
new_leader_uuid: "e65f179f7b664838b6770533692da7aa"
 from {username='slave'} at 127.0.0.1:40052
I20260709 12:08:18.321419 11602 raft_consensus.cc:606] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 1 LEADER]: Received request to transfer leadership to TS e65f179f7b664838b6770533692da7aa
I20260709 12:08:18.342768 11452 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "071aa019eff14096b27414993b8e9773"
dest_uuid: "e65f179f7b664838b6770533692da7aa"
mode: GRACEFUL
new_leader_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
 from {username='slave'} at 127.0.0.1:56982
I20260709 12:08:18.343369 11452 raft_consensus.cc:606] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 1 LEADER]: Received request to transfer leadership to TS 9a8e2255342c492eb37a3b0daee59dfb
I20260709 12:08:18.348371 11602 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af01620b86384cce826e034c1cf541fa"
dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
mode: GRACEFUL
new_leader_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
 from {username='slave'} at 127.0.0.1:40052
I20260709 12:08:18.348853 11602 raft_consensus.cc:606] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 1 LEADER]: Received request to transfer leadership to TS 9a8e2255342c492eb37a3b0daee59dfb
I20260709 12:08:18.353641 11602 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74"
dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
mode: GRACEFUL
new_leader_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
 from {username='slave'} at 127.0.0.1:40052
I20260709 12:08:18.354203 11602 raft_consensus.cc:606] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 2 LEADER]: Received request to transfer leadership to TS 9a8e2255342c492eb37a3b0daee59dfb
I20260709 12:08:18.362718 11452 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53e6057698b843b18fff49462dc3679e"
dest_uuid: "e65f179f7b664838b6770533692da7aa"
mode: GRACEFUL
new_leader_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
 from {username='slave'} at 127.0.0.1:56982
I20260709 12:08:18.363296 11452 raft_consensus.cc:606] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [term 1 LEADER]: Received request to transfer leadership to TS 31512bd2b8c74ab89e184a85332e6f9d
I20260709 12:08:18.509836 11738 raft_consensus.cc:993] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa: : Instructing follower 9a8e2255342c492eb37a3b0daee59dfb to start an election
I20260709 12:08:18.510478 11686 raft_consensus.cc:1081] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 1 LEADER]: Signalling peer 9a8e2255342c492eb37a3b0daee59dfb to start an election
I20260709 12:08:18.512528 11527 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "071aa019eff14096b27414993b8e9773"
dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
 from {username='slave'} at 127.0.0.1:36848
I20260709 12:08:18.513099 11527 raft_consensus.cc:493] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:18.513372 11527 raft_consensus.cc:3060] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:18.518436 11527 raft_consensus.cc:515] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:18.520211 11527 leader_election.cc:290] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 2 election: Requested vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:18.521193 11452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "071aa019eff14096b27414993b8e9773" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e65f179f7b664838b6770533692da7aa"
I20260709 12:08:18.521644 11602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "071aa019eff14096b27414993b8e9773" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
I20260709 12:08:18.522166 11452 raft_consensus.cc:3055] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:18.522298 11602 raft_consensus.cc:3060] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:18.522601 11452 raft_consensus.cc:740] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e65f179f7b664838b6770533692da7aa, State: Running, Role: LEADER
I20260709 12:08:18.523602 11452 consensus_queue.cc:260] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:18.525115 11452 raft_consensus.cc:3060] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:18.530262 11602 raft_consensus.cc:2468] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8e2255342c492eb37a3b0daee59dfb in term 2.
I20260709 12:08:18.531842 11491 leader_election.cc:304] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, 9a8e2255342c492eb37a3b0daee59dfb; no voters: 
I20260709 12:08:18.532130 11452 raft_consensus.cc:2468] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8e2255342c492eb37a3b0daee59dfb in term 2.
I20260709 12:08:18.532735 11679 raft_consensus.cc:2804] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:18.533952 11679 raft_consensus.cc:697] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 LEADER]: Becoming Leader. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Running, Role: LEADER
I20260709 12:08:18.535138 11679 consensus_queue.cc:237] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:18.542722 11350 catalog_manager.cc:5697] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb reported cstate change: term changed from 1 to 2, leader changed from e65f179f7b664838b6770533692da7aa (127.11.13.129) to 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130). New cstate: current_term: 2 leader_uuid: "9a8e2255342c492eb37a3b0daee59dfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:18.676074 11738 raft_consensus.cc:993] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa: : Instructing follower 31512bd2b8c74ab89e184a85332e6f9d to start an election
I20260709 12:08:18.676592 11738 raft_consensus.cc:1081] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [term 1 LEADER]: Signalling peer 31512bd2b8c74ab89e184a85332e6f9d to start an election
I20260709 12:08:18.679095 11602 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "53e6057698b843b18fff49462dc3679e"
dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
 from {username='slave'} at 127.0.0.1:40030
I20260709 12:08:18.679723 11602 raft_consensus.cc:493] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:18.680128 11602 raft_consensus.cc:3060] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:18.684913 11602 raft_consensus.cc:515] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:18.686646 11602 leader_election.cc:290] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 2 election: Requested vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537)
I20260709 12:08:18.687614 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6057698b843b18fff49462dc3679e" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
I20260709 12:08:18.687515 11452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53e6057698b843b18fff49462dc3679e" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e65f179f7b664838b6770533692da7aa"
I20260709 12:08:18.688235 11527 raft_consensus.cc:3060] T 53e6057698b843b18fff49462dc3679e P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:18.688289 11452 raft_consensus.cc:3055] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:18.688763 11452 raft_consensus.cc:740] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e65f179f7b664838b6770533692da7aa, State: Running, Role: LEADER
I20260709 12:08:18.689576 11452 consensus_queue.cc:260] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:18.690829 11452 raft_consensus.cc:3060] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:18.695528 11527 raft_consensus.cc:2468] T 53e6057698b843b18fff49462dc3679e P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 2.
I20260709 12:08:18.697196 11566 leader_election.cc:304] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, 9a8e2255342c492eb37a3b0daee59dfb; no voters: 
I20260709 12:08:18.697606 11452 raft_consensus.cc:2468] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 2.
I20260709 12:08:18.698236 11744 raft_consensus.cc:2804] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:18.698774 11744 raft_consensus.cc:697] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [term 2 LEADER]: Becoming Leader. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Running, Role: LEADER
I20260709 12:08:18.699667 11744 consensus_queue.cc:237] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:18.707779 11350 catalog_manager.cc:5697] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d reported cstate change: term changed from 1 to 2, leader changed from e65f179f7b664838b6770533692da7aa (127.11.13.129) to 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131). New cstate: current_term: 2 leader_uuid: "31512bd2b8c74ab89e184a85332e6f9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:18.762074 11732 raft_consensus.cc:993] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d: : Instructing follower 9a8e2255342c492eb37a3b0daee59dfb to start an election
I20260709 12:08:18.762588 11744 raft_consensus.cc:1081] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 2 LEADER]: Signalling peer 9a8e2255342c492eb37a3b0daee59dfb to start an election
I20260709 12:08:18.764132 11527 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74"
dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
 from {username='slave'} at 127.0.0.1:36854
I20260709 12:08:18.764659 11527 raft_consensus.cc:493] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:18.765187 11527 raft_consensus.cc:3060] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:18.769927 11527 raft_consensus.cc:515] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:18.771744 11527 leader_election.cc:290] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 3 election: Requested vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:18.772557 11452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "e65f179f7b664838b6770533692da7aa"
I20260709 12:08:18.772871 11602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
I20260709 12:08:18.773330 11452 raft_consensus.cc:3060] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:18.773361 11602 raft_consensus.cc:3055] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:08:18.773890 11602 raft_consensus.cc:740] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Running, Role: LEADER
I20260709 12:08:18.774771 11602 consensus_queue.cc:260] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:18.775959 11602 raft_consensus.cc:3060] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:18.780423 11452 raft_consensus.cc:2468] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8e2255342c492eb37a3b0daee59dfb in term 3.
I20260709 12:08:18.782249 11491 leader_election.cc:304] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a8e2255342c492eb37a3b0daee59dfb, e65f179f7b664838b6770533692da7aa; no voters: 
I20260709 12:08:18.783092 11602 raft_consensus.cc:2468] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8e2255342c492eb37a3b0daee59dfb in term 3.
I20260709 12:08:18.783254 11679 raft_consensus.cc:2804] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:08:18.783897 11679 raft_consensus.cc:697] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 3 LEADER]: Becoming Leader. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Running, Role: LEADER
I20260709 12:08:18.785079 11679 consensus_queue.cc:237] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:18.792912 11350 catalog_manager.cc:5697] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb reported cstate change: term changed from 2 to 3, leader changed from 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131) to 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130). New cstate: current_term: 3 leader_uuid: "9a8e2255342c492eb37a3b0daee59dfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:18.827517 11732 raft_consensus.cc:993] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d: : Instructing follower e65f179f7b664838b6770533692da7aa to start an election
I20260709 12:08:18.827965 11732 raft_consensus.cc:1081] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 LEADER]: Signalling peer e65f179f7b664838b6770533692da7aa to start an election
I20260709 12:08:18.829818 11452 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c7f41683de3849cfb6a5e4c93161ce46"
dest_uuid: "e65f179f7b664838b6770533692da7aa"
 from {username='slave'} at 127.0.0.1:56976
I20260709 12:08:18.830564 11452 raft_consensus.cc:493] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:18.830852 11452 raft_consensus.cc:3060] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:18.835656 11452 raft_consensus.cc:515] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:18.837930 11452 leader_election.cc:290] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 2 election: Requested vote from peers 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:18.839053 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f41683de3849cfb6a5e4c93161ce46" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
I20260709 12:08:18.839277 11602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f41683de3849cfb6a5e4c93161ce46" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
I20260709 12:08:18.839632 11527 raft_consensus.cc:3060] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:18.839892 11602 raft_consensus.cc:3055] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:18.840248 11602 raft_consensus.cc:740] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Running, Role: LEADER
I20260709 12:08:18.840970 11602 consensus_queue.cc:260] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:18.842012 11602 raft_consensus.cc:3060] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:18.845239 11527 raft_consensus.cc:2468] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65f179f7b664838b6770533692da7aa in term 2.
I20260709 12:08:18.846457 11417 leader_election.cc:304] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a8e2255342c492eb37a3b0daee59dfb, e65f179f7b664838b6770533692da7aa; no voters: 
I20260709 12:08:18.847048 11602 raft_consensus.cc:2468] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65f179f7b664838b6770533692da7aa in term 2.
I20260709 12:08:18.847450 11753 raft_consensus.cc:2804] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:18.847939 11753 raft_consensus.cc:697] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [term 2 LEADER]: Becoming Leader. State: Replica: e65f179f7b664838b6770533692da7aa, State: Running, Role: LEADER
I20260709 12:08:18.848706 11753 consensus_queue.cc:237] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:18.855155 11350 catalog_manager.cc:5697] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa reported cstate change: term changed from 1 to 2, leader changed from 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131) to e65f179f7b664838b6770533692da7aa (127.11.13.129). New cstate: current_term: 2 leader_uuid: "e65f179f7b664838b6770533692da7aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:18.863977 11744 raft_consensus.cc:993] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d: : Instructing follower e65f179f7b664838b6770533692da7aa to start an election
I20260709 12:08:18.864943 11732 raft_consensus.cc:1081] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 1 LEADER]: Signalling peer e65f179f7b664838b6770533692da7aa to start an election
I20260709 12:08:18.867141 11452 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "44710cf3b4a748969c249dcbeccef11d"
dest_uuid: "e65f179f7b664838b6770533692da7aa"
 from {username='slave'} at 127.0.0.1:56976
I20260709 12:08:18.867844 11452 raft_consensus.cc:493] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:18.868157 11452 raft_consensus.cc:3060] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:18.873399 11452 raft_consensus.cc:515] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:18.875415 11452 leader_election.cc:290] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 2 election: Requested vote from peers 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:18.876451 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44710cf3b4a748969c249dcbeccef11d" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
I20260709 12:08:18.876782 11602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44710cf3b4a748969c249dcbeccef11d" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
I20260709 12:08:18.877185 11527 raft_consensus.cc:3060] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:18.877449 11602 raft_consensus.cc:3055] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:18.877861 11602 raft_consensus.cc:740] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Running, Role: LEADER
I20260709 12:08:18.878787 11602 consensus_queue.cc:260] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:18.880146 11602 raft_consensus.cc:3060] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:18.882687 11527 raft_consensus.cc:2468] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65f179f7b664838b6770533692da7aa in term 2.
I20260709 12:08:18.883754 11417 leader_election.cc:304] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a8e2255342c492eb37a3b0daee59dfb, e65f179f7b664838b6770533692da7aa; no voters: 
I20260709 12:08:18.884424 11753 raft_consensus.cc:2804] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:18.884855 11753 raft_consensus.cc:697] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [term 2 LEADER]: Becoming Leader. State: Replica: e65f179f7b664838b6770533692da7aa, State: Running, Role: LEADER
I20260709 12:08:18.885711 11753 consensus_queue.cc:237] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:18.887195 11602 raft_consensus.cc:2468] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65f179f7b664838b6770533692da7aa in term 2.
I20260709 12:08:18.894537 11350 catalog_manager.cc:5697] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa reported cstate change: term changed from 1 to 2, leader changed from 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131) to e65f179f7b664838b6770533692da7aa (127.11.13.129). New cstate: current_term: 2 leader_uuid: "e65f179f7b664838b6770533692da7aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:18.913780 11756 raft_consensus.cc:993] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d: : Instructing follower 9a8e2255342c492eb37a3b0daee59dfb to start an election
I20260709 12:08:18.914497 11732 raft_consensus.cc:1081] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 1 LEADER]: Signalling peer 9a8e2255342c492eb37a3b0daee59dfb to start an election
I20260709 12:08:18.916381 11527 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "af01620b86384cce826e034c1cf541fa"
dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
 from {username='slave'} at 127.0.0.1:36854
I20260709 12:08:18.917048 11527 raft_consensus.cc:493] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:18.917414 11527 raft_consensus.cc:3060] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:18.923229 11527 raft_consensus.cc:515] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:18.925247 11527 leader_election.cc:290] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 2 election: Requested vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:18.926684 11452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af01620b86384cce826e034c1cf541fa" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e65f179f7b664838b6770533692da7aa"
I20260709 12:08:18.926839 11602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af01620b86384cce826e034c1cf541fa" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
I20260709 12:08:18.927481 11452 raft_consensus.cc:3060] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:18.927489 11602 raft_consensus.cc:3055] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:18.928048 11602 raft_consensus.cc:740] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Running, Role: LEADER
I20260709 12:08:18.928900 11602 consensus_queue.cc:260] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:18.930100 11602 raft_consensus.cc:3060] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:18.936142 11452 raft_consensus.cc:2468] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8e2255342c492eb37a3b0daee59dfb in term 2.
I20260709 12:08:18.937530 11491 leader_election.cc:304] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a8e2255342c492eb37a3b0daee59dfb, e65f179f7b664838b6770533692da7aa; no voters: 
I20260709 12:08:18.938330 11679 raft_consensus.cc:2804] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:18.938465 11602 raft_consensus.cc:2468] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8e2255342c492eb37a3b0daee59dfb in term 2.
I20260709 12:08:18.938804 11679 raft_consensus.cc:697] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [term 2 LEADER]: Becoming Leader. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Running, Role: LEADER
I20260709 12:08:18.939914 11679 consensus_queue.cc:237] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:18.947295 11350 catalog_manager.cc:5697] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb reported cstate change: term changed from 1 to 2, leader changed from 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131) to 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130). New cstate: current_term: 2 leader_uuid: "9a8e2255342c492eb37a3b0daee59dfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:19.032166 11452 raft_consensus.cc:1275] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Refusing update from remote peer 9a8e2255342c492eb37a3b0daee59dfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:19.033447 11679 consensus_queue.cc:1048] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:19.043493 11602 raft_consensus.cc:1275] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Refusing update from remote peer 9a8e2255342c492eb37a3b0daee59dfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:19.045284 11679 consensus_queue.cc:1048] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:19.173916 11527 raft_consensus.cc:1275] T 53e6057698b843b18fff49462dc3679e P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Refusing update from remote peer 31512bd2b8c74ab89e184a85332e6f9d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:19.175789 11732 consensus_queue.cc:1048] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:19.189402 11452 raft_consensus.cc:1275] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Refusing update from remote peer 31512bd2b8c74ab89e184a85332e6f9d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:19.191694 11756 consensus_queue.cc:1048] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:19.207760 11602 raft_consensus.cc:1275] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Refusing update from remote peer 9a8e2255342c492eb37a3b0daee59dfb: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:08:19.209372 11679 consensus_queue.cc:1048] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:19.219434 11452 raft_consensus.cc:1275] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Refusing update from remote peer 9a8e2255342c492eb37a3b0daee59dfb: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:08:19.221751 11767 consensus_queue.cc:1048] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:08:19.328287 11452 raft_consensus.cc:1275] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Refusing update from remote peer 9a8e2255342c492eb37a3b0daee59dfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:19.329584 11761 consensus_queue.cc:1048] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:19.339282 11602 raft_consensus.cc:1275] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Refusing update from remote peer 9a8e2255342c492eb37a3b0daee59dfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:19.341300 11767 consensus_queue.cc:1048] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:19.358448 11527 raft_consensus.cc:1275] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Refusing update from remote peer e65f179f7b664838b6770533692da7aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:19.359874 11738 consensus_queue.cc:1048] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:19.368515 11602 raft_consensus.cc:1275] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Refusing update from remote peer e65f179f7b664838b6770533692da7aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:19.371248 11601 raft_consensus.cc:1275] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Refusing update from remote peer e65f179f7b664838b6770533692da7aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:19.371255 11771 consensus_queue.cc:1048] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:08:19.373327 11753 consensus_queue.cc:1048] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:19.386323 11527 raft_consensus.cc:1275] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Refusing update from remote peer e65f179f7b664838b6770533692da7aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:19.389115 11771 consensus_queue.cc:1048] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:21.401598 11318 auto_leader_rebalancer-test.cc:294] The leader distribution is 
I20260709 12:08:21.402594 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:21.427522 11527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "071aa019eff14096b27414993b8e9773"
dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
mode: GRACEFUL
new_leader_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
 from {username='slave'} at 127.0.0.1:36860
I20260709 12:08:21.428200 11527 raft_consensus.cc:606] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 LEADER]: Received request to transfer leadership to TS 31512bd2b8c74ab89e184a85332e6f9d
I20260709 12:08:21.442590 11452 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7f41683de3849cfb6a5e4c93161ce46"
dest_uuid: "e65f179f7b664838b6770533692da7aa"
mode: GRACEFUL
new_leader_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
 from {username='slave'} at 127.0.0.1:56998
I20260709 12:08:21.443156 11452 raft_consensus.cc:606] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [term 2 LEADER]: Received request to transfer leadership to TS 31512bd2b8c74ab89e184a85332e6f9d
I20260709 12:08:21.444458 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:08:21.444972 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:21.592723 11772 raft_consensus.cc:993] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa: : Instructing follower 31512bd2b8c74ab89e184a85332e6f9d to start an election
I20260709 12:08:21.593185 11772 raft_consensus.cc:1081] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [term 2 LEADER]: Signalling peer 31512bd2b8c74ab89e184a85332e6f9d to start an election
I20260709 12:08:21.595026 11601 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c7f41683de3849cfb6a5e4c93161ce46"
dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
 from {username='slave'} at 127.0.0.1:40030
I20260709 12:08:21.595597 11601 raft_consensus.cc:493] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:21.595940 11601 raft_consensus.cc:3060] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:21.602743 11601 raft_consensus.cc:515] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:21.604890 11601 leader_election.cc:290] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 3 election: Requested vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537)
I20260709 12:08:21.606117 11452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f41683de3849cfb6a5e4c93161ce46" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e65f179f7b664838b6770533692da7aa"
I20260709 12:08:21.606698 11452 raft_consensus.cc:3055] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:08:21.606774 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f41683de3849cfb6a5e4c93161ce46" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
I20260709 12:08:21.607000 11452 raft_consensus.cc:740] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e65f179f7b664838b6770533692da7aa, State: Running, Role: LEADER
I20260709 12:08:21.607416 11527 raft_consensus.cc:3060] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:21.607883 11452 consensus_queue.cc:260] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:21.609104 11452 raft_consensus.cc:3060] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:21.613113 11527 raft_consensus.cc:2468] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 3.
I20260709 12:08:21.614030 11452 raft_consensus.cc:2468] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 3.
I20260709 12:08:21.615097 11566 leader_election.cc:304] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, 9a8e2255342c492eb37a3b0daee59dfb; no voters: 
I20260709 12:08:21.616201 11732 raft_consensus.cc:2804] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:08:21.617614 11732 raft_consensus.cc:697] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 LEADER]: Becoming Leader. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Running, Role: LEADER
I20260709 12:08:21.618544 11732 consensus_queue.cc:237] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:21.627398 11349 catalog_manager.cc:5697] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d reported cstate change: term changed from 2 to 3, leader changed from e65f179f7b664838b6770533692da7aa (127.11.13.129) to 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131). New cstate: current_term: 3 leader_uuid: "31512bd2b8c74ab89e184a85332e6f9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:21.662801 11679 raft_consensus.cc:993] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb: : Instructing follower 31512bd2b8c74ab89e184a85332e6f9d to start an election
I20260709 12:08:21.663305 11789 raft_consensus.cc:1081] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 LEADER]: Signalling peer 31512bd2b8c74ab89e184a85332e6f9d to start an election
I20260709 12:08:21.665025 11601 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "071aa019eff14096b27414993b8e9773"
dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
 from {username='slave'} at 127.0.0.1:40046
I20260709 12:08:21.665674 11601 raft_consensus.cc:493] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:21.665990 11601 raft_consensus.cc:3060] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:21.672745 11601 raft_consensus.cc:515] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:21.674845 11601 leader_election.cc:290] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 3 election: Requested vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537)
I20260709 12:08:21.676117 11452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "071aa019eff14096b27414993b8e9773" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e65f179f7b664838b6770533692da7aa"
I20260709 12:08:21.676189 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "071aa019eff14096b27414993b8e9773" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
I20260709 12:08:21.676862 11452 raft_consensus.cc:3060] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:21.676951 11527 raft_consensus.cc:3055] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:08:21.677367 11527 raft_consensus.cc:740] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Running, Role: LEADER
I20260709 12:08:21.678071 11527 consensus_queue.cc:260] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:21.679221 11527 raft_consensus.cc:3060] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:21.682938 11452 raft_consensus.cc:2468] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 3.
I20260709 12:08:21.684041 11565 leader_election.cc:304] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, e65f179f7b664838b6770533692da7aa; no voters: 
I20260709 12:08:21.684763 11732 raft_consensus.cc:2804] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:08:21.684989 11527 raft_consensus.cc:2468] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 3.
I20260709 12:08:21.685228 11732 raft_consensus.cc:697] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 LEADER]: Becoming Leader. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Running, Role: LEADER
I20260709 12:08:21.686241 11732 consensus_queue.cc:237] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:21.693185 11349 catalog_manager.cc:5697] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d reported cstate change: term changed from 2 to 3, leader changed from 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130) to 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131). New cstate: current_term: 3 leader_uuid: "31512bd2b8c74ab89e184a85332e6f9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:22.133445 11527 raft_consensus.cc:1275] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Refusing update from remote peer 31512bd2b8c74ab89e184a85332e6f9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:22.135097 11732 consensus_queue.cc:1048] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:08:22.147459 11452 raft_consensus.cc:1275] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Refusing update from remote peer 31512bd2b8c74ab89e184a85332e6f9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:22.149708 11793 consensus_queue.cc:1048] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:08:22.196229 11452 raft_consensus.cc:1275] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Refusing update from remote peer 31512bd2b8c74ab89e184a85332e6f9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:22.197748 11793 consensus_queue.cc:1048] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:08:22.209928 11527 raft_consensus.cc:1275] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Refusing update from remote peer 31512bd2b8c74ab89e184a85332e6f9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:22.211923 11783 consensus_queue.cc:1048] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:08:22.446403 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:22.453691 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:22.454257 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:23.455542 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:23.461069 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:23.461551 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:24.462656 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:24.467990 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:24.468490 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:25.469501 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:25.474758 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:25.475231 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:26.476440 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:26.483770 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:26.484256 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:27.485355 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:27.490603 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:27.491091 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:28.492192 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:28.497818 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:28.498363 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:29.499461 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:29.504884 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:29.505430 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:30.507073 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:30.515115 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:30.515645 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:31.553218 11318 auto_leader_rebalancer-test.cc:313] The leader distribution is 
I20260709 12:08:31.559712 11453 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f2fc2c80a37c43938dcb1dbaeacc2cc5"
dest_uuid: "e65f179f7b664838b6770533692da7aa"
mode: GRACEFUL
new_leader_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
 from {username='slave'} at 127.0.0.1:56982
I20260709 12:08:31.560554 11453 raft_consensus.cc:606] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 1 LEADER]: Received request to transfer leadership to TS 9a8e2255342c492eb37a3b0daee59dfb
I20260709 12:08:31.566991 11601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c7f41683de3849cfb6a5e4c93161ce46"
dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
mode: GRACEFUL
new_leader_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
 from {username='slave'} at 127.0.0.1:40052
I20260709 12:08:31.567577 11601 raft_consensus.cc:606] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 LEADER]: Received request to transfer leadership to TS 9a8e2255342c492eb37a3b0daee59dfb
I20260709 12:08:31.573987 11453 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "44710cf3b4a748969c249dcbeccef11d"
dest_uuid: "e65f179f7b664838b6770533692da7aa"
mode: GRACEFUL
new_leader_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
 from {username='slave'} at 127.0.0.1:56982
I20260709 12:08:31.574661 11453 raft_consensus.cc:606] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [term 2 LEADER]: Received request to transfer leadership to TS 9a8e2255342c492eb37a3b0daee59dfb
I20260709 12:08:31.581589 11453 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7bbcb6cf7a2e4076b40fbde18996ddce"
dest_uuid: "e65f179f7b664838b6770533692da7aa"
mode: GRACEFUL
new_leader_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
 from {username='slave'} at 127.0.0.1:56982
I20260709 12:08:31.582206 11453 raft_consensus.cc:606] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 1 LEADER]: Received request to transfer leadership to TS 9a8e2255342c492eb37a3b0daee59dfb
I20260709 12:08:31.588146 11601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "071aa019eff14096b27414993b8e9773"
dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
mode: GRACEFUL
new_leader_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
 from {username='slave'} at 127.0.0.1:40052
I20260709 12:08:31.588735 11601 raft_consensus.cc:606] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 LEADER]: Received request to transfer leadership to TS 9a8e2255342c492eb37a3b0daee59dfb
I20260709 12:08:31.712471 11801 raft_consensus.cc:993] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa: : Instructing follower 9a8e2255342c492eb37a3b0daee59dfb to start an election
I20260709 12:08:31.712873 11801 raft_consensus.cc:1081] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [term 2 LEADER]: Signalling peer 9a8e2255342c492eb37a3b0daee59dfb to start an election
I20260709 12:08:31.714848 11527 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "44710cf3b4a748969c249dcbeccef11d"
dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
 from {username='slave'} at 127.0.0.1:36848
I20260709 12:08:31.715372 11527 raft_consensus.cc:493] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:31.715826 11527 raft_consensus.cc:3060] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:31.721921 11527 raft_consensus.cc:515] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:31.724218 11527 leader_election.cc:290] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 3 election: Requested vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:31.725163 11453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44710cf3b4a748969c249dcbeccef11d" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e65f179f7b664838b6770533692da7aa"
I20260709 12:08:31.725383 11601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44710cf3b4a748969c249dcbeccef11d" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
I20260709 12:08:31.726078 11601 raft_consensus.cc:3060] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:31.726078 11453 raft_consensus.cc:3055] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:08:31.726747 11453 raft_consensus.cc:740] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e65f179f7b664838b6770533692da7aa, State: Running, Role: LEADER
I20260709 12:08:31.727768 11453 consensus_queue.cc:260] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:31.729233 11453 raft_consensus.cc:3060] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:31.734710 11601 raft_consensus.cc:2468] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8e2255342c492eb37a3b0daee59dfb in term 3.
I20260709 12:08:31.736935 11491 leader_election.cc:304] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, 9a8e2255342c492eb37a3b0daee59dfb; no voters: 
I20260709 12:08:31.737898 11805 raft_consensus.cc:2804] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:08:31.738171 11453 raft_consensus.cc:2468] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8e2255342c492eb37a3b0daee59dfb in term 3.
I20260709 12:08:31.739334 11805 raft_consensus.cc:697] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [term 3 LEADER]: Becoming Leader. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Running, Role: LEADER
I20260709 12:08:31.740487 11805 consensus_queue.cc:237] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:31.741698 11808 raft_consensus.cc:993] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa: : Instructing follower 9a8e2255342c492eb37a3b0daee59dfb to start an election
I20260709 12:08:31.742252 11801 raft_consensus.cc:1081] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 1 LEADER]: Signalling peer 9a8e2255342c492eb37a3b0daee59dfb to start an election
I20260709 12:08:31.744271 11527 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7bbcb6cf7a2e4076b40fbde18996ddce"
dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
 from {username='slave'} at 127.0.0.1:36848
I20260709 12:08:31.744999 11527 raft_consensus.cc:493] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:31.745522 11527 raft_consensus.cc:3060] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:31.748991 11347 catalog_manager.cc:5697] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb reported cstate change: term changed from 2 to 3, leader changed from e65f179f7b664838b6770533692da7aa (127.11.13.129) to 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130). New cstate: current_term: 3 leader_uuid: "9a8e2255342c492eb37a3b0daee59dfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:31.750761 11527 raft_consensus.cc:515] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:31.752519 11527 leader_election.cc:290] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 2 election: Requested vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:31.753351 11453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bbcb6cf7a2e4076b40fbde18996ddce" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e65f179f7b664838b6770533692da7aa"
I20260709 12:08:31.754030 11453 raft_consensus.cc:3055] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:31.754087 11601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bbcb6cf7a2e4076b40fbde18996ddce" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
I20260709 12:08:31.754580 11453 raft_consensus.cc:740] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e65f179f7b664838b6770533692da7aa, State: Running, Role: LEADER
I20260709 12:08:31.754750 11601 raft_consensus.cc:3060] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:31.755546 11453 consensus_queue.cc:260] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:31.757196 11453 raft_consensus.cc:3060] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:31.762107 11601 raft_consensus.cc:2468] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8e2255342c492eb37a3b0daee59dfb in term 2.
I20260709 12:08:31.763895 11453 raft_consensus.cc:2468] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8e2255342c492eb37a3b0daee59dfb in term 2.
I20260709 12:08:31.764314 11491 leader_election.cc:304] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, 9a8e2255342c492eb37a3b0daee59dfb; no voters: 
I20260709 12:08:31.765322 11805 raft_consensus.cc:2804] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:31.765901 11805 raft_consensus.cc:697] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 2 LEADER]: Becoming Leader. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Running, Role: LEADER
I20260709 12:08:31.766834 11805 consensus_queue.cc:237] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:31.774060 11347 catalog_manager.cc:5697] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb reported cstate change: term changed from 1 to 2, leader changed from e65f179f7b664838b6770533692da7aa (127.11.13.129) to 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130). New cstate: current_term: 2 leader_uuid: "9a8e2255342c492eb37a3b0daee59dfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:31.797724 11808 raft_consensus.cc:993] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa: : Instructing follower 9a8e2255342c492eb37a3b0daee59dfb to start an election
I20260709 12:08:31.798295 11801 raft_consensus.cc:1081] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 1 LEADER]: Signalling peer 9a8e2255342c492eb37a3b0daee59dfb to start an election
I20260709 12:08:31.800263 11527 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f2fc2c80a37c43938dcb1dbaeacc2cc5"
dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
 from {username='slave'} at 127.0.0.1:36848
I20260709 12:08:31.800820 11527 raft_consensus.cc:493] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:31.801136 11527 raft_consensus.cc:3060] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:31.805698 11527 raft_consensus.cc:515] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } } peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } }
I20260709 12:08:31.807518 11527 leader_election.cc:290] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 2 election: Requested vote from peers 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547), e65f179f7b664838b6770533692da7aa (127.11.13.129:35345)
I20260709 12:08:31.809089 11601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fc2c80a37c43938dcb1dbaeacc2cc5" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
I20260709 12:08:31.809283 11453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fc2c80a37c43938dcb1dbaeacc2cc5" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e65f179f7b664838b6770533692da7aa"
I20260709 12:08:31.809830 11601 raft_consensus.cc:3060] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:31.809859 11453 raft_consensus.cc:3055] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:31.810425 11453 raft_consensus.cc:740] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e65f179f7b664838b6770533692da7aa, State: Running, Role: LEADER
I20260709 12:08:31.811304 11453 consensus_queue.cc:260] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } } peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } }
I20260709 12:08:31.812541 11453 raft_consensus.cc:3060] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:31.816740 11601 raft_consensus.cc:2468] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8e2255342c492eb37a3b0daee59dfb in term 2.
I20260709 12:08:31.818066 11491 leader_election.cc:304] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, 9a8e2255342c492eb37a3b0daee59dfb; no voters: 
I20260709 12:08:31.819149 11805 raft_consensus.cc:2804] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:31.819650 11805 raft_consensus.cc:697] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 LEADER]: Becoming Leader. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Running, Role: LEADER
I20260709 12:08:31.819855 11453 raft_consensus.cc:2468] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8e2255342c492eb37a3b0daee59dfb in term 2.
I20260709 12:08:31.820765 11805 consensus_queue.cc:237] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } } peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } }
I20260709 12:08:31.829984 11347 catalog_manager.cc:5697] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb reported cstate change: term changed from 1 to 2, leader changed from e65f179f7b664838b6770533692da7aa (127.11.13.129) to 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130). New cstate: current_term: 2 leader_uuid: "9a8e2255342c492eb37a3b0daee59dfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:31.888980 11807 raft_consensus.cc:993] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d: : Instructing follower 9a8e2255342c492eb37a3b0daee59dfb to start an election
I20260709 12:08:31.889595 11817 raft_consensus.cc:1081] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 LEADER]: Signalling peer 9a8e2255342c492eb37a3b0daee59dfb to start an election
I20260709 12:08:31.891409 11527 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "071aa019eff14096b27414993b8e9773"
dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
 from {username='slave'} at 127.0.0.1:36854
I20260709 12:08:31.891945 11527 raft_consensus.cc:493] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:31.892270 11527 raft_consensus.cc:3060] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:08:31.898644 11807 raft_consensus.cc:993] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d: : Instructing follower 9a8e2255342c492eb37a3b0daee59dfb to start an election
I20260709 12:08:31.898783 11527 raft_consensus.cc:515] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:31.899433 11807 raft_consensus.cc:1081] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 LEADER]: Signalling peer 9a8e2255342c492eb37a3b0daee59dfb to start an election
I20260709 12:08:31.901147 11527 leader_election.cc:290] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 4 election: Requested vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:31.901994 11453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "071aa019eff14096b27414993b8e9773" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e65f179f7b664838b6770533692da7aa"
I20260709 12:08:31.902243 11601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "071aa019eff14096b27414993b8e9773" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
I20260709 12:08:31.902737 11453 raft_consensus.cc:3060] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:08:31.902958 11601 raft_consensus.cc:3055] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:08:31.903357 11527 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c7f41683de3849cfb6a5e4c93161ce46"
dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
 from {username='slave'} at 127.0.0.1:36854
I20260709 12:08:31.903481 11601 raft_consensus.cc:740] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Running, Role: LEADER
I20260709 12:08:31.904023 11527 raft_consensus.cc:493] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:31.904465 11527 raft_consensus.cc:3060] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:08:31.904642 11601 consensus_queue.cc:260] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:31.905992 11601 raft_consensus.cc:3060] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:08:31.908504 11453 raft_consensus.cc:2468] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8e2255342c492eb37a3b0daee59dfb in term 4.
I20260709 12:08:31.910869 11491 leader_election.cc:304] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a8e2255342c492eb37a3b0daee59dfb, e65f179f7b664838b6770533692da7aa; no voters: 
I20260709 12:08:31.911221 11527 raft_consensus.cc:515] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:31.912065 11809 raft_consensus.cc:2804] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:08:31.912791 11601 raft_consensus.cc:2468] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8e2255342c492eb37a3b0daee59dfb in term 4.
I20260709 12:08:31.912683 11809 raft_consensus.cc:697] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 4 LEADER]: Becoming Leader. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Running, Role: LEADER
I20260709 12:08:31.913825 11527 leader_election.cc:290] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 4 election: Requested vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:31.914369 11453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f41683de3849cfb6a5e4c93161ce46" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e65f179f7b664838b6770533692da7aa"
I20260709 12:08:31.914028 11809 consensus_queue.cc:237] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:31.915079 11453 raft_consensus.cc:3060] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:08:31.915251 11601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c7f41683de3849cfb6a5e4c93161ce46" candidate_uuid: "9a8e2255342c492eb37a3b0daee59dfb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
I20260709 12:08:31.915808 11601 raft_consensus.cc:3055] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:08:31.916055 11601 raft_consensus.cc:740] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Running, Role: LEADER
I20260709 12:08:31.916854 11601 consensus_queue.cc:260] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:31.917809 11601 raft_consensus.cc:3060] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:08:31.922888 11601 raft_consensus.cc:2468] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8e2255342c492eb37a3b0daee59dfb in term 4.
I20260709 12:08:31.924327 11453 raft_consensus.cc:2468] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9a8e2255342c492eb37a3b0daee59dfb in term 4.
I20260709 12:08:31.924355 11491 leader_election.cc:304] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, 9a8e2255342c492eb37a3b0daee59dfb; no voters: 
I20260709 12:08:31.925518 11809 raft_consensus.cc:2804] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:08:31.926054 11809 raft_consensus.cc:697] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [term 4 LEADER]: Becoming Leader. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Running, Role: LEADER
I20260709 12:08:31.927011 11809 consensus_queue.cc:237] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:31.927454 11347 catalog_manager.cc:5697] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb reported cstate change: term changed from 3 to 4, leader changed from 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131) to 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130). New cstate: current_term: 4 leader_uuid: "9a8e2255342c492eb37a3b0daee59dfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:31.939957 11347 catalog_manager.cc:5697] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb reported cstate change: term changed from 3 to 4, leader changed from 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131) to 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130). New cstate: current_term: 4 leader_uuid: "9a8e2255342c492eb37a3b0daee59dfb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:32.131527 11453 raft_consensus.cc:1275] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Refusing update from remote peer 9a8e2255342c492eb37a3b0daee59dfb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:32.133203 11809 consensus_queue.cc:1048] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:08:32.146438 11601 raft_consensus.cc:1275] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Refusing update from remote peer 9a8e2255342c492eb37a3b0daee59dfb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:32.147902 11805 consensus_queue.cc:1048] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:08:32.168301 11453 raft_consensus.cc:1275] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Refusing update from remote peer 9a8e2255342c492eb37a3b0daee59dfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:32.169729 11805 consensus_queue.cc:1048] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:08:32.179579 11601 raft_consensus.cc:1275] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Refusing update from remote peer 9a8e2255342c492eb37a3b0daee59dfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:32.181861 11809 consensus_queue.cc:1048] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:32.346714 11452 raft_consensus.cc:1275] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 4 FOLLOWER]: Refusing update from remote peer 9a8e2255342c492eb37a3b0daee59dfb: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:08:32.346778 11453 raft_consensus.cc:1275] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Refusing update from remote peer 9a8e2255342c492eb37a3b0daee59dfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:32.349392 11809 consensus_queue.cc:1048] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:08:32.350076 11821 consensus_queue.cc:1048] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:08:32.368781 11601 raft_consensus.cc:1275] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 4 FOLLOWER]: Refusing update from remote peer 9a8e2255342c492eb37a3b0daee59dfb: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:08:32.370793 11601 raft_consensus.cc:1275] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Refusing update from remote peer 9a8e2255342c492eb37a3b0daee59dfb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:32.370878 11821 consensus_queue.cc:1048] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:08:32.376483 11821 consensus_queue.cc:1048] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:08:32.502207 11453 raft_consensus.cc:1275] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [term 4 FOLLOWER]: Refusing update from remote peer 9a8e2255342c492eb37a3b0daee59dfb: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:08:32.503216 11602 raft_consensus.cc:1275] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 4 FOLLOWER]: Refusing update from remote peer 9a8e2255342c492eb37a3b0daee59dfb: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:08:32.505519 11832 consensus_queue.cc:1048] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:08:32.506521 11805 consensus_queue.cc:1048] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [LEADER]: Connected to new peer: Peer: permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:08:33.564551 11381 debug-util.cc:398] Leaking SignalData structure 0x7b0800062060 after lost signal to thread 11321
W20260709 12:08:33.565554 11381 debug-util.cc:398] Leaking SignalData structure 0x7b0800062740 after lost signal to thread 11384
W20260709 12:08:33.566737 11381 debug-util.cc:398] Leaking SignalData structure 0x7b08001d8940 after lost signal to thread 11477
W20260709 12:08:33.567703 11381 debug-util.cc:398] Leaking SignalData structure 0x7b08001ae0a0 after lost signal to thread 11552
W20260709 12:08:33.568409 11381 debug-util.cc:398] Leaking SignalData structure 0x7b08001571e0 after lost signal to thread 11626
I20260709 12:08:34.709336 11318 auto_leader_rebalancer-test.cc:328] The leader distribution is 
I20260709 12:08:34.710341 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:34.719249 11527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "071aa019eff14096b27414993b8e9773"
dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
mode: GRACEFUL
new_leader_uuid: "e65f179f7b664838b6770533692da7aa"
 from {username='slave'} at 127.0.0.1:36860
I20260709 12:08:34.720219 11527 raft_consensus.cc:606] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 4 LEADER]: Received request to transfer leadership to TS e65f179f7b664838b6770533692da7aa
I20260709 12:08:34.723596 11527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74"
dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
mode: GRACEFUL
new_leader_uuid: "e65f179f7b664838b6770533692da7aa"
 from {username='slave'} at 127.0.0.1:36860
I20260709 12:08:34.724246 11527 raft_consensus.cc:606] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 3 LEADER]: Received request to transfer leadership to TS e65f179f7b664838b6770533692da7aa
I20260709 12:08:34.727205 11527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7bbcb6cf7a2e4076b40fbde18996ddce"
dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
mode: GRACEFUL
new_leader_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
 from {username='slave'} at 127.0.0.1:36860
I20260709 12:08:34.727878 11527 raft_consensus.cc:606] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 2 LEADER]: Received request to transfer leadership to TS 31512bd2b8c74ab89e184a85332e6f9d
I20260709 12:08:34.731089 11527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "af01620b86384cce826e034c1cf541fa"
dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
mode: GRACEFUL
new_leader_uuid: "e65f179f7b664838b6770533692da7aa"
 from {username='slave'} at 127.0.0.1:36860
I20260709 12:08:34.731698 11527 raft_consensus.cc:606] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [term 2 LEADER]: Received request to transfer leadership to TS e65f179f7b664838b6770533692da7aa
I20260709 12:08:34.735119 11527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f2fc2c80a37c43938dcb1dbaeacc2cc5"
dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
mode: GRACEFUL
new_leader_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
 from {username='slave'} at 127.0.0.1:36860
I20260709 12:08:34.735638 11527 raft_consensus.cc:606] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 LEADER]: Received request to transfer leadership to TS 31512bd2b8c74ab89e184a85332e6f9d
I20260709 12:08:34.736886 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:08:34.737525 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:34.858202 11833 raft_consensus.cc:993] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb: : Instructing follower 31512bd2b8c74ab89e184a85332e6f9d to start an election
I20260709 12:08:34.858920 11842 raft_consensus.cc:1081] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 2 LEADER]: Signalling peer 31512bd2b8c74ab89e184a85332e6f9d to start an election
I20260709 12:08:34.860569 11602 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7bbcb6cf7a2e4076b40fbde18996ddce"
dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
 from {username='slave'} at 127.0.0.1:40046
I20260709 12:08:34.861104 11602 raft_consensus.cc:493] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:34.861393 11602 raft_consensus.cc:3060] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:34.866465 11602 raft_consensus.cc:515] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:34.868579 11602 leader_election.cc:290] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 3 election: Requested vote from peers e65f179f7b664838b6770533692da7aa (127.11.13.129:35345), 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537)
I20260709 12:08:34.869729 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bbcb6cf7a2e4076b40fbde18996ddce" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
I20260709 12:08:34.869737 11453 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7bbcb6cf7a2e4076b40fbde18996ddce" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e65f179f7b664838b6770533692da7aa"
I20260709 12:08:34.870410 11527 raft_consensus.cc:3055] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:08:34.870555 11453 raft_consensus.cc:3060] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:34.870772 11527 raft_consensus.cc:740] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Running, Role: LEADER
I20260709 12:08:34.871667 11527 consensus_queue.cc:260] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:34.872617 11527 raft_consensus.cc:3060] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:34.879693 11453 raft_consensus.cc:2468] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 3.
I20260709 12:08:34.881723 11565 leader_election.cc:304] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, e65f179f7b664838b6770533692da7aa; no voters: 
I20260709 12:08:34.882088 11527 raft_consensus.cc:2468] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 3.
I20260709 12:08:34.882946 11807 raft_consensus.cc:2804] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:08:34.884727 11807 raft_consensus.cc:697] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [term 3 LEADER]: Becoming Leader. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Running, Role: LEADER
I20260709 12:08:34.886093 11807 consensus_queue.cc:237] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:34.898203 11350 catalog_manager.cc:5697] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d reported cstate change: term changed from 2 to 3, leader changed from 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130) to 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131). New cstate: current_term: 3 leader_uuid: "31512bd2b8c74ab89e184a85332e6f9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:34.907640 11842 raft_consensus.cc:993] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb: : Instructing follower e65f179f7b664838b6770533692da7aa to start an election
I20260709 12:08:34.908257 11833 raft_consensus.cc:1081] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [term 2 LEADER]: Signalling peer e65f179f7b664838b6770533692da7aa to start an election
I20260709 12:08:34.910223 11453 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "af01620b86384cce826e034c1cf541fa"
dest_uuid: "e65f179f7b664838b6770533692da7aa"
 from {username='slave'} at 127.0.0.1:56980
I20260709 12:08:34.910784 11453 raft_consensus.cc:493] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:34.911183 11453 raft_consensus.cc:3060] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:34.918821 11833 raft_consensus.cc:993] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb: : Instructing follower e65f179f7b664838b6770533692da7aa to start an election
I20260709 12:08:34.919369 11833 raft_consensus.cc:1081] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 4 LEADER]: Signalling peer e65f179f7b664838b6770533692da7aa to start an election
I20260709 12:08:34.921214 11450 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "071aa019eff14096b27414993b8e9773"
dest_uuid: "e65f179f7b664838b6770533692da7aa"
 from {username='slave'} at 127.0.0.1:56980
I20260709 12:08:34.921814 11450 raft_consensus.cc:493] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:34.921439 11453 raft_consensus.cc:515] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:34.922230 11450 raft_consensus.cc:3060] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:08:34.924535 11453 leader_election.cc:290] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 3 election: Requested vote from peers 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:34.925457 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af01620b86384cce826e034c1cf541fa" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
I20260709 12:08:34.926084 11527 raft_consensus.cc:3055] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:08:34.926446 11527 raft_consensus.cc:740] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Running, Role: LEADER
I20260709 12:08:34.926692 11602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "af01620b86384cce826e034c1cf541fa" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
I20260709 12:08:34.927256 11527 consensus_queue.cc:260] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:34.927284 11602 raft_consensus.cc:3060] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:34.931969 11843 raft_consensus.cc:993] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb: : Instructing follower e65f179f7b664838b6770533692da7aa to start an election
I20260709 12:08:34.932485 11843 raft_consensus.cc:1081] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 3 LEADER]: Signalling peer e65f179f7b664838b6770533692da7aa to start an election
I20260709 12:08:34.932643 11527 raft_consensus.cc:3060] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:34.933753 11450 raft_consensus.cc:515] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:34.934525 11452 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74"
dest_uuid: "e65f179f7b664838b6770533692da7aa"
 from {username='slave'} at 127.0.0.1:56980
I20260709 12:08:34.935067 11452 raft_consensus.cc:493] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:34.935364 11452 raft_consensus.cc:3060] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:08:34.937641 11601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "071aa019eff14096b27414993b8e9773" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
I20260709 12:08:34.938287 11602 raft_consensus.cc:2468] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65f179f7b664838b6770533692da7aa in term 3.
I20260709 12:08:34.938396 11601 raft_consensus.cc:3060] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:08:34.939568 11526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "071aa019eff14096b27414993b8e9773" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
I20260709 12:08:34.940079 11526 raft_consensus.cc:3055] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:08:34.940321 11526 raft_consensus.cc:740] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Running, Role: LEADER
I20260709 12:08:34.941077 11526 consensus_queue.cc:260] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:34.942238 11526 raft_consensus.cc:3060] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:08:34.947033 11416 leader_election.cc:304] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, e65f179f7b664838b6770533692da7aa; no voters: 
I20260709 12:08:34.949630 11526 raft_consensus.cc:2468] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65f179f7b664838b6770533692da7aa in term 5.
I20260709 12:08:34.945254 11452 raft_consensus.cc:515] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:34.951272 11417 leader_election.cc:304] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9a8e2255342c492eb37a3b0daee59dfb, e65f179f7b664838b6770533692da7aa; no voters: 
I20260709 12:08:34.951759 11527 raft_consensus.cc:2468] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65f179f7b664838b6770533692da7aa in term 3.
I20260709 12:08:34.952093 11601 raft_consensus.cc:2468] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65f179f7b664838b6770533692da7aa in term 5.
I20260709 12:08:34.953657 11450 leader_election.cc:290] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 5 election: Requested vote from peers 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:34.955202 11452 leader_election.cc:290] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 4 election: Requested vote from peers 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537), 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131:37547)
I20260709 12:08:34.957398 11851 raft_consensus.cc:2804] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:08:34.958271 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
I20260709 12:08:34.959340 11527 raft_consensus.cc:3055] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:08:34.959775 11527 raft_consensus.cc:740] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Running, Role: LEADER
I20260709 12:08:34.959904 11601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70c6dea7ace4433a91c235307de05d74" candidate_uuid: "e65f179f7b664838b6770533692da7aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
I20260709 12:08:34.961110 11527 consensus_queue.cc:260] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:34.963063 11527 raft_consensus.cc:3060] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:08:34.960588 11601 raft_consensus.cc:3060] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:08:34.958691 11850 raft_consensus.cc:2804] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:08:34.973452 11601 raft_consensus.cc:2468] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65f179f7b664838b6770533692da7aa in term 4.
I20260709 12:08:34.974942 11416 leader_election.cc:304] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, e65f179f7b664838b6770533692da7aa; no voters: 
I20260709 12:08:34.978673 11851 raft_consensus.cc:697] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [term 3 LEADER]: Becoming Leader. State: Replica: e65f179f7b664838b6770533692da7aa, State: Running, Role: LEADER
I20260709 12:08:34.978960 11527 raft_consensus.cc:2468] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e65f179f7b664838b6770533692da7aa in term 4.
I20260709 12:08:34.980085 11851 consensus_queue.cc:237] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:34.980603 11850 raft_consensus.cc:697] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 5 LEADER]: Becoming Leader. State: Replica: e65f179f7b664838b6770533692da7aa, State: Running, Role: LEADER
I20260709 12:08:34.980458 11853 raft_consensus.cc:2804] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:08:34.981782 11850 consensus_queue.cc:237] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:34.982643 11853 raft_consensus.cc:697] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 4 LEADER]: Becoming Leader. State: Replica: e65f179f7b664838b6770533692da7aa, State: Running, Role: LEADER
I20260709 12:08:34.983843 11853 consensus_queue.cc:237] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } }
I20260709 12:08:35.004841 11350 catalog_manager.cc:5697] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa reported cstate change: term changed from 2 to 3, leader changed from 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130) to e65f179f7b664838b6770533692da7aa (127.11.13.129). New cstate: current_term: 3 leader_uuid: "e65f179f7b664838b6770533692da7aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:35.006644 11350 catalog_manager.cc:5697] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa reported cstate change: term changed from 4 to 5, leader changed from 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130) to e65f179f7b664838b6770533692da7aa (127.11.13.129). New cstate: current_term: 5 leader_uuid: "e65f179f7b664838b6770533692da7aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:35.008275 11350 catalog_manager.cc:5697] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa reported cstate change: term changed from 3 to 4, leader changed from 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130) to e65f179f7b664838b6770533692da7aa (127.11.13.129). New cstate: current_term: 4 leader_uuid: "e65f179f7b664838b6770533692da7aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:35.106966 11842 raft_consensus.cc:993] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb: : Instructing follower 31512bd2b8c74ab89e184a85332e6f9d to start an election
I20260709 12:08:35.107587 11833 raft_consensus.cc:1081] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 LEADER]: Signalling peer 31512bd2b8c74ab89e184a85332e6f9d to start an election
I20260709 12:08:35.109196 11601 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f2fc2c80a37c43938dcb1dbaeacc2cc5"
dest_uuid: "31512bd2b8c74ab89e184a85332e6f9d"
 from {username='slave'} at 127.0.0.1:40046
I20260709 12:08:35.109795 11601 raft_consensus.cc:493] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:35.110168 11601 raft_consensus.cc:3060] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:35.117393 11601 raft_consensus.cc:515] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } } peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } }
I20260709 12:08:35.119825 11601 leader_election.cc:290] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 3 election: Requested vote from peers 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537), e65f179f7b664838b6770533692da7aa (127.11.13.129:35345)
I20260709 12:08:35.120929 11527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fc2c80a37c43938dcb1dbaeacc2cc5" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "9a8e2255342c492eb37a3b0daee59dfb"
I20260709 12:08:35.121050 11452 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f2fc2c80a37c43938dcb1dbaeacc2cc5" candidate_uuid: "31512bd2b8c74ab89e184a85332e6f9d" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e65f179f7b664838b6770533692da7aa"
I20260709 12:08:35.121841 11527 raft_consensus.cc:3055] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:08:35.121881 11452 raft_consensus.cc:3060] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:35.122344 11527 raft_consensus.cc:740] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 9a8e2255342c492eb37a3b0daee59dfb, State: Running, Role: LEADER
I20260709 12:08:35.123243 11527 consensus_queue.cc:260] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } } peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } }
I20260709 12:08:35.124370 11527 raft_consensus.cc:3060] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:08:35.129328 11527 raft_consensus.cc:2468] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 3.
I20260709 12:08:35.130621 11566 leader_election.cc:304] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31512bd2b8c74ab89e184a85332e6f9d, 9a8e2255342c492eb37a3b0daee59dfb; no voters: 
I20260709 12:08:35.130676 11452 raft_consensus.cc:2468] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 31512bd2b8c74ab89e184a85332e6f9d in term 3.
I20260709 12:08:35.131415 11807 raft_consensus.cc:2804] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:08:35.131973 11807 raft_consensus.cc:697] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 LEADER]: Becoming Leader. State: Replica: 31512bd2b8c74ab89e184a85332e6f9d, State: Running, Role: LEADER
I20260709 12:08:35.132766 11807 consensus_queue.cc:237] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } } peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } }
I20260709 12:08:35.142062 11350 catalog_manager.cc:5697] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d reported cstate change: term changed from 2 to 3, leader changed from 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130) to 31512bd2b8c74ab89e184a85332e6f9d (127.11.13.131). New cstate: current_term: 3 leader_uuid: "31512bd2b8c74ab89e184a85332e6f9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:35.433223 11452 raft_consensus.cc:1275] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Refusing update from remote peer 31512bd2b8c74ab89e184a85332e6f9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:35.435075 11807 consensus_queue.cc:1048] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:08:35.440703 11527 raft_consensus.cc:1275] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 4 FOLLOWER]: Refusing update from remote peer e65f179f7b664838b6770533692da7aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:08:35.442711 11853 consensus_queue.cc:1048] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:08:35.450896 11526 raft_consensus.cc:1275] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Refusing update from remote peer 31512bd2b8c74ab89e184a85332e6f9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:35.454670 11807 consensus_queue.cc:1048] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:08:35.473228 11601 raft_consensus.cc:1275] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 4 FOLLOWER]: Refusing update from remote peer e65f179f7b664838b6770533692da7aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:08:35.474936 11850 consensus_queue.cc:1048] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:08:35.494202 11526 raft_consensus.cc:1275] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 5 FOLLOWER]: Refusing update from remote peer e65f179f7b664838b6770533692da7aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:08:35.495857 11853 consensus_queue.cc:1048] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:08:35.507697 11601 raft_consensus.cc:1275] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 5 FOLLOWER]: Refusing update from remote peer e65f179f7b664838b6770533692da7aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:08:35.510505 11853 consensus_queue.cc:1048] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:08:35.541040 11526 raft_consensus.cc:1275] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Refusing update from remote peer e65f179f7b664838b6770533692da7aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:35.542724 11851 consensus_queue.cc:1048] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:08:35.553695 11601 raft_consensus.cc:1275] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Refusing update from remote peer e65f179f7b664838b6770533692da7aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:35.555297 11850 consensus_queue.cc:1048] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "31512bd2b8c74ab89e184a85332e6f9d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 37547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:08:35.657745 11452 raft_consensus.cc:1275] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Refusing update from remote peer 31512bd2b8c74ab89e184a85332e6f9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:35.659497 11807 consensus_queue.cc:1048] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e65f179f7b664838b6770533692da7aa" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 35345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:08:35.669858 11526 raft_consensus.cc:1275] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Refusing update from remote peer 31512bd2b8c74ab89e184a85332e6f9d: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:08:35.671725 11862 consensus_queue.cc:1048] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "9a8e2255342c492eb37a3b0daee59dfb" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:08:35.738771 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:35.744078 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:35.744563 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:36.794811 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:36.804888 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:36.805409 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:08:37.789321 11474 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5060 after lost signal to thread 11321
W20260709 12:08:37.790248 11474 debug-util.cc:398] Leaking SignalData structure 0x7b08000e5740 after lost signal to thread 11384
W20260709 12:08:37.791134 11474 debug-util.cc:398] Leaking SignalData structure 0x7b08001dbc60 after lost signal to thread 11477
W20260709 12:08:37.791909 11474 debug-util.cc:398] Leaking SignalData structure 0x7b0800135620 after lost signal to thread 11552
W20260709 12:08:37.792654 11474 debug-util.cc:398] Leaking SignalData structure 0x7b080018a380 after lost signal to thread 11626
I20260709 12:08:37.853761 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:37.859417 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:37.859936 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:08:38.534658 11549 debug-util.cc:398] Leaking SignalData structure 0x7b080012a060 after lost signal to thread 11321
W20260709 12:08:38.535533 11549 debug-util.cc:398] Leaking SignalData structure 0x7b080012a740 after lost signal to thread 11384
W20260709 12:08:38.536353 11549 debug-util.cc:398] Leaking SignalData structure 0x7b0800197460 after lost signal to thread 11477
W20260709 12:08:38.537225 11549 debug-util.cc:398] Leaking SignalData structure 0x7b08001ec280 after lost signal to thread 11552
W20260709 12:08:38.538039 11549 debug-util.cc:398] Leaking SignalData structure 0x7b08001c4da0 after lost signal to thread 11626
I20260709 12:08:38.861268 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:38.870458 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:38.871147 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:39.872649 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:39.889009 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:39.889676 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:40.890843 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:40.895926 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:40.896425 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:41.897890 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:41.903934 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:41.904465 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:42.906111 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:42.912585 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:42.913188 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:43.914597 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:43.919854 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:43.920396 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:44.951601 11318 auto_leader_rebalancer-test.cc:340] The leader distribution is 
I20260709 12:08:44.952260 11318 tablet_server.cc:179] TabletServer@127.11.13.129:0 shutting down...
W20260709 12:08:44.985409 11565 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:08:44.992935 11565 consensus_peers.cc:597] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d -> Peer e65f179f7b664838b6770533692da7aa (127.11.13.129:35345): Couldn't send request to peer e65f179f7b664838b6770533692da7aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:08:44.996112 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:08:44.997123 11318 tablet_replica.cc:333] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa: stopping tablet replica
I20260709 12:08:44.997993 11318 raft_consensus.cc:2243] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:44.998814 11318 raft_consensus.cc:2272] T 53e6057698b843b18fff49462dc3679e P e65f179f7b664838b6770533692da7aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.001600 11318 tablet_replica.cc:333] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa: stopping tablet replica
I20260709 12:08:45.002602 11318 raft_consensus.cc:2243] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.003369 11318 raft_consensus.cc:2272] T c7f41683de3849cfb6a5e4c93161ce46 P e65f179f7b664838b6770533692da7aa [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.006831 11318 tablet_replica.cc:333] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa: stopping tablet replica
I20260709 12:08:45.007469 11318 raft_consensus.cc:2243] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.007992 11318 raft_consensus.cc:2272] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.010577 11318 tablet_replica.cc:333] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa: stopping tablet replica
I20260709 12:08:45.011229 11318 raft_consensus.cc:2243] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:08:45.012379 11318 raft_consensus.cc:2272] T 70c6dea7ace4433a91c235307de05d74 P e65f179f7b664838b6770533692da7aa [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.014508 11318 tablet_replica.cc:333] T 7ee33eb902604730a36d8bfe692dc297 P e65f179f7b664838b6770533692da7aa: stopping tablet replica
I20260709 12:08:45.015421 11318 raft_consensus.cc:2243] T 7ee33eb902604730a36d8bfe692dc297 P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.016026 11318 raft_consensus.cc:2272] T 7ee33eb902604730a36d8bfe692dc297 P e65f179f7b664838b6770533692da7aa [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:08:45.018003 11491 consensus_peers.cc:597] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb -> Peer e65f179f7b664838b6770533692da7aa (127.11.13.129:35345): Couldn't send request to peer e65f179f7b664838b6770533692da7aa. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:08:45.018957 11318 tablet_replica.cc:333] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa: stopping tablet replica
I20260709 12:08:45.019773 11318 raft_consensus.cc:2243] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.020352 11318 raft_consensus.cc:2272] T 7bbcb6cf7a2e4076b40fbde18996ddce P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.022483 11318 tablet_replica.cc:333] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa: stopping tablet replica
I20260709 12:08:45.023044 11318 raft_consensus.cc:2243] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:08:45.023885 11318 raft_consensus.cc:2272] T af01620b86384cce826e034c1cf541fa P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.025732 11318 tablet_replica.cc:333] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa: stopping tablet replica
I20260709 12:08:45.026335 11318 raft_consensus.cc:2243] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.026791 11318 raft_consensus.cc:2272] T 44710cf3b4a748969c249dcbeccef11d P e65f179f7b664838b6770533692da7aa [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.029078 11318 tablet_replica.cc:333] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa: stopping tablet replica
I20260709 12:08:45.029709 11318 raft_consensus.cc:2243] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:08:45.030930 11318 raft_consensus.cc:2272] T 071aa019eff14096b27414993b8e9773 P e65f179f7b664838b6770533692da7aa [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.150457 11318 tablet_server.cc:196] TabletServer@127.11.13.129:0 shutdown complete.
I20260709 12:08:45.175963 11318 tablet_server.cc:179] TabletServer@127.11.13.130:0 shutting down...
W20260709 12:08:45.214571 11566 consensus_peers.cc:597] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d -> Peer 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537): Couldn't send request to peer 9a8e2255342c492eb37a3b0daee59dfb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:08:45.215785 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:08:45.216702 11318 tablet_replica.cc:333] T 53e6057698b843b18fff49462dc3679e P 9a8e2255342c492eb37a3b0daee59dfb: stopping tablet replica
I20260709 12:08:45.217620 11318 raft_consensus.cc:2243] T 53e6057698b843b18fff49462dc3679e P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.218295 11318 raft_consensus.cc:2272] T 53e6057698b843b18fff49462dc3679e P 9a8e2255342c492eb37a3b0daee59dfb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.220448 11318 tablet_replica.cc:333] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb: stopping tablet replica
I20260709 12:08:45.221053 11318 raft_consensus.cc:2243] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.221539 11318 raft_consensus.cc:2272] T af01620b86384cce826e034c1cf541fa P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.223914 11318 tablet_replica.cc:333] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb: stopping tablet replica
I20260709 12:08:45.224550 11318 raft_consensus.cc:2243] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.225047 11318 raft_consensus.cc:2272] T 70c6dea7ace4433a91c235307de05d74 P 9a8e2255342c492eb37a3b0daee59dfb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.226943 11318 tablet_replica.cc:333] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb: stopping tablet replica
I20260709 12:08:45.227494 11318 raft_consensus.cc:2243] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.227998 11318 raft_consensus.cc:2272] T 7bbcb6cf7a2e4076b40fbde18996ddce P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.230304 11318 tablet_replica.cc:333] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb: stopping tablet replica
I20260709 12:08:45.230850 11318 raft_consensus.cc:2243] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:08:45.231760 11318 raft_consensus.cc:2272] T 7ee33eb902604730a36d8bfe692dc297 P 9a8e2255342c492eb37a3b0daee59dfb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.234191 11318 tablet_replica.cc:333] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb: stopping tablet replica
I20260709 12:08:45.234728 11318 raft_consensus.cc:2243] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.235198 11318 raft_consensus.cc:2272] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:08:45.235673 11565 consensus_peers.cc:597] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d -> Peer e65f179f7b664838b6770533692da7aa (127.11.13.129:35345): Couldn't send request to peer e65f179f7b664838b6770533692da7aa. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:35345: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:08:45.237104 11318 tablet_replica.cc:333] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb: stopping tablet replica
I20260709 12:08:45.237671 11318 raft_consensus.cc:2243] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:08:45.238590 11318 raft_consensus.cc:2272] T c7f41683de3849cfb6a5e4c93161ce46 P 9a8e2255342c492eb37a3b0daee59dfb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.240320 11318 tablet_replica.cc:333] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb: stopping tablet replica
I20260709 12:08:45.240833 11318 raft_consensus.cc:2243] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.241268 11318 raft_consensus.cc:2272] T 071aa019eff14096b27414993b8e9773 P 9a8e2255342c492eb37a3b0daee59dfb [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.243532 11318 tablet_replica.cc:333] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb: stopping tablet replica
I20260709 12:08:45.244189 11318 raft_consensus.cc:2243] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:08:45.245321 11318 raft_consensus.cc:2272] T 44710cf3b4a748969c249dcbeccef11d P 9a8e2255342c492eb37a3b0daee59dfb [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:08:45.259124 11566 consensus_peers.cc:597] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d -> Peer 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537): Couldn't send request to peer 9a8e2255342c492eb37a3b0daee59dfb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:08:45.298195 11318 tablet_server.cc:196] TabletServer@127.11.13.130:0 shutdown complete.
I20260709 12:08:45.320298 11318 tablet_server.cc:179] TabletServer@127.11.13.131:0 shutting down...
W20260709 12:08:45.341492 11566 consensus_peers.cc:597] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d -> Peer 9a8e2255342c492eb37a3b0daee59dfb (127.11.13.130:34537): Couldn't send request to peer 9a8e2255342c492eb37a3b0daee59dfb. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:34537: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:08:45.358753 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:08:45.359593 11318 tablet_replica.cc:333] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d: stopping tablet replica
I20260709 12:08:45.360234 11318 raft_consensus.cc:2243] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.360760 11318 raft_consensus.cc:2272] T af01620b86384cce826e034c1cf541fa P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.362669 11318 tablet_replica.cc:333] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d: stopping tablet replica
I20260709 12:08:45.363215 11318 raft_consensus.cc:2243] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:08:45.364010 11318 raft_consensus.cc:2272] T 53e6057698b843b18fff49462dc3679e P 31512bd2b8c74ab89e184a85332e6f9d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.365890 11318 tablet_replica.cc:333] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d: stopping tablet replica
I20260709 12:08:45.366470 11318 raft_consensus.cc:2243] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.366966 11318 raft_consensus.cc:2272] T 70c6dea7ace4433a91c235307de05d74 P 31512bd2b8c74ab89e184a85332e6f9d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.368716 11318 tablet_replica.cc:333] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d: stopping tablet replica
I20260709 12:08:45.369241 11318 raft_consensus.cc:2243] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:08:45.370239 11318 raft_consensus.cc:2272] T f2fc2c80a37c43938dcb1dbaeacc2cc5 P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.372139 11318 tablet_replica.cc:333] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d: stopping tablet replica
I20260709 12:08:45.372848 11318 raft_consensus.cc:2243] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.373382 11318 raft_consensus.cc:2272] T c7f41683de3849cfb6a5e4c93161ce46 P 31512bd2b8c74ab89e184a85332e6f9d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.375236 11318 tablet_replica.cc:333] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d: stopping tablet replica
I20260709 12:08:45.375716 11318 raft_consensus.cc:2243] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:08:45.376500 11318 raft_consensus.cc:2272] T 7bbcb6cf7a2e4076b40fbde18996ddce P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.378360 11318 tablet_replica.cc:333] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d: stopping tablet replica
I20260709 12:08:45.379055 11318 raft_consensus.cc:2243] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.379479 11318 raft_consensus.cc:2272] T 7ee33eb902604730a36d8bfe692dc297 P 31512bd2b8c74ab89e184a85332e6f9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.381215 11318 tablet_replica.cc:333] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d: stopping tablet replica
I20260709 12:08:45.381682 11318 raft_consensus.cc:2243] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.382081 11318 raft_consensus.cc:2272] T 071aa019eff14096b27414993b8e9773 P 31512bd2b8c74ab89e184a85332e6f9d [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.383919 11318 tablet_replica.cc:333] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d: stopping tablet replica
I20260709 12:08:45.384338 11318 raft_consensus.cc:2243] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:08:45.384770 11318 raft_consensus.cc:2272] T 44710cf3b4a748969c249dcbeccef11d P 31512bd2b8c74ab89e184a85332e6f9d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.416375 11318 tablet_server.cc:196] TabletServer@127.11.13.131:0 shutdown complete.
I20260709 12:08:45.436998 11318 master.cc:562] Master@127.11.13.190:33281 shutting down...
I20260709 12:08:45.665465 11318 raft_consensus.cc:2243] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:08:45.666239 11318 raft_consensus.cc:2272] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:08:45.666675 11318 tablet_replica.cc:333] T 00000000000000000000000000000000 P eef8fe4a3fbf4f829af15fb71d14c041: stopping tablet replica
I20260709 12:08:45.711864 11318 master.cc:584] Master@127.11.13.190:33281 shutdown complete.
31512bd2b8c74ab89e184a85332e6f9d  1
9a8e2255342c492eb37a3b0daee59dfb  4
e65f179f7b664838b6770533692da7aa  4
31512bd2b8c74ab89e184a85332e6f9d  3
9a8e2255342c492eb37a3b0daee59dfb  3
e65f179f7b664838b6770533692da7aa  3
31512bd2b8c74ab89e184a85332e6f9d  1
9a8e2255342c492eb37a3b0daee59dfb  8
e65f179f7b664838b6770533692da7aa  0
31512bd2b8c74ab89e184a85332e6f9d  3
9a8e2255342c492eb37a3b0daee59dfb  3
e65f179f7b664838b6770533692da7aa  3
[       OK ] LeaderRebalancerTest.FunctionalTestForDivided (33542 ms)
[ RUN      ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260709 12:08:45.779438 11318 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.11.13.190:34131
I20260709 12:08:45.780676 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:08:45.787580 11893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:45.787688 11892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:45.788239 11895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:45.791183 11318 server_base.cc:1061] running on GCE node
I20260709 12:08:45.792426 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:08:45.792639 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:08:45.792778 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783598925792766 us; error 0 us; skew 500 ppm
I20260709 12:08:45.793431 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:08:45.796383 11318 webserver.cc:533] Webserver started at http://127.11.13.190:35675/ using document root <none> and password file <none>
I20260709 12:08:45.796942 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:08:45.797111 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:08:45.797451 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:08:45.798780 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598892056905-11318-0/minicluster-data/master-0-root/instance:
uuid: "ceaf3fb618a745ecbf4fe6f92ed8a674"
format_stamp: "Formatted at 2026-07-09 12:08:45 on dist-test-slave-zqnl"
I20260709 12:08:45.803917 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:08:45.807796 11900 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:45.808861 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:08:45.809309 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598892056905-11318-0/minicluster-data/master-0-root
uuid: "ceaf3fb618a745ecbf4fe6f92ed8a674"
format_stamp: "Formatted at 2026-07-09 12:08:45 on dist-test-slave-zqnl"
I20260709 12:08:45.809684 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598892056905-11318-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598892056905-11318-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598892056905-11318-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:08:45.831997 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:08:45.833455 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:08:45.878885 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.190:34131
I20260709 12:08:45.879009 11951 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.190:34131 every 8 connection(s)
I20260709 12:08:45.883267 11952 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:45.894697 11952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674: Bootstrap starting.
I20260709 12:08:45.899943 11952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:45.904690 11952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674: No bootstrap required, opened a new log
I20260709 12:08:45.907141 11952 raft_consensus.cc:359] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceaf3fb618a745ecbf4fe6f92ed8a674" member_type: VOTER }
I20260709 12:08:45.907634 11952 raft_consensus.cc:385] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:45.907841 11952 raft_consensus.cc:740] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ceaf3fb618a745ecbf4fe6f92ed8a674, State: Initialized, Role: FOLLOWER
I20260709 12:08:45.908493 11952 consensus_queue.cc:260] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceaf3fb618a745ecbf4fe6f92ed8a674" member_type: VOTER }
I20260709 12:08:45.908985 11952 raft_consensus.cc:399] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:08:45.909188 11952 raft_consensus.cc:493] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:08:45.909407 11952 raft_consensus.cc:3060] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:45.913870 11952 raft_consensus.cc:515] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceaf3fb618a745ecbf4fe6f92ed8a674" member_type: VOTER }
I20260709 12:08:45.914701 11952 leader_election.cc:304] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ceaf3fb618a745ecbf4fe6f92ed8a674; no voters: 
I20260709 12:08:45.916160 11952 leader_election.cc:290] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:08:45.916589 11955 raft_consensus.cc:2804] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:45.918246 11955 raft_consensus.cc:697] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674 [term 1 LEADER]: Becoming Leader. State: Replica: ceaf3fb618a745ecbf4fe6f92ed8a674, State: Running, Role: LEADER
I20260709 12:08:45.919246 11955 consensus_queue.cc:237] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceaf3fb618a745ecbf4fe6f92ed8a674" member_type: VOTER }
I20260709 12:08:45.919950 11952 sys_catalog.cc:565] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:08:45.927269 11957 sys_catalog.cc:455] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ceaf3fb618a745ecbf4fe6f92ed8a674. Latest consensus state: current_term: 1 leader_uuid: "ceaf3fb618a745ecbf4fe6f92ed8a674" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceaf3fb618a745ecbf4fe6f92ed8a674" member_type: VOTER } }
I20260709 12:08:45.927237 11956 sys_catalog.cc:455] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ceaf3fb618a745ecbf4fe6f92ed8a674" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ceaf3fb618a745ecbf4fe6f92ed8a674" member_type: VOTER } }
I20260709 12:08:45.928542 11956 sys_catalog.cc:458] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674 [sys.catalog]: This master's current role is: LEADER
I20260709 12:08:45.928490 11957 sys_catalog.cc:458] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674 [sys.catalog]: This master's current role is: LEADER
I20260709 12:08:45.931169 11965 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:08:45.937188 11965 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:08:45.938755 11318 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:08:45.948182 11965 catalog_manager.cc:1383] Generated new cluster ID: 8a261adbf53f4b12b4e79d11cf8decd1
I20260709 12:08:45.948515 11965 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:08:45.998916 11965 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:08:46.000577 11965 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:08:46.017860 11965 catalog_manager.cc:6070] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674: Generated new TSK 0
I20260709 12:08:46.018836 11965 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:08:46.071237 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:08:46.079336 11973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:46.080204 11974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:46.082322 11318 server_base.cc:1061] running on GCE node
W20260709 12:08:46.082207 11976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:46.083540 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:08:46.083729 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:08:46.083882 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783598926083865 us; error 0 us; skew 500 ppm
I20260709 12:08:46.084352 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:08:46.086891 11318 webserver.cc:533] Webserver started at http://127.11.13.129:38601/ using document root <none> and password file <none>
I20260709 12:08:46.087424 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:08:46.087634 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:08:46.087913 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:08:46.088974 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598892056905-11318-0/minicluster-data/ts-0-root/instance:
uuid: "8eeaaedd1f81410fa74265d8338bd016"
format_stamp: "Formatted at 2026-07-09 12:08:46 on dist-test-slave-zqnl"
I20260709 12:08:46.093725 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:08:46.097222 11981 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:46.098230 11318 fs_manager.cc:730] Time spent opening block manager: real 0.002s	user 0.003s	sys 0.000s
I20260709 12:08:46.098490 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598892056905-11318-0/minicluster-data/ts-0-root
uuid: "8eeaaedd1f81410fa74265d8338bd016"
format_stamp: "Formatted at 2026-07-09 12:08:46 on dist-test-slave-zqnl"
I20260709 12:08:46.098747 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598892056905-11318-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598892056905-11318-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598892056905-11318-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:08:46.115751 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:08:46.116923 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:08:46.118340 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:08:46.120522 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:08:46.120693 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:46.120965 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:08:46.121136 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:46.165843 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.129:36687
I20260709 12:08:46.166042 12043 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.129:36687 every 8 connection(s)
I20260709 12:08:46.171736 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:08:46.181645 12048 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:46.182799 12049 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:46.186777 12051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:46.188719 11318 server_base.cc:1061] running on GCE node
I20260709 12:08:46.190515 12044 heartbeater.cc:344] Connected to a master server at 127.11.13.190:34131
I20260709 12:08:46.191027 12044 heartbeater.cc:461] Registering TS with master...
I20260709 12:08:46.191391 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:08:46.191730 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:08:46.191912 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783598926191895 us; error 0 us; skew 500 ppm
I20260709 12:08:46.192166 12044 heartbeater.cc:507] Master 127.11.13.190:34131 requested a full tablet report, sending...
I20260709 12:08:46.192631 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:08:46.195184 11917 ts_manager.cc:194] Registered new tserver with Master: 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687)
I20260709 12:08:46.196199 11318 webserver.cc:533] Webserver started at http://127.11.13.130:39609/ using document root <none> and password file <none>
I20260709 12:08:46.196956 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:08:46.197258 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:08:46.197625 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:08:46.198015 11917 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:32986
I20260709 12:08:46.199362 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598892056905-11318-0/minicluster-data/ts-1-root/instance:
uuid: "0a1d97359ea749a8a688b465e23abfe0"
format_stamp: "Formatted at 2026-07-09 12:08:46 on dist-test-slave-zqnl"
I20260709 12:08:46.204761 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.001s
I20260709 12:08:46.208547 12056 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:46.209596 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:08:46.210105 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598892056905-11318-0/minicluster-data/ts-1-root
uuid: "0a1d97359ea749a8a688b465e23abfe0"
format_stamp: "Formatted at 2026-07-09 12:08:46 on dist-test-slave-zqnl"
I20260709 12:08:46.210541 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598892056905-11318-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598892056905-11318-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598892056905-11318-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:08:46.228410 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:08:46.229678 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:08:46.231456 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:08:46.233695 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:08:46.233876 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:46.234193 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:08:46.234344 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:46.282387 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.130:44777
I20260709 12:08:46.282495 12118 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.130:44777 every 8 connection(s)
I20260709 12:08:46.287673 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:08:46.295362 12122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:46.297284 12123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:08:46.302596 12125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:08:46.303224 12119 heartbeater.cc:344] Connected to a master server at 127.11.13.190:34131
I20260709 12:08:46.303644 12119 heartbeater.cc:461] Registering TS with master...
I20260709 12:08:46.303686 11318 server_base.cc:1061] running on GCE node
I20260709 12:08:46.304822 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:08:46.304812 12119 heartbeater.cc:507] Master 127.11.13.190:34131 requested a full tablet report, sending...
W20260709 12:08:46.305271 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:08:46.305516 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783598926305495 us; error 0 us; skew 500 ppm
I20260709 12:08:46.306327 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:08:46.307838 11917 ts_manager.cc:194] Registered new tserver with Master: 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777)
I20260709 12:08:46.309219 11917 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:32992
I20260709 12:08:46.309278 11318 webserver.cc:533] Webserver started at http://127.11.13.131:41265/ using document root <none> and password file <none>
I20260709 12:08:46.309918 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:08:46.310232 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:08:46.310600 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:08:46.312110 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598892056905-11318-0/minicluster-data/ts-2-root/instance:
uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
format_stamp: "Formatted at 2026-07-09 12:08:46 on dist-test-slave-zqnl"
I20260709 12:08:46.317399 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:08:46.322027 12130 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:46.323123 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:08:46.323473 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598892056905-11318-0/minicluster-data/ts-2-root
uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
format_stamp: "Formatted at 2026-07-09 12:08:46 on dist-test-slave-zqnl"
I20260709 12:08:46.323791 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598892056905-11318-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598892056905-11318-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1783598892056905-11318-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:08:46.347843 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:08:46.349659 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:08:46.351619 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:08:46.354518 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:08:46.354745 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:46.355034 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:08:46.355228 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:08:46.407343 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.131:44393
I20260709 12:08:46.407446 12192 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.131:44393 every 8 connection(s)
I20260709 12:08:46.427578 12193 heartbeater.cc:344] Connected to a master server at 127.11.13.190:34131
I20260709 12:08:46.428050 12193 heartbeater.cc:461] Registering TS with master...
I20260709 12:08:46.429054 12193 heartbeater.cc:507] Master 127.11.13.190:34131 requested a full tablet report, sending...
I20260709 12:08:46.432149 11917 ts_manager.cc:194] Registered new tserver with Master: 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:08:46.433038 11318 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.02148414s
I20260709 12:08:46.433666 11917 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33008
I20260709 12:08:46.464115 11917 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33010:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:08:46.467511 11917 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:08:46.545627 12009 tablet_service.cc:1511] Processing CreateTablet for tablet 85eab2235a8d42d38f8ba3bc6a972cae (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:08:46.546607 12007 tablet_service.cc:1511] Processing CreateTablet for tablet 2cbf71a6592647c0993519af4377e95b (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:08:46.547302 12009 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85eab2235a8d42d38f8ba3bc6a972cae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.548092 12007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cbf71a6592647c0993519af4377e95b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.549858 12006 tablet_service.cc:1511] Processing CreateTablet for tablet c5bff46f93974ecb8067db3b62a12127 (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:08:46.551261 12008 tablet_service.cc:1511] Processing CreateTablet for tablet ff4965e3a40e45f89a81cbb4ce5c80ae (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:08:46.551743 12006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5bff46f93974ecb8067db3b62a12127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.552747 12008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff4965e3a40e45f89a81cbb4ce5c80ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.554287 12005 tablet_service.cc:1511] Processing CreateTablet for tablet df5303764ab648a987018adf0db182b8 (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:08:46.556237 12005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df5303764ab648a987018adf0db182b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.559803 12004 tablet_service.cc:1511] Processing CreateTablet for tablet ec40b30245c34cbead1a3c72ba543ad0 (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:08:46.561750 12004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec40b30245c34cbead1a3c72ba543ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.565744 12003 tablet_service.cc:1511] Processing CreateTablet for tablet 6654daaddcbc4d99814fb5d868018fb1 (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:08:46.567298 12003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6654daaddcbc4d99814fb5d868018fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.571291 12002 tablet_service.cc:1511] Processing CreateTablet for tablet 6b2682646881450b9d5c78c97b49c5d1 (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:08:46.572751 12002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b2682646881450b9d5c78c97b49c5d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.576769 12001 tablet_service.cc:1511] Processing CreateTablet for tablet 35d1b9c0fdf0425db8242897a276970c (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:08:46.578348 12001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35d1b9c0fdf0425db8242897a276970c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.582873 12000 tablet_service.cc:1511] Processing CreateTablet for tablet 1e866b8f6b68440294b311983cb3f078 (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:08:46.584342 12000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e866b8f6b68440294b311983cb3f078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.604148 12206 tablet_bootstrap.cc:492] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016: Bootstrap starting.
I20260709 12:08:46.615860 12158 tablet_service.cc:1511] Processing CreateTablet for tablet 85eab2235a8d42d38f8ba3bc6a972cae (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:08:46.616494 12157 tablet_service.cc:1511] Processing CreateTablet for tablet ff4965e3a40e45f89a81cbb4ce5c80ae (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:08:46.617484 12158 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85eab2235a8d42d38f8ba3bc6a972cae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.618216 12157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff4965e3a40e45f89a81cbb4ce5c80ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.618103 12155 tablet_service.cc:1511] Processing CreateTablet for tablet c5bff46f93974ecb8067db3b62a12127 (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:08:46.620915 12156 tablet_service.cc:1511] Processing CreateTablet for tablet 2cbf71a6592647c0993519af4377e95b (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:08:46.621810 12153 tablet_service.cc:1511] Processing CreateTablet for tablet ec40b30245c34cbead1a3c72ba543ad0 (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:08:46.622390 12155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5bff46f93974ecb8067db3b62a12127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.623373 12153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec40b30245c34cbead1a3c72ba543ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.620965 12154 tablet_service.cc:1511] Processing CreateTablet for tablet df5303764ab648a987018adf0db182b8 (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:08:46.625767 12152 tablet_service.cc:1511] Processing CreateTablet for tablet 6654daaddcbc4d99814fb5d868018fb1 (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:08:46.629703 12151 tablet_service.cc:1511] Processing CreateTablet for tablet 6b2682646881450b9d5c78c97b49c5d1 (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:08:46.632141 12150 tablet_service.cc:1511] Processing CreateTablet for tablet 35d1b9c0fdf0425db8242897a276970c (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:08:46.624032 12156 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cbf71a6592647c0993519af4377e95b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.637311 12150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35d1b9c0fdf0425db8242897a276970c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.636792 12149 tablet_service.cc:1511] Processing CreateTablet for tablet 1e866b8f6b68440294b311983cb3f078 (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:08:46.638188 12152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6654daaddcbc4d99814fb5d868018fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.640813 12149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e866b8f6b68440294b311983cb3f078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.641556 12154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df5303764ab648a987018adf0db182b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.644415 12151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b2682646881450b9d5c78c97b49c5d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.665150 12206 tablet_bootstrap.cc:654] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.710780 12082 tablet_service.cc:1511] Processing CreateTablet for tablet 2cbf71a6592647c0993519af4377e95b (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:08:46.714282 12083 tablet_service.cc:1511] Processing CreateTablet for tablet ff4965e3a40e45f89a81cbb4ce5c80ae (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:08:46.716375 12083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ff4965e3a40e45f89a81cbb4ce5c80ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.718751 12084 tablet_service.cc:1511] Processing CreateTablet for tablet 85eab2235a8d42d38f8ba3bc6a972cae (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:08:46.722239 12081 tablet_service.cc:1511] Processing CreateTablet for tablet c5bff46f93974ecb8067db3b62a12127 (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:08:46.722388 12080 tablet_service.cc:1511] Processing CreateTablet for tablet df5303764ab648a987018adf0db182b8 (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:08:46.725708 12078 tablet_service.cc:1511] Processing CreateTablet for tablet 6654daaddcbc4d99814fb5d868018fb1 (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:08:46.727628 12079 tablet_service.cc:1511] Processing CreateTablet for tablet ec40b30245c34cbead1a3c72ba543ad0 (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:08:46.729522 12077 tablet_service.cc:1511] Processing CreateTablet for tablet 6b2682646881450b9d5c78c97b49c5d1 (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:08:46.732182 12076 tablet_service.cc:1511] Processing CreateTablet for tablet 35d1b9c0fdf0425db8242897a276970c (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:08:46.734100 12075 tablet_service.cc:1511] Processing CreateTablet for tablet 1e866b8f6b68440294b311983cb3f078 (DEFAULT_TABLE table=test-workload [id=49311beaa3134b029570afe2949a269f]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:08:46.736135 12081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c5bff46f93974ecb8067db3b62a12127. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.743440 12076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 35d1b9c0fdf0425db8242897a276970c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.746776 12080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet df5303764ab648a987018adf0db182b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.747444 12079 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec40b30245c34cbead1a3c72ba543ad0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.748363 12078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6654daaddcbc4d99814fb5d868018fb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.749010 12075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e866b8f6b68440294b311983cb3f078. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.750530 12084 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 85eab2235a8d42d38f8ba3bc6a972cae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.759440 12082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2cbf71a6592647c0993519af4377e95b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.765761 12208 tablet_bootstrap.cc:492] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d: Bootstrap starting.
I20260709 12:08:46.778658 12077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6b2682646881450b9d5c78c97b49c5d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:08:46.784328 12208 tablet_bootstrap.cc:654] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.800015 12206 tablet_bootstrap.cc:492] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016: No bootstrap required, opened a new log
I20260709 12:08:46.800635 12206 ts_tablet_manager.cc:1403] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016: Time spent bootstrapping tablet: real 0.197s	user 0.039s	sys 0.054s
I20260709 12:08:46.809886 12206 raft_consensus.cc:359] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.810811 12206 raft_consensus.cc:385] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.811390 12206 raft_consensus.cc:740] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.813202 12206 consensus_queue.cc:260] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.814572 12208 tablet_bootstrap.cc:492] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d: No bootstrap required, opened a new log
I20260709 12:08:46.816363 12210 tablet_bootstrap.cc:492] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0: Bootstrap starting.
I20260709 12:08:46.824898 12210 tablet_bootstrap.cc:654] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.833889 12208 ts_tablet_manager.cc:1403] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d: Time spent bootstrapping tablet: real 0.069s	user 0.015s	sys 0.005s
I20260709 12:08:46.835852 12206 ts_tablet_manager.cc:1434] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016: Time spent starting tablet: real 0.035s	user 0.019s	sys 0.001s
I20260709 12:08:46.835757 12044 heartbeater.cc:499] Master 127.11.13.190:34131 was elected leader, sending a full tablet report...
I20260709 12:08:46.837229 12206 tablet_bootstrap.cc:492] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016: Bootstrap starting.
I20260709 12:08:46.839191 12208 raft_consensus.cc:359] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.839975 12208 raft_consensus.cc:385] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.840278 12208 raft_consensus.cc:740] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.841771 12208 consensus_queue.cc:260] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.846381 12206 tablet_bootstrap.cc:654] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.850952 12210 tablet_bootstrap.cc:492] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0: No bootstrap required, opened a new log
I20260709 12:08:46.851493 12210 ts_tablet_manager.cc:1403] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0: Time spent bootstrapping tablet: real 0.035s	user 0.006s	sys 0.018s
I20260709 12:08:46.852048 12193 heartbeater.cc:499] Master 127.11.13.190:34131 was elected leader, sending a full tablet report...
I20260709 12:08:46.853166 12208 ts_tablet_manager.cc:1434] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d: Time spent starting tablet: real 0.017s	user 0.010s	sys 0.003s
I20260709 12:08:46.854526 12208 tablet_bootstrap.cc:492] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d: Bootstrap starting.
I20260709 12:08:46.855156 12210 raft_consensus.cc:359] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.856280 12210 raft_consensus.cc:385] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.856634 12210 raft_consensus.cc:740] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.857467 12210 consensus_queue.cc:260] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.861397 12206 tablet_bootstrap.cc:492] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016: No bootstrap required, opened a new log
I20260709 12:08:46.862370 12206 ts_tablet_manager.cc:1403] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016: Time spent bootstrapping tablet: real 0.025s	user 0.011s	sys 0.003s
I20260709 12:08:46.867023 12208 tablet_bootstrap.cc:654] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.867058 12206 raft_consensus.cc:359] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.868675 12206 raft_consensus.cc:385] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.869536 12206 raft_consensus.cc:740] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.870924 12206 consensus_queue.cc:260] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.871824 12119 heartbeater.cc:499] Master 127.11.13.190:34131 was elected leader, sending a full tablet report...
I20260709 12:08:46.872373 12210 ts_tablet_manager.cc:1434] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0: Time spent starting tablet: real 0.020s	user 0.011s	sys 0.009s
I20260709 12:08:46.873486 12210 tablet_bootstrap.cc:492] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0: Bootstrap starting.
I20260709 12:08:46.873762 12206 ts_tablet_manager.cc:1434] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:08:46.874804 12206 tablet_bootstrap.cc:492] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016: Bootstrap starting.
I20260709 12:08:46.880824 12210 tablet_bootstrap.cc:654] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.881876 12206 tablet_bootstrap.cc:654] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.893597 12206 tablet_bootstrap.cc:492] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016: No bootstrap required, opened a new log
I20260709 12:08:46.894230 12206 ts_tablet_manager.cc:1403] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016: Time spent bootstrapping tablet: real 0.020s	user 0.007s	sys 0.011s
I20260709 12:08:46.898178 12206 raft_consensus.cc:359] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.899194 12206 raft_consensus.cc:385] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.899577 12206 raft_consensus.cc:740] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.900640 12206 consensus_queue.cc:260] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.903443 12206 ts_tablet_manager.cc:1434] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:08:46.904768 12206 tablet_bootstrap.cc:492] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016: Bootstrap starting.
I20260709 12:08:46.906342 12208 tablet_bootstrap.cc:492] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d: No bootstrap required, opened a new log
I20260709 12:08:46.907159 12208 ts_tablet_manager.cc:1403] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d: Time spent bootstrapping tablet: real 0.053s	user 0.008s	sys 0.009s
I20260709 12:08:46.912117 12206 tablet_bootstrap.cc:654] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016: Neither blocks nor log segments found. Creating new log.
W20260709 12:08:46.912606 12194 tablet.cc:2406] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:08:46.914237 12208 raft_consensus.cc:359] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.915027 12208 raft_consensus.cc:385] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.915338 12208 raft_consensus.cc:740] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.915693 12210 tablet_bootstrap.cc:492] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0: No bootstrap required, opened a new log
I20260709 12:08:46.916457 12210 ts_tablet_manager.cc:1403] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0: Time spent bootstrapping tablet: real 0.043s	user 0.022s	sys 0.019s
I20260709 12:08:46.916256 12208 consensus_queue.cc:260] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.918588 12208 ts_tablet_manager.cc:1434] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:08:46.919734 12208 tablet_bootstrap.cc:492] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d: Bootstrap starting.
I20260709 12:08:46.919684 12210 raft_consensus.cc:359] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.920639 12210 raft_consensus.cc:385] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.920991 12210 raft_consensus.cc:740] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.921931 12210 consensus_queue.cc:260] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.924204 12210 ts_tablet_manager.cc:1434] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:08:46.925266 12210 tablet_bootstrap.cc:492] T ff4965e3a40e45f89a81cbb4ce5c80ae P 0a1d97359ea749a8a688b465e23abfe0: Bootstrap starting.
I20260709 12:08:46.926676 12214 raft_consensus.cc:493] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:46.927353 12214 raft_consensus.cc:515] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.932452 12210 tablet_bootstrap.cc:654] T ff4965e3a40e45f89a81cbb4ce5c80ae P 0a1d97359ea749a8a688b465e23abfe0: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.932591 12214 leader_election.cc:290] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777)
W20260709 12:08:46.935032 11964 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:08:46.939203 12206 tablet_bootstrap.cc:492] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016: No bootstrap required, opened a new log
I20260709 12:08:46.940006 12206 ts_tablet_manager.cc:1403] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.013s
I20260709 12:08:46.944424 12206 raft_consensus.cc:359] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } }
I20260709 12:08:46.945312 12206 raft_consensus.cc:385] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.945696 12206 raft_consensus.cc:740] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.946615 12206 consensus_queue.cc:260] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } }
I20260709 12:08:46.949407 12206 ts_tablet_manager.cc:1434] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:08:46.950568 12206 tablet_bootstrap.cc:492] T 1e866b8f6b68440294b311983cb3f078 P 8eeaaedd1f81410fa74265d8338bd016: Bootstrap starting.
I20260709 12:08:46.954010 12208 tablet_bootstrap.cc:654] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.957186 12210 tablet_bootstrap.cc:492] T ff4965e3a40e45f89a81cbb4ce5c80ae P 0a1d97359ea749a8a688b465e23abfe0: No bootstrap required, opened a new log
I20260709 12:08:46.957677 12206 tablet_bootstrap.cc:654] T 1e866b8f6b68440294b311983cb3f078 P 8eeaaedd1f81410fa74265d8338bd016: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.957719 12210 ts_tablet_manager.cc:1403] T ff4965e3a40e45f89a81cbb4ce5c80ae P 0a1d97359ea749a8a688b465e23abfe0: Time spent bootstrapping tablet: real 0.033s	user 0.023s	sys 0.005s
I20260709 12:08:46.963363 12210 raft_consensus.cc:359] T ff4965e3a40e45f89a81cbb4ce5c80ae P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.964463 12210 raft_consensus.cc:385] T ff4965e3a40e45f89a81cbb4ce5c80ae P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.964854 12210 raft_consensus.cc:740] T ff4965e3a40e45f89a81cbb4ce5c80ae P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.965960 12210 consensus_queue.cc:260] T ff4965e3a40e45f89a81cbb4ce5c80ae P 0a1d97359ea749a8a688b465e23abfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.971513 12210 ts_tablet_manager.cc:1434] T ff4965e3a40e45f89a81cbb4ce5c80ae P 0a1d97359ea749a8a688b465e23abfe0: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 12:08:46.972642 12210 tablet_bootstrap.cc:492] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0: Bootstrap starting.
I20260709 12:08:46.975952 12208 tablet_bootstrap.cc:492] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d: No bootstrap required, opened a new log
I20260709 12:08:46.976635 12208 ts_tablet_manager.cc:1403] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d: Time spent bootstrapping tablet: real 0.057s	user 0.011s	sys 0.010s
I20260709 12:08:46.979593 12210 tablet_bootstrap.cc:654] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:46.979873 12208 raft_consensus.cc:359] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.981946 12206 tablet_bootstrap.cc:492] T 1e866b8f6b68440294b311983cb3f078 P 8eeaaedd1f81410fa74265d8338bd016: No bootstrap required, opened a new log
I20260709 12:08:46.982594 12206 ts_tablet_manager.cc:1403] T 1e866b8f6b68440294b311983cb3f078 P 8eeaaedd1f81410fa74265d8338bd016: Time spent bootstrapping tablet: real 0.032s	user 0.015s	sys 0.003s
I20260709 12:08:46.985881 12206 raft_consensus.cc:359] T 1e866b8f6b68440294b311983cb3f078 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.986865 12206 raft_consensus.cc:385] T 1e866b8f6b68440294b311983cb3f078 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.987226 12206 raft_consensus.cc:740] T 1e866b8f6b68440294b311983cb3f078 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.988094 12206 consensus_queue.cc:260] T 1e866b8f6b68440294b311983cb3f078 P 8eeaaedd1f81410fa74265d8338bd016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.991036 12206 ts_tablet_manager.cc:1434] T 1e866b8f6b68440294b311983cb3f078 P 8eeaaedd1f81410fa74265d8338bd016: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:08:46.991838 12210 tablet_bootstrap.cc:492] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0: No bootstrap required, opened a new log
I20260709 12:08:46.980926 12208 raft_consensus.cc:385] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.992189 12206 tablet_bootstrap.cc:492] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016: Bootstrap starting.
I20260709 12:08:46.992355 12210 ts_tablet_manager.cc:1403] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0: Time spent bootstrapping tablet: real 0.020s	user 0.013s	sys 0.004s
I20260709 12:08:46.992426 12208 raft_consensus.cc:740] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.993690 12208 consensus_queue.cc:260] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.995600 12210 raft_consensus.cc:359] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.996405 12210 raft_consensus.cc:385] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:46.996762 12210 raft_consensus.cc:740] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Initialized, Role: FOLLOWER
I20260709 12:08:46.998330 12208 ts_tablet_manager.cc:1434] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.000s
I20260709 12:08:46.997782 12210 consensus_queue.cc:260] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:46.999426 12208 tablet_bootstrap.cc:492] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d: Bootstrap starting.
I20260709 12:08:47.000427 12206 tablet_bootstrap.cc:654] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.002508 12210 ts_tablet_manager.cc:1434] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:08:47.003029 12212 raft_consensus.cc:493] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.004702 12210 tablet_bootstrap.cc:492] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0: Bootstrap starting.
I20260709 12:08:47.004657 12212 raft_consensus.cc:515] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } }
I20260709 12:08:47.009855 12212 leader_election.cc:290] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393), 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777)
I20260709 12:08:47.014310 12210 tablet_bootstrap.cc:654] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.028796 12094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d1b9c0fdf0425db8242897a276970c" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1d97359ea749a8a688b465e23abfe0" is_pre_election: true
I20260709 12:08:47.030799 12019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d1b9c0fdf0425db8242897a276970c" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eeaaedd1f81410fa74265d8338bd016" is_pre_election: true
W20260709 12:08:47.031482 12134 leader_election.cc:343] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:08:47.032770 12131 leader_election.cc:343] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:47.033360 12131 leader_election.cc:304] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 607b9de3a1844c9daf2e6b0ad22c727d; no voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016
I20260709 12:08:47.034554 12214 raft_consensus.cc:2749] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:47.038720 12212 raft_consensus.cc:493] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.039265 12212 raft_consensus.cc:515] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.041199 12212 leader_election.cc:290] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:08:47.043059 12208 tablet_bootstrap.cc:654] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.043965 12206 tablet_bootstrap.cc:492] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016: No bootstrap required, opened a new log
I20260709 12:08:47.044482 12206 ts_tablet_manager.cc:1403] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016: Time spent bootstrapping tablet: real 0.053s	user 0.022s	sys 0.000s
I20260709 12:08:47.047438 12206 raft_consensus.cc:359] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.048249 12206 raft_consensus.cc:385] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.048566 12206 raft_consensus.cc:740] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.049365 12206 consensus_queue.cc:260] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.052172 12206 ts_tablet_manager.cc:1434] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:08:47.053259 12206 tablet_bootstrap.cc:492] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016: Bootstrap starting.
I20260709 12:08:47.057188 12208 tablet_bootstrap.cc:492] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d: No bootstrap required, opened a new log
I20260709 12:08:47.057942 12208 ts_tablet_manager.cc:1403] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d: Time spent bootstrapping tablet: real 0.059s	user 0.005s	sys 0.014s
I20260709 12:08:47.060410 12206 tablet_bootstrap.cc:654] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.061457 12208 raft_consensus.cc:359] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } }
I20260709 12:08:47.062587 12208 raft_consensus.cc:385] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.065079 12094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eab2235a8d42d38f8ba3bc6a972cae" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1d97359ea749a8a688b465e23abfe0" is_pre_election: true
I20260709 12:08:47.066269 12093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf71a6592647c0993519af4377e95b" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1d97359ea749a8a688b465e23abfe0" is_pre_election: true
W20260709 12:08:47.067183 11985 leader_election.cc:343] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:47.062902 12208 raft_consensus.cc:740] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.069149 12208 consensus_queue.cc:260] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } }
W20260709 12:08:47.070006 11985 leader_election.cc:343] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:47.079314 12210 tablet_bootstrap.cc:492] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0: No bootstrap required, opened a new log
I20260709 12:08:47.079890 12210 ts_tablet_manager.cc:1403] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0: Time spent bootstrapping tablet: real 0.076s	user 0.035s	sys 0.024s
I20260709 12:08:47.083177 12208 ts_tablet_manager.cc:1434] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.000s
I20260709 12:08:47.083137 12210 raft_consensus.cc:359] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.083899 12210 raft_consensus.cc:385] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.084197 12210 raft_consensus.cc:740] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.085033 12210 consensus_queue.cc:260] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.086416 12167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf71a6592647c0993519af4377e95b" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" is_pre_election: true
I20260709 12:08:47.087754 12210 ts_tablet_manager.cc:1434] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:08:47.088804 12210 tablet_bootstrap.cc:492] T 85eab2235a8d42d38f8ba3bc6a972cae P 0a1d97359ea749a8a688b465e23abfe0: Bootstrap starting.
I20260709 12:08:47.086460 12214 raft_consensus.cc:493] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.089877 12214 raft_consensus.cc:515] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.094460 12093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df5303764ab648a987018adf0db182b8" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1d97359ea749a8a688b465e23abfe0" is_pre_election: true
I20260709 12:08:47.095389 12093 raft_consensus.cc:2468] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 607b9de3a1844c9daf2e6b0ad22c727d in term 0.
I20260709 12:08:47.096304 12208 tablet_bootstrap.cc:492] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d: Bootstrap starting.
I20260709 12:08:47.096570 12210 tablet_bootstrap.cc:654] T 85eab2235a8d42d38f8ba3bc6a972cae P 0a1d97359ea749a8a688b465e23abfe0: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.097908 12134 leader_election.cc:304] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 607b9de3a1844c9daf2e6b0ad22c727d; no voters: 
W20260709 12:08:47.102583 11985 leader_election.cc:343] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:47.103240 11985 leader_election.cc:304] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8eeaaedd1f81410fa74265d8338bd016; no voters: 0a1d97359ea749a8a688b465e23abfe0, 607b9de3a1844c9daf2e6b0ad22c727d
I20260709 12:08:47.104866 12212 raft_consensus.cc:2749] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:47.101676 12019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df5303764ab648a987018adf0db182b8" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eeaaedd1f81410fa74265d8338bd016" is_pre_election: true
I20260709 12:08:47.088418 12168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eab2235a8d42d38f8ba3bc6a972cae" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" is_pre_election: true
I20260709 12:08:47.107765 12168 raft_consensus.cc:2468] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8eeaaedd1f81410fa74265d8338bd016 in term 0.
I20260709 12:08:47.108700 12210 tablet_bootstrap.cc:492] T 85eab2235a8d42d38f8ba3bc6a972cae P 0a1d97359ea749a8a688b465e23abfe0: No bootstrap required, opened a new log
I20260709 12:08:47.109241 12210 ts_tablet_manager.cc:1403] T 85eab2235a8d42d38f8ba3bc6a972cae P 0a1d97359ea749a8a688b465e23abfe0: Time spent bootstrapping tablet: real 0.021s	user 0.009s	sys 0.008s
I20260709 12:08:47.110813 12206 tablet_bootstrap.cc:492] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016: No bootstrap required, opened a new log
I20260709 12:08:47.110954 12208 tablet_bootstrap.cc:654] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d: Neither blocks nor log segments found. Creating new log.
W20260709 12:08:47.111238 12131 leader_election.cc:343] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:47.111469 12206 ts_tablet_manager.cc:1403] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016: Time spent bootstrapping tablet: real 0.058s	user 0.024s	sys 0.031s
I20260709 12:08:47.112907 11985 leader_election.cc:304] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 607b9de3a1844c9daf2e6b0ad22c727d, 8eeaaedd1f81410fa74265d8338bd016; no voters: 0a1d97359ea749a8a688b465e23abfe0
I20260709 12:08:47.112335 12214 leader_election.cc:290] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777)
I20260709 12:08:47.114144 12214 raft_consensus.cc:2804] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:47.114812 12212 raft_consensus.cc:2804] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:47.114477 12210 raft_consensus.cc:359] T 85eab2235a8d42d38f8ba3bc6a972cae P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } }
I20260709 12:08:47.115000 12206 raft_consensus.cc:359] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.115840 12212 raft_consensus.cc:493] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:47.116010 12210 raft_consensus.cc:385] T 85eab2235a8d42d38f8ba3bc6a972cae P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.116107 12206 raft_consensus.cc:385] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.116297 12212 raft_consensus.cc:3060] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.116703 12210 raft_consensus.cc:740] T 85eab2235a8d42d38f8ba3bc6a972cae P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.116767 12206 raft_consensus.cc:740] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.117985 12210 consensus_queue.cc:260] T 85eab2235a8d42d38f8ba3bc6a972cae P 0a1d97359ea749a8a688b465e23abfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } }
I20260709 12:08:47.122473 12214 raft_consensus.cc:493] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:47.123273 12214 raft_consensus.cc:3060] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.118088 12206 consensus_queue.cc:260] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.125837 12210 ts_tablet_manager.cc:1434] T 85eab2235a8d42d38f8ba3bc6a972cae P 0a1d97359ea749a8a688b465e23abfe0: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 12:08:47.126959 12210 tablet_bootstrap.cc:492] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0: Bootstrap starting.
I20260709 12:08:47.135027 12210 tablet_bootstrap.cc:654] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.135047 12237 raft_consensus.cc:493] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.138860 12237 raft_consensus.cc:515] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.136950 12206 ts_tablet_manager.cc:1434] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016: Time spent starting tablet: real 0.025s	user 0.004s	sys 0.008s
I20260709 12:08:47.144193 12206 tablet_bootstrap.cc:492] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016: Bootstrap starting.
I20260709 12:08:47.142590 12093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec40b30245c34cbead1a3c72ba543ad0" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1d97359ea749a8a688b465e23abfe0" is_pre_election: true
W20260709 12:08:47.146459 11985 leader_election.cc:343] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:47.150007 12233 raft_consensus.cc:493] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:08:47.150866 12134 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:44777, user_credentials={real_user=slave}} blocked reactor thread for 53131.5us
I20260709 12:08:47.152053 12168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec40b30245c34cbead1a3c72ba543ad0" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" is_pre_election: true
I20260709 12:08:47.157977 12206 tablet_bootstrap.cc:654] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016: Neither blocks nor log segments found. Creating new log.
W20260709 12:08:47.158267 11985 leader_election.cc:343] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:47.158735 11985 leader_election.cc:304] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8eeaaedd1f81410fa74265d8338bd016; no voters: 0a1d97359ea749a8a688b465e23abfe0, 607b9de3a1844c9daf2e6b0ad22c727d
I20260709 12:08:47.152951 12235 raft_consensus.cc:493] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.160229 12235 raft_consensus.cc:515] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.163327 12235 leader_election.cc:290] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:08:47.164012 12235 raft_consensus.cc:2749] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:47.153348 12237 leader_election.cc:290] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:08:47.156606 12210 tablet_bootstrap.cc:492] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0: No bootstrap required, opened a new log
I20260709 12:08:47.165519 12210 ts_tablet_manager.cc:1403] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0: Time spent bootstrapping tablet: real 0.039s	user 0.015s	sys 0.014s
I20260709 12:08:47.167722 12206 tablet_bootstrap.cc:492] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016: No bootstrap required, opened a new log
I20260709 12:08:47.168260 12206 ts_tablet_manager.cc:1403] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016: Time spent bootstrapping tablet: real 0.024s	user 0.016s	sys 0.001s
I20260709 12:08:47.170095 12093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff4965e3a40e45f89a81cbb4ce5c80ae" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1d97359ea749a8a688b465e23abfe0" is_pre_election: true
I20260709 12:08:47.169966 12233 raft_consensus.cc:515] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.170881 12093 raft_consensus.cc:2468] T ff4965e3a40e45f89a81cbb4ce5c80ae P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8eeaaedd1f81410fa74265d8338bd016 in term 0.
I20260709 12:08:47.171475 12206 raft_consensus.cc:359] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.172246 12206 raft_consensus.cc:385] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.172644 12206 raft_consensus.cc:740] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.172958 12168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff4965e3a40e45f89a81cbb4ce5c80ae" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" is_pre_election: true
I20260709 12:08:47.173494 12206 consensus_queue.cc:260] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.133858 12212 raft_consensus.cc:515] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } }
I20260709 12:08:47.177096 12212 leader_election.cc:290] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 election: Requested vote from peers 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393), 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777)
I20260709 12:08:47.178253 12214 raft_consensus.cc:515] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.178625 12019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2682646881450b9d5c78c97b49c5d1" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eeaaedd1f81410fa74265d8338bd016" is_pre_election: true
I20260709 12:08:47.179517 12019 raft_consensus.cc:2468] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 607b9de3a1844c9daf2e6b0ad22c727d in term 0.
I20260709 12:08:47.181221 12131 leader_election.cc:304] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 607b9de3a1844c9daf2e6b0ad22c727d, 8eeaaedd1f81410fa74265d8338bd016; no voters: 
I20260709 12:08:47.175179 12206 ts_tablet_manager.cc:1434] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:08:47.183154 12019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df5303764ab648a987018adf0db182b8" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
I20260709 12:08:47.183434 12094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df5303764ab648a987018adf0db182b8" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
I20260709 12:08:47.184473 12094 raft_consensus.cc:3060] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.184743 12093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2682646881450b9d5c78c97b49c5d1" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1d97359ea749a8a688b465e23abfe0" is_pre_election: true
W20260709 12:08:47.187069 11985 leader_election.cc:343] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:47.187861 11985 leader_election.cc:304] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016; no voters: 607b9de3a1844c9daf2e6b0ad22c727d
I20260709 12:08:47.189800 12093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eab2235a8d42d38f8ba3bc6a972cae" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
I20260709 12:08:47.168744 12210 raft_consensus.cc:359] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.192672 12235 raft_consensus.cc:2804] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:47.198799 12235 raft_consensus.cc:493] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:47.199203 12235 raft_consensus.cc:3060] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.199240 12214 leader_election.cc:290] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 election: Requested vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777)
I20260709 12:08:47.200927 12094 raft_consensus.cc:2468] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 607b9de3a1844c9daf2e6b0ad22c727d in term 1.
I20260709 12:08:47.192801 12093 raft_consensus.cc:3060] T 85eab2235a8d42d38f8ba3bc6a972cae P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.202768 12208 tablet_bootstrap.cc:492] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d: No bootstrap required, opened a new log
I20260709 12:08:47.203382 12208 ts_tablet_manager.cc:1403] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d: Time spent bootstrapping tablet: real 0.107s	user 0.040s	sys 0.029s
I20260709 12:08:47.205219 12235 raft_consensus.cc:515] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
W20260709 12:08:47.190323 12134 leader_election.cc:343] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:47.193521 12206 tablet_bootstrap.cc:492] T 35d1b9c0fdf0425db8242897a276970c P 8eeaaedd1f81410fa74265d8338bd016: Bootstrap starting.
I20260709 12:08:47.206683 12208 raft_consensus.cc:359] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.207348 12235 leader_election.cc:290] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 election: Requested vote from peers 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:08:47.207540 12208 raft_consensus.cc:385] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.207942 12208 raft_consensus.cc:740] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.208757 12208 consensus_queue.cc:260] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.209820 12094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff4965e3a40e45f89a81cbb4ce5c80ae" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
I20260709 12:08:47.210304 12093 raft_consensus.cc:2468] T 85eab2235a8d42d38f8ba3bc6a972cae P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eeaaedd1f81410fa74265d8338bd016 in term 1.
I20260709 12:08:47.210518 12094 raft_consensus.cc:3060] T ff4965e3a40e45f89a81cbb4ce5c80ae P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.193269 12210 raft_consensus.cc:385] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.211774 11985 leader_election.cc:304] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016; no voters: 
I20260709 12:08:47.195233 12241 raft_consensus.cc:2804] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:47.195713 12168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eab2235a8d42d38f8ba3bc6a972cae" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
I20260709 12:08:47.213907 12241 raft_consensus.cc:493] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:47.214166 12168 raft_consensus.cc:3060] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.214391 12241 raft_consensus.cc:3060] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.216926 12206 tablet_bootstrap.cc:654] T 35d1b9c0fdf0425db8242897a276970c P 8eeaaedd1f81410fa74265d8338bd016: Neither blocks nor log segments found. Creating new log.
W20260709 12:08:47.195616 12131 leader_election.cc:343] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:47.198298 12233 leader_election.cc:290] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777)
I20260709 12:08:47.211949 12210 raft_consensus.cc:740] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.219421 12210 consensus_queue.cc:260] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.220902 12134 leader_election.cc:304] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 607b9de3a1844c9daf2e6b0ad22c727d; no voters: 8eeaaedd1f81410fa74265d8338bd016
I20260709 12:08:47.222218 12214 raft_consensus.cc:2804] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:47.222575 12167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff4965e3a40e45f89a81cbb4ce5c80ae" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
I20260709 12:08:47.213449 12208 ts_tablet_manager.cc:1434] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 12:08:47.225117 12208 tablet_bootstrap.cc:492] T ec40b30245c34cbead1a3c72ba543ad0 P 607b9de3a1844c9daf2e6b0ad22c727d: Bootstrap starting.
I20260709 12:08:47.226011 12241 raft_consensus.cc:515] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.228732 12241 leader_election.cc:290] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 election: Requested vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777)
I20260709 12:08:47.230281 12093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2682646881450b9d5c78c97b49c5d1" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
W20260709 12:08:47.231112 11985 leader_election.cc:343] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:47.231560 12019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2682646881450b9d5c78c97b49c5d1" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
I20260709 12:08:47.232388 12019 raft_consensus.cc:3060] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.235307 12168 raft_consensus.cc:2468] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eeaaedd1f81410fa74265d8338bd016 in term 1.
I20260709 12:08:47.236231 12235 raft_consensus.cc:2804] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:47.236558 12210 ts_tablet_manager.cc:1434] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0: Time spent starting tablet: real 0.071s	user 0.006s	sys 0.001s
I20260709 12:08:47.236850 12235 raft_consensus.cc:697] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [term 1 LEADER]: Becoming Leader. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Running, Role: LEADER
W20260709 12:08:47.237833 12134 leader_election.cc:343] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:47.237735 12235 consensus_queue.cc:237] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } }
I20260709 12:08:47.245378 12214 raft_consensus.cc:697] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 LEADER]: Becoming Leader. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Running, Role: LEADER
I20260709 12:08:47.245947 12210 tablet_bootstrap.cc:492] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0: Bootstrap starting.
I20260709 12:08:47.245712 12019 raft_consensus.cc:2468] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 607b9de3a1844c9daf2e6b0ad22c727d in term 1.
I20260709 12:08:47.246685 12214 consensus_queue.cc:237] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.248066 12131 leader_election.cc:304] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 607b9de3a1844c9daf2e6b0ad22c727d, 8eeaaedd1f81410fa74265d8338bd016; no voters: 0a1d97359ea749a8a688b465e23abfe0
I20260709 12:08:47.248549 12208 tablet_bootstrap.cc:654] T ec40b30245c34cbead1a3c72ba543ad0 P 607b9de3a1844c9daf2e6b0ad22c727d: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.249862 12242 raft_consensus.cc:2804] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:47.250535 12242 raft_consensus.cc:697] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 LEADER]: Becoming Leader. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Running, Role: LEADER
I20260709 12:08:47.252724 12242 consensus_queue.cc:237] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.253639 12206 tablet_bootstrap.cc:492] T 35d1b9c0fdf0425db8242897a276970c P 8eeaaedd1f81410fa74265d8338bd016: No bootstrap required, opened a new log
I20260709 12:08:47.254068 12210 tablet_bootstrap.cc:654] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.254251 12206 ts_tablet_manager.cc:1403] T 35d1b9c0fdf0425db8242897a276970c P 8eeaaedd1f81410fa74265d8338bd016: Time spent bootstrapping tablet: real 0.061s	user 0.009s	sys 0.007s
I20260709 12:08:47.256635 12094 raft_consensus.cc:2468] T ff4965e3a40e45f89a81cbb4ce5c80ae P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eeaaedd1f81410fa74265d8338bd016 in term 1.
I20260709 12:08:47.258145 11985 leader_election.cc:304] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016; no voters: 607b9de3a1844c9daf2e6b0ad22c727d
I20260709 12:08:47.259506 12212 raft_consensus.cc:2804] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:47.260047 12212 raft_consensus.cc:697] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 1 LEADER]: Becoming Leader. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Running, Role: LEADER
I20260709 12:08:47.260987 12212 consensus_queue.cc:237] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.265383 12208 tablet_bootstrap.cc:492] T ec40b30245c34cbead1a3c72ba543ad0 P 607b9de3a1844c9daf2e6b0ad22c727d: No bootstrap required, opened a new log
I20260709 12:08:47.266389 12208 ts_tablet_manager.cc:1403] T ec40b30245c34cbead1a3c72ba543ad0 P 607b9de3a1844c9daf2e6b0ad22c727d: Time spent bootstrapping tablet: real 0.042s	user 0.014s	sys 0.003s
I20260709 12:08:47.271622 12208 raft_consensus.cc:359] T ec40b30245c34cbead1a3c72ba543ad0 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.272795 12208 raft_consensus.cc:385] T ec40b30245c34cbead1a3c72ba543ad0 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.273216 12208 raft_consensus.cc:740] T ec40b30245c34cbead1a3c72ba543ad0 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.257544 12206 raft_consensus.cc:359] T 35d1b9c0fdf0425db8242897a276970c P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.274416 12216 raft_consensus.cc:493] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.273818 12206 raft_consensus.cc:385] T 35d1b9c0fdf0425db8242897a276970c P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.275017 12206 raft_consensus.cc:740] T 35d1b9c0fdf0425db8242897a276970c P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.275769 12206 consensus_queue.cc:260] T 35d1b9c0fdf0425db8242897a276970c P 8eeaaedd1f81410fa74265d8338bd016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.275486 11912 catalog_manager.cc:5697] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d reported cstate change: term changed from 0 to 1, leader changed from <none> to 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131). New cstate: current_term: 1 leader_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:47.277616 11917 catalog_manager.cc:5697] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "8eeaaedd1f81410fa74265d8338bd016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:47.278651 12206 ts_tablet_manager.cc:1434] T 35d1b9c0fdf0425db8242897a276970c P 8eeaaedd1f81410fa74265d8338bd016: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.002s
I20260709 12:08:47.278960 12212 raft_consensus.cc:493] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.275022 12216 raft_consensus.cc:515] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.279609 12212 raft_consensus.cc:515] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.280449 12210 tablet_bootstrap.cc:492] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0: No bootstrap required, opened a new log
I20260709 12:08:47.281365 12210 ts_tablet_manager.cc:1403] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0: Time spent bootstrapping tablet: real 0.036s	user 0.015s	sys 0.012s
I20260709 12:08:47.283347 12216 leader_election.cc:290] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:08:47.284147 12094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bff46f93974ecb8067db3b62a12127" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1d97359ea749a8a688b465e23abfe0" is_pre_election: true
I20260709 12:08:47.284852 12094 raft_consensus.cc:2468] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8eeaaedd1f81410fa74265d8338bd016 in term 0.
I20260709 12:08:47.286098 12210 raft_consensus.cc:359] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.279630 12206 tablet_bootstrap.cc:492] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016: Bootstrap starting.
I20260709 12:08:47.289004 11985 leader_election.cc:304] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016; no voters: 
I20260709 12:08:47.286839 12210 raft_consensus.cc:385] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.290038 12235 raft_consensus.cc:2804] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:47.290421 12210 raft_consensus.cc:740] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.290520 12235 raft_consensus.cc:493] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:47.290896 12235 raft_consensus.cc:3060] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.291163 12210 consensus_queue.cc:260] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.296692 12206 tablet_bootstrap.cc:654] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.301465 12212 leader_election.cc:290] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:08:47.302862 12210 ts_tablet_manager.cc:1434] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.003s
I20260709 12:08:47.282301 12208 consensus_queue.cc:260] T ec40b30245c34cbead1a3c72ba543ad0 P 607b9de3a1844c9daf2e6b0ad22c727d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.303730 12168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bff46f93974ecb8067db3b62a12127" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" is_pre_election: true
W20260709 12:08:47.306993 11985 leader_election.cc:343] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:47.310755 12208 ts_tablet_manager.cc:1434] T ec40b30245c34cbead1a3c72ba543ad0 P 607b9de3a1844c9daf2e6b0ad22c727d: Time spent starting tablet: real 0.044s	user 0.008s	sys 0.003s
I20260709 12:08:47.316710 12210 tablet_bootstrap.cc:492] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0: Bootstrap starting.
I20260709 12:08:47.321749 12019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e866b8f6b68440294b311983cb3f078" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eeaaedd1f81410fa74265d8338bd016" is_pre_election: true
I20260709 12:08:47.322494 12019 raft_consensus.cc:2468] T 1e866b8f6b68440294b311983cb3f078 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 0.
I20260709 12:08:47.321772 12208 tablet_bootstrap.cc:492] T 2cbf71a6592647c0993519af4377e95b P 607b9de3a1844c9daf2e6b0ad22c727d: Bootstrap starting.
I20260709 12:08:47.319640 12235 raft_consensus.cc:515] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.323833 12057 leader_election.cc:304] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016; no voters: 
I20260709 12:08:47.327358 12216 raft_consensus.cc:2804] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:47.327739 12216 raft_consensus.cc:493] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:47.328094 12216 raft_consensus.cc:3060] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.328416 12094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bff46f93974ecb8067db3b62a12127" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
I20260709 12:08:47.328974 12094 raft_consensus.cc:3060] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.330153 12210 tablet_bootstrap.cc:654] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.344362 12094 raft_consensus.cc:2468] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eeaaedd1f81410fa74265d8338bd016 in term 1.
I20260709 12:08:47.345902 12235 leader_election.cc:290] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 election: Requested vote from peers 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:08:47.347189 11985 leader_election.cc:304] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016; no voters: 
I20260709 12:08:47.349841 12235 raft_consensus.cc:2804] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:47.351107 12168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bff46f93974ecb8067db3b62a12127" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
I20260709 12:08:47.352005 12235 raft_consensus.cc:697] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 1 LEADER]: Becoming Leader. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Running, Role: LEADER
I20260709 12:08:47.340485 12216 raft_consensus.cc:515] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
W20260709 12:08:47.353520 11985 leader_election.cc:343] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:47.354357 12235 consensus_queue.cc:237] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.356719 12019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e866b8f6b68440294b311983cb3f078" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
I20260709 12:08:47.357421 12019 raft_consensus.cc:3060] T 1e866b8f6b68440294b311983cb3f078 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.363404 12206 tablet_bootstrap.cc:492] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016: No bootstrap required, opened a new log
I20260709 12:08:47.364058 12206 ts_tablet_manager.cc:1403] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016: Time spent bootstrapping tablet: real 0.085s	user 0.034s	sys 0.019s
I20260709 12:08:47.367992 12210 tablet_bootstrap.cc:492] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0: No bootstrap required, opened a new log
I20260709 12:08:47.367764 12206 raft_consensus.cc:359] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.368664 12206 raft_consensus.cc:385] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.368808 12210 ts_tablet_manager.cc:1403] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0: Time spent bootstrapping tablet: real 0.052s	user 0.015s	sys 0.001s
I20260709 12:08:47.369040 12206 raft_consensus.cc:740] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.370577 12206 consensus_queue.cc:260] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.369397 12216 leader_election.cc:290] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 1 election: Requested vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:08:47.367942 12019 raft_consensus.cc:2468] T 1e866b8f6b68440294b311983cb3f078 P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 1.
I20260709 12:08:47.373706 12208 tablet_bootstrap.cc:654] T 2cbf71a6592647c0993519af4377e95b P 607b9de3a1844c9daf2e6b0ad22c727d: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.375357 12057 leader_election.cc:304] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016; no voters: 
I20260709 12:08:47.391862 12206 ts_tablet_manager.cc:1434] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.006s
I20260709 12:08:47.394574 12167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e866b8f6b68440294b311983cb3f078" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
I20260709 12:08:47.391088 12210 raft_consensus.cc:359] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.396078 12210 raft_consensus.cc:385] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.395960 12168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e866b8f6b68440294b311983cb3f078" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" is_pre_election: true
I20260709 12:08:47.396617 12210 raft_consensus.cc:740] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Initialized, Role: FOLLOWER
W20260709 12:08:47.398576 12060 leader_election.cc:343] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:08:47.399688 12060 leader_election.cc:343] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:08:47.403005 12216 raft_consensus.cc:2804] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:47.403398 12210 consensus_queue.cc:260] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.414494 12216 raft_consensus.cc:697] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 LEADER]: Becoming Leader. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Running, Role: LEADER
I20260709 12:08:47.419294 12216 consensus_queue.cc:237] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.418771 11912 catalog_manager.cc:5697] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d reported cstate change: term changed from 0 to 1, leader changed from <none> to 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131). New cstate: current_term: 1 leader_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:47.426599 12210 ts_tablet_manager.cc:1434] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0: Time spent starting tablet: real 0.057s	user 0.019s	sys 0.005s
I20260709 12:08:47.434680 12208 tablet_bootstrap.cc:492] T 2cbf71a6592647c0993519af4377e95b P 607b9de3a1844c9daf2e6b0ad22c727d: No bootstrap required, opened a new log
I20260709 12:08:47.435321 12208 ts_tablet_manager.cc:1403] T 2cbf71a6592647c0993519af4377e95b P 607b9de3a1844c9daf2e6b0ad22c727d: Time spent bootstrapping tablet: real 0.114s	user 0.021s	sys 0.036s
I20260709 12:08:47.436980 12210 tablet_bootstrap.cc:492] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0: Bootstrap starting.
I20260709 12:08:47.439059 12208 raft_consensus.cc:359] T 2cbf71a6592647c0993519af4377e95b P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.440307 12208 raft_consensus.cc:385] T 2cbf71a6592647c0993519af4377e95b P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.439558 11917 catalog_manager.cc:5697] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "8eeaaedd1f81410fa74265d8338bd016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:47.442348 11917 catalog_manager.cc:5697] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "8eeaaedd1f81410fa74265d8338bd016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:47.446295 12208 raft_consensus.cc:740] T 2cbf71a6592647c0993519af4377e95b P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.447011 12210 tablet_bootstrap.cc:654] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.447541 12208 consensus_queue.cc:260] T 2cbf71a6592647c0993519af4377e95b P 607b9de3a1844c9daf2e6b0ad22c727d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.450510 12208 ts_tablet_manager.cc:1434] T 2cbf71a6592647c0993519af4377e95b P 607b9de3a1844c9daf2e6b0ad22c727d: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.001s
I20260709 12:08:47.451735 12208 tablet_bootstrap.cc:492] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d: Bootstrap starting.
I20260709 12:08:47.452685 11912 catalog_manager.cc:5697] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130). New cstate: current_term: 1 leader_uuid: "0a1d97359ea749a8a688b465e23abfe0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:47.456166 12210 tablet_bootstrap.cc:492] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0: No bootstrap required, opened a new log
I20260709 12:08:47.456674 12210 ts_tablet_manager.cc:1403] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0: Time spent bootstrapping tablet: real 0.020s	user 0.004s	sys 0.007s
I20260709 12:08:47.458431 12208 tablet_bootstrap.cc:654] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.459139 12214 raft_consensus.cc:493] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.459297 12210 raft_consensus.cc:359] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.459607 12214 raft_consensus.cc:515] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.459968 12210 raft_consensus.cc:385] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.460285 12210 raft_consensus.cc:740] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.460997 12210 consensus_queue.cc:260] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.461712 12214 leader_election.cc:290] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777)
I20260709 12:08:47.463323 12210 ts_tablet_manager.cc:1434] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:08:47.463470 12208 tablet_bootstrap.cc:492] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d: No bootstrap required, opened a new log
I20260709 12:08:47.463392 12094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6654daaddcbc4d99814fb5d868018fb1" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1d97359ea749a8a688b465e23abfe0" is_pre_election: true
I20260709 12:08:47.464201 12208 ts_tablet_manager.cc:1403] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d: Time spent bootstrapping tablet: real 0.013s	user 0.011s	sys 0.001s
I20260709 12:08:47.464186 12094 raft_consensus.cc:2468] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 607b9de3a1844c9daf2e6b0ad22c727d in term 0.
I20260709 12:08:47.465302 12134 leader_election.cc:304] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 607b9de3a1844c9daf2e6b0ad22c727d; no voters: 
I20260709 12:08:47.465794 12019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6654daaddcbc4d99814fb5d868018fb1" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eeaaedd1f81410fa74265d8338bd016" is_pre_election: true
I20260709 12:08:47.466143 12214 raft_consensus.cc:2804] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:47.466427 12019 raft_consensus.cc:2468] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 607b9de3a1844c9daf2e6b0ad22c727d in term 0.
I20260709 12:08:47.466506 12214 raft_consensus.cc:493] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:47.467468 12208 raft_consensus.cc:359] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.468775 12208 raft_consensus.cc:385] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.467003 12214 raft_consensus.cc:3060] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.469326 12208 raft_consensus.cc:740] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.470475 12208 consensus_queue.cc:260] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.471351 12216 raft_consensus.cc:493] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.471890 12216 raft_consensus.cc:515] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.473055 12208 ts_tablet_manager.cc:1434] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:08:47.474076 12208 tablet_bootstrap.cc:492] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d: Bootstrap starting.
I20260709 12:08:47.474221 12216 leader_election.cc:290] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:08:47.476907 12168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2682646881450b9d5c78c97b49c5d1" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" is_pre_election: true
I20260709 12:08:47.477707 12019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2682646881450b9d5c78c97b49c5d1" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eeaaedd1f81410fa74265d8338bd016" is_pre_election: true
I20260709 12:08:47.478690 12019 raft_consensus.cc:2393] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a1d97359ea749a8a688b465e23abfe0 in current term 1: Already voted for candidate 607b9de3a1844c9daf2e6b0ad22c727d in this term.
I20260709 12:08:47.479934 12057 leader_election.cc:304] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0; no voters: 607b9de3a1844c9daf2e6b0ad22c727d, 8eeaaedd1f81410fa74265d8338bd016
I20260709 12:08:47.479123 12216 raft_consensus.cc:493] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.481022 12261 raft_consensus.cc:3060] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.481725 12208 tablet_bootstrap.cc:654] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.487277 12214 raft_consensus.cc:515] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.481021 12216 raft_consensus.cc:515] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.492359 12214 leader_election.cc:290] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 election: Requested vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777)
I20260709 12:08:47.493474 12019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6654daaddcbc4d99814fb5d868018fb1" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
I20260709 12:08:47.494112 12019 raft_consensus.cc:3060] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.494233 12018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf71a6592647c0993519af4377e95b" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eeaaedd1f81410fa74265d8338bd016" is_pre_election: true
I20260709 12:08:47.494318 12094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6654daaddcbc4d99814fb5d868018fb1" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
I20260709 12:08:47.494940 12018 raft_consensus.cc:2468] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 0.
I20260709 12:08:47.495023 12094 raft_consensus.cc:3060] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.496119 12057 leader_election.cc:304] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016; no voters: 
I20260709 12:08:47.497022 12260 raft_consensus.cc:2804] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:47.497390 12260 raft_consensus.cc:493] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:47.497712 12260 raft_consensus.cc:3060] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.494313 12208 tablet_bootstrap.cc:492] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d: No bootstrap required, opened a new log
I20260709 12:08:47.502987 12208 ts_tablet_manager.cc:1403] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d: Time spent bootstrapping tablet: real 0.029s	user 0.011s	sys 0.007s
I20260709 12:08:47.502813 12168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf71a6592647c0993519af4377e95b" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" is_pre_election: true
I20260709 12:08:47.503566 12168 raft_consensus.cc:2468] T 2cbf71a6592647c0993519af4377e95b P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 0.
I20260709 12:08:47.504407 12216 leader_election.cc:290] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:08:47.505914 12208 raft_consensus.cc:359] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.506693 12208 raft_consensus.cc:385] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.506683 12094 raft_consensus.cc:2468] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 607b9de3a1844c9daf2e6b0ad22c727d in term 1.
I20260709 12:08:47.506742 12260 raft_consensus.cc:515] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.507356 12208 raft_consensus.cc:740] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.508006 12019 raft_consensus.cc:2468] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 607b9de3a1844c9daf2e6b0ad22c727d in term 1.
I20260709 12:08:47.508526 12134 leader_election.cc:304] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 607b9de3a1844c9daf2e6b0ad22c727d; no voters: 
I20260709 12:08:47.508416 12208 consensus_queue.cc:260] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.509889 12242 raft_consensus.cc:2804] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:47.510303 12018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf71a6592647c0993519af4377e95b" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
I20260709 12:08:47.510810 12208 ts_tablet_manager.cc:1434] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:08:47.510957 12018 raft_consensus.cc:3060] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.511420 12261 raft_consensus.cc:2749] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:47.512066 12260 leader_election.cc:290] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 1 election: Requested vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:08:47.513103 12208 tablet_bootstrap.cc:492] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d: Bootstrap starting.
I20260709 12:08:47.510435 12242 raft_consensus.cc:697] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 LEADER]: Becoming Leader. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Running, Role: LEADER
I20260709 12:08:47.516222 12168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf71a6592647c0993519af4377e95b" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
I20260709 12:08:47.516199 12242 consensus_queue.cc:237] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.517050 12168 raft_consensus.cc:3060] T 2cbf71a6592647c0993519af4377e95b P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.518791 12208 tablet_bootstrap.cc:654] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d: Neither blocks nor log segments found. Creating new log.
I20260709 12:08:47.521903 12018 raft_consensus.cc:2468] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 1.
I20260709 12:08:47.523790 12260 raft_consensus.cc:493] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.524006 12057 leader_election.cc:304] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016; no voters: 
I20260709 12:08:47.524340 12260 raft_consensus.cc:515] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.524914 12261 raft_consensus.cc:2804] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:47.525427 12261 raft_consensus.cc:697] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 1 LEADER]: Becoming Leader. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Running, Role: LEADER
I20260709 12:08:47.525941 12168 raft_consensus.cc:2468] T 2cbf71a6592647c0993519af4377e95b P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 1.
I20260709 12:08:47.528009 12019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec40b30245c34cbead1a3c72ba543ad0" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eeaaedd1f81410fa74265d8338bd016" is_pre_election: true
I20260709 12:08:47.527758 12261 consensus_queue.cc:237] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.528656 12019 raft_consensus.cc:2468] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 0.
I20260709 12:08:47.529938 12260 leader_election.cc:290] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:08:47.530200 12057 leader_election.cc:304] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016; no voters: 
I20260709 12:08:47.534168 12168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec40b30245c34cbead1a3c72ba543ad0" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" is_pre_election: true
I20260709 12:08:47.537599 12168 raft_consensus.cc:2468] T ec40b30245c34cbead1a3c72ba543ad0 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 0.
I20260709 12:08:47.541066 12260 raft_consensus.cc:2804] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:47.542502 12260 raft_consensus.cc:493] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:47.543123 12260 raft_consensus.cc:3060] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.551148 11912 catalog_manager.cc:5697] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130). New cstate: current_term: 1 leader_uuid: "0a1d97359ea749a8a688b465e23abfe0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:47.559047 12261 raft_consensus.cc:493] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.559381 12208 tablet_bootstrap.cc:492] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d: No bootstrap required, opened a new log
I20260709 12:08:47.559616 12261 raft_consensus.cc:515] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.562673 12018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d1b9c0fdf0425db8242897a276970c" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eeaaedd1f81410fa74265d8338bd016" is_pre_election: true
I20260709 12:08:47.562903 12261 leader_election.cc:290] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:08:47.563369 12018 raft_consensus.cc:2468] T 35d1b9c0fdf0425db8242897a276970c P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 0.
I20260709 12:08:47.563772 12168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d1b9c0fdf0425db8242897a276970c" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" is_pre_election: true
I20260709 12:08:47.563845 11917 catalog_manager.cc:5697] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d reported cstate change: term changed from 0 to 1, leader changed from <none> to 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131). New cstate: current_term: 1 leader_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:47.565029 12168 raft_consensus.cc:2468] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 0.
I20260709 12:08:47.565317 12057 leader_election.cc:304] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016; no voters: 
I20260709 12:08:47.566610 12261 raft_consensus.cc:2804] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:08:47.567083 12261 raft_consensus.cc:493] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:08:47.567441 12261 raft_consensus.cc:3060] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.567888 12242 raft_consensus.cc:493] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.568452 12242 raft_consensus.cc:515] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.570312 12260 raft_consensus.cc:515] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.571563 12018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bff46f93974ecb8067db3b62a12127" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eeaaedd1f81410fa74265d8338bd016" is_pre_election: true
I20260709 12:08:47.573511 12094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bff46f93974ecb8067db3b62a12127" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1d97359ea749a8a688b465e23abfe0" is_pre_election: true
I20260709 12:08:47.573766 12260 leader_election.cc:290] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 1 election: Requested vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:08:47.574836 12019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec40b30245c34cbead1a3c72ba543ad0" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
I20260709 12:08:47.560006 12208 ts_tablet_manager.cc:1403] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d: Time spent bootstrapping tablet: real 0.047s	user 0.019s	sys 0.022s
I20260709 12:08:47.575440 12019 raft_consensus.cc:3060] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.575848 12168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec40b30245c34cbead1a3c72ba543ad0" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
I20260709 12:08:47.576445 12168 raft_consensus.cc:3060] T ec40b30245c34cbead1a3c72ba543ad0 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.579150 12208 raft_consensus.cc:359] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.580156 12208 raft_consensus.cc:385] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:08:47.580474 12208 raft_consensus.cc:740] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Initialized, Role: FOLLOWER
I20260709 12:08:47.575306 12094 raft_consensus.cc:2393] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 607b9de3a1844c9daf2e6b0ad22c727d in current term 1: Already voted for candidate 8eeaaedd1f81410fa74265d8338bd016 in this term.
I20260709 12:08:47.575399 12242 leader_election.cc:290] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777)
I20260709 12:08:47.581530 12208 consensus_queue.cc:260] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.584414 12208 ts_tablet_manager.cc:1434] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:08:47.584589 12134 leader_election.cc:304] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 607b9de3a1844c9daf2e6b0ad22c727d; no voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016
I20260709 12:08:47.585601 12168 raft_consensus.cc:2468] T ec40b30245c34cbead1a3c72ba543ad0 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 1.
I20260709 12:08:47.586228 12242 raft_consensus.cc:3060] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.586371 12261 raft_consensus.cc:515] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.587131 12060 leader_election.cc:304] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 607b9de3a1844c9daf2e6b0ad22c727d; no voters: 
I20260709 12:08:47.588356 12260 raft_consensus.cc:2804] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:47.588855 12260 raft_consensus.cc:697] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 LEADER]: Becoming Leader. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Running, Role: LEADER
I20260709 12:08:47.589195 12261 leader_election.cc:290] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 1 election: Requested vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:08:47.589728 12260 consensus_queue.cc:237] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.590479 12018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d1b9c0fdf0425db8242897a276970c" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
I20260709 12:08:47.591188 12018 raft_consensus.cc:3060] T 35d1b9c0fdf0425db8242897a276970c P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.594193 12019 raft_consensus.cc:2468] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 1.
I20260709 12:08:47.595361 12242 raft_consensus.cc:2749] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:47.595854 12168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "35d1b9c0fdf0425db8242897a276970c" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
I20260709 12:08:47.596493 12168 raft_consensus.cc:3060] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.602726 12018 raft_consensus.cc:2468] T 35d1b9c0fdf0425db8242897a276970c P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 1.
I20260709 12:08:47.603901 12168 raft_consensus.cc:2468] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 1.
I20260709 12:08:47.604182 12057 leader_election.cc:304] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016; no voters: 
I20260709 12:08:47.605232 12216 raft_consensus.cc:2804] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:08:47.605736 12216 raft_consensus.cc:697] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 [term 1 LEADER]: Becoming Leader. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Running, Role: LEADER
I20260709 12:08:47.606607 12216 consensus_queue.cc:237] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.614578 11917 catalog_manager.cc:5697] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130). New cstate: current_term: 1 leader_uuid: "0a1d97359ea749a8a688b465e23abfe0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:47.622574 12242 raft_consensus.cc:493] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.623319 12242 raft_consensus.cc:515] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.627764 12242 leader_election.cc:290] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777)
I20260709 12:08:47.628712 12018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e866b8f6b68440294b311983cb3f078" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eeaaedd1f81410fa74265d8338bd016" is_pre_election: true
I20260709 12:08:47.629256 12094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e866b8f6b68440294b311983cb3f078" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1d97359ea749a8a688b465e23abfe0" is_pre_election: true
I20260709 12:08:47.629664 12018 raft_consensus.cc:2393] T 1e866b8f6b68440294b311983cb3f078 P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 607b9de3a1844c9daf2e6b0ad22c727d in current term 1: Already voted for candidate 0a1d97359ea749a8a688b465e23abfe0 in this term.
I20260709 12:08:47.630935 12134 leader_election.cc:304] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 607b9de3a1844c9daf2e6b0ad22c727d; no voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016
I20260709 12:08:47.631677 12242 raft_consensus.cc:3060] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.631322 11917 catalog_manager.cc:5697] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130). New cstate: current_term: 1 leader_uuid: "0a1d97359ea749a8a688b465e23abfe0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:47.637076 12242 raft_consensus.cc:2749] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:08:47.667088 12242 consensus_queue.cc:1048] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:47.679014 12018 raft_consensus.cc:3060] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.691008 12242 consensus_queue.cc:1048] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:47.720626 12235 consensus_queue.cc:1048] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:47.739070 12237 consensus_queue.cc:1048] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:47.755311 12237 consensus_queue.cc:1048] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:47.755322 12094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cbf71a6592647c0993519af4377e95b"
dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
mode: GRACEFUL
new_leader_uuid: "8eeaaedd1f81410fa74265d8338bd016"
 from {username='slave'} at 127.0.0.1:33866
I20260709 12:08:47.756445 12094 raft_consensus.cc:606] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 1 LEADER]: Received request to transfer leadership to TS 8eeaaedd1f81410fa74265d8338bd016
I20260709 12:08:47.772229 12281 consensus_queue.cc:1048] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:47.786832 12167 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df5303764ab648a987018adf0db182b8"
dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
mode: GRACEFUL
new_leader_uuid: "8eeaaedd1f81410fa74265d8338bd016"
 from {username='slave'} at 127.0.0.1:33066
I20260709 12:08:47.787534 12167 raft_consensus.cc:606] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 LEADER]: Received request to transfer leadership to TS 8eeaaedd1f81410fa74265d8338bd016
I20260709 12:08:47.797385 12167 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6654daaddcbc4d99814fb5d868018fb1"
dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
mode: GRACEFUL
new_leader_uuid: "0a1d97359ea749a8a688b465e23abfe0"
 from {username='slave'} at 127.0.0.1:33066
I20260709 12:08:47.797930 12167 raft_consensus.cc:606] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 LEADER]: Received request to transfer leadership to TS 0a1d97359ea749a8a688b465e23abfe0
I20260709 12:08:47.804462 12167 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b2682646881450b9d5c78c97b49c5d1"
dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
mode: GRACEFUL
new_leader_uuid: "0a1d97359ea749a8a688b465e23abfe0"
 from {username='slave'} at 127.0.0.1:33066
I20260709 12:08:47.805377 12167 raft_consensus.cc:606] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 LEADER]: Received request to transfer leadership to TS 0a1d97359ea749a8a688b465e23abfe0
I20260709 12:08:47.813899 12242 consensus_queue.cc:1048] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:47.815896 12094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1e866b8f6b68440294b311983cb3f078"
dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
mode: GRACEFUL
new_leader_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
 from {username='slave'} at 127.0.0.1:33866
I20260709 12:08:47.816610 12094 raft_consensus.cc:606] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 LEADER]: Received request to transfer leadership to TS 607b9de3a1844c9daf2e6b0ad22c727d
I20260709 12:08:47.823657 12241 raft_consensus.cc:993] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d: : Instructing follower 0a1d97359ea749a8a688b465e23abfe0 to start an election
I20260709 12:08:47.824199 12241 raft_consensus.cc:1081] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 LEADER]: Signalling peer 0a1d97359ea749a8a688b465e23abfe0 to start an election
I20260709 12:08:47.827157 12281 consensus_queue.cc:1048] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:47.829461 12214 consensus_queue.cc:1048] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:47.832105 12094 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6b2682646881450b9d5c78c97b49c5d1"
dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
 from {username='slave'} at 127.0.0.1:33842
I20260709 12:08:47.833213 12094 raft_consensus.cc:493] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:47.833823 12094 raft_consensus.cc:3060] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:47.848039 12094 raft_consensus.cc:515] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.853586 12018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2682646881450b9d5c78c97b49c5d1" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
I20260709 12:08:47.853860 12167 raft_consensus.cc:3060] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:08:47.854259 12018 raft_consensus.cc:3060] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:47.864228 12235 consensus_queue.cc:1048] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:47.866927 12214 raft_consensus.cc:493] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:08:47.867291 12168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2682646881450b9d5c78c97b49c5d1" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
I20260709 12:08:47.868161 12168 raft_consensus.cc:3055] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:47.868464 12168 raft_consensus.cc:740] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Running, Role: LEADER
I20260709 12:08:47.868050 12214 raft_consensus.cc:515] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.869272 12168 consensus_queue.cc:260] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.871018 12168 raft_consensus.cc:3060] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:47.873117 12019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff4965e3a40e45f89a81cbb4ce5c80ae" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8eeaaedd1f81410fa74265d8338bd016" is_pre_election: true
I20260709 12:08:47.871206 12214 leader_election.cc:290] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777)
I20260709 12:08:47.878587 12092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff4965e3a40e45f89a81cbb4ce5c80ae" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0a1d97359ea749a8a688b465e23abfe0" is_pre_election: true
I20260709 12:08:47.879248 12094 leader_election.cc:290] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 2 election: Requested vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:08:47.865139 12018 raft_consensus.cc:2468] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 2.
I20260709 12:08:47.880345 12134 leader_election.cc:304] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 607b9de3a1844c9daf2e6b0ad22c727d; no voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016
I20260709 12:08:47.881489 12057 leader_election.cc:304] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016; no voters: 
I20260709 12:08:47.882431 12168 raft_consensus.cc:2468] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 2.
I20260709 12:08:47.882944 12216 raft_consensus.cc:2804] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:47.884195 12216 raft_consensus.cc:697] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 LEADER]: Becoming Leader. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Running, Role: LEADER
I20260709 12:08:47.885969 12233 raft_consensus.cc:2749] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:08:47.885253 12216 consensus_queue.cc:237] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.894415 11912 catalog_manager.cc:5697] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 reported cstate change: term changed from 1 to 2, leader changed from 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131) to 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130). New cstate: current_term: 2 leader_uuid: "0a1d97359ea749a8a688b465e23abfe0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:47.937081 12216 consensus_queue.cc:1048] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:47.944533 12260 raft_consensus.cc:993] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0: : Instructing follower 8eeaaedd1f81410fa74265d8338bd016 to start an election
I20260709 12:08:47.944980 12260 raft_consensus.cc:1081] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 1 LEADER]: Signalling peer 8eeaaedd1f81410fa74265d8338bd016 to start an election
I20260709 12:08:47.948448 12019 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2cbf71a6592647c0993519af4377e95b"
dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
 from {username='slave'} at 127.0.0.1:39680
I20260709 12:08:47.949782 12019 raft_consensus.cc:493] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:47.950325 12019 raft_consensus.cc:3060] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:47.950771 12261 consensus_queue.cc:1048] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:47.958876 12019 raft_consensus.cc:515] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.963918 12167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf71a6592647c0993519af4377e95b" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
I20260709 12:08:47.964552 12167 raft_consensus.cc:3060] T 2cbf71a6592647c0993519af4377e95b P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:47.965193 12261 consensus_queue.cc:1048] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:47.961992 12019 leader_election.cc:290] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 2 election: Requested vote from peers 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:08:47.968241 12092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf71a6592647c0993519af4377e95b" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
I20260709 12:08:47.969537 12092 raft_consensus.cc:3055] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:47.969885 12092 raft_consensus.cc:740] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Running, Role: LEADER
I20260709 12:08:47.970785 12092 consensus_queue.cc:260] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.972222 12092 raft_consensus.cc:3060] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:47.973619 12167 raft_consensus.cc:2468] T 2cbf71a6592647c0993519af4377e95b P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eeaaedd1f81410fa74265d8338bd016 in term 2.
I20260709 12:08:47.973634 12233 consensus_queue.cc:1048] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:47.977927 11985 leader_election.cc:304] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 607b9de3a1844c9daf2e6b0ad22c727d, 8eeaaedd1f81410fa74265d8338bd016; no voters: 
I20260709 12:08:47.979624 12235 raft_consensus.cc:2804] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:47.980167 12235 raft_consensus.cc:697] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 2 LEADER]: Becoming Leader. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Running, Role: LEADER
I20260709 12:08:47.981062 12235 consensus_queue.cc:237] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:47.983300 12092 raft_consensus.cc:2468] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eeaaedd1f81410fa74265d8338bd016 in term 2.
I20260709 12:08:47.990947 12261 consensus_queue.cc:1048] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:47.999246 12233 consensus_queue.cc:1048] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:48.002058 12260 raft_consensus.cc:993] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0: : Instructing follower 607b9de3a1844c9daf2e6b0ad22c727d to start an election
I20260709 12:08:48.002756 12261 raft_consensus.cc:1081] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 LEADER]: Signalling peer 607b9de3a1844c9daf2e6b0ad22c727d to start an election
I20260709 12:08:48.002797 11914 catalog_manager.cc:5697] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 reported cstate change: term changed from 1 to 2, leader changed from 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130) to 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "8eeaaedd1f81410fa74265d8338bd016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:48.005946 12167 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1e866b8f6b68440294b311983cb3f078"
dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
 from {username='slave'} at 127.0.0.1:33052
I20260709 12:08:48.006876 12167 raft_consensus.cc:493] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:48.008157 12167 raft_consensus.cc:3060] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:48.012871 12233 raft_consensus.cc:993] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d: : Instructing follower 0a1d97359ea749a8a688b465e23abfe0 to start an election
I20260709 12:08:48.013356 12233 raft_consensus.cc:1081] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 LEADER]: Signalling peer 0a1d97359ea749a8a688b465e23abfe0 to start an election
I20260709 12:08:48.016727 12167 raft_consensus.cc:515] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:48.017333 12092 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6654daaddcbc4d99814fb5d868018fb1"
dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
 from {username='slave'} at 127.0.0.1:33842
I20260709 12:08:48.018421 12092 raft_consensus.cc:493] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:48.019110 12167 leader_election.cc:290] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 2 election: Requested vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777)
I20260709 12:08:48.021054 12019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e866b8f6b68440294b311983cb3f078" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
I20260709 12:08:48.021683 12019 raft_consensus.cc:3060] T 1e866b8f6b68440294b311983cb3f078 P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:48.018822 12092 raft_consensus.cc:3060] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:48.024627 12094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e866b8f6b68440294b311983cb3f078" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
I20260709 12:08:48.025276 12094 raft_consensus.cc:3055] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:48.025633 12094 raft_consensus.cc:740] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Running, Role: LEADER
I20260709 12:08:48.026489 12094 consensus_queue.cc:260] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:48.027839 12094 raft_consensus.cc:3060] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:48.030206 12019 raft_consensus.cc:2468] T 1e866b8f6b68440294b311983cb3f078 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 607b9de3a1844c9daf2e6b0ad22c727d in term 2.
I20260709 12:08:48.030550 12092 raft_consensus.cc:515] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:48.033831 12092 leader_election.cc:290] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 2 election: Requested vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:08:48.034106 12019 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6654daaddcbc4d99814fb5d868018fb1" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
I20260709 12:08:48.034861 12019 raft_consensus.cc:3060] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:48.034723 12260 consensus_queue.cc:1048] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:48.035032 12167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6654daaddcbc4d99814fb5d868018fb1" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
I20260709 12:08:48.035187 12131 leader_election.cc:304] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 607b9de3a1844c9daf2e6b0ad22c727d, 8eeaaedd1f81410fa74265d8338bd016; no voters: 
I20260709 12:08:48.035741 12167 raft_consensus.cc:3055] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:48.036118 12167 raft_consensus.cc:740] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Running, Role: LEADER
I20260709 12:08:48.036937 12167 consensus_queue.cc:260] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:48.037967 12167 raft_consensus.cc:3060] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:48.041253 12241 raft_consensus.cc:2804] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:48.041996 12241 raft_consensus.cc:697] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 LEADER]: Becoming Leader. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Running, Role: LEADER
I20260709 12:08:48.042936 12241 consensus_queue.cc:237] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:48.045032 12019 raft_consensus.cc:2468] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 2.
I20260709 12:08:48.046443 12057 leader_election.cc:304] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016; no voters: 
I20260709 12:08:48.046223 12167 raft_consensus.cc:2468] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 2.
I20260709 12:08:48.047449 12260 raft_consensus.cc:2804] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:48.047973 12260 raft_consensus.cc:697] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 LEADER]: Becoming Leader. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Running, Role: LEADER
I20260709 12:08:48.048185 12094 raft_consensus.cc:2468] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 607b9de3a1844c9daf2e6b0ad22c727d in term 2.
I20260709 12:08:48.048868 12260 consensus_queue.cc:237] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:48.057514 11917 catalog_manager.cc:5697] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 reported cstate change: term changed from 1 to 2, leader changed from 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131) to 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130). New cstate: current_term: 2 leader_uuid: "0a1d97359ea749a8a688b465e23abfe0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:48.059295 12260 consensus_queue.cc:1048] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:48.060624 11914 catalog_manager.cc:5697] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d reported cstate change: term changed from 1 to 2, leader changed from 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130) to 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131). New cstate: current_term: 2 leader_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:48.078672 12260 consensus_queue.cc:1048] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:08:48.091200 12261 consensus_queue.cc:1048] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:08:48.192252 12233 raft_consensus.cc:993] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d: : Instructing follower 8eeaaedd1f81410fa74265d8338bd016 to start an election
I20260709 12:08:48.192799 12241 raft_consensus.cc:1081] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 LEADER]: Signalling peer 8eeaaedd1f81410fa74265d8338bd016 to start an election
I20260709 12:08:48.194408 12019 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df5303764ab648a987018adf0db182b8"
dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
 from {username='slave'} at 127.0.0.1:39672
I20260709 12:08:48.194938 12019 raft_consensus.cc:493] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:48.195369 12019 raft_consensus.cc:3060] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:48.200663 12019 raft_consensus.cc:515] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:48.202551 12019 leader_election.cc:290] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 2 election: Requested vote from peers 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:08:48.204200 12168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df5303764ab648a987018adf0db182b8" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
I20260709 12:08:48.204200 12094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df5303764ab648a987018adf0db182b8" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
I20260709 12:08:48.205061 12094 raft_consensus.cc:3060] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:48.205016 12168 raft_consensus.cc:3055] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:48.205571 12168 raft_consensus.cc:740] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Running, Role: LEADER
I20260709 12:08:48.206542 12168 consensus_queue.cc:260] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:48.207868 12168 raft_consensus.cc:3060] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:48.213181 12094 raft_consensus.cc:2468] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eeaaedd1f81410fa74265d8338bd016 in term 2.
I20260709 12:08:48.215389 11985 leader_election.cc:304] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016; no voters: 
I20260709 12:08:48.216607 12235 raft_consensus.cc:2804] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:48.216672 12168 raft_consensus.cc:2468] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eeaaedd1f81410fa74265d8338bd016 in term 2.
I20260709 12:08:48.217187 12235 raft_consensus.cc:697] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 LEADER]: Becoming Leader. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Running, Role: LEADER
I20260709 12:08:48.218103 12235 consensus_queue.cc:237] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:48.225818 11914 catalog_manager.cc:5697] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 reported cstate change: term changed from 1 to 2, leader changed from 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131) to 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "8eeaaedd1f81410fa74265d8338bd016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: UNKNOWN } } }
I20260709 12:08:48.318818 12168 raft_consensus.cc:1275] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Refusing update from remote peer 0a1d97359ea749a8a688b465e23abfe0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.320788 12261 consensus_queue.cc:1048] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:08:48.331321 12019 raft_consensus.cc:1275] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Refusing update from remote peer 0a1d97359ea749a8a688b465e23abfe0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.334398 12216 consensus_queue.cc:1048] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:48.433223 12019 raft_consensus.cc:1275] T 1e866b8f6b68440294b311983cb3f078 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Refusing update from remote peer 607b9de3a1844c9daf2e6b0ad22c727d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.435550 12241 consensus_queue.cc:1048] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:48.440991 12094 raft_consensus.cc:1275] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Refusing update from remote peer 607b9de3a1844c9daf2e6b0ad22c727d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.443043 12241 consensus_queue.cc:1048] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:08:48.490610 12168 raft_consensus.cc:1275] T 2cbf71a6592647c0993519af4377e95b P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Refusing update from remote peer 8eeaaedd1f81410fa74265d8338bd016: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.492533 12237 consensus_queue.cc:1048] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:48.503682 12094 raft_consensus.cc:1275] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Refusing update from remote peer 8eeaaedd1f81410fa74265d8338bd016: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.505450 12212 consensus_queue.cc:1048] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:48.595304 12018 raft_consensus.cc:1275] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Refusing update from remote peer 0a1d97359ea749a8a688b465e23abfe0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.597098 12260 consensus_queue.cc:1048] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:48.598114 12168 raft_consensus.cc:1275] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Refusing update from remote peer 0a1d97359ea749a8a688b465e23abfe0: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.599766 12260 consensus_queue.cc:1048] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:08:48.657406 12094 raft_consensus.cc:1275] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Refusing update from remote peer 8eeaaedd1f81410fa74265d8338bd016: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.658800 12212 consensus_queue.cc:1048] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:48.660131 12168 raft_consensus.cc:1275] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Refusing update from remote peer 8eeaaedd1f81410fa74265d8338bd016: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:48.661731 12212 consensus_queue.cc:1048] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:50.858408 11318 auto_leader_rebalancer-test.cc:365] The leader distribution is 
I20260709 12:08:50.859189 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:50.884189 12018 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5bff46f93974ecb8067db3b62a12127"
dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
mode: GRACEFUL
new_leader_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
 from {username='slave'} at 127.0.0.1:39688
I20260709 12:08:50.884745 12018 raft_consensus.cc:606] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 1 LEADER]: Received request to transfer leadership to TS 607b9de3a1844c9daf2e6b0ad22c727d
I20260709 12:08:50.887760 12018 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff4965e3a40e45f89a81cbb4ce5c80ae"
dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
mode: GRACEFUL
new_leader_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
 from {username='slave'} at 127.0.0.1:39688
I20260709 12:08:50.888438 12018 raft_consensus.cc:606] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 1 LEADER]: Received request to transfer leadership to TS 607b9de3a1844c9daf2e6b0ad22c727d
I20260709 12:08:50.889746 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:08:50.890374 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:50.891408 12302 raft_consensus.cc:993] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016: : Instructing follower 607b9de3a1844c9daf2e6b0ad22c727d to start an election
I20260709 12:08:50.891867 12235 raft_consensus.cc:1081] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 1 LEADER]: Signalling peer 607b9de3a1844c9daf2e6b0ad22c727d to start an election
I20260709 12:08:50.893904 12168 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5bff46f93974ecb8067db3b62a12127"
dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
 from {username='slave'} at 127.0.0.1:33046
I20260709 12:08:50.894582 12168 raft_consensus.cc:493] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:50.894896 12168 raft_consensus.cc:3060] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:50.901533 12168 raft_consensus.cc:515] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:50.903290 12168 leader_election.cc:290] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 2 election: Requested vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777)
I20260709 12:08:50.904603 12094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bff46f93974ecb8067db3b62a12127" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
I20260709 12:08:50.905280 12094 raft_consensus.cc:3060] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:50.905179 12018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bff46f93974ecb8067db3b62a12127" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
I20260709 12:08:50.905793 12018 raft_consensus.cc:3055] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:50.906101 12018 raft_consensus.cc:740] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Running, Role: LEADER
I20260709 12:08:50.906867 12018 consensus_queue.cc:260] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:50.907863 12018 raft_consensus.cc:3060] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:50.910852 12094 raft_consensus.cc:2468] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 607b9de3a1844c9daf2e6b0ad22c727d in term 2.
I20260709 12:08:50.912261 12134 leader_election.cc:304] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 607b9de3a1844c9daf2e6b0ad22c727d; no voters: 
I20260709 12:08:50.913107 12018 raft_consensus.cc:2468] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 607b9de3a1844c9daf2e6b0ad22c727d in term 2.
I20260709 12:08:50.913250 12299 raft_consensus.cc:2804] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:50.914858 12299 raft_consensus.cc:697] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 LEADER]: Becoming Leader. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Running, Role: LEADER
I20260709 12:08:50.915681 12299 consensus_queue.cc:237] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:50.922914 11917 catalog_manager.cc:5697] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d reported cstate change: term changed from 1 to 2, leader changed from 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129) to 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131). New cstate: current_term: 2 leader_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:50.989486 12302 raft_consensus.cc:993] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016: : Instructing follower 607b9de3a1844c9daf2e6b0ad22c727d to start an election
I20260709 12:08:50.990959 12235 raft_consensus.cc:1081] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 1 LEADER]: Signalling peer 607b9de3a1844c9daf2e6b0ad22c727d to start an election
I20260709 12:08:50.992895 12168 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ff4965e3a40e45f89a81cbb4ce5c80ae"
dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
 from {username='slave'} at 127.0.0.1:33046
I20260709 12:08:50.993494 12168 raft_consensus.cc:493] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:08:50.993916 12168 raft_consensus.cc:3060] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:50.998987 12168 raft_consensus.cc:515] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:51.002279 12018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff4965e3a40e45f89a81cbb4ce5c80ae" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
I20260709 12:08:51.002548 12094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff4965e3a40e45f89a81cbb4ce5c80ae" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
I20260709 12:08:51.002978 12018 raft_consensus.cc:3055] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:08:51.003157 12094 raft_consensus.cc:3060] T ff4965e3a40e45f89a81cbb4ce5c80ae P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:51.003364 12018 raft_consensus.cc:740] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Running, Role: LEADER
I20260709 12:08:51.004293 12018 consensus_queue.cc:260] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:51.005438 12018 raft_consensus.cc:3060] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:08:51.006780 12168 leader_election.cc:290] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 2 election: Requested vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777)
I20260709 12:08:51.010380 12094 raft_consensus.cc:2468] T ff4965e3a40e45f89a81cbb4ce5c80ae P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 607b9de3a1844c9daf2e6b0ad22c727d in term 2.
I20260709 12:08:51.011209 12018 raft_consensus.cc:2468] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 607b9de3a1844c9daf2e6b0ad22c727d in term 2.
I20260709 12:08:51.011674 12134 leader_election.cc:304] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 607b9de3a1844c9daf2e6b0ad22c727d; no voters: 
I20260709 12:08:51.012581 12299 raft_consensus.cc:2804] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:08:51.013197 12299 raft_consensus.cc:697] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 LEADER]: Becoming Leader. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Running, Role: LEADER
I20260709 12:08:51.014392 12299 consensus_queue.cc:237] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:08:51.023852 11917 catalog_manager.cc:5697] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d reported cstate change: term changed from 1 to 2, leader changed from 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129) to 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131). New cstate: current_term: 2 leader_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: HEALTHY } } }
I20260709 12:08:51.308677 12017 raft_consensus.cc:1275] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Refusing update from remote peer 607b9de3a1844c9daf2e6b0ad22c727d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:51.310500 12299 consensus_queue.cc:1048] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:51.325214 12094 raft_consensus.cc:1275] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Refusing update from remote peer 607b9de3a1844c9daf2e6b0ad22c727d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:51.327625 12299 consensus_queue.cc:1048] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:08:51.591617 12017 raft_consensus.cc:1275] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Refusing update from remote peer 607b9de3a1844c9daf2e6b0ad22c727d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:51.593139 12299 consensus_queue.cc:1048] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:51.606832 12094 raft_consensus.cc:1275] T ff4965e3a40e45f89a81cbb4ce5c80ae P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Refusing update from remote peer 607b9de3a1844c9daf2e6b0ad22c727d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:08:51.609267 12299 consensus_queue.cc:1048] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:08:51.891557 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:51.897600 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:51.898113 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:52.899978 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:52.908373 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:52.908993 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:53.910211 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:53.916066 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:53.916651 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:54.917820 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:54.923827 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:54.924557 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:55.925746 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:55.932390 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:55.932878 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:56.934193 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:56.940383 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:56.940852 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:57.941963 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:57.947945 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:57.948431 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:58.949579 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:58.955253 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:58.955914 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:08:59.957041 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:08:59.963200 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:08:59.963757 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:01.072975 11318 auto_leader_rebalancer-test.cc:384] The leader distribution is 
I20260709 12:09:01.082787 12167 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ff4965e3a40e45f89a81cbb4ce5c80ae"
dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
mode: GRACEFUL
new_leader_uuid: "8eeaaedd1f81410fa74265d8338bd016"
 from {username='slave'} at 127.0.0.1:33066
I20260709 12:09:01.083606 12167 raft_consensus.cc:606] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 LEADER]: Received request to transfer leadership to TS 8eeaaedd1f81410fa74265d8338bd016
I20260709 12:09:01.092947 12167 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5bff46f93974ecb8067db3b62a12127"
dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
mode: GRACEFUL
new_leader_uuid: "8eeaaedd1f81410fa74265d8338bd016"
 from {username='slave'} at 127.0.0.1:33066
I20260709 12:09:01.093484 12167 raft_consensus.cc:606] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 LEADER]: Received request to transfer leadership to TS 8eeaaedd1f81410fa74265d8338bd016
I20260709 12:09:01.103533 12094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec40b30245c34cbead1a3c72ba543ad0"
dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
mode: GRACEFUL
new_leader_uuid: "8eeaaedd1f81410fa74265d8338bd016"
 from {username='slave'} at 127.0.0.1:33866
I20260709 12:09:01.104107 12094 raft_consensus.cc:606] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 LEADER]: Received request to transfer leadership to TS 8eeaaedd1f81410fa74265d8338bd016
I20260709 12:09:01.109978 12094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6654daaddcbc4d99814fb5d868018fb1"
dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
mode: GRACEFUL
new_leader_uuid: "8eeaaedd1f81410fa74265d8338bd016"
 from {username='slave'} at 127.0.0.1:33866
I20260709 12:09:01.110662 12094 raft_consensus.cc:606] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 LEADER]: Received request to transfer leadership to TS 8eeaaedd1f81410fa74265d8338bd016
I20260709 12:09:01.117390 12094 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6b2682646881450b9d5c78c97b49c5d1"
dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
mode: GRACEFUL
new_leader_uuid: "8eeaaedd1f81410fa74265d8338bd016"
 from {username='slave'} at 127.0.0.1:33866
I20260709 12:09:01.118112 12094 raft_consensus.cc:606] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 LEADER]: Received request to transfer leadership to TS 8eeaaedd1f81410fa74265d8338bd016
W20260709 12:09:01.137104 12040 debug-util.cc:398] Leaking SignalData structure 0x7b08001cfc80 after lost signal to thread 11321
W20260709 12:09:01.138345 12040 debug-util.cc:398] Leaking SignalData structure 0x7b08001b4ea0 after lost signal to thread 11951
W20260709 12:09:01.139469 12040 debug-util.cc:398] Leaking SignalData structure 0x7b08000a7540 after lost signal to thread 12043
W20260709 12:09:01.140480 12040 debug-util.cc:398] Leaking SignalData structure 0x7b080017ae80 after lost signal to thread 12118
W20260709 12:09:01.141482 12040 debug-util.cc:398] Leaking SignalData structure 0x7b08001b3740 after lost signal to thread 12192
I20260709 12:09:01.155161 12328 raft_consensus.cc:993] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0: : Instructing follower 8eeaaedd1f81410fa74265d8338bd016 to start an election
I20260709 12:09:01.155826 12327 raft_consensus.cc:1081] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 LEADER]: Signalling peer 8eeaaedd1f81410fa74265d8338bd016 to start an election
I20260709 12:09:01.157719 12018 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ec40b30245c34cbead1a3c72ba543ad0"
dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
 from {username='slave'} at 127.0.0.1:39680
I20260709 12:09:01.158301 12018 raft_consensus.cc:493] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:01.158711 12018 raft_consensus.cc:3060] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:01.165663 12018 raft_consensus.cc:515] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:01.168291 12018 leader_election.cc:290] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 2 election: Requested vote from peers 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:09:01.169626 12167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec40b30245c34cbead1a3c72ba543ad0" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
I20260709 12:09:01.169914 12094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec40b30245c34cbead1a3c72ba543ad0" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
I20260709 12:09:01.170457 12167 raft_consensus.cc:3060] T ec40b30245c34cbead1a3c72ba543ad0 P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:01.170639 12094 raft_consensus.cc:3055] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:09:01.171033 12094 raft_consensus.cc:740] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Running, Role: LEADER
I20260709 12:09:01.172061 12094 consensus_queue.cc:260] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:01.173645 12094 raft_consensus.cc:3060] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:01.177455 12167 raft_consensus.cc:2468] T ec40b30245c34cbead1a3c72ba543ad0 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eeaaedd1f81410fa74265d8338bd016 in term 2.
I20260709 12:09:01.179219 11985 leader_election.cc:304] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 607b9de3a1844c9daf2e6b0ad22c727d, 8eeaaedd1f81410fa74265d8338bd016; no voters: 
I20260709 12:09:01.180145 12325 raft_consensus.cc:2804] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:09:01.181454 12325 raft_consensus.cc:697] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 LEADER]: Becoming Leader. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Running, Role: LEADER
I20260709 12:09:01.182761 12325 consensus_queue.cc:237] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:01.183795 12094 raft_consensus.cc:2468] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eeaaedd1f81410fa74265d8338bd016 in term 2.
I20260709 12:09:01.194679 11914 catalog_manager.cc:5697] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 reported cstate change: term changed from 1 to 2, leader changed from 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130) to 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "8eeaaedd1f81410fa74265d8338bd016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:01.211781 12327 raft_consensus.cc:993] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0: : Instructing follower 8eeaaedd1f81410fa74265d8338bd016 to start an election
I20260709 12:09:01.212286 12327 raft_consensus.cc:1081] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 LEADER]: Signalling peer 8eeaaedd1f81410fa74265d8338bd016 to start an election
I20260709 12:09:01.214268 12018 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6654daaddcbc4d99814fb5d868018fb1"
dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
 from {username='slave'} at 127.0.0.1:39680
I20260709 12:09:01.214943 12018 raft_consensus.cc:493] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:01.215372 12018 raft_consensus.cc:3060] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:01.224746 12018 raft_consensus.cc:515] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:01.227456 12018 leader_election.cc:290] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 3 election: Requested vote from peers 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:09:01.227847 12094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6654daaddcbc4d99814fb5d868018fb1" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
I20260709 12:09:01.228403 12094 raft_consensus.cc:3055] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:09:01.228686 12094 raft_consensus.cc:740] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Running, Role: LEADER
I20260709 12:09:01.229509 12094 consensus_queue.cc:260] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:01.229929 12167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6654daaddcbc4d99814fb5d868018fb1" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
I20260709 12:09:01.230532 12167 raft_consensus.cc:3060] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:01.230593 12094 raft_consensus.cc:3060] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:01.238065 12094 raft_consensus.cc:2468] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eeaaedd1f81410fa74265d8338bd016 in term 3.
I20260709 12:09:01.238950 12167 raft_consensus.cc:2468] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eeaaedd1f81410fa74265d8338bd016 in term 3.
I20260709 12:09:01.239794 11985 leader_election.cc:304] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016; no voters: 
I20260709 12:09:01.240659 12325 raft_consensus.cc:2804] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:09:01.241222 12325 raft_consensus.cc:697] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 LEADER]: Becoming Leader. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Running, Role: LEADER
I20260709 12:09:01.242285 12325 consensus_queue.cc:237] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:01.251307 11914 catalog_manager.cc:5697] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 reported cstate change: term changed from 2 to 3, leader changed from 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130) to 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129). New cstate: current_term: 3 leader_uuid: "8eeaaedd1f81410fa74265d8338bd016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:01.281111 12328 raft_consensus.cc:993] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0: : Instructing follower 8eeaaedd1f81410fa74265d8338bd016 to start an election
I20260709 12:09:01.281909 12327 raft_consensus.cc:1081] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 LEADER]: Signalling peer 8eeaaedd1f81410fa74265d8338bd016 to start an election
I20260709 12:09:01.283818 12018 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6b2682646881450b9d5c78c97b49c5d1"
dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
 from {username='slave'} at 127.0.0.1:39680
I20260709 12:09:01.284386 12018 raft_consensus.cc:493] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:01.284811 12018 raft_consensus.cc:3060] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:01.290336 12299 raft_consensus.cc:993] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d: : Instructing follower 8eeaaedd1f81410fa74265d8338bd016 to start an election
I20260709 12:09:01.291011 12336 raft_consensus.cc:1081] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 LEADER]: Signalling peer 8eeaaedd1f81410fa74265d8338bd016 to start an election
I20260709 12:09:01.292243 12018 raft_consensus.cc:515] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:01.293237 12017 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ff4965e3a40e45f89a81cbb4ce5c80ae"
dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
 from {username='slave'} at 127.0.0.1:39672
I20260709 12:09:01.293886 12017 raft_consensus.cc:493] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:01.294291 12018 leader_election.cc:290] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 3 election: Requested vote from peers 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:09:01.294323 12017 raft_consensus.cc:3060] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:01.294992 12094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2682646881450b9d5c78c97b49c5d1" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
I20260709 12:09:01.295585 12094 raft_consensus.cc:3055] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:09:01.295889 12094 raft_consensus.cc:740] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Running, Role: LEADER
I20260709 12:09:01.296707 12094 consensus_queue.cc:260] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:01.297015 12167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6b2682646881450b9d5c78c97b49c5d1" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
I20260709 12:09:01.297737 12167 raft_consensus.cc:3060] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:01.297909 12094 raft_consensus.cc:3060] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:01.303335 12017 raft_consensus.cc:515] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:01.304560 12094 raft_consensus.cc:2468] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eeaaedd1f81410fa74265d8338bd016 in term 3.
I20260709 12:09:01.306334 12094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff4965e3a40e45f89a81cbb4ce5c80ae" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
I20260709 12:09:01.306282 11985 leader_election.cc:304] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016; no voters: 
I20260709 12:09:01.306957 12094 raft_consensus.cc:3060] T ff4965e3a40e45f89a81cbb4ce5c80ae P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:01.307559 12325 raft_consensus.cc:2804] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:09:01.310072 12325 raft_consensus.cc:697] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 LEADER]: Becoming Leader. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Running, Role: LEADER
I20260709 12:09:01.311132 12325 consensus_queue.cc:237] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:01.312628 12168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ff4965e3a40e45f89a81cbb4ce5c80ae" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
I20260709 12:09:01.313146 12168 raft_consensus.cc:3055] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:09:01.313395 12168 raft_consensus.cc:740] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Running, Role: LEADER
I20260709 12:09:01.314154 12168 consensus_queue.cc:260] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:01.315480 12168 raft_consensus.cc:3060] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:01.316959 12094 raft_consensus.cc:2468] T ff4965e3a40e45f89a81cbb4ce5c80ae P 0a1d97359ea749a8a688b465e23abfe0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eeaaedd1f81410fa74265d8338bd016 in term 3.
I20260709 12:09:01.318163 11985 leader_election.cc:304] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016; no voters: 
I20260709 12:09:01.319092 12329 raft_consensus.cc:2804] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:09:01.307857 12167 raft_consensus.cc:2468] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eeaaedd1f81410fa74265d8338bd016 in term 3.
I20260709 12:09:01.325644 12168 raft_consensus.cc:2468] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eeaaedd1f81410fa74265d8338bd016 in term 3.
I20260709 12:09:01.326351 12017 leader_election.cc:290] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 3 election: Requested vote from peers 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:09:01.329341 12329 raft_consensus.cc:697] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 3 LEADER]: Becoming Leader. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Running, Role: LEADER
I20260709 12:09:01.330538 12329 consensus_queue.cc:237] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:01.336294 11914 catalog_manager.cc:5697] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 reported cstate change: term changed from 2 to 3, leader changed from 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130) to 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129). New cstate: current_term: 3 leader_uuid: "8eeaaedd1f81410fa74265d8338bd016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:01.352690 11914 catalog_manager.cc:5697] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 reported cstate change: term changed from 2 to 3, leader changed from 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131) to 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129). New cstate: current_term: 3 leader_uuid: "8eeaaedd1f81410fa74265d8338bd016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:01.382520 12299 raft_consensus.cc:993] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d: : Instructing follower 8eeaaedd1f81410fa74265d8338bd016 to start an election
I20260709 12:09:01.383016 12336 raft_consensus.cc:1081] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 LEADER]: Signalling peer 8eeaaedd1f81410fa74265d8338bd016 to start an election
I20260709 12:09:01.385010 12017 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5bff46f93974ecb8067db3b62a12127"
dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
 from {username='slave'} at 127.0.0.1:39672
I20260709 12:09:01.385563 12017 raft_consensus.cc:493] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:01.385882 12017 raft_consensus.cc:3060] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:01.390770 12017 raft_consensus.cc:515] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:01.392618 12017 leader_election.cc:290] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 3 election: Requested vote from peers 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:09:01.394187 12094 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bff46f93974ecb8067db3b62a12127" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
I20260709 12:09:01.394770 12094 raft_consensus.cc:3060] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:01.394728 12168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bff46f93974ecb8067db3b62a12127" candidate_uuid: "8eeaaedd1f81410fa74265d8338bd016" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
I20260709 12:09:01.395411 12168 raft_consensus.cc:3055] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:09:01.395699 12168 raft_consensus.cc:740] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Running, Role: LEADER
I20260709 12:09:01.396658 12168 consensus_queue.cc:260] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:01.398031 12168 raft_consensus.cc:3060] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:01.402048 12094 raft_consensus.cc:2468] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eeaaedd1f81410fa74265d8338bd016 in term 3.
I20260709 12:09:01.403471 11985 leader_election.cc:304] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 8eeaaedd1f81410fa74265d8338bd016; no voters: 
I20260709 12:09:01.404325 12329 raft_consensus.cc:2804] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:09:01.404625 12168 raft_consensus.cc:2468] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8eeaaedd1f81410fa74265d8338bd016 in term 3.
I20260709 12:09:01.404903 12329 raft_consensus.cc:697] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 LEADER]: Becoming Leader. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Running, Role: LEADER
I20260709 12:09:01.406038 12329 consensus_queue.cc:237] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:01.414860 11914 catalog_manager.cc:5697] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 reported cstate change: term changed from 2 to 3, leader changed from 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131) to 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129). New cstate: current_term: 3 leader_uuid: "8eeaaedd1f81410fa74265d8338bd016" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:01.581387 12168 raft_consensus.cc:1275] T ec40b30245c34cbead1a3c72ba543ad0 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Refusing update from remote peer 8eeaaedd1f81410fa74265d8338bd016: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:01.583002 12329 consensus_queue.cc:1048] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:01.595870 12094 raft_consensus.cc:1275] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Refusing update from remote peer 8eeaaedd1f81410fa74265d8338bd016: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:01.597606 12329 consensus_queue.cc:1048] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:01.721400 12094 raft_consensus.cc:1275] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [term 3 FOLLOWER]: Refusing update from remote peer 8eeaaedd1f81410fa74265d8338bd016: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:01.722596 12168 raft_consensus.cc:1275] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Refusing update from remote peer 8eeaaedd1f81410fa74265d8338bd016: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:01.722797 12329 consensus_queue.cc:1048] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:01.723765 12325 consensus_queue.cc:1048] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:01.758704 12168 raft_consensus.cc:1275] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Refusing update from remote peer 8eeaaedd1f81410fa74265d8338bd016: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:01.760033 12329 consensus_queue.cc:1048] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:01.770304 12094 raft_consensus.cc:1275] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [term 3 FOLLOWER]: Refusing update from remote peer 8eeaaedd1f81410fa74265d8338bd016: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:01.780462 12338 consensus_queue.cc:1048] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:01.916044 12168 raft_consensus.cc:1275] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Refusing update from remote peer 8eeaaedd1f81410fa74265d8338bd016: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:01.916405 12094 raft_consensus.cc:1275] T ff4965e3a40e45f89a81cbb4ce5c80ae P 0a1d97359ea749a8a688b465e23abfe0 [term 3 FOLLOWER]: Refusing update from remote peer 8eeaaedd1f81410fa74265d8338bd016: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:01.917667 12349 consensus_queue.cc:1048] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:01.918751 12325 consensus_queue.cc:1048] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:01.958626 12168 raft_consensus.cc:1275] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Refusing update from remote peer 8eeaaedd1f81410fa74265d8338bd016: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:01.960458 12338 consensus_queue.cc:1048] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:01.978152 12094 raft_consensus.cc:1275] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0 [term 3 FOLLOWER]: Refusing update from remote peer 8eeaaedd1f81410fa74265d8338bd016: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:01.979871 12325 consensus_queue.cc:1048] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:04.171341 11318 auto_leader_rebalancer-test.cc:399] The leader distribution is 
I20260709 12:09:04.172421 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:04.181104 12017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2cbf71a6592647c0993519af4377e95b"
dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
mode: GRACEFUL
new_leader_uuid: "0a1d97359ea749a8a688b465e23abfe0"
 from {username='slave'} at 127.0.0.1:39688
I20260709 12:09:04.181867 12017 raft_consensus.cc:606] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 2 LEADER]: Received request to transfer leadership to TS 0a1d97359ea749a8a688b465e23abfe0
I20260709 12:09:04.185278 12017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "85eab2235a8d42d38f8ba3bc6a972cae"
dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
mode: GRACEFUL
new_leader_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
 from {username='slave'} at 127.0.0.1:39688
I20260709 12:09:04.185988 12017 raft_consensus.cc:606] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [term 1 LEADER]: Received request to transfer leadership to TS 607b9de3a1844c9daf2e6b0ad22c727d
I20260709 12:09:04.189162 12017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c5bff46f93974ecb8067db3b62a12127"
dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
mode: GRACEFUL
new_leader_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
 from {username='slave'} at 127.0.0.1:39688
I20260709 12:09:04.189738 12017 raft_consensus.cc:606] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 LEADER]: Received request to transfer leadership to TS 607b9de3a1844c9daf2e6b0ad22c727d
I20260709 12:09:04.190586 12329 raft_consensus.cc:993] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016: : Instructing follower 607b9de3a1844c9daf2e6b0ad22c727d to start an election
I20260709 12:09:04.191273 12357 raft_consensus.cc:1081] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [term 1 LEADER]: Signalling peer 607b9de3a1844c9daf2e6b0ad22c727d to start an election
I20260709 12:09:04.192488 12017 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "df5303764ab648a987018adf0db182b8"
dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
mode: GRACEFUL
new_leader_uuid: "0a1d97359ea749a8a688b465e23abfe0"
 from {username='slave'} at 127.0.0.1:39688
I20260709 12:09:04.193063 12017 raft_consensus.cc:606] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 LEADER]: Received request to transfer leadership to TS 0a1d97359ea749a8a688b465e23abfe0
I20260709 12:09:04.193189 12168 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "85eab2235a8d42d38f8ba3bc6a972cae"
dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
 from {username='slave'} at 127.0.0.1:33046
I20260709 12:09:04.193775 12168 raft_consensus.cc:493] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:04.194298 12168 raft_consensus.cc:3060] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:04.194787 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:09:04.195677 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:04.201128 12168 raft_consensus.cc:515] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } }
I20260709 12:09:04.203784 12168 leader_election.cc:290] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 2 election: Requested vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777)
I20260709 12:09:04.204632 12017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eab2235a8d42d38f8ba3bc6a972cae" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
I20260709 12:09:04.205096 12092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "85eab2235a8d42d38f8ba3bc6a972cae" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
I20260709 12:09:04.205520 12017 raft_consensus.cc:3055] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:09:04.205754 12092 raft_consensus.cc:3060] T 85eab2235a8d42d38f8ba3bc6a972cae P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:04.205857 12017 raft_consensus.cc:740] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Running, Role: LEADER
I20260709 12:09:04.206921 12017 consensus_queue.cc:260] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } }
I20260709 12:09:04.208105 12017 raft_consensus.cc:3060] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:04.212291 12092 raft_consensus.cc:2468] T 85eab2235a8d42d38f8ba3bc6a972cae P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 607b9de3a1844c9daf2e6b0ad22c727d in term 2.
I20260709 12:09:04.213792 12134 leader_election.cc:304] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 607b9de3a1844c9daf2e6b0ad22c727d; no voters: 
I20260709 12:09:04.214637 12017 raft_consensus.cc:2468] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 607b9de3a1844c9daf2e6b0ad22c727d in term 2.
I20260709 12:09:04.214974 12356 raft_consensus.cc:2804] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:09:04.217249 12356 raft_consensus.cc:697] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 LEADER]: Becoming Leader. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Running, Role: LEADER
I20260709 12:09:04.218433 12356 consensus_queue.cc:237] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } }
I20260709 12:09:04.226547 11917 catalog_manager.cc:5697] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d reported cstate change: term changed from 1 to 2, leader changed from 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129) to 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131). New cstate: current_term: 2 leader_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:04.492751 12329 raft_consensus.cc:993] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016: : Instructing follower 607b9de3a1844c9daf2e6b0ad22c727d to start an election
I20260709 12:09:04.493553 12357 raft_consensus.cc:1081] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 LEADER]: Signalling peer 607b9de3a1844c9daf2e6b0ad22c727d to start an election
I20260709 12:09:04.495522 12168 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c5bff46f93974ecb8067db3b62a12127"
dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
 from {username='slave'} at 127.0.0.1:33046
I20260709 12:09:04.496328 12168 raft_consensus.cc:493] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:04.496776 12168 raft_consensus.cc:3060] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:09:04.502337 12168 raft_consensus.cc:515] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:04.504180 12168 leader_election.cc:290] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 4 election: Requested vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130:44777)
I20260709 12:09:04.504819 12017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bff46f93974ecb8067db3b62a12127" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
I20260709 12:09:04.505414 12017 raft_consensus.cc:3055] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:09:04.505259 12092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c5bff46f93974ecb8067db3b62a12127" candidate_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
I20260709 12:09:04.505724 12017 raft_consensus.cc:740] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Running, Role: LEADER
I20260709 12:09:04.505899 12092 raft_consensus.cc:3060] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:09:04.506532 12017 consensus_queue.cc:260] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:04.507475 12017 raft_consensus.cc:3060] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:09:04.511487 12092 raft_consensus.cc:2468] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 607b9de3a1844c9daf2e6b0ad22c727d in term 4.
I20260709 12:09:04.512769 12017 raft_consensus.cc:2468] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 607b9de3a1844c9daf2e6b0ad22c727d in term 4.
I20260709 12:09:04.512778 12134 leader_election.cc:304] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 607b9de3a1844c9daf2e6b0ad22c727d; no voters: 
I20260709 12:09:04.513999 12356 raft_consensus.cc:2804] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:09:04.514529 12356 raft_consensus.cc:697] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 4 LEADER]: Becoming Leader. State: Replica: 607b9de3a1844c9daf2e6b0ad22c727d, State: Running, Role: LEADER
I20260709 12:09:04.514919 12329 raft_consensus.cc:993] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016: : Instructing follower 0a1d97359ea749a8a688b465e23abfe0 to start an election
I20260709 12:09:04.515451 12329 raft_consensus.cc:1081] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 LEADER]: Signalling peer 0a1d97359ea749a8a688b465e23abfe0 to start an election
I20260709 12:09:04.515532 12356 consensus_queue.cc:237] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:04.517410 12092 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "df5303764ab648a987018adf0db182b8"
dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
 from {username='slave'} at 127.0.0.1:33858
I20260709 12:09:04.517870 12092 raft_consensus.cc:493] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:04.518205 12092 raft_consensus.cc:3060] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:04.525025 12092 raft_consensus.cc:515] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:04.525498 11917 catalog_manager.cc:5697] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d reported cstate change: term changed from 3 to 4, leader changed from 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129) to 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131). New cstate: current_term: 4 leader_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:04.527665 12092 leader_election.cc:290] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 3 election: Requested vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:09:04.528723 12017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df5303764ab648a987018adf0db182b8" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
I20260709 12:09:04.529238 12168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "df5303764ab648a987018adf0db182b8" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
I20260709 12:09:04.529336 12017 raft_consensus.cc:3055] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:09:04.529727 12168 raft_consensus.cc:3060] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:04.529701 12017 raft_consensus.cc:740] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Running, Role: LEADER
I20260709 12:09:04.530810 12017 consensus_queue.cc:260] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:04.532143 12017 raft_consensus.cc:3060] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:04.538360 12168 raft_consensus.cc:2468] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 3.
I20260709 12:09:04.539969 12060 leader_election.cc:304] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 607b9de3a1844c9daf2e6b0ad22c727d; no voters: 
I20260709 12:09:04.540987 12327 raft_consensus.cc:2804] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:09:04.541002 12017 raft_consensus.cc:2468] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 3.
I20260709 12:09:04.542508 12327 raft_consensus.cc:697] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [term 3 LEADER]: Becoming Leader. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Running, Role: LEADER
I20260709 12:09:04.543501 12327 consensus_queue.cc:237] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:04.551998 11917 catalog_manager.cc:5697] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 reported cstate change: term changed from 2 to 3, leader changed from 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129) to 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130). New cstate: current_term: 3 leader_uuid: "0a1d97359ea749a8a688b465e23abfe0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:04.666990 12357 raft_consensus.cc:993] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016: : Instructing follower 0a1d97359ea749a8a688b465e23abfe0 to start an election
I20260709 12:09:04.667469 12329 raft_consensus.cc:1081] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 2 LEADER]: Signalling peer 0a1d97359ea749a8a688b465e23abfe0 to start an election
I20260709 12:09:04.668962 12092 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2cbf71a6592647c0993519af4377e95b"
dest_uuid: "0a1d97359ea749a8a688b465e23abfe0"
 from {username='slave'} at 127.0.0.1:33858
I20260709 12:09:04.669404 12092 raft_consensus.cc:493] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:04.669656 12092 raft_consensus.cc:3060] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:04.674757 12092 raft_consensus.cc:515] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:04.676805 12092 leader_election.cc:290] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 3 election: Requested vote from peers 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687), 607b9de3a1844c9daf2e6b0ad22c727d (127.11.13.131:44393)
I20260709 12:09:04.677729 12017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf71a6592647c0993519af4377e95b" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "8eeaaedd1f81410fa74265d8338bd016"
I20260709 12:09:04.677896 12168 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2cbf71a6592647c0993519af4377e95b" candidate_uuid: "0a1d97359ea749a8a688b465e23abfe0" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "607b9de3a1844c9daf2e6b0ad22c727d"
I20260709 12:09:04.678263 12017 raft_consensus.cc:3055] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:09:04.678574 12168 raft_consensus.cc:3060] T 2cbf71a6592647c0993519af4377e95b P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:04.678606 12017 raft_consensus.cc:740] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 8eeaaedd1f81410fa74265d8338bd016, State: Running, Role: LEADER
I20260709 12:09:04.679520 12017 consensus_queue.cc:260] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:04.680441 12017 raft_consensus.cc:3060] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:04.683597 12168 raft_consensus.cc:2468] T 2cbf71a6592647c0993519af4377e95b P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 3.
I20260709 12:09:04.685039 12060 leader_election.cc:304] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0a1d97359ea749a8a688b465e23abfe0, 607b9de3a1844c9daf2e6b0ad22c727d; no voters: 
I20260709 12:09:04.685567 12017 raft_consensus.cc:2468] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0a1d97359ea749a8a688b465e23abfe0 in term 3.
I20260709 12:09:04.685886 12327 raft_consensus.cc:2804] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:09:04.686486 12327 raft_consensus.cc:697] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 3 LEADER]: Becoming Leader. State: Replica: 0a1d97359ea749a8a688b465e23abfe0, State: Running, Role: LEADER
I20260709 12:09:04.687309 12327 consensus_queue.cc:237] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } }
I20260709 12:09:04.694375 11917 catalog_manager.cc:5697] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 reported cstate change: term changed from 2 to 3, leader changed from 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129) to 0a1d97359ea749a8a688b465e23abfe0 (127.11.13.130). New cstate: current_term: 3 leader_uuid: "0a1d97359ea749a8a688b465e23abfe0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:04.728976 12017 raft_consensus.cc:1275] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Refusing update from remote peer 607b9de3a1844c9daf2e6b0ad22c727d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:04.730489 12356 consensus_queue.cc:1048] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:04.734455 12092 raft_consensus.cc:1275] T 85eab2235a8d42d38f8ba3bc6a972cae P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Refusing update from remote peer 607b9de3a1844c9daf2e6b0ad22c727d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:04.735715 12356 consensus_queue.cc:1048] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:05.052316 12168 raft_consensus.cc:1275] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Refusing update from remote peer 0a1d97359ea749a8a688b465e23abfe0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:05.053927 12327 consensus_queue.cc:1048] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:05.066937 12017 raft_consensus.cc:1275] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 FOLLOWER]: Refusing update from remote peer 0a1d97359ea749a8a688b465e23abfe0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:05.068521 12375 consensus_queue.cc:1048] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:05.108465 12017 raft_consensus.cc:1275] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 4 FOLLOWER]: Refusing update from remote peer 607b9de3a1844c9daf2e6b0ad22c727d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:09:05.108415 12168 raft_consensus.cc:1275] T 2cbf71a6592647c0993519af4377e95b P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Refusing update from remote peer 0a1d97359ea749a8a688b465e23abfe0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:05.110404 12358 consensus_queue.cc:1048] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:05.110422 12375 consensus_queue.cc:1048] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "607b9de3a1844c9daf2e6b0ad22c727d" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 44393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:05.121222 12092 raft_consensus.cc:1275] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0 [term 4 FOLLOWER]: Refusing update from remote peer 607b9de3a1844c9daf2e6b0ad22c727d: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:09:05.123040 12358 consensus_queue.cc:1048] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0a1d97359ea749a8a688b465e23abfe0" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 44777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:05.133690 12017 raft_consensus.cc:1275] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 3 FOLLOWER]: Refusing update from remote peer 0a1d97359ea749a8a688b465e23abfe0: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:09:05.138228 12375 consensus_queue.cc:1048] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8eeaaedd1f81410fa74265d8338bd016" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36687 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:05.197233 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:05.204291 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:09:05.204799 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:06.206014 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:06.213979 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:09:06.214504 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:07.215425 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:07.222858 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:09:07.223502 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:08.224509 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:08.231596 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:09:08.232100 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:09.233093 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:09.243093 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:09:09.243839 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:10.245030 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:10.252336 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:09:10.252939 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:11.254058 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:11.260417 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:09:11.260901 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:12.261987 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:12.267861 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:09:12.268392 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:13.269789 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:13.276721 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:09:13.277238 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:14.321344 11318 auto_leader_rebalancer-test.cc:410] The leader distribution is 
I20260709 12:09:14.322167 11318 tablet_server.cc:179] TabletServer@127.11.13.129:0 shutting down...
I20260709 12:09:14.366678 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:09:14.367566 11318 tablet_replica.cc:333] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016: stopping tablet replica
I20260709 12:09:14.368733 11318 raft_consensus.cc:2243] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:09:14.370023 11318 raft_consensus.cc:2272] T 6654daaddcbc4d99814fb5d868018fb1 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.372325 11318 tablet_replica.cc:333] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016: stopping tablet replica
I20260709 12:09:14.372877 11318 raft_consensus.cc:2243] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:09:14.373922 11318 raft_consensus.cc:2272] T 6b2682646881450b9d5c78c97b49c5d1 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.376957 11318 tablet_replica.cc:333] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016: stopping tablet replica
I20260709 12:09:14.377614 11318 raft_consensus.cc:2243] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.378365 11318 raft_consensus.cc:2272] T df5303764ab648a987018adf0db182b8 P 8eeaaedd1f81410fa74265d8338bd016 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.381093 11318 tablet_replica.cc:333] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016: stopping tablet replica
I20260709 12:09:14.381745 11318 raft_consensus.cc:2243] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.382320 11318 raft_consensus.cc:2272] T 2cbf71a6592647c0993519af4377e95b P 8eeaaedd1f81410fa74265d8338bd016 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.384346 11318 tablet_replica.cc:333] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016: stopping tablet replica
I20260709 12:09:14.384882 11318 raft_consensus.cc:2243] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:09:14.385753 11318 raft_consensus.cc:2272] T ec40b30245c34cbead1a3c72ba543ad0 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.387801 11318 tablet_replica.cc:333] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016: stopping tablet replica
I20260709 12:09:14.388839 11318 raft_consensus.cc:2243] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.389397 11318 raft_consensus.cc:2272] T 85eab2235a8d42d38f8ba3bc6a972cae P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.391563 11318 tablet_replica.cc:333] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016: stopping tablet replica
I20260709 12:09:14.392150 11318 raft_consensus.cc:2243] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:09:14.393034 11318 raft_consensus.cc:2272] T ff4965e3a40e45f89a81cbb4ce5c80ae P 8eeaaedd1f81410fa74265d8338bd016 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.395396 11318 tablet_replica.cc:333] T 1e866b8f6b68440294b311983cb3f078 P 8eeaaedd1f81410fa74265d8338bd016: stopping tablet replica
I20260709 12:09:14.395939 11318 raft_consensus.cc:2243] T 1e866b8f6b68440294b311983cb3f078 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.396490 11318 raft_consensus.cc:2272] T 1e866b8f6b68440294b311983cb3f078 P 8eeaaedd1f81410fa74265d8338bd016 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.398490 11318 tablet_replica.cc:333] T 35d1b9c0fdf0425db8242897a276970c P 8eeaaedd1f81410fa74265d8338bd016: stopping tablet replica
I20260709 12:09:14.399149 11318 raft_consensus.cc:2243] T 35d1b9c0fdf0425db8242897a276970c P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.399652 11318 raft_consensus.cc:2272] T 35d1b9c0fdf0425db8242897a276970c P 8eeaaedd1f81410fa74265d8338bd016 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.401649 11318 tablet_replica.cc:333] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016: stopping tablet replica
I20260709 12:09:14.402181 11318 raft_consensus.cc:2243] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.402652 11318 raft_consensus.cc:2272] T c5bff46f93974ecb8067db3b62a12127 P 8eeaaedd1f81410fa74265d8338bd016 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.456238 11318 tablet_server.cc:196] TabletServer@127.11.13.129:0 shutdown complete.
W20260709 12:09:14.471258 12131 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.13.129:36687: connect: Connection refused (error 111) [suppressed 5 similar messages]
W20260709 12:09:14.475894 12131 consensus_peers.cc:597] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d -> Peer 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687): Couldn't send request to peer 8eeaaedd1f81410fa74265d8338bd016. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:36687: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:14.484556 11318 tablet_server.cc:179] TabletServer@127.11.13.130:0 shutting down...
I20260709 12:09:14.528146 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:09:14.527976 12057 consensus_peers.cc:597] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 -> Peer 8eeaaedd1f81410fa74265d8338bd016 (127.11.13.129:36687): Couldn't send request to peer 8eeaaedd1f81410fa74265d8338bd016. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:36687: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:14.529140 11318 tablet_replica.cc:333] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0: stopping tablet replica
I20260709 12:09:14.530483 11318 raft_consensus.cc:2243] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.531118 11318 raft_consensus.cc:2272] T 6b2682646881450b9d5c78c97b49c5d1 P 0a1d97359ea749a8a688b465e23abfe0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.533816 11318 tablet_replica.cc:333] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0: stopping tablet replica
I20260709 12:09:14.534498 11318 raft_consensus.cc:2243] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.535085 11318 raft_consensus.cc:2272] T 6654daaddcbc4d99814fb5d868018fb1 P 0a1d97359ea749a8a688b465e23abfe0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.537107 11318 tablet_replica.cc:333] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0: stopping tablet replica
I20260709 12:09:14.537667 11318 raft_consensus.cc:2243] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.538165 11318 raft_consensus.cc:2272] T ec40b30245c34cbead1a3c72ba543ad0 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.540123 11318 tablet_replica.cc:333] T 85eab2235a8d42d38f8ba3bc6a972cae P 0a1d97359ea749a8a688b465e23abfe0: stopping tablet replica
I20260709 12:09:14.540683 11318 raft_consensus.cc:2243] T 85eab2235a8d42d38f8ba3bc6a972cae P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.541124 11318 raft_consensus.cc:2272] T 85eab2235a8d42d38f8ba3bc6a972cae P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.543102 11318 tablet_replica.cc:333] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0: stopping tablet replica
I20260709 12:09:14.543742 11318 raft_consensus.cc:2243] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.544199 11318 raft_consensus.cc:2272] T c5bff46f93974ecb8067db3b62a12127 P 0a1d97359ea749a8a688b465e23abfe0 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.546289 11318 tablet_replica.cc:333] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0: stopping tablet replica
I20260709 12:09:14.546831 11318 raft_consensus.cc:2243] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:09:14.547646 11318 raft_consensus.cc:2272] T 35d1b9c0fdf0425db8242897a276970c P 0a1d97359ea749a8a688b465e23abfe0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.549569 11318 tablet_replica.cc:333] T ff4965e3a40e45f89a81cbb4ce5c80ae P 0a1d97359ea749a8a688b465e23abfe0: stopping tablet replica
I20260709 12:09:14.550189 11318 raft_consensus.cc:2243] T ff4965e3a40e45f89a81cbb4ce5c80ae P 0a1d97359ea749a8a688b465e23abfe0 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.550704 11318 raft_consensus.cc:2272] T ff4965e3a40e45f89a81cbb4ce5c80ae P 0a1d97359ea749a8a688b465e23abfe0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.552488 11318 tablet_replica.cc:333] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0: stopping tablet replica
I20260709 12:09:14.552986 11318 raft_consensus.cc:2243] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:09:14.553788 11318 raft_consensus.cc:2272] T 2cbf71a6592647c0993519af4377e95b P 0a1d97359ea749a8a688b465e23abfe0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.555913 11318 tablet_replica.cc:333] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0: stopping tablet replica
I20260709 12:09:14.556536 11318 raft_consensus.cc:2243] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:09:14.557329 11318 raft_consensus.cc:2272] T df5303764ab648a987018adf0db182b8 P 0a1d97359ea749a8a688b465e23abfe0 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.559092 11318 tablet_replica.cc:333] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0: stopping tablet replica
I20260709 12:09:14.559598 11318 raft_consensus.cc:2243] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.560057 11318 raft_consensus.cc:2272] T 1e866b8f6b68440294b311983cb3f078 P 0a1d97359ea749a8a688b465e23abfe0 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.591842 11318 tablet_server.cc:196] TabletServer@127.11.13.130:0 shutdown complete.
I20260709 12:09:14.614770 11318 tablet_server.cc:179] TabletServer@127.11.13.131:0 shutting down...
I20260709 12:09:14.651353 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:09:14.652063 11318 tablet_replica.cc:333] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d: stopping tablet replica
I20260709 12:09:14.652690 11318 raft_consensus.cc:2243] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.653168 11318 raft_consensus.cc:2272] T ff4965e3a40e45f89a81cbb4ce5c80ae P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.655639 11318 tablet_replica.cc:333] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d: stopping tablet replica
I20260709 12:09:14.656282 11318 raft_consensus.cc:2243] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:09:14.657280 11318 raft_consensus.cc:2272] T 1e866b8f6b68440294b311983cb3f078 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.659395 11318 tablet_replica.cc:333] T ec40b30245c34cbead1a3c72ba543ad0 P 607b9de3a1844c9daf2e6b0ad22c727d: stopping tablet replica
I20260709 12:09:14.659909 11318 raft_consensus.cc:2243] T ec40b30245c34cbead1a3c72ba543ad0 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.660334 11318 raft_consensus.cc:2272] T ec40b30245c34cbead1a3c72ba543ad0 P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.662727 11318 tablet_replica.cc:333] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d: stopping tablet replica
I20260709 12:09:14.663506 11318 raft_consensus.cc:2243] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:09:14.664263 11318 raft_consensus.cc:2272] T 85eab2235a8d42d38f8ba3bc6a972cae P 607b9de3a1844c9daf2e6b0ad22c727d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.666021 11318 tablet_replica.cc:333] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d: stopping tablet replica
I20260709 12:09:14.666636 11318 raft_consensus.cc:2243] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.667074 11318 raft_consensus.cc:2272] T 6b2682646881450b9d5c78c97b49c5d1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.668780 11318 tablet_replica.cc:333] T 2cbf71a6592647c0993519af4377e95b P 607b9de3a1844c9daf2e6b0ad22c727d: stopping tablet replica
I20260709 12:09:14.669275 11318 raft_consensus.cc:2243] T 2cbf71a6592647c0993519af4377e95b P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.669703 11318 raft_consensus.cc:2272] T 2cbf71a6592647c0993519af4377e95b P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.671597 11318 tablet_replica.cc:333] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d: stopping tablet replica
I20260709 12:09:14.672060 11318 raft_consensus.cc:2243] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.672442 11318 raft_consensus.cc:2272] T df5303764ab648a987018adf0db182b8 P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.674373 11318 tablet_replica.cc:333] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d: stopping tablet replica
I20260709 12:09:14.674852 11318 raft_consensus.cc:2243] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.675338 11318 raft_consensus.cc:2272] T 6654daaddcbc4d99814fb5d868018fb1 P 607b9de3a1844c9daf2e6b0ad22c727d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.677062 11318 tablet_replica.cc:333] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d: stopping tablet replica
I20260709 12:09:14.677520 11318 raft_consensus.cc:2243] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:09:14.678331 11318 raft_consensus.cc:2272] T c5bff46f93974ecb8067db3b62a12127 P 607b9de3a1844c9daf2e6b0ad22c727d [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.680083 11318 tablet_replica.cc:333] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d: stopping tablet replica
I20260709 12:09:14.680508 11318 raft_consensus.cc:2243] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:14.680912 11318 raft_consensus.cc:2272] T 35d1b9c0fdf0425db8242897a276970c P 607b9de3a1844c9daf2e6b0ad22c727d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.713227 11318 tablet_server.cc:196] TabletServer@127.11.13.131:0 shutdown complete.
I20260709 12:09:14.737641 11318 master.cc:562] Master@127.11.13.190:34131 shutting down...
I20260709 12:09:14.769986 11318 raft_consensus.cc:2243] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:09:14.770730 11318 raft_consensus.cc:2272] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:14.771109 11318 tablet_replica.cc:333] T 00000000000000000000000000000000 P ceaf3fb618a745ecbf4fe6f92ed8a674: stopping tablet replica
I20260709 12:09:14.793942 11318 master.cc:584] Master@127.11.13.190:34131 shutdown complete.
0a1d97359ea749a8a688b465e23abfe0  4
607b9de3a1844c9daf2e6b0ad22c727d  1
8eeaaedd1f81410fa74265d8338bd016  5
0a1d97359ea749a8a688b465e23abfe0  4
607b9de3a1844c9daf2e6b0ad22c727d  3
8eeaaedd1f81410fa74265d8338bd016  3
0a1d97359ea749a8a688b465e23abfe0  1
607b9de3a1844c9daf2e6b0ad22c727d  1
8eeaaedd1f81410fa74265d8338bd016  8
0a1d97359ea749a8a688b465e23abfe0  3
607b9de3a1844c9daf2e6b0ad22c727d  3
8eeaaedd1f81410fa74265d8338bd016  4
[       OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (29063 ms)
[ RUN      ] LeaderRebalancerTest.AddTserver
I20260709 12:09:14.842376 11318 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.11.13.190:46629
I20260709 12:09:14.843606 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:09:14.850191 12405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:09:14.850242 12404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:09:14.851739 12407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:09:14.852979 11318 server_base.cc:1061] running on GCE node
I20260709 12:09:14.854054 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:09:14.854367 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:09:14.854524 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783598954854509 us; error 0 us; skew 500 ppm
I20260709 12:09:14.855144 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:09:14.857965 11318 webserver.cc:533] Webserver started at http://127.11.13.190:41949/ using document root <none> and password file <none>
I20260709 12:09:14.858592 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:09:14.858772 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:09:14.859144 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:09:14.860435 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/master-0-root/instance:
uuid: "d5d8248387cf49cfbd3416fcacedae86"
format_stamp: "Formatted at 2026-07-09 12:09:14 on dist-test-slave-zqnl"
I20260709 12:09:14.866044 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.002s
I20260709 12:09:14.870311 12412 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:14.871658 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:09:14.872192 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/master-0-root
uuid: "d5d8248387cf49cfbd3416fcacedae86"
format_stamp: "Formatted at 2026-07-09 12:09:14 on dist-test-slave-zqnl"
I20260709 12:09:14.872706 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:09:14.903942 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:09:14.905419 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:09:14.956305 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.190:46629
I20260709 12:09:14.956512 12463 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.190:46629 every 8 connection(s)
I20260709 12:09:14.961092 12464 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:14.977589 12464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Bootstrap starting.
I20260709 12:09:14.984277 12464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:14.990402 12464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: No bootstrap required, opened a new log
I20260709 12:09:14.992726 12464 raft_consensus.cc:359] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d8248387cf49cfbd3416fcacedae86" member_type: VOTER }
I20260709 12:09:14.993237 12464 raft_consensus.cc:385] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:14.993422 12464 raft_consensus.cc:740] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d8248387cf49cfbd3416fcacedae86, State: Initialized, Role: FOLLOWER
I20260709 12:09:14.994050 12464 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d8248387cf49cfbd3416fcacedae86" member_type: VOTER }
I20260709 12:09:14.994571 12464 raft_consensus.cc:399] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:09:14.994768 12464 raft_consensus.cc:493] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:09:14.994997 12464 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:14.999552 12464 raft_consensus.cc:515] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d8248387cf49cfbd3416fcacedae86" member_type: VOTER }
I20260709 12:09:15.000082 12464 leader_election.cc:304] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d5d8248387cf49cfbd3416fcacedae86; no voters: 
I20260709 12:09:15.001747 12464 leader_election.cc:290] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:09:15.002497 12467 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:15.004236 12467 raft_consensus.cc:697] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86 [term 1 LEADER]: Becoming Leader. State: Replica: d5d8248387cf49cfbd3416fcacedae86, State: Running, Role: LEADER
I20260709 12:09:15.005209 12464 sys_catalog.cc:565] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:09:15.005244 12467 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d8248387cf49cfbd3416fcacedae86" member_type: VOTER }
I20260709 12:09:15.010741 12469 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5d8248387cf49cfbd3416fcacedae86. Latest consensus state: current_term: 1 leader_uuid: "d5d8248387cf49cfbd3416fcacedae86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d8248387cf49cfbd3416fcacedae86" member_type: VOTER } }
I20260709 12:09:15.010926 12468 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d5d8248387cf49cfbd3416fcacedae86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d5d8248387cf49cfbd3416fcacedae86" member_type: VOTER } }
I20260709 12:09:15.011526 12469 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86 [sys.catalog]: This master's current role is: LEADER
I20260709 12:09:15.011724 12468 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86 [sys.catalog]: This master's current role is: LEADER
I20260709 12:09:15.014765 12474 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:09:15.023721 12474 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:09:15.024909 11318 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:09:15.034897 12474 catalog_manager.cc:1383] Generated new cluster ID: 45d44476153b46a685c60e8f8830f926
I20260709 12:09:15.035272 12474 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:09:15.056147 12474 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:09:15.057555 12474 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:09:15.083287 12474 catalog_manager.cc:6070] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Generated new TSK 0
I20260709 12:09:15.084004 12474 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:09:15.092077 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:09:15.099442 12485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:09:15.100570 12486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:09:15.102965 12488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:09:15.104257 11318 server_base.cc:1061] running on GCE node
I20260709 12:09:15.105507 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:09:15.105721 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:09:15.106416 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783598955106373 us; error 0 us; skew 500 ppm
I20260709 12:09:15.107198 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:09:15.109906 11318 webserver.cc:533] Webserver started at http://127.11.13.129:42733/ using document root <none> and password file <none>
I20260709 12:09:15.110476 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:09:15.110656 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:09:15.111006 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:09:15.112206 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-0-root/instance:
uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
format_stamp: "Formatted at 2026-07-09 12:09:15 on dist-test-slave-zqnl"
I20260709 12:09:15.117506 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:09:15.121714 12493 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:15.122823 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:09:15.123268 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-0-root
uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
format_stamp: "Formatted at 2026-07-09 12:09:15 on dist-test-slave-zqnl"
I20260709 12:09:15.123782 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:09:15.138413 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:09:15.140005 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:09:15.142012 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:09:15.145124 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:09:15.145435 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:15.145773 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:09:15.145987 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:15.193542 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.129:38811
I20260709 12:09:15.193672 12555 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.129:38811 every 8 connection(s)
I20260709 12:09:15.198936 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:09:15.208302 12560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:09:15.213352 12561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:09:15.216800 12563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:09:15.217046 11318 server_base.cc:1061] running on GCE node
I20260709 12:09:15.219233 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:09:15.219480 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:09:15.219748 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783598955219721 us; error 0 us; skew 500 ppm
I20260709 12:09:15.220479 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:09:15.221485 12556 heartbeater.cc:344] Connected to a master server at 127.11.13.190:46629
I20260709 12:09:15.221943 12556 heartbeater.cc:461] Registering TS with master...
I20260709 12:09:15.223363 12556 heartbeater.cc:507] Master 127.11.13.190:46629 requested a full tablet report, sending...
I20260709 12:09:15.223476 11318 webserver.cc:533] Webserver started at http://127.11.13.130:33283/ using document root <none> and password file <none>
I20260709 12:09:15.224136 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:09:15.224359 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:09:15.224742 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:09:15.226195 12429 ts_manager.cc:194] Registered new tserver with Master: a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:09:15.226346 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-1-root/instance:
uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
format_stamp: "Formatted at 2026-07-09 12:09:15 on dist-test-slave-zqnl"
I20260709 12:09:15.228194 12429 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58980
I20260709 12:09:15.232172 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:09:15.236546 12568 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:15.237638 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:09:15.238013 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-1-root
uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
format_stamp: "Formatted at 2026-07-09 12:09:15 on dist-test-slave-zqnl"
I20260709 12:09:15.238428 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:09:15.279593 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:09:15.280946 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:09:15.282773 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:09:15.285537 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:09:15.285810 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:15.286111 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:09:15.286337 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:09:15.336195 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.130:39365
I20260709 12:09:15.336572 12630 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.130:39365 every 8 connection(s)
I20260709 12:09:15.341671 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:09:15.350786 12634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:09:15.353721 12635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:09:15.354141 11318 server_base.cc:1061] running on GCE node
W20260709 12:09:15.355046 12637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:09:15.356415 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:09:15.356766 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:09:15.357044 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783598955357021 us; error 0 us; skew 500 ppm
I20260709 12:09:15.357923 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:09:15.358552 12631 heartbeater.cc:344] Connected to a master server at 127.11.13.190:46629
I20260709 12:09:15.358959 12631 heartbeater.cc:461] Registering TS with master...
I20260709 12:09:15.359980 12631 heartbeater.cc:507] Master 127.11.13.190:46629 requested a full tablet report, sending...
I20260709 12:09:15.361317 11318 webserver.cc:533] Webserver started at http://127.11.13.131:35131/ using document root <none> and password file <none>
I20260709 12:09:15.362317 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:09:15.362761 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:09:15.363134 12429 ts_manager.cc:194] Registered new tserver with Master: f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:15.363320 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:09:15.365448 12429 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58992
I20260709 12:09:15.365406 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-2-root/instance:
uuid: "193a40da6e364ead8ab561e14c3cc8d1"
format_stamp: "Formatted at 2026-07-09 12:09:15 on dist-test-slave-zqnl"
I20260709 12:09:15.372180 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:09:15.376850 12642 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:15.377974 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:09:15.378479 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-2-root
uuid: "193a40da6e364ead8ab561e14c3cc8d1"
format_stamp: "Formatted at 2026-07-09 12:09:15 on dist-test-slave-zqnl"
I20260709 12:09:15.378826 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:09:15.422672 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:09:15.423945 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:09:15.425606 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:09:15.427946 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:09:15.428345 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:15.428632 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:09:15.428840 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:15.477547 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.131:38613
I20260709 12:09:15.477684 12704 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.131:38613 every 8 connection(s)
I20260709 12:09:15.495261 12705 heartbeater.cc:344] Connected to a master server at 127.11.13.190:46629
I20260709 12:09:15.495997 12705 heartbeater.cc:461] Registering TS with master...
I20260709 12:09:15.496948 12705 heartbeater.cc:507] Master 127.11.13.190:46629 requested a full tablet report, sending...
I20260709 12:09:15.499859 12429 ts_manager.cc:194] Registered new tserver with Master: 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:15.500754 11318 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019492264s
I20260709 12:09:15.501783 12429 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59006
I20260709 12:09:15.534564 12429 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59018:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:09:15.540863 12429 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:09:15.776153 12669 tablet_service.cc:1511] Processing CreateTablet for tablet 10395d6e75b84c3982521323f92e3309 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:09:15.776823 12596 tablet_service.cc:1511] Processing CreateTablet for tablet b65110412db1492aa47f4a845ec89242 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:09:15.777426 12670 tablet_service.cc:1511] Processing CreateTablet for tablet b65110412db1492aa47f4a845ec89242 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:09:15.778424 12596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65110412db1492aa47f4a845ec89242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.778540 12594 tablet_service.cc:1511] Processing CreateTablet for tablet e15d21db797e4a96b5732711a5379980 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:09:15.779537 12668 tablet_service.cc:1511] Processing CreateTablet for tablet e15d21db797e4a96b5732711a5379980 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:09:15.780054 12594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e15d21db797e4a96b5732711a5379980. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.779981 12593 tablet_service.cc:1511] Processing CreateTablet for tablet bdfb480850c24efa8a38a916d53bdb1f (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:09:15.781512 12593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdfb480850c24efa8a38a916d53bdb1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.782085 12667 tablet_service.cc:1511] Processing CreateTablet for tablet bdfb480850c24efa8a38a916d53bdb1f (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:09:15.783463 12592 tablet_service.cc:1511] Processing CreateTablet for tablet 33a0b4d6cfba4ab2b3d54f591439b179 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:09:15.783519 12595 tablet_service.cc:1511] Processing CreateTablet for tablet 10395d6e75b84c3982521323f92e3309 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:09:15.784672 12666 tablet_service.cc:1511] Processing CreateTablet for tablet 33a0b4d6cfba4ab2b3d54f591439b179 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:09:15.778020 12669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10395d6e75b84c3982521323f92e3309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.788959 12666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33a0b4d6cfba4ab2b3d54f591439b179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.790485 12670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65110412db1492aa47f4a845ec89242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.794201 12668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e15d21db797e4a96b5732711a5379980. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.793743 12591 tablet_service.cc:1511] Processing CreateTablet for tablet 569d6ab756014cd88db2198d5d25dccd (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:09:15.798616 12667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdfb480850c24efa8a38a916d53bdb1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.798454 12665 tablet_service.cc:1511] Processing CreateTablet for tablet 569d6ab756014cd88db2198d5d25dccd (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:09:15.799836 12665 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 569d6ab756014cd88db2198d5d25dccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.801146 12590 tablet_service.cc:1511] Processing CreateTablet for tablet 3585ecc88ce149c5b13cb3e294485b62 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:09:15.805492 12664 tablet_service.cc:1511] Processing CreateTablet for tablet 3585ecc88ce149c5b13cb3e294485b62 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:09:15.807152 12664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3585ecc88ce149c5b13cb3e294485b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.808450 12589 tablet_service.cc:1511] Processing CreateTablet for tablet 70b8c9b89e3d43fbb70422110d76702c (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:09:15.810846 12592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33a0b4d6cfba4ab2b3d54f591439b179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.814324 12663 tablet_service.cc:1511] Processing CreateTablet for tablet 70b8c9b89e3d43fbb70422110d76702c (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:09:15.819886 12595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10395d6e75b84c3982521323f92e3309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.823695 12520 tablet_service.cc:1511] Processing CreateTablet for tablet 10395d6e75b84c3982521323f92e3309 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:09:15.825296 12520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 10395d6e75b84c3982521323f92e3309. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.823724 12590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3585ecc88ce149c5b13cb3e294485b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.826640 12521 tablet_service.cc:1511] Processing CreateTablet for tablet b65110412db1492aa47f4a845ec89242 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:09:15.827559 12519 tablet_service.cc:1511] Processing CreateTablet for tablet e15d21db797e4a96b5732711a5379980 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:09:15.828421 12521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b65110412db1492aa47f4a845ec89242. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.829128 12519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e15d21db797e4a96b5732711a5379980. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.826812 12588 tablet_service.cc:1511] Processing CreateTablet for tablet b7cccd56fa854cbc84a922238ad4add9 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:09:15.830325 12588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7cccd56fa854cbc84a922238ad4add9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.831419 12518 tablet_service.cc:1511] Processing CreateTablet for tablet bdfb480850c24efa8a38a916d53bdb1f (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:09:15.833343 12589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70b8c9b89e3d43fbb70422110d76702c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.833894 12517 tablet_service.cc:1511] Processing CreateTablet for tablet 33a0b4d6cfba4ab2b3d54f591439b179 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:09:15.834758 12663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70b8c9b89e3d43fbb70422110d76702c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.835704 12514 tablet_service.cc:1511] Processing CreateTablet for tablet 70b8c9b89e3d43fbb70422110d76702c (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:09:15.836464 12516 tablet_service.cc:1511] Processing CreateTablet for tablet 569d6ab756014cd88db2198d5d25dccd (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:09:15.839939 12662 tablet_service.cc:1511] Processing CreateTablet for tablet b7cccd56fa854cbc84a922238ad4add9 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:09:15.839516 12515 tablet_service.cc:1511] Processing CreateTablet for tablet 3585ecc88ce149c5b13cb3e294485b62 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:09:15.843151 12518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bdfb480850c24efa8a38a916d53bdb1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.844061 12517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33a0b4d6cfba4ab2b3d54f591439b179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.846807 12516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 569d6ab756014cd88db2198d5d25dccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.843354 12591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 569d6ab756014cd88db2198d5d25dccd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.849336 12514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70b8c9b89e3d43fbb70422110d76702c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.850066 12587 tablet_service.cc:1511] Processing CreateTablet for tablet 952cdab9198b44e4bdf1b472109715c3 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:09:15.853781 12515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3585ecc88ce149c5b13cb3e294485b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.855301 12661 tablet_service.cc:1511] Processing CreateTablet for tablet 952cdab9198b44e4bdf1b472109715c3 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:09:15.841746 12662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7cccd56fa854cbc84a922238ad4add9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.862865 12587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 952cdab9198b44e4bdf1b472109715c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.867318 12513 tablet_service.cc:1511] Processing CreateTablet for tablet b7cccd56fa854cbc84a922238ad4add9 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:09:15.868865 12513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7cccd56fa854cbc84a922238ad4add9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.871488 12512 tablet_service.cc:1511] Processing CreateTablet for tablet 952cdab9198b44e4bdf1b472109715c3 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:09:15.874562 12661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 952cdab9198b44e4bdf1b472109715c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.874645 12512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 952cdab9198b44e4bdf1b472109715c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.875203 12596 tablet_service.cc:1511] Processing CreateTablet for tablet 05ff02a879ef4c94966348127fec27f7 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:09:15.891683 12596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05ff02a879ef4c94966348127fec27f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.900090 12666 tablet_service.cc:1511] Processing CreateTablet for tablet 05ff02a879ef4c94966348127fec27f7 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:09:15.925406 12720 tablet_bootstrap.cc:492] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:15.949415 12720 tablet_bootstrap.cc:654] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:15.950868 12666 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05ff02a879ef4c94966348127fec27f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.970813 12667 tablet_service.cc:1511] Processing CreateTablet for tablet d7bd864b75c34027b05582a55b152877 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:09:15.972280 12667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7bd864b75c34027b05582a55b152877. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.978881 12669 tablet_service.cc:1511] Processing CreateTablet for tablet cf6806e0707d42a0b80b9b9a65444604 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:09:15.980379 12669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf6806e0707d42a0b80b9b9a65444604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.983415 12663 tablet_service.cc:1511] Processing CreateTablet for tablet aeb1937b72c34b78909bb9d4840f1f06 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:09:15.984347 12595 tablet_service.cc:1511] Processing CreateTablet for tablet cf6806e0707d42a0b80b9b9a65444604 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:09:15.985029 12663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeb1937b72c34b78909bb9d4840f1f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.985700 12595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf6806e0707d42a0b80b9b9a65444604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:15.999992 12668 tablet_service.cc:1511] Processing CreateTablet for tablet dd783245c5154ded95c4fe8434d0b380 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:09:16.001874 12721 tablet_bootstrap.cc:492] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
W20260709 12:09:16.019699 12477 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:16.034335 12668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd783245c5154ded95c4fe8434d0b380. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.040863 12594 tablet_service.cc:1511] Processing CreateTablet for tablet d7bd864b75c34027b05582a55b152877 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:09:16.042372 12594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7bd864b75c34027b05582a55b152877. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.042035 12591 tablet_service.cc:1511] Processing CreateTablet for tablet aeb1937b72c34b78909bb9d4840f1f06 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:09:16.043427 12591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeb1937b72c34b78909bb9d4840f1f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.048197 12662 tablet_service.cc:1511] Processing CreateTablet for tablet 618bbf28983a42beaaf337df12784ee1 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:09:16.049459 12722 tablet_bootstrap.cc:492] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:16.048199 12593 tablet_service.cc:1511] Processing CreateTablet for tablet dd783245c5154ded95c4fe8434d0b380 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:09:16.050554 12593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd783245c5154ded95c4fe8434d0b380. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.060359 12518 tablet_service.cc:1511] Processing CreateTablet for tablet cf6806e0707d42a0b80b9b9a65444604 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:09:16.062492 12518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf6806e0707d42a0b80b9b9a65444604. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.066386 12513 tablet_service.cc:1511] Processing CreateTablet for tablet d7bd864b75c34027b05582a55b152877 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:09:16.068030 12513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7bd864b75c34027b05582a55b152877. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.057524 12516 tablet_service.cc:1511] Processing CreateTablet for tablet 05ff02a879ef4c94966348127fec27f7 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:09:16.075205 12516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 05ff02a879ef4c94966348127fec27f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.062007 12590 tablet_service.cc:1511] Processing CreateTablet for tablet 618bbf28983a42beaaf337df12784ee1 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:09:16.091945 12590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 618bbf28983a42beaaf337df12784ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.095171 12670 tablet_service.cc:1511] Processing CreateTablet for tablet d29f84bdbc4c49278fb44e72ddb1a33c (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:09:16.105538 12664 tablet_service.cc:1511] Processing CreateTablet for tablet 816b6f34d124434882148e507cf5b612 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:09:16.102299 12595 tablet_service.cc:1511] Processing CreateTablet for tablet d29f84bdbc4c49278fb44e72ddb1a33c (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:09:16.107606 12595 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d29f84bdbc4c49278fb44e72ddb1a33c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.110458 12662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 618bbf28983a42beaaf337df12784ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.111358 12588 tablet_service.cc:1511] Processing CreateTablet for tablet 816b6f34d124434882148e507cf5b612 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:09:16.112699 12588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 816b6f34d124434882148e507cf5b612. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.115720 12722 tablet_bootstrap.cc:654] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:16.117436 12664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 816b6f34d124434882148e507cf5b612. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.120043 12668 tablet_service.cc:1511] Processing CreateTablet for tablet f0d97a077cac41709974a6cde7865fcb (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:09:16.129709 12670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d29f84bdbc4c49278fb44e72ddb1a33c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.132033 12515 tablet_service.cc:1511] Processing CreateTablet for tablet aeb1937b72c34b78909bb9d4840f1f06 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:09:16.133312 12589 tablet_service.cc:1511] Processing CreateTablet for tablet f0d97a077cac41709974a6cde7865fcb (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:09:16.136125 12518 tablet_service.cc:1511] Processing CreateTablet for tablet dd783245c5154ded95c4fe8434d0b380 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:09:16.141965 12517 tablet_service.cc:1511] Processing CreateTablet for tablet 618bbf28983a42beaaf337df12784ee1 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:09:16.142949 12720 tablet_bootstrap.cc:492] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:16.145341 12520 tablet_service.cc:1511] Processing CreateTablet for tablet d29f84bdbc4c49278fb44e72ddb1a33c (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:09:16.147795 12512 tablet_service.cc:1511] Processing CreateTablet for tablet 816b6f34d124434882148e507cf5b612 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:09:16.148567 12587 tablet_service.cc:1511] Processing CreateTablet for tablet 15456f360e78465994eb4d0d39e90b87 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:09:16.151298 12668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0d97a077cac41709974a6cde7865fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.154282 12513 tablet_service.cc:1511] Processing CreateTablet for tablet f0d97a077cac41709974a6cde7865fcb (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:09:16.176450 12661 tablet_service.cc:1511] Processing CreateTablet for tablet 15456f360e78465994eb4d0d39e90b87 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:09:16.177882 12661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15456f360e78465994eb4d0d39e90b87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.161473 12721 tablet_bootstrap.cc:654] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:16.195384 12514 tablet_service.cc:1511] Processing CreateTablet for tablet ade78efb80994e1b835a9aeec05fd8cd (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:09:16.167050 12515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aeb1937b72c34b78909bb9d4840f1f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.167050 12589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0d97a077cac41709974a6cde7865fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.199013 12514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ade78efb80994e1b835a9aeec05fd8cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.204087 12722 tablet_bootstrap.cc:492] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:16.204707 12722 ts_tablet_manager.cc:1403] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.156s	user 0.008s	sys 0.008s
I20260709 12:09:16.206496 12517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 618bbf28983a42beaaf337df12784ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.207517 12593 tablet_service.cc:1511] Processing CreateTablet for tablet ade78efb80994e1b835a9aeec05fd8cd (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:09:16.209019 12593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ade78efb80994e1b835a9aeec05fd8cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.210479 12587 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15456f360e78465994eb4d0d39e90b87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.211419 12668 tablet_service.cc:1511] Processing CreateTablet for tablet ade78efb80994e1b835a9aeec05fd8cd (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:09:16.187052 12720 ts_tablet_manager.cc:1403] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.262s	user 0.014s	sys 0.003s
I20260709 12:09:16.213284 12668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ade78efb80994e1b835a9aeec05fd8cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.215950 12720 raft_consensus.cc:359] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:16.216795 12720 raft_consensus.cc:385] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:16.217186 12720 raft_consensus.cc:740] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:16.220479 12722 raft_consensus.cc:359] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:16.226099 12722 raft_consensus.cc:385] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:16.226619 12722 raft_consensus.cc:740] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:16.230505 12520 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d29f84bdbc4c49278fb44e72ddb1a33c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.231086 12556 heartbeater.cc:499] Master 127.11.13.190:46629 was elected leader, sending a full tablet report...
I20260709 12:09:16.231102 12722 consensus_queue.cc:260] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:16.170575 12516 tablet_service.cc:1511] Processing CreateTablet for tablet 15456f360e78465994eb4d0d39e90b87 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:09:16.233505 12592 tablet_service.cc:1511] Processing CreateTablet for tablet 43d4539f52e24ea8adc368a4c9529e66 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:09:16.236889 12518 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd783245c5154ded95c4fe8434d0b380. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.237522 12592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43d4539f52e24ea8adc368a4c9529e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.252370 12512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 816b6f34d124434882148e507cf5b612. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.271538 12663 tablet_service.cc:1511] Processing CreateTablet for tablet 43d4539f52e24ea8adc368a4c9529e66 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:09:16.267592 12513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f0d97a077cac41709974a6cde7865fcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.279987 12663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43d4539f52e24ea8adc368a4c9529e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.279625 12519 tablet_service.cc:1511] Processing CreateTablet for tablet 52c63945e7484332bf4c454ec7423a21 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:09:16.281890 12669 tablet_service.cc:1511] Processing CreateTablet for tablet 52c63945e7484332bf4c454ec7423a21 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:09:16.283188 12516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15456f360e78465994eb4d0d39e90b87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.283556 12669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52c63945e7484332bf4c454ec7423a21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.286327 12519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52c63945e7484332bf4c454ec7423a21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.287371 12596 tablet_service.cc:1511] Processing CreateTablet for tablet 52c63945e7484332bf4c454ec7423a21 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:09:16.289063 12596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 52c63945e7484332bf4c454ec7423a21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.291325 12514 tablet_service.cc:1511] Processing CreateTablet for tablet 6fa9b4ceed3443f2a6992343ab6ad877 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:09:16.292801 12514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fa9b4ceed3443f2a6992343ab6ad877. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.302045 12591 tablet_service.cc:1511] Processing CreateTablet for tablet 6fa9b4ceed3443f2a6992343ab6ad877 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:09:16.303723 12591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fa9b4ceed3443f2a6992343ab6ad877. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.305070 12667 tablet_service.cc:1511] Processing CreateTablet for tablet 6fa9b4ceed3443f2a6992343ab6ad877 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:09:16.306732 12667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fa9b4ceed3443f2a6992343ab6ad877. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.306596 12515 tablet_service.cc:1511] Processing CreateTablet for tablet 57b456cf0a3b4f0f9f6cae5a1862caca (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:09:16.249058 12720 consensus_queue.cc:260] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:16.324680 12594 tablet_service.cc:1511] Processing CreateTablet for tablet 57b456cf0a3b4f0f9f6cae5a1862caca (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:09:16.326308 12594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57b456cf0a3b4f0f9f6cae5a1862caca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.326634 12662 tablet_service.cc:1511] Processing CreateTablet for tablet 57b456cf0a3b4f0f9f6cae5a1862caca (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:09:16.328014 12662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57b456cf0a3b4f0f9f6cae5a1862caca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.334414 12521 tablet_service.cc:1511] Processing CreateTablet for tablet 43d4539f52e24ea8adc368a4c9529e66 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:09:16.341786 12590 tablet_service.cc:1511] Processing CreateTablet for tablet cdb0bb14b2b745b398e14d8c1a7139a2 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:09:16.343431 12590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdb0bb14b2b745b398e14d8c1a7139a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.349450 12515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57b456cf0a3b4f0f9f6cae5a1862caca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.365839 12721 tablet_bootstrap.cc:492] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:16.366616 12721 ts_tablet_manager.cc:1403] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.365s	user 0.019s	sys 0.019s
I20260709 12:09:16.365717 12631 heartbeater.cc:499] Master 127.11.13.190:46629 was elected leader, sending a full tablet report...
I20260709 12:09:16.371999 12588 tablet_service.cc:1511] Processing CreateTablet for tablet 3af899b7fab042d0ac433430f162d106 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:09:16.373392 12588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3af899b7fab042d0ac433430f162d106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.369642 12721 raft_consensus.cc:359] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:16.383124 12721 raft_consensus.cc:385] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:16.383621 12721 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:16.385181 12721 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:16.385870 12515 tablet_service.cc:1511] Processing CreateTablet for tablet 3af899b7fab042d0ac433430f162d106 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:09:16.387295 12515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3af899b7fab042d0ac433430f162d106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.390406 12661 tablet_service.cc:1511] Processing CreateTablet for tablet e40144c1c60847ee81b8252b107df67d (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:09:16.391834 12661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e40144c1c60847ee81b8252b107df67d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.398530 12722 ts_tablet_manager.cc:1434] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.193s	user 0.012s	sys 0.005s
I20260709 12:09:16.399682 12722 tablet_bootstrap.cc:492] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:16.363775 12669 tablet_service.cc:1511] Processing CreateTablet for tablet cdb0bb14b2b745b398e14d8c1a7139a2 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:09:16.403311 12669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdb0bb14b2b745b398e14d8c1a7139a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.383503 12667 tablet_service.cc:1511] Processing CreateTablet for tablet 3af899b7fab042d0ac433430f162d106 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:09:16.407652 12593 tablet_service.cc:1511] Processing CreateTablet for tablet e40144c1c60847ee81b8252b107df67d (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:09:16.407387 12515 tablet_service.cc:1511] Processing CreateTablet for tablet e40144c1c60847ee81b8252b107df67d (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:09:16.409215 12515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e40144c1c60847ee81b8252b107df67d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.363574 12720 ts_tablet_manager.cc:1434] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.151s	user 0.005s	sys 0.008s
I20260709 12:09:16.411461 12720 tablet_bootstrap.cc:492] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:16.409209 12593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e40144c1c60847ee81b8252b107df67d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.408138 12667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3af899b7fab042d0ac433430f162d106. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.419122 12521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43d4539f52e24ea8adc368a4c9529e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.421958 12670 tablet_service.cc:1511] Processing CreateTablet for tablet ee30d7cd60b04485b7e19c8d119d67bd (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:09:16.423508 12670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee30d7cd60b04485b7e19c8d119d67bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.425558 12517 tablet_service.cc:1511] Processing CreateTablet for tablet cdb0bb14b2b745b398e14d8c1a7139a2 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:09:16.459224 12517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdb0bb14b2b745b398e14d8c1a7139a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.428574 12512 tablet_service.cc:1511] Processing CreateTablet for tablet ee30d7cd60b04485b7e19c8d119d67bd (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:09:16.463658 12705 heartbeater.cc:499] Master 127.11.13.190:46629 was elected leader, sending a full tablet report...
I20260709 12:09:16.463564 12591 tablet_service.cc:1511] Processing CreateTablet for tablet 8f2a3bfdbbd6440ba9cf687448fb678c (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:09:16.464665 12512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee30d7cd60b04485b7e19c8d119d67bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.465117 12591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f2a3bfdbbd6440ba9cf687448fb678c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.431656 12589 tablet_service.cc:1511] Processing CreateTablet for tablet ee30d7cd60b04485b7e19c8d119d67bd (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:09:16.446166 12720 tablet_bootstrap.cc:654] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:16.475553 12721 ts_tablet_manager.cc:1434] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.109s	user 0.009s	sys 0.000s
I20260709 12:09:16.477008 12721 tablet_bootstrap.cc:492] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:16.480113 12515 tablet_service.cc:1511] Processing CreateTablet for tablet 8f2a3bfdbbd6440ba9cf687448fb678c (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:09:16.481876 12515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f2a3bfdbbd6440ba9cf687448fb678c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.490535 12589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee30d7cd60b04485b7e19c8d119d67bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.499385 12722 tablet_bootstrap.cc:654] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:16.506314 12727 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:16.506898 12727 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:16.509786 12727 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:16.537009 12721 tablet_bootstrap.cc:654] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:16.540158 12720 tablet_bootstrap.cc:492] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:16.553122 12720 ts_tablet_manager.cc:1403] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.142s	user 0.012s	sys 0.006s
I20260709 12:09:16.548570 12722 tablet_bootstrap.cc:492] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:16.554876 12722 ts_tablet_manager.cc:1403] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.155s	user 0.012s	sys 0.006s
I20260709 12:09:16.558195 12722 raft_consensus.cc:359] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:16.559063 12722 raft_consensus.cc:385] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:16.561033 12722 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:16.562048 12722 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:16.564359 12722 ts_tablet_manager.cc:1434] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:09:16.564924 12720 raft_consensus.cc:359] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:09:16.565493 12722 tablet_bootstrap.cc:492] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:16.565724 12720 raft_consensus.cc:385] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:16.566107 12720 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:16.569099 12720 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:09:16.572475 12720 ts_tablet_manager.cc:1434] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.019s	user 0.005s	sys 0.002s
I20260709 12:09:16.573558 12720 tablet_bootstrap.cc:492] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:16.584369 12588 tablet_service.cc:1511] Processing CreateTablet for tablet 852ac65e35c7462ba906c02b9422dc5b (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:09:16.585129 12721 tablet_bootstrap.cc:492] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:16.585868 12588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 852ac65e35c7462ba906c02b9422dc5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.588847 12669 tablet_service.cc:1511] Processing CreateTablet for tablet 852ac65e35c7462ba906c02b9422dc5b (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:09:16.590709 12669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 852ac65e35c7462ba906c02b9422dc5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.590873 12670 tablet_service.cc:1511] Processing CreateTablet for tablet 8f2a3bfdbbd6440ba9cf687448fb678c (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:09:16.592139 12590 tablet_service.cc:1511] Processing CreateTablet for tablet 29bbd70aa24b4e81b3d2764c3584ad60 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:09:16.593701 12590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29bbd70aa24b4e81b3d2764c3584ad60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.592677 12670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8f2a3bfdbbd6440ba9cf687448fb678c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.597956 12521 tablet_service.cc:1511] Processing CreateTablet for tablet 29bbd70aa24b4e81b3d2764c3584ad60 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:09:16.600256 12521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29bbd70aa24b4e81b3d2764c3584ad60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.602727 12720 tablet_bootstrap.cc:654] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:16.607218 12662 tablet_service.cc:1511] Processing CreateTablet for tablet 29bbd70aa24b4e81b3d2764c3584ad60 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:09:16.608682 12662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 29bbd70aa24b4e81b3d2764c3584ad60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.611152 12517 tablet_service.cc:1511] Processing CreateTablet for tablet 852ac65e35c7462ba906c02b9422dc5b (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:09:16.612529 12517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 852ac65e35c7462ba906c02b9422dc5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.585860 12721 ts_tablet_manager.cc:1403] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.109s	user 0.011s	sys 0.001s
I20260709 12:09:16.628887 12512 tablet_service.cc:1511] Processing CreateTablet for tablet 8c7e0eddcc2b4449b7ed8821854646e2 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:09:16.630365 12512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c7e0eddcc2b4449b7ed8821854646e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.625638 12721 raft_consensus.cc:359] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:16.649225 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:16.648981 12596 tablet_service.cc:1511] Processing CreateTablet for tablet 8c7e0eddcc2b4449b7ed8821854646e2 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
W20260709 12:09:16.651192 12571 leader_election.cc:343] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:16.651930 12596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c7e0eddcc2b4449b7ed8821854646e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.652951 12667 tablet_service.cc:1511] Processing CreateTablet for tablet 8c7e0eddcc2b4449b7ed8821854646e2 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:09:16.654383 12667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c7e0eddcc2b4449b7ed8821854646e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.644411 12721 raft_consensus.cc:385] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:16.672659 12726 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:16.673219 12726 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:16.674058 12519 tablet_service.cc:1511] Processing CreateTablet for tablet 502d264440884ecda3d4641918b20d0c (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:09:16.675480 12519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 502d264440884ecda3d4641918b20d0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.669901 12721 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:16.686444 12661 tablet_service.cc:1511] Processing CreateTablet for tablet 502d264440884ecda3d4641918b20d0c (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:09:16.687868 12661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 502d264440884ecda3d4641918b20d0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.682557 12726 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:16.697376 12516 tablet_service.cc:1511] Processing CreateTablet for tablet 53977095735a4e34bc297c91e8c6f17f (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:09:16.698935 12516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53977095735a4e34bc297c91e8c6f17f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.699275 12589 tablet_service.cc:1511] Processing CreateTablet for tablet 53977095735a4e34bc297c91e8c6f17f (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:09:16.700959 12728 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:16.701462 12728 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:16.706290 12720 tablet_bootstrap.cc:492] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:16.706759 12728 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:16.706964 12722 tablet_bootstrap.cc:654] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:16.706894 12720 ts_tablet_manager.cc:1403] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.134s	user 0.016s	sys 0.000s
I20260709 12:09:16.717641 12589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53977095735a4e34bc297c91e8c6f17f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.726368 12720 raft_consensus.cc:359] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:16.727185 12720 raft_consensus.cc:385] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:16.727483 12720 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:16.743031 12667 tablet_service.cc:1511] Processing CreateTablet for tablet 53977095735a4e34bc297c91e8c6f17f (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:09:16.744362 12667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 53977095735a4e34bc297c91e8c6f17f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.731525 12721 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:16.761113 12720 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:16.767347 12721 ts_tablet_manager.cc:1434] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.145s	user 0.007s	sys 0.001s
I20260709 12:09:16.768846 12721 tablet_bootstrap.cc:492] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:16.782618 12591 tablet_service.cc:1511] Processing CreateTablet for tablet 502d264440884ecda3d4641918b20d0c (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:09:16.784365 12591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 502d264440884ecda3d4641918b20d0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.795974 12720 ts_tablet_manager.cc:1434] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.088s	user 0.007s	sys 0.002s
I20260709 12:09:16.797235 12720 tablet_bootstrap.cc:492] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
W20260709 12:09:16.812830 12415 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 74251.4us
I20260709 12:09:16.816529 12517 tablet_service.cc:1511] Processing CreateTablet for tablet 6530c20e384d4ff0915a09d0641c82f8 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:09:16.828506 12519 tablet_service.cc:1511] Processing CreateTablet for tablet a92d1d91cce04cec98366aa6021356df (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:09:16.831606 12514 tablet_service.cc:1511] Processing CreateTablet for tablet 5771c48bd1d24152ab96cfc24da505ed (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:09:16.844019 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:09:16.845700 12494 leader_election.cc:343] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:16.846378 12720 tablet_bootstrap.cc:654] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:16.849172 12667 tablet_service.cc:1511] Processing CreateTablet for tablet a92d1d91cce04cec98366aa6021356df (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:09:16.849566 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:16.850569 12667 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a92d1d91cce04cec98366aa6021356df. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:09:16.851243 12646 leader_election.cc:343] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:16.859340 12514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5771c48bd1d24152ab96cfc24da505ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.859467 12670 tablet_service.cc:1511] Processing CreateTablet for tablet 5771c48bd1d24152ab96cfc24da505ed (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:09:16.861225 12670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5771c48bd1d24152ab96cfc24da505ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.861775 12721 tablet_bootstrap.cc:654] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:16.864933 12519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a92d1d91cce04cec98366aa6021356df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.865530 12661 tablet_service.cc:1511] Processing CreateTablet for tablet e1ff01ff4ede4fc990b9a07b23102b0d (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:09:16.866983 12661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1ff01ff4ede4fc990b9a07b23102b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.871331 12517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6530c20e384d4ff0915a09d0641c82f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.875319 12669 tablet_service.cc:1511] Processing CreateTablet for tablet 6530c20e384d4ff0915a09d0641c82f8 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:09:16.877061 12669 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6530c20e384d4ff0915a09d0641c82f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.879483 12521 tablet_service.cc:1511] Processing CreateTablet for tablet e1ff01ff4ede4fc990b9a07b23102b0d (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:09:16.881278 12521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1ff01ff4ede4fc990b9a07b23102b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.890748 12596 tablet_service.cc:1511] Processing CreateTablet for tablet 6530c20e384d4ff0915a09d0641c82f8 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:09:16.891242 12588 tablet_service.cc:1511] Processing CreateTablet for tablet a92d1d91cce04cec98366aa6021356df (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:09:16.892644 12596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6530c20e384d4ff0915a09d0641c82f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.908545 12590 tablet_service.cc:1511] Processing CreateTablet for tablet 5771c48bd1d24152ab96cfc24da505ed (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:09:16.910533 12512 tablet_service.cc:1511] Processing CreateTablet for tablet 6cb14d97e16d424a82a91a8ce1b6cc8d (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:09:16.910811 12662 tablet_service.cc:1511] Processing CreateTablet for tablet 6cb14d97e16d424a82a91a8ce1b6cc8d (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:09:16.911878 12512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cb14d97e16d424a82a91a8ce1b6cc8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.912180 12662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cb14d97e16d424a82a91a8ce1b6cc8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.914572 12588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a92d1d91cce04cec98366aa6021356df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.917021 12590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5771c48bd1d24152ab96cfc24da505ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.917120 12592 tablet_service.cc:1511] Processing CreateTablet for tablet e1ff01ff4ede4fc990b9a07b23102b0d (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:09:16.919160 12592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1ff01ff4ede4fc990b9a07b23102b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.927687 12593 tablet_service.cc:1511] Processing CreateTablet for tablet 6cb14d97e16d424a82a91a8ce1b6cc8d (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:09:16.929713 12593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6cb14d97e16d424a82a91a8ce1b6cc8d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.935858 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:16.939807 12668 tablet_service.cc:1511] Processing CreateTablet for tablet 43ba56f9f94346338994fdfb09969ba8 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:09:16.940474 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:16.941359 12668 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43ba56f9f94346338994fdfb09969ba8. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:09:16.942277 12496 leader_election.cc:343] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:16.942761 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:16.943667 12726 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:16.939983 12515 tablet_service.cc:1511] Processing CreateTablet for tablet 43ba56f9f94346338994fdfb09969ba8 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:09:16.945351 12515 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43ba56f9f94346338994fdfb09969ba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.963972 12722 tablet_bootstrap.cc:492] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:16.964437 12720 tablet_bootstrap.cc:492] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:16.964599 12722 ts_tablet_manager.cc:1403] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.399s	user 0.018s	sys 0.010s
I20260709 12:09:16.965060 12720 ts_tablet_manager.cc:1403] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.168s	user 0.043s	sys 0.020s
I20260709 12:09:16.973042 12720 raft_consensus.cc:359] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:16.974766 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:16.978379 12720 raft_consensus.cc:385] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:16.978303 12513 tablet_service.cc:1511] Processing CreateTablet for tablet 317be0a176ee473b9b3631bdc1030bcc (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:09:16.978864 12720 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:16.979769 12513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 317be0a176ee473b9b3631bdc1030bcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.976132 12664 tablet_service.cc:1511] Processing CreateTablet for tablet ee47f7c4414b4ae98233a31696e99cc0 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:09:16.967855 12722 raft_consensus.cc:359] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:16.975589 12572 leader_election.cc:343] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:16.976763 12516 tablet_service.cc:1511] Processing CreateTablet for tablet ee47f7c4414b4ae98233a31696e99cc0 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:09:16.980410 12720 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:16.985479 12572 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:16.986048 12664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee47f7c4414b4ae98233a31696e99cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:16.982579 12722 raft_consensus.cc:385] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:16.988929 12722 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:16.990309 12721 tablet_bootstrap.cc:492] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:16.989997 12722 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:16.990831 12721 ts_tablet_manager.cc:1403] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.222s	user 0.013s	sys 0.002s
I20260709 12:09:16.994062 12726 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:16.994652 12726 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:16.997426 12706 tablet.cc:2406] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:09:16.998100 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:16.999461 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:09:17.002341 12496 leader_election.cc:343] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:17.003232 12494 leader_election.cc:343] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.003686 12494 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:17.004958 12726 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:17.005627 12726 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:17.007741 12727 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:16.993999 12721 raft_consensus.cc:359] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.012709 12728 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:17.012889 12721 raft_consensus.cc:385] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.013407 12728 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.014745 12670 tablet_service.cc:1511] Processing CreateTablet for tablet 317be0a176ee473b9b3631bdc1030bcc (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:09:17.016141 12728 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:17.021636 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:09:17.022346 12477 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:17.009539 12722 ts_tablet_manager.cc:1434] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.045s	user 0.010s	sys 0.001s
I20260709 12:09:17.024801 12514 tablet_service.cc:1511] Processing CreateTablet for tablet 0d1b28b57b3d4a9184fbb79b00227f3a (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
W20260709 12:09:17.025765 12646 leader_election.cc:343] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.026724 12514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d1b28b57b3d4a9184fbb79b00227f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.029367 12720 ts_tablet_manager.cc:1434] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.059s	user 0.005s	sys 0.002s
I20260709 12:09:17.029275 12661 tablet_service.cc:1511] Processing CreateTablet for tablet 0d1b28b57b3d4a9184fbb79b00227f3a (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:09:17.013347 12721 raft_consensus.cc:740] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.034901 12721 consensus_queue.cc:260] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.039196 12516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee47f7c4414b4ae98233a31696e99cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.040145 12663 tablet_service.cc:1511] Processing CreateTablet for tablet ef80db1de7994402abd14cc926713fb0 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:09:17.043641 12721 ts_tablet_manager.cc:1434] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.052s	user 0.005s	sys 0.003s
I20260709 12:09:17.044962 12722 tablet_bootstrap.cc:492] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:17.016175 12670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 317be0a176ee473b9b3631bdc1030bcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.053099 12721 tablet_bootstrap.cc:492] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:17.060621 12661 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d1b28b57b3d4a9184fbb79b00227f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.060884 12521 tablet_service.cc:1511] Processing CreateTablet for tablet ef80db1de7994402abd14cc926713fb0 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:09:17.062278 12521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef80db1de7994402abd14cc926713fb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.067272 12663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef80db1de7994402abd14cc926713fb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.070180 12519 tablet_service.cc:1511] Processing CreateTablet for tablet e479222164574b9ea8944c530fbeb919 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:09:17.071491 12519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e479222164574b9ea8944c530fbeb919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.078084 12662 tablet_service.cc:1511] Processing CreateTablet for tablet e479222164574b9ea8944c530fbeb919 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
W20260709 12:09:17.081738 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 78561.9us
W20260709 12:09:17.082490 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 107074us
I20260709 12:09:17.080037 12662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e479222164574b9ea8944c530fbeb919. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:09:17.085712 12643 leader_election.cc:343] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.106619 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:17.107961 12728 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:17.087630 12517 tablet_service.cc:1511] Processing CreateTablet for tablet 3c9b5315396948b2af3fa5480c9bee87 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:09:17.094513 12727 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:17.109401 12727 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:09:17.110275 12517 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c9b5315396948b2af3fa5480c9bee87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.112201 12727 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:09:17.094760 12748 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:17.113286 12748 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.114286 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:17.117839 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:17.118746 12748 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:09:17.119937 12571 leader_election.cc:343] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.120553 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:17.121244 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:09:17.122679 12571 leader_election.cc:343] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.095964 12592 tablet_service.cc:1511] Processing CreateTablet for tablet 43ba56f9f94346338994fdfb09969ba8 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
W20260709 12:09:17.122879 12572 leader_election.cc:343] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.096135 12596 tablet_service.cc:1511] Processing CreateTablet for tablet ee47f7c4414b4ae98233a31696e99cc0 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:09:17.098210 12590 tablet_service.cc:1511] Processing CreateTablet for tablet 317be0a176ee473b9b3631bdc1030bcc (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:09:17.098488 12593 tablet_service.cc:1511] Processing CreateTablet for tablet 0d1b28b57b3d4a9184fbb79b00227f3a (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:09:17.100590 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:17.100534 12589 tablet_service.cc:1511] Processing CreateTablet for tablet ef80db1de7994402abd14cc926713fb0 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:09:17.100903 12594 tablet_service.cc:1511] Processing CreateTablet for tablet e479222164574b9ea8944c530fbeb919 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:09:17.127031 12592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43ba56f9f94346338994fdfb09969ba8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.127933 12594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e479222164574b9ea8944c530fbeb919. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.128878 12596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee47f7c4414b4ae98233a31696e99cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.103632 12720 tablet_bootstrap.cc:492] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:17.130577 12590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 317be0a176ee473b9b3631bdc1030bcc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.134526 12593 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0d1b28b57b3d4a9184fbb79b00227f3a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.138623 12589 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ef80db1de7994402abd14cc926713fb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.103940 12722 tablet_bootstrap.cc:654] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.106034 12721 tablet_bootstrap.cc:654] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.153122 12728 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:17.153640 12728 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.155829 12728 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:17.157723 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:09:17.159408 12646 leader_election.cc:343] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.162819 12747 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:17.162400 12670 tablet_service.cc:1511] Processing CreateTablet for tablet 3c9b5315396948b2af3fa5480c9bee87 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:09:17.163371 12747 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.164530 12670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c9b5315396948b2af3fa5480c9bee87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.168778 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:17.168519 12516 tablet_service.cc:1511] Processing CreateTablet for tablet 19eaf42a0f664c919842f3c2f5cb868f (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:09:17.169560 12680 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:17.170491 12747 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:17.170720 12516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19eaf42a0f664c919842f3c2f5cb868f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.171487 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:09:17.173221 12494 leader_election.cc:343] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.174611 12496 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:17.176138 12747 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:17.176584 12747 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:17.176946 12747 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:17.183019 12662 tablet_service.cc:1511] Processing CreateTablet for tablet 19eaf42a0f664c919842f3c2f5cb868f (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:09:17.186993 12662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19eaf42a0f664c919842f3c2f5cb868f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.187422 12519 tablet_service.cc:1511] Processing CreateTablet for tablet 4270d5e0462b49d2982145525764c7fa (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:09:17.188784 12519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4270d5e0462b49d2982145525764c7fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.124189 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:17.190213 12748 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:09:17.190516 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 67643.5us
W20260709 12:09:17.191601 12572 leader_election.cc:343] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.192143 12572 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:17.193018 12748 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:17.194612 12748 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:17.195101 12748 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.197342 12747 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.198580 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:17.199235 12748 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:17.202656 12664 tablet_service.cc:1511] Processing CreateTablet for tablet 4270d5e0462b49d2982145525764c7fa (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:09:17.203491 12720 tablet_bootstrap.cc:654] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.204411 12664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4270d5e0462b49d2982145525764c7fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.210395 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:17.211030 12680 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:17.211453 12643 leader_election.cc:343] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.211963 12643 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:17.213996 12728 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:17.215372 12521 tablet_service.cc:1511] Processing CreateTablet for tablet 7b825dff40dc4290a25d71534b598330 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:09:17.215591 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:17.216419 12521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b825dff40dc4290a25d71534b598330. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:09:17.216861 12571 leader_election.cc:343] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:17.218189 12494 leader_election.cc:343] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.219686 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:17.223419 12591 tablet_service.cc:1511] Processing CreateTablet for tablet 3c9b5315396948b2af3fa5480c9bee87 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:09:17.225597 12591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c9b5315396948b2af3fa5480c9bee87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.207219 12747 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:17.237969 12663 tablet_service.cc:1511] Processing CreateTablet for tablet 7b825dff40dc4290a25d71534b598330 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:09:17.240212 12663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b825dff40dc4290a25d71534b598330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.240221 12514 tablet_service.cc:1511] Processing CreateTablet for tablet 6fca0ed05cd34c8e92dc950351b864a2 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:09:17.241847 12514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fca0ed05cd34c8e92dc950351b864a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.247599 12721 tablet_bootstrap.cc:492] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:17.248339 12721 ts_tablet_manager.cc:1403] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.195s	user 0.021s	sys 0.022s
I20260709 12:09:17.251721 12721 raft_consensus.cc:359] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.252485 12721 raft_consensus.cc:385] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.252799 12721 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.254014 12721 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.256634 12721 ts_tablet_manager.cc:1434] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:09:17.267838 12594 tablet_service.cc:1511] Processing CreateTablet for tablet 4270d5e0462b49d2982145525764c7fa (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:09:17.269021 12680 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:17.269716 12594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4270d5e0462b49d2982145525764c7fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.269977 12596 tablet_service.cc:1511] Processing CreateTablet for tablet 7b825dff40dc4290a25d71534b598330 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:09:17.267961 12588 tablet_service.cc:1511] Processing CreateTablet for tablet 19eaf42a0f664c919842f3c2f5cb868f (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:09:17.271731 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:17.276736 12596 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7b825dff40dc4290a25d71534b598330. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.272053 12592 tablet_service.cc:1511] Processing CreateTablet for tablet 6fca0ed05cd34c8e92dc950351b864a2 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:09:17.281667 12592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fca0ed05cd34c8e92dc950351b864a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.282928 12496 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:09:17.273598 12572 leader_election.cc:343] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.290597 12572 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:17.291265 12747 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:09:17.292044 12643 leader_election.cc:343] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.292768 12720 tablet_bootstrap.cc:492] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:17.293236 12588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 19eaf42a0f664c919842f3c2f5cb868f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.293319 12720 ts_tablet_manager.cc:1403] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.213s	user 0.015s	sys 0.011s
I20260709 12:09:17.294275 12748 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:17.297327 12720 raft_consensus.cc:359] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.298177 12720 raft_consensus.cc:385] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.292574 12747 raft_consensus.cc:697] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:09:17.299571 12722 tablet_bootstrap.cc:492] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:17.288517 12662 tablet_service.cc:1511] Processing CreateTablet for tablet 6fca0ed05cd34c8e92dc950351b864a2 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:09:17.300105 12722 ts_tablet_manager.cc:1403] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.255s	user 0.019s	sys 0.018s
I20260709 12:09:17.299127 12747 consensus_queue.cc:237] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.292518 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:17.280719 12721 tablet_bootstrap.cc:492] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:17.301249 12728 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:17.302491 12662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fca0ed05cd34c8e92dc950351b864a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.307289 12720 raft_consensus.cc:740] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.314370 12720 consensus_queue.cc:260] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.319561 12720 ts_tablet_manager.cc:1434] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.002s
I20260709 12:09:17.321056 12722 raft_consensus.cc:359] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.321879 12722 raft_consensus.cc:385] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.322371 12722 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.323138 12722 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.326005 12722 ts_tablet_manager.cc:1434] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.026s	user 0.004s	sys 0.004s
I20260709 12:09:17.327173 12722 tablet_bootstrap.cc:492] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:17.327435 12726 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:17.328241 12721 tablet_bootstrap.cc:654] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.327935 12726 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.336659 12726 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:17.337404 12590 tablet_service.cc:1511] Processing CreateTablet for tablet 5dd1b2c01b5e495f9f31aee68f93d394 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:09:17.338335 12521 tablet_service.cc:1511] Processing CreateTablet for tablet 5dd1b2c01b5e495f9f31aee68f93d394 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:09:17.338562 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:17.340041 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:17.339458 12663 tablet_service.cc:1511] Processing CreateTablet for tablet 5dd1b2c01b5e495f9f31aee68f93d394 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
W20260709 12:09:17.346503 12494 leader_election.cc:343] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.333855 12722 tablet_bootstrap.cc:654] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
W20260709 12:09:17.347273 12496 leader_election.cc:343] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.347095 12521 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dd1b2c01b5e495f9f31aee68f93d394. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.347997 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:17.349635 12726 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:17.347167 12663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dd1b2c01b5e495f9f31aee68f93d394. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.338917 12590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dd1b2c01b5e495f9f31aee68f93d394. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.354780 12720 tablet_bootstrap.cc:492] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:17.361728 12514 tablet_service.cc:1511] Processing CreateTablet for tablet 47b417a13f2d44ef8922eb717c8ab20d (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:09:17.363269 12514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47b417a13f2d44ef8922eb717c8ab20d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.377779 12721 tablet_bootstrap.cc:492] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:17.378477 12721 ts_tablet_manager.cc:1403] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.098s	user 0.012s	sys 0.010s
I20260709 12:09:17.383705 12721 raft_consensus.cc:359] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.384153 12670 tablet_service.cc:1511] Processing CreateTablet for tablet 47b417a13f2d44ef8922eb717c8ab20d (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:09:17.384967 12721 raft_consensus.cc:385] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.384714 12519 tablet_service.cc:1511] Processing CreateTablet for tablet 111842005e864eab9966b3a00ab9ae8a (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:09:17.385386 12721 raft_consensus.cc:740] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.385972 12670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47b417a13f2d44ef8922eb717c8ab20d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.386329 12519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 111842005e864eab9966b3a00ab9ae8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.386480 12721 consensus_queue.cc:260] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.373417 12427 catalog_manager.cc:5697] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:17.392952 12590 tablet_service.cc:1511] Processing CreateTablet for tablet 47b417a13f2d44ef8922eb717c8ab20d (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:09:17.394738 12590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 47b417a13f2d44ef8922eb717c8ab20d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.402087 12722 tablet_bootstrap.cc:492] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:17.402765 12722 ts_tablet_manager.cc:1403] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.076s	user 0.012s	sys 0.002s
I20260709 12:09:17.405081 12720 tablet_bootstrap.cc:654] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.405999 12722 raft_consensus.cc:359] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.410241 12588 tablet_service.cc:1511] Processing CreateTablet for tablet 111842005e864eab9966b3a00ab9ae8a (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:09:17.406926 12722 raft_consensus.cc:385] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.414211 12588 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 111842005e864eab9966b3a00ab9ae8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.419577 12721 ts_tablet_manager.cc:1434] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.001s
I20260709 12:09:17.419883 12594 tablet_service.cc:1511] Processing CreateTablet for tablet 33b7d46307bc4c33acd6c9a58055c7af (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:09:17.420457 12662 tablet_service.cc:1511] Processing CreateTablet for tablet 111842005e864eab9966b3a00ab9ae8a (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:09:17.421730 12662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 111842005e864eab9966b3a00ab9ae8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.422062 12514 tablet_service.cc:1511] Processing CreateTablet for tablet 33b7d46307bc4c33acd6c9a58055c7af (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:09:17.414597 12722 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.423442 12721 tablet_bootstrap.cc:492] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:17.424183 12594 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33b7d46307bc4c33acd6c9a58055c7af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.424130 12722 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.423442 12514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33b7d46307bc4c33acd6c9a58055c7af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.430522 12721 tablet_bootstrap.cc:654] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.437404 12726 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:17.438197 12726 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.442360 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:17.443550 12606 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:17.445288 12664 tablet_service.cc:1511] Processing CreateTablet for tablet 33b7d46307bc4c33acd6c9a58055c7af (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:09:17.450989 12664 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33b7d46307bc4c33acd6c9a58055c7af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.455680 12494 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:17.457069 12747 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:17.457616 12747 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:17.458055 12747 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:17.458775 12722 ts_tablet_manager.cc:1434] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.002s
I20260709 12:09:17.460315 12722 tablet_bootstrap.cc:492] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:17.473973 12726 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:17.476742 12591 tablet_service.cc:1511] Processing CreateTablet for tablet 9b55aace8c0343b9808746fb1275f5cc (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:09:17.477102 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:17.477757 12680 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:17.478083 12591 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b55aace8c0343b9808746fb1275f5cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.476760 12516 tablet_service.cc:1511] Processing CreateTablet for tablet 9b55aace8c0343b9808746fb1275f5cc (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:09:17.483466 12516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b55aace8c0343b9808746fb1275f5cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.500571 12590 tablet_service.cc:1511] Processing CreateTablet for tablet 3c00651839ae4132bb331bc6c2602641 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:09:17.501899 12590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c00651839ae4132bb331bc6c2602641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.501747 12663 tablet_service.cc:1511] Processing CreateTablet for tablet 3c00651839ae4132bb331bc6c2602641 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:09:17.503126 12663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c00651839ae4132bb331bc6c2602641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.504011 12721 tablet_bootstrap.cc:492] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:17.504532 12721 ts_tablet_manager.cc:1403] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.081s	user 0.014s	sys 0.034s
I20260709 12:09:17.505870 12519 tablet_service.cc:1511] Processing CreateTablet for tablet 3c00651839ae4132bb331bc6c2602641 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:09:17.507421 12519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c00651839ae4132bb331bc6c2602641. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.505861 12670 tablet_service.cc:1511] Processing CreateTablet for tablet 9b55aace8c0343b9808746fb1275f5cc (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:09:17.517179 12721 raft_consensus.cc:359] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.519479 12670 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9b55aace8c0343b9808746fb1275f5cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.509936 12747 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.521943 12721 raft_consensus.cc:385] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.523996 12721 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.525017 12721 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.527818 12721 ts_tablet_manager.cc:1434] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.002s
I20260709 12:09:17.529162 12720 tablet_bootstrap.cc:492] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:17.529814 12720 ts_tablet_manager.cc:1403] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.175s	user 0.008s	sys 0.009s
I20260709 12:09:17.535516 12516 tablet_service.cc:1511] Processing CreateTablet for tablet 3a68955866d543408acca9e959ad1b26 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:09:17.539021 12516 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a68955866d543408acca9e959ad1b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.538829 12592 tablet_service.cc:1511] Processing CreateTablet for tablet 3a68955866d543408acca9e959ad1b26 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:09:17.540200 12592 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a68955866d543408acca9e959ad1b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.542790 12720 raft_consensus.cc:359] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.551187 12722 tablet_bootstrap.cc:654] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.556416 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:17.557072 12606 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:17.557518 12720 raft_consensus.cc:385] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.558672 12720 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.552963 12721 tablet_bootstrap.cc:492] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:17.564138 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:17.564849 12680 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:17.565294 12720 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.577639 12747 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:17.568423 12720 ts_tablet_manager.cc:1434] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 12:09:17.581483 12720 tablet_bootstrap.cc:492] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:17.583019 12606 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:17.583771 12721 tablet_bootstrap.cc:654] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.584338 12494 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:17.585268 12747 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:17.586498 12722 tablet_bootstrap.cc:492] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:17.587014 12722 ts_tablet_manager.cc:1403] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.127s	user 0.012s	sys 0.002s
I20260709 12:09:17.589520 12663 tablet_service.cc:1511] Processing CreateTablet for tablet 3a68955866d543408acca9e959ad1b26 (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:09:17.591058 12663 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3a68955866d543408acca9e959ad1b26. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.591495 12722 raft_consensus.cc:359] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.592269 12722 raft_consensus.cc:385] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.592608 12722 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.593369 12722 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.594192 12747 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:09:17.595170 12747 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.598047 12519 tablet_service.cc:1511] Processing CreateTablet for tablet bcef55b2ef044bcb9d9a9ffd855c20ed (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:09:17.599465 12519 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcef55b2ef044bcb9d9a9ffd855c20ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.599709 12662 tablet_service.cc:1511] Processing CreateTablet for tablet bcef55b2ef044bcb9d9a9ffd855c20ed (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:09:17.600855 12662 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcef55b2ef044bcb9d9a9ffd855c20ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.598407 12590 tablet_service.cc:1511] Processing CreateTablet for tablet bcef55b2ef044bcb9d9a9ffd855c20ed (DEFAULT_TABLE table=test-workload [id=8634097bac3246c9a57e272441058c28]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:09:17.607779 12590 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcef55b2ef044bcb9d9a9ffd855c20ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:17.608304 12722 ts_tablet_manager.cc:1434] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 12:09:17.609802 12722 tablet_bootstrap.cc:492] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:17.615221 12748 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:17.618489 12748 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.620823 12748 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:17.622674 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:17.623836 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:09:17.624346 12571 leader_election.cc:343] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.624647 12531 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:17.617041 12680 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:17.626313 12572 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:09:17.627592 12748 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:17.627992 12748 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:17.628314 12748 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:17.633677 12722 tablet_bootstrap.cc:654] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.637805 12748 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.641323 12720 tablet_bootstrap.cc:654] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.641489 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:17.645308 12531 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:17.654479 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:09:17.656867 12571 leader_election.cc:343] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.657905 12748 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:17.660461 12531 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:17.662413 12572 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:09:17.664181 12748 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:17.669153 12748 raft_consensus.cc:697] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:09:17.670080 12748 consensus_queue.cc:237] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.676685 12722 tablet_bootstrap.cc:492] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:17.677435 12722 ts_tablet_manager.cc:1403] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.068s	user 0.030s	sys 0.009s
I20260709 12:09:17.676631 12427 catalog_manager.cc:5697] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:17.679000 12721 tablet_bootstrap.cc:492] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:17.679507 12721 ts_tablet_manager.cc:1403] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.127s	user 0.008s	sys 0.011s
I20260709 12:09:17.681679 12720 tablet_bootstrap.cc:492] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:17.682018 12720 ts_tablet_manager.cc:1403] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.101s	user 0.018s	sys 0.007s
I20260709 12:09:17.683197 12722 raft_consensus.cc:359] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.682592 12721 raft_consensus.cc:359] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.684413 12721 raft_consensus.cc:385] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.684782 12721 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.685671 12721 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.689184 12721 ts_tablet_manager.cc:1434] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:09:17.683899 12722 raft_consensus.cc:385] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.689612 12722 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.686745 12720 raft_consensus.cc:359] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.698807 12720 raft_consensus.cc:385] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.698587 12722 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.698633 12721 tablet_bootstrap.cc:492] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:17.701251 12722 ts_tablet_manager.cc:1434] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.023s	user 0.002s	sys 0.011s
I20260709 12:09:17.699872 12426 catalog_manager.cc:5697] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 0 to 1, leader changed from <none> to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:17.699222 12720 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.706208 12720 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.708792 12722 tablet_bootstrap.cc:492] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:17.710798 12721 tablet_bootstrap.cc:654] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.708854 12720 ts_tablet_manager.cc:1434] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.027s	user 0.004s	sys 0.001s
I20260709 12:09:17.715193 12720 tablet_bootstrap.cc:492] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:17.716264 12721 tablet_bootstrap.cc:492] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:17.716950 12721 ts_tablet_manager.cc:1403] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.019s	user 0.007s	sys 0.006s
I20260709 12:09:17.716276 12722 tablet_bootstrap.cc:654] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.719758 12721 raft_consensus.cc:359] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:09:17.720541 12721 raft_consensus.cc:385] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.720878 12721 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.721442 12720 tablet_bootstrap.cc:654] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.721788 12721 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:09:17.729944 12721 ts_tablet_manager.cc:1434] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.000s
I20260709 12:09:17.731400 12721 tablet_bootstrap.cc:492] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:17.740159 12721 tablet_bootstrap.cc:654] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.752841 12720 tablet_bootstrap.cc:492] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:17.753363 12720 ts_tablet_manager.cc:1403] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.038s	user 0.023s	sys 0.010s
I20260709 12:09:17.753832 12722 tablet_bootstrap.cc:492] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:17.754247 12721 tablet_bootstrap.cc:492] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:17.754367 12722 ts_tablet_manager.cc:1403] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.046s	user 0.008s	sys 0.035s
I20260709 12:09:17.754662 12721 ts_tablet_manager.cc:1403] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.024s	user 0.017s	sys 0.004s
I20260709 12:09:17.755952 12720 raft_consensus.cc:359] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.756532 12720 raft_consensus.cc:385] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.756763 12720 raft_consensus.cc:740] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.757411 12720 consensus_queue.cc:260] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.757364 12722 raft_consensus.cc:359] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.757536 12721 raft_consensus.cc:359] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.758261 12722 raft_consensus.cc:385] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.759268 12720 ts_tablet_manager.cc:1434] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.006s	user 0.007s	sys 0.000s
I20260709 12:09:17.762569 12722 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.758289 12721 raft_consensus.cc:385] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.763113 12721 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.763470 12720 tablet_bootstrap.cc:492] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:17.763450 12722 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.763954 12721 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.766036 12722 ts_tablet_manager.cc:1434] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.011s	user 0.005s	sys 0.003s
I20260709 12:09:17.768152 12721 ts_tablet_manager.cc:1434] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 12:09:17.768507 12722 tablet_bootstrap.cc:492] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:17.769374 12721 tablet_bootstrap.cc:492] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:17.773494 12720 tablet_bootstrap.cc:654] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.775203 12722 tablet_bootstrap.cc:654] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.775580 12721 tablet_bootstrap.cc:654] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.813390 12728 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:17.814194 12728 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.817955 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:17.818324 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:17.819691 12720 tablet_bootstrap.cc:492] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:17.817649 12728 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:09:17.819584 12646 leader_election.cc:343] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.820233 12720 ts_tablet_manager.cc:1403] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.057s	user 0.009s	sys 0.039s
W20260709 12:09:17.820287 12643 leader_election.cc:343] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.820967 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:17.821763 12728 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:17.822171 12721 tablet_bootstrap.cc:492] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:17.822672 12721 ts_tablet_manager.cc:1403] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.053s	user 0.038s	sys 0.011s
I20260709 12:09:17.823352 12722 tablet_bootstrap.cc:492] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:17.823308 12720 raft_consensus.cc:359] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.823922 12722 ts_tablet_manager.cc:1403] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.056s	user 0.033s	sys 0.013s
I20260709 12:09:17.824203 12720 raft_consensus.cc:385] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.824515 12720 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.825121 12721 raft_consensus.cc:359] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.825253 12720 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.825845 12721 raft_consensus.cc:385] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.826493 12721 raft_consensus.cc:740] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.827366 12722 raft_consensus.cc:359] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.827828 12720 ts_tablet_manager.cc:1434] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:09:17.832247 12722 raft_consensus.cc:385] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.831836 12747 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:17.827550 12721 consensus_queue.cc:260] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.832547 12722 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.833216 12720 tablet_bootstrap.cc:492] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:17.833216 12722 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.835002 12606 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:17.838428 12722 ts_tablet_manager.cc:1434] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.005s
I20260709 12:09:17.839409 12722 tablet_bootstrap.cc:492] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:17.840022 12720 tablet_bootstrap.cc:654] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.844877 12721 ts_tablet_manager.cc:1434] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.000s
I20260709 12:09:17.846161 12721 tablet_bootstrap.cc:492] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:17.847436 12722 tablet_bootstrap.cc:654] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.852995 12720 tablet_bootstrap.cc:492] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:17.853843 12720 ts_tablet_manager.cc:1403] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.003s
I20260709 12:09:17.857921 12720 raft_consensus.cc:359] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.859261 12720 raft_consensus.cc:385] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.859730 12720 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.860379 12721 tablet_bootstrap.cc:654] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.860980 12720 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.864589 12720 ts_tablet_manager.cc:1434] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:09:17.866653 12720 tablet_bootstrap.cc:492] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:17.868832 12728 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:17.869580 12728 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.870536 12722 tablet_bootstrap.cc:492] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:17.871256 12722 ts_tablet_manager.cc:1403] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.032s	user 0.011s	sys 0.019s
I20260709 12:09:17.872443 12728 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:17.873034 12728 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:17.873508 12728 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.875731 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:17.883431 12605 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:17.875537 12722 raft_consensus.cc:359] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.884083 12722 raft_consensus.cc:385] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.884420 12722 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.875638 12747 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:17.885139 12722 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.885370 12747 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.886493 12728 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:17.887444 12722 ts_tablet_manager.cc:1434] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.000s
I20260709 12:09:17.880756 12720 tablet_bootstrap.cc:654] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.888595 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:09:17.890688 12646 leader_election.cc:343] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.894604 12722 tablet_bootstrap.cc:492] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:17.897492 12721 tablet_bootstrap.cc:492] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:17.898300 12721 ts_tablet_manager.cc:1403] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.052s	user 0.005s	sys 0.013s
I20260709 12:09:17.900965 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:17.901475 12722 tablet_bootstrap.cc:654] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.901633 12721 raft_consensus.cc:359] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.902631 12721 raft_consensus.cc:385] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.902717 12726 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:17.903040 12721 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.903923 12721 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.906324 12721 ts_tablet_manager.cc:1434] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:09:17.907471 12721 tablet_bootstrap.cc:492] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:17.902022 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:17.909590 12531 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:17.911679 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:17.901684 12606 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:17.915066 12494 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:17.916531 12721 tablet_bootstrap.cc:654] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.917044 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:17.918287 12605 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:17.918749 12720 tablet_bootstrap.cc:492] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:17.919325 12720 ts_tablet_manager.cc:1403] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.053s	user 0.009s	sys 0.009s
I20260709 12:09:17.920883 12747 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:17.921680 12747 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:17.922188 12747 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:17.922703 12747 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:17.923033 12720 raft_consensus.cc:359] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.924167 12720 raft_consensus.cc:385] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.924569 12720 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.927639 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:17.925601 12720 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:17.929580 12496 leader_election.cc:343] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.930032 12722 tablet_bootstrap.cc:492] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:17.930583 12722 ts_tablet_manager.cc:1403] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.036s	user 0.010s	sys 0.023s
I20260709 12:09:17.930640 12720 ts_tablet_manager.cc:1434] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:09:17.931335 12728 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:17.931739 12728 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:17.932070 12728 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:17.933362 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:17.933460 12722 raft_consensus.cc:359] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.934172 12722 raft_consensus.cc:385] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.934513 12722 raft_consensus.cc:740] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.935284 12722 consensus_queue.cc:260] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.937321 12722 ts_tablet_manager.cc:1434] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.006s	user 0.006s	sys 0.000s
I20260709 12:09:17.939314 12787 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:17.939754 12787 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:17.940172 12721 tablet_bootstrap.cc:492] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:17.940527 12722 tablet_bootstrap.cc:492] T e15d21db797e4a96b5732711a5379980 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:17.940754 12721 ts_tablet_manager.cc:1403] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.009s
I20260709 12:09:17.943104 12748 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:17.943904 12748 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.943681 12721 raft_consensus.cc:359] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.940085 12787 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:17.946403 12721 raft_consensus.cc:385] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.947093 12721 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:17.947456 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:17.948232 12531 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:17.948899 12721 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.949644 12572 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:17.951473 12766 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:17.951887 12766 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:17.952427 12766 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:17.954551 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:17.955183 12679 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:17.955969 12722 tablet_bootstrap.cc:654] T e15d21db797e4a96b5732711a5379980 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.956213 12726 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:17.956684 12726 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.958923 12726 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:17.959333 12720 tablet_bootstrap.cc:492] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:17.959151 12787 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.961238 12787 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:17.962325 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:17.962915 12605 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:17.966256 12722 tablet_bootstrap.cc:492] T e15d21db797e4a96b5732711a5379980 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:17.966724 12722 ts_tablet_manager.cc:1403] T e15d21db797e4a96b5732711a5379980 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.029s	user 0.013s	sys 0.000s
I20260709 12:09:17.966713 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:17.969604 12722 raft_consensus.cc:359] T e15d21db797e4a96b5732711a5379980 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.970366 12722 raft_consensus.cc:385] T e15d21db797e4a96b5732711a5379980 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:17.970656 12722 raft_consensus.cc:740] T e15d21db797e4a96b5732711a5379980 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
W20260709 12:09:17.970764 12494 leader_election.cc:343] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:17.971328 12722 consensus_queue.cc:260] T e15d21db797e4a96b5732711a5379980 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.973461 12722 ts_tablet_manager.cc:1434] T e15d21db797e4a96b5732711a5379980 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:09:17.974388 12722 tablet_bootstrap.cc:492] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:17.962364 12728 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.978701 12748 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:17.978845 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:17.979516 12530 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:17.980810 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:17.981487 12680 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:17.981848 12722 tablet_bootstrap.cc:654] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:17.983633 12496 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:17.985376 12726 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:17.985909 12726 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:17.986415 12726 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:17.990670 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:17.883628 12779 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:17.992659 12531 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:17.997370 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:17.998054 12606 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.002084 12766 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:17.989770 12530 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:18.008169 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:18.009109 12529 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:17.993057 12779 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.013566 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:18.004189 12728 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:18.014539 12528 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:18.016234 12646 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:18.018060 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:18.020228 12789 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:18.020808 12789 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:18.021301 12789 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.027189 12747 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.027693 12606 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:18.036845 12720 tablet_bootstrap.cc:654] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:18.038352 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:18.039099 12603 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:18.022428 12728 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:18.040143 12728 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:18.041085 12728 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.042440 12766 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:18.043177 12781 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:18.043678 12781 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.048753 12605 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:18.049886 12781 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:18.051553 12787 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:18.051893 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:18.052165 12766 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:18.052536 12680 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.052989 12766 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:18.054270 12477 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:18.052112 12787 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:09:18.056444 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:18.057178 12679 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:18.057437 12529 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:18.057483 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:18.036162 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:18.062762 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:18.056447 12787 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:09:18.065117 12721 ts_tablet_manager.cc:1434] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.124s	user 0.053s	sys 0.045s
I20260709 12:09:18.065706 12787 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:18.066277 12721 tablet_bootstrap.cc:492] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:18.066190 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:18.066334 12787 raft_consensus.cc:697] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:18.066946 12530 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:18.064492 12496 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:18.063304 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:18.044938 12779 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:18.060328 12766 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:18.064670 12606 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.065037 12572 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:18.072901 12781 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:18.075757 12531 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:18.081303 12781 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:18.085029 12726 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.084749 12781 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.087594 12766 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:18.090835 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:18.092089 12789 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.065105 12605 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
W20260709 12:09:18.093161 12496 leader_election.cc:343] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:18.093335 12679 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 1: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:09:18.093379 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:18.094357 12603 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:18.082173 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:18.096760 12722 tablet_bootstrap.cc:492] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:18.089915 12680 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:18.100198 12722 ts_tablet_manager.cc:1403] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.126s	user 0.011s	sys 0.014s
I20260709 12:09:18.086292 12747 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:18.096529 12766 raft_consensus.cc:697] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:09:18.084462 12572 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:18.100821 12747 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:18.104806 12747 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.115553 12748 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:18.116698 12748 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:18.119936 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:18.120452 12748 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.120859 12680 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.120602 12766 consensus_queue.cc:237] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.116593 12643 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:18.123219 12728 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:18.123500 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:18.123785 12728 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:18.116683 12428 catalog_manager.cc:5697] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:18.124848 12679 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:18.126451 12496 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:18.127106 12606 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:18.107677 12787 consensus_queue.cc:237] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.117435 12726 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:18.128657 12726 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:18.129148 12726 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:18.129573 12726 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.130270 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:18.130926 12531 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.134289 12789 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:18.124759 12728 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.128057 12747 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:18.154093 12680 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:18.155637 12496 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:18.156927 12747 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:18.161990 12781 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.137538 12748 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.171298 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:09:18.171741 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 44863.2us
I20260709 12:09:18.204922 12748 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:18.210381 12748 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:09:18.215101 12722 raft_consensus.cc:359] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:09:18.217221 12722 raft_consensus.cc:385] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:18.217518 12722 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:18.219180 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:18.220062 12722 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:09:18.223426 12793 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:09:18.223945 12494 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:09:18.225756 12720 tablet_bootstrap.cc:492] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:18.226490 12720 ts_tablet_manager.cc:1403] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.267s	user 0.015s	sys 0.020s
I20260709 12:09:18.227460 12529 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:18.227494 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:18.228602 12680 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 1: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:09:18.229869 12722 ts_tablet_manager.cc:1434] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.129s	user 0.005s	sys 0.003s
I20260709 12:09:18.230908 12722 tablet_bootstrap.cc:492] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:18.231046 12721 tablet_bootstrap.cc:654] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:18.227571 12793 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:18.230667 12720 raft_consensus.cc:359] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.232344 12720 raft_consensus.cc:385] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:18.232813 12720 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:18.233940 12720 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.231830 12793 raft_consensus.cc:697] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:09:18.215497 12747 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:09:18.236258 12789 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:18.239882 12793 consensus_queue.cc:237] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.240239 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:18.242720 12606 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 1: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:09:18.253782 12720 ts_tablet_manager.cc:1434] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 12:09:18.255245 12720 tablet_bootstrap.cc:492] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:18.241039 12726 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.252646 12789 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.265242 12781 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:09:18.272053 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 48197.5us
I20260709 12:09:18.278048 12572 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:18.280375 12427 catalog_manager.cc:5697] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 0 to 1, leader changed from <none> to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:18.281327 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:18.281837 12748 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:09:18.275151 12747 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.271812 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:18.276506 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:09:18.276468 12494 leader_election.cc:343] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:18.297362 12531 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:18.303512 12679 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.308568 12680 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.308832 12726 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:18.311565 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:18.312233 12678 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:18.340879 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 64437.9us
I20260709 12:09:18.284905 12789 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:18.341415 12728 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:09:18.349737 12728 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:09:18.351859 12766 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:18.353140 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:18.354288 12646 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:18.354687 12722 tablet_bootstrap.cc:654] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:18.355319 12789 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:18.358433 12789 raft_consensus.cc:697] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:18.359354 12789 consensus_queue.cc:237] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.363335 12747 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:09:18.365387 12494 leader_election.cc:343] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:18.365949 12747 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.368678 12747 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:18.368546 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:18.370041 12529 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.374609 12643 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:18.375264 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:18.376091 12603 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.376601 12720 tablet_bootstrap.cc:654] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:18.372545 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:18.377311 12606 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.381562 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:18.392970 12678 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:18.401630 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:18.403342 12605 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:18.405055 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:09:18.407218 12496 leader_election.cc:343] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:18.413378 12679 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:18.432565 12720 tablet_bootstrap.cc:492] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:18.447121 12429 catalog_manager.cc:5697] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:18.495306 12529 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:18.505365 12646 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:18.506567 12728 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:18.507197 12728 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:18.507484 12787 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:18.508289 12728 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:09:18.455929 12496 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:09:18.511600 12494 leader_election.cc:343] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:18.512787 12766 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:18.513137 12494 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:09:18.530190 12795 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:18.530988 12795 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.517649 12603 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:18.534899 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:09:18.537283 12572 leader_election.cc:343] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:18.538439 12427 catalog_manager.cc:5697] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:18.548399 12720 ts_tablet_manager.cc:1403] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.293s	user 0.030s	sys 0.010s
I20260709 12:09:18.550994 12795 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:18.551220 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:18.545806 12606 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
W20260709 12:09:18.552747 12571 leader_election.cc:343] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:18.552541 12720 raft_consensus.cc:359] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.553194 12571 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:18.553540 12721 tablet_bootstrap.cc:492] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:18.556668 12721 ts_tablet_manager.cc:1403] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.491s	user 0.034s	sys 0.065s
I20260709 12:09:18.557266 12722 tablet_bootstrap.cc:492] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:18.557791 12722 ts_tablet_manager.cc:1403] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.327s	user 0.014s	sys 0.009s
I20260709 12:09:18.553588 12720 raft_consensus.cc:385] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:18.553984 12795 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:18.554466 12747 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:09:18.561789 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 48752.1us
I20260709 12:09:18.560658 12720 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:18.563844 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:18.565052 12726 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:18.565474 12726 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:18.565645 12781 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:18.565861 12726 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.566248 12781 raft_consensus.cc:697] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:09:18.567191 12781 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.568626 12796 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:18.569154 12796 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:09:18.570014 12796 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.571215 12722 raft_consensus.cc:359] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.572335 12722 raft_consensus.cc:385] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:18.572736 12722 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:18.573611 12722 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:18.575609 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 138315us
I20260709 12:09:18.576093 12726 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.578598 12722 ts_tablet_manager.cc:1434] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.000s
I20260709 12:09:18.581686 12722 tablet_bootstrap.cc:492] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:18.579695 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:18.583190 12726 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:18.583709 12604 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.587498 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:18.540984 12427 catalog_manager.cc:5697] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:18.581876 12721 raft_consensus.cc:359] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.592211 12721 raft_consensus.cc:385] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:18.592631 12721 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:18.579499 12720 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.593156 12604 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:18.594762 12494 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:18.595675 12796 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:18.596347 12720 ts_tablet_manager.cc:1434] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 12:09:18.596285 12796 raft_consensus.cc:697] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:09:18.597882 12720 tablet_bootstrap.cc:492] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:18.597748 12796 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.595536 12721 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.598845 12680 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.602828 12748 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:18.613269 12789 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:18.625676 12680 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:18.630553 12722 tablet_bootstrap.cc:654] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:18.632431 12721 ts_tablet_manager.cc:1434] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.075s	user 0.009s	sys 0.000s
I20260709 12:09:18.635253 12721 tablet_bootstrap.cc:492] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:18.641420 12728 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:18.645460 12720 tablet_bootstrap.cc:654] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:18.651222 12426 catalog_manager.cc:5697] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
W20260709 12:09:18.653985 12496 consensus_peers.cc:597] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 -> Peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Couldn't send request to peer 193a40da6e364ead8ab561e14c3cc8d1. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:18.653431 12426 catalog_manager.cc:5697] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:18.658470 12722 tablet_bootstrap.cc:492] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:18.659379 12722 ts_tablet_manager.cc:1403] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.078s	user 0.011s	sys 0.006s
I20260709 12:09:18.669684 12722 raft_consensus.cc:359] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.670542 12722 raft_consensus.cc:385] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:18.670846 12722 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:18.672024 12722 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.674299 12722 ts_tablet_manager.cc:1434] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.000s
I20260709 12:09:18.675490 12722 tablet_bootstrap.cc:492] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:18.682415 12721 tablet_bootstrap.cc:654] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:18.690207 12722 tablet_bootstrap.cc:654] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:18.692970 12720 tablet_bootstrap.cc:492] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:18.693611 12720 ts_tablet_manager.cc:1403] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.096s	user 0.009s	sys 0.013s
I20260709 12:09:18.696152 12720 raft_consensus.cc:359] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.696967 12720 raft_consensus.cc:385] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:18.697284 12720 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:18.699970 12720 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.701229 12722 tablet_bootstrap.cc:492] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:18.701710 12720 ts_tablet_manager.cc:1434] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.005s
I20260709 12:09:18.701717 12722 ts_tablet_manager.cc:1403] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.027s	user 0.001s	sys 0.013s
I20260709 12:09:18.702873 12720 tablet_bootstrap.cc:492] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:18.703814 12721 tablet_bootstrap.cc:492] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:18.704280 12721 ts_tablet_manager.cc:1403] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.069s	user 0.005s	sys 0.008s
I20260709 12:09:18.705159 12722 raft_consensus.cc:359] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.705960 12722 raft_consensus.cc:385] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:18.706383 12722 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:18.707268 12728 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:18.707558 12722 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.710111 12722 ts_tablet_manager.cc:1434] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:09:18.710530 12720 tablet_bootstrap.cc:654] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:18.709540 12796 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:18.711473 12722 tablet_bootstrap.cc:492] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:18.715314 12721 raft_consensus.cc:359] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.716122 12721 raft_consensus.cc:385] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:18.718040 12781 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:18.718601 12781 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.720631 12722 tablet_bootstrap.cc:654] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:18.720952 12781 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:18.724625 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:09:18.726459 12496 leader_election.cc:343] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:18.729108 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:18.729974 12606 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:18.731703 12494 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:09:18.733070 12796 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:18.734705 12796 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:18.735074 12796 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:18.735437 12796 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.740692 12816 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:18.741300 12816 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.716432 12721 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:18.743156 12721 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.745116 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:18.745510 12721 ts_tablet_manager.cc:1434] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.041s	user 0.004s	sys 0.003s
I20260709 12:09:18.746001 12678 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 1: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:09:18.746894 12816 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:18.752898 12721 tablet_bootstrap.cc:492] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:18.753249 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:18.755021 12728 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:18.755517 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:18.757066 12816 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.769176 12796 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.783039 12721 tablet_bootstrap.cc:654] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:18.791749 12796 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:18.794829 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:18.797037 12722 tablet_bootstrap.cc:492] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:18.796160 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:18.797565 12722 ts_tablet_manager.cc:1403] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.086s	user 0.025s	sys 0.031s
W20260709 12:09:18.798625 12496 leader_election.cc:343] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:18.798861 12779 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:18.799482 12604 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:18.818653 12494 consensus_peers.cc:597] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:18.818914 12766 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:18.819664 12766 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.823700 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:18.824615 12529 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:18.826665 12572 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:18.827852 12748 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:18.828332 12748 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:18.828755 12748 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.829193 12722 raft_consensus.cc:359] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.829906 12722 raft_consensus.cc:385] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:18.830255 12722 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:18.831321 12766 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:18.831183 12722 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.834975 12604 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:18.837467 12494 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:09:18.833243 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:18.840246 12427 catalog_manager.cc:5697] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:18.842339 12722 ts_tablet_manager.cc:1434] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.044s	user 0.005s	sys 0.004s
I20260709 12:09:18.843879 12722 tablet_bootstrap.cc:492] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:18.844782 12796 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:18.843829 12427 catalog_manager.cc:5697] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:18.849357 12427 catalog_manager.cc:5697] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
W20260709 12:09:18.851063 12571 leader_election.cc:343] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:18.851604 12720 tablet_bootstrap.cc:492] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:18.852035 12720 ts_tablet_manager.cc:1403] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.149s	user 0.009s	sys 0.034s
I20260709 12:09:18.844532 12816 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:18.853802 12799 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:18.860066 12799 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.850355 12796 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:09:18.863528 12796 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.864025 12720 raft_consensus.cc:359] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.864823 12720 raft_consensus.cc:385] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:18.865258 12720 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:18.867918 12720 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.868232 12779 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:18.868913 12722 tablet_bootstrap.cc:654] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:18.871423 12799 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:18.872660 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:09:18.875799 12494 leader_election.cc:343] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:18.874001 12720 ts_tablet_manager.cc:1434] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.002s
I20260709 12:09:18.877353 12720 tablet_bootstrap.cc:492] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:18.878912 12721 tablet_bootstrap.cc:492] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:18.877916 12748 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.879415 12721 ts_tablet_manager.cc:1403] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.127s	user 0.021s	sys 0.022s
I20260709 12:09:18.881044 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:09:18.890292 12496 leader_election.cc:343] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:18.890957 12496 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:18.892442 12799 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:18.896615 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:18.897521 12529 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.899952 12779 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:18.900378 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:18.900502 12779 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.905525 12779 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:09:18.904264 12571 leader_election.cc:343] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:18.907400 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:18.908211 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:18.903100 12748 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:18.913882 12646 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:18.915622 12720 tablet_bootstrap.cc:654] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:18.915818 12728 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:18.927003 12529 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:18.928784 12572 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:09:18.930114 12748 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:18.930322 12721 raft_consensus.cc:359] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.930722 12748 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:09:18.931102 12721 raft_consensus.cc:385] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:18.931403 12721 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:18.931855 12748 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.932135 12721 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.935896 12721 ts_tablet_manager.cc:1434] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.051s	user 0.007s	sys 0.000s
I20260709 12:09:18.938189 12721 tablet_bootstrap.cc:492] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:18.958379 12816 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:18.957644 12429 catalog_manager.cc:5697] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:18.962571 12796 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:18.965029 12722 tablet_bootstrap.cc:492] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:18.965567 12722 ts_tablet_manager.cc:1403] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.122s	user 0.025s	sys 0.010s
I20260709 12:09:18.960821 12816 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.968876 12722 raft_consensus.cc:359] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.966076 12789 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:18.969638 12722 raft_consensus.cc:385] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:18.969452 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:18.970199 12722 raft_consensus.cc:740] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
W20260709 12:09:18.971343 12572 leader_election.cc:343] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:18.970945 12722 consensus_queue.cc:260] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:18.969749 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:18.975893 12680 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:18.976869 12722 ts_tablet_manager.cc:1434] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.011s	user 0.000s	sys 0.008s
I20260709 12:09:18.977056 12816 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:18.977895 12722 tablet_bootstrap.cc:492] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:18.979228 12796 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:18.980890 12571 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:18.982501 12748 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:18.982919 12748 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:18.983300 12748 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:18.995471 12427 catalog_manager.cc:5697] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 0 to 1, leader changed from <none> to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:18.997782 12720 tablet_bootstrap.cc:492] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:18.998327 12720 ts_tablet_manager.cc:1403] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.121s	user 0.016s	sys 0.003s
I20260709 12:09:19.009084 12722 tablet_bootstrap.cc:654] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.016585 12748 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.018704 12796 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.019493 12796 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.020938 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:09:19.022871 12572 leader_election.cc:343] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.035924 12748 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:19.037209 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:19.037911 12678 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.077775 12720 raft_consensus.cc:359] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.078622 12720 raft_consensus.cc:385] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.078881 12720 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.079502 12720 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.082453 12720 ts_tablet_manager.cc:1434] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.084s	user 0.017s	sys 0.015s
I20260709 12:09:19.083448 12720 tablet_bootstrap.cc:492] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:19.084585 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:19.087050 12721 tablet_bootstrap.cc:654] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
W20260709 12:09:19.092352 12494 leader_election.cc:343] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.094242 12789 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:19.087178 12678 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:19.109794 12571 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:19.096463 12796 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:19.098587 12726 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:19.111117 12795 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:19.111769 12795 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:09:19.112867 12795 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.099354 12799 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:19.101006 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:09:19.116371 12496 leader_election.cc:343] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.116835 12496 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:19.118352 12799 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:09:19.104005 12477 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:19.140714 12799 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:19.108134 12720 tablet_bootstrap.cc:654] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.143559 12721 tablet_bootstrap.cc:492] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:19.144222 12721 ts_tablet_manager.cc:1403] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.206s	user 0.007s	sys 0.009s
I20260709 12:09:19.146930 12722 tablet_bootstrap.cc:492] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:19.147488 12722 ts_tablet_manager.cc:1403] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.170s	user 0.030s	sys 0.033s
I20260709 12:09:19.162793 12722 raft_consensus.cc:359] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.161613 12721 raft_consensus.cc:359] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.164702 12721 raft_consensus.cc:385] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.165006 12721 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.166018 12721 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.163587 12722 raft_consensus.cc:385] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.168823 12722 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.170559 12429 catalog_manager.cc:5697] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 0 to 1, leader changed from <none> to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:19.169638 12722 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.178339 12781 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:19.179119 12722 ts_tablet_manager.cc:1434] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.031s	user 0.008s	sys 0.000s
I20260709 12:09:19.180378 12721 ts_tablet_manager.cc:1434] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.036s	user 0.006s	sys 0.004s
I20260709 12:09:19.181324 12721 tablet_bootstrap.cc:492] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:19.183748 12722 tablet_bootstrap.cc:492] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:19.191699 12722 tablet_bootstrap.cc:654] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.204656 12721 tablet_bootstrap.cc:654] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.212528 12721 tablet_bootstrap.cc:492] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:19.213217 12721 ts_tablet_manager.cc:1403] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.000s
I20260709 12:09:19.216593 12722 tablet_bootstrap.cc:492] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:19.217095 12722 ts_tablet_manager.cc:1403] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.037s	user 0.014s	sys 0.002s
I20260709 12:09:19.217402 12721 raft_consensus.cc:359] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.218163 12721 raft_consensus.cc:385] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.218544 12721 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.218796 12678 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.219360 12721 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.219772 12722 raft_consensus.cc:359] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.220738 12722 raft_consensus.cc:385] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.221087 12722 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.222039 12721 ts_tablet_manager.cc:1434] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:09:19.221930 12722 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.223006 12721 tablet_bootstrap.cc:492] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:19.224079 12722 ts_tablet_manager.cc:1434] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:09:19.224956 12722 tablet_bootstrap.cc:492] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:19.230530 12721 tablet_bootstrap.cc:654] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.231460 12722 tablet_bootstrap.cc:654] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.242787 12721 tablet_bootstrap.cc:492] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:19.243503 12721 ts_tablet_manager.cc:1403] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.021s	user 0.011s	sys 0.007s
I20260709 12:09:19.245402 12722 tablet_bootstrap.cc:492] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:19.245880 12722 ts_tablet_manager.cc:1403] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.021s	user 0.006s	sys 0.011s
I20260709 12:09:19.249197 12722 raft_consensus.cc:359] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.249933 12722 raft_consensus.cc:385] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.250252 12722 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.250824 12721 raft_consensus.cc:359] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.251693 12721 raft_consensus.cc:385] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.251367 12722 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.252100 12721 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.252758 12789 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.253796 12722 ts_tablet_manager.cc:1434] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:09:19.253791 12721 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.255527 12722 tablet_bootstrap.cc:492] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:19.257048 12721 ts_tablet_manager.cc:1434] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:09:19.258141 12721 tablet_bootstrap.cc:492] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:19.262696 12722 tablet_bootstrap.cc:654] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.265928 12721 tablet_bootstrap.cc:654] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.253791 12789 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.273033 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:19.277043 12789 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:19.277769 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:19.278687 12604 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 1: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:09:19.279779 12722 tablet_bootstrap.cc:492] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:19.279937 12643 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:19.280277 12722 ts_tablet_manager.cc:1403] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.000s
I20260709 12:09:19.283373 12721 tablet_bootstrap.cc:492] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:19.284076 12721 ts_tablet_manager.cc:1403] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.003s
I20260709 12:09:19.283876 12722 raft_consensus.cc:359] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.284632 12722 raft_consensus.cc:385] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.284962 12722 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.285802 12722 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.287282 12721 raft_consensus.cc:359] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.288136 12721 raft_consensus.cc:385] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.288398 12722 ts_tablet_manager.cc:1434] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:09:19.288509 12721 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.289453 12722 tablet_bootstrap.cc:492] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:19.289443 12721 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.290933 12779 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.299240 12722 tablet_bootstrap.cc:654] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.306819 12779 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:19.312779 12721 ts_tablet_manager.cc:1434] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.000s
I20260709 12:09:19.316394 12721 tablet_bootstrap.cc:492] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:19.319779 12720 tablet_bootstrap.cc:492] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:19.325169 12720 ts_tablet_manager.cc:1403] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.242s	user 0.081s	sys 0.089s
I20260709 12:09:19.329162 12720 raft_consensus.cc:359] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.330168 12720 raft_consensus.cc:385] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.330533 12720 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.331257 12720 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.333498 12721 tablet_bootstrap.cc:654] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.333523 12722 tablet_bootstrap.cc:492] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:19.334239 12722 ts_tablet_manager.cc:1403] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.045s	user 0.013s	sys 0.013s
I20260709 12:09:19.334420 12779 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.334874 12779 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.337232 12779 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:19.337299 12722 raft_consensus.cc:359] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.337778 12720 ts_tablet_manager.cc:1434] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:09:19.338001 12722 raft_consensus.cc:385] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.338356 12722 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.338807 12720 tablet_bootstrap.cc:492] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:19.338999 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:09:19.341180 12646 leader_election.cc:343] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.342283 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:19.343537 12781 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:09:19.344050 12643 leader_election.cc:343] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.344515 12643 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:19.345568 12720 tablet_bootstrap.cc:654] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.345908 12722 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.349092 12779 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:19.359948 12722 ts_tablet_manager.cc:1434] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:09:19.366026 12799 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:19.371806 12722 tablet_bootstrap.cc:492] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:19.371665 12795 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:19.393025 12720 tablet_bootstrap.cc:492] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:19.393842 12720 ts_tablet_manager.cc:1403] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.055s	user 0.019s	sys 0.011s
I20260709 12:09:19.393074 12722 tablet_bootstrap.cc:654] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.396981 12720 raft_consensus.cc:359] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.397799 12720 raft_consensus.cc:385] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.398183 12720 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.404531 12720 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.405355 12679 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.407030 12720 ts_tablet_manager.cc:1434] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.013s	user 0.005s	sys 0.001s
I20260709 12:09:19.409742 12779 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.410593 12720 tablet_bootstrap.cc:492] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:19.410413 12779 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.412756 12779 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:19.414695 12795 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:19.415333 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:19.415980 12531 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:19.417102 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:09:19.418907 12643 leader_election.cc:343] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.420281 12646 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:19.421913 12779 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:19.422344 12779 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:19.423882 12720 tablet_bootstrap.cc:654] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.422796 12779 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.431017 12789 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.431660 12789 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.434080 12789 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:19.435737 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:19.439932 12720 tablet_bootstrap.cc:492] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:19.437461 12605 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:19.440850 12643 leader_election.cc:343] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.437294 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:19.446465 12795 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:09:19.447432 12646 leader_election.cc:343] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.447146 12795 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.447916 12646 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:19.449095 12789 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:19.454015 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:19.454439 12720 ts_tablet_manager.cc:1403] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.047s	user 0.010s	sys 0.013s
I20260709 12:09:19.454912 12531 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:19.456611 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:19.457535 12816 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:19.457948 12816 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:19.458346 12816 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.466317 12799 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.466936 12799 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.467152 12816 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.469755 12799 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:19.470109 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:09:19.472615 12494 leader_election.cc:343] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.460108 12720 raft_consensus.cc:359] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.473721 12720 raft_consensus.cc:385] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.474293 12720 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.475164 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:19.475529 12799 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.476094 12799 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:19.476887 12496 leader_election.cc:343] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.475545 12720 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.477397 12496 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:19.478785 12781 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:19.479537 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:09:19.482079 12494 leader_election.cc:343] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.483274 12720 ts_tablet_manager.cc:1434] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.028s	user 0.009s	sys 0.000s
I20260709 12:09:19.483640 12799 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:19.486112 12795 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:19.487851 12720 tablet_bootstrap.cc:492] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:19.485904 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:19.487694 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:09:19.490624 12571 leader_election.cc:343] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:19.492215 12496 leader_election.cc:343] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.492733 12496 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:19.493523 12799 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:19.497186 12529 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.498036 12720 tablet_bootstrap.cc:654] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.511415 12720 tablet_bootstrap.cc:492] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:19.512466 12720 ts_tablet_manager.cc:1403] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.007s
I20260709 12:09:19.516201 12720 raft_consensus.cc:359] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.517319 12720 raft_consensus.cc:385] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.517747 12720 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.519500 12795 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:19.519008 12720 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.520807 12721 tablet_bootstrap.cc:492] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:19.521320 12721 ts_tablet_manager.cc:1403] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.205s	user 0.040s	sys 0.011s
I20260709 12:09:19.524739 12720 ts_tablet_manager.cc:1434] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:09:19.525631 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:19.528580 12531 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.529666 12720 tablet_bootstrap.cc:492] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:19.536103 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:09:19.537602 12571 leader_election.cc:343] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.538208 12816 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:19.538910 12720 tablet_bootstrap.cc:654] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.541275 12779 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.544480 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:19.545141 12529 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.524812 12721 raft_consensus.cc:359] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.548745 12721 raft_consensus.cc:385] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.549130 12721 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.550194 12721 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.555253 12748 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:19.558147 12781 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.558756 12781 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.561537 12781 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:19.567488 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:19.569113 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:09:19.571568 12496 leader_election.cc:343] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:19.573083 12494 leader_election.cc:343] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.562021 12529 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:19.573886 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:19.574892 12796 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:19.575450 12646 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:19.575702 12720 tablet_bootstrap.cc:492] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:19.576396 12789 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:19.576547 12720 ts_tablet_manager.cc:1403] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.047s	user 0.021s	sys 0.011s
I20260709 12:09:19.576948 12789 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:19.578634 12721 ts_tablet_manager.cc:1434] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.057s	user 0.005s	sys 0.004s
I20260709 12:09:19.578290 12789 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.582037 12796 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.582614 12796 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.583873 12531 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:19.584868 12796 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:19.585248 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:09:19.586225 12795 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:19.586953 12795 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:09:19.587548 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:19.588270 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:19.587999 12795 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:19.589212 12496 leader_election.cc:343] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.591467 12721 tablet_bootstrap.cc:492] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
W20260709 12:09:19.591701 12494 leader_election.cc:343] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.592317 12494 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:19.593529 12796 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:19.596536 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:09:19.598384 12643 leader_election.cc:343] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.599483 12779 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:19.605146 12720 raft_consensus.cc:359] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.605962 12720 raft_consensus.cc:385] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.606397 12720 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.607188 12721 tablet_bootstrap.cc:654] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.607151 12720 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.607703 12426 catalog_manager.cc:5697] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:19.609555 12720 ts_tablet_manager.cc:1434] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.026s	user 0.006s	sys 0.001s
I20260709 12:09:19.610913 12720 tablet_bootstrap.cc:492] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:19.615391 12722 tablet_bootstrap.cc:492] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:19.615957 12722 ts_tablet_manager.cc:1403] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.244s	user 0.053s	sys 0.074s
I20260709 12:09:19.617690 12720 tablet_bootstrap.cc:654] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.618362 12795 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.618798 12722 raft_consensus.cc:359] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.618896 12795 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.619692 12722 raft_consensus.cc:385] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.620009 12722 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.621397 12795 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:19.622491 12721 tablet_bootstrap.cc:492] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:19.623013 12721 ts_tablet_manager.cc:1403] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.005s
I20260709 12:09:19.623133 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:19.623863 12531 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:19.623546 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:19.624686 12678 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:19.620798 12722 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.626314 12721 raft_consensus.cc:359] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.627028 12721 raft_consensus.cc:385] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.627331 12721 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.627995 12722 ts_tablet_manager.cc:1434] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.003s
I20260709 12:09:19.628089 12721 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.629524 12722 tablet_bootstrap.cc:492] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:19.631057 12721 ts_tablet_manager.cc:1434] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:09:19.632934 12721 tablet_bootstrap.cc:492] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:19.636850 12722 tablet_bootstrap.cc:654] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.630875 12571 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:19.639264 12721 tablet_bootstrap.cc:654] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.639345 12720 tablet_bootstrap.cc:492] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:19.639904 12720 ts_tablet_manager.cc:1403] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.029s	user 0.008s	sys 0.005s
I20260709 12:09:19.640378 12795 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:19.640867 12795 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:19.641343 12795 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.643023 12720 raft_consensus.cc:359] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.644083 12720 raft_consensus.cc:385] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.644483 12720 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.645442 12720 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.649220 12720 ts_tablet_manager.cc:1434] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:09:19.649501 12722 tablet_bootstrap.cc:492] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:19.650221 12722 ts_tablet_manager.cc:1403] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.003s
I20260709 12:09:19.650681 12720 tablet_bootstrap.cc:492] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:19.653399 12722 raft_consensus.cc:359] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.653762 12427 catalog_manager.cc:5697] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 0 to 1, leader changed from <none> to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:19.657778 12722 raft_consensus.cc:385] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.658179 12722 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.658949 12816 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.658910 12722 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.659500 12816 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.661595 12816 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:19.661933 12722 ts_tablet_manager.cc:1434] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:09:19.663003 12722 tablet_bootstrap.cc:492] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:19.664011 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:19.664708 12678 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:19.666447 12571 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:19.667007 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:19.667750 12531 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:19.668867 12816 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:19.669318 12816 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:19.669775 12720 tablet_bootstrap.cc:654] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.669684 12816 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.672447 12722 tablet_bootstrap.cc:654] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.656677 12721 tablet_bootstrap.cc:492] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:19.675107 12721 ts_tablet_manager.cc:1403] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.043s	user 0.016s	sys 0.008s
I20260709 12:09:19.689349 12816 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.697682 12766 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.653311 12795 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.698518 12766 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.701030 12795 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:19.701550 12766 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:19.704632 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:09:19.707625 12572 leader_election.cc:343] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.708820 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:19.709367 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:19.709687 12680 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.713912 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:19.732360 12531 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:19.736845 12571 leader_election.cc:343] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.737552 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:19.739109 12766 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:19.773180 12680 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:19.775249 12571 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:19.776865 12795 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:19.794275 12721 raft_consensus.cc:359] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.795279 12721 raft_consensus.cc:385] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.795620 12721 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.797431 12531 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:19.799806 12721 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.802839 12721 ts_tablet_manager.cc:1434] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.127s	user 0.009s	sys 0.000s
I20260709 12:09:19.805732 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:19.806542 12529 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.828001 12721 tablet_bootstrap.cc:492] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:19.823738 12816 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:19.828985 12795 raft_consensus.cc:697] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:09:19.828760 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:19.830654 12680 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.831305 12529 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:19.830928 12795 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.832729 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:19.833542 12766 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:19.834020 12766 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:09:19.834883 12766 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.847940 12720 tablet_bootstrap.cc:492] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:19.848475 12720 ts_tablet_manager.cc:1403] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.198s	user 0.056s	sys 0.093s
I20260709 12:09:19.849629 12722 tablet_bootstrap.cc:492] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:19.850092 12722 ts_tablet_manager.cc:1403] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.187s	user 0.064s	sys 0.066s
I20260709 12:09:19.852748 12680 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:19.852918 12722 raft_consensus.cc:359] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.854202 12720 raft_consensus.cc:359] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.857801 12722 raft_consensus.cc:385] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.858214 12722 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.858017 12720 raft_consensus.cc:385] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.859133 12720 raft_consensus.cc:740] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.858918 12722 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.857335 12721 tablet_bootstrap.cc:654] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.859884 12720 consensus_queue.cc:260] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.861829 12722 ts_tablet_manager.cc:1434] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.003s
I20260709 12:09:19.862314 12720 ts_tablet_manager.cc:1434] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:09:19.863224 12720 tablet_bootstrap.cc:492] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:19.863287 12722 tablet_bootstrap.cc:492] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:19.871407 12720 tablet_bootstrap.cc:654] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.871407 12722 tablet_bootstrap.cc:654] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.880127 12721 tablet_bootstrap.cc:492] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:19.880970 12721 ts_tablet_manager.cc:1403] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.053s	user 0.012s	sys 0.002s
I20260709 12:09:19.883905 12721 raft_consensus.cc:359] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.884763 12722 tablet_bootstrap.cc:492] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:19.884800 12721 raft_consensus.cc:385] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.885378 12721 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.885779 12722 ts_tablet_manager.cc:1403] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.023s	user 0.016s	sys 0.005s
I20260709 12:09:19.886480 12721 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.888926 12721 ts_tablet_manager.cc:1434] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:09:19.890388 12721 tablet_bootstrap.cc:492] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:19.891104 12722 raft_consensus.cc:359] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.892192 12722 raft_consensus.cc:385] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.886471 12429 catalog_manager.cc:5697] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 0 to 1, leader changed from <none> to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:19.892614 12722 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.893908 12722 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.900014 12722 ts_tablet_manager.cc:1434] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:09:19.901962 12721 tablet_bootstrap.cc:654] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.902451 12722 tablet_bootstrap.cc:492] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:19.902952 12720 tablet_bootstrap.cc:492] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:19.903436 12720 ts_tablet_manager.cc:1403] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.040s	user 0.021s	sys 0.003s
I20260709 12:09:19.900691 12789 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.910687 12789 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.922286 12789 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:19.924228 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:09:19.925942 12646 leader_election.cc:343] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.927666 12795 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.934481 12720 raft_consensus.cc:359] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.935257 12720 raft_consensus.cc:385] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:19.935930 12720 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:19.936640 12720 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.933633 12795 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.938803 12720 ts_tablet_manager.cc:1434] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.035s	user 0.009s	sys 0.000s
I20260709 12:09:19.939741 12720 tablet_bootstrap.cc:492] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:19.935060 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:19.935679 12722 tablet_bootstrap.cc:654] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.941962 12795 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:19.949406 12429 catalog_manager.cc:5697] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 0 to 1, leader changed from <none> to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
W20260709 12:09:19.950254 12643 leader_election.cc:343] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:19.950557 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:19.950990 12643 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:19.951366 12529 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:19.952349 12789 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:19.954476 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:19.955250 12680 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:19.959190 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:19.961827 12766 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:19.962296 12766 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:19.963317 12766 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:19.968634 12720 tablet_bootstrap.cc:654] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:19.969070 12721 tablet_bootstrap.cc:492] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:19.969842 12721 ts_tablet_manager.cc:1403] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.080s	user 0.012s	sys 0.022s
I20260709 12:09:19.977684 12796 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:19.978304 12796 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.983027 12796 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:19.993764 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:19.988720 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:19.990974 12766 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.995721 12605 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:19.995934 12678 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:19.997922 12496 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:19.994594 12721 raft_consensus.cc:359] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:19.999677 12766 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:19.999925 12721 raft_consensus.cc:385] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.000227 12721 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.001096 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:20.000963 12721 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.001717 12678 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.002737 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:20.003553 12531 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.009124 12721 ts_tablet_manager.cc:1434] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.000s
I20260709 12:09:20.011126 12721 tablet_bootstrap.cc:492] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:20.010237 12796 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:20.014103 12796 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:20.014513 12796 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.011077 12766 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.017036 12766 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.020043 12799 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.020414 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:20.020774 12799 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.021634 12529 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 1: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:09:20.023854 12799 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:20.028856 12531 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:20.032919 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:20.032919 12572 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:20.036019 12879 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:20.036769 12879 raft_consensus.cc:697] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:09:20.037997 12879 consensus_queue.cc:237] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.044108 12606 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:20.047303 12494 leader_election.cc:343] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.075178 12678 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:20.075767 12722 tablet_bootstrap.cc:492] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:20.076299 12722 ts_tablet_manager.cc:1403] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.174s	user 0.042s	sys 0.052s
I20260709 12:09:20.077092 12429 catalog_manager.cc:5697] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 0 to 1, leader changed from <none> to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:20.079259 12722 raft_consensus.cc:359] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.079969 12722 raft_consensus.cc:385] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.080262 12722 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.081148 12722 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.082434 12766 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:20.083384 12722 ts_tablet_manager.cc:1434] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:09:20.084295 12722 tablet_bootstrap.cc:492] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:20.086710 12720 tablet_bootstrap.cc:492] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:20.087801 12789 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.088408 12789 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.092144 12722 tablet_bootstrap.cc:654] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.092934 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:20.094182 12531 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 1: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:09:20.096271 12789 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:20.097656 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:20.099228 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:20.101555 12789 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.104154 12720 ts_tablet_manager.cc:1403] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.148s	user 0.020s	sys 0.004s
I20260709 12:09:20.107649 12720 raft_consensus.cc:359] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.108392 12720 raft_consensus.cc:385] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.108657 12720 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.073534 12796 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:20.164335 12477 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:20.114986 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:20.124784 12722 tablet_bootstrap.cc:492] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:20.130281 12766 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.065s
I20260709 12:09:20.195405 12722 ts_tablet_manager.cc:1403] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.111s	user 0.025s	sys 0.037s
I20260709 12:09:20.196861 12571 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:20.197907 12766 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:20.198367 12722 raft_consensus.cc:359] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.199131 12722 raft_consensus.cc:385] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.199400 12722 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.200320 12766 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.201092 12766 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.128175 12789 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:20.210044 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:20.211189 12529 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:20.213492 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:20.214890 12795 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:20.215444 12795 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:20.215931 12795 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.171172 12879 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.259496 12879 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.140544 12721 tablet_bootstrap.cc:654] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.117731 12728 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.146s
W20260709 12:09:20.263842 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 67127.2us
I20260709 12:09:20.263698 12722 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.265237 12748 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:20.266453 12722 ts_tablet_manager.cc:1434] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.071s	user 0.025s	sys 0.045s
I20260709 12:09:20.267505 12722 tablet_bootstrap.cc:492] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:20.109311 12720 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.274952 12722 tablet_bootstrap.cc:654] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.275880 12779 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:20.142457 12799 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.115262 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:20.294356 12748 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:20.303718 12766 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:20.294234 12799 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.320786 12799 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:20.326428 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:20.329398 12799 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.329332 12528 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:20.332108 12796 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:20.332866 12572 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:20.334276 12816 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:20.334841 12816 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:20.335292 12816 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.348760 12799 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:20.353452 12496 leader_election.cc:343] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.378746 12496 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:20.382805 12896 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:20.390045 12879 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:20.313755 12720 ts_tablet_manager.cc:1434] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.209s	user 0.007s	sys 0.002s
I20260709 12:09:20.399283 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:20.400895 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:20.401322 12720 tablet_bootstrap.cc:492] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
W20260709 12:09:20.403677 12571 leader_election.cc:343] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:20.405205 12571 leader_election.cc:343] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.405778 12799 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:20.406920 12795 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.409139 12795 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:20.415236 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:20.417761 12721 tablet_bootstrap.cc:492] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:20.418406 12721 ts_tablet_manager.cc:1403] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.408s	user 0.012s	sys 0.005s
I20260709 12:09:20.420188 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:20.420899 12604 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.406299 12766 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:20.430346 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:20.431414 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:20.431059 12603 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:20.432049 12678 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.432446 12604 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:20.431950 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:20.433954 12494 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:20.436408 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:20.436556 12781 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:20.437037 12528 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.437175 12781 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:20.437726 12781 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:20.443574 12571 leader_election.cc:343] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.354699 12816 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.445930 12816 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:20.446831 12494 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:20.442477 12678 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:20.451828 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:20.452921 12676 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:20.455088 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:20.455824 12675 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:20.461114 12892 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:20.461714 12892 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:09:20.462682 12892 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.463466 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:20.464097 12530 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.467121 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:09:20.469168 12494 leader_election.cc:343] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.455096 12781 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:20.476933 12571 leader_election.cc:343] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.480062 12721 raft_consensus.cc:359] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.480895 12721 raft_consensus.cc:385] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.481166 12721 raft_consensus.cc:740] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.486999 12496 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:20.490195 12721 consensus_queue.cc:260] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.492309 12799 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:20.492753 12799 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:20.493114 12799 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.499128 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:20.500291 12606 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.502049 12720 tablet_bootstrap.cc:654] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.510665 12781 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:20.515617 12528 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:20.527848 12721 ts_tablet_manager.cc:1434] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.109s	user 0.009s	sys 0.000s
I20260709 12:09:20.529357 12721 tablet_bootstrap.cc:492] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:20.530699 12530 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:20.541189 12879 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.022s
I20260709 12:09:20.566581 12721 tablet_bootstrap.cc:654] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.565049 12899 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.023s
W20260709 12:09:20.562500 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 72840.9us
I20260709 12:09:20.575039 12816 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:09:20.561685 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:09:20.567802 12606 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:20.610635 12722 tablet_bootstrap.cc:492] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:20.611553 12722 ts_tablet_manager.cc:1403] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.344s	user 0.084s	sys 0.138s
I20260709 12:09:20.592685 12748 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:20.580363 12799 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:20.608873 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 59004.1us
I20260709 12:09:20.630406 12494 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:20.630934 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:20.631678 12680 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.627301 12720 tablet_bootstrap.cc:492] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:20.634696 12720 ts_tablet_manager.cc:1403] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.234s	user 0.023s	sys 0.011s
I20260709 12:09:20.622483 12748 raft_consensus.cc:697] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:09:20.638621 12911 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:20.642724 12911 raft_consensus.cc:697] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:09:20.643669 12911 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.651161 12796 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.651448 12799 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:20.651762 12796 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.652105 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:20.656163 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:20.656634 12796 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:20.656796 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:20.657397 12675 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.658309 12572 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:09:20.659605 12899 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:20.660101 12899 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:09:20.660748 12879 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:20.660955 12899 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.653491 12748 consensus_queue.cc:237] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.662377 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:09:20.662892 12494 leader_election.cc:343] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:20.664719 12910 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.050s	user 0.002s	sys 0.002s
W20260709 12:09:20.670555 12496 leader_election.cc:343] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.671913 12722 raft_consensus.cc:359] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.672719 12722 raft_consensus.cc:385] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.673050 12722 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.671092 12496 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:20.675661 12781 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:20.673750 12722 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.695708 12722 ts_tablet_manager.cc:1434] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.084s	user 0.006s	sys 0.001s
I20260709 12:09:20.693801 12429 catalog_manager.cc:5697] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:20.700683 12721 tablet_bootstrap.cc:492] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:20.701540 12721 ts_tablet_manager.cc:1403] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.172s	user 0.029s	sys 0.013s
I20260709 12:09:20.703315 12722 tablet_bootstrap.cc:492] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:20.712455 12675 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:20.712917 12680 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:20.727012 12721 raft_consensus.cc:359] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.727851 12721 raft_consensus.cc:385] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.728056 12721 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.730268 12496 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:20.738272 12721 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.738797 12781 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:20.739538 12781 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:09:20.740679 12781 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:20.745095 12494 leader_election.cc:343] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.747110 12721 ts_tablet_manager.cc:1434] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.045s	user 0.004s	sys 0.005s
I20260709 12:09:20.748440 12721 tablet_bootstrap.cc:492] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:20.751631 12720 raft_consensus.cc:359] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.752780 12720 raft_consensus.cc:385] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.753014 12720 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.753592 12720 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.756239 12720 ts_tablet_manager.cc:1434] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.121s	user 0.002s	sys 0.005s
I20260709 12:09:20.757432 12720 tablet_bootstrap.cc:492] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:20.769878 12427 catalog_manager.cc:5697] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 0 to 1, leader changed from <none> to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:20.777338 12721 tablet_bootstrap.cc:654] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.790704 12722 tablet_bootstrap.cc:654] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.793093 12720 tablet_bootstrap.cc:654] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.798883 12721 tablet_bootstrap.cc:492] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:20.799628 12721 ts_tablet_manager.cc:1403] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.052s	user 0.009s	sys 0.007s
I20260709 12:09:20.803604 12721 raft_consensus.cc:359] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.804373 12721 raft_consensus.cc:385] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.804685 12721 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.806556 12721 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.811717 12722 tablet_bootstrap.cc:492] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:20.812247 12722 ts_tablet_manager.cc:1403] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.109s	user 0.010s	sys 0.011s
I20260709 12:09:20.815140 12722 raft_consensus.cc:359] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.815845 12721 ts_tablet_manager.cc:1434] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.001s
I20260709 12:09:20.815843 12722 raft_consensus.cc:385] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.816442 12722 raft_consensus.cc:740] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.817013 12721 tablet_bootstrap.cc:492] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:20.817175 12722 consensus_queue.cc:260] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.819669 12722 ts_tablet_manager.cc:1434] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:09:20.820577 12722 tablet_bootstrap.cc:492] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:20.824160 12721 tablet_bootstrap.cc:654] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.826771 12722 tablet_bootstrap.cc:654] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.829725 12428 catalog_manager.cc:5697] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:20.831664 12428 catalog_manager.cc:5697] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:20.838852 12720 tablet_bootstrap.cc:492] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:20.839570 12720 ts_tablet_manager.cc:1403] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.082s	user 0.021s	sys 0.003s
I20260709 12:09:20.843822 12894 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.844769 12894 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.849614 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:20.851307 12530 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 1: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:09:20.869472 12911 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.869993 12722 tablet_bootstrap.cc:492] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:20.870540 12722 ts_tablet_manager.cc:1403] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.050s	user 0.017s	sys 0.019s
I20260709 12:09:20.870270 12911 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.872977 12911 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:20.874559 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:20.876175 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:09:20.876734 12496 leader_election.cc:343] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.877453 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:09:20.877614 12494 leader_election.cc:343] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:20.878057 12494 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:20.880364 12643 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:20.882058 12728 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.881645 12426 catalog_manager.cc:5697] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 0 to 1, leader changed from <none> to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:20.876171 12894 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:20.888381 12722 raft_consensus.cc:359] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.889369 12722 raft_consensus.cc:385] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.889830 12722 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.890533 12722 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.893456 12722 ts_tablet_manager.cc:1434] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 12:09:20.894456 12722 tablet_bootstrap.cc:492] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:20.889321 12728 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:20.889465 12911 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:20.907909 12722 tablet_bootstrap.cc:654] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.901335 12720 raft_consensus.cc:359] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.911129 12720 raft_consensus.cc:385] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.935793 12720 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.927932 12748 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:20.936651 12720 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.936889 12748 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.927104 12722 tablet_bootstrap.cc:492] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:20.938310 12722 ts_tablet_manager.cc:1403] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.044s	user 0.013s	sys 0.002s
I20260709 12:09:20.939201 12748 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:20.941025 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:20.941515 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:20.941933 12674 raft_consensus.cc:2393] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 1: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:09:20.942240 12720 ts_tablet_manager.cc:1434] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.102s	user 0.005s	sys 0.004s
I20260709 12:09:20.943238 12720 tablet_bootstrap.cc:492] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:20.947448 12571 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:20.948768 12748 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:20.951445 12720 tablet_bootstrap.cc:654] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.952128 12721 tablet_bootstrap.cc:492] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:20.952602 12721 ts_tablet_manager.cc:1403] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.136s	user 0.056s	sys 0.033s
I20260709 12:09:20.955410 12721 raft_consensus.cc:359] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.956113 12721 raft_consensus.cc:385] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.956380 12721 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.957048 12721 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.959302 12721 ts_tablet_manager.cc:1434] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.003s
I20260709 12:09:20.960703 12721 tablet_bootstrap.cc:492] T e40144c1c60847ee81b8252b107df67d P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:20.967600 12721 tablet_bootstrap.cc:654] T e40144c1c60847ee81b8252b107df67d P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:20.962244 12748 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:20.982857 12722 raft_consensus.cc:359] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.983640 12722 raft_consensus.cc:385] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.983953 12722 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.984792 12722 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.986367 12720 tablet_bootstrap.cc:492] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:20.986863 12720 ts_tablet_manager.cc:1403] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.044s	user 0.027s	sys 0.011s
I20260709 12:09:20.987450 12722 ts_tablet_manager.cc:1434] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.000s
I20260709 12:09:20.989481 12722 tablet_bootstrap.cc:492] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:20.989589 12720 raft_consensus.cc:359] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.990715 12720 raft_consensus.cc:385] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:20.991009 12720 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:20.992162 12720 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:20.994450 12720 ts_tablet_manager.cc:1434] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:09:20.996080 12720 tablet_bootstrap.cc:492] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:20.996580 12722 tablet_bootstrap.cc:654] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.002094 12720 tablet_bootstrap.cc:654] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.007334 12721 tablet_bootstrap.cc:492] T e40144c1c60847ee81b8252b107df67d P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:21.015213 12721 ts_tablet_manager.cc:1403] T e40144c1c60847ee81b8252b107df67d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.055s	user 0.020s	sys 0.008s
I20260709 12:09:21.017692 12722 tablet_bootstrap.cc:492] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:21.018304 12722 ts_tablet_manager.cc:1403] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.029s	user 0.025s	sys 0.002s
I20260709 12:09:21.018187 12721 raft_consensus.cc:359] T e40144c1c60847ee81b8252b107df67d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.018935 12721 raft_consensus.cc:385] T e40144c1c60847ee81b8252b107df67d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.019254 12721 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.019600 12911 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:21.019904 12721 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.021593 12722 raft_consensus.cc:359] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.022249 12721 ts_tablet_manager.cc:1434] T e40144c1c60847ee81b8252b107df67d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:09:21.022315 12722 raft_consensus.cc:385] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.023952 12722 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.024577 12721 tablet_bootstrap.cc:492] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:21.024667 12722 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.029541 12781 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.030900 12722 ts_tablet_manager.cc:1434] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:09:21.030529 12781 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.031232 12721 tablet_bootstrap.cc:654] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.031968 12722 tablet_bootstrap.cc:492] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:21.034977 12896 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.035905 12720 tablet_bootstrap.cc:492] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:21.035602 12896 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.036530 12720 ts_tablet_manager.cc:1403] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.041s	user 0.012s	sys 0.003s
I20260709 12:09:21.039129 12722 tablet_bootstrap.cc:654] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.039537 12720 raft_consensus.cc:359] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.040254 12720 raft_consensus.cc:385] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.040875 12720 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.041546 12720 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.041780 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:21.042666 12674 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:21.043720 12720 ts_tablet_manager.cc:1434] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:09:21.044924 12720 tablet_bootstrap.cc:492] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:21.046548 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:21.047951 12496 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:21.048957 12796 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:21.049367 12796 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:21.049775 12796 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.054350 12720 tablet_bootstrap.cc:654] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.060858 12721 tablet_bootstrap.cc:492] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:21.060659 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:21.062342 12721 ts_tablet_manager.cc:1403] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.038s	user 0.007s	sys 0.006s
I20260709 12:09:21.057667 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:21.064142 12606 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:21.066864 12721 raft_consensus.cc:359] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.067643 12721 raft_consensus.cc:385] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.067896 12721 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.068549 12721 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:21.069939 12494 leader_election.cc:343] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.070729 12721 ts_tablet_manager.cc:1434] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
W20260709 12:09:21.075857 12496 leader_election.cc:343] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.079232 12496 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:21.080644 12799 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:21.081017 12892 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:21.075937 12721 tablet_bootstrap.cc:492] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:21.088229 12781 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:21.040218 12896 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:21.093009 12720 tablet_bootstrap.cc:492] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:21.093849 12720 ts_tablet_manager.cc:1403] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.049s	user 0.017s	sys 0.001s
I20260709 12:09:21.098510 12781 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.099146 12781 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.101011 12728 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.101848 12721 tablet_bootstrap.cc:654] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.101588 12728 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.109297 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:21.110298 12530 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:21.112675 12728 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:21.113540 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:21.097601 12720 raft_consensus.cc:359] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.114737 12720 raft_consensus.cc:385] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.114888 12606 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:21.113690 12646 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:21.116462 12720 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.117828 12720 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.120206 12720 ts_tablet_manager.cc:1434] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.026s	user 0.007s	sys 0.000s
I20260709 12:09:21.121570 12748 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:21.121565 12720 tablet_bootstrap.cc:492] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:21.123651 12728 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:21.126858 12728 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:21.127574 12728 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.134593 12892 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.135118 12892 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.151638 12892 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:21.158013 12720 tablet_bootstrap.cc:654] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.147255 12728 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.159932 12728 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:21.162032 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:21.162976 12604 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.168140 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:21.168819 12530 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.184989 12680 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.153772 12721 tablet_bootstrap.cc:492] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:21.154412 12781 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:21.189060 12721 ts_tablet_manager.cc:1403] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.113s	user 0.017s	sys 0.005s
I20260709 12:09:21.193497 12748 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:21.198545 12721 raft_consensus.cc:359] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.199615 12721 raft_consensus.cc:385] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.200001 12721 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.200968 12721 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.153162 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:21.206821 12679 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:21.207429 12721 ts_tablet_manager.cc:1434] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.004s
I20260709 12:09:21.208812 12721 tablet_bootstrap.cc:492] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:21.209178 12496 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:21.215421 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:21.189255 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:21.218412 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:21.218823 12675 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:21.219705 12911 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:21.220142 12911 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:21.220474 12911 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.224538 12748 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:21.224978 12721 tablet_bootstrap.cc:654] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.228382 12496 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:21.235423 12604 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:21.237332 12799 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:21.250362 12799 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:21.250804 12799 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.264278 12643 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:21.237673 12796 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:21.241317 12494 leader_election.cc:343] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.241592 12721 tablet_bootstrap.cc:492] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:21.268498 12721 ts_tablet_manager.cc:1403] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.060s	user 0.013s	sys 0.003s
W20260709 12:09:21.269994 12494 leader_election.cc:343] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.272764 12721 raft_consensus.cc:359] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.273798 12721 raft_consensus.cc:385] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.274228 12721 raft_consensus.cc:740] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.275197 12721 consensus_queue.cc:260] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.277035 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:21.242470 12530 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:21.265264 12728 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:21.278534 12728 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:21.277769 12604 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.279435 12728 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.280352 12721 ts_tablet_manager.cc:1434] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.003s
I20260709 12:09:21.282086 12721 tablet_bootstrap.cc:492] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:21.265246 12799 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.291913 12799 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:21.291680 12879 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:21.296098 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:21.296794 12675 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.299471 12721 tablet_bootstrap.cc:654] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.302603 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:21.303365 12680 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.311295 12721 tablet_bootstrap.cc:492] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:21.312263 12721 ts_tablet_manager.cc:1403] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.003s
I20260709 12:09:21.316244 12721 raft_consensus.cc:359] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.316972 12721 raft_consensus.cc:385] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.317229 12721 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.323997 12892 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:21.325608 12721 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.328919 12721 ts_tablet_manager.cc:1434] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:09:21.342096 12896 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:21.355374 12796 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:21.362217 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:21.367036 12721 tablet_bootstrap.cc:492] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:21.377117 12911 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.387231 12680 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:21.393837 12721 tablet_bootstrap.cc:654] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.409060 12799 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
W20260709 12:09:21.412211 12494 leader_election.cc:343] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.411619 12728 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.425266 12796 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:21.419224 12911 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:21.431802 12721 tablet_bootstrap.cc:492] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:21.432309 12721 ts_tablet_manager.cc:1403] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.066s	user 0.018s	sys 0.002s
I20260709 12:09:21.433281 12748 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.434888 12748 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.424664 12496 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:21.431898 12429 catalog_manager.cc:5697] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:21.439543 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:21.441416 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:21.442092 12528 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:21.422300 12728 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.443538 12572 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:21.444854 12879 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:21.445276 12879 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:21.445607 12879 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.447049 12728 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:21.448661 12679 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:21.450937 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:21.452006 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:21.438853 12748 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:21.452647 12528 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:21.453831 12646 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:21.454280 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:21.496501 12722 tablet_bootstrap.cc:492] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:21.499653 12722 ts_tablet_manager.cc:1403] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.468s	user 0.134s	sys 0.220s
I20260709 12:09:21.454922 12679 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.454741 12728 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:21.455068 12675 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:21.467041 12893 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:09:21.460728 12477 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:21.473313 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:21.507783 12728 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:21.510445 12728 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.455168 12892 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:21.458534 12604 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:21.525866 12720 tablet_bootstrap.cc:492] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:21.526798 12720 ts_tablet_manager.cc:1403] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.406s	user 0.061s	sys 0.099s
I20260709 12:09:21.530612 12722 raft_consensus.cc:359] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.531538 12722 raft_consensus.cc:385] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.531872 12722 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.532723 12722 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.533262 12720 raft_consensus.cc:359] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.534157 12720 raft_consensus.cc:385] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.534546 12720 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.535586 12722 ts_tablet_manager.cc:1434] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 12:09:21.536907 12722 tablet_bootstrap.cc:492] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:21.538551 12720 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.476152 12879 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.544729 12879 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:21.546350 12722 tablet_bootstrap.cc:654] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.524971 12892 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:09:21.515872 12893 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.558715 12494 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:21.559894 12721 raft_consensus.cc:359] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.562775 12721 raft_consensus.cc:385] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.563105 12721 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.559746 12892 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.563882 12721 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.561856 12606 raft_consensus.cc:2393] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 1: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:09:21.561350 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:21.567067 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:21.576018 12528 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.576361 12721 ts_tablet_manager.cc:1434] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.144s	user 0.011s	sys 0.001s
I20260709 12:09:21.576627 12720 ts_tablet_manager.cc:1434] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.050s	user 0.010s	sys 0.000s
I20260709 12:09:21.577616 12720 tablet_bootstrap.cc:492] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:21.573477 12796 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:21.585150 12796 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:09:21.590854 12748 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.597223 12893 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:21.597193 12679 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:21.598644 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:21.605531 12721 tablet_bootstrap.cc:492] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
W20260709 12:09:21.588114 12494 leader_election.cc:343] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:21.613962 12643 leader_election.cc:343] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.595613 12748 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.586056 12796 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:21.620093 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 45138.9us
I20260709 12:09:21.627583 12496 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:21.648694 12799 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:21.651078 12799 raft_consensus.cc:697] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:09:21.652269 12799 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.656052 12728 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.658627 12675 raft_consensus.cc:2393] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 1: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:09:21.658478 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:21.659150 12530 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:21.666579 12720 tablet_bootstrap.cc:654] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.667448 12728 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:21.678471 12961 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.682597 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:21.683506 12603 raft_consensus.cc:2393] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 1: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:09:21.682842 12961 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.686256 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:21.693015 12646 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:21.693253 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:21.694478 12531 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:21.695861 12728 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:21.696265 12728 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:21.696599 12728 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.699473 12572 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:21.687767 12748 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:21.700605 12748 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:21.700968 12748 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:21.701321 12748 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.705669 12528 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:21.720149 12646 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:21.721422 12960 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:09:21.716311 12572 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:09:21.727478 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:21.728129 12675 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:21.726013 12964 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:21.730082 12964 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:09:21.731266 12964 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.724253 12961 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:21.740767 12721 tablet_bootstrap.cc:654] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.733788 12748 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.743170 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:21.752206 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:21.753168 12530 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:21.760361 12426 catalog_manager.cc:5697] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:21.761235 12646 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:09:21.764550 12643 leader_election.cc:343] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.755959 12728 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.765637 12961 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:21.766065 12961 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:21.766445 12961 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.767666 12722 tablet_bootstrap.cc:492] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:21.768118 12722 ts_tablet_manager.cc:1403] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.231s	user 0.016s	sys 0.014s
I20260709 12:09:21.768756 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:21.769538 12530 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.771908 12426 catalog_manager.cc:5697] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:21.777627 12722 raft_consensus.cc:359] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.778404 12722 raft_consensus.cc:385] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.778678 12722 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.778167 12426 catalog_manager.cc:5697] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:21.779297 12722 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.781978 12722 ts_tablet_manager.cc:1434] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.014s	user 0.008s	sys 0.000s
I20260709 12:09:21.782986 12722 tablet_bootstrap.cc:492] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:21.773118 12728 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:21.789274 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:21.793917 12720 tablet_bootstrap.cc:492] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:21.795770 12530 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:21.795182 12720 ts_tablet_manager.cc:1403] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.218s	user 0.029s	sys 0.015s
I20260709 12:09:21.797436 12646 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:21.798708 12728 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:21.799985 12728 raft_consensus.cc:697] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:21.800916 12728 consensus_queue.cc:237] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.801642 12961 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.804817 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:21.804961 12961 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:21.805505 12530 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.806700 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:09:21.809594 12643 leader_election.cc:343] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.810902 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:21.811594 12680 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.811671 12748 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:09:21.815286 12643 leader_election.cc:343] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.817061 12722 tablet_bootstrap.cc:654] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.816766 12720 raft_consensus.cc:359] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.817624 12720 raft_consensus.cc:385] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.818153 12720 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.819141 12720 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.820994 12680 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:21.821996 12720 ts_tablet_manager.cc:1434] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 12:09:21.822208 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:21.823149 12720 tablet_bootstrap.cc:492] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:21.823315 12748 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:21.823989 12722 tablet_bootstrap.cc:492] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:21.824025 12748 raft_consensus.cc:697] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:09:21.824522 12722 ts_tablet_manager.cc:1403] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.042s	user 0.008s	sys 0.002s
I20260709 12:09:21.825322 12748 consensus_queue.cc:237] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.825246 12428 catalog_manager.cc:5697] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 0 to 1, leader changed from <none> to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:21.823402 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:21.827178 12528 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.831672 12722 raft_consensus.cc:359] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.832427 12722 raft_consensus.cc:385] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.832726 12722 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.833431 12722 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.836047 12721 tablet_bootstrap.cc:492] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:21.836505 12721 ts_tablet_manager.cc:1403] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.231s	user 0.015s	sys 0.009s
I20260709 12:09:21.839238 12721 raft_consensus.cc:359] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.839975 12721 raft_consensus.cc:385] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.840246 12721 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.841619 12720 tablet_bootstrap.cc:654] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.842041 12530 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:21.843415 12646 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:21.844655 12960 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:21.845193 12960 raft_consensus.cc:697] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:21.846158 12960 consensus_queue.cc:237] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.851971 12721 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.854892 12721 ts_tablet_manager.cc:1434] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.018s	user 0.008s	sys 0.001s
I20260709 12:09:21.855923 12721 tablet_bootstrap.cc:492] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:21.858566 12961 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:21.860008 12722 ts_tablet_manager.cc:1434] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.035s	user 0.008s	sys 0.000s
I20260709 12:09:21.861737 12722 tablet_bootstrap.cc:492] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:21.862300 12720 tablet_bootstrap.cc:492] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:21.862821 12720 ts_tablet_manager.cc:1403] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.040s	user 0.012s	sys 0.009s
I20260709 12:09:21.868427 12720 raft_consensus.cc:359] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.872547 12722 tablet_bootstrap.cc:654] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.871304 12721 tablet_bootstrap.cc:654] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.872572 12720 raft_consensus.cc:385] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.878916 12720 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.879645 12720 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.881829 12720 ts_tablet_manager.cc:1434] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.019s	user 0.003s	sys 0.001s
I20260709 12:09:21.882748 12720 tablet_bootstrap.cc:492] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:21.884147 12911 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.884651 12911 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.884833 12956 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:21.891455 12911 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:21.891708 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:09:21.893409 12494 leader_election.cc:343] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.896353 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:09:21.897657 12496 leader_election.cc:343] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:21.898149 12496 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:21.900748 12720 tablet_bootstrap.cc:654] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.901039 12796 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:21.903717 12721 tablet_bootstrap.cc:492] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:21.904242 12721 ts_tablet_manager.cc:1403] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.049s	user 0.012s	sys 0.004s
I20260709 12:09:21.909587 12528 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:21.910300 12721 raft_consensus.cc:359] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.911088 12721 raft_consensus.cc:385] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.911391 12721 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.912150 12721 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.913798 12721 ts_tablet_manager.cc:1434] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:09:21.915900 12903 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.916553 12721 tablet_bootstrap.cc:492] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:21.916405 12903 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.923970 12721 tablet_bootstrap.cc:654] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.925562 12903 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:21.926275 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:21.928424 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:21.929616 12675 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 1: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:09:21.931932 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:21.933745 12722 tablet_bootstrap.cc:492] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:21.934145 12748 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:21.934319 12722 ts_tablet_manager.cc:1403] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.073s	user 0.015s	sys 0.004s
I20260709 12:09:21.938097 12722 raft_consensus.cc:359] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.938851 12722 raft_consensus.cc:385] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.939117 12722 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.939901 12722 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.942102 12748 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:21.943791 12722 ts_tablet_manager.cc:1434] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:09:21.946022 12720 tablet_bootstrap.cc:492] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:21.946602 12720 ts_tablet_manager.cc:1403] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.064s	user 0.012s	sys 0.007s
I20260709 12:09:21.946967 12426 catalog_manager.cc:5697] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 0 to 1, leader changed from <none> to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:21.949344 12720 raft_consensus.cc:359] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.956741 12720 raft_consensus.cc:385] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.957058 12720 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.957763 12720 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.958899 12428 catalog_manager.cc:5697] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:21.965639 12720 ts_tablet_manager.cc:1434] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.017s	user 0.005s	sys 0.000s
I20260709 12:09:21.966655 12720 tablet_bootstrap.cc:492] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:21.972880 12720 tablet_bootstrap.cc:654] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:21.984104 12720 tablet_bootstrap.cc:492] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:21.984606 12720 ts_tablet_manager.cc:1403] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.018s	user 0.008s	sys 0.006s
I20260709 12:09:21.987322 12720 raft_consensus.cc:359] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.988265 12720 raft_consensus.cc:385] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:21.988500 12722 tablet_bootstrap.cc:492] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:21.988554 12720 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:21.989513 12720 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.991662 12720 ts_tablet_manager.cc:1434] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:09:21.992493 12720 tablet_bootstrap.cc:492] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:21.996650 12796 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:21.997218 12796 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:21.998430 12722 tablet_bootstrap.cc:654] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.000334 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:22.000633 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:09:22.001756 12496 leader_election.cc:343] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:22.002393 12494 leader_election.cc:343] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.002837 12494 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:22.003305 12720 tablet_bootstrap.cc:654] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.003698 12911 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:22.005088 12903 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:22.005795 12796 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:22.005687 12903 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.008345 12903 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:22.009167 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:22.016816 12528 raft_consensus.cc:2393] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 1: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:09:22.017313 12720 tablet_bootstrap.cc:492] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:22.017786 12720 ts_tablet_manager.cc:1403] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.025s	user 0.012s	sys 0.000s
I20260709 12:09:22.010313 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:22.013454 12722 tablet_bootstrap.cc:492] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:22.019085 12722 ts_tablet_manager.cc:1403] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.031s	user 0.007s	sys 0.007s
I20260709 12:09:22.020530 12720 raft_consensus.cc:359] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.021240 12720 raft_consensus.cc:385] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.021548 12720 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.022356 12722 raft_consensus.cc:359] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.022408 12720 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.023792 12722 raft_consensus.cc:385] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.024683 12722 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.024794 12720 ts_tablet_manager.cc:1434] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:09:22.025869 12720 tablet_bootstrap.cc:492] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:22.022831 12571 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:22.025610 12722 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.027091 12903 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.027879 12722 ts_tablet_manager.cc:1434] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:09:22.027587 12748 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:22.028364 12748 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.031615 12748 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:22.028820 12722 tablet_bootstrap.cc:492] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:22.033972 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:22.034436 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:22.034780 12680 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 1: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:09:22.036140 12903 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:22.038717 12722 tablet_bootstrap.cc:654] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.041306 12572 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:22.041231 12427 catalog_manager.cc:5697] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:22.042239 12903 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.052464 12903 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:22.059482 12748 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:22.061734 12748 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.064757 12748 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:22.061070 12720 tablet_bootstrap.cc:654] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.066331 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:22.066720 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:22.067476 12528 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
W20260709 12:09:22.068048 12571 leader_election.cc:343] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.068861 12572 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:09:22.069919 12748 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:22.070250 12722 tablet_bootstrap.cc:492] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:22.070392 12748 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:22.070863 12748 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.070927 12722 ts_tablet_manager.cc:1403] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.042s	user 0.011s	sys 0.024s
I20260709 12:09:22.074749 12722 raft_consensus.cc:359] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.075673 12722 raft_consensus.cc:385] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.076040 12722 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.077399 12748 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.078193 12722 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.080266 12728 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:22.080322 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:22.081022 12528 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.080919 12728 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.082329 12748 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:22.084450 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:09:22.087018 12571 leader_election.cc:343] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.087280 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:22.089136 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:22.089800 12530 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
W20260709 12:09:22.091789 12643 leader_election.cc:343] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.091753 12528 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:22.092458 12646 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:22.093744 12961 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:22.094292 12961 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:22.094645 12961 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.096989 12728 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:22.096966 12796 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:22.093793 12572 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:09:22.099666 12964 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:22.100240 12964 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:09:22.101105 12964 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.103638 12799 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:22.106899 12799 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:22.111002 12720 tablet_bootstrap.cc:492] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:22.111824 12722 ts_tablet_manager.cc:1434] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.040s	user 0.010s	sys 0.000s
I20260709 12:09:22.116412 12721 tablet_bootstrap.cc:492] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:22.117185 12721 ts_tablet_manager.cc:1403] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.201s	user 0.034s	sys 0.037s
I20260709 12:09:22.118594 12960 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:22.111851 12720 ts_tablet_manager.cc:1403] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.086s	user 0.013s	sys 0.007s
I20260709 12:09:22.119213 12960 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.120273 12722 tablet_bootstrap.cc:492] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:22.120466 12721 raft_consensus.cc:359] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.121253 12721 raft_consensus.cc:385] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.121578 12721 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.122805 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:22.123493 12605 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:22.124333 12720 raft_consensus.cc:359] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.125017 12720 raft_consensus.cc:385] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.125932 12643 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:22.126881 12728 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:22.127272 12728 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:22.127633 12728 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.125321 12720 raft_consensus.cc:740] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.131423 12720 consensus_queue.cc:260] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.135584 12720 ts_tablet_manager.cc:1434] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.016s	user 0.004s	sys 0.002s
I20260709 12:09:22.136420 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:22.137372 12530 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:22.140973 12720 tablet_bootstrap.cc:492] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:22.141619 12728 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.130029 12960 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:22.147226 12728 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:22.138413 12961 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.157923 12721 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.162894 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:22.163681 12605 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.170086 12961 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:22.169847 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:22.170893 12530 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.178397 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:22.181272 12799 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:22.183985 12964 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:22.176463 12427 catalog_manager.cc:5697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 0 to 1, leader changed from <none> to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:22.187747 12721 ts_tablet_manager.cc:1434] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.070s	user 0.010s	sys 0.000s
I20260709 12:09:22.188817 12721 tablet_bootstrap.cc:492] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
W20260709 12:09:22.198988 12643 leader_election.cc:343] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.201421 12964 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:22.215623 12721 tablet_bootstrap.cc:654] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.218353 12892 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:22.247766 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:22.256556 12529 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.249668 12605 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:22.294586 12799 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:22.299223 12799 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.303683 12960 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:22.305364 12720 tablet_bootstrap.cc:654] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.309501 12722 tablet_bootstrap.cc:654] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.311570 12529 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:22.313424 12646 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:22.314218 12728 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:22.314738 12728 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:22.315691 12728 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.317950 12799 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:22.323627 12964 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:22.325968 12964 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.328743 12964 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:22.338059 12530 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:22.340160 12646 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:22.340965 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:22.342414 12721 tablet_bootstrap.cc:492] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:22.354435 12960 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:22.359192 12960 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:22.354171 12911 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:22.360492 12911 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.360317 12960 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.378259 12721 ts_tablet_manager.cc:1403] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.190s	user 0.008s	sys 0.023s
I20260709 12:09:22.408860 12720 tablet_bootstrap.cc:492] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:22.409643 12720 ts_tablet_manager.cc:1403] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.269s	user 0.022s	sys 0.021s
I20260709 12:09:22.411574 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:22.404179 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:09:22.415885 12643 consensus_peers.cc:597] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:22.429255 12721 raft_consensus.cc:359] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.430904 12721 raft_consensus.cc:385] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.432318 12721 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.433231 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:22.432961 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:22.434461 12721 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.442304 12956 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:09:22.445904 12572 leader_election.cc:343] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.449265 12721 ts_tablet_manager.cc:1434] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.071s	user 0.005s	sys 0.003s
I20260709 12:09:22.450784 12964 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:22.453739 12892 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:09:22.454409 12494 leader_election.cc:343] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:22.456704 12496 leader_election.cc:343] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.456804 12956 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:22.457185 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:22.457268 12963 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:22.458597 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:22.459380 12674 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
W20260709 12:09:22.460529 12494 leader_election.cc:343] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.457428 12956 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.450238 12721 tablet_bootstrap.cc:492] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:22.468279 12796 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:22.470255 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:22.460013 12911 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:22.491632 12720 raft_consensus.cc:359] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.492765 12720 raft_consensus.cc:385] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.493170 12720 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.481015 12796 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:09:22.483749 12571 leader_election.cc:343] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:22.499958 12796 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:09:22.506402 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 54043.8us
I20260709 12:09:22.515478 12796 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.508301 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:22.551709 12903 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:22.555006 12956 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:09:22.559506 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 78619.9us
W20260709 12:09:22.585502 13000 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:09:22.580724 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:22.596391 12603 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:22.597795 12722 tablet_bootstrap.cc:492] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:22.598441 12722 ts_tablet_manager.cc:1403] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.479s	user 0.017s	sys 0.015s
I20260709 12:09:22.603716 12722 raft_consensus.cc:359] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.604703 12722 raft_consensus.cc:385] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.605036 12721 tablet_bootstrap.cc:654] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.605054 12722 raft_consensus.cc:740] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.606312 12722 consensus_queue.cc:260] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.606945 12796 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.609058 12722 ts_tablet_manager.cc:1434] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:09:22.609278 12796 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:22.610162 12722 tablet_bootstrap.cc:492] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:22.610942 12721 tablet_bootstrap.cc:492] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:22.611469 12721 ts_tablet_manager.cc:1403] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.161s	user 0.015s	sys 0.000s
I20260709 12:09:22.614678 12721 raft_consensus.cc:359] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.615454 12721 raft_consensus.cc:385] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.615792 12721 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.616613 12721 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.663820 12722 tablet_bootstrap.cc:654] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.668141 12721 ts_tablet_manager.cc:1434] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.056s	user 0.006s	sys 0.006s
I20260709 12:09:22.670976 12721 tablet_bootstrap.cc:492] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:22.681483 12721 tablet_bootstrap.cc:654] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.682379 12722 tablet_bootstrap.cc:492] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:22.683235 12722 ts_tablet_manager.cc:1403] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.073s	user 0.021s	sys 0.002s
I20260709 12:09:22.687211 12722 raft_consensus.cc:359] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.688341 12722 raft_consensus.cc:385] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.688663 12722 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.689465 12722 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.693727 12721 tablet_bootstrap.cc:492] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:22.694593 12721 ts_tablet_manager.cc:1403] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.003s
I20260709 12:09:22.695479 12722 ts_tablet_manager.cc:1434] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.005s
I20260709 12:09:22.698433 12722 tablet_bootstrap.cc:492] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:22.698267 12721 raft_consensus.cc:359] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.699146 12721 raft_consensus.cc:385] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.699481 12721 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.700316 12721 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.705407 12721 ts_tablet_manager.cc:1434] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.001s
I20260709 12:09:22.706959 12721 tablet_bootstrap.cc:492] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:22.715794 12721 tablet_bootstrap.cc:654] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.727567 12722 tablet_bootstrap.cc:654] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.748198 12721 tablet_bootstrap.cc:492] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:22.748988 12721 ts_tablet_manager.cc:1403] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.014s
I20260709 12:09:22.753316 12721 raft_consensus.cc:359] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.754230 12721 raft_consensus.cc:385] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.749969 12720 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.754601 12721 raft_consensus.cc:740] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.755553 12721 consensus_queue.cc:260] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.757368 12720 ts_tablet_manager.cc:1434] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.347s	user 0.016s	sys 0.000s
I20260709 12:09:22.757403 12721 ts_tablet_manager.cc:1434] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.008s	user 0.002s	sys 0.003s
I20260709 12:09:22.759388 12721 tablet_bootstrap.cc:492] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:22.762208 12722 tablet_bootstrap.cc:492] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:22.765164 12722 ts_tablet_manager.cc:1403] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.068s	user 0.022s	sys 0.008s
I20260709 12:09:22.764602 12720 tablet_bootstrap.cc:492] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:22.767419 12721 tablet_bootstrap.cc:654] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.776108 12722 raft_consensus.cc:359] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.777032 12722 raft_consensus.cc:385] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.777364 12722 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.778752 12720 tablet_bootstrap.cc:654] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.778357 12722 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.782649 12722 ts_tablet_manager.cc:1434] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.001s
I20260709 12:09:22.784308 12722 tablet_bootstrap.cc:492] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:22.785748 12720 tablet_bootstrap.cc:492] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:22.786382 12720 ts_tablet_manager.cc:1403] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.022s	user 0.011s	sys 0.006s
I20260709 12:09:22.788910 12721 tablet_bootstrap.cc:492] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:22.789309 12721 ts_tablet_manager.cc:1403] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.002s
I20260709 12:09:22.790979 12720 raft_consensus.cc:359] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.792331 12720 raft_consensus.cc:385] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.792717 12722 tablet_bootstrap.cc:654] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.792755 12720 raft_consensus.cc:740] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.799993 12721 raft_consensus.cc:359] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.800827 12721 raft_consensus.cc:385] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.794469 12720 consensus_queue.cc:260] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.801144 12721 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.801787 12721 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.805660 12720 ts_tablet_manager.cc:1434] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.019s	user 0.006s	sys 0.005s
I20260709 12:09:22.816154 12721 ts_tablet_manager.cc:1434] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.003s
I20260709 12:09:22.820537 12721 tablet_bootstrap.cc:492] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:22.822204 12720 tablet_bootstrap.cc:492] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:22.836974 12721 tablet_bootstrap.cc:654] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.848999 12722 tablet_bootstrap.cc:492] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:22.849646 12722 ts_tablet_manager.cc:1403] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.066s	user 0.025s	sys 0.028s
I20260709 12:09:22.849959 13004 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:22.850956 12720 tablet_bootstrap.cc:654] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.850798 13004 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.853031 12722 raft_consensus.cc:359] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.853909 12722 raft_consensus.cc:385] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.854246 12722 raft_consensus.cc:740] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:22.854363 13004 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:22.855104 12722 consensus_queue.cc:260] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.857373 12722 ts_tablet_manager.cc:1434] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:09:22.858670 12722 tablet_bootstrap.cc:492] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:22.868650 12722 tablet_bootstrap.cc:654] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:22.887204 12679 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.910835 12721 tablet_bootstrap.cc:492] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:22.911487 12721 ts_tablet_manager.cc:1403] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.091s	user 0.020s	sys 0.015s
I20260709 12:09:22.909873 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:22.917984 12643 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:22.920506 12956 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:22.920962 12956 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:22.921455 12956 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:22.921902 12722 tablet_bootstrap.cc:492] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:22.922981 12722 ts_tablet_manager.cc:1403] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.065s	user 0.022s	sys 0.026s
I20260709 12:09:22.922276 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:22.950435 13004 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:09:22.952696 13003 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:22.975314 12527 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:22.953294 13003 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.987000 13003 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:09:22.987766 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52724us
I20260709 12:09:22.991554 12964 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:22.991812 12721 raft_consensus.cc:359] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:22.992715 12721 raft_consensus.cc:385] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:22.993064 12721 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:23.006227 12721 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.008919 12721 ts_tablet_manager.cc:1434] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.097s	user 0.007s	sys 0.000s
I20260709 12:09:22.980870 12722 raft_consensus.cc:359] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.016016 12722 raft_consensus.cc:385] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.016674 12722 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:23.032723 12722 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.041214 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:23.043565 12672 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.048111 12892 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:23.048807 12892 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.069968 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:23.070747 12720 tablet_bootstrap.cc:492] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:23.070856 12531 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:23.071290 12720 ts_tablet_manager.cc:1403] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.250s	user 0.029s	sys 0.014s
I20260709 12:09:23.077811 12720 raft_consensus.cc:359] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.082134 12720 raft_consensus.cc:385] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.082819 13004 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:23.084583 12720 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:23.087924 12722 ts_tablet_manager.cc:1434] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.165s	user 0.011s	sys 0.000s
I20260709 12:09:23.079463 12606 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.089854 12720 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.089046 13004 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.089128 12963 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:23.107172 12963 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.109086 12601 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.156725 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:23.157557 13027 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:23.157634 12531 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:23.162242 13027 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.165234 13027 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:23.216944 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:23.301730 13032 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:23.302410 13032 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.304963 13032 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:23.345502 12672 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:23.347563 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:23.347709 12892 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:23.348095 12679 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:23.350728 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:23.351750 13010 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:23.351334 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:23.352187 13010 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:23.352591 13010 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.360275 13010 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.362689 13010 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:23.364689 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:23.365451 12679 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.368880 12963 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:23.372372 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:23.374548 12679 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:23.376315 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:23.377246 12963 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:23.377759 12963 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:09:23.378729 12963 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.387580 12892 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:23.388319 12892 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.390231 12892 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:23.391760 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:23.392428 12679 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:23.413230 12720 ts_tablet_manager.cc:1434] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.342s	user 0.010s	sys 0.001s
I20260709 12:09:23.417529 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:23.421238 13002 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:23.421861 13002 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.425107 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:23.425804 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:23.426136 12605 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:23.426539 12671 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:23.427289 13002 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:23.428131 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:09:23.432137 12643 leader_election.cc:343] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.434792 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:23.442692 12720 tablet_bootstrap.cc:492] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:23.442785 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:23.452514 12528 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:23.451346 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:23.449560 13005 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:23.456447 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:23.458058 12526 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 1: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:09:23.462350 12527 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.476151 12722 tablet_bootstrap.cc:492] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
W20260709 12:09:23.476991 12496 leader_election.cc:343] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:23.472770 12956 consensus_meta.cc:287] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.550s	user 0.014s	sys 0.013s
I20260709 12:09:23.477864 12956 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.484556 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:23.491748 12721 tablet_bootstrap.cc:492] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
W20260709 12:09:23.492642 12494 leader_election.cc:343] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.493285 12494 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:23.494704 12796 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:23.508685 12494 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:23.513604 12999 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260709 12:09:23.514717 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 54474.7us
I20260709 12:09:23.514775 12999 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:23.515478 12999 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.515872 12646 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:23.518220 13003 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:23.509419 12956 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:23.514225 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:09:23.523947 12494 leader_election.cc:343] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.522217 12527 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:23.520021 13003 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:09:23.529770 12720 tablet_bootstrap.cc:654] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:23.530555 12796 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:23.532927 12796 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:09:23.508759 13032 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260709 12:09:23.534003 13004 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:23.535470 12646 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:23.533931 12796 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:23.525619 12496 leader_election.cc:343] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.543018 12960 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:23.543540 12960 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:23.544116 12960 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:23.550163 12494 leader_election.cc:343] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.550689 12494 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:23.551524 12781 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:09:23.555722 12494 leader_election.cc:343] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.537377 12999 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.560484 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:23.561407 12527 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 1: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:09:23.565275 12572 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:09:23.566258 12477 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:23.571864 12426 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58992 (request call id 66) took 1063 ms. Trace:
I20260709 12:09:23.573944 12960 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:23.577093 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 51380.9us
I20260709 12:09:23.581794 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:23.578751 12960 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:23.561987 12496 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:23.589917 12721 tablet_bootstrap.cc:654] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:23.572562 12426 rpcz_store.cc:276] 0709 12:09:22.509657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:22.510933 (+  1276us) service_pool.cc:224] Handling call
0709 12:09:23.571636 (+1060703us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4,"reported_tablets":2}
I20260709 12:09:23.655624 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:23.656334 12529 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.651418 13027 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:23.658460 13027 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:23.658829 13027 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.659265 12427 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58980 (request call id 66) took 1134 ms. Trace:
I20260709 12:09:23.659853 12427 rpcz_store.cc:276] 0709 12:09:22.519566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:22.520497 (+   931us) service_pool.cc:224] Handling call
0709 12:09:23.625872 (+1105375us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13723,"reported_tablets":4}
I20260709 12:09:23.615655 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:23.611630 12999 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:23.588601 12963 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:23.617417 13037 raft_consensus.cc:2764] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
W20260709 12:09:23.672152 12643 leader_election.cc:343] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.616019 12429 catalog_manager.cc:5697] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:23.685946 12722 tablet_bootstrap.cc:654] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:23.615291 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:09:23.666215 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 98040.3us
I20260709 12:09:23.674561 12531 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.691984 12429 catalog_manager.cc:5697] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:23.703401 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:23.704145 12605 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:23.713575 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:23.706372 12600 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.707340 12529 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:23.720917 13005 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:23.714687 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:23.738168 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:23.726018 13005 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:23.739334 13005 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.727799 12606 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.748030 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:23.749106 12646 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:23.750330 12956 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:23.750857 12956 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:23.751754 12956 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.749627 13004 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:09:23.754581 12643 leader_election.cc:343] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.781191 13027 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.809617 12531 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
W20260709 12:09:23.818938 13040 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 12:09:23.824054 13045 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:09:23.824229 12903 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:23.826045 13027 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:23.826804 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:09:23.828531 12494 leader_election.cc:343] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:23.834744 12600 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:23.835155 12606 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
W20260709 12:09:23.840101 13043 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:09:23.840344 12643 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:23.841274 13037 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:23.841787 13037 raft_consensus.cc:697] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:23.840130 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:23.843191 12680 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:23.849182 12494 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:23.836784 13005 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.853524 13002 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:23.856452 12677 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 1: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:09:23.856331 13002 raft_consensus.cc:697] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:09:23.857488 13005 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:23.860703 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:23.861657 12606 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 1: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:09:23.868160 12680 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:23.871430 12720 tablet_bootstrap.cc:492] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:23.872285 12720 ts_tablet_manager.cc:1403] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.430s	user 0.034s	sys 0.021s
I20260709 12:09:23.842940 13037 consensus_queue.cc:237] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.902663 12496 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:23.904186 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:23.906373 12999 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:23.906904 12999 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:09:23.907145 12646 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:23.907788 12999 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.909339 12728 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:09:23.857590 13002 consensus_queue.cc:237] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.932690 12963 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:23.932458 12720 raft_consensus.cc:359] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.935031 12720 raft_consensus.cc:385] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.935389 12720 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
W20260709 12:09:23.935958 13050 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:09:23.936163 12720 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.949687 12911 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:23.952490 12720 ts_tablet_manager.cc:1434] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.080s	user 0.008s	sys 0.002s
I20260709 12:09:23.953758 12720 tablet_bootstrap.cc:492] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:23.955119 12721 tablet_bootstrap.cc:492] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:23.955611 12721 ts_tablet_manager.cc:1403] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.488s	user 0.014s	sys 0.036s
I20260709 12:09:23.964346 12721 raft_consensus.cc:359] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:23.966377 12721 raft_consensus.cc:385] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:23.967013 12721 raft_consensus.cc:740] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:23.973896 12722 tablet_bootstrap.cc:492] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:23.974586 12722 ts_tablet_manager.cc:1403] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.499s	user 0.034s	sys 0.028s
I20260709 12:09:23.983536 12721 consensus_queue.cc:260] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.003041 12721 ts_tablet_manager.cc:1434] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.047s	user 0.010s	sys 0.002s
I20260709 12:09:24.017745 12721 tablet_bootstrap.cc:492] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
W20260709 12:09:24.151778 12627 debug-util.cc:398] Leaking SignalData structure 0x7b080016c6a0 after lost signal to thread 11321
W20260709 12:09:24.152967 12627 debug-util.cc:398] Leaking SignalData structure 0x7b080016cd80 after lost signal to thread 12463
W20260709 12:09:24.154487 12627 debug-util.cc:398] Leaking SignalData structure 0x7b080009d9a0 after lost signal to thread 12555
W20260709 12:09:24.155431 12627 debug-util.cc:398] Leaking SignalData structure 0x7b080013d760 after lost signal to thread 12630
W20260709 12:09:24.156281 12627 debug-util.cc:398] Leaking SignalData structure 0x7b080022c1a0 after lost signal to thread 12704
W20260709 12:09:24.176139 12494 consensus_peers.cc:597] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:24.175299 12722 raft_consensus.cc:359] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.177681 12722 raft_consensus.cc:385] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.192095 12722 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.191939 13038 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:09:24.192993 12722 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.196069 12722 ts_tablet_manager.cc:1434] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.221s	user 0.005s	sys 0.008s
I20260709 12:09:24.197603 12722 tablet_bootstrap.cc:492] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:24.190251 12720 tablet_bootstrap.cc:654] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.178263 12748 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.199869 12748 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.202198 12748 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:24.203469 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:24.203884 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:24.204434 12677 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 1: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:09:24.207031 13038 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.215590 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:24.216522 12748 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.220782 12426 catalog_manager.cc:5697] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:24.207517 13038 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.223048 12426 catalog_manager.cc:5697] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:24.224401 13038 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:24.227001 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:24.228504 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:09:24.229192 12643 leader_election.cc:343] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:24.229269 12529 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:24.237558 12646 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:24.239064 13004 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:24.239601 13004 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:24.240034 13004 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.246090 12721 tablet_bootstrap.cc:654] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.251375 12748 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:24.255432 12427 catalog_manager.cc:5697] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 0 to 1, leader changed from <none> to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:24.265717 13004 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.268491 12722 tablet_bootstrap.cc:654] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.270254 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:24.270849 13004 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:09:24.272158 12643 leader_election.cc:343] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:24.273483 12720 tablet_bootstrap.cc:492] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:24.273965 12720 ts_tablet_manager.cc:1403] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.320s	user 0.018s	sys 0.006s
I20260709 12:09:24.272501 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:24.274847 12526 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.278458 13002 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.279088 13002 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.281132 13002 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:24.281893 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:24.282732 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:09:24.284050 12494 leader_election.cc:343] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:24.284539 12496 leader_election.cc:343] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:24.284940 12526 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:24.285250 12496 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:24.287722 13002 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:24.297103 12720 raft_consensus.cc:359] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.297894 12720 raft_consensus.cc:385] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:09:24.297957 12643 consensus_peers.cc:597] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:24.298243 12720 raft_consensus.cc:740] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.298933 12720 consensus_queue.cc:260] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.299597 12721 tablet_bootstrap.cc:492] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:24.300050 12721 ts_tablet_manager.cc:1403] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.295s	user 0.012s	sys 0.004s
I20260709 12:09:24.303036 12720 ts_tablet_manager.cc:1434] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.029s	user 0.003s	sys 0.004s
I20260709 12:09:24.303273 12721 raft_consensus.cc:359] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.303900 12721 raft_consensus.cc:385] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.304075 12721 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.304199 12720 tablet_bootstrap.cc:492] T a92d1d91cce04cec98366aa6021356df P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:24.304687 12721 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.306509 12646 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:24.312433 12720 tablet_bootstrap.cc:654] T a92d1d91cce04cec98366aa6021356df P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.320120 12722 tablet_bootstrap.cc:492] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:24.320676 12722 ts_tablet_manager.cc:1403] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.124s	user 0.013s	sys 0.012s
I20260709 12:09:24.323765 12722 raft_consensus.cc:359] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.324939 12722 raft_consensus.cc:385] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.325351 12722 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.326454 12722 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.329334 12722 ts_tablet_manager.cc:1434] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.001s
I20260709 12:09:24.331219 13039 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:24.331939 12721 ts_tablet_manager.cc:1434] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.032s	user 0.008s	sys 0.000s
I20260709 12:09:24.333475 12722 tablet_bootstrap.cc:492] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:24.333468 12721 tablet_bootstrap.cc:492] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:24.338531 12720 tablet_bootstrap.cc:492] T a92d1d91cce04cec98366aa6021356df P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:24.338670 13039 raft_consensus.cc:697] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:24.339084 12720 ts_tablet_manager.cc:1403] T a92d1d91cce04cec98366aa6021356df P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.035s	user 0.018s	sys 0.000s
I20260709 12:09:24.339726 13039 consensus_queue.cc:237] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.341670 12722 tablet_bootstrap.cc:654] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.342274 12720 raft_consensus.cc:359] T a92d1d91cce04cec98366aa6021356df P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.342979 12720 raft_consensus.cc:385] T a92d1d91cce04cec98366aa6021356df P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.343257 12720 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.343436 12721 tablet_bootstrap.cc:654] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.344084 12720 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.347047 12720 ts_tablet_manager.cc:1434] T a92d1d91cce04cec98366aa6021356df P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:09:24.348377 12720 tablet_bootstrap.cc:492] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:24.350574 12429 catalog_manager.cc:5697] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:24.355821 12429 catalog_manager.cc:5697] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:24.355810 12720 tablet_bootstrap.cc:654] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.358505 12721 tablet_bootstrap.cc:492] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:24.359019 12721 ts_tablet_manager.cc:1403] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.026s	user 0.011s	sys 0.002s
I20260709 12:09:24.361852 12721 raft_consensus.cc:359] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.362533 12721 raft_consensus.cc:385] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.362804 12721 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.363442 12721 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.365073 12721 ts_tablet_manager.cc:1434] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.000s
I20260709 12:09:24.365927 12721 tablet_bootstrap.cc:492] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:24.368047 12722 tablet_bootstrap.cc:492] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:24.368643 12722 ts_tablet_manager.cc:1403] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.035s	user 0.013s	sys 0.006s
I20260709 12:09:24.375875 12722 raft_consensus.cc:359] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.376664 12722 raft_consensus.cc:385] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.376753 12721 tablet_bootstrap.cc:654] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.377018 12722 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.379801 12911 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.380283 12911 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.382531 12911 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:24.386932 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:24.387650 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:09:24.388803 12496 leader_election.cc:343] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:24.391721 12720 tablet_bootstrap.cc:492] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:24.392203 12720 ts_tablet_manager.cc:1403] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.044s	user 0.011s	sys 0.000s
W20260709 12:09:24.393277 12494 leader_election.cc:343] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:24.393705 12494 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:24.395212 12720 raft_consensus.cc:359] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.396015 12720 raft_consensus.cc:385] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.396291 12720 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.396709 12911 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:24.396950 12720 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.399212 12720 ts_tablet_manager.cc:1434] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:09:24.400195 12720 tablet_bootstrap.cc:492] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:24.401137 12427 catalog_manager.cc:5697] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:24.402580 12722 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.415243 12722 ts_tablet_manager.cc:1434] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.046s	user 0.003s	sys 0.005s
I20260709 12:09:24.416280 12722 tablet_bootstrap.cc:492] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:24.425624 12721 tablet_bootstrap.cc:492] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:24.426254 12721 ts_tablet_manager.cc:1403] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.061s	user 0.011s	sys 0.010s
I20260709 12:09:24.440491 12721 raft_consensus.cc:359] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.441190 12721 raft_consensus.cc:385] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.441447 12721 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.442193 12721 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.444226 12720 tablet_bootstrap.cc:654] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.454821 12721 ts_tablet_manager.cc:1434] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.028s	user 0.007s	sys 0.001s
I20260709 12:09:24.467381 12721 tablet_bootstrap.cc:492] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:24.478458 13039 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.479064 13039 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.481679 13039 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:24.481889 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:24.482990 12525 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:24.485248 12646 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:24.487661 13004 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:24.488080 13004 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:24.488837 13004 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.490952 13002 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:24.491839 12602 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.495668 12748 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.496197 12748 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.498097 12748 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:24.500304 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:24.505383 12426 catalog_manager.cc:5697] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:24.515705 12911 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:24.520128 12722 tablet_bootstrap.cc:654] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.522945 12720 tablet_bootstrap.cc:492] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:24.525738 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:24.525420 12720 ts_tablet_manager.cc:1403] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.125s	user 0.013s	sys 0.000s
I20260709 12:09:24.523808 12721 tablet_bootstrap.cc:654] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.528137 13004 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:24.537408 12643 leader_election.cc:343] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:24.537464 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:24.538596 12529 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 1: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:09:24.540169 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:24.540903 12529 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.546633 12572 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:24.547441 12748 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.553450 12748 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:24.556638 12796 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:09:24.531556 12720 raft_consensus.cc:359] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.559695 12529 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:24.560204 12720 raft_consensus.cc:385] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.560523 12720 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.561221 12720 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.562290 13037 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:09:24.566026 12646 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:24.567625 13039 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:24.568145 13039 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:24.571739 12748 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.572209 12748 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.576289 12748 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:24.578238 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:24.579104 12680 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:24.583850 13002 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:24.586074 13003 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:24.588587 12720 ts_tablet_manager.cc:1434] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.061s	user 0.007s	sys 0.000s
I20260709 12:09:24.568930 13039 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.606822 13004 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:24.624220 12720 tablet_bootstrap.cc:492] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:24.624712 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:09:24.628077 12643 leader_election.cc:343] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:24.633781 12721 tablet_bootstrap.cc:492] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:24.634182 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:24.634580 12721 ts_tablet_manager.cc:1403] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.171s	user 0.038s	sys 0.021s
I20260709 12:09:24.635087 12526 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:24.638173 12721 raft_consensus.cc:359] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.638919 12721 raft_consensus.cc:385] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.639333 12721 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.643455 12721 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.662834 12721 ts_tablet_manager.cc:1434] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.001s
I20260709 12:09:24.670403 12571 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:24.674630 12721 tablet_bootstrap.cc:492] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:24.676834 12722 tablet_bootstrap.cc:492] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:24.677299 12722 ts_tablet_manager.cc:1403] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.261s	user 0.009s	sys 0.021s
I20260709 12:09:24.680641 12963 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:24.681180 12963 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:24.681514 12963 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.679147 13031 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.688305 13031 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.731920 13031 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:24.746182 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:09:24.751812 12477 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:24.827723 12722 raft_consensus.cc:359] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.833508 12722 raft_consensus.cc:385] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:24.833856 12722 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:24.844436 12722 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.855370 12722 ts_tablet_manager.cc:1434] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.178s	user 0.009s	sys 0.002s
I20260709 12:09:24.856736 12722 tablet_bootstrap.cc:492] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
W20260709 12:09:24.870111 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 138683us
I20260709 12:09:24.874800 12720 tablet_bootstrap.cc:654] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.868453 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:24.894321 12600 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:24.892274 13086 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.884531 13004 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 12:09:24.914283 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:24.925026 12722 tablet_bootstrap.cc:654] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.943001 12721 tablet_bootstrap.cc:654] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:24.949981 12963 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.947249 13096 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.957305 13096 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.959743 13096 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:24.926533 13086 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.975442 13038 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:24.975992 13038 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:24.976383 12426 catalog_manager.cc:5697] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:24.978017 13038 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:24.981508 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:25.003986 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:25.006417 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:09:25.008603 12643 leader_election.cc:343] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:25.018947 13086 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:25.025597 12720 tablet_bootstrap.cc:492] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:25.026094 12720 ts_tablet_manager.cc:1403] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.407s	user 0.034s	sys 0.030s
I20260709 12:09:24.990793 12963 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:25.011862 12528 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:25.045154 12643 leader_election.cc:343] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:25.057660 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:25.061996 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:25.062758 12672 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:25.066530 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:25.072402 12673 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:25.096297 12496 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:25.101209 12999 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:25.101648 12999 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:25.102015 12999 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:25.107709 12494 leader_election.cc:343] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:25.109326 12672 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:25.116807 12722 tablet_bootstrap.cc:492] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:25.117338 12722 ts_tablet_manager.cc:1403] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.261s	user 0.031s	sys 0.011s
I20260709 12:09:25.117393 12720 raft_consensus.cc:359] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.118165 12720 raft_consensus.cc:385] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.118397 12720 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:25.119004 12720 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.122354 12571 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:25.125521 12721 tablet_bootstrap.cc:492] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:25.126066 12721 ts_tablet_manager.cc:1403] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.452s	user 0.020s	sys 0.026s
I20260709 12:09:25.128005 12722 raft_consensus.cc:359] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.130439 12722 raft_consensus.cc:385] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.131011 13092 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:25.133852 12722 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:25.133450 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:25.134953 12964 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:25.136572 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:25.144814 12964 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:09:25.145967 12964 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.137766 12720 ts_tablet_manager.cc:1434] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.111s	user 0.002s	sys 0.005s
I20260709 12:09:25.131858 13092 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.137679 12528 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:25.133859 12721 raft_consensus.cc:359] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.149290 12529 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:25.149461 12721 raft_consensus.cc:385] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.149816 12721 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:25.153373 13087 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:25.151863 12721 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.163069 12721 ts_tablet_manager.cc:1434] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.037s	user 0.009s	sys 0.000s
I20260709 12:09:25.170764 12720 tablet_bootstrap.cc:492] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:25.176512 13092 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:25.177644 12646 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:25.178855 13087 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:25.179368 13087 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:25.179834 13087 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:25.180809 12722 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.184351 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:25.194422 13031 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:25.202669 13095 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:25.209939 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:09:25.212527 12494 leader_election.cc:343] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:25.215349 12721 tablet_bootstrap.cc:492] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:25.220356 12722 ts_tablet_manager.cc:1434] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.068s	user 0.007s	sys 0.003s
I20260709 12:09:25.193432 12999 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.234302 12531 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 1: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:09:25.236974 12646 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:25.238626 13003 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:25.238912 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:09:25.240360 12494 leader_election.cc:343] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:25.241671 12722 tablet_bootstrap.cc:492] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:25.243359 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:25.241793 12999 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:25.256789 12720 tablet_bootstrap.cc:654] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:25.238199 13087 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.261400 12678 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 1: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:09:25.267828 12496 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:25.269989 12999 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:25.282696 13003 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:25.283895 12722 tablet_bootstrap.cc:654] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:25.287562 12429 catalog_manager.cc:5697] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 0 to 1, leader changed from <none> to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:25.292194 12999 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:25.298746 12600 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:25.300299 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:09:25.302023 12643 leader_election.cc:343] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:25.302546 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:25.297930 13087 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:25.296485 12671 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:25.310664 12721 tablet_bootstrap.cc:654] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:25.320070 12720 tablet_bootstrap.cc:492] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:25.320616 12720 ts_tablet_manager.cc:1403] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.154s	user 0.011s	sys 0.005s
I20260709 12:09:25.324514 12528 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:25.326390 12496 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:25.327275 12999 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:25.327739 13094 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:25.327919 12999 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:09:25.328275 13094 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.329208 12999 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.311062 13087 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 12:09:25.332382 13094 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:25.333148 12646 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:25.334605 13003 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:25.335129 13003 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:25.335510 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:25.336163 12529 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:25.335984 13003 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.337548 12646 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:25.338866 13095 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:25.339246 13095 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:25.339614 13095 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:25.349298 12722 tablet_bootstrap.cc:492] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:25.350001 12722 ts_tablet_manager.cc:1403] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.128s	user 0.010s	sys 0.005s
I20260709 12:09:25.353228 13095 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.339466 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:09:25.355301 12643 leader_election.cc:343] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:25.355726 13095 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:25.357697 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:25.358371 12529 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:25.373322 12529 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:25.358062 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:25.376938 12646 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:25.377815 13094 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:25.378659 13094 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:25.379743 13094 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:25.385586 12643 leader_election.cc:343] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:25.404837 12722 raft_consensus.cc:359] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.405887 12722 raft_consensus.cc:385] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.424957 12722 raft_consensus.cc:740] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:25.428462 12721 tablet_bootstrap.cc:492] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:25.429098 12721 ts_tablet_manager.cc:1403] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.233s	user 0.014s	sys 0.001s
I20260709 12:09:25.361829 12720 raft_consensus.cc:359] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.430639 12720 raft_consensus.cc:385] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.431066 12720 raft_consensus.cc:740] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:25.432299 12720 consensus_queue.cc:260] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.425902 12722 consensus_queue.cc:260] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.441434 12748 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:25.442099 12748 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.452852 12748 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:25.454433 12720 ts_tablet_manager.cc:1434] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.134s	user 0.010s	sys 0.001s
I20260709 12:09:25.455936 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:25.464620 12722 ts_tablet_manager.cc:1434] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.114s	user 0.010s	sys 0.000s
I20260709 12:09:25.466974 12722 tablet_bootstrap.cc:492] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:25.467844 13097 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:25.468554 13097 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.469125 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:25.469806 12529 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
I20260709 12:09:25.472436 13097 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:25.475299 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:25.476419 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:09:25.477142 12571 leader_election.cc:343] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:25.478358 12496 leader_election.cc:343] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:25.479308 12494 leader_election.cc:343] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:25.479727 12494 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:25.482375 12720 tablet_bootstrap.cc:492] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:25.486275 12572 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:09:25.488536 12748 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:25.488982 12748 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:25.489346 12748 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:25.487010 13097 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:25.503022 12721 raft_consensus.cc:359] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.503731 12721 raft_consensus.cc:385] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.503986 12721 raft_consensus.cc:740] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:25.504487 12721 consensus_queue.cc:260] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.524809 12748 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.520763 12429 catalog_manager.cc:5697] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:25.524204 12427 catalog_manager.cc:5697] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:25.528868 12748 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:25.530222 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:25.530843 12678 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:25.535373 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:25.536006 12529 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:25.538321 12721 ts_tablet_manager.cc:1434] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.109s	user 0.004s	sys 0.004s
I20260709 12:09:25.548022 12678 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:25.549466 12571 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:25.550267 12748 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:25.550997 12748 raft_consensus.cc:697] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:09:25.551788 12748 consensus_queue.cc:237] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.556550 12721 tablet_bootstrap.cc:492] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:25.561379 12722 tablet_bootstrap.cc:654] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:25.562500 12720 tablet_bootstrap.cc:654] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:25.571789 12721 tablet_bootstrap.cc:654] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:25.548512 12529 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:25.584259 12720 tablet_bootstrap.cc:492] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:25.584532 12722 tablet_bootstrap.cc:492] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:25.584987 12722 ts_tablet_manager.cc:1403] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.118s	user 0.016s	sys 0.000s
I20260709 12:09:25.585605 12720 ts_tablet_manager.cc:1403] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.107s	user 0.012s	sys 0.006s
I20260709 12:09:25.587956 12722 raft_consensus.cc:359] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.588738 12722 raft_consensus.cc:385] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.589058 12722 raft_consensus.cc:740] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:25.588385 12720 raft_consensus.cc:359] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.590281 12720 raft_consensus.cc:385] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.590157 12722 consensus_queue.cc:260] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.599004 12721 tablet_bootstrap.cc:492] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:25.590904 12720 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:25.610592 12721 ts_tablet_manager.cc:1403] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.054s	user 0.010s	sys 0.005s
I20260709 12:09:25.610844 12720 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.613245 12720 ts_tablet_manager.cc:1434] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.027s	user 0.000s	sys 0.004s
I20260709 12:09:25.613673 12721 raft_consensus.cc:359] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.614466 12721 raft_consensus.cc:385] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.619707 12721 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:25.614404 12720 tablet_bootstrap.cc:492] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:25.620558 12721 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.622999 12721 ts_tablet_manager.cc:1434] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:09:25.624006 12721 tablet_bootstrap.cc:492] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:25.628748 12720 tablet_bootstrap.cc:654] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:25.632316 12722 ts_tablet_manager.cc:1434] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.047s	user 0.017s	sys 0.005s
I20260709 12:09:25.633766 12721 tablet_bootstrap.cc:654] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:25.646270 12722 tablet_bootstrap.cc:492] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:25.658419 12722 tablet_bootstrap.cc:654] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:25.668977 12720 tablet_bootstrap.cc:492] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:25.669278 12721 tablet_bootstrap.cc:492] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:25.669636 12720 ts_tablet_manager.cc:1403] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.056s	user 0.009s	sys 0.010s
I20260709 12:09:25.669709 12721 ts_tablet_manager.cc:1403] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.046s	user 0.013s	sys 0.000s
I20260709 12:09:25.673059 12721 raft_consensus.cc:359] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.673851 12721 raft_consensus.cc:385] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.674046 12720 raft_consensus.cc:359] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.674197 12721 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:25.675318 12720 raft_consensus.cc:385] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.675757 12720 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:25.676769 12720 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.677979 12722 tablet_bootstrap.cc:492] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:25.678517 12722 ts_tablet_manager.cc:1403] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.033s	user 0.007s	sys 0.005s
I20260709 12:09:25.679566 12720 ts_tablet_manager.cc:1434] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.001s
I20260709 12:09:25.684126 12720 tablet_bootstrap.cc:492] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:25.682435 12721 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.687299 12721 ts_tablet_manager.cc:1434] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:09:25.681493 12722 raft_consensus.cc:359] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.688280 12722 raft_consensus.cc:385] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.688715 12722 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:25.689548 12722 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.696852 12722 ts_tablet_manager.cc:1434] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:09:25.702852 12722 tablet_bootstrap.cc:492] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:25.710237 12720 tablet_bootstrap.cc:654] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:25.716732 12722 tablet_bootstrap.cc:654] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:25.721450 13010 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:25.722002 13010 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.739696 12721 tablet_bootstrap.cc:492] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:25.743011 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:25.744167 12678 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 1: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:09:25.747784 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:25.747689 13031 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:25.748126 12720 tablet_bootstrap.cc:492] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:25.748693 12720 ts_tablet_manager.cc:1403] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.065s	user 0.009s	sys 0.006s
I20260709 12:09:25.749832 12572 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:25.751451 12720 raft_consensus.cc:359] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.752133 12720 raft_consensus.cc:385] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.752406 12720 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:25.758096 12721 tablet_bootstrap.cc:654] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:25.758165 12722 tablet_bootstrap.cc:492] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:25.758881 12722 ts_tablet_manager.cc:1403] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.056s	user 0.009s	sys 0.006s
I20260709 12:09:25.759102 12720 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.758837 12964 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:25.778826 12964 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:25.779067 13087 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:25.779556 13087 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.764756 13010 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:09:25.831043 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50976.8us
I20260709 12:09:25.833710 12720 ts_tablet_manager.cc:1434] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.085s	user 0.010s	sys 0.003s
I20260709 12:09:25.833902 12722 raft_consensus.cc:359] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.834954 12722 raft_consensus.cc:385] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.835156 12720 tablet_bootstrap.cc:492] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:25.835330 12722 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:25.837597 12722 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.838255 13087 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:25.856003 12722 ts_tablet_manager.cc:1434] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.097s	user 0.022s	sys 0.012s
I20260709 12:09:25.861336 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:25.862095 12528 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:25.862529 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:25.863137 12748 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:09:25.864737 12646 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:25.869956 13094 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:25.870445 13094 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:25.870818 13094 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:25.876123 12722 tablet_bootstrap.cc:492] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:25.871557 13088 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:09:25.884495 12606 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:25.882653 12643 leader_election.cc:343] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:25.900060 12721 tablet_bootstrap.cc:492] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:25.900600 12721 ts_tablet_manager.cc:1403] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.161s	user 0.018s	sys 0.017s
W20260709 12:09:25.903622 12643 consensus_peers.cc:597] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:25.913255 13002 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:25.917968 12748 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:25.918699 12748 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.934402 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:09:25.936983 12572 leader_election.cc:343] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:25.927841 12427 catalog_manager.cc:5697] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:25.942881 12748 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:25.944698 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:09:25.946080 12571 leader_election.cc:343] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:25.945205 13126 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:25.946525 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:09:25.950029 12477 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:25.949963 13031 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:25.942255 13088 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260709 12:09:25.966722 12721 raft_consensus.cc:359] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.973507 12721 raft_consensus.cc:385] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:25.973896 12721 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:25.976456 12721 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:25.982421 12722 tablet_bootstrap.cc:654] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:25.983777 12721 ts_tablet_manager.cc:1434] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.083s	user 0.006s	sys 0.001s
I20260709 12:09:25.970801 12999 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
I20260709 12:09:25.992947 12721 tablet_bootstrap.cc:492] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:25.999393 12964 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:26.000144 12964 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:26.001119 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 55208.3us
I20260709 12:09:26.046660 12720 tablet_bootstrap.cc:654] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:26.083328 13141 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:26.083624 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:26.083962 13141 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:26.077366 12964 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:26.052963 13094 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:26.116927 13002 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:26.304495 13002 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:26.323104 13141 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:09:26.370707 13135 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 12:09:26.378999 13002 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:26.380169 13094 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:26.391701 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:26.393556 12721 tablet_bootstrap.cc:654] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
W20260709 12:09:26.408285 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 317467us
I20260709 12:09:26.420969 12720 tablet_bootstrap.cc:492] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:26.421522 12720 ts_tablet_manager.cc:1403] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.587s	user 0.028s	sys 0.007s
I20260709 12:09:26.436308 12722 tablet_bootstrap.cc:492] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:26.436769 12722 ts_tablet_manager.cc:1403] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.562s	user 0.011s	sys 0.006s
I20260709 12:09:26.417209 13132 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:26.434273 13138 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:26.451692 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:26.444996 13132 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:26.455489 13138 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:26.524313 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 72241.3us
I20260709 12:09:26.556960 12722 raft_consensus.cc:359] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:26.578043 12722 raft_consensus.cc:385] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:26.578497 12722 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:26.522735 12720 raft_consensus.cc:359] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:26.581566 12720 raft_consensus.cc:385] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:26.581871 12720 raft_consensus.cc:740] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:26.582470 12722 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:26.583418 12720 consensus_queue.cc:260] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:26.600991 12722 ts_tablet_manager.cc:1434] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.164s	user 0.007s	sys 0.007s
I20260709 12:09:26.616326 12720 ts_tablet_manager.cc:1434] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.194s	user 0.012s	sys 0.015s
I20260709 12:09:26.605228 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:26.628108 12720 tablet_bootstrap.cc:492] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
W20260709 12:09:26.643105 12494 leader_election.cc:343] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:26.651722 12721 tablet_bootstrap.cc:492] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:26.698871 12721 ts_tablet_manager.cc:1403] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.706s	user 0.014s	sys 0.028s
I20260709 12:09:26.637530 12527 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:26.719805 13138 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:09:26.721668 13157 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:09:26.725247 13132 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:26.725939 13160 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:26.726532 13160 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:26.727417 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:09:26.730885 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 86030.2us
I20260709 12:09:26.749184 13160 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:26.753063 12721 raft_consensus.cc:359] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:26.754925 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:26.754011 12721 raft_consensus.cc:385] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:26.760776 12721 raft_consensus.cc:740] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:26.761597 12721 consensus_queue.cc:260] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:26.763437 12646 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:26.767436 13129 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:26.768267 13129 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:26.768803 13129 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:26.786777 12646 leader_election.cc:343] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:26.806598 12720 tablet_bootstrap.cc:654] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:26.799237 12721 ts_tablet_manager.cc:1434] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.100s	user 0.005s	sys 0.004s
I20260709 12:09:26.817286 13132 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:26.826756 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:26.817848 13132 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:26.840029 13132 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:26.842957 13129 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:26.848726 13129 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:26.854564 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:26.855544 12530 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:26.858453 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:26.859192 12677 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 0.
W20260709 12:09:26.937732 12494 leader_election.cc:343] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:26.947616 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:26.857988 13127 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.041s
W20260709 12:09:26.950700 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 89773.3us
I20260709 12:09:26.964267 12571 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:26.973320 13173 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:26.973748 13173 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:26.974059 13173 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:26.975950 12720 tablet_bootstrap.cc:492] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:26.983275 12720 ts_tablet_manager.cc:1403] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.355s	user 0.034s	sys 0.006s
I20260709 12:09:26.988574 12721 tablet_bootstrap.cc:492] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:26.997463 12676 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:27.003656 12720 raft_consensus.cc:359] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.004720 12720 raft_consensus.cc:385] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:27.005111 12720 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:27.016695 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:27.017493 12524 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:27.021795 13126 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:09:27.022514 13126 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:27.030628 12494 leader_election.cc:343] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:27.049278 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:27.059937 12496 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:09:27.060808 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 93983.9us
I20260709 12:09:27.061388 12426 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58980 (request call id 83) took 1086 ms. Trace:
I20260709 12:09:27.061781 12426 rpcz_store.cc:276] 0709 12:09:25.974964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:25.975106 (+   142us) service_pool.cc:224] Handling call
0709 12:09:27.061326 (+1086220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":387952,"reported_tablets":2}
I20260709 12:09:27.006002 12720 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.064335 12429 catalog_manager.cc:5697] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 0 to 1, leader changed from <none> to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:27.078403 13133 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:27.078940 13133 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:27.079315 13133 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:27.085633 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:27.092533 12721 tablet_bootstrap.cc:654] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:27.092623 12748 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:27.094503 12748 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.096184 12720 ts_tablet_manager.cc:1434] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.113s	user 0.014s	sys 0.017s
I20260709 12:09:27.104455 13173 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:27.114110 12496 leader_election.cc:343] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:27.116844 12496 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:27.125116 12524 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:27.114477 13158 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:09:27.128002 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:27.128612 13158 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.136687 12646 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:27.146963 12720 tablet_bootstrap.cc:492] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:27.150483 13131 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:27.150926 13173 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:27.152945 13126 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:27.164229 13038 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:27.165491 13038 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:09:27.185293 12477 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:27.196237 12748 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:27.198747 12572 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:27.202697 12748 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:27.173269 13133 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:27.206557 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 48818.5us
I20260709 12:09:27.208876 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:27.216182 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:09:27.218499 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 81907.5us
I20260709 12:09:27.179409 12530 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:27.213997 13031 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.069s
I20260709 12:09:27.219389 13158 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:27.333688 12605 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:27.283700 13161 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:09:27.338531 13161 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.283493 13153 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:27.339581 13153 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:09:27.284691 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:27.267022 12722 tablet_bootstrap.cc:492] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:27.287905 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:27.268329 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:27.263343 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:27.353901 12529 raft_consensus.cc:2393] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 1: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:09:27.218014 13038 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.340970 13161 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:27.377585 13097 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:27.378399 13097 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.380072 13133 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:27.381223 13097 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:27.386837 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:27.387177 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:09:27.388332 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 101180us
W20260709 12:09:27.390758 12643 leader_election.cc:343] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:27.451900 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 61185.3us
I20260709 12:09:27.390327 12598 raft_consensus.cc:2393] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 1: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:09:27.389549 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:27.483093 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:27.477759 13131 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:09:27.455452 12597 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
W20260709 12:09:27.529088 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 64547.4us
W20260709 12:09:27.533975 12643 leader_election.cc:343] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:27.533840 13161 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:09:27.537407 13161 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.539687 13161 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:27.540064 12721 tablet_bootstrap.cc:492] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:27.540462 12964 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:27.541092 12964 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.540515 12721 ts_tablet_manager.cc:1403] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.552s	user 0.026s	sys 0.013s
I20260709 12:09:27.529983 12528 raft_consensus.cc:2393] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 1: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:09:27.552507 12722 tablet_bootstrap.cc:654] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:27.549613 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:27.555433 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:27.646930 13031 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:27.557361 13097 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:27.545719 13131 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.649012 12721 raft_consensus.cc:359] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.712154 12721 raft_consensus.cc:385] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:27.712538 12721 raft_consensus.cc:740] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:27.713271 12721 consensus_queue.cc:260] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.648051 13161 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:27.751546 13162 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:27.712381 12675 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:27.752130 13162 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.775422 12721 ts_tablet_manager.cc:1434] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.233s	user 0.006s	sys 0.002s
I20260709 12:09:27.715437 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:09:27.779523 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 130136us
W20260709 12:09:27.785619 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 140829us
I20260709 12:09:27.748569 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:27.793148 12720 tablet_bootstrap.cc:654] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:27.712723 13031 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.735273 13097 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:27.801527 12496 leader_election.cc:343] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:27.802101 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:27.807044 13131 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:27.807889 13131 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:27.758343 13161 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.819752 13131 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:09:27.870746 13161 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:27.877311 13031 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:27.880009 13162 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:27.880578 13148 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:09:27.881084 13148 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.883625 13148 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:27.890499 13097 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:27.893431 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:27.897459 12675 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:09:27.899276 13161 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:09:27.900189 13161 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.901491 12722 tablet_bootstrap.cc:492] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:27.902055 12722 ts_tablet_manager.cc:1403] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.683s	user 0.013s	sys 0.027s
I20260709 12:09:27.902912 13161 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:27.902420 13153 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:09:27.906774 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:27.907948 12675 raft_consensus.cc:2393] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 1: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:09:27.909727 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:27.873023 13131 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.912616 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:27.914894 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:27.916479 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:27.916749 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:27.919922 13031 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:27.920488 13031 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.923913 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:27.931146 13097 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:09:27.931696 13097 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.933249 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:27.933734 13097 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:27.927858 13131 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:27.932071 13173 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:27.936710 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:27.937562 13031 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:27.937772 12494 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:27.938849 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:27.939580 13132 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:27.940095 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:27.941498 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:27.943953 13196 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:27.942939 12571 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:27.944481 13196 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.945470 13168 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:27.946012 13168 raft_consensus.cc:697] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:09:27.947186 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:27.948241 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:27.952309 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:27.953033 12605 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:27.953511 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:27.956081 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:27.957324 13196 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:27.960902 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:27.964423 12721 tablet_bootstrap.cc:492] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:27.965471 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:27.964424 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:27.967891 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:27.948415 13168 consensus_queue.cc:237] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.973706 12892 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:09:27.974092 12892 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:27.974619 12892 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:27.980736 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:27.984335 12429 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58992 (request call id 76) took 2093 ms. Trace:
I20260709 12:09:27.968672 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:27.980685 12722 raft_consensus.cc:359] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:27.989277 12722 raft_consensus.cc:385] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:27.989979 12722 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:27.992041 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:27.984730 12429 rpcz_store.cc:276] 0709 12:09:25.913302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:25.921714 (+  8412us) service_pool.cc:224] Handling call
0709 12:09:27.984280 (+2062566us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:09:27.284130 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:09:27.294597 (+ 10467us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:09:27.294633 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:09:27.294721 (+    88us) tablet.cc:662] Decoding operations
0709 12:09:27.295126 (+   405us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:09:27.295200 (+    74us) write_op.cc:641] Partition lock acquired for write op
0709 12:09:27.295237 (+    37us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:09:27.295432 (+   195us) tablet.cc:701] Row locks acquired
0709 12:09:27.295452 (+    20us) write_op.cc:260] PREPARE: finished
0709 12:09:27.295599 (+   147us) write_op.cc:270] Start()
0709 12:09:27.295719 (+   120us) write_op.cc:276] Timestamp: P: 1783598967295553 usec, L: 0
0709 12:09:27.295745 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:09:27.296471 (+   726us) log.cc:844] Serialized 540 byte log entry
0709 12:09:27.470422 (+173951us) op_driver.cc:464] REPLICATION: finished
0709 12:09:27.792645 (+322223us) write_op.cc:301] APPLY: starting
0709 12:09:27.799426 (+  6781us) tablet.cc:1367] starting BulkCheckPresence
0709 12:09:27.817221 (+ 17795us) tablet.cc:1370] finished BulkCheckPresence
0709 12:09:27.826724 (+  9503us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:09:27.826955 (+   231us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:09:27.827104 (+   149us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:09:27.827150 (+    46us) write_op.cc:312] APPLY: finished
0709 12:09:27.875797 (+ 48647us) log.cc:844] Serialized 22 byte log entry
0709 12:09:27.876009 (+   212us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:09:27.876098 (+    89us) write_op.cc:454] Released schema lock
0709 12:09:27.876147 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 12:09:27.876479 (+   332us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:09:27.876509 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":37997,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":284401,"apply.run_cpu_time_us":2278,"apply.run_wall_time_us":96661,"num_ops":1,"prepare.queue_time_us":199335,"prepare.run_cpu_time_us":17342,"prepare.run_wall_time_us":98059,"raft.queue_time_us":105247,"raft.run_cpu_time_us":29908,"raft.run_wall_time_us":196236,"replication_time_us":173751,"thread_start_us":281380,"threads_started":4,"wal-append.queue_time_us":65429}]]}
I20260709 12:09:27.994282 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:27.992064 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:28.002226 12646 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:28.002302 12722 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.009660 12722 ts_tablet_manager.cc:1434] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.106s	user 0.007s	sys 0.000s
I20260709 12:09:28.011000 12722 tablet_bootstrap.cc:492] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:28.008126 12721 tablet_bootstrap.cc:654] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:28.004890 12892 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.015352 12964 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:28.015597 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:28.017274 12892 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:09:28.020098 12646 leader_election.cc:343] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.023480 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:28.012995 13196 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:28.023972 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:28.024305 13196 raft_consensus.cc:697] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:28.024132 12598 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:28.025281 13196 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.026383 12720 tablet_bootstrap.cc:492] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:28.026894 12720 ts_tablet_manager.cc:1403] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.908s	user 0.013s	sys 0.024s
I20260709 12:09:28.033294 12496 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:28.036268 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:28.038322 12496 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:28.044404 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:28.079818 13165 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:28.080857 13144 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:09:28.084039 12643 leader_election.cc:343] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.102137 13181 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:28.102774 13181 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:28.103219 13181 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:09:28.114686 13185 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.391s	user 0.002s	sys 0.008s
I20260709 12:09:28.138177 12722 tablet_bootstrap.cc:654] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:28.179474 12720 raft_consensus.cc:359] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.184842 12720 raft_consensus.cc:385] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:09:28.182384 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 144133us
I20260709 12:09:28.185521 12720 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:28.189534 12720 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:28.190361 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 106380us
I20260709 12:09:28.192325 12643 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:28.195924 13129 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:28.209291 13092 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:09:28.210932 13092 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.211385 13031 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:28.211967 13031 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.212951 12720 ts_tablet_manager.cc:1434] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.186s	user 0.016s	sys 0.004s
I20260709 12:09:28.213158 13092 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:28.215632 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:28.214011 12720 tablet_bootstrap.cc:492] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:28.221918 12496 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:28.223855 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:28.229509 12496 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:28.237443 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:28.237892 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:28.241294 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:28.241109 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:09:28.245237 12643 consensus_peers.cc:597] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:28.225247 13031 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:28.242211 13192 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:28.253001 12892 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:28.257642 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:28.261324 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:09:28.265960 12643 leader_election.cc:343] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.266472 12643 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:28.268092 13038 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:28.269479 12572 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:28.270326 13168 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:28.270236 12496 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:28.271931 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:28.271894 13181 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.272823 13168 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:28.273162 13092 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:28.276099 12496 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:28.276844 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:28.281003 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:28.282333 12721 tablet_bootstrap.cc:492] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:28.282828 12722 tablet_bootstrap.cc:492] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:28.282917 12721 ts_tablet_manager.cc:1403] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.319s	user 0.027s	sys 0.006s
I20260709 12:09:28.283385 12722 ts_tablet_manager.cc:1403] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.273s	user 0.010s	sys 0.025s
I20260709 12:09:28.283905 12572 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:28.285135 12496 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:09:28.285501 12646 leader_election.cc:343] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.285740 13192 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:28.286039 13145 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:28.254477 12598 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:09:28.291035 12496 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:28.293581 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:28.297588 12496 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:28.298055 13170 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:28.302557 13197 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:28.303375 13193 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:09:28.304073 13193 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:09:28.289829 13161 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:28.308256 12720 tablet_bootstrap.cc:654] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:28.313135 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:28.316445 13181 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:28.293288 13192 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.323010 12496 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:28.325707 13132 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:28.326443 13155 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:28.332991 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:28.333374 13092 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:28.320123 12722 raft_consensus.cc:359] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.334689 12722 raft_consensus.cc:385] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:28.335052 12722 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
W20260709 12:09:28.336220 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 45591.2us
I20260709 12:09:28.336395 12496 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:28.336970 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:28.325645 13192 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:28.346547 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:28.347678 12722 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.351605 12722 ts_tablet_manager.cc:1434] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.068s	user 0.004s	sys 0.004s
I20260709 12:09:28.339687 13092 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:28.344832 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:09:28.353214 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 70745.7us
I20260709 12:09:28.355510 12721 raft_consensus.cc:359] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:28.356611 12496 leader_election.cc:343] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.357535 13088 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:28.356187 12721 raft_consensus.cc:385] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:28.358543 12721 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:28.305292 13193 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.359815 12721 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.315744 13197 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:28.344569 13031 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:09:28.345652 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52093.4us
I20260709 12:09:28.376796 12605 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:09:28.378791 12721 ts_tablet_manager.cc:1434] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.095s	user 0.008s	sys 0.000s
I20260709 12:09:28.387662 13037 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:28.389057 13037 raft_consensus.cc:3055] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:09:28.389439 13037 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:28.389952 12721 tablet_bootstrap.cc:492] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:28.413779 12722 tablet_bootstrap.cc:492] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:28.415472 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:28.416553 12599 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:28.404152 13037 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.447469 13205 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:09:28.452598 12494 leader_election.cc:343] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.454970 12494 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:28.464481 13144 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:28.465898 13190 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:28.466508 13190 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.468523 13190 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:28.472819 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:28.501531 12526 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:28.504812 12646 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:28.505522 13190 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:28.505882 13190 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:28.506456 13190 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:28.510427 12720 tablet_bootstrap.cc:492] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:28.510934 12720 ts_tablet_manager.cc:1403] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.297s	user 0.012s	sys 0.020s
I20260709 12:09:28.509984 13175 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:28.513051 13175 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.514868 13175 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:28.517556 12721 tablet_bootstrap.cc:654] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:28.522671 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:28.528359 12722 tablet_bootstrap.cc:654] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:28.539516 13037 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:28.540081 12720 raft_consensus.cc:359] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.541303 12720 raft_consensus.cc:385] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:28.544018 12605 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:28.541615 12720 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
W20260709 12:09:28.550509 12477 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:28.551622 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:28.557092 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:28.557837 12528 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 1: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:09:28.559763 12572 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:28.559609 12720 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.561614 12720 ts_tablet_manager.cc:1434] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.002s
I20260709 12:09:28.562203 12604 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:28.563496 12720 tablet_bootstrap.cc:492] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:28.566542 12426 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59006 (request call id 80) took 1422 ms. Trace:
I20260709 12:09:28.566932 12426 rpcz_store.cc:276] 0709 12:09:27.144685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:27.146271 (+  1586us) service_pool.cc:224] Handling call
0709 12:09:28.566477 (+1420206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":84213,"reported_tablets":7}
I20260709 12:09:28.548604 13133 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:28.586477 13133 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.587520 13190 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.590374 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:28.601763 12676 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 1: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:09:28.593798 12428 catalog_manager.cc:5697] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 0 to 1, leader changed from <none> to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:28.588747 13133 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:28.597594 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:28.598845 12599 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:28.609328 13206 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:28.613176 13206 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:28.616541 12494 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:28.620262 12721 tablet_bootstrap.cc:492] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:28.622283 12721 ts_tablet_manager.cc:1403] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.236s	user 0.017s	sys 0.010s
I20260709 12:09:28.623641 13192 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:28.625164 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:28.627627 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:28.628582 12600 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:28.628814 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:28.629680 12529 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:28.640381 12529 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:28.646867 12646 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:28.650430 13088 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:28.642607 13190 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:28.653023 12721 raft_consensus.cc:359] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.654101 12721 raft_consensus.cc:385] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:28.654631 12721 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:28.655652 13190 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:28.655685 13180 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:28.658157 13180 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:28.659309 13180 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.648396 12600 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:28.671339 12721 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.655269 13126 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:28.676286 12721 ts_tablet_manager.cc:1434] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.054s	user 0.006s	sys 0.000s
I20260709 12:09:28.676865 13126 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:28.677531 12721 tablet_bootstrap.cc:492] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:28.684886 12722 tablet_bootstrap.cc:492] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:28.685433 12722 ts_tablet_manager.cc:1403] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.272s	user 0.033s	sys 0.017s
I20260709 12:09:28.678251 13126 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.709133 13192 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:28.712587 12722 raft_consensus.cc:359] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.717523 12722 raft_consensus.cc:385] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:28.717890 12722 raft_consensus.cc:740] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:28.724747 12721 tablet_bootstrap.cc:654] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:28.741410 12722 consensus_queue.cc:260] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.747452 12720 tablet_bootstrap.cc:654] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:28.770001 12722 ts_tablet_manager.cc:1434] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.084s	user 0.008s	sys 0.000s
I20260709 12:09:28.791292 12722 tablet_bootstrap.cc:492] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Bootstrap starting.
I20260709 12:09:28.880074 12720 tablet_bootstrap.cc:492] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:28.883035 12720 ts_tablet_manager.cc:1403] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.320s	user 0.015s	sys 0.018s
I20260709 12:09:28.887228 13190 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:28.889199 13190 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.890965 12721 tablet_bootstrap.cc:492] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:28.891722 12721 ts_tablet_manager.cc:1403] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.214s	user 0.035s	sys 0.027s
I20260709 12:09:28.894752 12722 tablet_bootstrap.cc:654] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:28.902745 13190 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:28.907500 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:09:28.909080 12643 leader_election.cc:343] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:28.912113 12720 raft_consensus.cc:359] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.913235 12720 raft_consensus.cc:385] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:28.913549 12720 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:28.914706 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:28.915668 12523 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:28.918185 12720 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.920122 12646 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:28.922943 13126 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:28.923478 13126 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:28.923986 13126 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:28.928234 12721 raft_consensus.cc:359] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.929343 12721 raft_consensus.cc:385] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:28.929710 12721 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:28.937224 12720 ts_tablet_manager.cc:1434] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.046s	user 0.004s	sys 0.002s
I20260709 12:09:28.947101 12721 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.930950 13092 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:28.954092 12720 tablet_bootstrap.cc:492] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:28.960678 12604 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:28.965395 13092 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:28.969841 12721 ts_tablet_manager.cc:1434] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 0.077s	user 0.010s	sys 0.001s
I20260709 12:09:28.968397 13133 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:09:29.014328 12721 tablet_bootstrap.cc:492] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Bootstrap starting.
I20260709 12:09:29.055651 13092 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:29.061189 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:29.062305 12676 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:29.074496 12429 catalog_manager.cc:5697] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:29.090732 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:29.108211 12429 catalog_manager.cc:5697] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:29.138584 12678 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:09:29.140353 12494 leader_election.cc:343] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:29.145273 12720 tablet_bootstrap.cc:654] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:29.156459 12496 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:29.158995 13144 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:29.159409 13144 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:29.159740 13144 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:29.167086 12722 tablet_bootstrap.cc:492] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: No bootstrap required, opened a new log
I20260709 12:09:29.167672 12722 ts_tablet_manager.cc:1403] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent bootstrapping tablet: real 0.382s	user 0.028s	sys 0.030s
I20260709 12:09:29.151755 13126 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:29.155694 13133 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:09:29.193634 12721 tablet_bootstrap.cc:654] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:29.212899 13126 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:29.213932 12722 raft_consensus.cc:359] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:29.215596 12722 raft_consensus.cc:385] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:29.215898 12722 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Initialized, Role: FOLLOWER
I20260709 12:09:29.221702 12605 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:29.237401 12722 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:29.242365 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:29.243045 12531 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:29.254869 12720 tablet_bootstrap.cc:492] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:29.256794 12720 ts_tablet_manager.cc:1403] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.303s	user 0.012s	sys 0.009s
I20260709 12:09:29.261471 12721 tablet_bootstrap.cc:492] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: No bootstrap required, opened a new log
I20260709 12:09:29.263190 12721 ts_tablet_manager.cc:1403] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Time spent bootstrapping tablet: real 0.262s	user 0.021s	sys 0.012s
I20260709 12:09:29.289312 13196 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:29.291199 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:29.294544 12722 ts_tablet_manager.cc:1434] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent starting tablet: real 0.126s	user 0.009s	sys 0.000s
I20260709 12:09:29.354436 13126 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:29.355189 13126 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:29.493353 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 245497us
I20260709 12:09:29.277595 12720 raft_consensus.cc:359] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:29.495368 12720 raft_consensus.cc:385] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:29.495697 12720 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:29.496362 12720 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:29.497634 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 183291us
I20260709 12:09:29.508976 12531 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:29.527081 12720 ts_tablet_manager.cc:1434] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.270s	user 0.012s	sys 0.017s
I20260709 12:09:29.522615 13190 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 12:09:29.509128 12597 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:29.492233 13144 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:29.521361 13196 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.026s
I20260709 12:09:29.570506 12720 tablet_bootstrap.cc:492] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:29.591298 12721 raft_consensus.cc:359] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:29.651132 12721 raft_consensus.cc:385] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:29.651551 12721 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Initialized, Role: FOLLOWER
I20260709 12:09:29.372294 12426 catalog_manager.cc:5697] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 1 to 2, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:29.667989 12721 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:29.695104 13126 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:29.699404 13144 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:29.790987 12429 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59006 (request call id 81) took 1125 ms. Trace:
I20260709 12:09:29.791446 12429 rpcz_store.cc:276] 0709 12:09:28.665468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:28.665629 (+   161us) service_pool.cc:224] Handling call
0709 12:09:29.790893 (+1125264us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:09:29.159316 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:09:29.159883 (+   567us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:09:29.159917 (+    34us) write_op.cc:435] Acquired schema lock
0709 12:09:29.159935 (+    18us) tablet.cc:662] Decoding operations
0709 12:09:29.160355 (+   420us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:09:29.160415 (+    60us) write_op.cc:641] Partition lock acquired for write op
0709 12:09:29.160451 (+    36us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:09:29.160795 (+   344us) tablet.cc:701] Row locks acquired
0709 12:09:29.160812 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:09:29.160958 (+   146us) write_op.cc:270] Start()
0709 12:09:29.161068 (+   110us) write_op.cc:276] Timestamp: P: 1783598969160913 usec, L: 0
0709 12:09:29.161094 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:09:29.175553 (+ 14459us) log.cc:844] Serialized 928 byte log entry
0709 12:09:29.190503 (+ 14950us) op_driver.cc:464] REPLICATION: finished
0709 12:09:29.190932 (+   429us) write_op.cc:301] APPLY: starting
0709 12:09:29.191009 (+    77us) tablet.cc:1367] starting BulkCheckPresence
0709 12:09:29.191139 (+   130us) tablet.cc:1370] finished BulkCheckPresence
0709 12:09:29.191245 (+   106us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:09:29.191573 (+   328us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:09:29.191742 (+   169us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:09:29.191785 (+    43us) write_op.cc:312] APPLY: finished
0709 12:09:29.191980 (+   195us) log.cc:844] Serialized 28 byte log entry
0709 12:09:29.192132 (+   152us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:09:29.192212 (+    80us) write_op.cc:454] Released schema lock
0709 12:09:29.192248 (+    36us) write_op.cc:341] FINISH: Updating metrics
0709 12:09:29.192516 (+   268us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:09:29.192542 (+    26us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":246,"apply.run_cpu_time_us":1554,"apply.run_wall_time_us":1554,"num_ops":2,"prepare.queue_time_us":91,"prepare.run_cpu_time_us":3142,"prepare.run_wall_time_us":16530,"replication_time_us":29257}]]}
I20260709 12:09:30.177232 13237 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:09:30.177970 13237 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.180459 13237 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:09:30.196770 12597 consensus_meta.cc:287] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.616s	user 0.015s	sys 0.000s
I20260709 12:09:30.197269 12597 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
W20260709 12:09:30.219184 12496 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.688s	user 0.044s	sys 0.054s
W20260709 12:09:30.219796 12496 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.044s	sys 0.054s
W20260709 12:09:30.230764 12605 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.004s	sys 0.010s
W20260709 12:09:30.231215 12605 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.004s	sys 0.010s
I20260709 12:09:30.232263 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 245) took 1017 ms. Trace:
I20260709 12:09:30.232728 12605 rpcz_store.cc:276] 0709 12:09:29.214741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:29.215789 (+  1048us) service_pool.cc:224] Handling call
0709 12:09:29.221547 (+  5758us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:09:29.494356 (+272809us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:09:29.494384 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:09:29.494857 (+   473us) log.cc:844] Serialized 26 byte log entry
0709 12:09:29.565433 (+ 70576us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:09:30.231634 (+666201us) log.cc:844] Serialized 14 byte log entry
0709 12:09:30.231902 (+   268us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:09:30.231990 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:09:30.232045 (+    55us) raft_consensus.cc:1685] finished
0709 12:09:30.232060 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:09:30.232200 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":156,"prepare.queue_time_us":197944,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":4003,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":14241,"thread_start_us":735924,"threads_started":2,"wal-append.queue_time_us":224234}
W20260709 12:09:30.247673 12571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.009s	sys 0.051s
W20260709 12:09:30.248055 12571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.009s	sys 0.052s
I20260709 12:09:30.249747 13231 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:30.250336 13231 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.252621 13231 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:30.259037 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:09:30.267678 12494 leader_election.cc:343] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:30.269611 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:09:30.272540 12572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.692s	user 0.006s	sys 0.015s
W20260709 12:09:30.272939 12572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.006s	sys 0.015s
I20260709 12:09:30.293349 12426 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58980 (request call id 86) took 1566 ms. Trace:
I20260709 12:09:30.297309 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:30.297390 12426 rpcz_store.cc:276] 0709 12:09:28.726940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:28.727286 (+   346us) service_pool.cc:224] Handling call
0709 12:09:30.293256 (+1565970us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:09:29.703157 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:09:29.703912 (+   755us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:09:29.703961 (+    49us) write_op.cc:435] Acquired schema lock
0709 12:09:29.703984 (+    23us) tablet.cc:662] Decoding operations
0709 12:09:29.704432 (+   448us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:09:29.704508 (+    76us) write_op.cc:641] Partition lock acquired for write op
0709 12:09:29.704541 (+    33us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:09:29.704820 (+   279us) tablet.cc:701] Row locks acquired
0709 12:09:29.704847 (+    27us) write_op.cc:260] PREPARE: finished
0709 12:09:29.705037 (+   190us) write_op.cc:270] Start()
0709 12:09:29.705173 (+   136us) write_op.cc:276] Timestamp: P: 1783598969704981 usec, L: 0
0709 12:09:29.705193 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:09:29.706026 (+   833us) log.cc:844] Serialized 540 byte log entry
0709 12:09:29.764051 (+ 58025us) op_driver.cc:464] REPLICATION: finished
0709 12:09:29.766708 (+  2657us) write_op.cc:301] APPLY: starting
0709 12:09:29.766790 (+    82us) tablet.cc:1367] starting BulkCheckPresence
0709 12:09:29.766894 (+   104us) tablet.cc:1370] finished BulkCheckPresence
0709 12:09:29.767024 (+   130us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:09:29.767300 (+   276us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:09:29.767462 (+   162us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:09:29.767515 (+    53us) write_op.cc:312] APPLY: finished
0709 12:09:29.767720 (+   205us) log.cc:844] Serialized 22 byte log entry
0709 12:09:29.767889 (+   169us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:09:29.767969 (+    80us) write_op.cc:454] Released schema lock
0709 12:09:29.768006 (+    37us) write_op.cc:341] FINISH: Updating metrics
0709 12:09:29.768354 (+   348us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:09:29.768383 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":2438,"apply.run_cpu_time_us":1641,"apply.run_wall_time_us":1640,"num_ops":1,"prepare.queue_time_us":35990,"prepare.run_cpu_time_us":3350,"prepare.run_wall_time_us":11977,"replication_time_us":58690,"thread_start_us":2073,"threads_started":1}]]}
I20260709 12:09:30.303691 13191 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:09:30.313401 13191 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:30.308135 13182 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.597s	user 0.011s	sys 0.033s
I20260709 12:09:30.285689 13038 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:09:30.315784 13182 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.011s	sys 0.034s
I20260709 12:09:30.315689 13038 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.317273 13205 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:30.318001 13205 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.318567 13191 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:30.319649 13191 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:09:30.320237 13191 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.321532 13205 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:30.322647 13191 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:30.327522 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 273) took 1087 ms. Trace:
I20260709 12:09:30.328006 12527 rpcz_store.cc:276] 0709 12:09:29.240103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:29.240258 (+   155us) service_pool.cc:224] Handling call
0709 12:09:29.303550 (+ 63292us) spinlock_profiling.cc:243] Waited 4.86 ms on lock 0x7b5800002a98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:09:30.326656 (+1023106us) spinlock_profiling.cc:243] Waited 3.26 ms on lock 0x7b5800002a98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fbeee 000000000036eb39 00007f148d96f1e0 00007f148da380c4 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:09:30.326822 (+   166us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:09:30.327210 (+   388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:09:30.327401 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18688000}
I20260709 12:09:30.339630 13038 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:09:30.340934 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46800.9us
W20260709 12:09:30.343216 12721 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.014s	sys 0.001s
W20260709 12:09:30.343539 12721 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.014s	sys 0.001s
I20260709 12:09:30.352499 12721 ts_tablet_manager.cc:1434] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Time spent starting tablet: real 1.089s	user 0.030s	sys 0.015s
W20260709 12:09:30.352905 12721 ts_tablet_manager.cc:1467] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Tablet startup took 1089ms
I20260709 12:09:30.362103 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:30.358911 13144 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:30.368121 13144 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:30.368594 12646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.703s	user 0.007s	sys 0.002s
W20260709 12:09:30.368991 12646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.008s	sys 0.002s
I20260709 12:09:30.371012 13144 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:30.372736 12720 tablet_bootstrap.cc:654] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
W20260709 12:09:30.353238 12721 ts_tablet_manager.cc:1469] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Trace:
0709 12:09:17.598820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:17.598945 (+   125us) service_pool.cc:224] Handling call
0709 12:09:17.600420 (+  1475us) ts_tablet_manager.cc:669] Acquired tablet manager lock
0709 12:09:17.600499 (+    79us) ts_tablet_manager.cc:682] Creating new metadata...
0709 12:09:17.632374 (+ 31875us) tablet_metadata.cc:665] Metadata flushed
0709 12:09:17.688769 (+ 56395us) tablet_replica.cc:206] Creating consensus instance
0709 12:09:17.689324 (+   555us) inbound_call.cc:177] Queueing success response
0709 12:09:28.974514 (+11285190us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:09:29.263935 (+289421us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:09:29.271005 (+  7070us) tablet_replica.cc:261] Starting instrumentation
0709 12:09:29.495287 (+224282us) tablet_replica.cc:280] Starting consensus
Metrics: {"mutex_wait_us":19314,"raft.queue_time_us":26124,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":115,"tablet-open.queue_time_us":11285219,"thread_start_us":674329,"threads_started":1}
I20260709 12:09:30.377429 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:30.379494 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:09:30.383723 12494 leader_election.cc:343] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:30.385114 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:30.395093 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:30.399353 13191 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:09:30.403160 12643 leader_election.cc:343] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:09:30.405614 12643 leader_election.cc:343] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:30.402437 13126 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:09:30.407778 13126 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.408936 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 276) took 1072 ms. Trace:
I20260709 12:09:30.409687 12525 rpcz_store.cc:276] 0709 12:09:29.336195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:29.336360 (+   165us) service_pool.cc:224] Handling call
0709 12:09:30.407797 (+1071437us) spinlock_profiling.cc:243] Waited 159 ms on lock 0x7b5800002a98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:09:30.407967 (+   170us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:09:30.408127 (+   160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:09:30.408256 (+   129us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:09:30.408310 (+    54us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:09:30.408732 (+   422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:09:30.408795 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:09:30.408881 (+    86us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":365200128}
I20260709 12:09:30.413019 13168 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:09:30.413543 13168 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:30.414335 12494 leader_election.cc:343] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:30.415620 13168 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:30.410246 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 249) took 1192 ms. Trace:
I20260709 12:09:30.418004 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:30.399917 13191 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.421469 13191 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:30.422173 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:30.422859 12671 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:30.411008 13240 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:30.428996 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:30.427242 12597 rpcz_store.cc:276] 0709 12:09:29.217369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:29.221969 (+  4600us) service_pool.cc:224] Handling call
0709 12:09:29.291050 (+ 69081us) spinlock_profiling.cc:243] Waited 2.15 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:09:30.402545 (+1111495us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b5400271ad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:09:30.402657 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":477228672}
I20260709 12:09:30.434659 13126 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:30.432296 13240 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.438803 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:30.439783 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:30.440687 13240 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:30.442332 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:30.445135 12671 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:09:30.446444 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:30.447770 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:30.449155 13255 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:09:30.449651 13255 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.450412 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:30.451030 12528 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:30.451290 13038 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:30.451808 13038 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:30.453212 13255 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:30.452713 13038 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.456089 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:30.458199 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:30.469918 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:30.470836 12680 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 0.
I20260709 12:09:30.494688 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:30.500052 13192 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:30.500612 13192 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.504525 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:30.505666 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:30.514750 13248 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:09:30.515394 13248 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.517607 13248 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:30.513839 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:30.521732 12522 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 0.
I20260709 12:09:30.521437 12426 catalog_manager.cc:5697] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:30.526324 12426 catalog_manager.cc:5697] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:30.531021 13192 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:30.543231 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:09:30.547992 12477 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:30.550009 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:30.557627 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:30.558480 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:30.558977 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:30.562937 13226 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:30.563536 13226 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.571775 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:30.573011 13254 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:30.573729 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:30.574579 13038 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:30.575301 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:30.573899 12646 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:30.576428 13254 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:30.577323 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:30.577594 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:30.577813 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:30.579088 13250 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:30.579377 13170 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:30.579490 13250 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:30.579928 13250 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:30.580848 13088 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:30.589637 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:30.590533 12720 tablet_bootstrap.cc:492] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:30.591027 12720 ts_tablet_manager.cc:1403] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 1.021s	user 0.033s	sys 0.028s
I20260709 12:09:30.592839 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:30.593698 12720 raft_consensus.cc:359] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.594388 12720 raft_consensus.cc:385] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:30.594655 12720 raft_consensus.cc:740] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:30.596546 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:30.597488 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:30.598495 12496 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:30.600164 13133 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:30.600735 13133 raft_consensus.cc:697] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:09:30.602176 12496 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:30.603219 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:30.604288 13133 consensus_queue.cc:237] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.607273 13250 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.607844 13170 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:30.609334 13226 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:30.602308 12646 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:30.611482 13190 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:30.611930 12646 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:30.595999 12720 consensus_queue.cc:260] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.614506 12646 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:30.613247 13248 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:30.619637 12676 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 1: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:09:30.622526 13226 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:30.622742 12646 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:30.624913 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:30.625217 13190 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:30.622503 12496 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:30.626390 13192 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:30.625576 13191 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:09:30.619858 13250 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:30.626873 13191 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:09:30.626754 12643 leader_election.cc:343] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:30.627388 13191 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:30.626754 13192 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:30.628301 13192 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:30.628985 12496 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:30.630191 13144 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:30.634382 12646 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:30.636950 13258 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:30.637305 12720 ts_tablet_manager.cc:1434] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.046s	user 0.006s	sys 0.000s
I20260709 12:09:30.636101 12496 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:30.638412 12720 tablet_bootstrap.cc:492] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
I20260709 12:09:30.638609 13158 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:30.654376 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:30.659754 12525 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:30.661509 13191 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.673017 12496 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:30.682194 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:30.687820 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:09:30.690191 12643 leader_election.cc:343] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:30.689854 13191 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:30.671515 13192 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.691951 12720 tablet_bootstrap.cc:654] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
I20260709 12:09:30.697460 12530 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 1: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:09:30.703189 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:30.703653 13192 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:09:30.705351 12494 leader_election.cc:343] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:09:30.707278 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:30.708084 12676 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 1: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:09:30.708429 12646 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:30.709465 13088 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:09:30.711144 12496 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:30.712581 13192 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:09:30.725586 12525 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:30.727733 13144 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:30.728209 12720 tablet_bootstrap.cc:492] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:30.728730 12720 ts_tablet_manager.cc:1403] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.091s	user 0.006s	sys 0.019s
I20260709 12:09:30.730425 12646 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:30.731535 13088 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:09:30.731603 12720 raft_consensus.cc:359] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.732084 13088 raft_consensus.cc:697] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:30.732322 12720 raft_consensus.cc:385] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:30.732628 12720 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:30.732942 13088 consensus_queue.cc:237] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.733264 12720 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.738782 12720 ts_tablet_manager.cc:1434] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.002s
I20260709 12:09:30.741736 12720 tablet_bootstrap.cc:492] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Bootstrap starting.
W20260709 12:09:30.745294 12460 debug-util.cc:398] Leaking SignalData structure 0x7b08000349a0 after lost signal to thread 11321
W20260709 12:09:30.754208 12460 debug-util.cc:398] Leaking SignalData structure 0x7b080003e080 after lost signal to thread 12463
I20260709 12:09:30.755173 12720 tablet_bootstrap.cc:654] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Neither blocks nor log segments found. Creating new log.
W20260709 12:09:30.755303 12460 debug-util.cc:398] Leaking SignalData structure 0x7b080016a220 after lost signal to thread 12555
W20260709 12:09:30.757026 12460 debug-util.cc:398] Leaking SignalData structure 0x7b08002c4260 after lost signal to thread 12630
W20260709 12:09:30.765622 12460 debug-util.cc:398] Leaking SignalData structure 0x7b08002975e0 after lost signal to thread 12704
I20260709 12:09:30.781834 13170 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:30.782398 13170 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.784546 13170 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:30.788241 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:30.790112 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:30.791007 12672 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 1: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:09:30.791520 13226 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:09:30.792028 13226 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.800160 12571 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:30.801378 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:30.801962 13252 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:30.814975 13226 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:30.820201 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:30.828361 13252 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:30.848806 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:30.850308 13192 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:30.883900 12428 catalog_manager.cc:5697] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 0 to 1, leader changed from <none> to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:30.917822 12429 catalog_manager.cc:5697] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:30.930953 12429 catalog_manager.cc:5697] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:09:30.934192 12720 tablet_bootstrap.cc:492] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: No bootstrap required, opened a new log
I20260709 12:09:30.941433 12720 ts_tablet_manager.cc:1403] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent bootstrapping tablet: real 0.200s	user 0.023s	sys 0.052s
I20260709 12:09:30.944085 12720 raft_consensus.cc:359] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.945089 12720 raft_consensus.cc:385] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:09:30.945438 12720 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Initialized, Role: FOLLOWER
I20260709 12:09:30.946388 12720 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:30.949548 12720 ts_tablet_manager.cc:1434] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:09:30.961699 13250 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:30.998337 13190 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:31.028458 13144 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:31.054071 13144 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:09:31.124040 13254 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:31.124644 13254 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:31.128672 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:31.128770 13254 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:31.128991 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:31.129834 12527 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 1: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:09:31.131415 12572 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:31.133163 13254 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:31.142110 13254 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:31.170240 13088 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:09:31.210035 13170 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:31.210599 13170 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:31.212778 13170 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:31.218178 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:31.219188 12675 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 1: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:09:31.246078 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:31.247298 12526 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 1: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:09:31.252810 12572 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:31.278369 12964 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:09:31.341797 12964 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:09:31.350940 13190 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:09:31.441247 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 80566.2us
W20260709 12:09:31.599323 12477 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:09:32.389839 13088 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:32.390460 13088 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:32.396209 13088 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:32.397260 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:32.397888 12597 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:32.399377 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:32.396091 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:32.400276 13088 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:09:32.400393 12523 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:09:32.400754 13088 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:09:32.401348 13088 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:32.409264 13088 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:32.414533 13088 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:32.428437 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:32.429194 12603 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:32.456600 12603 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:09:32.457639 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:32.458441 12523 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:32.460371 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:32.461719 13190 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:09:32.462239 13190 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:32.463105 13190 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:32.489162 12523 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:09:32.497591 12426 catalog_manager.cc:5697] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 0 to 2, leader changed from <none> to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 2 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
W20260709 12:09:32.957763 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 70755.5us
I20260709 12:09:32.970758 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
I20260709 12:09:33.004271 12523 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:09:33.007989 12603 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:09:33.009303 13285 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
W20260709 12:09:33.014371 13288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:09:33.038609 13289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:09:33.055502 13250 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:09:33.183139 13295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:09:33.188319 11318 server_base.cc:1061] running on GCE node
I20260709 12:09:33.198477 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:09:33.198843 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:09:33.199083 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783598973199059 us; error 0 us; skew 500 ppm
I20260709 12:09:33.200074 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:09:33.242976 11318 webserver.cc:533] Webserver started at http://127.11.13.132:36083/ using document root <none> and password file <none>
I20260709 12:09:33.243944 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:09:33.244274 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:09:33.244762 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:09:33.247447 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-3-root/instance:
uuid: "e56d487365394ac5ac5f1002926a5645"
format_stamp: "Formatted at 2026-07-09 12:09:33 on dist-test-slave-zqnl"
I20260709 12:09:33.265362 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.016s	user 0.007s	sys 0.011s
I20260709 12:09:33.278091 13301 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:33.279434 11318 fs_manager.cc:730] Time spent opening block manager: real 0.010s	user 0.004s	sys 0.007s
I20260709 12:09:33.280020 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-3-root
uuid: "e56d487365394ac5ac5f1002926a5645"
format_stamp: "Formatted at 2026-07-09 12:09:33 on dist-test-slave-zqnl"
I20260709 12:09:33.280514 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:09:33.328717 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:09:33.331286 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:09:33.336202 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:09:33.346714 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:09:33.347064 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:33.347415 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:09:33.347677 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:09:33.957724 13363 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.132:45801 every 8 connection(s)
I20260709 12:09:33.958143 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.132:45801
I20260709 12:09:34.006556 11318 auto_leader_rebalancer-test.cc:439] add a tserver: e56d487365394ac5ac5f1002926a5645
I20260709 12:09:34.065627 13364 heartbeater.cc:344] Connected to a master server at 127.11.13.190:46629
I20260709 12:09:34.066236 13364 heartbeater.cc:461] Registering TS with master...
I20260709 12:09:34.067301 13364 heartbeater.cc:507] Master 127.11.13.190:46629 requested a full tablet report, sending...
I20260709 12:09:34.070214 12426 ts_manager.cc:194] Registered new tserver with Master: e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:09:34.072805 12426 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:47524
I20260709 12:09:35.076630 13364 heartbeater.cc:499] Master 127.11.13.190:46629 was elected leader, sending a full tablet report...
I20260709 12:09:35.197568 12597 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:35.214608 12525 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:35.216344 13252 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:35.217448 12671 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:35.221720 13252 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:35.254658 13283 raft_consensus.cc:2955] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:35.254729 12671 raft_consensus.cc:2955] T e40144c1c60847ee81b8252b107df67d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:35.259680 12525 raft_consensus.cc:2955] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:35.287725 12426 catalog_manager.cc:5697] T e40144c1c60847ee81b8252b107df67d P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
W20260709 12:09:35.290532 12569 consensus_peers.cc:597] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e40144c1c60847ee81b8252b107df67d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:35.361452 12675 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:35.397089 12527 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:35.398921 13248 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:35.400743 12597 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:35.403555 13190 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:35.445020 13248 raft_consensus.cc:2955] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:35.450673 12527 raft_consensus.cc:2955] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:35.458822 12603 raft_consensus.cc:2955] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
W20260709 12:09:35.473204 12643 consensus_peers.cc:597] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b7cccd56fa854cbc84a922238ad4add9. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:35.474633 12428 catalog_manager.cc:5697] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:35.523994 12675 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:35.556108 12603 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:35.556272 12523 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:09:35.558207 12643 consensus_peers.cc:597] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e1ff01ff4ede4fc990b9a07b23102b0d. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:35.558657 13248 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:35.568300 13368 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:35.613766 13088 raft_consensus.cc:2955] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:35.642055 12597 raft_consensus.cc:2955] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:35.647106 12675 consensus_queue.cc:237] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:35.645748 12525 raft_consensus.cc:2955] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:35.658864 12428 catalog_manager.cc:5697] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260709 12:09:35.689361 12643 consensus_peers.cc:597] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e15d21db797e4a96b5732711a5379980. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:35.694509 12527 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:35.696162 13389 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:35.714337 12602 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:35.718578 13368 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:35.728428 13389 raft_consensus.cc:2955] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:35.740564 12523 raft_consensus.cc:2955] T e15d21db797e4a96b5732711a5379980 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:35.786484 12428 catalog_manager.cc:5697] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:35.790282 12602 raft_consensus.cc:2955] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:35.813314 13394 ts_tablet_manager.cc:933] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:35.876924 13394 tablet_copy_client.cc:323] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.130:39365
I20260709 12:09:36.011111 13398 ts_tablet_manager.cc:933] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:36.018837 12616 tablet_copy_service.cc:140] P f5fc807f7ecc4d1ca0fd6be524f734ce: Received BeginTabletCopySession request for tablet e40144c1c60847ee81b8252b107df67d from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:48492)
I20260709 12:09:36.019526 12616 tablet_copy_service.cc:161] P f5fc807f7ecc4d1ca0fd6be524f734ce: Beginning new tablet copy session on tablet e40144c1c60847ee81b8252b107df67d from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:48492: session id = e56d487365394ac5ac5f1002926a5645-e40144c1c60847ee81b8252b107df67d
I20260709 12:09:36.035302 12616 tablet_copy_source_session.cc:215] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:36.054584 13398 tablet_copy_client.cc:323] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.131:38613
I20260709 12:09:36.070393 13394 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e40144c1c60847ee81b8252b107df67d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:36.099087 13394 tablet_copy_client.cc:806] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:36.099802 13394 tablet_copy_client.cc:670] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:36.122439 13394 tablet_copy_client.cc:538] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:36.155429 13394 tablet_bootstrap.cc:492] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
I20260709 12:09:36.191017 12690 tablet_copy_service.cc:140] P 193a40da6e364ead8ab561e14c3cc8d1: Received BeginTabletCopySession request for tablet b7cccd56fa854cbc84a922238ad4add9 from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:40246)
I20260709 12:09:36.191798 12690 tablet_copy_service.cc:161] P 193a40da6e364ead8ab561e14c3cc8d1: Beginning new tablet copy session on tablet b7cccd56fa854cbc84a922238ad4add9 from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:40246: session id = e56d487365394ac5ac5f1002926a5645-b7cccd56fa854cbc84a922238ad4add9
I20260709 12:09:36.200595 13403 ts_tablet_manager.cc:933] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:36.222674 13403 tablet_copy_client.cc:323] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.131:38613
I20260709 12:09:36.226440 12690 tablet_copy_source_session.cc:215] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:36.236632 12689 tablet_copy_service.cc:140] P 193a40da6e364ead8ab561e14c3cc8d1: Received BeginTabletCopySession request for tablet e1ff01ff4ede4fc990b9a07b23102b0d from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:40246)
I20260709 12:09:36.237340 12689 tablet_copy_service.cc:161] P 193a40da6e364ead8ab561e14c3cc8d1: Beginning new tablet copy session on tablet e1ff01ff4ede4fc990b9a07b23102b0d from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:40246: session id = e56d487365394ac5ac5f1002926a5645-e1ff01ff4ede4fc990b9a07b23102b0d
I20260709 12:09:36.257673 13398 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b7cccd56fa854cbc84a922238ad4add9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:36.258810 13404 ts_tablet_manager.cc:933] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:36.281662 12689 tablet_copy_source_session.cc:215] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:36.397930 13403 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e1ff01ff4ede4fc990b9a07b23102b0d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:36.410053 13404 tablet_copy_client.cc:323] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.131:38613
I20260709 12:09:36.415578 12689 tablet_copy_service.cc:140] P 193a40da6e364ead8ab561e14c3cc8d1: Received BeginTabletCopySession request for tablet e15d21db797e4a96b5732711a5379980 from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:40246)
I20260709 12:09:36.416180 12689 tablet_copy_service.cc:161] P 193a40da6e364ead8ab561e14c3cc8d1: Beginning new tablet copy session on tablet e15d21db797e4a96b5732711a5379980 from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:40246: session id = e56d487365394ac5ac5f1002926a5645-e15d21db797e4a96b5732711a5379980
I20260709 12:09:36.410104 13398 tablet_copy_client.cc:806] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:36.443050 13398 tablet_copy_client.cc:670] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:36.486104 12689 tablet_copy_source_session.cc:215] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:36.491133 13404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e15d21db797e4a96b5732711a5379980. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:36.492555 13398 tablet_copy_client.cc:538] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:36.541059 13394 tablet_bootstrap.cc:492] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:36.542527 13394 tablet_bootstrap.cc:492] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
I20260709 12:09:36.532017 13398 tablet_bootstrap.cc:492] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
I20260709 12:09:36.541746 13403 tablet_copy_client.cc:806] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:36.577834 13403 tablet_copy_client.cc:670] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:36.580364 13404 tablet_copy_client.cc:806] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:36.581039 13404 tablet_copy_client.cc:670] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:36.581596 13394 ts_tablet_manager.cc:1403] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 0.426s	user 0.010s	sys 0.039s
I20260709 12:09:36.609124 13403 tablet_copy_client.cc:538] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:36.612018 13394 raft_consensus.cc:359] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:36.613204 13394 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
I20260709 12:09:36.614054 13394 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:36.621016 13403 tablet_bootstrap.cc:492] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
I20260709 12:09:36.633230 13404 tablet_copy_client.cc:538] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:36.667058 13394 ts_tablet_manager.cc:1434] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.085s	user 0.015s	sys 0.007s
I20260709 12:09:36.719272 13404 tablet_bootstrap.cc:492] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
I20260709 12:09:36.729465 12616 tablet_copy_service.cc:342] P f5fc807f7ecc4d1ca0fd6be524f734ce: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-e40144c1c60847ee81b8252b107df67d received from {username='slave'} at 127.0.0.1:48492
I20260709 12:09:36.730255 12616 tablet_copy_service.cc:434] P f5fc807f7ecc4d1ca0fd6be524f734ce: ending tablet copy session e56d487365394ac5ac5f1002926a5645-e40144c1c60847ee81b8252b107df67d on tablet e40144c1c60847ee81b8252b107df67d with peer e56d487365394ac5ac5f1002926a5645
I20260709 12:09:36.762449 13338 raft_consensus.cc:1217] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:36.778074 13398 tablet_bootstrap.cc:492] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:36.784571 13398 tablet_bootstrap.cc:492] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
I20260709 12:09:36.785444 13398 ts_tablet_manager.cc:1403] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 0.254s	user 0.026s	sys 0.018s
I20260709 12:09:36.794107 13403 tablet_bootstrap.cc:492] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:36.809309 13403 tablet_bootstrap.cc:492] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
I20260709 12:09:36.810289 13403 ts_tablet_manager.cc:1403] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 0.189s	user 0.019s	sys 0.011s
I20260709 12:09:36.797232 13398 raft_consensus.cc:359] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:36.814018 13398 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
I20260709 12:09:36.814903 13398 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:36.817996 13398 ts_tablet_manager.cc:1434] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.024s	user 0.005s	sys 0.001s
I20260709 12:09:36.821960 12690 tablet_copy_service.cc:342] P 193a40da6e364ead8ab561e14c3cc8d1: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-b7cccd56fa854cbc84a922238ad4add9 received from {username='slave'} at 127.0.0.1:40246
I20260709 12:09:36.822474 12690 tablet_copy_service.cc:434] P 193a40da6e364ead8ab561e14c3cc8d1: ending tablet copy session e56d487365394ac5ac5f1002926a5645-b7cccd56fa854cbc84a922238ad4add9 on tablet b7cccd56fa854cbc84a922238ad4add9 with peer e56d487365394ac5ac5f1002926a5645
I20260709 12:09:36.834331 13403 raft_consensus.cc:359] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:36.836401 13403 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
I20260709 12:09:36.854669 13403 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:36.873126 13403 ts_tablet_manager.cc:1434] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.062s	user 0.005s	sys 0.002s
I20260709 12:09:36.876660 12690 tablet_copy_service.cc:342] P 193a40da6e364ead8ab561e14c3cc8d1: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-e1ff01ff4ede4fc990b9a07b23102b0d received from {username='slave'} at 127.0.0.1:40246
I20260709 12:09:36.877127 12690 tablet_copy_service.cc:434] P 193a40da6e364ead8ab561e14c3cc8d1: ending tablet copy session e56d487365394ac5ac5f1002926a5645-e1ff01ff4ede4fc990b9a07b23102b0d on tablet e1ff01ff4ede4fc990b9a07b23102b0d with peer e56d487365394ac5ac5f1002926a5645
I20260709 12:09:36.889259 13404 tablet_bootstrap.cc:492] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:36.890415 13404 tablet_bootstrap.cc:492] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
I20260709 12:09:36.891219 13404 ts_tablet_manager.cc:1403] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 0.182s	user 0.020s	sys 0.013s
I20260709 12:09:36.894513 13404 raft_consensus.cc:359] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:36.895468 13404 raft_consensus.cc:740] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
I20260709 12:09:36.900427 13404 consensus_queue.cc:260] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:36.912829 13404 ts_tablet_manager.cc:1434] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.021s	user 0.007s	sys 0.001s
I20260709 12:09:36.915596 12690 tablet_copy_service.cc:342] P 193a40da6e364ead8ab561e14c3cc8d1: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-e15d21db797e4a96b5732711a5379980 received from {username='slave'} at 127.0.0.1:40246
I20260709 12:09:36.916119 12690 tablet_copy_service.cc:434] P 193a40da6e364ead8ab561e14c3cc8d1: ending tablet copy session e56d487365394ac5ac5f1002926a5645-e15d21db797e4a96b5732711a5379980 on tablet e15d21db797e4a96b5732711a5379980 with peer e56d487365394ac5ac5f1002926a5645
I20260709 12:09:36.977989 13338 raft_consensus.cc:1217] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:37.025367 13338 raft_consensus.cc:1217] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:37.143002 13338 raft_consensus.cc:1217] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:37.173975 13283 raft_consensus.cc:1064] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: attempting to promote NON_VOTER e56d487365394ac5ac5f1002926a5645 to VOTER
I20260709 12:09:37.184209 13283 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:37.215054 13338 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:37.217085 13283 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:37.223619 12671 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:37.225678 13283 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:37.333662 13410 raft_consensus.cc:2955] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:37.351657 12676 raft_consensus.cc:2955] T e40144c1c60847ee81b8252b107df67d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:37.357868 12523 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:37.437994 13368 raft_consensus.cc:1064] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: attempting to promote NON_VOTER e56d487365394ac5ac5f1002926a5645 to VOTER
I20260709 12:09:37.440817 13368 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:09:37.482327 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 63612.8us
W20260709 12:09:37.534174 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 69622.6us
I20260709 12:09:37.550102 13283 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:37.567566 13338 raft_consensus.cc:2955] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:37.580487 12428 catalog_manager.cc:5697] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:09:37.644697 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 94721.6us
I20260709 12:09:37.645555 13338 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:37.712006 12525 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:37.713042 13433 raft_consensus.cc:1064] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: attempting to promote NON_VOTER e56d487365394ac5ac5f1002926a5645 to VOTER
I20260709 12:09:37.715842 13433 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:37.719121 12523 raft_consensus.cc:2955] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:09:37.723768 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 59889.2us
I20260709 12:09:37.718111 13432 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.012s
W20260709 12:09:37.776119 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 53451.7us
I20260709 12:09:37.793799 13415 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:37.811710 12599 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:37.830406 13368 raft_consensus.cc:1064] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1: attempting to promote NON_VOTER e56d487365394ac5ac5f1002926a5645 to VOTER
I20260709 12:09:37.833652 13368 consensus_queue.cc:237] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:37.840828 13430 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:37.849668 13339 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:37.853760 12525 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:37.855193 12605 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:37.853672 13429 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260709 12:09:37.863034 13429 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:37.865146 13429 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:37.920588 13338 raft_consensus.cc:2955] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:37.917024 13415 raft_consensus.cc:2955] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:37.948506 12530 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:37.946838 13368 raft_consensus.cc:2955] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:37.950184 12600 raft_consensus.cc:2955] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:37.953035 13430 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:37.953575 12605 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:37.953665 12598 raft_consensus.cc:2955] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:37.973606 13433 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:37.974061 13339 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:37.984751 13432 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:37.994349 12525 raft_consensus.cc:2955] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.027568 12429 catalog_manager.cc:5697] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:38.024096 12525 raft_consensus.cc:2955] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.045192 13337 raft_consensus.cc:2955] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.082854 12678 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:38.103960 12599 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:38.106000 13338 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:38.108456 13430 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:38.110096 13248 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:38.112036 12427 catalog_manager.cc:5697] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 reported cstate change: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.136210 13432 raft_consensus.cc:2955] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.149142 12675 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:38.177716 13431 raft_consensus.cc:2955] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.178774 12605 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:38.224946 12523 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:38.225606 13339 raft_consensus.cc:2955] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.230298 13283 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:38.234433 13337 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:38.236884 13338 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:38.239116 13424 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:38.245570 12598 raft_consensus.cc:2955] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.249678 13433 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:38.253499 13410 raft_consensus.cc:2955] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.259611 12530 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:38.263660 12530 raft_consensus.cc:2955] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.267009 13338 raft_consensus.cc:2955] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.276515 13368 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:38.285475 12413 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e40144c1c60847ee81b8252b107df67d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:38.290889 12426 catalog_manager.cc:5697] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 reported cstate change: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.299044 12429 catalog_manager.cc:5697] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: config changed from index 3 to 4, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:38.298861 13430 raft_consensus.cc:2955] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:09:38.323150 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58007.2us
I20260709 12:09:38.302695 12523 raft_consensus.cc:2955] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.339084 12662 tablet_service.cc:1558] Processing DeleteTablet for tablet e40144c1c60847ee81b8252b107df67d with delete_type TABLET_DATA_TOMBSTONED (TS 193a40da6e364ead8ab561e14c3cc8d1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51290
I20260709 12:09:38.307356 12602 raft_consensus.cc:2955] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.307579 13337 raft_consensus.cc:2955] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.376600 12530 raft_consensus.cc:2955] T e15d21db797e4a96b5732711a5379980 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.387761 12415 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet b7cccd56fa854cbc84a922238ad4add9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:38.397818 12429 catalog_manager.cc:5697] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 reported cstate change: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:09:38.424484 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 88256.4us
I20260709 12:09:38.433252 12427 catalog_manager.cc:5697] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.441723 12519 tablet_service.cc:1558] Processing DeleteTablet for tablet e1ff01ff4ede4fc990b9a07b23102b0d with delete_type TABLET_DATA_TOMBSTONED (TS a2fdbe7eef6748a9b4611f6c9bc9b201 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53846
I20260709 12:09:38.441640 13337 raft_consensus.cc:2955] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.453596 12590 tablet_service.cc:1558] Processing DeleteTablet for tablet b7cccd56fa854cbc84a922238ad4add9 with delete_type TABLET_DATA_TOMBSTONED (TS f5fc807f7ecc4d1ca0fd6be524f734ce not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51774
I20260709 12:09:38.504215 12415 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e1ff01ff4ede4fc990b9a07b23102b0d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:38.519477 13447 tablet_replica.cc:333] T e40144c1c60847ee81b8252b107df67d P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:09:38.520723 13447 raft_consensus.cc:2243] T e40144c1c60847ee81b8252b107df67d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:38.522058 13449 tablet_replica.cc:333] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:09:38.523348 13449 raft_consensus.cc:2243] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:38.524073 13449 raft_consensus.cc:2272] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:38.529775 13448 tablet_replica.cc:333] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:09:38.531000 13448 raft_consensus.cc:2243] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:38.531736 13448 raft_consensus.cc:2272] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:38.544168 13447 raft_consensus.cc:2272] T e40144c1c60847ee81b8252b107df67d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:38.552865 13448 ts_tablet_manager.cc:1916] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:38.560330 13447 ts_tablet_manager.cc:1916] T e40144c1c60847ee81b8252b107df67d P 193a40da6e364ead8ab561e14c3cc8d1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:38.563782 13449 ts_tablet_manager.cc:1916] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:38.582577 12678 consensus_queue.cc:237] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:38.593498 12605 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:38.593165 13337 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:38.598959 13431 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:38.600346 13432 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:38.619469 13448 ts_tablet_manager.cc:1929] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:38.620011 13448 log.cc:1199] T e1ff01ff4ede4fc990b9a07b23102b0d P a2fdbe7eef6748a9b4611f6c9bc9b201: Deleting WAL directory at /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-0-root/wals/e1ff01ff4ede4fc990b9a07b23102b0d
I20260709 12:09:38.621805 12416 catalog_manager.cc:5028] TS a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): tablet e1ff01ff4ede4fc990b9a07b23102b0d (table test-workload [id=8634097bac3246c9a57e272441058c28]) successfully deleted
I20260709 12:09:38.632154 13447 ts_tablet_manager.cc:1929] T e40144c1c60847ee81b8252b107df67d P 193a40da6e364ead8ab561e14c3cc8d1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:38.632674 13447 log.cc:1199] T e40144c1c60847ee81b8252b107df67d P 193a40da6e364ead8ab561e14c3cc8d1: Deleting WAL directory at /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-2-root/wals/e40144c1c60847ee81b8252b107df67d
I20260709 12:09:38.634383 12415 catalog_manager.cc:5028] TS 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): tablet e40144c1c60847ee81b8252b107df67d (table test-workload [id=8634097bac3246c9a57e272441058c28]) successfully deleted
I20260709 12:09:38.645684 13449 ts_tablet_manager.cc:1929] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:38.646246 13449 log.cc:1199] T b7cccd56fa854cbc84a922238ad4add9 P f5fc807f7ecc4d1ca0fd6be524f734ce: Deleting WAL directory at /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-1-root/wals/b7cccd56fa854cbc84a922238ad4add9
I20260709 12:09:38.647010 13450 raft_consensus.cc:2955] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.654664 12413 catalog_manager.cc:5028] TS f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): tablet b7cccd56fa854cbc84a922238ad4add9 (table test-workload [id=8634097bac3246c9a57e272441058c28]) successfully deleted
I20260709 12:09:38.683122 13337 raft_consensus.cc:2955] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.703896 12429 catalog_manager.cc:5697] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 reported cstate change: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.707397 12605 raft_consensus.cc:2955] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:38.716825 12519 tablet_service.cc:1558] Processing DeleteTablet for tablet e15d21db797e4a96b5732711a5379980 with delete_type TABLET_DATA_TOMBSTONED (TS a2fdbe7eef6748a9b4611f6c9bc9b201 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53846
I20260709 12:09:38.717880 13448 tablet_replica.cc:333] T e15d21db797e4a96b5732711a5379980 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:09:38.719007 13448 raft_consensus.cc:2243] T e15d21db797e4a96b5732711a5379980 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:38.719725 13448 raft_consensus.cc:2272] T e15d21db797e4a96b5732711a5379980 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:38.725382 13448 ts_tablet_manager.cc:1916] T e15d21db797e4a96b5732711a5379980 P a2fdbe7eef6748a9b4611f6c9bc9b201: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:38.726595 12415 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet e15d21db797e4a96b5732711a5379980 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:38.766345 13448 ts_tablet_manager.cc:1929] T e15d21db797e4a96b5732711a5379980 P a2fdbe7eef6748a9b4611f6c9bc9b201: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:38.766991 13448 log.cc:1199] T e15d21db797e4a96b5732711a5379980 P a2fdbe7eef6748a9b4611f6c9bc9b201: Deleting WAL directory at /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-0-root/wals/e15d21db797e4a96b5732711a5379980
I20260709 12:09:38.771246 12416 catalog_manager.cc:5028] TS a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): tablet e15d21db797e4a96b5732711a5379980 (table test-workload [id=8634097bac3246c9a57e272441058c28]) successfully deleted
W20260709 12:09:39.790638 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 73049us
I20260709 12:09:39.863139 12526 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:39.893368 12675 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:39.896581 12605 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:39.898370 13411 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:39.902204 13455 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:39.921176 13451 raft_consensus.cc:2955] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
W20260709 12:09:39.955739 12494 consensus_peers.cc:597] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: d7bd864b75c34027b05582a55b152877. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:39.957350 12605 raft_consensus.cc:2955] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:39.963092 12675 raft_consensus.cc:2955] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:39.984781 12526 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:40.005748 12675 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:40.007104 12426 catalog_manager.cc:5697] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:40.008453 13287 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:40.013622 12602 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:09:40.015677 12494 consensus_peers.cc:597] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a92d1d91cce04cec98366aa6021356df. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:40.022276 13287 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:40.085590 12671 raft_consensus.cc:2955] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:40.093164 12602 raft_consensus.cc:2955] T a92d1d91cce04cec98366aa6021356df P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:40.081629 13411 raft_consensus.cc:2955] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:40.185650 12429 catalog_manager.cc:5697] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:40.189124 12526 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:40.231909 12671 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:09:40.233317 12494 consensus_peers.cc:597] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 33a0b4d6cfba4ab2b3d54f591439b179. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:40.234956 12605 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:40.237040 13455 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:40.240290 13287 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:40.260162 13455 raft_consensus.cc:2955] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:40.267936 12671 raft_consensus.cc:2955] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:40.306301 12427 catalog_manager.cc:5697] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:40.322167 12600 raft_consensus.cc:2955] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
W20260709 12:09:40.369689 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 55021.5us
I20260709 12:09:40.381145 12526 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
W20260709 12:09:40.394449 12494 consensus_peers.cc:597] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ee30d7cd60b04485b7e19c8d119d67bd. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:40.396322 12600 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:40.400096 13455 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:40.397550 12671 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:40.405956 13411 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:40.459431 13455 raft_consensus.cc:2955] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:40.463235 13479 ts_tablet_manager.cc:933] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:09:40.471344 12675 raft_consensus.cc:2955] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:40.483436 13479 tablet_copy_client.cc:323] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.129:38811
I20260709 12:09:40.516165 13481 ts_tablet_manager.cc:933] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:09:40.538645 13481 tablet_copy_client.cc:323] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.129:38811
I20260709 12:09:40.549489 12427 catalog_manager.cc:5697] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:40.586015 12602 raft_consensus.cc:2955] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:40.672617 12541 tablet_copy_service.cc:140] P a2fdbe7eef6748a9b4611f6c9bc9b201: Received BeginTabletCopySession request for tablet d7bd864b75c34027b05582a55b152877 from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:60222)
I20260709 12:09:40.673301 12541 tablet_copy_service.cc:161] P a2fdbe7eef6748a9b4611f6c9bc9b201: Beginning new tablet copy session on tablet d7bd864b75c34027b05582a55b152877 from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:60222: session id = e56d487365394ac5ac5f1002926a5645-d7bd864b75c34027b05582a55b152877
I20260709 12:09:40.678494 12540 tablet_copy_service.cc:140] P a2fdbe7eef6748a9b4611f6c9bc9b201: Received BeginTabletCopySession request for tablet a92d1d91cce04cec98366aa6021356df from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:60222)
I20260709 12:09:40.679194 12540 tablet_copy_service.cc:161] P a2fdbe7eef6748a9b4611f6c9bc9b201: Beginning new tablet copy session on tablet a92d1d91cce04cec98366aa6021356df from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:60222: session id = e56d487365394ac5ac5f1002926a5645-a92d1d91cce04cec98366aa6021356df
I20260709 12:09:40.691572 12541 tablet_copy_source_session.cc:215] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:40.710393 13479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7bd864b75c34027b05582a55b152877. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:40.694868 12540 tablet_copy_source_session.cc:215] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:40.721962 13481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a92d1d91cce04cec98366aa6021356df. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:40.750252 13479 tablet_copy_client.cc:806] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:40.751071 13479 tablet_copy_client.cc:670] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:40.755541 13479 tablet_copy_client.cc:538] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:40.774186 13479 tablet_bootstrap.cc:492] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
I20260709 12:09:40.811721 13485 ts_tablet_manager.cc:933] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:09:40.818287 13481 tablet_copy_client.cc:806] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:40.819231 13481 tablet_copy_client.cc:670] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:40.824492 13481 tablet_copy_client.cc:538] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:40.849553 13481 tablet_bootstrap.cc:492] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
I20260709 12:09:40.850876 13485 tablet_copy_client.cc:323] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.129:38811
I20260709 12:09:40.872038 12541 tablet_copy_service.cc:140] P a2fdbe7eef6748a9b4611f6c9bc9b201: Received BeginTabletCopySession request for tablet 33a0b4d6cfba4ab2b3d54f591439b179 from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:60222)
I20260709 12:09:40.872637 12541 tablet_copy_service.cc:161] P a2fdbe7eef6748a9b4611f6c9bc9b201: Beginning new tablet copy session on tablet 33a0b4d6cfba4ab2b3d54f591439b179 from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:60222: session id = e56d487365394ac5ac5f1002926a5645-33a0b4d6cfba4ab2b3d54f591439b179
I20260709 12:09:40.878005 13479 tablet_bootstrap.cc:492] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:40.892222 13479 tablet_bootstrap.cc:492] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
I20260709 12:09:40.894809 13479 ts_tablet_manager.cc:1403] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 0.129s	user 0.028s	sys 0.001s
I20260709 12:09:40.905342 13479 raft_consensus.cc:359] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:40.911410 13479 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
I20260709 12:09:40.913120 12541 tablet_copy_source_session.cc:215] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:40.913751 13479 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:40.922710 13485 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 33a0b4d6cfba4ab2b3d54f591439b179. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:40.935808 13481 tablet_bootstrap.cc:492] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:40.936039 13337 raft_consensus.cc:1217] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:40.937184 13481 tablet_bootstrap.cc:492] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
I20260709 12:09:40.937275 13479 ts_tablet_manager.cc:1434] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.035s	user 0.005s	sys 0.004s
I20260709 12:09:40.938108 13481 ts_tablet_manager.cc:1403] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 0.089s	user 0.021s	sys 0.008s
I20260709 12:09:40.942057 13481 raft_consensus.cc:359] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:40.943207 13481 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
I20260709 12:09:40.944072 13481 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:40.950080 12541 tablet_copy_service.cc:342] P a2fdbe7eef6748a9b4611f6c9bc9b201: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-d7bd864b75c34027b05582a55b152877 received from {username='slave'} at 127.0.0.1:60222
I20260709 12:09:40.950707 12541 tablet_copy_service.cc:434] P a2fdbe7eef6748a9b4611f6c9bc9b201: ending tablet copy session e56d487365394ac5ac5f1002926a5645-d7bd864b75c34027b05582a55b152877 on tablet d7bd864b75c34027b05582a55b152877 with peer e56d487365394ac5ac5f1002926a5645
I20260709 12:09:40.972442 13479 ts_tablet_manager.cc:933] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:09:40.990643 13479 tablet_copy_client.cc:323] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.129:38811
I20260709 12:09:41.003952 12541 tablet_copy_service.cc:140] P a2fdbe7eef6748a9b4611f6c9bc9b201: Received BeginTabletCopySession request for tablet ee30d7cd60b04485b7e19c8d119d67bd from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:60222)
I20260709 12:09:41.004855 12541 tablet_copy_service.cc:161] P a2fdbe7eef6748a9b4611f6c9bc9b201: Beginning new tablet copy session on tablet ee30d7cd60b04485b7e19c8d119d67bd from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:60222: session id = e56d487365394ac5ac5f1002926a5645-ee30d7cd60b04485b7e19c8d119d67bd
I20260709 12:09:41.058983 12541 tablet_copy_source_session.cc:215] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:41.061115 13481 ts_tablet_manager.cc:1434] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.122s	user 0.031s	sys 0.054s
I20260709 12:09:41.079228 13479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee30d7cd60b04485b7e19c8d119d67bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:41.087460 12541 tablet_copy_service.cc:342] P a2fdbe7eef6748a9b4611f6c9bc9b201: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-a92d1d91cce04cec98366aa6021356df received from {username='slave'} at 127.0.0.1:60222
I20260709 12:09:41.089077 12541 tablet_copy_service.cc:434] P a2fdbe7eef6748a9b4611f6c9bc9b201: ending tablet copy session e56d487365394ac5ac5f1002926a5645-a92d1d91cce04cec98366aa6021356df on tablet a92d1d91cce04cec98366aa6021356df with peer e56d487365394ac5ac5f1002926a5645
W20260709 12:09:41.145381 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 45528.5us
W20260709 12:09:41.163818 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 62930.6us
I20260709 12:09:41.171061 13337 raft_consensus.cc:1217] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:09:41.242692 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 85439.2us
I20260709 12:09:41.273890 13479 tablet_copy_client.cc:806] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:41.274638 13485 tablet_copy_client.cc:806] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:41.274742 13479 tablet_copy_client.cc:670] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:41.275663 13485 tablet_copy_client.cc:670] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:41.283070 13479 tablet_copy_client.cc:538] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:41.309250 13485 tablet_copy_client.cc:538] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:41.321128 13485 tablet_bootstrap.cc:492] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
I20260709 12:09:41.323767 13479 tablet_bootstrap.cc:492] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
I20260709 12:09:41.363443 13485 tablet_bootstrap.cc:492] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:41.364575 13485 tablet_bootstrap.cc:492] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
I20260709 12:09:41.365355 13485 ts_tablet_manager.cc:1403] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 0.045s	user 0.011s	sys 0.023s
I20260709 12:09:41.368813 13485 raft_consensus.cc:359] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:41.369781 13485 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
I20260709 12:09:41.370978 13485 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:41.373821 13485 ts_tablet_manager.cc:1434] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:09:41.376223 12540 tablet_copy_service.cc:342] P a2fdbe7eef6748a9b4611f6c9bc9b201: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-33a0b4d6cfba4ab2b3d54f591439b179 received from {username='slave'} at 127.0.0.1:60222
I20260709 12:09:41.376737 12540 tablet_copy_service.cc:434] P a2fdbe7eef6748a9b4611f6c9bc9b201: ending tablet copy session e56d487365394ac5ac5f1002926a5645-33a0b4d6cfba4ab2b3d54f591439b179 on tablet 33a0b4d6cfba4ab2b3d54f591439b179 with peer e56d487365394ac5ac5f1002926a5645
I20260709 12:09:41.392210 13479 tablet_bootstrap.cc:492] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:41.393380 13479 tablet_bootstrap.cc:492] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
I20260709 12:09:41.394233 13479 ts_tablet_manager.cc:1403] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 0.071s	user 0.027s	sys 0.006s
I20260709 12:09:41.398435 13479 raft_consensus.cc:359] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:41.399442 13479 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
I20260709 12:09:41.400369 13479 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:41.403438 13479 ts_tablet_manager.cc:1434] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:09:41.405455 12540 tablet_copy_service.cc:342] P a2fdbe7eef6748a9b4611f6c9bc9b201: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-ee30d7cd60b04485b7e19c8d119d67bd received from {username='slave'} at 127.0.0.1:60222
I20260709 12:09:41.405939 12540 tablet_copy_service.cc:434] P a2fdbe7eef6748a9b4611f6c9bc9b201: ending tablet copy session e56d487365394ac5ac5f1002926a5645-ee30d7cd60b04485b7e19c8d119d67bd on tablet ee30d7cd60b04485b7e19c8d119d67bd with peer e56d487365394ac5ac5f1002926a5645
I20260709 12:09:41.448344 13337 raft_consensus.cc:1217] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:41.468914 13287 raft_consensus.cc:1064] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: attempting to promote NON_VOTER e56d487365394ac5ac5f1002926a5645 to VOTER
I20260709 12:09:41.471887 13287 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:41.491609 12605 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:41.494928 13337 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:41.496992 13287 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:41.498183 12678 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:41.523725 13287 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:41.539682 13451 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:41.561753 13411 raft_consensus.cc:2955] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:41.593641 12675 raft_consensus.cc:2955] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:41.606304 13455 raft_consensus.cc:1064] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: attempting to promote NON_VOTER e56d487365394ac5ac5f1002926a5645 to VOTER
W20260709 12:09:41.641165 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 83694.5us
I20260709 12:09:41.667332 13455 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:41.739398 13337 raft_consensus.cc:1217] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:41.876163 12602 raft_consensus.cc:2955] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:41.899739 13337 raft_consensus.cc:2955] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:41.914848 12427 catalog_manager.cc:5697] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:41.933259 13338 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:41.935400 12675 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:41.935077 13455 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:41.937345 12602 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:41.938884 13411 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:41.949419 13455 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:42.016484 13509 raft_consensus.cc:2955] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.018929 12675 raft_consensus.cc:2955] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.049108 13337 raft_consensus.cc:2955] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.068727 13287 raft_consensus.cc:1064] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: attempting to promote NON_VOTER e56d487365394ac5ac5f1002926a5645 to VOTER
I20260709 12:09:42.074237 13287 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:42.085827 12597 raft_consensus.cc:2955] T a92d1d91cce04cec98366aa6021356df P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.120187 12526 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:42.120100 13337 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:42.127878 12428 catalog_manager.cc:5697] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 reported cstate change: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.131843 13508 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:42.136380 12597 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:42.138003 13337 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:42.139770 13508 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:42.147898 13508 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:42.151882 12602 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:42.156293 12675 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:42.154347 13509 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:42.160033 13287 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:42.180085 13287 raft_consensus.cc:2955] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.195099 12598 raft_consensus.cc:2955] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.202874 12526 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:42.208194 12416 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet d7bd864b75c34027b05582a55b152877 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:42.229027 12671 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:42.230639 13509 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:42.218791 13337 raft_consensus.cc:2955] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.242393 13519 raft_consensus.cc:2955] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.247862 13338 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:42.248044 12676 raft_consensus.cc:2955] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.261608 13509 raft_consensus.cc:2955] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.268071 13411 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:42.264101 12427 catalog_manager.cc:5697] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: config changed from index 3 to 4, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:42.265153 12605 raft_consensus.cc:2955] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.272846 13508 raft_consensus.cc:1064] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: attempting to promote NON_VOTER e56d487365394ac5ac5f1002926a5645 to VOTER
I20260709 12:09:42.272338 12429 catalog_manager.cc:5697] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.275595 13508 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:42.284848 12675 raft_consensus.cc:2955] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.338788 13338 raft_consensus.cc:2955] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.348516 12662 tablet_service.cc:1558] Processing DeleteTablet for tablet d7bd864b75c34027b05582a55b152877 with delete_type TABLET_DATA_TOMBSTONED (TS 193a40da6e364ead8ab561e14c3cc8d1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51290
I20260709 12:09:42.360832 12590 tablet_service.cc:1558] Processing DeleteTablet for tablet a92d1d91cce04cec98366aa6021356df with delete_type TABLET_DATA_TOMBSTONED (TS f5fc807f7ecc4d1ca0fd6be524f734ce not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51774
I20260709 12:09:42.394918 12416 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet a92d1d91cce04cec98366aa6021356df with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:42.410521 12426 catalog_manager.cc:5697] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.414031 13525 tablet_replica.cc:333] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:09:42.415227 13525 raft_consensus.cc:2243] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:42.417901 13339 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:42.417816 13337 raft_consensus.cc:2955] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.436025 13525 raft_consensus.cc:2272] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:42.453298 13525 ts_tablet_manager.cc:1916] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:42.445367 13519 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:09:42.474226 12605 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:42.476684 13509 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:42.483202 13526 tablet_replica.cc:333] T a92d1d91cce04cec98366aa6021356df P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:09:42.485667 13526 raft_consensus.cc:2243] T a92d1d91cce04cec98366aa6021356df P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:42.490298 12675 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:42.498855 13526 raft_consensus.cc:2272] T a92d1d91cce04cec98366aa6021356df P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:42.508781 13287 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:42.574011 13526 ts_tablet_manager.cc:1916] T a92d1d91cce04cec98366aa6021356df P f5fc807f7ecc4d1ca0fd6be524f734ce: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:42.603868 13455 raft_consensus.cc:2955] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.605741 12605 raft_consensus.cc:2955] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:09:42.618222 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 75989.2us
I20260709 12:09:42.621863 13337 raft_consensus.cc:2955] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.629036 12526 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:42.637919 12427 catalog_manager.cc:5697] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:09:42.643537 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 85474.9us
I20260709 12:09:42.653964 13338 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:42.656350 13519 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:42.662322 12675 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:42.680907 13455 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:42.684408 13287 raft_consensus.cc:2955] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.688472 13338 raft_consensus.cc:2955] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.694705 12675 raft_consensus.cc:2955] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.696132 12416 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ee30d7cd60b04485b7e19c8d119d67bd with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:42.702328 12676 raft_consensus.cc:2955] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.714520 12530 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:42.731267 12605 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:42.732540 12427 catalog_manager.cc:5697] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 reported cstate change: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.734283 13525 ts_tablet_manager.cc:1929] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:42.738426 13525 log.cc:1199] T d7bd864b75c34027b05582a55b152877 P 193a40da6e364ead8ab561e14c3cc8d1: Deleting WAL directory at /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-2-root/wals/d7bd864b75c34027b05582a55b152877
I20260709 12:09:42.740088 13455 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:42.744431 13338 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:42.745460 12415 catalog_manager.cc:5028] TS 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): tablet d7bd864b75c34027b05582a55b152877 (table test-workload [id=8634097bac3246c9a57e272441058c28]) successfully deleted
I20260709 12:09:42.745971 13526 ts_tablet_manager.cc:1929] T a92d1d91cce04cec98366aa6021356df P f5fc807f7ecc4d1ca0fd6be524f734ce: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:42.746572 13526 log.cc:1199] T a92d1d91cce04cec98366aa6021356df P f5fc807f7ecc4d1ca0fd6be524f734ce: Deleting WAL directory at /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-1-root/wals/a92d1d91cce04cec98366aa6021356df
I20260709 12:09:42.747095 13509 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:42.748893 12413 catalog_manager.cc:5028] TS f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): tablet a92d1d91cce04cec98366aa6021356df (table test-workload [id=8634097bac3246c9a57e272441058c28]) successfully deleted
I20260709 12:09:42.759343 13451 raft_consensus.cc:2955] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.767035 12602 raft_consensus.cc:2955] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.777684 13337 raft_consensus.cc:2955] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.790868 12426 catalog_manager.cc:5697] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: config changed from index 3 to 4, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:42.806362 12416 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 33a0b4d6cfba4ab2b3d54f591439b179 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:42.806388 12590 tablet_service.cc:1558] Processing DeleteTablet for tablet ee30d7cd60b04485b7e19c8d119d67bd with delete_type TABLET_DATA_TOMBSTONED (TS f5fc807f7ecc4d1ca0fd6be524f734ce not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51774
I20260709 12:09:42.807701 13526 tablet_replica.cc:333] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:09:42.808853 13526 raft_consensus.cc:2243] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:42.809584 13526 raft_consensus.cc:2272] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:42.814726 13526 ts_tablet_manager.cc:1916] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:42.825862 12662 tablet_service.cc:1558] Processing DeleteTablet for tablet 33a0b4d6cfba4ab2b3d54f591439b179 with delete_type TABLET_DATA_TOMBSTONED (TS 193a40da6e364ead8ab561e14c3cc8d1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51290
I20260709 12:09:42.827091 13525 tablet_replica.cc:333] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:09:42.828251 13525 raft_consensus.cc:2243] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:42.829826 13525 raft_consensus.cc:2272] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:42.848340 13525 ts_tablet_manager.cc:1916] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:42.868058 13526 ts_tablet_manager.cc:1929] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:42.868772 13526 log.cc:1199] T ee30d7cd60b04485b7e19c8d119d67bd P f5fc807f7ecc4d1ca0fd6be524f734ce: Deleting WAL directory at /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-1-root/wals/ee30d7cd60b04485b7e19c8d119d67bd
I20260709 12:09:42.871543 12413 catalog_manager.cc:5028] TS f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): tablet ee30d7cd60b04485b7e19c8d119d67bd (table test-workload [id=8634097bac3246c9a57e272441058c28]) successfully deleted
I20260709 12:09:42.898478 13525 ts_tablet_manager.cc:1929] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:42.899085 13525 log.cc:1199] T 33a0b4d6cfba4ab2b3d54f591439b179 P 193a40da6e364ead8ab561e14c3cc8d1: Deleting WAL directory at /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-2-root/wals/33a0b4d6cfba4ab2b3d54f591439b179
I20260709 12:09:42.940678 12415 catalog_manager.cc:5028] TS 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): tablet 33a0b4d6cfba4ab2b3d54f591439b179 (table test-workload [id=8634097bac3246c9a57e272441058c28]) successfully deleted
W20260709 12:09:43.913054 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46163.5us
I20260709 12:09:44.039115 12675 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:44.064028 12530 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:44.064781 12605 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:44.066463 13511 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:09:44.069172 12643 consensus_peers.cc:597] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6fca0ed05cd34c8e92dc950351b864a2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:44.072757 13538 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:44.082412 13538 raft_consensus.cc:2955] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:44.095043 12523 raft_consensus.cc:2955] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:44.108325 12605 raft_consensus.cc:2955] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:44.121359 12426 catalog_manager.cc:5697] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:44.164851 12523 consensus_queue.cc:237] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:44.186043 12676 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:09:44.189956 12494 consensus_peers.cc:597] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3c00651839ae4132bb331bc6c2602641. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:44.192728 12605 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:44.195358 13451 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:44.200667 13540 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:44.259626 13540 raft_consensus.cc:2955] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:44.269735 12602 raft_consensus.cc:2955] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:44.280400 12671 raft_consensus.cc:2955] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:44.309432 12671 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:44.339150 12600 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:09:44.339881 12643 consensus_peers.cc:597] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 4270d5e0462b49d2982145525764c7fa. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:44.340835 13538 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:44.341285 12429 catalog_manager.cc:5697] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:44.342290 12530 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:44.344017 13538 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:44.369732 13536 raft_consensus.cc:2955] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:44.374190 12600 raft_consensus.cc:2955] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:44.384900 12523 raft_consensus.cc:2955] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:44.413645 12429 catalog_manager.cc:5697] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:44.417351 12530 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:44.449093 12676 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:44.451097 13539 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:44.463064 12605 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:09:44.466650 12494 consensus_peers.cc:597] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 618bbf28983a42beaaf337df12784ee1. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:44.472014 13509 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:44.487968 13540 raft_consensus.cc:2955] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:44.493851 12675 raft_consensus.cc:2955] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:44.548799 12429 catalog_manager.cc:5697] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:44.596539 13560 ts_tablet_manager.cc:933] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:09:44.592077 12605 raft_consensus.cc:2955] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:44.623140 13560 tablet_copy_client.cc:323] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.129:38811
I20260709 12:09:44.625794 12540 tablet_copy_service.cc:140] P a2fdbe7eef6748a9b4611f6c9bc9b201: Received BeginTabletCopySession request for tablet 3c00651839ae4132bb331bc6c2602641 from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:60222)
I20260709 12:09:44.626597 12540 tablet_copy_service.cc:161] P a2fdbe7eef6748a9b4611f6c9bc9b201: Beginning new tablet copy session on tablet 3c00651839ae4132bb331bc6c2602641 from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:60222: session id = e56d487365394ac5ac5f1002926a5645-3c00651839ae4132bb331bc6c2602641
I20260709 12:09:44.635429 13559 ts_tablet_manager.cc:933] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:44.662493 13559 tablet_copy_client.cc:323] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.131:38613
I20260709 12:09:44.664809 12690 tablet_copy_service.cc:140] P 193a40da6e364ead8ab561e14c3cc8d1: Received BeginTabletCopySession request for tablet 6fca0ed05cd34c8e92dc950351b864a2 from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:40246)
I20260709 12:09:44.665434 12690 tablet_copy_service.cc:161] P 193a40da6e364ead8ab561e14c3cc8d1: Beginning new tablet copy session on tablet 6fca0ed05cd34c8e92dc950351b864a2 from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:40246: session id = e56d487365394ac5ac5f1002926a5645-6fca0ed05cd34c8e92dc950351b864a2
I20260709 12:09:44.674909 12690 tablet_copy_source_session.cc:215] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:44.680382 13559 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6fca0ed05cd34c8e92dc950351b864a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:44.724843 13559 tablet_copy_client.cc:806] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:44.725602 13559 tablet_copy_client.cc:670] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:44.731263 13559 tablet_copy_client.cc:538] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:44.751612 12540 tablet_copy_source_session.cc:215] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:44.742039 13559 tablet_bootstrap.cc:492] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
I20260709 12:09:44.803217 13559 tablet_bootstrap.cc:492] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:44.811071 13559 tablet_bootstrap.cc:492] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
I20260709 12:09:44.812011 13559 ts_tablet_manager.cc:1403] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 0.070s	user 0.022s	sys 0.013s
I20260709 12:09:44.815742 13559 raft_consensus.cc:359] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:44.816802 13559 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
I20260709 12:09:44.818751 13559 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:44.824595 13560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c00651839ae4132bb331bc6c2602641. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:09:44.893461 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 71859.2us
I20260709 12:09:44.907438 13564 ts_tablet_manager.cc:933] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:44.916239 13564 tablet_copy_client.cc:323] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.131:38613
I20260709 12:09:44.918352 12690 tablet_copy_service.cc:140] P 193a40da6e364ead8ab561e14c3cc8d1: Received BeginTabletCopySession request for tablet 4270d5e0462b49d2982145525764c7fa from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:40246)
I20260709 12:09:44.919199 12690 tablet_copy_service.cc:161] P 193a40da6e364ead8ab561e14c3cc8d1: Beginning new tablet copy session on tablet 4270d5e0462b49d2982145525764c7fa from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:40246: session id = e56d487365394ac5ac5f1002926a5645-4270d5e0462b49d2982145525764c7fa
I20260709 12:09:44.930269 12690 tablet_copy_source_session.cc:215] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:44.913081 13559 ts_tablet_manager.cc:1434] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.101s	user 0.006s	sys 0.004s
I20260709 12:09:44.936525 13564 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4270d5e0462b49d2982145525764c7fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:44.959426 12690 tablet_copy_service.cc:342] P 193a40da6e364ead8ab561e14c3cc8d1: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-6fca0ed05cd34c8e92dc950351b864a2 received from {username='slave'} at 127.0.0.1:40246
I20260709 12:09:44.964198 12690 tablet_copy_service.cc:434] P 193a40da6e364ead8ab561e14c3cc8d1: ending tablet copy session e56d487365394ac5ac5f1002926a5645-6fca0ed05cd34c8e92dc950351b864a2 on tablet 6fca0ed05cd34c8e92dc950351b864a2 with peer e56d487365394ac5ac5f1002926a5645
I20260709 12:09:44.965600 13564 tablet_copy_client.cc:806] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:44.977883 13560 tablet_copy_client.cc:806] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:44.978871 13560 tablet_copy_client.cc:670] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:44.982448 13564 tablet_copy_client.cc:670] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:44.987149 13560 tablet_copy_client.cc:538] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:44.988318 13564 tablet_copy_client.cc:538] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:45.001909 13564 tablet_bootstrap.cc:492] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
I20260709 12:09:45.006556 13560 tablet_bootstrap.cc:492] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
W20260709 12:09:45.069813 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 48680.2us
I20260709 12:09:45.078210 13337 raft_consensus.cc:1217] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:45.078603 13564 tablet_bootstrap.cc:492] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:45.085613 13564 tablet_bootstrap.cc:492] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
I20260709 12:09:45.086548 13564 ts_tablet_manager.cc:1403] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 0.085s	user 0.028s	sys 0.008s
I20260709 12:09:45.090560 13560 tablet_bootstrap.cc:492] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:45.092068 13560 tablet_bootstrap.cc:492] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
I20260709 12:09:45.093138 13560 ts_tablet_manager.cc:1403] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 0.087s	user 0.025s	sys 0.004s
W20260709 12:09:45.140373 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 67140.3us
W20260709 12:09:45.145833 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 50053.2us
I20260709 12:09:45.154151 13560 raft_consensus.cc:359] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:45.155061 13560 raft_consensus.cc:740] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
I20260709 12:09:45.155793 13560 consensus_queue.cc:260] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:45.152555 13564 raft_consensus.cc:359] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:45.184672 13560 ts_tablet_manager.cc:1434] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.091s	user 0.010s	sys 0.000s
I20260709 12:09:45.169188 13564 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
I20260709 12:09:45.188575 13564 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:45.197250 12540 tablet_copy_service.cc:342] P a2fdbe7eef6748a9b4611f6c9bc9b201: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-3c00651839ae4132bb331bc6c2602641 received from {username='slave'} at 127.0.0.1:60222
I20260709 12:09:45.197885 12540 tablet_copy_service.cc:434] P a2fdbe7eef6748a9b4611f6c9bc9b201: ending tablet copy session e56d487365394ac5ac5f1002926a5645-3c00651839ae4132bb331bc6c2602641 on tablet 3c00651839ae4132bb331bc6c2602641 with peer e56d487365394ac5ac5f1002926a5645
I20260709 12:09:45.200492 13564 ts_tablet_manager.cc:1434] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.114s	user 0.007s	sys 0.001s
I20260709 12:09:45.216981 12690 tablet_copy_service.cc:342] P 193a40da6e364ead8ab561e14c3cc8d1: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-4270d5e0462b49d2982145525764c7fa received from {username='slave'} at 127.0.0.1:40246
I20260709 12:09:45.217461 12690 tablet_copy_service.cc:434] P 193a40da6e364ead8ab561e14c3cc8d1: ending tablet copy session e56d487365394ac5ac5f1002926a5645-4270d5e0462b49d2982145525764c7fa on tablet 4270d5e0462b49d2982145525764c7fa with peer e56d487365394ac5ac5f1002926a5645
I20260709 12:09:45.222151 13560 ts_tablet_manager.cc:933] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:09:45.222960 13337 raft_consensus.cc:1217] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:45.252663 13560 tablet_copy_client.cc:323] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.129:38811
I20260709 12:09:45.265491 12540 tablet_copy_service.cc:140] P a2fdbe7eef6748a9b4611f6c9bc9b201: Received BeginTabletCopySession request for tablet 618bbf28983a42beaaf337df12784ee1 from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:60222)
I20260709 12:09:45.266091 12540 tablet_copy_service.cc:161] P a2fdbe7eef6748a9b4611f6c9bc9b201: Beginning new tablet copy session on tablet 618bbf28983a42beaaf337df12784ee1 from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:60222: session id = e56d487365394ac5ac5f1002926a5645-618bbf28983a42beaaf337df12784ee1
I20260709 12:09:45.284785 12540 tablet_copy_source_session.cc:215] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:45.290661 13560 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 618bbf28983a42beaaf337df12784ee1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:45.295042 13337 raft_consensus.cc:1217] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:45.341514 13560 tablet_copy_client.cc:806] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:45.342635 13560 tablet_copy_client.cc:670] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:45.347559 13560 tablet_copy_client.cc:538] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:45.373311 13560 tablet_bootstrap.cc:492] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
I20260709 12:09:45.457904 13560 tablet_bootstrap.cc:492] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:45.459342 13560 tablet_bootstrap.cc:492] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
I20260709 12:09:45.460302 13560 ts_tablet_manager.cc:1403] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 0.087s	user 0.027s	sys 0.005s
I20260709 12:09:45.463778 13560 raft_consensus.cc:359] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:45.464809 13560 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
I20260709 12:09:45.465549 13560 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:45.468428 13560 ts_tablet_manager.cc:1434] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:09:45.471028 12540 tablet_copy_service.cc:342] P a2fdbe7eef6748a9b4611f6c9bc9b201: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-618bbf28983a42beaaf337df12784ee1 received from {username='slave'} at 127.0.0.1:60222
I20260709 12:09:45.471570 12540 tablet_copy_service.cc:434] P a2fdbe7eef6748a9b4611f6c9bc9b201: ending tablet copy session e56d487365394ac5ac5f1002926a5645-618bbf28983a42beaaf337df12784ee1 on tablet 618bbf28983a42beaaf337df12784ee1 with peer e56d487365394ac5ac5f1002926a5645
I20260709 12:09:45.550410 13337 raft_consensus.cc:1217] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:45.617460 13574 raft_consensus.cc:1064] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: attempting to promote NON_VOTER e56d487365394ac5ac5f1002926a5645 to VOTER
I20260709 12:09:45.620208 13574 consensus_queue.cc:237] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:45.649650 13338 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:45.650620 12602 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:45.655112 12671 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:45.657354 13509 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:45.659529 13509 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:45.664211 13572 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:45.679032 13511 raft_consensus.cc:1064] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: attempting to promote NON_VOTER e56d487365394ac5ac5f1002926a5645 to VOTER
I20260709 12:09:45.681514 13511 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:45.697825 13540 raft_consensus.cc:2955] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:45.706489 13538 raft_consensus.cc:1064] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: attempting to promote NON_VOTER e56d487365394ac5ac5f1002926a5645 to VOTER
I20260709 12:09:45.709450 13538 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:45.723399 13338 raft_consensus.cc:2955] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:09:45.860816 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 107892us
I20260709 12:09:45.879491 13337 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:45.886570 12429 catalog_manager.cc:5697] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 reported cstate change: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:45.890686 12523 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:45.917412 13570 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:45.918315 12602 raft_consensus.cc:2955] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:45.950241 12523 consensus_queue.cc:237] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:09:45.957276 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 44237.6us
I20260709 12:09:45.969007 12676 raft_consensus.cc:2955] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:45.982729 12602 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:45.998602 13338 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:46.063472 12523 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:46.066179 13339 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:46.069613 12600 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:46.071996 12602 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:46.078392 13511 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:46.078800 13569 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:46.084007 13540 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:46.090543 13572 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:46.112877 13509 raft_consensus.cc:1064] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: attempting to promote NON_VOTER e56d487365394ac5ac5f1002926a5645 to VOTER
I20260709 12:09:46.139571 13509 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:09:46.163534 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 62405.7us
I20260709 12:09:46.165904 13501 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:46.170854 13592 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:09:46.163587 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 46139.3us
I20260709 12:09:46.181581 13538 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:09:46.233597 13580 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.136s	user 0.018s	sys 0.081s
W20260709 12:09:46.233608 13579 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.124s	user 0.007s	sys 0.014s
W20260709 12:09:46.239473 13581 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.139s	user 0.000s	sys 0.021s
I20260709 12:09:46.241477 13287 raft_consensus.cc:2955] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:09:46.256263 13600 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.135s	user 0.011s	sys 0.000s
I20260709 12:09:46.284528 12597 raft_consensus.cc:2955] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.275727 13538 raft_consensus.cc:2955] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.284387 13338 raft_consensus.cc:2955] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.302089 12598 raft_consensus.cc:2955] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.322067 12416 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 3c00651839ae4132bb331bc6c2602641 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:46.329350 12676 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:46.338387 12602 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:46.340298 12526 raft_consensus.cc:2955] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.363288 13540 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:46.340006 13339 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:46.378868 12427 catalog_manager.cc:5697] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 reported cstate change: config changed from index 3 to 4, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.408105 13598 raft_consensus.cc:2955] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.444039 12429 catalog_manager.cc:5697] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.388135 13574 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:09:46.446621 13575 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:46.432669 13337 raft_consensus.cc:2955] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.436681 13339 raft_consensus.cc:2955] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.456416 12662 tablet_service.cc:1558] Processing DeleteTablet for tablet 3c00651839ae4132bb331bc6c2602641 with delete_type TABLET_DATA_TOMBSTONED (TS 193a40da6e364ead8ab561e14c3cc8d1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51290
I20260709 12:09:46.527251 13609 tablet_replica.cc:333] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:09:46.528514 13609 raft_consensus.cc:2243] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:46.535302 12522 raft_consensus.cc:2955] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.559719 12676 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:46.577692 13337 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260709 12:09:46.577970 13611 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.056s	user 0.000s	sys 0.002s
I20260709 12:09:46.579514 13570 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:46.580906 13609 raft_consensus.cc:2272] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:46.594553 13501 raft_consensus.cc:2955] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:09:46.599494 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 184946us
I20260709 12:09:46.599713 13337 raft_consensus.cc:2955] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.620831 13609 ts_tablet_manager.cc:1916] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260709 12:09:46.622908 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 56299.9us
I20260709 12:09:46.613786 12427 catalog_manager.cc:5697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 reported cstate change: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.654059 13509 raft_consensus.cc:2955] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.673899 12426 catalog_manager.cc:5697] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.675838 12671 raft_consensus.cc:2955] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.681565 12599 raft_consensus.cc:2955] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.685472 12598 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:46.686695 12600 raft_consensus.cc:2955] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.684702 13338 raft_consensus.cc:2955] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.694538 12415 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 6fca0ed05cd34c8e92dc950351b864a2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:46.706439 13538 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:46.737090 12519 tablet_service.cc:1558] Processing DeleteTablet for tablet 6fca0ed05cd34c8e92dc950351b864a2 with delete_type TABLET_DATA_TOMBSTONED (TS a2fdbe7eef6748a9b4611f6c9bc9b201 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53846
I20260709 12:09:46.745887 12602 raft_consensus.cc:2955] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.738478 12429 catalog_manager.cc:5697] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:46.762053 13614 tablet_replica.cc:333] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:09:46.777618 13609 ts_tablet_manager.cc:1929] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:46.778350 13609 log.cc:1199] T 3c00651839ae4132bb331bc6c2602641 P 193a40da6e364ead8ab561e14c3cc8d1: Deleting WAL directory at /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-2-root/wals/3c00651839ae4132bb331bc6c2602641
I20260709 12:09:46.780264 12415 catalog_manager.cc:5028] TS 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): tablet 3c00651839ae4132bb331bc6c2602641 (table test-workload [id=8634097bac3246c9a57e272441058c28]) successfully deleted
I20260709 12:09:46.807078 12671 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:46.816190 13614 raft_consensus.cc:2243] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:46.817098 13614 raft_consensus.cc:2272] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:46.824779 13614 ts_tablet_manager.cc:1916] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:46.826215 13338 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:46.827790 13501 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:46.830886 12600 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:46.835954 12522 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:46.843370 13337 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:46.847323 13583 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:46.850528 13570 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:46.857770 13607 raft_consensus.cc:2955] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.860564 13511 raft_consensus.cc:2955] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.864182 13337 raft_consensus.cc:2955] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.869235 12671 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:46.871572 13608 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:46.877750 13338 raft_consensus.cc:2955] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.879742 12671 raft_consensus.cc:2955] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.908239 12429 catalog_manager.cc:5697] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 reported cstate change: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.913086 12416 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 618bbf28983a42beaaf337df12784ee1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:46.915196 12415 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 4270d5e0462b49d2982145525764c7fa with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:46.916209 12600 raft_consensus.cc:2955] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.933319 12590 tablet_service.cc:1558] Processing DeleteTablet for tablet 618bbf28983a42beaaf337df12784ee1 with delete_type TABLET_DATA_TOMBSTONED (TS f5fc807f7ecc4d1ca0fd6be524f734ce not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51774
I20260709 12:09:46.940250 12429 catalog_manager.cc:5697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 reported cstate change: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:46.941416 13615 tablet_replica.cc:333] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:09:46.968359 12519 tablet_service.cc:1558] Processing DeleteTablet for tablet 4270d5e0462b49d2982145525764c7fa with delete_type TABLET_DATA_TOMBSTONED (TS a2fdbe7eef6748a9b4611f6c9bc9b201 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53846
I20260709 12:09:46.990378 13615 raft_consensus.cc:2243] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:46.991356 13615 raft_consensus.cc:2272] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:46.996865 13615 ts_tablet_manager.cc:1916] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:47.014065 13614 ts_tablet_manager.cc:1929] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:47.014647 13614 log.cc:1199] T 6fca0ed05cd34c8e92dc950351b864a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Deleting WAL directory at /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-0-root/wals/6fca0ed05cd34c8e92dc950351b864a2
I20260709 12:09:47.019860 12416 catalog_manager.cc:5028] TS a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): tablet 6fca0ed05cd34c8e92dc950351b864a2 (table test-workload [id=8634097bac3246c9a57e272441058c28]) successfully deleted
I20260709 12:09:47.016692 13614 tablet_replica.cc:333] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:09:47.025619 13614 raft_consensus.cc:2243] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:47.026731 13614 raft_consensus.cc:2272] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:47.027822 13615 ts_tablet_manager.cc:1929] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:47.028929 13615 log.cc:1199] T 618bbf28983a42beaaf337df12784ee1 P f5fc807f7ecc4d1ca0fd6be524f734ce: Deleting WAL directory at /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-1-root/wals/618bbf28983a42beaaf337df12784ee1
I20260709 12:09:47.032063 12413 catalog_manager.cc:5028] TS f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): tablet 618bbf28983a42beaaf337df12784ee1 (table test-workload [id=8634097bac3246c9a57e272441058c28]) successfully deleted
I20260709 12:09:47.034453 13614 ts_tablet_manager.cc:1916] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:47.056000 13614 ts_tablet_manager.cc:1929] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:47.056581 13614 log.cc:1199] T 4270d5e0462b49d2982145525764c7fa P a2fdbe7eef6748a9b4611f6c9bc9b201: Deleting WAL directory at /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-0-root/wals/4270d5e0462b49d2982145525764c7fa
I20260709 12:09:47.058753 12416 catalog_manager.cc:5028] TS a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): tablet 4270d5e0462b49d2982145525764c7fa (table test-workload [id=8634097bac3246c9a57e272441058c28]) successfully deleted
W20260709 12:09:48.299690 12645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.006s	sys 0.007s
W20260709 12:09:48.300309 12645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.007s	sys 0.007s
I20260709 12:09:48.393102 12598 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:48.406900 12676 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:48.411307 12530 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:48.411646 13619 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:48.413872 13619 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:09:48.417531 12569 consensus_peers.cc:597] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dd783245c5154ded95c4fe8434d0b380. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:48.439024 13618 raft_consensus.cc:2955] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:48.441080 12676 raft_consensus.cc:2955] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:48.444000 12522 raft_consensus.cc:2955] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:48.459523 12527 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:48.476456 12676 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:09:48.476876 12494 consensus_peers.cc:597] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 43d4539f52e24ea8adc368a4c9529e66. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:48.476469 12598 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:48.473035 12429 catalog_manager.cc:5697] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:48.479107 13624 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:48.480113 13629 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:48.494686 13624 raft_consensus.cc:2955] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:48.498286 12598 raft_consensus.cc:2955] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:48.500581 12680 raft_consensus.cc:2955] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:48.532949 12527 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
W20260709 12:09:48.567070 12494 consensus_peers.cc:597] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cdb0bb14b2b745b398e14d8c1a7139a2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:48.567679 12676 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:09:48.571712 12598 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:09:48.572446 12428 catalog_manager.cc:5697] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:48.575886 13630 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:48.578045 13630 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:48.612149 13624 raft_consensus.cc:2955] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:48.615299 12680 raft_consensus.cc:2955] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:48.620638 12598 raft_consensus.cc:2955] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:48.650014 12676 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:48.651695 12428 catalog_manager.cc:5697] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index -1 to 3, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New cstate: current_term: 2 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:48.755529 12529 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260709 12:09:48.756599 12643 consensus_peers.cc:597] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5dd1b2c01b5e495f9f31aee68f93d394. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:48.757630 13511 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:48.760407 12603 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:48.771732 13511 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:48.828044 13511 raft_consensus.cc:2955] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:48.830907 12597 raft_consensus.cc:2955] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:48.839042 12529 raft_consensus.cc:2955] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:48.867143 12428 catalog_manager.cc:5697] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:48.977255 13649 ts_tablet_manager.cc:933] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:48.979007 13649 tablet_copy_client.cc:323] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.130:39365
I20260709 12:09:48.980979 12616 tablet_copy_service.cc:140] P f5fc807f7ecc4d1ca0fd6be524f734ce: Received BeginTabletCopySession request for tablet dd783245c5154ded95c4fe8434d0b380 from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:48492)
I20260709 12:09:48.981566 12616 tablet_copy_service.cc:161] P f5fc807f7ecc4d1ca0fd6be524f734ce: Beginning new tablet copy session on tablet dd783245c5154ded95c4fe8434d0b380 from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:48492: session id = e56d487365394ac5ac5f1002926a5645-dd783245c5154ded95c4fe8434d0b380
I20260709 12:09:48.983937 13651 ts_tablet_manager.cc:933] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:09:48.985941 13651 tablet_copy_client.cc:323] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.129:38811
I20260709 12:09:48.987824 12540 tablet_copy_service.cc:140] P a2fdbe7eef6748a9b4611f6c9bc9b201: Received BeginTabletCopySession request for tablet 43d4539f52e24ea8adc368a4c9529e66 from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:60222)
I20260709 12:09:48.988406 12540 tablet_copy_service.cc:161] P a2fdbe7eef6748a9b4611f6c9bc9b201: Beginning new tablet copy session on tablet 43d4539f52e24ea8adc368a4c9529e66 from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:60222: session id = e56d487365394ac5ac5f1002926a5645-43d4539f52e24ea8adc368a4c9529e66
I20260709 12:09:48.990468 12616 tablet_copy_source_session.cc:215] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:48.994449 13649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dd783245c5154ded95c4fe8434d0b380. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:48.996927 12540 tablet_copy_source_session.cc:215] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:48.999845 13651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 43d4539f52e24ea8adc368a4c9529e66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:49.012418 13649 tablet_copy_client.cc:806] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:49.013155 13649 tablet_copy_client.cc:670] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:49.017068 13651 tablet_copy_client.cc:806] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:49.018028 13651 tablet_copy_client.cc:670] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:49.019749 13649 tablet_copy_client.cc:538] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:49.022527 13651 tablet_copy_client.cc:538] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:49.030315 13649 tablet_bootstrap.cc:492] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
I20260709 12:09:49.033025 13651 tablet_bootstrap.cc:492] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
I20260709 12:09:49.056833 13649 tablet_bootstrap.cc:492] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:49.057904 13649 tablet_bootstrap.cc:492] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
I20260709 12:09:49.058704 13649 ts_tablet_manager.cc:1403] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 0.029s	user 0.023s	sys 0.008s
I20260709 12:09:49.061798 13649 raft_consensus.cc:359] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:49.062784 13649 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
I20260709 12:09:49.063578 13649 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:49.070221 13651 tablet_bootstrap.cc:492] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:49.071662 13651 tablet_bootstrap.cc:492] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
I20260709 12:09:49.072508 13651 ts_tablet_manager.cc:1403] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 0.040s	user 0.020s	sys 0.005s
I20260709 12:09:49.075846 13651 raft_consensus.cc:359] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:49.076813 13651 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
I20260709 12:09:49.077610 13651 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:49.081719 13651 ts_tablet_manager.cc:1434] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.009s	user 0.003s	sys 0.004s
I20260709 12:09:49.084612 12540 tablet_copy_service.cc:342] P a2fdbe7eef6748a9b4611f6c9bc9b201: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-43d4539f52e24ea8adc368a4c9529e66 received from {username='slave'} at 127.0.0.1:60222
I20260709 12:09:49.085181 12540 tablet_copy_service.cc:434] P a2fdbe7eef6748a9b4611f6c9bc9b201: ending tablet copy session e56d487365394ac5ac5f1002926a5645-43d4539f52e24ea8adc368a4c9529e66 on tablet 43d4539f52e24ea8adc368a4c9529e66 with peer e56d487365394ac5ac5f1002926a5645
I20260709 12:09:49.085700 13649 ts_tablet_manager.cc:1434] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.027s	user 0.005s	sys 0.009s
I20260709 12:09:49.087931 12616 tablet_copy_service.cc:342] P f5fc807f7ecc4d1ca0fd6be524f734ce: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-dd783245c5154ded95c4fe8434d0b380 received from {username='slave'} at 127.0.0.1:48492
I20260709 12:09:49.088459 12616 tablet_copy_service.cc:434] P f5fc807f7ecc4d1ca0fd6be524f734ce: ending tablet copy session e56d487365394ac5ac5f1002926a5645-dd783245c5154ded95c4fe8434d0b380 on tablet dd783245c5154ded95c4fe8434d0b380 with peer e56d487365394ac5ac5f1002926a5645
W20260709 12:09:49.111378 12627 debug-util.cc:398] Leaking SignalData structure 0x7b080013d600 after lost signal to thread 11321
W20260709 12:09:49.112402 12627 debug-util.cc:398] Leaking SignalData structure 0x7b080023bd00 after lost signal to thread 12463
W20260709 12:09:49.113842 12627 debug-util.cc:398] Leaking SignalData structure 0x7b080016cba0 after lost signal to thread 12555
W20260709 12:09:49.115232 12627 debug-util.cc:398] Leaking SignalData structure 0x7b08000cf2c0 after lost signal to thread 12630
W20260709 12:09:49.116500 12627 debug-util.cc:398] Leaking SignalData structure 0x7b08002267c0 after lost signal to thread 12704
W20260709 12:09:49.117563 12627 debug-util.cc:398] Leaking SignalData structure 0x7b08003377e0 after lost signal to thread 13363
I20260709 12:09:49.155856 13649 ts_tablet_manager.cc:933] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:09:49.160799 13649 tablet_copy_client.cc:323] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.129:38811
I20260709 12:09:49.162879 12540 tablet_copy_service.cc:140] P a2fdbe7eef6748a9b4611f6c9bc9b201: Received BeginTabletCopySession request for tablet cdb0bb14b2b745b398e14d8c1a7139a2 from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:60222)
I20260709 12:09:49.163276 13651 ts_tablet_manager.cc:933] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:49.163497 12540 tablet_copy_service.cc:161] P a2fdbe7eef6748a9b4611f6c9bc9b201: Beginning new tablet copy session on tablet cdb0bb14b2b745b398e14d8c1a7139a2 from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:60222: session id = e56d487365394ac5ac5f1002926a5645-cdb0bb14b2b745b398e14d8c1a7139a2
I20260709 12:09:49.196815 12540 tablet_copy_source_session.cc:215] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:49.201457 13649 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cdb0bb14b2b745b398e14d8c1a7139a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:49.245026 13651 tablet_copy_client.cc:323] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.131:38613
I20260709 12:09:49.247799 12690 tablet_copy_service.cc:140] P 193a40da6e364ead8ab561e14c3cc8d1: Received BeginTabletCopySession request for tablet 5dd1b2c01b5e495f9f31aee68f93d394 from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:40246)
I20260709 12:09:49.248636 12690 tablet_copy_service.cc:161] P 193a40da6e364ead8ab561e14c3cc8d1: Beginning new tablet copy session on tablet 5dd1b2c01b5e495f9f31aee68f93d394 from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:40246: session id = e56d487365394ac5ac5f1002926a5645-5dd1b2c01b5e495f9f31aee68f93d394
I20260709 12:09:49.261124 12690 tablet_copy_source_session.cc:215] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:49.264832 13651 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5dd1b2c01b5e495f9f31aee68f93d394. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:49.284894 13649 tablet_copy_client.cc:806] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:49.286679 13649 tablet_copy_client.cc:670] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:49.300683 13649 tablet_copy_client.cc:538] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:49.312211 13649 tablet_bootstrap.cc:492] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
I20260709 12:09:49.345628 13651 tablet_copy_client.cc:806] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:49.346411 13651 tablet_copy_client.cc:670] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:49.369355 13338 raft_consensus.cc:1217] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:49.374231 13651 tablet_copy_client.cc:538] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:49.426257 13651 tablet_bootstrap.cc:492] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
I20260709 12:09:49.429687 13649 tablet_bootstrap.cc:492] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:49.434870 13338 raft_consensus.cc:1217] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:49.445683 13649 tablet_bootstrap.cc:492] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
I20260709 12:09:49.448159 13649 ts_tablet_manager.cc:1403] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 0.136s	user 0.043s	sys 0.017s
I20260709 12:09:49.453495 13649 raft_consensus.cc:359] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:49.454576 13649 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
I20260709 12:09:49.455821 13649 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:49.461784 13649 ts_tablet_manager.cc:1434] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.000s
I20260709 12:09:49.463847 12540 tablet_copy_service.cc:342] P a2fdbe7eef6748a9b4611f6c9bc9b201: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-cdb0bb14b2b745b398e14d8c1a7139a2 received from {username='slave'} at 127.0.0.1:60222
I20260709 12:09:49.464377 12540 tablet_copy_service.cc:434] P a2fdbe7eef6748a9b4611f6c9bc9b201: ending tablet copy session e56d487365394ac5ac5f1002926a5645-cdb0bb14b2b745b398e14d8c1a7139a2 on tablet cdb0bb14b2b745b398e14d8c1a7139a2 with peer e56d487365394ac5ac5f1002926a5645
I20260709 12:09:49.488909 13651 tablet_bootstrap.cc:492] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:49.490428 13651 tablet_bootstrap.cc:492] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
I20260709 12:09:49.491645 13651 ts_tablet_manager.cc:1403] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 0.066s	user 0.027s	sys 0.013s
I20260709 12:09:49.495214 13651 raft_consensus.cc:359] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:49.496385 13651 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
I20260709 12:09:49.497135 13651 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:49.502979 13651 ts_tablet_manager.cc:1434] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:09:49.505081 12690 tablet_copy_service.cc:342] P 193a40da6e364ead8ab561e14c3cc8d1: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-5dd1b2c01b5e495f9f31aee68f93d394 received from {username='slave'} at 127.0.0.1:40246
I20260709 12:09:49.505596 12690 tablet_copy_service.cc:434] P 193a40da6e364ead8ab561e14c3cc8d1: ending tablet copy session e56d487365394ac5ac5f1002926a5645-5dd1b2c01b5e495f9f31aee68f93d394 on tablet 5dd1b2c01b5e495f9f31aee68f93d394 with peer e56d487365394ac5ac5f1002926a5645
I20260709 12:09:49.705547 13338 raft_consensus.cc:1217] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3]   Dedup: 2.3->[]
I20260709 12:09:49.722100 13338 raft_consensus.cc:1217] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:09:50.065562 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 358377us
I20260709 12:09:50.092818 13618 raft_consensus.cc:1064] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: attempting to promote NON_VOTER e56d487365394ac5ac5f1002926a5645 to VOTER
I20260709 12:09:50.098176 13618 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:09:50.110546 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 72896.5us
I20260709 12:09:50.118458 13669 raft_consensus.cc:1064] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: attempting to promote NON_VOTER e56d487365394ac5ac5f1002926a5645 to VOTER
I20260709 12:09:50.155141 13669 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:09:50.182430 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 74114us
I20260709 12:09:50.203503 13337 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:50.207085 12674 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:50.209431 12598 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:50.212061 13337 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:50.212742 13674 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:50.219570 13511 raft_consensus.cc:1064] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: attempting to promote NON_VOTER e56d487365394ac5ac5f1002926a5645 to VOTER
I20260709 12:09:50.222674 12527 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:50.222345 13664 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.023s
I20260709 12:09:50.246964 13668 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:50.267791 12671 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:50.267953 13681 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
W20260709 12:09:50.273175 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 57205.4us
I20260709 12:09:50.274380 13511 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:50.279129 13663 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:50.281781 13624 raft_consensus.cc:1064] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: attempting to promote NON_VOTER e56d487365394ac5ac5f1002926a5645 to VOTER
I20260709 12:09:50.284641 13624 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:09:50.303179 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 49162.7us
I20260709 12:09:50.311175 13666 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:50.339798 13681 raft_consensus.cc:2955] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:50.341400 12598 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:09:50.343103 13630 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:50.343403 12674 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:09:50.349082 13624 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:50.353554 12598 raft_consensus.cc:2955] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:50.370505 13338 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 2 LEARNER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260709 12:09:50.390827 13674 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:50.386437 12428 catalog_manager.cc:5697] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:50.394994 13338 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:50.432328 12597 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:09:50.433961 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 45317us
I20260709 12:09:50.436216 12527 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:09:50.438227 13689 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:09:50.446429 13678 raft_consensus.cc:2955] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:09:50.449471 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 53048.2us
I20260709 12:09:50.454375 13679 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:50.468768 13632 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:50.472999 13511 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:50.474861 12676 raft_consensus.cc:2955] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:50.457980 12527 raft_consensus.cc:2955] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:50.484972 12680 raft_consensus.cc:2955] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:50.505913 13337 raft_consensus.cc:2955] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:09:50.590660 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 59594.4us
I20260709 12:09:50.753562 13334 raft_consensus.cc:2955] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:50.908995 13674 raft_consensus.cc:2955] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:09:50.931181 12701 debug-util.cc:398] Leaking SignalData structure 0x7b08001394a0 after lost signal to thread 11321
W20260709 12:09:50.932418 12701 debug-util.cc:398] Leaking SignalData structure 0x7b080017aea0 after lost signal to thread 12463
W20260709 12:09:50.933763 12701 debug-util.cc:398] Leaking SignalData structure 0x7b080033e140 after lost signal to thread 12555
W20260709 12:09:50.936471 12701 debug-util.cc:398] Leaking SignalData structure 0x7b080033e940 after lost signal to thread 12630
W20260709 12:09:50.940121 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 114682us
W20260709 12:09:50.959738 13693 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 12:09:50.994622 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 196348us
I20260709 12:09:50.991516 12674 raft_consensus.cc:2955] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:09:50.978935 12701 debug-util.cc:398] Leaking SignalData structure 0x7b08001dee40 after lost signal to thread 12704
I20260709 12:09:51.047778 13336 raft_consensus.cc:2955] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:51.052582 13339 raft_consensus.cc:2955] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:09:51.056183 12701 debug-util.cc:398] Leaking SignalData structure 0x7b0800133de0 after lost signal to thread 13363
I20260709 12:09:51.021147 13679 raft_consensus.cc:2955] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:51.070801 12600 raft_consensus.cc:2955] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:09:51.129359 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 51768us
W20260709 12:09:51.138810 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 59018us
I20260709 12:09:51.165060 12524 raft_consensus.cc:2955] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:51.199218 12600 raft_consensus.cc:2955] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:51.310986 12429 catalog_manager.cc:5697] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:51.317817 12429 catalog_manager.cc:5697] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: config changed from index 3 to 4, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:51.329756 12429 catalog_manager.cc:5697] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:51.370366 12673 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:51.400060 12597 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:51.411753 13336 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:51.416854 13672 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:51.417918 12600 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:51.419741 13511 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:51.420776 13336 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:51.423180 13618 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:51.429520 12671 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:51.429644 13671 raft_consensus.cc:2955] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:51.431218 13336 raft_consensus.cc:2955] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:51.436677 13702 raft_consensus.cc:2955] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:51.438859 13339 raft_consensus.cc:2955] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:51.452316 12600 raft_consensus.cc:2955] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:51.458956 13700 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:51.469743 12671 raft_consensus.cc:2955] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:51.474370 12524 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:51.477838 12530 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:51.503019 12426 catalog_manager.cc:5697] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New cstate: current_term: 1 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:51.517132 13336 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:09:51.517271 12600 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:09:51.527457 13661 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:09:51.529589 13661 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260709 12:09:51.544456 12429 catalog_manager.cc:5697] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:51.548661 12519 tablet_service.cc:1558] Processing DeleteTablet for tablet 5dd1b2c01b5e495f9f31aee68f93d394 with delete_type TABLET_DATA_TOMBSTONED (TS a2fdbe7eef6748a9b4611f6c9bc9b201 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53846
W20260709 12:09:51.625248 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 45791us
I20260709 12:09:51.630923 12597 raft_consensus.cc:2955] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:51.619863 13709 tablet_replica.cc:333] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:09:51.676307 13709 raft_consensus.cc:2243] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:51.677139 13709 raft_consensus.cc:2272] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:51.694815 12519 tablet_service.cc:1558] Processing DeleteTablet for tablet dd783245c5154ded95c4fe8434d0b380 with delete_type TABLET_DATA_TOMBSTONED (TS a2fdbe7eef6748a9b4611f6c9bc9b201 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53846
I20260709 12:09:51.695554 12415 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 5dd1b2c01b5e495f9f31aee68f93d394 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:51.710466 13709 ts_tablet_manager.cc:1916] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:51.713699 12413 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet dd783245c5154ded95c4fe8434d0b380 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:51.747408 12426 catalog_manager.cc:5697] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: config changed from index 4 to 5, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New cstate: current_term: 2 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:51.778384 13334 raft_consensus.cc:2955] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:51.758744 13701 raft_consensus.cc:2955] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:51.791492 13333 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:51.814435 12679 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:51.820125 13704 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:51.833506 12662 tablet_service.cc:1558] Processing DeleteTablet for tablet cdb0bb14b2b745b398e14d8c1a7139a2 with delete_type TABLET_DATA_TOMBSTONED (TS 193a40da6e364ead8ab561e14c3cc8d1 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:51290
I20260709 12:09:51.836292 13709 ts_tablet_manager.cc:1929] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:51.836908 13709 log.cc:1199] T 5dd1b2c01b5e495f9f31aee68f93d394 P a2fdbe7eef6748a9b4611f6c9bc9b201: Deleting WAL directory at /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-0-root/wals/5dd1b2c01b5e495f9f31aee68f93d394
I20260709 12:09:51.847033 12416 catalog_manager.cc:5028] TS a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): tablet 5dd1b2c01b5e495f9f31aee68f93d394 (table test-workload [id=8634097bac3246c9a57e272441058c28]) successfully deleted
I20260709 12:09:51.845997 13709 tablet_replica.cc:333] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:09:51.847805 12416 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet cdb0bb14b2b745b398e14d8c1a7139a2 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:51.856220 13701 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260709 12:09:51.883350 13709 raft_consensus.cc:2243] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:51.886205 13709 raft_consensus.cc:2272] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:51.890436 13719 tablet_replica.cc:333] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:09:51.895725 13719 raft_consensus.cc:2243] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:51.896924 13719 raft_consensus.cc:2272] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:51.908516 13719 ts_tablet_manager.cc:1916] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:51.914536 13709 ts_tablet_manager.cc:1916] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:51.914736 13701 raft_consensus.cc:2955] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:51.920181 13334 raft_consensus.cc:2955] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:51.959635 12416 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 43d4539f52e24ea8adc368a4c9529e66 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:51.968065 12428 catalog_manager.cc:5697] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:51.977542 13719 ts_tablet_manager.cc:1929] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260709 12:09:51.978108 13719 log.cc:1199] T cdb0bb14b2b745b398e14d8c1a7139a2 P 193a40da6e364ead8ab561e14c3cc8d1: Deleting WAL directory at /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-2-root/wals/cdb0bb14b2b745b398e14d8c1a7139a2
I20260709 12:09:51.980751 12590 tablet_service.cc:1558] Processing DeleteTablet for tablet 43d4539f52e24ea8adc368a4c9529e66 with delete_type TABLET_DATA_TOMBSTONED (TS f5fc807f7ecc4d1ca0fd6be524f734ce not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51774
I20260709 12:09:51.989497 12415 catalog_manager.cc:5028] TS 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): tablet cdb0bb14b2b745b398e14d8c1a7139a2 (table test-workload [id=8634097bac3246c9a57e272441058c28]) successfully deleted
I20260709 12:09:51.990465 13709 ts_tablet_manager.cc:1929] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:51.999054 13709 log.cc:1199] T dd783245c5154ded95c4fe8434d0b380 P a2fdbe7eef6748a9b4611f6c9bc9b201: Deleting WAL directory at /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-0-root/wals/dd783245c5154ded95c4fe8434d0b380
I20260709 12:09:52.001063 12416 catalog_manager.cc:5028] TS a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): tablet dd783245c5154ded95c4fe8434d0b380 (table test-workload [id=8634097bac3246c9a57e272441058c28]) successfully deleted
I20260709 12:09:52.009173 13722 tablet_replica.cc:333] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:09:52.026243 12679 raft_consensus.cc:2955] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:52.057389 13722 raft_consensus.cc:2243] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:52.059839 13722 raft_consensus.cc:2272] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:09:52.067602 13722 ts_tablet_manager.cc:1916] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:52.147305 13722 ts_tablet_manager.cc:1929] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:09:52.147857 13722 log.cc:1199] T 43d4539f52e24ea8adc368a4c9529e66 P f5fc807f7ecc4d1ca0fd6be524f734ce: Deleting WAL directory at /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-1-root/wals/43d4539f52e24ea8adc368a4c9529e66
I20260709 12:09:52.151731 12413 catalog_manager.cc:5028] TS f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): tablet 43d4539f52e24ea8adc368a4c9529e66 (table test-workload [id=8634097bac3246c9a57e272441058c28]) successfully deleted
W20260709 12:09:52.603149 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 145895us
W20260709 12:09:52.721383 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 58349.1us
W20260709 12:09:52.739303 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 59258.6us
W20260709 12:09:52.743747 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 52925.5us
I20260709 12:09:53.281395 12530 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
W20260709 12:09:53.307744 12494 consensus_peers.cc:597] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 952cdab9198b44e4bdf1b472109715c3. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:53.308068 12602 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:53.309825 13674 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:53.310477 12673 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:53.313427 13724 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:53.357276 13704 raft_consensus.cc:2955] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:53.363713 12673 raft_consensus.cc:2955] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:53.379534 12602 raft_consensus.cc:2955] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:53.397001 12530 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:53.398288 12428 catalog_manager.cc:5697] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
W20260709 12:09:53.419170 12494 consensus_peers.cc:597] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: ee47f7c4414b4ae98233a31696e99cc0. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:53.423264 12673 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:53.424588 12597 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:53.425220 13724 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:53.426909 13726 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:53.454412 13724 raft_consensus.cc:2955] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:53.462395 12673 raft_consensus.cc:2955] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:53.467849 12602 raft_consensus.cc:2955] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:53.483808 12530 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:53.497228 12428 catalog_manager.cc:5697] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:53.520197 12673 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:53.521829 12597 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:53.522977 13704 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:09:53.526335 12494 consensus_peers.cc:597] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8c7e0eddcc2b4449b7ed8821854646e2. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:53.527845 13704 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:53.552212 13729 raft_consensus.cc:2955] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:53.559460 12674 raft_consensus.cc:2955] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:53.586140 12597 raft_consensus.cc:2955] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:53.604076 12428 catalog_manager.cc:5697] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:53.632524 12597 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:53.650393 12674 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:53.652416 13714 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:09:53.670531 12569 consensus_peers.cc:597] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 816b6f34d124434882148e507cf5b612. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:09:53.670802 12530 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:09:53.673827 13723 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:53.676177 13714 raft_consensus.cc:2955] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:53.678179 12673 raft_consensus.cc:2955] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:53.707582 12428 catalog_manager.cc:5697] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:53.743878 12530 raft_consensus.cc:2955] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:09:53.841455 13754 ts_tablet_manager.cc:933] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:09:53.850363 13754 tablet_copy_client.cc:323] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.129:38811
I20260709 12:09:53.852993 12540 tablet_copy_service.cc:140] P a2fdbe7eef6748a9b4611f6c9bc9b201: Received BeginTabletCopySession request for tablet ee47f7c4414b4ae98233a31696e99cc0 from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:60222)
I20260709 12:09:53.853688 12540 tablet_copy_service.cc:161] P a2fdbe7eef6748a9b4611f6c9bc9b201: Beginning new tablet copy session on tablet ee47f7c4414b4ae98233a31696e99cc0 from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:60222: session id = e56d487365394ac5ac5f1002926a5645-ee47f7c4414b4ae98233a31696e99cc0
I20260709 12:09:53.863587 12540 tablet_copy_source_session.cc:215] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:53.867698 13754 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ee47f7c4414b4ae98233a31696e99cc0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:53.898970 13754 tablet_copy_client.cc:806] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:53.899753 13754 tablet_copy_client.cc:670] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:53.931034 13754 tablet_copy_client.cc:538] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:53.995141 13756 ts_tablet_manager.cc:933] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:09:54.005249 13754 tablet_bootstrap.cc:492] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
W20260709 12:09:54.040411 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 70598.2us
W20260709 12:09:54.042827 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52790.5us
I20260709 12:09:54.064584 13756 tablet_copy_client.cc:323] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.129:38811
I20260709 12:09:54.066929 12540 tablet_copy_service.cc:140] P a2fdbe7eef6748a9b4611f6c9bc9b201: Received BeginTabletCopySession request for tablet 952cdab9198b44e4bdf1b472109715c3 from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:60222)
I20260709 12:09:54.067556 12540 tablet_copy_service.cc:161] P a2fdbe7eef6748a9b4611f6c9bc9b201: Beginning new tablet copy session on tablet 952cdab9198b44e4bdf1b472109715c3 from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:60222: session id = e56d487365394ac5ac5f1002926a5645-952cdab9198b44e4bdf1b472109715c3
I20260709 12:09:54.077338 12540 tablet_copy_source_session.cc:215] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:54.082418 13756 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 952cdab9198b44e4bdf1b472109715c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:54.104429 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:54.123147 13754 tablet_bootstrap.cc:492] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:54.130525 13754 tablet_bootstrap.cc:492] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
I20260709 12:09:54.143270 13754 ts_tablet_manager.cc:1403] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 0.138s	user 0.020s	sys 0.014s
I20260709 12:09:54.153404 13754 raft_consensus.cc:359] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:54.155105 13754 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
I20260709 12:09:54.164484 13754 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:54.168190 13765 ts_tablet_manager.cc:933] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:09:54.186309 13766 ts_tablet_manager.cc:933] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:54.196277 13756 tablet_copy_client.cc:806] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:54.204331 13766 tablet_copy_client.cc:323] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.130:39365
I20260709 12:09:54.201344 13756 tablet_copy_client.cc:670] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:54.206715 13754 ts_tablet_manager.cc:1434] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.063s	user 0.009s	sys 0.008s
I20260709 12:09:54.209793 12540 tablet_copy_service.cc:342] P a2fdbe7eef6748a9b4611f6c9bc9b201: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-ee47f7c4414b4ae98233a31696e99cc0 received from {username='slave'} at 127.0.0.1:60222
I20260709 12:09:54.210577 12540 tablet_copy_service.cc:434] P a2fdbe7eef6748a9b4611f6c9bc9b201: ending tablet copy session e56d487365394ac5ac5f1002926a5645-ee47f7c4414b4ae98233a31696e99cc0 on tablet ee47f7c4414b4ae98233a31696e99cc0 with peer e56d487365394ac5ac5f1002926a5645
I20260709 12:09:54.218609 12616 tablet_copy_service.cc:140] P f5fc807f7ecc4d1ca0fd6be524f734ce: Received BeginTabletCopySession request for tablet 816b6f34d124434882148e507cf5b612 from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:48492)
I20260709 12:09:54.219738 12616 tablet_copy_service.cc:161] P f5fc807f7ecc4d1ca0fd6be524f734ce: Beginning new tablet copy session on tablet 816b6f34d124434882148e507cf5b612 from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:48492: session id = e56d487365394ac5ac5f1002926a5645-816b6f34d124434882148e507cf5b612
I20260709 12:09:54.239230 13756 tablet_copy_client.cc:538] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:54.264159 13765 tablet_copy_client.cc:323] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.129:38811
I20260709 12:09:54.266775 12541 tablet_copy_service.cc:140] P a2fdbe7eef6748a9b4611f6c9bc9b201: Received BeginTabletCopySession request for tablet 8c7e0eddcc2b4449b7ed8821854646e2 from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:60222)
I20260709 12:09:54.267503 12541 tablet_copy_service.cc:161] P a2fdbe7eef6748a9b4611f6c9bc9b201: Beginning new tablet copy session on tablet 8c7e0eddcc2b4449b7ed8821854646e2 from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:60222: session id = e56d487365394ac5ac5f1002926a5645-8c7e0eddcc2b4449b7ed8821854646e2
I20260709 12:09:54.276091 13332 raft_consensus.cc:1217] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:54.259570 12616 tablet_copy_source_session.cc:215] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:54.288460 12541 tablet_copy_source_session.cc:215] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Tablet Copy: opened 0 blocks and 1 log segments
I20260709 12:09:54.294957 13766 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 816b6f34d124434882148e507cf5b612. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:54.321730 13765 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8c7e0eddcc2b4449b7ed8821854646e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:09:54.338644 13756 tablet_bootstrap.cc:492] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
I20260709 12:09:54.366727 13765 tablet_copy_client.cc:806] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:54.367409 13765 tablet_copy_client.cc:670] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:54.371783 13766 tablet_copy_client.cc:806] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:09:54.372539 13766 tablet_copy_client.cc:670] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
I20260709 12:09:54.375656 13765 tablet_copy_client.cc:538] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:54.376958 13766 tablet_copy_client.cc:538] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:09:54.407068 13765 tablet_bootstrap.cc:492] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
I20260709 12:09:54.413208 13766 tablet_bootstrap.cc:492] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
I20260709 12:09:54.426620 13756 tablet_bootstrap.cc:492] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:54.427894 13756 tablet_bootstrap.cc:492] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
I20260709 12:09:54.428840 13756 ts_tablet_manager.cc:1403] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 0.090s	user 0.024s	sys 0.003s
I20260709 12:09:54.508913 13766 tablet_bootstrap.cc:492] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:54.510352 13766 tablet_bootstrap.cc:492] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
I20260709 12:09:54.511129 13766 ts_tablet_manager.cc:1403] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 0.098s	user 0.029s	sys 0.012s
I20260709 12:09:54.498378 13756 raft_consensus.cc:359] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:54.531590 13756 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
I20260709 12:09:54.532464 13756 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:54.535888 13756 ts_tablet_manager.cc:1434] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.107s	user 0.006s	sys 0.002s
I20260709 12:09:54.617183 13766 raft_consensus.cc:359] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:54.675565 13766 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
W20260709 12:09:54.815124 13765 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:09:54.918192 13765 tablet_bootstrap.cc:492] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:09:54.760317 13766 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:54.923979 13766 ts_tablet_manager.cc:1434] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.412s	user 0.012s	sys 0.001s
W20260709 12:09:54.935148 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 178098us
I20260709 12:09:55.137920 13765 tablet_bootstrap.cc:492] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
I20260709 12:09:55.138718 13765 ts_tablet_manager.cc:1403] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 0.732s	user 0.047s	sys 0.033s
I20260709 12:09:55.141381 13765 raft_consensus.cc:359] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:55.142391 13765 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
I20260709 12:09:55.143518 13765 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:09:55.150641 13765 ts_tablet_manager.cc:1434] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.001s
I20260709 12:09:55.200170 12616 tablet_copy_service.cc:342] P f5fc807f7ecc4d1ca0fd6be524f734ce: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-816b6f34d124434882148e507cf5b612 received from {username='slave'} at 127.0.0.1:48492
I20260709 12:09:55.200796 12616 tablet_copy_service.cc:434] P f5fc807f7ecc4d1ca0fd6be524f734ce: ending tablet copy session e56d487365394ac5ac5f1002926a5645-816b6f34d124434882148e507cf5b612 on tablet 816b6f34d124434882148e507cf5b612 with peer e56d487365394ac5ac5f1002926a5645
I20260709 12:09:55.206616 13336 raft_consensus.cc:1217] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
W20260709 12:09:55.209460 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 462037us
I20260709 12:09:55.216210 12541 tablet_copy_service.cc:342] P a2fdbe7eef6748a9b4611f6c9bc9b201: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-952cdab9198b44e4bdf1b472109715c3 received from {username='slave'} at 127.0.0.1:60222
I20260709 12:09:55.216758 12541 tablet_copy_service.cc:434] P a2fdbe7eef6748a9b4611f6c9bc9b201: ending tablet copy session e56d487365394ac5ac5f1002926a5645-952cdab9198b44e4bdf1b472109715c3 on tablet 952cdab9198b44e4bdf1b472109715c3 with peer e56d487365394ac5ac5f1002926a5645
W20260709 12:09:55.256623 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 47915.6us
I20260709 12:09:55.267267 13332 raft_consensus.cc:1217] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:55.279819 13788 raft_consensus.cc:1064] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: attempting to promote NON_VOTER e56d487365394ac5ac5f1002926a5645 to VOTER
W20260709 12:09:55.282564 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 71858.9us
I20260709 12:09:55.285737 13788 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:55.300271 12541 tablet_copy_service.cc:342] P a2fdbe7eef6748a9b4611f6c9bc9b201: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-8c7e0eddcc2b4449b7ed8821854646e2 received from {username='slave'} at 127.0.0.1:60222
I20260709 12:09:55.300776 12541 tablet_copy_service.cc:434] P a2fdbe7eef6748a9b4611f6c9bc9b201: ending tablet copy session e56d487365394ac5ac5f1002926a5645-8c7e0eddcc2b4449b7ed8821854646e2 on tablet 8c7e0eddcc2b4449b7ed8821854646e2 with peer e56d487365394ac5ac5f1002926a5645
I20260709 12:09:55.302318 13332 raft_consensus.cc:1217] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2]   Dedup: 1.2->[]
I20260709 12:09:55.320101 12671 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:09:55.320886 12671 raft_consensus.cc:606] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:55.322005 13332 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:55.323932 13788 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:55.338550 12673 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:55.340896 13704 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:55.342839 12673 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:09:55.343641 12673 raft_consensus.cc:606] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:55.354553 13787 raft_consensus.cc:1064] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201: attempting to promote NON_VOTER e56d487365394ac5ac5f1002926a5645 to VOTER
I20260709 12:09:55.357348 12672 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "e56d487365394ac5ac5f1002926a5645"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:09:55.358237 12672 raft_consensus.cc:606] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Received request to transfer leadership to TS e56d487365394ac5ac5f1002926a5645
I20260709 12:09:55.358773 12598 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:55.366876 12672 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:09:55.368089 12672 raft_consensus.cc:606] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:55.374804 13758 raft_consensus.cc:1064] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201: attempting to promote NON_VOTER e56d487365394ac5ac5f1002926a5645 to VOTER
I20260709 12:09:55.377641 13758 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:55.387979 13788 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:55.397135 13787 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:55.464249 12674 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:55.484202 12605 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:09:55.486497 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 70369.6us
I20260709 12:09:55.489208 13780 raft_consensus.cc:993] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: : Instructing follower f5fc807f7ecc4d1ca0fd6be524f734ce to start an election
I20260709 12:09:55.489619 12674 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:55.489768 13780 raft_consensus.cc:1081] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Signalling peer f5fc807f7ecc4d1ca0fd6be524f734ce to start an election
I20260709 12:09:55.473778 13777 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260709 12:09:55.495628 13332 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:55.497596 13332 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:09:55.527112 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50744.1us
I20260709 12:09:55.540095 13791 raft_consensus.cc:993] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: : Instructing follower f5fc807f7ecc4d1ca0fd6be524f734ce to start an election
I20260709 12:09:55.540616 13791 raft_consensus.cc:1081] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Signalling peer f5fc807f7ecc4d1ca0fd6be524f734ce to start an election
I20260709 12:09:55.544770 13778 raft_consensus.cc:993] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: : Instructing follower f5fc807f7ecc4d1ca0fd6be524f734ce to start an election
I20260709 12:09:55.544735 13761 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:55.545606 13778 raft_consensus.cc:1081] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Signalling peer f5fc807f7ecc4d1ca0fd6be524f734ce to start an election
I20260709 12:09:55.548084 12605 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:32832
I20260709 12:09:55.548671 12605 raft_consensus.cc:606] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:09:55.551995 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 50885us
I20260709 12:09:55.554632 12597 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:51792
I20260709 12:09:55.554045 12605 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:51792
I20260709 12:09:55.555243 12602 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:51792
I20260709 12:09:55.555326 12597 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:55.556819 12597 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:55.555961 12602 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:55.559302 12602 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:55.555660 12605 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:55.561604 12605 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:09:55.566730 13799 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:09:55.570324 13761 raft_consensus.cc:2955] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:55.580664 12602 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:55.581863 13784 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:55.592082 12600 raft_consensus.cc:2955] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:55.598608 13784 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:55.598807 13758 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:09:55.583560 12597 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:55.584218 12598 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:55.613873 13336 raft_consensus.cc:2955] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:55.619824 12671 raft_consensus.cc:2955] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:55.572960 12605 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:55.632043 12605 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:55.636065 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:55.637020 12672 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:09:55.637454 12672 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:55.643584 12597 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:55.645937 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:55.646620 12673 raft_consensus.cc:3055] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:09:55.646979 12673 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:55.647966 12673 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:55.649715 12673 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:55.657847 12672 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:55.659456 12672 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:55.663136 12427 catalog_manager.cc:5697] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:55.665223 12602 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:55.670153 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:55.670769 12671 raft_consensus.cc:3055] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:09:55.671068 12671 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:55.671833 12671 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:55.673055 12671 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:55.691272 13777 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:09:55.698557 12671 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:09:55.700314 13801 raft_consensus.cc:2955] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:55.712234 12571 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:55.713291 13714 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:09:55.714468 13714 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:09:55.715538 13714 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:55.721470 12672 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:09:55.722905 12571 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:55.724036 13781 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:09:55.724607 13781 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:09:55.725787 13781 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:55.734827 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:55.735579 12522 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:55.736091 12671 raft_consensus.cc:2955] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:55.741024 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:55.741267 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:55.747862 12522 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:09:55.748009 12530 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:09:55.756556 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 58150.5us
I20260709 12:09:55.747483 12531 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:55.759953 12673 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:09:55.767153 13678 raft_consensus.cc:1064] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce: attempting to promote NON_VOTER e56d487365394ac5ac5f1002926a5645 to VOTER
I20260709 12:09:55.807411 12571 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:55.808586 13733 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:09:55.809028 13678 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:55.769104 12530 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:09:55.766005 12531 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:09:55.785614 12522 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:55.809067 13733 raft_consensus.cc:697] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:09:55.791010 12605 raft_consensus.cc:2955] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:55.811655 13733 consensus_queue.cc:237] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:55.828029 12429 catalog_manager.cc:5697] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:55.863421 13774 raft_consensus.cc:993] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: : Instructing follower 193a40da6e364ead8ab561e14c3cc8d1 to start an election
W20260709 12:09:55.871171 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 48198.6us
I20260709 12:09:55.853273 13788 raft_consensus.cc:2955] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:55.884579 12522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7bd864b75c34027b05582a55b152877"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "e56d487365394ac5ac5f1002926a5645"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:09:55.885234 12522 raft_consensus.cc:606] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Received request to transfer leadership to TS e56d487365394ac5ac5f1002926a5645
I20260709 12:09:55.886992 13774 raft_consensus.cc:1081] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Signalling peer 193a40da6e364ead8ab561e14c3cc8d1 to start an election
I20260709 12:09:55.889812 12671 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:51304
I20260709 12:09:55.890431 12671 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:55.890774 12671 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:55.898289 12673 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "e56d487365394ac5ac5f1002926a5645"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:09:55.899132 12671 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:55.898962 12673 raft_consensus.cc:606] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Received request to transfer leadership to TS e56d487365394ac5ac5f1002926a5645
I20260709 12:09:55.901865 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 12:09:55.903716 13780 raft_consensus.cc:993] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: : Instructing follower e56d487365394ac5ac5f1002926a5645 to start an election
I20260709 12:09:55.904065 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:09:55.904225 13780 raft_consensus.cc:1081] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Signalling peer e56d487365394ac5ac5f1002926a5645 to start an election
I20260709 12:09:55.904729 12671 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:55.905462 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:09:55.912633 13339 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa"
dest_uuid: "e56d487365394ac5ac5f1002926a5645"
 from {username='slave'} at 127.0.0.1:38482
I20260709 12:09:55.913440 13339 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:55.913851 13339 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:55.922195 12428 catalog_manager.cc:5697] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 1 to 2, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 2 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:55.933204 12428 catalog_manager.cc:5697] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 1 to 2, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 2 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:55.935875 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:55.936525 12522 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:09:55.935401 12552 debug-util.cc:398] Leaking SignalData structure 0x7b0800171780 after lost signal to thread 11321
W20260709 12:09:55.944756 12552 debug-util.cc:398] Leaking SignalData structure 0x7b08000d5220 after lost signal to thread 12463
I20260709 12:09:55.951848 12522 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:09:55.958536 12605 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:55.957996 13333 raft_consensus.cc:2955] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:09:55.946053 12552 debug-util.cc:398] Leaking SignalData structure 0x7b0800345640 after lost signal to thread 12555
I20260709 12:09:55.978220 13339 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:55.950367 13334 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:55.962666 12646 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:09:55.985000 12671 raft_consensus.cc:2955] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:55.987856 12674 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260709 12:09:55.991676 12552 debug-util.cc:398] Leaking SignalData structure 0x7b080013aea0 after lost signal to thread 12630
W20260709 12:09:55.986649 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 97886.2us
I20260709 12:09:55.997388 13339 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:55.999693 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:55.999927 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:56.000465 12672 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:09:56.000645 12605 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:56.000881 12672 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:56.006052 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:56.006810 12597 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:09:56.006238 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 43757.7us
I20260709 12:09:56.007139 12597 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:09:56.008621 12672 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:56.009524 12522 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:56.009953 12672 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:56.001633 13335 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260709 12:09:56.010954 12597 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:56.023751 12597 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:56.016394 13733 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:09:56.032716 12605 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
I20260709 12:09:56.016170 13780 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:09:56.043483 13302 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:56.048959 13767 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:09:56.046803 13678 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260709 12:09:56.049852 13780 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:56.055979 13801 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:56.057567 13780 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:56.052043 12552 debug-util.cc:398] Leaking SignalData structure 0x7b08002fc5a0 after lost signal to thread 12704
I20260709 12:09:56.080690 12597 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
W20260709 12:09:56.085055 12552 debug-util.cc:398] Leaking SignalData structure 0x7b08002bb560 after lost signal to thread 13363
I20260709 12:09:56.088640 12429 catalog_manager.cc:5697] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:56.092120 13787 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:09:56.096045 13339 raft_consensus.cc:2955] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:56.100818 12602 raft_consensus.cc:2955] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:09:56.114233 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 98862.6us
I20260709 12:09:56.111580 13767 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 2 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:09:56.124402 13781 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 12:09:56.161435 12671 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:56.158538 12428 catalog_manager.cc:5697] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 1 to 2, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 2 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:56.154430 12672 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
W20260709 12:09:56.206364 13823 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:09:56.188071 13767 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:56.222831 13795 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:09:56.242486 12429 catalog_manager.cc:5697] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 1 to 2, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 2 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
W20260709 12:09:56.245782 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 82770.3us
I20260709 12:09:56.265558 12522 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:09:56.287482 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 129210us
I20260709 12:09:56.308351 13830 raft_consensus.cc:2955] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:56.269677 12530 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:56.297220 12531 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:56.302886 13795 raft_consensus.cc:2955] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:56.314428 13829 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:09:56.319042 12674 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:56.320736 12672 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:56.321576 12530 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:56.323107 13761 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:56.324926 13761 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:09:56.330762 13678 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:09:56.342490 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49859us
I20260709 12:09:56.347771 12530 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:56.351218 13774 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:09:56.352213 12425 catalog_manager.cc:5697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 1 to 2, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 2 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:56.355904 13801 raft_consensus.cc:2955] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:56.356688 13816 raft_consensus.cc:2955] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:09:56.361120 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:09:56.369829 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 43718.6us
I20260709 12:09:56.366561 12674 raft_consensus.cc:2955] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:56.378566 12598 raft_consensus.cc:2955] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:09:56.395780 13831 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.107s	user 0.000s	sys 0.001s
W20260709 12:09:56.476256 13810 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:09:56.482110 12416 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet ee47f7c4414b4ae98233a31696e99cc0 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:09:56.482877 12416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 72129.5us
I20260709 12:09:56.483423 12676 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:56.484411 12680 raft_consensus.cc:2955] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:56.490209 12416 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 952cdab9198b44e4bdf1b472109715c3 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:56.495847 13335 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:56.397589 12672 raft_consensus.cc:2955] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:56.520140 13333 raft_consensus.cc:2955] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:56.523437 13334 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:56.545832 12674 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:09:56.579334 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 200487us
I20260709 12:09:56.596788 12605 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:56.577441 13795 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.046s
I20260709 12:09:56.697333 12524 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:09:56.709920 13791 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:09:56.730496 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 338721us
I20260709 12:09:56.742779 12416 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 8c7e0eddcc2b4449b7ed8821854646e2 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260709 12:09:56.748921 13334 raft_consensus.cc:2955] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:56.776213 13802 raft_consensus.cc:993] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: : Instructing follower e56d487365394ac5ac5f1002926a5645 to start an election
I20260709 12:09:56.777010 13802 raft_consensus.cc:1081] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Signalling peer e56d487365394ac5ac5f1002926a5645 to start an election
W20260709 12:09:56.796437 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 234507us
I20260709 12:09:56.797550 12427 catalog_manager.cc:5697] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:09:56.843272 12599 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:09:56.834450 13336 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d"
dest_uuid: "e56d487365394ac5ac5f1002926a5645"
 from {username='slave'} at 127.0.0.1:38482
I20260709 12:09:56.862275 13704 raft_consensus.cc:993] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: : Instructing follower e56d487365394ac5ac5f1002926a5645 to start an election
I20260709 12:09:56.862859 13704 raft_consensus.cc:1081] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Signalling peer e56d487365394ac5ac5f1002926a5645 to start an election
I20260709 12:09:56.875886 12529 raft_consensus.cc:2955] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:56.881052 12674 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:09:56.861477 13336 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:56.866019 13678 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:09:56.885030 13336 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:56.886899 13817 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:09:56.928061 13332 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d7bd864b75c34027b05582a55b152877"
dest_uuid: "e56d487365394ac5ac5f1002926a5645"
 from {username='slave'} at 127.0.0.1:38484
I20260709 12:09:56.884902 13777 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.044s
I20260709 12:09:56.950891 13767 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:09:56.968645 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 135352us
I20260709 12:09:56.947551 13332 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:57.005445 12600 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:56.988274 13332 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:57.075093 13336 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:57.081020 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:57.081911 12602 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:57.044101 13767 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:09:57.100685 13336 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:09:57.107375 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 187118us
I20260709 12:09:57.178465 13850 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:09:57.183542 12602 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
I20260709 12:09:57.179207 13850 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:57.192387 13850 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:09:57.196378 13332 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:57.214645 12425 catalog_manager.cc:5697] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: config changed from index 3 to 4, VOTER 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) evicted. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:09:57.222765 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 45330.3us
I20260709 12:09:57.265841 13332 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:57.276737 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:09:57.277541 12679 raft_consensus.cc:3055] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:09:57.277864 12679 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:09:57.278692 12679 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:09:57.290612 12646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.008s	sys 0.034s
W20260709 12:09:57.291064 12646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.008s	sys 0.034s
I20260709 12:09:57.313678 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:57.314378 12529 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:09:57.314706 12529 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:09:57.315500 12529 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:57.316349 13339 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:09:57.346753 12679 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:57.395308 12529 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:57.408491 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:57.414566 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:57.363363 13336 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260709 12:09:57.383822 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:09:57.432565 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:09:57.507099 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 411678us
I20260709 12:09:57.509430 13302 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:09:57.510938 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 79222.3us
W20260709 12:09:57.536232 13334 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.014s	sys 0.025s
I20260709 12:09:57.543756 12529 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
W20260709 12:09:57.581642 13859 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
W20260709 12:09:57.552287 13334 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.015s	sys 0.025s
W20260709 12:09:57.612385 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 255283us
I20260709 12:09:57.593116 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1148) took 1229 ms. Trace:
I20260709 12:09:57.586678 13337 raft_consensus.cc:2955] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:57.633803 13305 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:09:57.647956 13861 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:09:57.653388 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 236037us
I20260709 12:09:57.652953 13817 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:09:57.617444 12530 rpcz_store.cc:276] 0709 12:09:56.353837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:56.380230 (+ 26393us) service_pool.cc:224] Handling call
0709 12:09:56.380538 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:09:56.841832 (+461294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:09:56.841869 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:09:56.872897 (+ 31028us) log.cc:844] Serialized 26 byte log entry
0709 12:09:57.351861 (+478964us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:09:57.351922 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:09:57.352137 (+   215us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:09:57.583354 (+231217us) raft_consensus.cc:1685] finished
0709 12:09:57.583385 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:09:57.583629 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19321,"raft.queue_time_us":30194,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":23605,"thread_start_us":851200,"threads_started":2,"wal-append.queue_time_us":498446}
I20260709 12:09:57.673488 13795 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:09:57.704227 13795 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:57.673151 13857 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:09:57.672835 13817 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:57.712461 13817 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:57.578619 13335 raft_consensus.cc:2955] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:57.725880 12679 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
I20260709 12:09:57.729065 12530 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:09:57.729929 12530 raft_consensus.cc:606] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:09:57.736354 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:57.742290 13795 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:09:57.746186 12597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:32832
I20260709 12:09:57.747071 12597 raft_consensus.cc:606] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:09:57.752981 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:57.763093 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:09:57.771008 12569 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:09:57.773067 13851 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:57.798700 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:09:57.799402 12597 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:57.812420 13851 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 12:09:57.817277 13829 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:09:57.834177 13857 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 2 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:09:57.858667 13866 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.174s	user 0.005s	sys 0.000s
I20260709 12:09:57.830642 13829 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:09:57.861771 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 248445us
I20260709 12:09:57.862329 12597 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
I20260709 12:09:57.843576 13861 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 2 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:09:57.876411 12643 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:09:57.877279 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 95129.3us
I20260709 12:09:57.860863 13857 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:57.858019 13801 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:57.898550 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:57.864684 13780 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
W20260709 12:09:57.962643 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 100002us
I20260709 12:09:57.964762 12429 catalog_manager.cc:5697] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:57.903940 13801 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:57.934808 13872 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:57.986855 12429 catalog_manager.cc:5697] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: config changed from index 3 to 4, VOTER f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) evicted. New cstate: current_term: 1 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:09:57.996452 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:58.011099 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:09:58.038673 13879 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 12:09:58.074786 12678 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a68955866d543408acca9e959ad1b26"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:09:58.075515 12678 raft_consensus.cc:606] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:58.081015 12662 tablet_service.cc:1558] Processing DeleteTablet for tablet ee47f7c4414b4ae98233a31696e99cc0 with delete_type TABLET_DATA_TOMBSTONED (TS 193a40da6e364ead8ab561e14c3cc8d1 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51290
I20260709 12:09:58.081717 13801 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:58.079015 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:09:58.089663 13851 raft_consensus.cc:993] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: : Instructing follower 193a40da6e364ead8ab561e14c3cc8d1 to start an election
I20260709 12:09:58.090940 13817 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:09:58.104045 13855 raft_consensus.cc:993] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: : Instructing follower 193a40da6e364ead8ab561e14c3cc8d1 to start an election
I20260709 12:09:58.104637 13855 raft_consensus.cc:1081] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Signalling peer 193a40da6e364ead8ab561e14c3cc8d1 to start an election
I20260709 12:09:58.151499 13868 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:09:58.152140 13868 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:58.155819 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 213882us
I20260709 12:09:57.956248 13861 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:09:58.156752 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 51194.3us
I20260709 12:09:58.105569 13678 raft_consensus.cc:1081] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Signalling peer 193a40da6e364ead8ab561e14c3cc8d1 to start an election
I20260709 12:09:57.938050 13863 raft_consensus.cc:2955] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:58.193075 12674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "e56d487365394ac5ac5f1002926a5645"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:09:58.194887 12674 raft_consensus.cc:606] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Received request to transfer leadership to TS e56d487365394ac5ac5f1002926a5645
W20260709 12:09:58.229528 13883 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 12:09:58.206037 13829 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:09:58.263692 13857 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:09:58.251513 13854 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:58.286734 12674 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "e56d487365394ac5ac5f1002926a5645"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:09:58.287811 12674 raft_consensus.cc:606] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Received request to transfer leadership to TS e56d487365394ac5ac5f1002926a5645
W20260709 12:09:58.289242 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 131671us
I20260709 12:09:58.264485 13857 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:09:58.291326 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 184234us
I20260709 12:09:58.281637 13758 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:58.301273 13887 raft_consensus.cc:993] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: : Instructing follower e56d487365394ac5ac5f1002926a5645 to start an election
I20260709 12:09:58.281932 13704 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:09:58.327778 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 337018us
I20260709 12:09:58.331653 12676 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:51304
I20260709 12:09:58.332412 12676 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:58.332813 12676 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:09:58.302282 13758 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:58.318269 13887 raft_consensus.cc:1081] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Signalling peer e56d487365394ac5ac5f1002926a5645 to start an election
I20260709 12:09:58.314854 13704 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:58.299470 13854 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:58.365396 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:58.376405 12671 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e15d21db797e4a96b5732711a5379980"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "e56d487365394ac5ac5f1002926a5645"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:09:58.381197 13337 raft_consensus.cc:2955] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:58.382484 12671 raft_consensus.cc:606] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Received request to transfer leadership to TS e56d487365394ac5ac5f1002926a5645
I20260709 12:09:58.368932 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:09:58.406354 12571 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:09:58.411180 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:58.446480 12675 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:51308
I20260709 12:09:58.447517 12675 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:58.448012 12675 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:58.477794 13704 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:09:58.410810 13890 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
W20260709 12:09:58.455399 13867 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.669s	user 0.010s	sys 0.001s
I20260709 12:09:58.482173 13900 tablet_replica.cc:333] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:09:58.496780 13895 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:09:58.502197 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 95997us
I20260709 12:09:58.484547 13784 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:58.529032 13854 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:58.530743 13900 raft_consensus.cc:2243] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:09:58.531842 13900 raft_consensus.cc:2272] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:09:58.540048 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 158563us
I20260709 12:09:58.558365 13896 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:09:58.558748 13335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 262) took 1328 ms. Trace:
I20260709 12:09:58.558660 13332 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394"
dest_uuid: "e56d487365394ac5ac5f1002926a5645"
 from {username='slave'} at 127.0.0.1:38482
I20260709 12:09:58.559674 13332 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:09:58.571710 12678 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:09:58.543478 13829 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:09:58.584554 13863 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:51774 (request call id 66) took 1588 ms. Trace:
I20260709 12:09:58.539659 13784 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:09:58.533380 13895 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:58.591353 13895 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:09:58.564986 13758 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:58.566231 13733 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:58.596652 13829 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:58.559412 13335 rpcz_store.cc:276] 0709 12:09:57.230718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:57.236338 (+  5620us) service_pool.cc:224] Handling call
0709 12:09:57.277415 (+ 41077us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:09:57.278065 (+   650us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:09:57.278103 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:09:57.279527 (+  1424us) log.cc:844] Serialized 501 byte log entry
0709 12:09:57.551850 (+272323us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:09:57.859945 (+308095us) log.cc:844] Serialized 14 byte log entry
0709 12:09:57.872249 (+ 12304us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:09:57.872399 (+   150us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:09:58.286313 (+413914us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:09:58.553112 (+266799us) raft_consensus.cc:1685] finished
0709 12:09:58.558464 (+  5352us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:09:58.558658 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5139,"raft.queue_time_us":12505,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":185,"spinlock_wait_cycles":4661632,"thread_start_us":250362,"threads_started":1,"wal-append.queue_time_us":393034}
I20260709 12:09:58.575335 13332 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:58.626641 13733 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:58.627449 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:58.573582 13904 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:09:58.648979 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:58.649973 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:58.629804 12676 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:58.585192 13863 rpcz_store.cc:276] 0709 12:09:56.995813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:57.001180 (+  5367us) service_pool.cc:224] Handling call
0709 12:09:57.159930 (+158750us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b540002cc20. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f148899564d 00007f1488978fe9 00007f1488978f3e 00007f14889fe888 00007f1488a1b75c 00007f1488a1c76a 00007f1488a1b91b 00007f1488a2703c 00007f1488a266b3 00007f1488a1e9fa
0709 12:09:57.166857 (+  6927us) log.cc:844] Serialized 501 byte log entry
0709 12:09:58.584398 (+1417541us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6733,"raft.queue_time_us":8819,"raft.run_cpu_time_us":2369,"raft.run_wall_time_us":2740,"spinlock_wait_cycles":23001856}
I20260709 12:09:58.661577 12605 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
W20260709 12:09:58.657526 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 55966.5us
I20260709 12:09:58.668246 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:09:58.670485 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:58.591488 13784 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:58.666153 13904 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:58.632755 13829 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:58.690786 12676 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:58.706696 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:09:58.956571 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 214550us
W20260709 12:09:58.729538 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 51225.3us
I20260709 12:09:59.019395 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:59.086664 12673 raft_consensus.cc:2955] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:09:58.707247 12675 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:59.103935 13337 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.014s	sys 0.006s
W20260709 12:09:59.105360 13337 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.015s	sys 0.006s
I20260709 12:09:58.675783 13854 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:59.174566 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:09:59.065840 13861 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.011s	sys 0.012s
I20260709 12:09:58.705842 13733 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:09:58.686008 13863 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:59.130707 12494 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:09:58.955631 13817 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:09:58.951892 13893 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:58.753225 13856 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:58.740012 13868 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:58.740288 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:09:58.778028 13332 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:09:59.203116 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 495204us
I20260709 12:09:59.366617 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:59.571686 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:09:59.572448 12530 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:09:59.573239 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 484305us
I20260709 12:09:59.338970 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:59.135643 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:59.536129 13872 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:09:59.015741 13758 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:59.179190 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:09:59.574537 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 443913us
I20260709 12:09:58.951504 13877 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:09:59.583117 12413 catalog_manager.cc:5210] ChangeConfig:REMOVE_PEER RPC for tablet 816b6f34d124434882148e507cf5b612 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260709 12:09:59.254370 13910 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.012s	sys 0.023s
I20260709 12:09:59.577102 13914 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:09:59.315366 13303 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.007s	sys 0.034s
W20260709 12:09:59.574505 13861 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.037s	user 0.022s	sys 0.057s
I20260709 12:09:58.701695 13904 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:59.131870 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:09:59.363456 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:09:59.302424 13909 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:09:59.650105 13818 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:09:59.273190 13898 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:59.256201 13788 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:59.577253 13761 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:09:59.583384 13877 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:59.652252 12674 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:09:59.577153 13828 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:09:59.599503 13910 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.016s	user 0.035s	sys 0.039s
I20260709 12:09:59.576830 13872 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:59.582355 13868 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:09:59.235374 13854 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:09:59.653987 13303 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.151s	user 0.014s	sys 0.036s
I20260709 12:09:59.582019 13863 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:59.661005 13788 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:59.712656 13872 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:59.658035 13761 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:59.713095 13902 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:09:59.713408 13761 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:09:59.579247 13817 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } attrs { replace: true } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:59.602054 13758 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:59.674374 13816 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:09:59.650668 13914 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:59.653134 13818 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:59.751194 13733 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:59.652558 13909 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:59.610388 13893 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:59.746019 12530 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
W20260709 12:09:59.723551 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 140929us
I20260709 12:09:59.663441 13898 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:59.651566 13856 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:59.702509 13868 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:59.713606 13902 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:59.752388 13816 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:59.707273 13828 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:59.860883 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:00.167888 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:00.185146 12523 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 1.
I20260709 12:10:00.162415 13920 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:00.127624 13919 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:00.157503 13918 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:00.171677 12675 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:59.782721 13733 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:59.882893 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:00.218726 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:51308 (request call id 1377) took 1810 ms. Trace:
W20260709 12:09:59.954401 13916 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 12:09:59.759801 13851 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:00.219290 12675 rpcz_store.cc:276] 0709 12:09:58.379382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:58.379544 (+   162us) service_pool.cc:224] Handling call
0709 12:10:00.154280 (+1774736us) spinlock_profiling.cc:243] Waited 48.8 ms on lock 0x7b4c00333000. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1486ceca20 00007f1486cfda29 00007f1486cfe13f 00007f1486cc22ab 00007f1486ceff3c 00007f1486cf03fe 00007f1486f7d88b 00007f148897c495 00007f14889cd30f 00007f1488a17a55 00007f148d9e7cd0
0709 12:10:00.189354 (+ 35074us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":112330112}
I20260709 12:09:59.782513 13900 ts_tablet_manager.cc:1916] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:09:59.927430 13332 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:00.187613 13920 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:00.300925 13332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:38482 (request call id 218) took 1744 ms. Trace:
I20260709 12:10:00.301393 13332 rpcz_store.cc:276] 0709 12:09:58.556700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:58.558471 (+  1771us) service_pool.cc:224] Handling call
0709 12:10:00.300807 (+1742336us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:00.301980 13907 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:00.303004 13920 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:00.303992 13920 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:00.304683 13920 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:09:59.937443 13915 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:00.306751 13915 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:00.167479 13872 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:00.300061 13902 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:09:59.961781 13818 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:59.966032 13817 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:09:59.786967 13873 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:00.172782 13863 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:00.184317 13914 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:59.881469 13877 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:00.297837 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:00.259505 13898 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:59.960464 13909 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:09:59.876303 13893 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:59.941494 13881 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:10:00.349280 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 76883.7us
I20260709 12:10:00.350935 13930 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:00.351790 13930 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:00.126726 13917 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:00.313237 13927 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:09:59.957129 13758 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:00.326321 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:09:59.964221 13761 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:09:59.919955 13906 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:00.187821 13919 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:00.460327 13930 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:09:59.929903 13854 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:00.254750 13926 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:00.319110 13872 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:00.302956 13907 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:00.346652 13850 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:00.367218 13909 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:00.345220 13877 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:00.253975 13922 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:00.521544 13935 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:00.526001 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1378) took 1880 ms. Trace:
I20260709 12:10:00.315877 13851 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:00.546587 12602 rpcz_store.cc:276] 0709 12:09:58.623064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:58.623812 (+   748us) service_pool.cc:224] Handling call
0709 12:09:58.629519 (+  5707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:09:58.634906 (+  5387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:09:58.634937 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:09:58.634980 (+    43us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:09:58.635008 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:00.503105 (+1868097us) spinlock_profiling.cc:243] Waited 4.46 ms on lock 0x7b540007b054. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:00.503185 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:00.503398 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10264320}
I20260709 12:10:00.500334 13876 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:00.558084 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:00.559321 12530 raft_consensus.cc:2393] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 2: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:00.313938 13918 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:00.536345 13930 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:00.544627 13915 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:00.574272 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1305) took 1485 ms. Trace:
I20260709 12:10:00.323248 13873 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:00.576026 13940 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:00.574177 13930 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:00.555200 13863 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:00.571089 13914 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:00.539059 13909 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:00.582693 13931 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:00.557332 13841 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:00.542006 13850 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:00.558302 13935 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:00.574879 12525 rpcz_store.cc:276] 0709 12:09:59.053734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:59.053942 (+   208us) service_pool.cc:224] Handling call
0709 12:09:59.066228 (+ 12286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:09:59.088505 (+ 22277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:09:59.088536 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:09:59.113518 (+ 24982us) log.cc:844] Serialized 26 byte log entry
0709 12:09:59.537981 (+424463us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:09:59.724086 (+186105us) log.cc:844] Serialized 14 byte log entry
0709 12:09:59.729898 (+  5812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:09:59.730012 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:00.538775 (+808763us) raft_consensus.cc:1685] finished
0709 12:10:00.538798 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:00.538922 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5505,"prepare.queue_time_us":417535,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":57462,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":30813,"thread_start_us":432988,"threads_started":2,"wal-append.queue_time_us":457680}
I20260709 12:10:00.452561 13881 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:00.500258 13926 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:00.472978 13927 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:00.560360 13877 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:00.594272 13938 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:00.542369 13906 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:00.576515 13923 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:00.598361 13816 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:00.571976 13876 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:00.582960 13937 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:00.462661 13917 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:00.600629 13935 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:00.599536 13828 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:00.614249 12530 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:00.602002 13868 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:00.587672 13863 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:00.597049 13733 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:00.589963 13914 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:00.561838 13922 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:00.623481 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:39924 (request call id 8) took 2265 ms. Trace:
I20260709 12:10:00.616236 13939 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:00.616447 13934 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:00.624265 12671 rpcz_store.cc:276] 0709 12:09:58.358496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:58.358659 (+   163us) service_pool.cc:224] Handling call
0709 12:10:00.617848 (+2259189us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b5400284720. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f148899d520 00007f1488a1a652 00007f148d9e860a 00007f1486f6e7d4 00007f1486f6e762 00007f1486f6e691 00007f1486f6e60c 00007f1486f6d8c2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:00.623365 (+  5517us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4206848}
I20260709 12:10:00.620785 13761 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:00.643677 13907 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:00.656018 13857 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:00.596804 13940 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:00.621492 13937 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:00.602352 13841 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:00.633941 13941 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:00.607852 13931 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:00.610826 13938 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:00.637192 13920 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:00.618793 13923 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:00.700089 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1310) took 1118 ms. Trace:
I20260709 12:10:00.937175 12524 rpcz_store.cc:276] 0709 12:09:59.581987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:59.582111 (+   124us) service_pool.cc:224] Handling call
0709 12:09:59.582427 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:09:59.722807 (+140380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:09:59.722855 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:09:59.722912 (+    57us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:09:59.722957 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:00.699735 (+976778us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54003f7d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:00.699810 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:00.700006 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":322172672}
I20260709 12:10:00.790575 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1380) took 1650 ms. Trace:
I20260709 12:10:00.940264 12673 rpcz_store.cc:276] 0709 12:09:59.092877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:59.592972 (+500095us) service_pool.cc:224] Handling call
0709 12:09:59.655462 (+ 62490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:00.269788 (+614326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:00.269840 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:00.269904 (+    64us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:00.269952 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:00.743395 (+473443us) spinlock_profiling.cc:243] Waited 614 ms on lock 0x7b54000c26d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:00.743469 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:00.743657 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1412198272}
I20260709 12:10:00.678385 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:10:00.637601 13872 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:00.949709 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 329856us
I20260709 12:10:00.700865 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1309) took 1479 ms. Trace:
I20260709 12:10:00.961637 13944 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:00.962404 13944 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:00.964964 13944 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:00.653700 13939 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:00.960026 12522 rpcz_store.cc:276] 0709 12:09:59.221186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:59.577593 (+356407us) service_pool.cc:224] Handling call
0709 12:09:59.577939 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:09:59.714020 (+136081us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:09:59.714057 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:09:59.714104 (+    47us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:09:59.714156 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:00.700596 (+986440us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b5400054454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:00.700657 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:00.700799 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":312503680}
W20260709 12:10:00.724311 12569 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.132:45801 timed out after 1.983s (SENT) [suppressed 1 similar messages]
I20260709 12:10:00.696512 13930 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:00.658224 13852 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:00.873075 13863 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:00.923485 13909 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:00.887111 13919 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:00.731611 13853 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:00.690201 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:00.855154 13850 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:00.982805 13852 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:00.984861 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:00.985188 13939 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:00.974165 13906 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:00.957898 13914 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:00.974941 13944 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:00.986784 13917 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:00.966315 13922 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:00.979738 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 255484us
I20260709 12:10:00.656333 13918 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:00.881382 13926 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:00.997918 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 59967us
I20260709 12:10:00.720090 13942 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:00.739302 13868 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:01.000154 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:00.698266 13945 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:00.972793 13857 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:01.004455 13943 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.118s
I20260709 12:10:00.681731 13934 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.020128 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:01.022383 12676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1385) took 1910 ms (client timeout 1896 ms). Trace:
I20260709 12:10:01.017400 13780 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:01.001682 13877 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:01.014612 13841 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:00.978190 13941 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.038600 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.371218 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:01.003491 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 79897us
I20260709 12:10:01.329787 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.377512 13337 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:10:01.378139 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.021412 13850 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:01.373279 12676 rpcz_store.cc:269] 0709 12:09:59.097797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:59.301500 (+203703us) service_pool.cc:224] Handling call
0709 12:10:00.984679 (+1683179us) spinlock_profiling.cc:243] Waited 2.01 ms on lock 0x7b58001f2c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:10:01.007671 (+ 22992us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4621184}
I20260709 12:10:01.031389 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:01.389549 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.389608 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.391343 12675 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:01.392422 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.090799 12677 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:01.394629 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.389804 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:01.397073 13338 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:01.018327 13853 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.122696 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:01.402038 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.323905 13937 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:01.252288 13852 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:01.179066 13914 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:01.307813 13931 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:01.033758 13863 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:01.404203 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.121673 13947 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:01.064353 13917 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:01.184685 13782 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:10:01.406843 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 2.0362e+06us
I20260709 12:10:01.458264 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.305711 13923 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:01.304324 13940 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:01.312204 13802 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:01.477931 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1204) took 1148 ms. Trace:
I20260709 12:10:01.477629 13802 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.478030 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.478366 12528 rpcz_store.cc:276] 0709 12:10:00.306539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:00.306667 (+   128us) service_pool.cc:224] Handling call
0709 12:10:00.306975 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:00.615603 (+308628us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:00.615640 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:00.615685 (+    45us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:00.615747 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:01.454334 (+838587us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b54003f6954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:01.454383 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:01.454541 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":709316480}
I20260709 12:10:01.479027 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.385917 13918 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:01.390434 13945 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.413214 13841 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:01.390822 13850 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.414963 13887 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:01.039407 13944 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.416476 13905 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:01.417727 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.398532 13942 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:01.419956 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.423874 13906 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:01.390583 13895 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:01.405800 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:01.411177 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.511897 12527 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:01.433594 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:01.512892 12530 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:01.513376 12530 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:01.434986 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.387967 13891 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:01.324240 13938 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:01.314904 13881 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:01.437417 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.417217 13853 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:01.428020 13927 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:01.474480 12606 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:10:01.474923 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.436089 13795 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:01.383049 13780 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.480433 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.388602 13868 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:01.481024 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 68171.8us
I20260709 12:10:01.483210 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.546082 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.491328 13841 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:01.491065 12599 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 1.
I20260709 12:10:01.559973 13953 raft_consensus.cc:993] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1: : Instructing follower e56d487365394ac5ac5f1002926a5645 to start an election
I20260709 12:10:01.382419 13857 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:01.560494 13953 raft_consensus.cc:1081] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Signalling peer e56d487365394ac5ac5f1002926a5645 to start an election
I20260709 12:10:01.561122 13841 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:01.562083 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.562639 13857 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:01.563503 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.564234 12606 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 1.
I20260709 12:10:01.492851 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.497978 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.443477 13861 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260709 12:10:01.516753 13906 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.567349 13780 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:01.501039 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:01.568174 12603 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:01.568398 13906 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:01.568557 12603 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:01.528416 13939 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:01.513650 13895 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.570585 13939 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.572630 13895 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:01.485458 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:01.503108 13781 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:01.573323 13939 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:01.573915 13781 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:01.569446 12603 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.575587 12603 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:01.577373 13781 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:10:01.503649 13850 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:01.579175 12600 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.514427 12675 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:01.507377 13952 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:01.626582 13952 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.628021 13873 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:01.628011 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 81689.9us
I20260709 12:10:01.629069 12603 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:01.514879 12531 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:10:01.486877 13948 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:01.698547 13948 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.698988 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.700299 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.700389 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.700973 13948 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:01.701087 12525 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:01.453048 13914 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:01.701416 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.701568 12572 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:10:01.523313 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 66163.4us
I20260709 12:10:01.702653 12496 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:01.703473 13914 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:01.703806 13932 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.479785 13917 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.704643 13924 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.705092 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.487198 13856 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:01.531869 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.707451 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.708189 12525 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
W20260709 12:10:01.708307 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 117930us
I20260709 12:10:01.480458 13947 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.550459 13795 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.711256 13947 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:01.711606 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.711913 13795 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:01.551947 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.544735 13943 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:01.714360 12571 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:01.714613 13943 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:01.715721 13931 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:01.717211 13946 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 3 current term: 3 is higher.
I20260709 12:10:01.497330 13887 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.719879 13887 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:01.525795 13892 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:01.720206 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.721108 12603 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:10:01.525309 13891 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:01.557266 13944 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:01.558616 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.722910 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.723851 12677 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 1.
I20260709 12:10:01.724889 13931 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.725091 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.542927 13927 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:01.726478 13927 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.728955 13927 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:01.504858 13905 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.497232 13918 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.569446 13941 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:01.733062 13918 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:01.569305 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.734592 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.735378 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.479830 13782 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:01.502351 13733 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:01.736657 13733 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.738988 13733 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:01.571233 12598 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 1.
I20260709 12:10:01.740186 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.740448 12425 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47524 (request call id 89) took 3730 ms. Trace:
I20260709 12:10:01.741077 12425 rpcz_store.cc:276] 0709 12:09:58.074990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:58.080709 (+  5719us) service_pool.cc:224] Handling call
0709 12:10:01.740303 (+3659594us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":113287,"reported_tablets":2}
I20260709 12:10:01.503541 13920 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:01.742331 13920 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:01.742978 13782 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:01.581467 13935 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.959s	user 0.014s	sys 0.000s
I20260709 12:10:01.581333 13788 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:01.744206 13905 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:01.582862 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:01.583701 13942 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:01.586737 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.491639 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.747044 13891 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:01.591645 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1304) took 2558 ms. Trace:
I20260709 12:10:01.717818 13943 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:01.593355 13339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 273) took 2741 ms. Trace:
I20260709 12:10:01.565569 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.726478 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:01.749696 12606 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 1.
I20260709 12:10:01.751103 13943 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.603489 13330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 278) took 1785 ms. Trace:
I20260709 12:10:01.604228 13851 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:01.598759 13868 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:01.610970 13861 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:10:01.755358 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 48232.5us
I20260709 12:10:01.613488 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.748324 12526 rpcz_store.cc:276] 0709 12:09:59.032864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:59.042276 (+  9412us) service_pool.cc:224] Handling call
0709 12:09:59.065594 (+ 23318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:00.314455 (+1248861us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:00.314486 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:00.314527 (+    41us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:00.314559 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:01.591081 (+1276522us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54003cc8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:01.591156 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:01.591370 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2871662464}
I20260709 12:10:01.614651 12569 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:01.757169 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 142609us
I20260709 12:10:01.617084 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.620642 13934 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:01.749339 13339 rpcz_store.cc:276] 0709 12:09:58.851916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:58.955217 (+103301us) service_pool.cc:224] Handling call
0709 12:09:58.955672 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:09:59.729613 (+773941us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:09:59.731752 (+  2139us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:09:59.746555 (+ 14803us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:09:59.746621 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:01.592942 (+1846321us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b5400145fd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:01.593022 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:01.593234 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1752941952}
I20260709 12:10:01.758739 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.620857 13704 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:01.760426 13920 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:01.756541 12674 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 1.
I20260709 12:10:01.760234 13704 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.761205 13733 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.761713 13733 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:01.622090 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:01.761834 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.762169 13733 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.762493 12674 raft_consensus.cc:1813] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Handling vote request from an unknown peer f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:01.625592 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.630328 13952 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:01.763144 12674 raft_consensus.cc:2410] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 2 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:10:01.495056 13923 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:01.763706 13923 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.637091 13802 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:01.609771 13900 ts_tablet_manager.cc:1929] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
W20260709 12:10:01.621955 12494 connection.cc:256] RPC call timeout handler was delayed by -1.6002885919999699s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1555s for a response
I20260709 12:10:01.764962 13900 log.cc:1199] T ee47f7c4414b4ae98233a31696e99cc0 P 193a40da6e364ead8ab561e14c3cc8d1: Deleting WAL directory at /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-2-root/wals/ee47f7c4414b4ae98233a31696e99cc0
I20260709 12:10:01.765300 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:01.765604 12494 connection.cc:256] RPC call timeout handler was delayed by -1.2933885919999284s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1896s for a response
I20260709 12:10:01.632449 13876 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:01.766076 12494 connection.cc:256] RPC call timeout handler was delayed by -1.0305885919999582s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21880000000000002s for a response
I20260709 12:10:01.765976 13923 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:01.766296 13900 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51290 (request call id 70) took 3764 ms. Trace:
I20260709 12:10:01.646060 12646 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:10:01.768695 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 122700us
I20260709 12:10:01.769670 12429 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58980 (request call id 180) took 4046 ms. Trace:
I20260709 12:10:01.771395 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.735389 12677 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:01.754674 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:01.772519 12646 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:01.651783 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1203) took 1346 ms. Trace:
I20260709 12:10:01.773692 13920 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.773819 12529 rpcz_store.cc:276] 0709 12:10:00.305101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:00.305261 (+   160us) service_pool.cc:224] Handling call
0709 12:10:00.305594 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:00.621569 (+315975us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:00.621602 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:00.621652 (+    50us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:00.621682 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:01.651459 (+1029777us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b54002107d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:01.651523 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:01.651704 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":726167424}
I20260709 12:10:01.654074 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.776005 12678 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 1.
I20260709 12:10:01.776525 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.777468 12677 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 1.
I20260709 12:10:01.657907 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:01.779171 13334 raft_consensus.cc:1813] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Handling vote request from an unknown peer a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:01.780230 13334 raft_consensus.cc:2410] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 2 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:10:01.661311 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1202) took 1359 ms. Trace:
I20260709 12:10:01.781574 12523 rpcz_store.cc:276] 0709 12:10:00.301520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:00.303860 (+  2340us) service_pool.cc:224] Handling call
0709 12:10:00.304347 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:00.343781 (+ 39434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:00.343827 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:00.343910 (+    83us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:00.343955 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:01.661011 (+1317056us) spinlock_profiling.cc:243] Waited 39.2 ms on lock 0x7b54003c0854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:01.661074 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:01.661232 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":90046336}
I20260709 12:10:01.663304 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.691900 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.693655 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.784359 12672 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 1.
I20260709 12:10:01.685529 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:01.697443 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:01.695974 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 141553us
I20260709 12:10:01.455920 13863 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.785387 12680 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:01.784361 12597 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 1.
I20260709 12:10:01.785917 12680 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:01.786638 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.786767 12680 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:01.787420 13863 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:01.704092 12496 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:01.496987 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:01.518286 12530 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.706082 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.705960 13917 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:01.788386 12604 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.706176 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.789606 12601 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:10:01.707158 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:10:01.790612 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.790956 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:01.791321 12601 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:01.704118 13333 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:01.792389 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:01.708530 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.785553 12605 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:10:01.785387 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.794502 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:01.794679 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.794817 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:01.795146 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.795411 13331 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:01.531735 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.796552 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.528201 13926 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:01.720711 13892 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:01.751030 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.529119 13938 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:01.751693 13330 rpcz_store.cc:276] 0709 12:09:59.818529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:59.820164 (+  1635us) service_pool.cc:224] Handling call
0709 12:10:01.602649 (+1782485us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b5800163298. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fbeee 000000000036eb39 00007f148d96f1e0 00007f148da380c4 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:10:01.602862 (+   213us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:01.603078 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:01.603107 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:01.603153 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:01.603187 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:01.603247 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:01.603416 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":25997184}
I20260709 12:10:01.757525 12427 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58992 (request call id 166) took 3748 ms. Trace:
I20260709 12:10:01.751698 13935 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:01.759392 12673 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 1.
I20260709 12:10:01.764017 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.647276 12643 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:01.767072 13900 rpcz_store.cc:276] 0709 12:09:58.017137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:58.080723 (+ 63586us) service_pool.cc:224] Handling call
0709 12:09:58.393401 (+312678us) ts_tablet_manager.cc:1151] Deleting tablet ee47f7c4414b4ae98233a31696e99cc0
0709 12:09:58.401623 (+  8222us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:10:01.422784 (+3021161us) tablet_metadata.cc:665] Metadata flushed
0709 12:10:01.609568 (+186784us) tablet_metadata.cc:665] Metadata flushed
0709 12:10:01.766205 (+156637us) inbound_call.cc:177] Queueing success response
Metrics: {"tablet-delete.queue_time_us":288338,"thread_start_us":226018,"threads_started":1}
I20260709 12:10:01.772703 13943 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:01.788823 12679 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:01.799911 12569 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:01.788908 12530 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.795316 13335 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 2: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:01.796816 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.792407 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.741516 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.787962 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:01.798776 12605 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:10:01.787761 12680 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.770092 12429 rpcz_store.cc:276] 0709 12:09:57.723738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:57.724188 (+   450us) service_pool.cc:224] Handling call
0709 12:10:01.769603 (+4045415us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:09:58.245760 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:09:58.371372 (+125612us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:09:58.375050 (+  3678us) write_op.cc:435] Acquired schema lock
0709 12:09:58.380027 (+  4977us) tablet.cc:662] Decoding operations
0709 12:09:58.457091 (+ 77064us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:09:58.478923 (+ 21832us) write_op.cc:641] Partition lock acquired for write op
0709 12:09:58.478976 (+    53us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:09:58.479540 (+   564us) tablet.cc:701] Row locks acquired
0709 12:09:58.479567 (+    27us) write_op.cc:260] PREPARE: finished
0709 12:09:58.479791 (+   224us) write_op.cc:270] Start()
0709 12:09:58.479979 (+   188us) write_op.cc:276] Timestamp: P: 1783598998479731 usec, L: 0
0709 12:09:58.480010 (+    31us) op_driver.cc:348] REPLICATION: starting
0709 12:09:58.480968 (+   958us) log.cc:844] Serialized 918 byte log entry
0709 12:09:59.720478 (+1239510us) op_driver.cc:464] REPLICATION: finished
0709 12:10:00.255353 (+534875us) write_op.cc:301] APPLY: starting
0709 12:10:00.255498 (+   145us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:00.255667 (+   169us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:00.255843 (+   176us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:00.256546 (+   703us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:00.256793 (+   247us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=2
0709 12:10:00.256864 (+    71us) write_op.cc:312] APPLY: finished
0709 12:10:00.257156 (+   292us) log.cc:844] Serialized 28 byte log entry
0709 12:10:00.259918 (+  2762us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:00.260027 (+   109us) write_op.cc:454] Released schema lock
0709 12:10:00.260066 (+    39us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:00.261195 (+  1129us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:00.261224 (+    29us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":170947,"reported_tablets":9,"child_traces":[["op",{"apply.queue_time_us":530733,"apply.run_cpu_time_us":2626,"apply.run_wall_time_us":5889,"num_ops":2,"prepare.queue_time_us":218117,"prepare.run_cpu_time_us":15381,"prepare.run_wall_time_us":353645,"raft.queue_time_us":1018583,"raft.run_cpu_time_us":14809,"raft.run_wall_time_us":479174,"replication_time_us":1182727,"thread_start_us":1593582,"threads_started":4,"wal-append.queue_time_us":100022,"wal-append.run_cpu_time_us":78512,"wal-append.run_wall_time_us":2698985}]]}
I20260709 12:10:01.772554 12674 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 1.
I20260709 12:10:01.803756 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:01.671811 12600 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:01.749102 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.558710 13945 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:01.833359 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 76861us
I20260709 12:10:01.802359 12427 rpcz_store.cc:276] 0709 12:09:58.008805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:09:58.008977 (+   172us) service_pool.cc:224] Handling call
0709 12:10:01.757446 (+3748469us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32245,"reported_tablets":2}
I20260709 12:10:01.828524 12603 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:01.823736 13943 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.829941 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:01.829847 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:01.802258 13861 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:01.797387 13926 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:01.831568 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 184372us
W20260709 12:10:01.833359 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 129319us
I20260709 12:10:01.834055 13952 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:01.833999 13788 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.834736 13957 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.856447 13957 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:01.856921 13957 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.835326 13876 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.860430 13876 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:01.860864 13876 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.835392 12569 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:01.823597 13938 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:01.797387 13905 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:01.836406 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:01.870791 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.840911 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.873232 12527 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:01.842012 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.845047 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.875432 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.846189 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.876698 13332 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:01.877372 13895 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:01.877811 13895 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:01.862944 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.823998 13856 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:01.836752 13923 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.848909 12415 catalog_manager.cc:5028] TS 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): tablet ee47f7c4414b4ae98233a31696e99cc0 (table test-workload [id=8634097bac3246c9a57e272441058c28]) successfully deleted
I20260709 12:10:01.878499 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.876518 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.880561 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.874872 12571 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:01.895416 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:01.896158 12676 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:01.896711 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:01.863654 13851 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.863471 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:01.882154 13330 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:01.883378 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.889875 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.890079 12597 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:01.892745 12494 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:01.904663 12527 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:10:01.905802 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.882040 13935 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:01.899200 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:01.900403 12415 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 51828.7us
I20260709 12:10:01.902386 12530 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:01.843250 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:01.902729 12680 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
I20260709 12:10:01.865367 13920 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.891685 13895 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:01.922919 12571 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:01.926857 13959 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:01.927105 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:01.907069 13336 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e15d21db797e4a96b5732711a5379980"
dest_uuid: "e56d487365394ac5ac5f1002926a5645"
 from {username='slave'} at 127.0.0.1:38482
I20260709 12:10:01.927871 13336 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:10:01.928325 13336 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.931648 13733 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:01.907917 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.908571 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:01.934583 13332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 86) took 1624 ms. Trace:
I20260709 12:10:01.909155 13924 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:01.934918 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:01.935276 13924 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:01.935647 13339 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:10:01.935961 13924 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:01.936921 12602 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:01.911732 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:01.939510 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.940086 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.900261 13918 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.913844 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.866452 13861 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:01.942145 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.942781 12676 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:01.944109 13861 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:01.919108 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 75986.4us
I20260709 12:10:01.944964 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:01.945091 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.946063 12676 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:01.947369 13918 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:01.877377 13704 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:01.904580 12522 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:01.949474 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.950179 12572 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:01.950634 12679 raft_consensus.cc:1813] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Handling vote request from an unknown peer f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:01.922830 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:01.906075 12496 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:01.900417 13941 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:01.924134 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.924813 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:01.927284 13959 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:01.935068 13332 rpcz_store.cc:276] 0709 12:10:00.310644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:00.315828 (+  5184us) service_pool.cc:224] Handling call
0709 12:10:00.505908 (+190080us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:01.877300 (+1371392us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:01.934263 (+ 56963us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54004439d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:01.934511 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3056518912}
I20260709 12:10:01.961210 13959 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.890640 13905 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:01.961349 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.890267 13892 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:01.938620 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:01.914582 13957 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:01.894029 13923 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:01.940925 12674 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 2: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:01.912919 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:01.916683 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:01.951524 12679 raft_consensus.cc:2410] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 2 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:10:01.953330 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.898533 12646 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:01.947853 12429 catalog_manager.cc:5697] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 1 to 2, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 2 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:01.955092 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.955142 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:02.002717 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 104251us
I20260709 12:10:01.956835 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.903669 13851 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:01.964319 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58316.1us
I20260709 12:10:01.965788 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:01.965960 13302 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:01.906519 12527 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:10:01.971267 13861 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:10:01.968564 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 45828.8us
I20260709 12:10:01.922148 13920 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:01.969348 13704 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.055666 13920 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.048117 13861 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:01.970782 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.056871 12680 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:01.966945 13733 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:02.001101 12604 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
I20260709 12:10:02.039299 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:02.039639 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:01.947937 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:01.966544 13932 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:02.003253 12671 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:02.003332 12429 catalog_manager.cc:5697] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 reported cstate change: config changed from index 3 to 4, VOTER a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) evicted. New cstate: current_term: 1 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:10:02.046607 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:01.947849 13946 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:01.924825 13960 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.048887 12606 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:01.947876 13918 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:02.054850 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:02.108398 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:01.924682 13931 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.058581 12646 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:02.009897 13851 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:01.938424 13856 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:02.087127 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:02.111666 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 53177.1us
W20260709 12:10:02.112531 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 66025.2us
I20260709 12:10:02.097179 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:02.113348 12643 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:02.101377 13957 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:02.103180 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:02.123049 13891 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.123581 13891 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:02.123980 13891 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:02.047113 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 81236.9us
I20260709 12:10:02.062289 13935 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.010733 13952 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.117107 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.117837 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.118741 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.034413 13923 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.137892 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:02.127107 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.107769 13946 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:02.074944 13861 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:02.132774 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 214166us
I20260709 12:10:02.081637 13934 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.111430 13931 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:02.134589 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:02.135412 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 95887.8us
I20260709 12:10:02.106098 12429 catalog_manager.cc:5697] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 1 to 2, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 2 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:02.110767 13917 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.131319 12643 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:02.135469 13942 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.104988 13932 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:02.146112 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:02.121187 13918 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:02.153573 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:02.133919 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:02.112250 13960 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:02.127115 13954 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.124909 13791 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.007997 13336 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:02.158185 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:02.102936 13905 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:02.069473 13943 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:02.126976 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 197929us
I20260709 12:10:02.133642 13935 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:02.169591 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:02.170210 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:02.142903 13951 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.189236 12529 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 2: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:10:02.189530 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 55704.7us
I20260709 12:10:02.233274 13302 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:02.189994 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.190387 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:02.234917 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 65403.7us
I20260709 12:10:02.188511 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.191382 12599 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:02.192229 13868 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:02.192649 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:02.203678 13914 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:02.156852 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.145097 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:02.162173 13931 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.193709 13960 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.193308 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:02.171419 13932 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:02.165549 13942 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:02.193168 13887 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.254379 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.236325 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:02.237634 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:02.201761 13926 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:02.242317 13917 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.154829 13946 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:02.240716 12603 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:02.241425 13914 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:02.209338 13714 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.215382 13938 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:02.250237 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.251657 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:02.273886 12673 raft_consensus.cc:2376] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2. Re-sending same reply.
I20260709 12:10:02.275106 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.252661 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:02.254351 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.276999 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.236677 13951 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:02.259264 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.233676 13935 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.269949 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.255327 13887 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:02.281914 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:02.284485 12527 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:02.256032 13942 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.270144 13914 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:02.286780 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.269485 13714 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:02.298524 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.254540 13868 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:02.280063 13951 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.301205 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:02.301946 12527 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:02.282972 13938 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.286352 13887 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.310474 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.326314 12679 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 1.
I20260709 12:10:02.339947 13932 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:02.343838 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:02.301230 13714 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.346361 12496 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:02.362479 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:02.363768 12527 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:02.314754 13856 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:02.304948 13938 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:02.366436 13932 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:02.365921 13336 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:02.377985 13938 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:02.406925 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 173777us
W20260709 12:10:02.410053 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 173814us
I20260709 12:10:02.537267 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:02.537478 13302 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:10:02.432857 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 280880us
I20260709 12:10:02.538580 12494 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:10:02.523084 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 167402us
I20260709 12:10:02.293144 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:02.531545 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:02.533557 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:02.539831 12496 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:02.541251 12523 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
W20260709 12:10:02.552106 13895 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.602s	user 0.013s	sys 0.013s
W20260709 12:10:02.551390 13876 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.690s	user 0.014s	sys 0.014s
W20260709 12:10:02.569736 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 260123us
I20260709 12:10:02.552796 13943 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:02.581626 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:02.582670 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:02.585491 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:02.589380 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:02.562851 13963 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.592128 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.590965 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:02.598579 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:02.600399 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:02.576556 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 291507us
W20260709 12:10:02.604305 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 66545.3us
I20260709 12:10:02.606760 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:02.607745 12598 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:02.607125 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:02.610564 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:02.588975 13784 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.585269 13970 raft_consensus.cc:2764] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:10:02.578364 13895 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:02.600286 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:02.581285 13876 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:02.629976 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:02.618489 13802 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.646314 12646 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:02.647480 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:02.626082 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:02.621479 13963 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.652608 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:02.659193 12598 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:10:02.657197 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:02.639837 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 95167.9us
I20260709 12:10:02.628619 13974 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.673934 12496 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:02.680819 13972 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:10:02.663655 13918 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:02.647554 13802 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:02.686978 13802 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:02.688834 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 79813.7us
I20260709 12:10:02.653605 13944 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:02.647722 13880 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.663523 13963 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:02.688364 13974 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:02.709477 13954 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.718868 13944 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:02.732064 13944 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:02.710959 13880 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:02.719991 13963 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.718677 13905 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:02.723893 13969 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:10:02.747519 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 73729.5us
I20260709 12:10:02.748466 12496 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:02.749516 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 103411us
W20260709 12:10:02.710253 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 80144.2us
I20260709 12:10:02.754503 12496 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:02.730446 13974 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.742038 13880 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.757167 12646 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:02.757527 12496 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:02.767403 12496 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:02.769765 12496 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:02.746340 13905 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:02.768627 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:02.741503 13981 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:10:02.784228 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 245719us
W20260709 12:10:02.784559 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 127517us
I20260709 12:10:02.771947 13788 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.791014 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.754426 13856 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.786826 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:02.836333 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49582.3us
I20260709 12:10:02.771531 13964 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.771654 13966 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:10:02.784951 13891 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.660s	user 0.012s	sys 0.013s
I20260709 12:10:02.847661 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:02.843701 13302 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:02.766108 13895 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:02.775862 13905 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:02.762356 13980 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:02.851591 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:02.845718 13920 consensus_meta.cc:287] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.789s	user 0.021s	sys 0.018s
W20260709 12:10:02.870875 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 78662.8us
I20260709 12:10:02.843219 13968 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:02.783466 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:02.894878 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1388) took 1188 ms. Trace:
I20260709 12:10:02.858335 13876 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:02.843274 13975 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:02.905007 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:02.888542 12599 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
W20260709 12:10:02.907574 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 69644us
I20260709 12:10:02.878062 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.911021 12494 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:02.909687 12603 rpcz_store.cc:276] 0709 12:10:01.697796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:01.849481 (+151685us) service_pool.cc:224] Handling call
0709 12:10:01.849780 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:02.264192 (+414412us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:02.886340 (+622148us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b5400374954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:02.886549 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":898512640}
W20260709 12:10:02.883490 13959 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.922s	user 0.016s	sys 0.014s
I20260709 12:10:02.875315 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:02.878764 13968 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:02.888178 13918 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:02.858891 13891 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:02.895488 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:02.921514 13968 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.940760 12598 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
W20260709 12:10:02.924496 13931 consensus_meta.cc:287] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.660s	user 0.009s	sys 0.013s
W20260709 12:10:02.927505 13946 consensus_meta.cc:287] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.664s	user 0.008s	sys 0.004s
W20260709 12:10:02.930088 13887 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.605s	user 0.004s	sys 0.008s
I20260709 12:10:02.894940 13952 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:02.929507 13938 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:02.946208 12673 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
W20260709 12:10:02.947894 13851 consensus_meta.cc:287] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.812s	user 0.012s	sys 0.003s
I20260709 12:10:02.920864 12677 raft_consensus.cc:1813] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Handling vote request from an unknown peer a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:02.883983 13920 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:02.952348 13943 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:02.952653 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:02.938032 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:02.916606 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:02.953723 13956 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:03.003103 13956 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:03.005095 13714 consensus_meta.cc:287] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.648s	user 0.023s	sys 0.000s
W20260709 12:10:03.005450 13960 consensus_meta.cc:287] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.747s	user 0.008s	sys 0.002s
I20260709 12:10:03.006388 13918 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.006875 13918 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:02.954547 13856 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:10:03.012809 13856 raft_consensus.cc:3055] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.013221 13856 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:02.930891 13987 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:03.014420 13987 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:03.014235 13856 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.014916 13987 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:02.944356 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 176504us
I20260709 12:10:02.921126 13975 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:10:02.993678 13923 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.856s	user 0.019s	sys 0.007s
I20260709 12:10:02.962961 13952 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.961874 12523 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:02.962787 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:02.991119 12677 raft_consensus.cc:2410] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 2 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260709 12:10:02.950771 13333 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:02.992744 13851 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:02.997953 13943 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:02.955757 13965 consensus_queue.cc:1059] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:02.913270 13977 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:02.999626 13868 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:03.080518 13331 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:03.080731 13943 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:02.920826 13959 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.005683 13956 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:02.949923 13861 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.783s	user 0.009s	sys 0.002s
I20260709 12:10:03.010392 13918 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.011475 13985 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:10:02.955483 13951 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.642s	user 0.011s	sys 0.000s
I20260709 12:10:03.099321 13951 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:02.991935 13924 consensus_meta.cc:287] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.028s	user 0.008s	sys 0.013s
I20260709 12:10:02.956745 12494 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:02.958617 13934 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:03.005590 13714 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.092150 13965 raft_consensus.cc:3055] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.093895 13943 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:03.102087 13951 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:03.029147 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.110656 12523 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:03.026577 12590 tablet_service.cc:1558] Processing DeleteTablet for tablet 8c7e0eddcc2b4449b7ed8821854646e2 with delete_type TABLET_DATA_TOMBSTONED (TS f5fc807f7ecc4d1ca0fd6be524f734ce not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51774
I20260709 12:10:02.961552 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:03.034566 12606 raft_consensus.cc:2393] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 2: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:03.037705 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.037781 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:03.047367 12592 tablet_service.cc:1558] Processing DeleteTablet for tablet 952cdab9198b44e4bdf1b472109715c3 with delete_type TABLET_DATA_TOMBSTONED (TS f5fc807f7ecc4d1ca0fd6be524f734ce not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:51774
I20260709 12:10:03.047887 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.044096 13975 raft_consensus.cc:697] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:03.049623 13935 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.763s	user 0.012s	sys 0.001s
I20260709 12:10:03.026743 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:03.055627 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:02.957193 13988 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.100s
I20260709 12:10:02.951195 13931 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.058669 13330 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.059033 12643 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:03.058995 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:03.147001 12605 raft_consensus.cc:2393] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 2: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:10:03.148355 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 89425.1us
I20260709 12:10:03.059769 13917 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:03.153115 13917 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.154547 13979 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.062609 13920 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:03.063633 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:03.065348 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:03.054857 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:03.170221 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 115431us
I20260709 12:10:03.066749 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:03.068019 13891 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:03.042901 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.073817 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:03.018741 13856 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.005861 13960 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:02.966053 13978 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:02.958959 13946 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.083552 12569 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:03.186800 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 103336us
I20260709 12:10:03.083734 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:03.188992 13896 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:03.189479 13896 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:03.189879 13896 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:03.084941 12571 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:03.092296 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:03.191742 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 106926us
I20260709 12:10:03.196815 12572 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:03.198379 13932 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.199111 13932 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:03.199506 13932 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:03.199919 13932 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.203181 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:03.205410 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:02.963182 13887 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.098227 13851 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:03.102510 13924 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.208473 12674 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 2: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:03.208562 12531 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:03.209661 13924 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:03.102996 13934 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.210821 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 39) took 1048 ms. Trace:
I20260709 12:10:03.211237 12674 rpcz_store.cc:276] 0709 12:10:02.162288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:02.190148 (+ 27860us) service_pool.cc:224] Handling call
0709 12:10:03.210627 (+1020479us) spinlock_profiling.cc:243] Waited 973 ms on lock 0x7b540015dbd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:03.210764 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2237368064}
I20260709 12:10:03.212020 13934 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:03.103896 13918 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.212448 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:03.214067 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:03.104645 13714 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:03.113556 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 55335.1us
I20260709 12:10:03.113071 13880 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.219010 13304 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:03.220211 13880 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:03.220418 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:03.220919 13880 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:03.221293 13880 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:03.221675 13880 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:03.114799 13942 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.812s	user 0.011s	sys 0.012s
I20260709 12:10:03.222337 13985 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:10:03.115139 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 158504us
I20260709 12:10:03.222786 13985 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:03.223205 13985 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.117645 13980 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.106673 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.119184 13333 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:03.230917 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:03.107167 13965 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:03.098285 13959 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:03.041576 13923 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:03.024060 13914 consensus_meta.cc:287] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.723s	user 0.005s	sys 0.018s
I20260709 12:10:03.239166 12677 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.030948 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:03.123772 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.071209 13974 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.245579 12599 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.133626 13984 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:03.142575 13784 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.142933 13983 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:03.144016 13931 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:03.149720 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:03.152029 13975 consensus_queue.cc:237] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.237970 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:03.258836 13984 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.259483 13984 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:03.260831 13984 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.262986 12679 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
W20260709 12:10:03.263769 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 114113us
I20260709 12:10:03.265264 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:03.150386 13935 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.209877 13887 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:03.174096 13951 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.228703 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:03.232635 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:03.241539 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:03.124010 13861 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.250458 13983 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:03.262449 13335 raft_consensus.cc:2393] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 2: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:03.229892 13938 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:03.244531 13914 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.277064 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:03.184180 13802 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.248365 13784 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:03.222397 13942 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.287731 13861 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:03.281392 13963 consensus_meta.cc:287] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.538s	user 0.006s	sys 0.005s
I20260709 12:10:03.293695 13946 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:03.294023 13963 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.285442 13891 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.295099 13952 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.287761 13914 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:03.281615 13992 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 12:10:03.285239 13984 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.296510 13784 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.246173 13304 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:03.278007 13996 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.278469 13945 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.248972 13978 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.293383 13917 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:03.295475 13960 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:03.290704 13970 raft_consensus.cc:2764] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 12:10:03.289264 13332 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 2: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:03.305784 13334 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.303409 13935 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:03.288755 13987 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.312805 13932 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.280295 13983 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.314651 13983 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:03.284797 13965 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.323176 13965 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.308204 13968 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.308897 13978 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:03.306293 13914 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.310844 13942 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:03.296546 13963 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:03.297777 13952 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:03.325935 13987 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:03.327764 13968 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:03.329538 13932 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:03.333102 13985 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.337056 13998 tablet_replica.cc:333] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:10:03.096042 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:10:03.363803 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 128426us
I20260709 12:10:03.130373 12496 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:03.390712 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 112795us
W20260709 12:10:03.139721 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:03.393240 12643 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:03.394711 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:03.396361 12643 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:03.365828 12527 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.366250 12524 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
W20260709 12:10:03.403455 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 273410us
I20260709 12:10:03.252290 12569 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:03.405792 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 153496us
I20260709 12:10:03.226325 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:03.412741 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 186342us
I20260709 12:10:03.227885 12572 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:03.414781 13960 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:03.415948 12679 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.415817 13968 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.416265 13951 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.251879 12597 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:10:03.417055 13968 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:10:03.300595 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 71302.4us
I20260709 12:10:03.387853 13896 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.419773 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:03.420192 13975 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.379601 12672 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.421381 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:03.305244 13784 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.424722 13914 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:03.425401 13914 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:03.425591 13918 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:03.416976 13946 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:03.238873 12605 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:10:03.426884 13946 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:03.427657 13946 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.381875 12676 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.426327 13918 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:03.428356 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:03.429327 13918 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.419848 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:03.430078 13951 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:03.415841 13960 raft_consensus.cc:697] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:03.430619 13951 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:03.431157 13951 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:03.416432 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 188723us
I20260709 12:10:03.432960 13960 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.435667 12572 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:03.437532 13922 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:03.438102 13922 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:03.438668 13922 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.425942 13914 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.446928 13896 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:03.456146 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:03.457114 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:03.457261 13923 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:03.467628 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:03.468525 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:03.470947 12604 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.461315 13880 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.472292 13974 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:03.465518 12572 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:03.478163 12606 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
W20260709 12:10:03.479179 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:03.487092 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58960.1us
I20260709 12:10:03.490111 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 28) took 1451 ms. Trace:
I20260709 12:10:03.489967 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:03.491124 12598 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:03.490706 12597 rpcz_store.cc:276] 0709 12:10:02.038498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:02.038779 (+   281us) service_pool.cc:224] Handling call
0709 12:10:02.039293 (+   514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:03.417908 (+1378615us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:03.489563 (+ 71655us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54001167d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:03.489905 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2787915776}
I20260709 12:10:03.493862 13802 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:03.496075 12526 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
W20260709 12:10:03.503142 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:03.504273 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:03.513000 13334 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.513475 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:03.523276 13880 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:03.526822 13985 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:03.530308 13932 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:03.547045 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:03.532902 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 67722.5us
I20260709 12:10:03.535739 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.536338 13334 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
W20260709 12:10:03.544575 13905 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.669s	user 0.005s	sys 0.013s
I20260709 12:10:03.541814 12598 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
W20260709 12:10:03.531031 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:03.545238 12494 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:03.546690 13932 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:03.530975 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:03.537052 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.565236 13704 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:10:03.566944 13704 raft_consensus.cc:3055] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.567487 13704 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:03.568833 13704 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.569415 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:03.570490 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 174548us
I20260709 12:10:03.570725 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.571008 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:03.571713 12643 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:03.572907 13704 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.573634 13932 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:03.570524 12597 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:03.571916 12606 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.577093 12671 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:03.573650 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:03.585558 12602 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.587373 13791 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.589246 13975 consensus_queue.cc:1059] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:03.592628 13963 consensus_queue.cc:1059] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:10:03.595439 13304 leader_election.cc:336] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.606s (SENT)
I20260709 12:10:03.590533 13924 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:10:03.597223 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:03.580556 13896 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:10:03.601528 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 56445.8us
I20260709 12:10:03.605523 13932 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:03.607245 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.604746 13896 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:03.611778 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:03.599325 13988 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.596421 13918 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.614902 12496 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:03.560367 13905 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.603114 12604 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.625181 13968 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:10:03.624694 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 45910.4us
I20260709 12:10:03.610015 13932 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.614609 13896 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:03.629658 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:03.625561 12601 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.616230 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:03.637799 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:03.632910 12569 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:03.634284 12494 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:03.625690 13968 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:03.629227 13932 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:03.628096 13963 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:03.634274 13918 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:03.648717 13965 raft_consensus.cc:3055] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.649282 13965 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:03.650707 13965 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.652691 13965 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.640731 13968 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.644240 13980 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:03.658085 12525 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
I20260709 12:10:03.660476 13905 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:03.645238 13963 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:03.662225 13963 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.663909 13951 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.665503 12599 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.665668 13932 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.667665 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:03.668113 13980 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.670370 13905 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.670640 13920 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.668497 13951 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:03.655831 13906 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.669840 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:03.675122 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:03.663097 13922 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.654412 13784 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.682693 13784 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:03.683725 13784 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.684916 13784 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.682720 13880 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.681905 13922 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:03.644265 13975 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:03.687309 13856 raft_consensus.cc:3055] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.708312 13856 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:03.689929 13975 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.712622 12429 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47524 (request call id 90) took 1777 ms. Trace:
I20260709 12:10:03.713208 13975 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:03.709883 13856 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.693234 13946 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.716895 13896 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.717100 12427 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58992 (request call id 167) took 1792 ms. Trace:
I20260709 12:10:03.703267 13932 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.723179 12427 rpcz_store.cc:276] 0709 12:10:01.924486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:01.924804 (+   318us) service_pool.cc:224] Handling call
0709 12:10:03.716844 (+1792040us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68375,"reported_tablets":5,"spinlock_wait_cycles":35584}
I20260709 12:10:03.704939 13980 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:03.705422 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 110055us
I20260709 12:10:03.726524 13984 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:10:03.726975 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 52028.5us
I20260709 12:10:03.728132 13905 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:03.729173 13791 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:03.729970 13791 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:03.730665 13941 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:03.731539 13791 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.732447 13923 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:03.741662 13923 raft_consensus.cc:3055] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:03.741768 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:03.742331 13923 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:03.743041 12527 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:10:03.743878 13923 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.745743 13923 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:03.746667 12527 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:03.750347 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.750859 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.751541 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:03.752545 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.754208 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.755447 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.757395 13856 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.758831 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.762063 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.762431 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:03.732743 13922 consensus_queue.cc:1059] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:03.733352 13984 consensus_queue.cc:1059] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:03.733913 13791 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:03.764770 13984 consensus_queue.cc:1243] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:10:03.766335 13951 raft_consensus.cc:3055] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.767091 13951 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:03.733352 13924 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:03.768576 13951 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.769448 13924 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.770608 13951 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.772446 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.773100 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.774271 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.775454 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.776346 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.777884 13802 raft_consensus.cc:3055] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.778283 12526 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:03.739521 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.780372 13924 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:03.773758 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:03.739989 13905 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:03.762933 12530 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.783594 13996 raft_consensus.cc:3055] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.783991 13996 raft_consensus.cc:740] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:03.785064 13996 consensus_queue.cc:260] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:03.721765 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 53352.8us
I20260709 12:10:03.788087 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:03.778330 13802 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:03.789206 13334 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 2: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:03.790369 13802 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.796239 13802 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.794960 13996 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.765935 13968 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.804966 13968 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:03.738509 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 68895.9us
I20260709 12:10:03.808868 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:03.811298 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:03.812836 13791 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.813665 13791 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.815234 13922 consensus_queue.cc:1059] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:10:03.816655 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 212192us
I20260709 12:10:03.817466 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:03.818372 13784 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.822052 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:03.822985 13922 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.823583 13920 raft_consensus.cc:3055] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.824093 13920 raft_consensus.cc:740] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:03.824331 12571 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:03.825233 13922 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.825345 13920 consensus_queue.cc:260] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.826558 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:03.827606 13922 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.717960 13946 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:03.829142 12571 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:03.831990 13946 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:03.833621 13946 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:03.835042 13922 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.835467 13922 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:03.836426 13922 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.827800 13920 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.722541 12606 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:03.840148 12525 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:03.842386 12530 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:03.843309 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.843700 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:03.839921 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 193293us
I20260709 12:10:03.846590 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.856742 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.857944 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.858997 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:03.859408 12676 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
W20260709 12:10:03.860884 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 43518.5us
I20260709 12:10:03.862388 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:03.853164 13922 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.879699 13704 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:10:03.886950 12605 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:03.890535 12599 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.875972 13943 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.895756 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:03.896886 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:03.897621 12672 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:03.898943 13896 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:03.909065 12522 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:03.892632 13988 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.910563 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.912063 13704 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:03.907033 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:03.906916 13305 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:03.912930 12679 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:03.913271 12606 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.915272 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:03.915310 12599 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:03.917608 13704 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:03.921165 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:03.924217 12599 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:03.924990 12494 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:03.910617 13988 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:03.713217 12429 rpcz_store.cc:276] 0709 12:10:01.935402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:01.936563 (+  1161us) service_pool.cc:224] Handling call
0709 12:10:01.942901 (+  6338us) spinlock_profiling.cc:243] Waited 1.47 ms on lock 0x7b54000104d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148898a090 00007f1488a28f3a 00007f148e7dee9c 00007f148e8dd04e 00007f148cd9e2a4 00007f148cd9e232 00007f148cd9e161 00007f148cd9e0dc 00007f148cd9d392 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:03.712327 (+1769426us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:02.600033 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:02.738006 (+137973us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:02.742942 (+  4936us) write_op.cc:435] Acquired schema lock
0709 12:10:02.744818 (+  1876us) tablet.cc:662] Decoding operations
0709 12:10:02.857660 (+112842us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:02.868660 (+ 11000us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:02.876733 (+  8073us) tablet.cc:685] Acquiring locks for 3 operations
0709 12:10:02.922257 (+ 45524us) tablet.cc:701] Row locks acquired
0709 12:10:02.930672 (+  8415us) write_op.cc:260] PREPARE: finished
0709 12:10:02.948879 (+ 18207us) write_op.cc:270] Start()
0709 12:10:02.949044 (+   165us) write_op.cc:276] Timestamp: P: 1783599002948804 usec, L: 0
0709 12:10:02.949070 (+    26us) op_driver.cc:348] REPLICATION: starting
0709 12:10:02.952540 (+  3470us) log.cc:844] Serialized 1301 byte log entry
0709 12:10:03.289029 (+336489us) op_driver.cc:464] REPLICATION: finished
0709 12:10:03.337189 (+ 48160us) write_op.cc:301] APPLY: starting
0709 12:10:03.337276 (+    87us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:03.337414 (+   138us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:03.337541 (+   127us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:03.337921 (+   380us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:03.388306 (+ 50385us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=3
0709 12:10:03.388414 (+   108us) write_op.cc:312] APPLY: finished
0709 12:10:03.483288 (+ 94874us) log.cc:844] Serialized 34 byte log entry
0709 12:10:03.509737 (+ 26449us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:03.509987 (+   250us) write_op.cc:454] Released schema lock
0709 12:10:03.510069 (+    82us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:03.617423 (+107354us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:03.617478 (+    55us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":657,"reported_tablets":6,"spinlock_wait_cycles":3387776,"child_traces":[["op",{"apply.queue_time_us":47911,"apply.run_cpu_time_us":4412,"apply.run_wall_time_us":280052,"num_ops":3,"prepare.queue_time_us":366135,"prepare.run_cpu_time_us":33953,"prepare.run_wall_time_us":507987,"raft.queue_time_us":133055,"raft.run_cpu_time_us":36486,"raft.run_wall_time_us":46114,"replication_time_us":339780,"thread_start_us":369336,"threads_started":4,"wal-append.queue_time_us":183210}]]}
I20260709 12:10:03.930706 13987 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:03.931411 13987 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 2 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:03.932629 13987 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.940755 12572 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:03.954643 13704 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.944443 13963 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:03.957749 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:03.945448 12519 tablet_service.cc:1558] Processing DeleteTablet for tablet 816b6f34d124434882148e507cf5b612 with delete_type TABLET_DATA_TOMBSTONED (TS a2fdbe7eef6748a9b4611f6c9bc9b201 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:53846
W20260709 12:10:03.962396 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:03.963660 14005 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:03.959405 13946 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:03.951530 13880 consensus_queue.cc:1059] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.015s
I20260709 12:10:03.966356 13924 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.966851 13880 raft_consensus.cc:3055] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:03.966820 13924 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:03.967300 13880 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:03.968462 13924 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.969677 12646 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:10:03.970649 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 57107us
I20260709 12:10:03.968462 13880 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.971942 13791 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:03.972378 13791 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:03.974260 12572 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:03.972780 13791 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.978430 13965 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:03.979564 13965 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:03.979929 13965 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:03.970085 13924 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:03.975533 13988 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.983572 13965 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:03.985769 12646 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:03.986753 13968 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:03.987159 13905 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:03.988005 13905 raft_consensus.cc:697] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:03.989616 13905 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:03.996454 13965 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:03.996764 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:03.999533 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:03.976056 13914 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.532s	user 0.013s	sys 0.000s
W20260709 12:10:04.005743 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52250.4us
W20260709 12:10:04.025696 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 51542.7us
I20260709 12:10:04.028162 12572 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:04.029309 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52653.4us
I20260709 12:10:04.035701 13896 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:04.039338 13334 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:10:04.039410 12572 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:04.041443 12494 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:04.047106 13951 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:04.047724 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:04.048367 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:04.049257 12606 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:04.050426 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:04.051355 12676 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 2: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:04.053922 12643 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:04.024673 13914 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:04.029817 13880 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:04.050190 13791 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:04.054965 14005 consensus_queue.cc:1059] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:04.060918 12606 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
I20260709 12:10:04.031909 13941 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:10:04.082222 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.082970 13923 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:04.073864 12679 raft_consensus.cc:2410] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 12:10:04.085716 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 44358us
I20260709 12:10:04.070794 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:04.045979 13918 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:04.075153 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:04.075819 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:04.076396 13791 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:04.076812 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:04.077109 14008 tablet_replica.cc:333] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:10:04.077665 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:04.078219 13332 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.080845 13906 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:04.106248 12599 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 2: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:04.087286 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:10:04.091264 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 69190.4us
I20260709 12:10:04.098291 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:04.099180 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:04.098291 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:04.103818 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49976.5us
I20260709 12:10:04.058285 13932 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:10:04.113022 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:04.115497 13906 raft_consensus.cc:697] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:04.111370 12678 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:10:04.124215 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 53536.1us
I20260709 12:10:04.125655 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:04.126844 13941 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:04.130070 12572 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:04.134171 13914 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:04.106951 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:04.135051 14010 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:10:04.136214 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49013.1us
I20260709 12:10:04.137130 12571 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:04.138032 13941 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:04.138541 12571 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:04.116811 13906 consensus_queue.cc:237] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:04.129891 13932 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:04.121532 13918 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } attrs { replace: true } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:04.145714 13932 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:04.145943 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:04.146847 13941 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:04.146307 13932 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:04.148597 13302 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:04.149534 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.149405 13998 raft_consensus.cc:2243] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:10:04.123318 13920 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:04.150813 13918 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:04.154093 12672 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.156024 13920 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:04.157246 13923 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:04.158619 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:04.158782 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:10:04.161808 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.161973 13985 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:04.164028 12572 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:04.168601 12528 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:04.173538 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:04.177754 12679 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:04.178198 12572 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:04.159698 13965 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:04.177552 12571 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:04.184329 12646 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:04.185583 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:04.186309 13951 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:04.188026 13984 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:04.188721 13968 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:04.189327 13968 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:04.167670 13802 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:04.194494 12598 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
I20260709 12:10:04.195026 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:04.167927 13914 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:04.170481 13941 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:04.204902 14008 raft_consensus.cc:2243] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:10:04.176510 13959 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:04.182173 13918 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:04.188722 13984 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:04.225288 13932 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:04.230222 13906 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:04.198987 13802 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:04.206363 13951 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:10:04.263893 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 177372us
I20260709 12:10:04.232403 12604 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
I20260709 12:10:04.199457 13965 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:04.219442 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:04.279634 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:04.232031 14011 consensus_queue.cc:1059] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260709 12:10:04.190514 13968 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:04.293071 12598 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:04.292650 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:04.300004 14009 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:10:04.299506 13802 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:04.300938 13802 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:04.301426 14008 raft_consensus.cc:2272] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:10:04.232695 13791 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 12:10:04.303280 13856 raft_consensus.cc:3055] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:04.303589 12605 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:04.303797 13856 raft_consensus.cc:740] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:04.287644 13918 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:04.305655 14008 ts_tablet_manager.cc:1916] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:10:04.280930 13988 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:04.305621 13335 raft_consensus.cc:1813] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Handling vote request from an unknown peer f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:04.305217 13856 consensus_queue.cc:260] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:04.316465 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 112641us
W20260709 12:10:04.319490 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 63487.5us
I20260709 12:10:04.320633 13330 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.305444 13918 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:04.325562 13918 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:04.315582 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:04.325681 12646 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:04.326673 12572 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:10:04.316884 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 84690.6us
W20260709 12:10:04.327419 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:04.327766 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.278110 13975 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:10:04.297086 13985 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 3 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:04.305438 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:04.343174 13984 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:04.343936 13336 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 2: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:04.347369 14012 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:04.347836 14012 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:04.348671 14012 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:04.349501 13335 raft_consensus.cc:2410] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 2 because replica has last-logged OpId of term: 1 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
W20260709 12:10:04.337531 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 138036us
I20260709 12:10:04.343212 12678 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:04.368265 13704 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:04.368784 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:04.369256 13932 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:04.369728 13330 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 2: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:04.331657 12601 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:04.372881 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:04.373126 12531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:10:04.373864 12531 raft_consensus.cc:606] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:04.367017 12429 catalog_manager.cc:5697] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 1 to 2, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 2 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:04.375176 12531 raft_consensus.cc:614] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:04.379168 12427 catalog_manager.cc:5697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 1 to 2, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 2 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:04.381978 12531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "e56d487365394ac5ac5f1002926a5645"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:10:04.382689 12531 raft_consensus.cc:606] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Received request to transfer leadership to TS e56d487365394ac5ac5f1002926a5645
I20260709 12:10:04.388180 12571 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:04.392020 13956 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260709 12:10:04.394987 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:04.353382 13918 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:10:04.389463 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 49846.4us
I20260709 12:10:04.392467 12678 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.392055 14023 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:04.390101 14012 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:04.402505 13956 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:04.404549 13918 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:04.405752 13975 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:04.395040 13965 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:04.410084 12671 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "e56d487365394ac5ac5f1002926a5645"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:10:04.399400 14011 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:10:04.414320 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:04.414001 13975 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:04.414881 14023 raft_consensus.cc:697] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:04.406502 13963 consensus_queue.cc:1059] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:10:04.416455 14023 consensus_queue.cc:237] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:04.417207 13975 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:04.417815 13941 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:04.418358 13918 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:04.411195 12671 raft_consensus.cc:606] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Received request to transfer leadership to TS e56d487365394ac5ac5f1002926a5645
I20260709 12:10:04.422338 12671 raft_consensus.cc:614] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:04.424863 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:04.425631 12678 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:10:04.425850 14020 raft_consensus.cc:2706] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Received pre-election callback for term 2 while not running: Illegal state: RaftConsensus is not running: State = Stopping
I20260709 12:10:04.427232 12496 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:04.430833 13302 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:04.431684 13987 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:04.431854 13302 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:04.409102 13956 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:04.433614 12572 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:04.432550 13704 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:04.443435 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:04.445465 13704 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:04.445948 13704 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:04.446287 13704 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:04.447328 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:04.448652 12494 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:04.452275 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:04.429690 13932 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:10:04.432209 13987 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 2 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:04.449635 13880 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:04.456648 13920 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:10:04.481662 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.462787 13906 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:04.467658 13984 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:04.462534 14025 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:04.466377 13963 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 12:10:04.490761 13956 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:04.451470 13985 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:04.499854 13791 raft_consensus.cc:3055] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:04.502501 12599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:32832
I20260709 12:10:04.502951 13333 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.484962 13906 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:04.506042 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:10:04.506695 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.510900 12494 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:04.523931 13880 raft_consensus.cc:697] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 2 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:04.531394 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:04.532147 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:04.525665 12599 raft_consensus.cc:606] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:04.504583 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:04.491657 12601 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:04.534806 12523 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:10:04.535738 12674 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:04.537145 12605 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:04.545795 12602 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.550699 12604 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:04.551563 14017 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.552359 14017 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:04.543002 14025 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:04.525309 13791 raft_consensus.cc:740] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:04.557179 13791 consensus_queue.cc:260] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:04.558561 13791 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:04.534194 13922 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:04.559962 13922 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:04.562453 13987 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:04.527247 13965 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:04.555006 13956 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:04.564003 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 58061.2us
I20260709 12:10:04.568571 13704 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:04.574213 12531 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.576378 12604 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:04.542454 13975 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:04.597882 14010 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:04.598416 14010 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:04.586997 14021 raft_consensus.cc:3055] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:04.586992 13906 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:04.592559 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:04.592794 13914 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:04.594890 12601 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:10:04.594688 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:04.596961 14005 raft_consensus.cc:3055] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:04.564638 14017 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:04.617785 14021 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:04.629074 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:10:04.629506 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:04.648376 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 137532us
I20260709 12:10:04.649801 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:04.651315 12494 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:04.651201 12671 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.660904 12599 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
I20260709 12:10:04.584233 13922 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:04.672201 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 79743.5us
I20260709 12:10:04.677464 13302 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:04.679188 12599 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:04.680886 12599 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.681562 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:04.683576 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 55150.9us
I20260709 12:10:04.683954 14048 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:04.685554 14048 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:04.691891 14024 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:04.692955 14024 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:04.693513 14024 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:04.694424 14036 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.695029 14036 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:04.695622 14013 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:04.696532 14013 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:04.697603 14036 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:04.699249 14013 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:04.685963 14048 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:04.636968 14005 raft_consensus.cc:740] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:04.711684 12674 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:04.781719 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:04.673096 14025 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:04.674574 13980 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:04.785234 12523 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:10:04.787415 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:04.599519 14010 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:04.790395 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 80955.9us
I20260709 12:10:04.713757 13906 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:04.802448 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.782387 14048 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:04.790026 13943 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.703653 13963 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.098s
I20260709 12:10:04.831136 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:04.800670 13946 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.034s
I20260709 12:10:04.815349 13998 raft_consensus.cc:2272] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:10:04.832314 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:04.857600 14040 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:10:04.858280 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:04.858544 12680 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "e56d487365394ac5ac5f1002926a5645"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:10:04.859163 14039 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:04.855365 14037 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:04.856773 14044 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:04.838277 13980 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:04.800704 13985 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260709 12:10:04.835047 14036 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:04.881820 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.845942 12671 raft_consensus.cc:2393] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 2: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:04.786449 13880 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:04.868395 12680 raft_consensus.cc:606] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Received request to transfer leadership to TS e56d487365394ac5ac5f1002926a5645
I20260709 12:10:04.881482 14013 raft_consensus.cc:3055] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:04.884589 14036 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:04.884630 14013 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:04.887066 14036 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:04.886686 14013 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:04.882937 12601 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
I20260709 12:10:04.890652 13334 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:04.891156 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:04.893126 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:04.928146 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:04.929456 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:04.929759 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:04.930353 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 117129us
I20260709 12:10:04.938330 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:04.841904 13943 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:04.940814 12605 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:04.943492 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:04.942184 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:04.945555 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:04.948895 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:04.918826 13946 raft_consensus.cc:3055] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:04.940871 13332 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 2: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:04.940608 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:04.959497 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:04.840716 14005 consensus_queue.cc:260] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:04.883194 13980 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:04.968650 12602 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 2: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:04.938081 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:04.976832 12675 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:10:04.977406 12675 pending_rounds.cc:85] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 2
I20260709 12:10:04.977751 12675 pending_rounds.cc:107] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:10:04.978061 12675 raft_consensus.cc:2889] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:04.979718 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:04.981099 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:04.982404 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:04.982791 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:04.983569 12680 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:04.983935 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:04.987884 13330 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:10:04.955106 13946 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:04.992965 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:04.993825 12602 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:04.988225 14052 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:05.010080 12598 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:04.919093 14039 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.013509 12606 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:05.015496 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:05.016940 13943 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:05.017738 14052 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:05.018368 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:05.016975 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:05.019662 12676 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:05.020638 14052 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:05.024255 12671 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
I20260709 12:10:05.028806 13998 ts_tablet_manager.cc:1916] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:10:05.024885 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:05.034291 14049 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.091s	user 0.002s	sys 0.000s
I20260709 12:10:05.034886 12676 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
I20260709 12:10:05.035614 13304 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:05.038645 14052 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:10:05.038957 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 45034.6us
I20260709 12:10:05.040128 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:05.041154 14052 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:05.040306 12676 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:05.037791 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:05.012996 14025 consensus_queue.cc:1059] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:10:05.046015 13704 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:05.059973 14039 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:05.061147 12673 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:05.046506 12674 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:05.064291 13965 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:05.065251 13965 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:05.065439 12678 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e15d21db797e4a96b5732711a5379980"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "e56d487365394ac5ac5f1002926a5645"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:10:05.066244 12678 raft_consensus.cc:606] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Received request to transfer leadership to TS e56d487365394ac5ac5f1002926a5645
I20260709 12:10:05.066623 12678 raft_consensus.cc:614] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:10:05.069806 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 354746us
W20260709 12:10:05.079069 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 75540.9us
I20260709 12:10:05.083793 12674 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
I20260709 12:10:05.041713 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:05.014062 14005 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:05.055073 13988 raft_consensus.cc:3055] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:05.092751 13906 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:05.093641 14050 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:05.054368 14025 raft_consensus.cc:3055] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:05.101006 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:10:05.121210 12602 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:05.106549 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:05.123948 12604 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 2: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:10:05.107868 14026 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.630s	user 0.001s	sys 0.003s
I20260709 12:10:05.115527 12679 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:05.116228 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:05.117830 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:05.094916 14039 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:10:05.119041 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 161688us
I20260709 12:10:05.092275 12680 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
I20260709 12:10:05.130416 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:05.055351 13946 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.134416 13791 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:05.135574 13946 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:05.130870 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:05.099541 13988 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:05.142788 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:05.144165 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 43698.1us
I20260709 12:10:05.152065 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:05.152902 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:05.145027 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:05.156394 13302 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:05.159238 12679 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:05.120776 14052 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:05.091662 14024 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:05.119184 13968 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:05.121011 14025 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:05.172819 12602 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:05.136032 13920 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:05.175841 12599 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:05.175072 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:05.178613 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:05.171195 13914 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:05.180184 13332 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:05.193914 12676 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:05.171474 13924 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:05.181849 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:05.183452 13980 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:05.130926 14048 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:05.195938 12603 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:05.196292 13988 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:05.196911 14017 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:05.197333 14017 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:05.170761 13932 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:05.198251 13932 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:05.198536 14017 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:05.199290 13932 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:05.199951 14017 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:05.187928 12601 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:05.188450 13963 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:05.137984 14013 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:05.190655 12598 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
I20260709 12:10:05.191782 14025 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:05.192983 13965 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.197422 14048 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:05.187115 14037 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:05.203059 12599 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:05.205241 13302 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:05.144299 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 102718us
I20260709 12:10:05.207310 12602 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:05.208997 12528 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
I20260709 12:10:05.194624 13924 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:05.189705 13920 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:05.193750 13914 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:05.238790 12496 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:05.248456 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:05.248728 12606 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:10:05.259332 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:05.271490 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:05.262586 12597 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:05.267900 12526 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:05.279220 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:05.209980 13987 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:10:05.251849 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:05.237483 14044 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.038s
I20260709 12:10:05.201989 13968 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.248714 13880 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:05.263136 13914 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:05.292814 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54238.8us
I20260709 12:10:05.313946 14057 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:05.314204 14051 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:05.242343 13943 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.042s
I20260709 12:10:05.307535 14025 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:05.339708 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 88081.6us
I20260709 12:10:05.330272 14065 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.265913 13906 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.072s
I20260709 12:10:05.342422 13924 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:05.324973 13920 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.356297 12643 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:05.359365 13923 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:05.361985 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:05.363018 13332 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:10:05.363953 13965 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:05.330827 14011 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:05.365547 12494 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:05.266803 14020 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.074s
I20260709 12:10:05.372555 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:05.356179 14057 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.370416 14065 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.373558 13924 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:10:05.369275 14012 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:05.377951 12678 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 2. Current term is 3. Ops: []
W20260709 12:10:05.379001 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 130648us
I20260709 12:10:05.380174 13941 raft_consensus.cc:3055] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:05.380602 13941 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:05.380642 14036 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:05.377995 14065 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:05.381182 14036 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:05.381757 13965 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:05.381596 14048 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:05.382438 13965 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:05.382876 14008 ts_tablet_manager.cc:1929] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:10:05.383841 14008 log.cc:1199] T 816b6f34d124434882148e507cf5b612 P a2fdbe7eef6748a9b4611f6c9bc9b201: Deleting WAL directory at /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-0-root/wals/816b6f34d124434882148e507cf5b612
I20260709 12:10:05.384342 14048 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.385730 14048 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:05.386497 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:05.390491 12603 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
W20260709 12:10:05.392696 14033 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.589s	user 0.003s	sys 0.000s
I20260709 12:10:05.382910 13965 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:05.395452 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:05.384675 13968 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:05.286247 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:05.404961 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:05.407649 14011 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:05.409590 12643 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:05.410801 12606 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
I20260709 12:10:05.413808 12494 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:05.414446 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:05.414987 12606 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:05.415798 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:05.416334 12606 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:05.416726 13302 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:05.411922 12602 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:05.373946 13856 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:05.400707 14036 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:05.421841 12494 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:05.423830 14005 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:05.378001 14057 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:05.423363 14069 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.381616 13941 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.433494 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:05.424046 13968 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:05.395922 14008 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:53846 (request call id 76) took 1475 ms. Trace:
I20260709 12:10:05.369552 14066 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:05.424465 14011 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:05.434455 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:05.430486 14048 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:05.442600 14058 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:05.439323 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:05.430929 13987 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:05.443403 13987 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:05.437322 14065 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260709 12:10:05.437458 13791 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:05.432493 13985 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:05.434064 13923 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:05.449889 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 161302us
I20260709 12:10:05.454772 14011 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.459460 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:05.470801 12671 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:10:05.443843 13987 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:05.437196 14059 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:05.478504 14059 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:05.437783 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 168105us
I20260709 12:10:05.473204 14066 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:05.482318 13304 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:05.484313 13880 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:05.484393 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:05.435703 14012 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:10:05.485286 13880 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:05.485999 14012 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:05.486498 14012 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:05.487566 14012 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.467288 12602 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:05.491175 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:05.491991 12599 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:05.468005 14062 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:10:05.462939 13923 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.478912 14040 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:05.482555 14066 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:05.450695 13968 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:10:05.513502 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:05.497653 12530 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:10:05.500260 14055 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.268s	user 0.001s	sys 0.000s
I20260709 12:10:05.500790 12522 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:10:05.510346 14063 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.186s	user 0.002s	sys 0.000s
I20260709 12:10:05.511332 14070 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.439716 14069 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:05.513075 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 63924us
I20260709 12:10:05.513269 12524 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:10:05.522928 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:05.469295 12494 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:05.513935 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:05.524662 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 55457.1us
I20260709 12:10:05.504417 14012 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:05.526185 12494 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:05.527521 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:05.532902 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:05.533618 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:05.534382 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:05.535899 14019 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:05.538813 13985 raft_consensus.cc:2764] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won
I20260709 12:10:05.538350 12572 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:05.505080 14040 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:05.514400 13998 ts_tablet_manager.cc:1929] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:10:05.498553 14059 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:05.512588 13856 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:05.513190 14066 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:05.553123 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:05.554188 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:05.543357 13906 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:05.588791 12676 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:10:05.550634 13784 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:05.590219 12679 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:05.591040 12530 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:05.590327 13998 log.cc:1199] T 8c7e0eddcc2b4449b7ed8821854646e2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Deleting WAL directory at /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-1-root/wals/8c7e0eddcc2b4449b7ed8821854646e2
I20260709 12:10:05.587841 13980 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:05.603166 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:05.524049 14070 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:05.606261 12605 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 2: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:05.599048 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:05.589774 13906 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:05.438890 14008 rpcz_store.cc:276] 0709 12:10:03.928231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:03.945074 (+ 16843us) service_pool.cc:224] Handling call
0709 12:10:04.076515 (+131441us) ts_tablet_manager.cc:1151] Deleting tablet 816b6f34d124434882148e507cf5b612
0709 12:10:04.076613 (+    98us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:10:05.050990 (+974377us) tablet_metadata.cc:665] Metadata flushed
0709 12:10:05.382726 (+331736us) tablet_metadata.cc:665] Metadata flushed
0709 12:10:05.385219 (+  2493us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":99887,"tablet-delete.queue_time_us":113362,"thread_start_us":63443,"threads_started":1}
I20260709 12:10:05.610416 12605 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:05.599521 14005 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:05.615162 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:05.625944 14075 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.626688 14075 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.627177 14070 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:05.603237 13784 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:05.595786 13856 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.634217 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:05.613514 13906 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:05.638842 12605 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
W20260709 12:10:05.641366 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:05.646927 12494 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:05.653486 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:05.629706 14075 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:05.615731 13980 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:05.664966 14081 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:10:05.636845 14005 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:05.638029 14030 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.686961 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:05.688057 12605 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:05.654031 14070 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:05.662755 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:05.681137 14078 raft_consensus.cc:993] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: : Instructing follower e56d487365394ac5ac5f1002926a5645 to start an election
I20260709 12:10:05.694286 13784 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.695351 14078 raft_consensus.cc:1081] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Signalling peer e56d487365394ac5ac5f1002926a5645 to start an election
I20260709 12:10:05.696925 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:05.679350 13923 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:05.681883 13856 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:05.682511 14081 raft_consensus.cc:3055] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:05.644714 13905 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.019s
I20260709 12:10:05.690320 12599 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
I20260709 12:10:05.698226 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:05.701792 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:05.704051 12600 raft_consensus.cc:2393] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 2: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:05.631729 14040 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.698930 14070 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:05.682533 13980 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:05.713937 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:05.703837 14010 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:05.708120 14078 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:10:05.710031 13856 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:05.719000 12599 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:05.708711 13998 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51774 (request call id 67) took 2682 ms. Trace:
I20260709 12:10:05.717902 14079 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.736826 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:05.720364 14077 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
I20260709 12:10:05.701159 14081 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:05.683357 13914 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:05.708236 13946 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.734459 12494 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:05.736395 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 194724us
I20260709 12:10:05.719887 14070 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:05.763417 12675 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:05.719344 14058 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:05.700821 13965 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:05.765309 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:05.755254 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:05.772909 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:05.825520 12678 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:05.698231 14030 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.828183 12600 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:05.829483 13333 raft_consensus.cc:2368] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4.
I20260709 12:10:05.829255 13336 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9"
dest_uuid: "e56d487365394ac5ac5f1002926a5645"
 from {username='slave'} at 127.0.0.1:38482
I20260709 12:10:05.764700 13943 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.735157 13998 rpcz_store.cc:276] 0709 12:10:03.023036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:03.026234 (+  3198us) service_pool.cc:224] Handling call
0709 12:10:03.336682 (+310448us) ts_tablet_manager.cc:1151] Deleting tablet 8c7e0eddcc2b4449b7ed8821854646e2
0709 12:10:03.336754 (+    72us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:10:04.585693 (+1248939us) spinlock_profiling.cc:243] Waited 47.4 ms on lock 0x7b540007ab54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a28362 00007f148c41334c 00007f148da3660a 00007f148da672d0 00007f148da4d83d 00007f148da4d7e9 00007f148da4d779 00007f148da4d741 00007f148da4c96d 0000000000388634
0709 12:10:05.062092 (+476399us) tablet_metadata.cc:665] Metadata flushed
0709 12:10:05.502499 (+440407us) tablet_metadata.cc:665] Metadata flushed
0709 12:10:05.690212 (+187713us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":731507,"spinlock_wait_cycles":109093632,"tablet-delete.queue_time_us":205400,"thread_start_us":226230,"threads_started":1}
I20260709 12:10:05.764433 13334 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:05.770946 13946 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.773602 14069 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:05.821125 14075 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:05.835016 13965 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:05.835697 12673 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:10:05.836620 13704 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:05.839927 14024 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:05.840468 12523 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:05.840727 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:05.864950 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 166986us
I20260709 12:10:05.770378 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:05.826069 14079 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.902947 12677 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
W20260709 12:10:05.867272 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 102100us
I20260709 12:10:05.905292 14079 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:05.906775 13802 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:05.908561 13802 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:10:05.765491 13980 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.821154 13932 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.898177 13704 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.858659 14052 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:05.898295 14085 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:05.876399 13336 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:10:05.898738 13922 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.875708 13906 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.894960 14075 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:05.903350 14069 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:05.952010 12599 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:05.897442 13880 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.944664 12671 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:10:05.933176 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:05.877545 13943 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.919863 14053 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.939728 13336 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:05.913645 14025 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.968631 13704 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:05.973076 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:05.949182 14075 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:05.973160 14074 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.412s	user 0.003s	sys 0.000s
I20260709 12:10:05.992105 12678 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
I20260709 12:10:05.964269 14079 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:05.975739 14017 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:10:05.963546 13980 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:05.992734 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:05.950897 13984 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 12:10:05.981734 14044 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:05.944947 14052 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:05.973676 14080 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260709 12:10:05.962550 13932 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.969345 14069 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:05.969417 13922 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.009228 14021 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:05.965798 13880 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:05.961534 13987 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:06.007345 14084 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:10:05.985409 14085 raft_consensus.cc:697] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:06.010241 14091 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.106s
I20260709 12:10:06.017661 13980 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:06.018213 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:10:06.018517 12427 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47524 (request call id 91) took 1676 ms. Trace:
I20260709 12:10:06.021762 14069 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:06.029433 13335 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:06.029728 14038 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:10:06.029917 14012 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:06.031602 13880 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:06.032758 13943 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:06.033548 14017 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:10:05.989462 14025 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.035953 13998 tablet_replica.cc:333] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:10:06.003516 14079 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.034941 13946 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:06.043665 14036 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.606s	user 0.013s	sys 0.000s
I20260709 12:10:06.034952 14044 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.041181 13914 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:06.044147 14021 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.053289 13943 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.068274 13334 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:10:06.142627 14044 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:06.141913 14082 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:06.152029 14030 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:06.153508 13998 raft_consensus.cc:2243] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:10:06.059907 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 69333.5us
I20260709 12:10:06.050689 14012 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.049422 14085 consensus_queue.cc:237] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.070034 14036 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:06.163542 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:06.165508 14036 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:06.158073 14090 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 12:10:06.166399 13914 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:06.167013 13914 raft_consensus.cc:697] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 3 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:06.167285 13998 raft_consensus.cc:2272] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:10:06.167971 13914 consensus_queue.cc:237] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:05.997725 14053 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.038147 12427 rpcz_store.cc:276] 0709 12:10:04.342657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:04.346243 (+  3586us) service_pool.cc:224] Handling call
0709 12:10:06.018437 (+1672194us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:04.501170 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:04.501990 (+   820us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:04.502048 (+    58us) write_op.cc:435] Acquired schema lock
0709 12:10:04.502073 (+    25us) tablet.cc:662] Decoding operations
0709 12:10:04.502588 (+   515us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:04.502699 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:04.502732 (+    33us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:10:04.503068 (+   336us) tablet.cc:701] Row locks acquired
0709 12:10:04.503116 (+    48us) write_op.cc:260] PREPARE: finished
0709 12:10:04.503303 (+   187us) write_op.cc:270] Start()
0709 12:10:04.503496 (+   193us) write_op.cc:276] Timestamp: P: 1783599004503229 usec, L: 0
0709 12:10:04.503526 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 12:10:04.504286 (+   760us) log.cc:844] Serialized 535 byte log entry
0709 12:10:04.678950 (+174664us) op_driver.cc:464] REPLICATION: finished
0709 12:10:04.888739 (+209789us) write_op.cc:301] APPLY: starting
0709 12:10:04.888842 (+   103us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:04.888992 (+   150us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:04.889126 (+   134us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:04.889433 (+   307us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:04.889604 (+   171us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:10:04.889649 (+    45us) write_op.cc:312] APPLY: finished
0709 12:10:04.889859 (+   210us) log.cc:844] Serialized 22 byte log entry
0709 12:10:04.893844 (+  3985us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:04.940285 (+ 46441us) write_op.cc:454] Released schema lock
0709 12:10:04.942075 (+  1790us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:04.977038 (+ 34963us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:04.977076 (+    38us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":209490,"apply.run_cpu_time_us":8331,"apply.run_wall_time_us":73046,"num_ops":1,"prepare.queue_time_us":104672,"prepare.run_cpu_time_us":3644,"prepare.run_wall_time_us":31080,"replication_time_us":175201,"thread_start_us":180639,"threads_started":2}]]}
I20260709 12:10:06.173257 13922 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:06.174866 14025 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:05.664461 12416 catalog_manager.cc:5028] TS a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): tablet 816b6f34d124434882148e507cf5b612 (table test-workload [id=8634097bac3246c9a57e272441058c28]) successfully deleted
W20260709 12:10:06.176636 12416 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 531691us
I20260709 12:10:06.177875 13932 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:06.041144 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:06.146660 13906 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:06.161216 14082 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:06.183209 12429 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58992 (request call id 168) took 1970 ms. Trace:
I20260709 12:10:06.171949 13336 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:06.192890 14040 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.191665 12429 rpcz_store.cc:276] 0709 12:10:04.289570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:04.300380 (+ 10810us) service_pool.cc:224] Handling call
0709 12:10:06.182970 (+1882590us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:04.551933 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:04.552504 (+   571us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:04.552532 (+    28us) write_op.cc:435] Acquired schema lock
0709 12:10:04.552549 (+    17us) tablet.cc:662] Decoding operations
0709 12:10:04.552894 (+   345us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:04.552940 (+    46us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:04.552968 (+    28us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:10:04.553162 (+   194us) tablet.cc:701] Row locks acquired
0709 12:10:04.553180 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:10:04.553316 (+   136us) write_op.cc:270] Start()
0709 12:10:04.553423 (+   107us) write_op.cc:276] Timestamp: P: 1783599004553263 usec, L: 0
0709 12:10:04.553442 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:10:04.554083 (+   641us) log.cc:844] Serialized 535 byte log entry
0709 12:10:04.841438 (+287355us) op_driver.cc:464] REPLICATION: finished
0709 12:10:04.948326 (+106888us) write_op.cc:301] APPLY: starting
0709 12:10:04.954590 (+  6264us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:04.970725 (+ 16135us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:04.987781 (+ 17056us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:04.988099 (+   318us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:04.988306 (+   207us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:10:04.988352 (+    46us) write_op.cc:312] APPLY: finished
0709 12:10:05.005098 (+ 16746us) log.cc:844] Serialized 22 byte log entry
0709 12:10:05.021639 (+ 16541us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:05.021758 (+   119us) write_op.cc:454] Released schema lock
0709 12:10:05.021813 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:05.022487 (+   674us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:05.022526 (+    39us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":998739,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":83794,"apply.run_cpu_time_us":3485,"apply.run_wall_time_us":81196,"num_ops":1,"prepare.queue_time_us":160659,"prepare.run_cpu_time_us":2594,"prepare.run_wall_time_us":23993,"replication_time_us":279712,"thread_start_us":10019,"threads_started":1}]]}
I20260709 12:10:06.207569 13336 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:06.136050 13943 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:06.214108 13334 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:06.215337 13335 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:10:06.216238 14066 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.620s	user 0.009s	sys 0.005s
I20260709 12:10:06.216663 14066 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.218070 13918 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:06.219868 14079 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:06.221760 12525 raft_consensus.cc:2368] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3.
I20260709 12:10:06.222875 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:06.223735 13334 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:06.224932 14012 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:06.218482 14092 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.100s	user 0.001s	sys 0.000s
I20260709 12:10:06.227207 12569 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:06.228578 13959 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:06.229012 13959 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.229375 13959 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:06.230437 13791 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.231303 13998 ts_tablet_manager.cc:1916] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260709 12:10:06.231045 13791 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.235467 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:06.236322 12680 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:06.236709 12680 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:06.192519 14070 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.213758 13943 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:06.173300 14053 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:06.190634 14005 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:06.237584 12680 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:06.258612 12680 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.261369 13963 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.262006 13963 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.193796 14082 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.263270 14082 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.264230 13963 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:06.272521 14081 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.275074 14081 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.191757 14036 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
W20260709 12:10:06.282629 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:06.286429 12680 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
I20260709 12:10:06.299638 14011 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.300366 14011 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:06.302978 14011 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:06.306094 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:06.307595 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:06.307966 13959 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:06.310264 13959 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:06.312374 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:06.313063 12671 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:06.314040 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:06.314874 12676 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:06.315593 13941 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:06.317433 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:06.325941 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:06.320029 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:06.318172 12413 catalog_manager.cc:5028] TS f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): tablet 8c7e0eddcc2b4449b7ed8821854646e2 (table test-workload [id=8634097bac3246c9a57e272441058c28]) successfully deleted
I20260709 12:10:06.322542 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:06.321772 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:06.325759 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:06.289089 14052 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:06.352711 14099 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:10:06.323961 13988 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.324326 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:06.355876 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:06.362637 12604 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:06.364022 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:06.365648 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:06.325189 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:06.327782 12599 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:06.367244 12522 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:06.327684 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:06.368167 12602 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 2: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:06.368242 12604 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 2: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:06.369071 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:06.328380 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:06.329954 12526 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:06.322954 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:06.330722 12528 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:06.323673 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:06.371081 12529 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:06.313964 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:06.329797 12678 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
W20260709 12:10:06.333213 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:06.333483 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:06.333386 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:06.373670 12676 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.335472 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:06.376009 12600 raft_consensus.cc:2393] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 2: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:06.336746 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:06.324003 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:06.337273 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:06.338083 14010 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:06.338732 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:06.341197 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:06.342521 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:06.343483 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:06.346493 14066 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:06.348165 14075 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.331239 12603 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:06.360323 14070 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:06.290024 14012 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:06.400429 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:06.404139 12572 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:06.404515 12672 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:06.324326 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:06.378993 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:06.379536 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:06.382011 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:06.388245 13338 raft_consensus.cc:2368] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3.
W20260709 12:10:06.387964 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:06.406380 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:06.407104 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:06.394505 12676 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:06.389540 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:06.417121 12604 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.398509 12600 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:06.405120 12572 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:06.435505 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:06.439739 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:06.440881 12605 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:06.453686 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:06.363480 14011 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.387728 14005 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:06.400992 13905 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.296525 13304 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:06.469779 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:06.465710 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 60659.6us
I20260709 12:10:06.401129 13975 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:06.464715 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:06.396940 13984 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:06.404671 13331 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:06.469987 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:06.463265 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:06.470616 14066 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.470333 14030 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:06.467542 14079 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:06.463282 13922 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.472580 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:06.465626 13943 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.398219 14010 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.471808 12599 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.445331 14019 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:06.456712 14091 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:06.473022 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1442) took 1087 ms. Trace:
I20260709 12:10:06.500607 12572 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:06.498332 14052 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:06.457973 13932 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:06.465034 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:06.525175 13932 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.520048 13791 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:06.467808 14025 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:06.468174 14048 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:06.470502 14070 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.464200 14021 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:06.474151 13951 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.506320 12494 leader_election.cc:400] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3.
W20260709 12:10:06.541967 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 51396.7us
I20260709 12:10:06.507462 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:06.506932 12604 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:06.474376 13960 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:06.497536 14024 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:06.508183 13943 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:06.482743 14100 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:10:06.494438 14005 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:06.464681 14012 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.536872 13336 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:06.508137 14044 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:10:06.495784 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 204823us
I20260709 12:10:06.534590 12530 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:06.505827 13330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38482 (request call id 238) took 1353 ms. Trace:
I20260709 12:10:06.523170 13918 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.506172 13922 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:06.534477 14023 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:06.485707 13984 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:06.494357 13975 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.745827 13975 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:06.527679 13932 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.512683 12601 rpcz_store.cc:276] 0709 12:10:05.395134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:05.399431 (+  4297us) service_pool.cc:224] Handling call
0709 12:10:05.433898 (+ 34467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:05.463252 (+ 29354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:05.464757 (+  1505us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:05.475224 (+ 10467us) log.cc:844] Serialized 26 byte log entry
0709 12:10:06.349776 (+874552us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b18000a20a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:10:06.443828 (+ 94052us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:06.443917 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:06.452389 (+  8472us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:06.462658 (+ 10269us) raft_consensus.cc:1685] finished
0709 12:10:06.462692 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:06.472945 (+ 10253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5448,"raft.queue_time_us":7575,"raft.run_cpu_time_us":204,"raft.run_wall_time_us":7158,"spinlock_wait_cycles":49337728,"thread_start_us":42798,"threads_started":1,"wal-append.queue_time_us":908240}
I20260709 12:10:06.747761 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:06.747903 13922 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:06.748598 14023 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.494437 14080 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.027s
I20260709 12:10:06.489786 13905 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:06.527578 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:06.511732 14091 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.752744 14091 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.753050 13905 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:06.745775 13984 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:06.753427 14005 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:06.499204 14066 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:06.526026 13956 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:06.755777 13956 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.756201 13956 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:06.757102 14066 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:06.544939 12599 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:06.545053 12600 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:10:06.499460 14030 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.758744 13943 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:06.758944 14030 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:06.759030 13932 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.761359 13932 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:06.536412 14052 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 2 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:06.472500 14011 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:06.552178 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 44812.8us
I20260709 12:10:06.762665 12496 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:06.763003 12603 raft_consensus.cc:2393] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:06.553894 14075 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:06.766590 14075 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:06.767038 14075 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:06.550014 12494 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:06.768070 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 218120us
I20260709 12:10:06.547339 13951 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.767416 14075 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.716571 12426 catalog_manager.cc:5697] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 1 to 2, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 2 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:10:06.559710 13856 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:10:06.715775 13302 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.510s (SENT) [suppressed 5 similar messages]
I20260709 12:10:06.547629 14024 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:06.776657 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:06.545998 13960 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.723944 14012 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:06.708244 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:06.726884 12604 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:06.562317 14044 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.802971 14044 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.803145 13951 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:06.803557 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:06.541899 14070 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:06.546494 14021 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.728965 14037 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:10:06.264968 14078 consensus_queue.cc:1175] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e56d487365394ac5ac5f1002926a5645" responder_term: 2 status { last_received { term: 1 index: 4 } last_committed_idx: 4 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:10:06.738816 14102 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.204s	user 0.000s	sys 0.001s
I20260709 12:10:06.562567 13330 rpcz_store.cc:276] 0709 12:10:05.144252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:05.145590 (+  1338us) service_pool.cc:224] Handling call
0709 12:10:05.178282 (+ 32692us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:05.191560 (+ 13278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:05.191602 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:05.192139 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 12:10:05.378547 (+186408us) spinlock_profiling.cc:243] Waited 7.15 ms on lock 0x7b18004359a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f148c8f458f 00007f1485933571 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd
0709 12:10:06.395868 (+1017321us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b18004359a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1
0709 12:10:06.431662 (+ 35794us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:06.456575 (+ 24913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:06.467860 (+ 11285us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:06.470915 (+  3055us) raft_consensus.cc:1685] finished
0709 12:10:06.475178 (+  4263us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:06.496209 (+ 21031us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12015,"raft.queue_time_us":20360,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":23339,"spinlock_wait_cycles":41614592,"thread_start_us":40666,"threads_started":1,"wal-append.queue_time_us":119670,"wal-append.run_cpu_time_us":16142,"wal-append.run_wall_time_us":1393168}
I20260709 12:10:06.742326 13791 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:06.767395 14010 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:06.772522 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:06.770089 12494 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:10:06.830583 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 60576.8us
I20260709 12:10:06.711967 12605 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
W20260709 12:10:06.778043 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:06.791328 13956 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.793793 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:06.789075 13905 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:06.847625 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:06.848479 12680 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:06.849789 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:06.851258 12678 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:06.798435 14024 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.800166 12572 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:06.800859 14011 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:06.801357 12677 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:06.773824 14065 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:06.771242 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:06.865365 12600 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.804765 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:06.865528 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1473) took 1665 ms. Trace:
I20260709 12:10:06.805284 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:06.798378 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:06.866153 12597 rpcz_store.cc:276] 0709 12:10:05.200510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:05.201630 (+  1120us) service_pool.cc:224] Handling call
0709 12:10:05.242027 (+ 40397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:05.279804 (+ 37777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:06.865241 (+1585437us) spinlock_profiling.cc:243] Waited 20.3 ms on lock 0x7b54001171d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:06.865461 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46757248}
I20260709 12:10:06.547964 14025 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:06.871824 14070 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:06.809862 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:06.872342 13956 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:06.762684 14052 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:06.836440 13791 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.771613 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:06.849452 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:06.874501 12531 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:06.875020 13336 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:06.805327 14020 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.805727 13960 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.876545 14020 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.751160 14023 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:06.810303 13856 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:06.878008 13856 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:06.823230 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:06.879638 12679 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.774406 13984 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:06.832193 12494 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:06.881668 13984 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:06.882526 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50493.1us
I20260709 12:10:06.883602 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:06.797346 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:06.811535 14057 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:06.885020 14057 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.888094 14057 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:06.888664 14057 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:06.889109 14057 raft_consensus.cc:697] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:06.890774 12528 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:06.839452 13975 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:06.860288 13905 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:06.891026 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 90961.1us
I20260709 12:10:06.849468 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:06.892282 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:06.872998 12676 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.866250 12604 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:06.853358 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:06.891822 12672 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:10:06.897850 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:06.812688 13918 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:06.906629 13918 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:06.908154 14066 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:06.907822 14075 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.908666 14066 raft_consensus.cc:697] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:06.909622 14066 consensus_queue.cc:237] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.911936 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:06.859858 13906 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.900628 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:06.914719 12528 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:06.921845 12679 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:06.914670 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:06.817267 14037 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:06.937906 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:06.938766 12528 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:06.937976 12572 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:06.899878 14104 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
I20260709 12:10:06.963582 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:06.957758 12572 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:06.969081 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:06.969363 14010 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:06.969909 14010 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:06.970446 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:06.971050 12675 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:10:06.971946 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:06.972724 14010 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:06.894297 13896 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:06.894830 13975 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:06.899652 13956 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:06.974807 12676 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
W20260709 12:10:06.977094 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 93469us
I20260709 12:10:06.994052 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1395) took 1487 ms. Trace:
W20260709 12:10:06.962944 14101 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.511s	user 0.000s	sys 0.009s
I20260709 12:10:06.994721 12523 rpcz_store.cc:276] 0709 12:10:05.490843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:05.491014 (+   171us) service_pool.cc:224] Handling call
0709 12:10:05.497329 (+  6315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:05.910393 (+413064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:06.977625 (+1067232us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b540027f6d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:06.977818 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":788662016}
I20260709 12:10:06.937914 14075 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:06.955238 13951 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:07.003674 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:07.004613 12676 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:10:06.913743 14070 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.083s
I20260709 12:10:06.974923 13922 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:06.889983 14057 consensus_queue.cc:237] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.014679 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:06.954947 14011 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.064s
I20260709 12:10:06.988186 13975 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:06.988197 13896 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.027596 13905 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:07.028182 13896 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:06.999207 13956 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:07.008289 13943 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:06.954867 13906 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:07.022385 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 306657us
I20260709 12:10:07.006523 13951 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:07.038861 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 73758.9us
I20260709 12:10:07.046191 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:07.019539 12676 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 2: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:07.048789 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:07.049722 12676 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:07.009498 14065 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:07.018066 13922 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:07.019825 14075 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:07.064714 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:07.030378 13956 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:07.072444 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:07.073640 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:07.073925 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:07.010082 14030 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.077199 12528 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
W20260709 12:10:07.057994 14106 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.370s	user 0.001s	sys 0.004s
I20260709 12:10:07.037338 13943 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:07.081650 12676 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:10:07.040647 14023 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:07.113026 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:07.113373 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48790.7us
I20260709 12:10:07.113757 12671 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:07.114578 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:07.067477 13922 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:07.067806 14021 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:07.105479 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1443) took 1704 ms. Trace:
I20260709 12:10:07.054611 13951 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:07.040812 13333 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 297) took 1336 ms. Trace:
I20260709 12:10:07.054965 13905 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:07.057464 14012 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:07.116272 12496 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:07.090413 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:07.117165 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:07.117861 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:07.066776 13932 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:07.121753 12675 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 4: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:07.105199 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:07.030437 13856 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.124786 12598 rpcz_store.cc:276] 0709 12:10:05.401334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:05.459136 (+ 57802us) service_pool.cc:224] Handling call
0709 12:10:05.459579 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:05.468746 (+  9167us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:05.468780 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:05.735114 (+266334us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b18000be900. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a
0709 12:10:06.239147 (+504033us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b2c00000e98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:10:06.239877 (+   730us) log.cc:844] Serialized 26 byte log entry
0709 12:10:06.401999 (+162122us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:06.402057 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:06.402208 (+   151us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:06.912756 (+510548us) spinlock_profiling.cc:243] Waited 15.4 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:07.105195 (+192439us) raft_consensus.cc:1685] finished
0709 12:10:07.105217 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:07.105410 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10688,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":23425,"spinlock_wait_cycles":45989888,"thread_start_us":137067,"threads_started":1,"wal-append.queue_time_us":191064}
I20260709 12:10:07.079187 14116 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:10:07.166810 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50638.8us
I20260709 12:10:07.111832 13943 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:07.138942 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:07.197149 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58327.4us
W20260709 12:10:07.133850 14107 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.362s	user 0.001s	sys 0.001s
I20260709 12:10:07.139106 14012 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:07.078980 14075 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:07.165359 13988 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.759s	user 0.013s	sys 0.000s
I20260709 12:10:07.224500 13896 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:07.225226 13896 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.130975 13802 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:07.132359 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:07.125093 14066 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:07.221933 13988 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:07.226071 14130 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:07.228251 12524 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:07.171396 14040 raft_consensus.cc:2764] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:10:07.134886 14118 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:07.201937 12671 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:07.122198 14023 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.215559 12530 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:07.189636 14080 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:07.221158 14075 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:07.179626 14021 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:07.267452 14021 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.268961 13896 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:07.189374 13941 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:07.180614 14065 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.246312 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1400) took 1746 ms. Trace:
W20260709 12:10:07.220765 14121 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 12:10:07.167253 13905 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.216223 14012 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:07.309103 14103 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.775s	user 0.002s	sys 0.000s
I20260709 12:10:07.212016 14070 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:07.310623 14070 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.134100 13333 rpcz_store.cc:276] 0709 12:10:05.685269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:05.690173 (+  4904us) service_pool.cc:224] Handling call
0709 12:10:07.006812 (+1316639us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5400469954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:07.021038 (+ 14226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":296928512}
I20260709 12:10:07.277442 13941 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:07.275712 12525 rpcz_store.cc:276] 0709 12:10:05.499571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:05.515012 (+ 15441us) service_pool.cc:224] Handling call
0709 12:10:07.245924 (+1730912us) spinlock_profiling.cc:243] Waited 692 ms on lock 0x7b54000eef54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:07.246182 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1592269056}
I20260709 12:10:07.324676 14017 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:07.325387 14017 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:07.328083 14017 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:07.330025 13856 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:07.284060 12530 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:07.198144 13932 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.233654 14011 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:07.269917 14021 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:07.286300 13331 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.800s	user 0.007s	sys 0.006s
I20260709 12:10:07.347954 13331 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:07.348888 13331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38472 (request call id 122) took 1011 ms. Trace:
I20260709 12:10:07.253007 14037 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:07.283761 14085 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:07.351054 14085 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.277503 14030 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:07.289361 12673 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 2: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:10:07.291747 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:07.354867 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1474) took 1042 ms. Trace:
I20260709 12:10:07.300192 14119 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:07.362145 14037 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:07.362501 14119 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:07.362954 14037 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.366048 14037 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:07.367012 13339 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 4. Ops: []
I20260709 12:10:07.370460 13932 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:07.376431 13941 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:07.384219 14023 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:07.272224 14080 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.387238 14080 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:07.389287 13906 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:07.390707 14021 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:10:07.305085 14081 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.029s	user 0.005s	sys 0.005s
I20260709 12:10:07.307315 13905 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:07.226485 13802 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.228906 13988 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.342651 12524 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:07.349696 13987 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:07.355273 12673 rpcz_store.cc:276] 0709 12:10:06.312581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:06.313687 (+  1106us) service_pool.cc:224] Handling call
0709 12:10:07.289013 (+975326us) spinlock_profiling.cc:243] Waited 444 ms on lock 0x7b5400284c20. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488997dc0 00007f1488a23e18 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:07.354784 (+ 65771us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1020983680}
I20260709 12:10:07.403616 13802 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:07.284716 13960 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.412174 13960 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:07.339694 14044 consensus_meta.cc:287] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.536s	user 0.004s	sys 0.007s
I20260709 12:10:07.422438 14044 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.216897 14116 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.400187 12672 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:07.391316 14021 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.396963 13906 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:07.402315 13856 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:07.402717 14123 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:07.401095 12494 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:07.355926 14011 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:07.420691 13988 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:07.254719 14091 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.273077 13975 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:07.349478 13331 rpcz_store.cc:276] 0709 12:10:06.337962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:06.338283 (+   321us) service_pool.cc:224] Handling call
0709 12:10:07.348816 (+1010533us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:07.455461 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 1321) took 1131 ms. Trace:
I20260709 12:10:07.481369 12524 rpcz_store.cc:276] 0709 12:10:06.323573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:06.340895 (+ 17322us) service_pool.cc:224] Handling call
0709 12:10:07.455262 (+1114367us) spinlock_profiling.cc:243] Waited 840 ms on lock 0x7b5400407754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:07.455390 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1931450112}
I20260709 12:10:07.396486 14119 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:07.483363 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:07.432982 12600 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.537s	user 0.010s	sys 0.000s
I20260709 12:10:07.484553 12600 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
W20260709 12:10:07.485595 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 84623.2us
I20260709 12:10:07.425206 14044 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:07.435438 14021 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:07.491065 12680 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 3: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:07.463661 13920 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:07.462849 13791 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:07.434517 14070 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:07.445876 14025 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:07.447525 13984 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:07.471002 14085 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:07.453811 14020 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:07.468961 13943 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.502045 13339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 303) took 1038 ms. Trace:
I20260709 12:10:07.499389 13998 ts_tablet_manager.cc:1929] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260709 12:10:07.490963 13988 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:07.427888 14116 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:07.474825 13956 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.482448 13906 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:07.490000 13856 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:07.448762 13922 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:07.491811 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 384443us
I20260709 12:10:07.531544 14091 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:07.531762 13906 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:07.528723 13998 log.cc:1199] T 952cdab9198b44e4bdf1b472109715c3 P f5fc807f7ecc4d1ca0fd6be524f734ce: Deleting WAL directory at /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0/minicluster-data/ts-1-root/wals/952cdab9198b44e4bdf1b472109715c3
I20260709 12:10:07.517652 14070 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:07.520069 14044 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:07.559185 14044 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:07.527570 13988 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:07.519956 13339 rpcz_store.cc:276] 0709 12:10:06.463086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:06.483621 (+ 20535us) service_pool.cc:224] Handling call
0709 12:10:07.366649 (+883028us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:07.367890 (+  1241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:07.485112 (+117222us) spinlock_profiling.cc:243] Waited 871 ms on lock 0x7b54006450d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:07.501401 (+ 16289us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2003693952}
I20260709 12:10:07.490620 14123 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:07.565299 13988 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:07.579638 12680 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:07.579754 12494 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:07.581383 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:07.581419 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:07.565079 13956 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:07.548287 13951 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:07.616194 14091 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:07.615962 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:07.616966 14118 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:07.558043 14070 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:07.621567 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:07.625650 12680 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
I20260709 12:10:07.641661 12603 raft_consensus.cc:2368] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3.
I20260709 12:10:07.574522 13975 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:07.583748 14024 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.689s	user 0.014s	sys 0.001s
I20260709 12:10:07.574584 14020 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:07.615543 14116 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:10:07.650080 12495 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.020s	sys 0.021s
W20260709 12:10:07.661954 12495 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.655s	user 0.021s	sys 0.021s
I20260709 12:10:07.656072 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:07.662822 12530 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:10:07.589110 13856 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:07.664811 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:07.672905 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:07.673599 12680 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:07.685389 12680 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:07.685413 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:07.686259 12528 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
W20260709 12:10:07.686671 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:07.646539 13998 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51774 (request call id 68) took 4599 ms. Trace:
I20260709 12:10:07.622352 14091 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.649012 14085 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:07.692639 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:07.693853 12606 raft_consensus.cc:2368] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3.
W20260709 12:10:07.668612 13918 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.001s	sys 0.046s
W20260709 12:10:07.693246 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 495255us
I20260709 12:10:07.697108 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:07.698846 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:07.699012 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:07.699550 12528 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
W20260709 12:10:07.694797 13918 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.002s	sys 0.046s
I20260709 12:10:07.672735 14123 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:07.692860 13998 rpcz_store.cc:276] 0709 12:10:03.023567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:03.041181 (+ 17614us) service_pool.cc:224] Handling call
0709 12:10:05.978870 (+2937689us) ts_tablet_manager.cc:1151] Deleting tablet 952cdab9198b44e4bdf1b472109715c3
0709 12:10:05.990705 (+ 11835us) ts_tablet_manager.cc:1071] Acquired tablet manager lock
0709 12:10:06.581250 (+590545us) tablet_metadata.cc:665] Metadata flushed
0709 12:10:07.489914 (+908664us) tablet_metadata.cc:665] Metadata flushed
0709 12:10:07.619362 (+129448us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91824,"tablet-delete.queue_time_us":2835503}
I20260709 12:10:07.720299 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:07.706358 14025 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:07.710065 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:07.721360 12522 raft_consensus.cc:2368] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 2. Current term is 3.
I20260709 12:10:07.721151 14025 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.711604 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:07.713133 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:07.714901 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:07.722507 14011 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:07.723420 12604 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:07.714583 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:07.720335 13943 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:07.715763 14044 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:07.716223 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:07.725142 12597 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 2: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:07.725412 12599 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:07.672348 14116 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:07.727953 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:07.689770 12530 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:07.723985 14025 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:07.728341 12571 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:07.728853 12522 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 2: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:07.728400 14133 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.033s
I20260709 12:10:07.746905 12526 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:07.650980 14024 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.731703 12571 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:07.736171 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:07.714573 13922 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:07.756206 13305 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.004s	sys 0.016s
I20260709 12:10:07.739312 12605 consensus_queue.cc:237] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
W20260709 12:10:07.766489 13305 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.004s	sys 0.016s
W20260709 12:10:07.748772 13302 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.007s	sys 0.030s
I20260709 12:10:07.664644 14020 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:07.716372 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:07.767970 12600 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:07.798079 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:07.799082 12606 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:07.768600 12599 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:10:07.770092 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:07.802187 12598 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:07.771924 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:07.802980 12604 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:07.798983 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:07.803679 12678 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:07.803869 12526 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
W20260709 12:10:07.768654 12572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.008s	sys 0.008s
W20260709 12:10:07.804540 12572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.685s	user 0.008s	sys 0.008s
W20260709 12:10:07.805047 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:07.805446 12674 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.013s	sys 0.035s
I20260709 12:10:07.730695 14070 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:07.732053 13918 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:07.817242 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:07.695106 14012 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.698143 14085 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:07.807030 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:07.769094 13302 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.008s	sys 0.030s
W20260709 12:10:07.877862 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 810254us
I20260709 12:10:07.802971 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:07.878960 12680 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 4: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:10:07.810313 12674 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.013s	sys 0.035s
I20260709 12:10:07.731832 13951 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:07.847536 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:07.883057 12597 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:07.807582 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:07.769668 13956 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:07.855252 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:07.767104 14048 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.007s	sys 0.007s
W20260709 12:10:07.805814 13946 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.010s	sys 0.013s
I20260709 12:10:07.851670 12604 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
I20260709 12:10:07.734942 14052 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.141s
I20260709 12:10:07.733633 13304 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:10:07.891258 12496 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.011s	sys 0.018s
W20260709 12:10:07.907699 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:07.898274 12526 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:10:07.912503 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1486) took 1019 ms. Trace:
I20260709 12:10:07.901863 12598 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:07.814440 14132 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:07.919330 12606 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:07.801651 14129 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:08.037724 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 366843us
I20260709 12:10:07.850440 12601 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:07.993856 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:07.950344 12676 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:07.967558 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:07.956657 12671 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:08.053550 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:08.049456 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:08.056350 12673 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
W20260709 12:10:08.053861 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 378778us
I20260709 12:10:08.055583 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:07.995844 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:08.053311 12496 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.858s	user 0.014s	sys 0.019s
I20260709 12:10:08.069046 12678 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.056419 12675 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:08.043612 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:08.070854 13337 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.075408 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:08.069141 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 873818us
I20260709 12:10:08.076313 12604 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:08.076570 12496 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:07.882637 14025 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:08.051921 12674 rpcz_store.cc:276] 0709 12:10:06.893425 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:06.954401 (+ 60976us) service_pool.cc:224] Handling call
0709 12:10:06.991671 (+ 37270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:06.991937 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:07.244612 (+252675us) spinlock_profiling.cc:243] Waited 9.14 ms on lock 0x7b18002fa700. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf
0709 12:10:07.244862 (+   250us) log.cc:844] Serialized 14 byte log entry
0709 12:10:07.912048 (+667186us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:07.912138 (+    90us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:07.912208 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:07.912295 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:07.912436 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51428,"prepare.queue_time_us":87493,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":32872,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":7141,"spinlock_wait_cycles":21016960,"thread_start_us":682780,"threads_started":2}
I20260709 12:10:08.078306 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:07.986522 14102 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 12:10:07.929828 13946 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.711s	user 0.010s	sys 0.013s
W20260709 12:10:08.079285 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:08.064492 14129 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:07.968242 13941 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:07.898617 14091 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:08.043881 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:08.079255 12525 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:08.079691 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:08.070474 14139 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:07.895974 13956 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:08.097747 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:07.929131 14048 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.007s	sys 0.008s
I20260709 12:10:08.100076 12413 catalog_manager.cc:5028] TS f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): tablet 952cdab9198b44e4bdf1b472109715c3 (table test-workload [id=8634097bac3246c9a57e272441058c28]) successfully deleted
I20260709 12:10:07.980793 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:08.045264 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:07.990276 14012 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:07.984216 14024 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:07.982574 14020 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:07.949254 13918 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:07.965862 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:08.123958 12678 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:07.955813 14116 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:08.126358 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:08.126456 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:08.126767 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:08.104444 14025 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:08.130447 12572 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:08.166759 14085 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:08.136317 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:08.148412 12413 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 48352.9us
I20260709 12:10:08.152410 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:08.190587 12526 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:07.991667 13988 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:08.147739 13956 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.103971 14057 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:08.136497 13941 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:08.147975 13791 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:08.173903 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:08.210784 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:08.211477 12672 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.152009 14139 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:08.098905 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:08.122471 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 373454us
I20260709 12:10:08.179286 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:08.216342 13304 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:08.216915 12523 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:08.218457 13987 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:08.133608 14143 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:08.185186 12529 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:08.219563 14143 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:08.194546 14020 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:08.220032 14143 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.221122 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:08.222157 12678 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 3: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:08.144127 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:08.224700 12522 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
W20260709 12:10:08.188036 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 57698.9us
I20260709 12:10:07.974598 14140 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.221s
I20260709 12:10:08.226779 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:08.229089 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:08.230216 12678 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.192273 12527 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.186964 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:08.205389 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:08.189653 13339 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:08.207142 13332 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 4: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:10:08.208163 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 105486us
I20260709 12:10:08.280526 12496 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:08.208461 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:08.281489 12671 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.282531 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:08.063827 14132 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:08.238322 12525 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.185098 13336 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:08.210668 13334 raft_consensus.cc:2410] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 12:10:08.277141 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 50460.3us
I20260709 12:10:08.293926 12572 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:08.133499 12494 leader_election.cc:400] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3.
I20260709 12:10:08.220556 14085 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:08.295476 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:08.300484 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:08.301188 12529 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.295419 12494 leader_election.cc:403] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:10:08.271399 14011 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:08.316310 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:08.317684 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:08.318861 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:08.319645 12522 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.324450 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:08.325119 12531 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:08.309101 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 183250us
I20260709 12:10:08.332525 12494 leader_election.cc:400] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3.
I20260709 12:10:08.276321 14020 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:08.333348 12494 leader_election.cc:403] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:10:08.345645 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:08.352953 12671 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:08.239307 13791 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:08.285413 13333 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:08.301153 13946 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:08.373795 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:08.380518 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:08.426908 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 53245.8us
I20260709 12:10:08.307708 13920 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:08.308482 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:08.437319 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:08.369812 14057 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3.
I20260709 12:10:08.333142 14125 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:08.310513 14085 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:08.369742 14052 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:08.432153 13920 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:08.497915 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:08.438184 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:08.427723 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:08.432013 14134 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:10:08.442524 12603 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.733s	user 0.023s	sys 0.061s
W20260709 12:10:08.401834 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:08.449872 14125 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:08.350281 14011 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:08.308527 13918 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:08.519977 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:08.518096 12603 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.024s	sys 0.061s
I20260709 12:10:08.511520 12671 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260709 12:10:08.477631 14135 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.035s	sys 0.060s
I20260709 12:10:08.545001 12522 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:08.395942 12525 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:08.562779 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:10:08.565933 12605 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.011s	sys 0.014s
W20260709 12:10:08.587539 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:08.611431 12605 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.012s	sys 0.014s
I20260709 12:10:08.407546 12526 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:08.541043 12678 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:08.478601 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:08.482239 14136 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.756s	user 0.024s	sys 0.068s
I20260709 12:10:08.501770 13920 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:08.500067 14129 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:08.616823 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:08.517549 14134 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:08.517606 14139 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:08.511440 13984 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:08.511291 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:08.606725 14135 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.035s	sys 0.061s
I20260709 12:10:08.637924 13302 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:08.693545 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 55843.7us
I20260709 12:10:08.496645 14052 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:08.712862 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:08.716063 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:08.644164 14125 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:08.718017 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:08.718222 13984 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:08.719646 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:08.720367 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:08.721223 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:08.522861 13951 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:08.610617 14057 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:10:08.631865 14136 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.024s	sys 0.072s
I20260709 12:10:08.699774 14025 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:08.691962 13336 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 3: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:08.705467 12531 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:08.706384 13941 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:10:08.706687 14132 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:08.690284 13988 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:08.707655 14011 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:08.709173 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:08.682269 14134 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:08.721663 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:08.619908 14020 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:08.756726 14144 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.477s	user 0.000s	sys 0.001s
I20260709 12:10:08.749356 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:08.767308 13331 raft_consensus.cc:2393] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 2: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:08.768318 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:08.769073 13331 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 2: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:08.748378 14057 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:08.734997 12527 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:08.722398 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:08.772811 12525 raft_consensus.cc:2393] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:08.743820 12529 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
W20260709 12:10:08.731979 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 416044us
I20260709 12:10:08.773844 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:08.774739 12526 raft_consensus.cc:2368] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3.
I20260709 12:10:08.774968 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:08.722065 12522 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:08.633706 13332 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:08.757954 13791 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:08.758795 14052 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:08.771926 14048 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.016s	sys 0.012s
W20260709 12:10:08.754063 14150 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.542s	user 0.002s	sys 0.000s
W20260709 12:10:08.770401 12600 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.700s	user 0.010s	sys 0.018s
W20260709 12:10:08.750260 12672 consensus_meta.cc:287] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.523s	user 0.011s	sys 0.001s
I20260709 12:10:08.759591 13984 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:08.760852 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:08.755863 13951 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:08.781697 14052 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:08.782886 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:08.763747 14132 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:08.794100 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:08.724635 13337 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.641s	user 0.010s	sys 0.001s
I20260709 12:10:08.886234 12672 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:08.886699 13337 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
W20260709 12:10:08.830884 13305 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.013s	sys 0.032s
W20260709 12:10:08.887758 12679 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.023s	sys 0.013s
W20260709 12:10:08.888088 13305 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.612s	user 0.013s	sys 0.035s
W20260709 12:10:08.888342 12679 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.024s	sys 0.013s
W20260709 12:10:08.888634 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 615185us
I20260709 12:10:08.773380 14057 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:08.776856 14070 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3.
W20260709 12:10:08.795966 14048 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.016s	sys 0.012s
I20260709 12:10:08.772523 13333 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:08.777824 13956 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.000s
I20260709 12:10:08.797919 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:08.884431 12600 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.012s	sys 0.019s
I20260709 12:10:08.884886 12522 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:08.959527 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:08.961196 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:08.962601 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:08.886813 14146 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.588s	user 0.005s	sys 0.023s
I20260709 12:10:08.968196 12526 raft_consensus.cc:2393] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:10:08.890704 14012 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.020s	sys 0.017s
I20260709 12:10:08.973726 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:08.975291 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:08.976938 12524 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.810s	user 0.016s	sys 0.031s
I20260709 12:10:08.977532 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:08.891929 14052 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:08.979074 12525 raft_consensus.cc:2368] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3.
W20260709 12:10:08.969269 14146 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.729s	user 0.005s	sys 0.023s
W20260709 12:10:08.981869 12528 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.013s	sys 0.020s
I20260709 12:10:08.893587 14132 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 3 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:08.980002 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:08.984002 12569 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.002s	sys 0.044s
W20260709 12:10:08.985680 12569 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.003s	sys 0.045s
W20260709 12:10:08.988090 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 601308us
W20260709 12:10:08.979825 12524 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.813s	user 0.016s	sys 0.031s
I20260709 12:10:08.980839 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:08.984313 12526 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:10:08.992205 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 61) took 1340 ms. Trace:
I20260709 12:10:08.992731 12524 rpcz_store.cc:276] 0709 12:10:07.651967 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:07.652182 (+   215us) service_pool.cc:224] Handling call
0709 12:10:07.652675 (+   493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:07.653234 (+   559us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:10:08.155246 (+502012us) log.cc:844] Serialized 14 byte log entry
0709 12:10:08.991726 (+836480us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:08.991807 (+    81us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:08.991883 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:08.991995 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:08.992144 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":955343,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":7691,"thread_start_us":1306221,"threads_started":2}
I20260709 12:10:08.994288 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:08.995136 12529 raft_consensus.cc:2368] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3.
W20260709 12:10:08.984709 12528 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.014s	sys 0.020s
W20260709 12:10:08.973244 14012 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.020s	sys 0.017s
W20260709 12:10:08.983219 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 690681us
I20260709 12:10:08.970408 13331 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:10:08.989560 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:09.008507 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:08.984791 14143 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.712s	user 0.008s	sys 0.004s
I20260709 12:10:08.954687 13956 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:09.004159 12496 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:09.007373 12522 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:10:08.982316 12571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.013s	sys 0.020s
I20260709 12:10:09.009236 12531 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:09.015022 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:09.016263 12522 raft_consensus.cc:2368] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3.
I20260709 12:10:09.016572 12531 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:10:09.008258 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:08.996101 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:09.019323 12571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.013s	sys 0.021s
W20260709 12:10:09.020390 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 596390us
I20260709 12:10:08.984261 14085 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:09.032163 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:09.038637 14159 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 12:10:09.042956 13951 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:09.007037 14024 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:10:09.029951 13987 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.791s	user 0.012s	sys 0.056s
I20260709 12:10:09.057881 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:09.066471 13988 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:10:09.058501 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:09.056571 14065 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:10:09.020041 13303 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.010s	sys 0.024s
I20260709 12:10:09.014869 14143 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:09.067950 13987 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.849s	user 0.013s	sys 0.056s
W20260709 12:10:09.076887 13946 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.012s	sys 0.034s
W20260709 12:10:09.083143 12676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.006s	sys 0.026s
W20260709 12:10:09.089812 12494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.018s	sys 0.015s
W20260709 12:10:09.089494 12523 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.001s	sys 0.016s
W20260709 12:10:09.091048 12523 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.002s	sys 0.016s
W20260709 12:10:09.090270 12676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.007s	sys 0.026s
W20260709 12:10:09.091915 12674 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.018s	sys 0.015s
W20260709 12:10:09.093658 12674 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s	user 0.018s	sys 0.015s
W20260709 12:10:09.090193 12494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.018s	sys 0.015s
W20260709 12:10:09.113286 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 685607us
I20260709 12:10:09.114722 12494 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:09.089511 13946 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.013s	sys 0.034s
W20260709 12:10:09.087828 13303 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.787s	user 0.011s	sys 0.024s
I20260709 12:10:09.133450 13338 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 3: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:10:09.119534 13330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.014s	sys 0.004s
I20260709 12:10:09.141360 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:09.089368 13987 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:10:09.149858 13330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.015s	sys 0.005s
I20260709 12:10:09.128583 13946 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:10:09.158274 14161 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:10:09.069041 14052 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:09.198568 12680 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.018s	sys 0.012s
W20260709 12:10:09.199195 12680 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.018s	sys 0.012s
I20260709 12:10:09.184490 14085 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:09.087988 14132 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:09.181644 14140 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
W20260709 12:10:09.201251 13920 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.572s	user 0.013s	sys 0.000s
I20260709 12:10:09.212580 14065 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:10:09.249162 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:09.245532 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 470195us
W20260709 12:10:09.250684 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 237066us
I20260709 12:10:09.252048 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1471) took 1599 ms. Trace:
I20260709 12:10:09.252563 12603 rpcz_store.cc:276] 0709 12:10:07.653108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:07.657174 (+  4066us) service_pool.cc:224] Handling call
0709 12:10:07.672140 (+ 14966us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:07.672619 (+   479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:07.672662 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:07.695689 (+ 23027us) log.cc:844] Serialized 52 byte log entry
0709 12:10:08.607834 (+912145us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:09.142299 (+534465us) log.cc:844] Serialized 14 byte log entry
0709 12:10:09.251683 (+109384us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:09.251782 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:09.251835 (+    53us) raft_consensus.cc:1685] finished
0709 12:10:09.251851 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:09.251988 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":109440,"prepare.queue_time_us":114839,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":7348,"raft.run_cpu_time_us":211,"raft.run_wall_time_us":17323,"thread_start_us":1353606,"threads_started":2,"wal-append.queue_time_us":965218}
I20260709 12:10:09.255503 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1408) took 1429 ms. Trace:
I20260709 12:10:09.256145 12530 rpcz_store.cc:276] 0709 12:10:07.826223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:08.064799 (+238576us) service_pool.cc:224] Handling call
0709 12:10:08.065537 (+   738us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:08.066164 (+   627us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:08.066221 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:08.081882 (+ 15661us) log.cc:844] Serialized 26 byte log entry
0709 12:10:08.606931 (+525049us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:08.606976 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:08.607249 (+   273us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:09.255193 (+647944us) raft_consensus.cc:1685] finished
0709 12:10:09.255236 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:09.255427 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":118,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":24525,"thread_start_us":455483,"threads_started":1,"wal-append.queue_time_us":918099}
I20260709 12:10:09.252166 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:10:09.256742 14116 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.011s	sys 0.019s
W20260709 12:10:09.258903 14116 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.510s	user 0.012s	sys 0.019s
I20260709 12:10:09.272388 12602 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:09.218896 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:09.152413 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 453269us
I20260709 12:10:09.259588 14116 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:09.313524 12605 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:09.242444 13956 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:09.281354 14135 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.000s	sys 0.046s
I20260709 12:10:09.244853 14136 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:09.403123 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1481) took 1354 ms. Trace:
W20260709 12:10:09.403867 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 153237us
I20260709 12:10:09.247031 14065 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:09.218791 14085 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:09.226320 13920 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:09.413223 12600 rpcz_store.cc:276] 0709 12:10:08.049011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:08.049566 (+   555us) service_pool.cc:224] Handling call
0709 12:10:08.049902 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:08.050101 (+   199us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:10:08.916326 (+866225us) log.cc:844] Serialized 14 byte log entry
0709 12:10:09.402652 (+486326us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:09.402720 (+    68us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:09.402793 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:09.402884 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:09.403047 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":195812,"prepare.queue_time_us":641095,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":196912,"thread_start_us":1135339,"threads_started":2}
W20260709 12:10:09.423840 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 400333us
W20260709 12:10:09.409129 14135 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.001s	sys 0.046s
W20260709 12:10:09.416363 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 164985us
I20260709 12:10:09.415189 12569 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:09.402838 14172 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:09.451785 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:09.465782 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 213678us
W20260709 12:10:09.467710 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 326423us
W20260709 12:10:09.470356 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 55314.9us
I20260709 12:10:09.471504 12569 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:09.436939 13339 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 2: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:09.426824 14065 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:09.435603 14134 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.669s	user 0.012s	sys 0.000s
I20260709 12:10:09.469403 12572 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:09.245645 13987 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:09.469285 12494 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:09.506916 12572 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:09.424575 14136 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:09.411083 13946 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:09.505614 14125 consensus_meta.cc:287] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.778s	user 0.010s	sys 0.003s
I20260709 12:10:09.473305 13941 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:09.492033 13956 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:09.488086 14179 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:09.495944 13918 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:10:09.544997 14163 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.358s	user 0.000s	sys 0.002s
I20260709 12:10:09.543282 14141 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:10:09.568053 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 99318.2us
I20260709 12:10:09.539378 14147 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:09.576713 14147 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:09.577361 14147 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:09.581583 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:09.582559 12679 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:10:09.543706 13339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 244) took 1188 ms. Trace:
I20260709 12:10:09.475257 14172 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:09.596704 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 127527us
I20260709 12:10:09.505095 14134 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:09.584779 14141 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:09.603960 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:09.604187 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:09.604759 12678 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:09.605158 12673 raft_consensus.cc:2393] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 3: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:09.605311 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:09.604403 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:09.612488 12671 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:09.613461 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:09.605659 12494 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:09.560336 14125 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:09.620901 12671 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:09.624177 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:09.624383 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:09.570322 13956 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:09.623044 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:09.625298 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 132823us
I20260709 12:10:09.625226 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:09.630105 12571 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:09.625223 12672 raft_consensus.cc:2368] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3.
W20260709 12:10:09.619443 14162 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.343s	user 0.001s	sys 0.001s
I20260709 12:10:09.630573 12675 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:10:09.633492 12571 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:09.635694 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:09.626832 12678 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
W20260709 12:10:09.637707 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:09.638310 12527 raft_consensus.cc:2368] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3.
W20260709 12:10:09.636093 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:09.610725 14141 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:09.644948 14180 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:09.634297 13922 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:10:09.632246 14177 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.221s	user 0.002s	sys 0.000s
I20260709 12:10:09.591903 13339 rpcz_store.cc:276] 0709 12:10:08.355511 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:08.428478 (+ 72967us) service_pool.cc:224] Handling call
0709 12:10:09.209593 (+781115us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b5800163298. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:10:09.538094 (+328501us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29062528}
I20260709 12:10:09.638146 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:09.648911 13338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38472 (request call id 129) took 1596 ms. Trace:
I20260709 12:10:09.656931 12571 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:09.660224 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:09.657763 13338 rpcz_store.cc:276] 0709 12:10:08.052718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:08.052864 (+   146us) service_pool.cc:224] Handling call
0709 12:10:09.648678 (+1595814us) spinlock_profiling.cc:243] Waited 980 ms on lock 0x7b5400690354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:09.648845 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2254615936}
I20260709 12:10:09.664031 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:09.646615 14178 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:10:09.641206 13951 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:09.645630 14180 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:09.648034 14194 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
W20260709 12:10:09.762830 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 123861us
W20260709 12:10:09.646162 14171 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.394s	user 0.002s	sys 0.001s
I20260709 12:10:09.647465 14133 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:10:09.760900 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 96838.1us
I20260709 12:10:09.777259 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:09.761687 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:09.648795 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:09.648243 13920 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:09.656109 14116 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:10:09.659790 14191 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.190s	user 0.000s	sys 0.001s
I20260709 12:10:09.738441 13922 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:09.772200 13305 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:09.763456 14185 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:10:09.664486 14131 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:10:09.764644 14186 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 12:10:09.739823 13918 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:09.651538 14192 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:09.739006 14085 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:09.768615 14193 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 12:10:09.764233 14202 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:09.765792 14174 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:09.779938 14133 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:09.789718 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 282875us
I20260709 12:10:09.812789 12572 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:10:09.809145 12646 connection.cc:256] RPC call timeout handler was delayed by -1.1200076300001456s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1801s for a response
I20260709 12:10:09.814096 12572 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:10:09.814360 12646 connection.cc:256] RPC call timeout handler was delayed by -1.0835610960000395s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1904s for a response
I20260709 12:10:09.794835 14129 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:09.815845 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:10:09.816434 12646 connection.cc:256] RPC call timeout handler was delayed by -1.0152223060001688s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1523s for a response
I20260709 12:10:09.816651 13339 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:09.816824 12646 connection.cc:256] RPC call timeout handler was delayed by -1.0133397850001984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1516s for a response
I20260709 12:10:09.815956 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:09.820878 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1412) took 1726 ms. Trace:
I20260709 12:10:09.784646 14206 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:09.784584 14116 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:09.791548 14138 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:09.794273 14170 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:09.821532 12528 rpcz_store.cc:276] 0709 12:10:08.098630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:08.102371 (+  3741us) service_pool.cc:224] Handling call
0709 12:10:08.132123 (+ 29752us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:08.280799 (+148676us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:08.280871 (+    72us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:08.315439 (+ 34568us) log.cc:844] Serialized 26 byte log entry
0709 12:10:08.996000 (+680561us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:08.996060 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:09.645467 (+649407us) spinlock_profiling.cc:243] Waited 23 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:09.760057 (+114590us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54001072d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:09.760118 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:09.820574 (+ 60456us) raft_consensus.cc:1685] finished
0709 12:10:09.820610 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:09.820793 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2274,"raft.queue_time_us":2608,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":18645,"spinlock_wait_cycles":327731200,"thread_start_us":616712,"threads_started":1,"wal-append.queue_time_us":859966}
I20260709 12:10:09.803529 14192 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:09.800755 14131 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:09.817589 14133 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:09.826996 14196 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.192s	user 0.000s	sys 0.001s
W20260709 12:10:09.837774 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 48448.1us
I20260709 12:10:09.843566 14174 raft_consensus.cc:697] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:09.848269 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:09.830058 14116 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:09.801326 14167 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260709 12:10:09.827840 14202 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:09.853436 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:09.825012 14129 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:09.841125 13956 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:09.857760 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:09.839735 14138 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:09.846254 14085 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:09.829823 14206 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:09.852067 14172 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:09.835587 14204 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:10:09.856133 14205 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:09.875329 14129 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:09.879565 14198 log.cc:927] Time spent T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:10:09.870198 14207 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:10:09.880344 14138 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:09.882942 14202 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:09.884074 14212 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:10:09.880089 14182 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.287s	user 0.006s	sys 0.009s
I20260709 12:10:09.856549 14210 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:09.886615 14210 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:09.886974 12675 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:09.887084 14210 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:09.890512 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:09.880576 14085 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:09.888244 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:09.892174 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:09.893293 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:09.896726 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:09.896330 14131 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:09.880935 14024 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:09.897608 14024 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:09.890445 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:09.899893 12604 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:09.906301 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:09.907593 12604 raft_consensus.cc:2410] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:09.905965 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:09.913527 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:09.858049 14195 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:09.917519 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:09.840034 14170 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:09.919451 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:09.920265 13338 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 2: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:09.920765 14170 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:09.923436 14180 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:09.925946 14206 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:09.926419 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:09.927649 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:09.928874 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:09.929468 12679 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:09.931263 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:09.931669 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:09.932427 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:09.934274 13332 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 3: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:09.901328 13941 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:09.870303 13920 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:09.934999 13941 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:09.935359 13920 raft_consensus.cc:697] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:09.936252 13941 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:09.936574 13920 consensus_queue.cc:237] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:09.938534 14192 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:09.938328 14195 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:09.940583 14195 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:09.941236 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:09.921778 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:09.948457 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:09.950069 13339 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
W20260709 12:10:09.951294 14057 consensus_meta.cc:287] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.056s	user 0.012s	sys 0.003s
I20260709 12:10:09.952479 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:09.956370 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1349) took 1646 ms. Trace:
I20260709 12:10:09.956861 12680 rpcz_store.cc:276] 0709 12:10:08.303732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:08.304086 (+   354us) service_pool.cc:224] Handling call
0709 12:10:08.304453 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:09.199635 (+895182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:09.199794 (+   159us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:09.206633 (+  6839us) log.cc:844] Serialized 26 byte log entry
0709 12:10:09.425699 (+219066us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:09.462486 (+ 36787us) log.cc:844] Serialized 14 byte log entry
0709 12:10:09.465728 (+  3242us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:09.941732 (+476004us) spinlock_profiling.cc:243] Waited 6.36 ms on lock 0x7b180039a660. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419
0709 12:10:09.949407 (+  7675us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b5400092ed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:09.949461 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:09.949506 (+    45us) raft_consensus.cc:1685] finished
0709 12:10:09.949521 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:09.949687 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":40736,"raft.queue_time_us":94777,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":6626,"spinlock_wait_cycles":774306816,"thread_start_us":753287,"threads_started":2,"wal-append.queue_time_us":424402}
I20260709 12:10:09.942559 14085 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:09.958700 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:09.979277 12680 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:09.980569 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:09.980815 14012 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:09.981303 12680 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:09.981390 14012 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:09.985195 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:09.986652 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:09.987762 12531 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:09.988174 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:09.989465 12526 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:09.990496 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:09.990638 13333 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.014s	user 0.015s	sys 0.000s
I20260709 12:10:09.991186 13333 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:09.991725 13333 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38472 (request call id 132) took 1691 ms. Trace:
I20260709 12:10:09.991896 14070 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:09.992159 13333 rpcz_store.cc:276] 0709 12:10:08.300459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:08.721415 (+420956us) service_pool.cc:224] Handling call
0709 12:10:09.991650 (+1270235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:09.992596 14070 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:09.993327 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:09.993583 12677 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260709 12:10:09.994525 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:10.000434 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:09.949998 14036 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:09.995054 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:10.004092 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:10.005573 12604 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:09.994616 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:10.006278 12531 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:10.006740 12603 raft_consensus.cc:2393] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:10.007534 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:10.007714 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:10.008360 12531 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:10:10.008466 12603 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:09.998567 14183 raft_consensus.cc:2764] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:10:09.955866 14192 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:10.010164 14192 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:10.010749 14197 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 12:10:10.012598 14192 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:09.997280 14179 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:10.013842 14179 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.014983 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:10.016346 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1416) took 1533 ms. Trace:
I20260709 12:10:10.017578 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:10.016215 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:10.018765 12599 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:09.991735 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:10.003976 14036 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:10.016881 12679 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:10.023339 12599 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:09.953189 13330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 125) took 2064 ms. Trace:
I20260709 12:10:10.006573 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:10.017691 12523 rpcz_store.cc:276] 0709 12:10:08.483131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:08.516563 (+ 33432us) service_pool.cc:224] Handling call
0709 12:10:08.519283 (+  2720us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:09.091730 (+572447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:09.091779 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:09.144889 (+ 53110us) log.cc:844] Serialized 26 byte log entry
0709 12:10:09.427231 (+282342us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:09.427277 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:09.427353 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:10.016119 (+588766us) raft_consensus.cc:1685] finished
0709 12:10:10.016141 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:10.016285 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":105833,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":199,"thread_start_us":746593,"threads_started":2,"wal-append.queue_time_us":624953}
I20260709 12:10:10.050762 14172 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:10.051101 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:10.055217 14172 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:10.055536 12526 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:10.056311 12604 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:10.056624 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:10.057353 14172 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:10.058480 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:10.058620 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:10.057871 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:10.059326 12526 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:10:10.060225 12680 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:10.060254 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:10.062268 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:10.061772 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:09.951787 14057 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.066954 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:10.041405 14065 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.532s	user 0.005s	sys 0.005s
I20260709 12:10:10.071826 12522 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:09.997481 14195 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:10:10.097548 13302 leader_election.cc:336] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.510s (SENT)
I20260709 12:10:10.060039 13922 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:09.959250 14174 consensus_queue.cc:237] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:10.271700 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 51017.5us
I20260709 12:10:10.132781 12529 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:10.284786 12571 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:10.157807 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 300093us
I20260709 12:10:10.215140 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 71) took 1711 ms. Trace:
W20260709 12:10:10.281669 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 184195us
I20260709 12:10:10.228719 12526 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:10.294878 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:10.295233 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:10.074410 14204 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:10.102634 14206 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:10.319239 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:10.330035 12673 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:10.101466 14175 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:10.079739 14179 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:10.317233 12571 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:10.306033 12531 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:10:10.172472 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:10.319137 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:10.080214 14172 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:10.323037 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:10.316751 12526 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
W20260709 12:10:10.361380 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 44293.5us
I20260709 12:10:10.362104 12571 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:10.366250 12496 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:10.072000 13951 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:10.286371 12676 rpcz_store.cc:276] 0709 12:10:08.504058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:08.504167 (+   109us) service_pool.cc:224] Handling call
0709 12:10:08.504508 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:09.092620 (+588112us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:09.092669 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:09.126720 (+ 34051us) log.cc:844] Serialized 26 byte log entry
0709 12:10:09.508734 (+382014us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:09.508781 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:10.214546 (+705765us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b54001acf54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:10.214624 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:10.214699 (+    75us) raft_consensus.cc:1685] finished
0709 12:10:10.214722 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:10.215052 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":209407,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":136,"spinlock_wait_cycles":24360192,"thread_start_us":856675,"threads_started":2,"wal-append.queue_time_us":611373}
I20260709 12:10:10.234742 14085 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:10.303663 14203 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:10.344092 12572 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:10.264245 14223 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:10.238721 14139 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:10.308759 14138 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:10.308096 14070 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:10.122210 14195 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:10.213984 14147 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.636s	user 0.009s	sys 0.000s
I20260709 12:10:10.131407 14202 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:10.358583 12525 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:10.079805 14065 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.358608 12680 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:10.366250 12571 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:10.368551 12496 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:10.165495 14012 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:10.371142 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:10.308178 14140 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
I20260709 12:10:10.342584 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:10.383548 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:10.389417 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 105969us
W20260709 12:10:10.434525 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 68369.3us
W20260709 12:10:10.373904 14141 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.718s	user 0.010s	sys 0.000s
W20260709 12:10:10.397364 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 77279.9us
I20260709 12:10:10.444410 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:10.398079 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 54075.3us
I20260709 12:10:10.363530 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:10.339699 14206 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.378309 14179 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:10.374068 14213 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:10.331131 13946 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.057s
I20260709 12:10:10.333344 14204 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.423130 12531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10395d6e75b84c3982521323f92e3309"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
W20260709 12:10:10.427740 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 59294.1us
I20260709 12:10:10.374619 14230 raft_consensus.cc:2764] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:10:10.378065 14217 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:10.382030 13956 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:10.400430 14138 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:10.458447 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:10:10.498678 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 52659.3us
I20260709 12:10:10.398633 14203 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:10.460166 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:10.461040 12523 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:10.458952 13941 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:10.404084 14139 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.432827 14221 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:10.466925 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:10.486030 12529 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1439) took 1772 ms (client timeout 2280 ms). Trace:
I20260709 12:10:10.489683 12531 raft_consensus.cc:606] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:10.372265 14175 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.399106 13951 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.390424 14085 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:10.459684 14036 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:10.513278 12529 rpcz_store.cc:269] 0709 12:10:08.713658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:08.995837 (+282179us) service_pool.cc:224] Handling call
0709 12:10:10.485827 (+1489990us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54003f6e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:10.485957 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2561111808}
I20260709 12:10:10.373827 14172 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.487254 14192 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:10.423871 14147 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.401266 14223 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.429452 14202 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.513558 12531 raft_consensus.cc:614] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:10.525079 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:10.497473 13988 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 12:10:10.566187 12603 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:10.499275 14138 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:10.510566 14233 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
W20260709 12:10:10.549736 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 59224.7us
I20260709 12:10:10.438303 13922 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:10.556793 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 112180us
I20260709 12:10:10.499104 13956 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.571538 12496 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:10.532330 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 94332us
W20260709 12:10:10.537297 14226 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.187s	user 0.001s	sys 0.000s
I20260709 12:10:10.563513 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:10.554976 14020 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:10.538672 14204 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:10.574676 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:10.461517 14141 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.604329 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:10.525992 14057 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:10.524214 13941 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:10.527386 14179 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.075s
W20260709 12:10:10.601068 14214 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.013s	sys 0.003s
I20260709 12:10:10.606036 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:10.052713 13330 rpcz_store.cc:276] 0709 12:10:07.886832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:07.899172 (+ 12340us) service_pool.cc:224] Handling call
0709 12:10:07.966645 (+ 67473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:08.391357 (+424712us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:10:08.494157 (+102800us) log.cc:844] Serialized 14 byte log entry
0709 12:10:09.205189 (+711032us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:09.206988 (+  1799us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:09.212540 (+  5552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:09.910453 (+697913us) spinlock_profiling.cc:243] Waited 364 ms on lock 0x7b54006b0154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:09.916940 (+  6487us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:09.950703 (+ 33763us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":54378,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":83175,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":93748,"spinlock_wait_cycles":837604608,"thread_start_us":667849,"threads_started":1,"wal-append.queue_time_us":1021175}
I20260709 12:10:10.563107 14192 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.605041 14217 raft_consensus.cc:697] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:10.607010 14020 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:10.626067 14234 raft_consensus.cc:3055] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:10.671819 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:10.662801 12646 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:10.671669 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 172049us
I20260709 12:10:10.630064 14213 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:10.652040 12601 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:10.609789 14239 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:10.651269 14204 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:10:10.712827 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 108630us
I20260709 12:10:10.653654 13941 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:10.672592 14116 consensus_meta.cc:287] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.814s	user 0.012s	sys 0.000s
I20260709 12:10:10.716914 12496 leader_election.cc:400] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3.
W20260709 12:10:10.717499 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 54832.3us
I20260709 12:10:10.720222 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:10.722172 12646 leader_election.cc:400] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3.
I20260709 12:10:10.722507 12646 leader_election.cc:403] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:10:10.723323 14195 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:10.723919 14175 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:10.724737 14232 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 12:10:10.686712 14057 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:10.705945 14234 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:10.737778 14234 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:10.739207 14234 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:10.739339 14242 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3.
I20260709 12:10:10.741531 14241 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:10.727103 14116 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:10.743523 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:10.715817 14210 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.824s	user 0.011s	sys 0.001s
I20260709 12:10:10.730386 14065 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:10.730671 14139 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:10.713201 14020 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:10.731675 13951 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:10.732206 14057 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.732115 14217 consensus_queue.cc:237] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.733556 14192 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:10.733855 14227 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
I20260709 12:10:10.728551 14214 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:10:10.734509 14024 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.836s	user 0.012s	sys 0.000s
I20260709 12:10:10.753710 13920 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:10:10.734678 14224 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
W20260709 12:10:10.734838 14236 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:10:10.735437 14133 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.883s	user 0.011s	sys 0.000s
I20260709 12:10:10.759330 14195 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:10.747321 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 141411us
I20260709 12:10:10.731329 14085 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:10.779757 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:10.769249 14203 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:10.783136 14249 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:10.766855 13951 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:10.783790 14249 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:10.721417 14233 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:10.784296 14249 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:10.784024 13951 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.786262 14233 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:10.772799 14133 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:10.787249 14136 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:10.787618 12599 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:10.787703 14116 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:10.730674 14218 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 12:10:10.743784 14172 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:10.788800 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:10.788936 14133 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:10.731671 14223 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:10.758808 14147 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:10.789522 12671 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
I20260709 12:10:10.751394 12646 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:10.790247 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 72) took 1148 ms. Trace:
I20260709 12:10:10.759260 14175 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:10.755012 14222 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:10.770345 14024 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:10.780350 12527 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1422) took 2071 ms (client timeout 1905 ms). Trace:
W20260709 12:10:10.794932 12527 rpcz_store.cc:269] 0709 12:10:08.709029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:08.774718 (+ 65689us) service_pool.cc:224] Handling call
0709 12:10:10.780163 (+2005445us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b54001068d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:10.780290 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1956736384}
I20260709 12:10:10.792604 14085 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:10.796325 14085 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:10.796725 14085 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:10.796726 14024 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:10.753151 14210 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.786758 14233 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:10.758944 14141 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:10.770925 13920 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:10.761257 12523 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:10.766413 14206 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:10.791726 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:10.791828 12646 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:10.790655 12671 rpcz_store.cc:276] 0709 12:10:09.642113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:09.642287 (+   174us) service_pool.cc:224] Handling call
0709 12:10:10.788601 (+1146314us) spinlock_profiling.cc:243] Waited 6.25 ms on lock 0x7b58001f2c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:10:10.790108 (+  1507us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14374784}
I20260709 12:10:10.744549 12572 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:10.773291 14214 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:10.806113 13920 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:10.800477 14175 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.818377 14141 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:10.818856 14141 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:10:10.811147 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 66681.9us
I20260709 12:10:10.769196 14213 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.793071 14195 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.808707 14214 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.811117 12598 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:10.816238 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:10.825376 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:10.817608 12646 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:10.807483 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 63850us
I20260709 12:10:10.814672 14222 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:10.819219 14141 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:10.831151 12527 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:10.820035 13951 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:10.833024 14134 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:10.820709 14012 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:10.828161 14222 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:10.823643 14216 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:10.839529 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:10.840296 12646 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:10.842664 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:10.847360 13333 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:10.842939 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:10.849150 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:10.850445 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:10.851629 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:10.852185 12525 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:10.853379 14249 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.843569 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:10.856242 14247 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:10:10.859043 12646 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:10.865623 14125 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:10.860762 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:10.869228 12525 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:10.859913 12426 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59006 (request call id 173) took 5980 ms. Trace:
I20260709 12:10:10.864419 14143 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:10.861828 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:10.859526 14195 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:10.867183 14210 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:10.861056 14175 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:10.826305 14242 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:10.860201 14222 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:10.835112 14240 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:10:10.903146 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 44179.6us
I20260709 12:10:10.899308 14020 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:10.906180 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:10.916821 13337 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:10.917431 12646 leader_election.cc:400] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3.
I20260709 12:10:10.892099 14250 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:10.910581 14057 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:10.898445 12426 rpcz_store.cc:276] 0709 12:10:04.879427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:04.884690 (+  5263us) service_pool.cc:224] Handling call
0709 12:10:10.859817 (+5975127us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:07.019067 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:07.230261 (+211194us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:07.230309 (+    48us) write_op.cc:435] Acquired schema lock
0709 12:10:07.230335 (+    26us) tablet.cc:662] Decoding operations
0709 12:10:07.253887 (+ 23552us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:07.262683 (+  8796us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:07.272556 (+  9873us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:10:07.333752 (+ 61196us) tablet.cc:701] Row locks acquired
0709 12:10:07.333786 (+    34us) write_op.cc:260] PREPARE: finished
0709 12:10:07.390325 (+ 56539us) write_op.cc:270] Start()
0709 12:10:07.390478 (+   153us) write_op.cc:276] Timestamp: P: 1783599007356498 usec, L: 0
0709 12:10:07.390505 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:10:07.391330 (+   825us) log.cc:844] Serialized 540 byte log entry
0709 12:10:09.199497 (+1808167us) op_driver.cc:464] REPLICATION: finished
0709 12:10:09.644454 (+444957us) write_op.cc:301] APPLY: starting
0709 12:10:09.644551 (+    97us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:09.644669 (+   118us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:09.644819 (+   150us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:09.645116 (+   297us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:09.645295 (+   179us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:10:09.645353 (+    58us) write_op.cc:312] APPLY: finished
0709 12:10:09.645580 (+   227us) log.cc:844] Serialized 22 byte log entry
0709 12:10:09.645793 (+   213us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:09.645918 (+   125us) write_op.cc:454] Released schema lock
0709 12:10:09.645962 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:09.646440 (+   478us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:09.646474 (+    34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":447,"reported_tablets":8,"child_traces":[["op",{"apply.queue_time_us":435481,"apply.run_cpu_time_us":1971,"apply.run_wall_time_us":1968,"num_ops":1,"prepare.queue_time_us":208922,"prepare.run_cpu_time_us":30406,"prepare.run_wall_time_us":790057,"raft.queue_time_us":943323,"raft.run_cpu_time_us":35529,"raft.run_wall_time_us":311978,"replication_time_us":1768417,"thread_start_us":1495793,"threads_started":4,"wal-append.queue_time_us":712045,"wal-append.run_cpu_time_us":32471,"wal-append.run_wall_time_us":2662047}]]}
I20260709 12:10:10.897238 13941 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.913316 14249 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:10.911136 14085 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:10.931301 13941 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:10.917829 12646 leader_election.cc:403] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:10:10.894330 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:10.921849 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:10.914094 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:10.925012 14240 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:10.925132 14138 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.926432 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:10.928555 14233 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:10.932749 13339 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:10.923892 14020 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:10:10.933058 12531 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:10.933317 13337 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:10.905123 14012 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:10.941267 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:10.951037 13333 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:10.951109 14233 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:10.959242 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:11.008881 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 91580.3us
I20260709 12:10:10.958073 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:10.958004 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:11.014668 12646 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:11.015489 12526 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:11.017441 12646 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:11.015884 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:11.018615 12494 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:11.019577 14057 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:11.018711 14085 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:11.019893 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:11.020010 14212 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:11.020964 14212 raft_consensus.cc:697] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:11.021924 14212 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:10.956043 12530 raft_consensus.cc:2376] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4. Re-sending same reply.
I20260709 12:10:10.967211 12531 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:11.023425 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:11.024003 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.026057 12646 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:10.982805 13339 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
W20260709 12:10:11.023865 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 184117us
I20260709 12:10:11.020005 14057 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:10.955226 14238 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 12:10:10.939208 14234 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3.
I20260709 12:10:10.954653 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:11.023272 14240 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:11.036192 12526 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:11.028995 12646 leader_election.cc:400] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3.
I20260709 12:10:11.026603 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:11.028241 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:10.960879 14216 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:10.952786 14138 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:11.026939 14036 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:11.034597 13335 raft_consensus.cc:2376] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3. Re-sending same reply.
I20260709 12:10:11.025596 14085 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:10.952937 13330 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:11.042064 12531 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:11.028367 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:11.026628 12527 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:11.011955 13956 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:11.011718 13920 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:11.042763 12646 leader_election.cc:403] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:10:11.094588 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.095109 12496 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:11.031078 14233 raft_consensus.cc:2764] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:10:11.081648 14216 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:11.037549 14213 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:11.031006 14057 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:11.105407 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:11.037441 14240 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:10:11.106601 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 78835.9us
I20260709 12:10:11.112912 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:10:11.113058 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 84256.2us
I20260709 12:10:11.106391 14125 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:11.102767 14214 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:11.110251 12676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "e56d487365394ac5ac5f1002926a5645"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:10:11.117820 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:11.118553 12671 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 3
W20260709 12:10:11.120023 14255 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:10:11.120958 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:11.121318 14012 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:11.121846 14012 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:11.122360 14012 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:11.123185 14208 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:11.123607 14208 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:11.124961 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1507) took 1261 ms. Trace:
I20260709 12:10:11.125401 12601 rpcz_store.cc:276] 0709 12:10:09.863326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:09.864864 (+  1538us) service_pool.cc:224] Handling call
0709 12:10:09.930379 (+ 65515us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:10.714076 (+783697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:11.124623 (+410547us) spinlock_profiling.cc:243] Waited 673 ms on lock 0x7b5400116cd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:11.124881 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1547179264}
I20260709 12:10:11.114281 14085 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3.
I20260709 12:10:11.127394 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:11.128343 13920 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:11.093689 14207 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260709 12:10:11.113842 14200 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:11.131104 14200 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:11.132091 14200 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:11.111264 14250 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:11.115649 12527 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:11.135008 14214 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:11.121937 13335 raft_consensus.cc:2368] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3.
I20260709 12:10:11.135988 14214 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:11.136793 12643 leader_election.cc:400] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3.
I20260709 12:10:11.137110 12643 leader_election.cc:403] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:10:11.111773 14233 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:11.138684 12679 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:11.138643 14233 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:11.140053 12676 raft_consensus.cc:606] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Received request to transfer leadership to TS e56d487365394ac5ac5f1002926a5645
I20260709 12:10:11.038990 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:11.140528 12676 raft_consensus.cc:614] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:11.141103 13987 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:11.141498 13987 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:11.141918 13987 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:11.142369 14250 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:11.116117 12674 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:11.146615 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 114089us
I20260709 12:10:11.150651 14208 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:11.152225 14208 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:11.154646 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:11.155541 12527 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:11.156721 12671 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:11.114950 12646 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:11.127326 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:11.158982 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 44105.8us
I20260709 12:10:11.117074 14240 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:11.162734 14234 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:11.164399 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.182852 12602 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:11.166513 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.167052 12531 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:11.171403 12674 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:11.172410 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.184930 12676 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:10:11.180563 12646 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:11.141721 14233 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:11.186961 14233 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:10:11.187155 14225 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.616s	user 0.006s	sys 0.000s
I20260709 12:10:11.188464 12646 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:11.189730 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 1368) took 1161 ms. Trace:
I20260709 12:10:11.190199 12523 rpcz_store.cc:276] 0709 12:10:10.027854 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:10.051324 (+ 23470us) service_pool.cc:224] Handling call
0709 12:10:11.189433 (+1138109us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b5400408654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:11.189668 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":924351744}
I20260709 12:10:11.190563 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:11.191628 12646 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:11.110277 14213 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:11.194886 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.208763 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.194886 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:11.209415 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.207804 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1521) took 1279 ms. Trace:
I20260709 12:10:11.211356 12646 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:11.194372 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.210721 12602 rpcz_store.cc:276] 0709 12:10:09.911771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:09.911952 (+   181us) service_pool.cc:224] Handling call
0709 12:10:11.164168 (+1252216us) spinlock_profiling.cc:243] Waited 3.13 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:10:11.190877 (+ 26709us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7190400}
I20260709 12:10:11.212719 14258 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:11.213182 14258 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:11.213442 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.215044 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:11.216065 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:11.216651 12603 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:11.216651 12602 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:11.216817 12679 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:10:11.222985 12676 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:11.224617 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:11.225848 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.231914 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:11.232391 12674 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:11.228510 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.232713 12671 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 2: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:11.225905 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.232561 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:11.189711 14233 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:11.229776 12601 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 2: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:11.184024 14141 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:11.220355 14258 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:11.243927 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.257613 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.258802 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.234207 12527 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:11.265470 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:11.266630 12676 raft_consensus.cc:2368] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4.
I20260709 12:10:11.267704 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.269155 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:11.270735 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.272406 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.273180 12671 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:11.274618 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.275492 12678 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:11.281338 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.283221 12527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:10:11.282215 12674 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:11.284813 12527 raft_consensus.cc:606] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:11.285719 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.286823 12674 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:11.288172 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.289216 12674 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:11.229292 14180 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 12:10:11.243398 14233 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:11.305006 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 61) took 1263 ms. Trace:
W20260709 12:10:11.299719 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 67241.1us
W20260709 12:10:11.308706 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 97525.4us
I20260709 12:10:11.305612 12599 rpcz_store.cc:276] 0709 12:10:10.041718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:10.059564 (+ 17846us) service_pool.cc:224] Handling call
0709 12:10:11.304701 (+1245137us) spinlock_profiling.cc:243] Waited 724 ms on lock 0x7b54001171d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:11.304921 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1665905536}
I20260709 12:10:11.293179 14234 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:11.316841 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.321524 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.324572 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.309175 14233 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:11.326711 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.374681 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1519) took 1479 ms. Trace:
W20260709 12:10:11.359550 12569 connection.cc:256] RPC call timeout handler was delayed by -1.3571804069999871s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2502s for a response
I20260709 12:10:11.374630 14213 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:11.327513 14242 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:11.383453 12605 rpcz_store.cc:276] 0709 12:10:09.896279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:09.896468 (+   189us) service_pool.cc:224] Handling call
0709 12:10:11.374405 (+1477937us) spinlock_profiling.cc:243] Waited 191 ms on lock 0x7b5400376254. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:11.374591 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":438920448}
I20260709 12:10:11.383584 12674 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:11.359431 14256 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:11.369359 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.376753 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:11.384153 12569 connection.cc:256] RPC call timeout handler was delayed by -1.2455548689997613s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3033s for a response
I20260709 12:10:11.348378 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:11.399601 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.402196 12605 raft_consensus.cc:2410] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:11.402397 12569 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:11.382228 14268 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:11.429755 12674 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260709 12:10:11.434418 12674 pending_rounds.cc:85] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 4
I20260709 12:10:11.434728 12674 pending_rounds.cc:107] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:10:11.434986 12674 raft_consensus.cc:2889] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:11.435385 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.352958 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.448551 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.394775 14213 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:11.399736 14183 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:11.386804 14222 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:11.458916 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.459951 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:11.465536 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:11.468465 12605 raft_consensus.cc:2368] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3.
I20260709 12:10:11.462571 12603 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:11.472738 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.473320 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:11.473909 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.474555 13339 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 3: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:11.474892 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.475639 12605 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:11.476048 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.455353 13920 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:11.432008 13333 raft_consensus.cc:2376] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3. Re-sending same reply.
I20260709 12:10:11.477145 12603 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:10:11.463406 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 153197us
W20260709 12:10:11.463124 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 60860.6us
I20260709 12:10:11.479461 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:11.487013 12646 connection.cc:256] RPC call timeout handler was delayed by -1.3604464500001541s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22799999999999998s for a response
I20260709 12:10:11.479944 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:11.488526 12605 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:11.493564 12603 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:11.494850 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 63) took 1429 ms. Trace:
I20260709 12:10:11.495342 12603 rpcz_store.cc:276] 0709 12:10:10.067735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.492799 (+1425064us) service_pool.cc:224] Handling call
0709 12:10:11.493295 (+   496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:11.494472 (+  1177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:11.494772 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:11.499953 12680 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
I20260709 12:10:11.502018 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1529) took 1509 ms. Trace:
I20260709 12:10:11.502660 12604 rpcz_store.cc:276] 0709 12:10:09.992441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:10.006290 (+ 13849us) service_pool.cc:224] Handling call
0709 12:10:11.501748 (+1495458us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b54002e54d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:11.501930 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":32158080}
W20260709 12:10:11.503580 12569 consensus_peers.cc:597] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70b8c9b89e3d43fbb70422110d76702c. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:10:11.512894 12604 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:11.516088 12604 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:11.518980 12600 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:11.460145 14213 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:11.520408 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1509) took 1633 ms. Trace:
I20260709 12:10:11.466173 14183 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:11.524514 12600 rpcz_store.cc:276] 0709 12:10:09.887224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:09.910453 (+ 23229us) service_pool.cc:224] Handling call
0709 12:10:11.518370 (+1607917us) spinlock_profiling.cc:243] Waited 4.55 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:10:11.518681 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:11.520047 (+  1366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:11.520324 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10468608}
I20260709 12:10:11.526634 12600 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:11.505733 14270 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:11.508810 12677 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:11.558280 14222 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:11.558503 14183 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:11.533973 14259 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:10:11.548904 14250 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3.
I20260709 12:10:11.553227 12605 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:11.561702 13331 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: []
W20260709 12:10:11.566663 13305 leader_election.cc:336] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.764s (SENT)
I20260709 12:10:11.555097 14270 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:10:11.559906 14264 log.cc:927] Time spent T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:10:11.575481 12604 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 12:10:11.578804 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 201574us
I20260709 12:10:11.566026 12601 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:11.595721 12605 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:10:11.598098 12605 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:11.605753 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.615226 13987 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:11.652717 13987 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:11.644264 14213 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:11.657955 13330 consensus_meta.cc:287] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.550s	user 0.013s	sys 0.000s
I20260709 12:10:11.659740 13330 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:11.660630 14140 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:11.661278 14140 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:11.662770 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 65156.6us
W20260709 12:10:11.663237 14057 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.550s	user 0.008s	sys 0.003s
I20260709 12:10:11.663715 14057 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:11.664757 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:11.665570 12671 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:11.665905 14057 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:11.667383 14216 consensus_meta.cc:287] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.556s	user 0.012s	sys 0.000s
I20260709 12:10:11.667825 14216 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:11.578878 14270 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:11.670922 14012 consensus_meta.cc:287] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.548s	user 0.011s	sys 0.002s
I20260709 12:10:11.670857 14216 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:11.675094 12676 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:11.683115 12671 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:11.660820 14240 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:11.685598 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:11.679553 14012 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:11.687378 14280 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:11.687933 14280 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:11.688136 14202 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:11.688851 14280 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:11.689456 14208 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.537s	user 0.007s	sys 0.007s
I20260709 12:10:11.692113 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 65) took 1610 ms. Trace:
I20260709 12:10:11.694063 12679 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:11.693925 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1361) took 1672 ms. Trace:
I20260709 12:10:11.704548 14240 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:11.705359 12522 rpcz_store.cc:276] 0709 12:10:10.021691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:10.080404 (+ 58713us) service_pool.cc:224] Handling call
0709 12:10:10.157720 (+ 77316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:10.503117 (+345397us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:10.503146 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:10.503192 (+    46us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:10.503218 (+    26us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:11.693721 (+1190503us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b540038f4d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:11.693765 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:11.693872 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":793871232}
I20260709 12:10:11.706696 14012 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:11.692494 12599 rpcz_store.cc:276] 0709 12:10:10.082178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.492600 (+1410422us) service_pool.cc:224] Handling call
0709 12:10:11.501967 (+  9367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:11.502487 (+   520us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:11.502533 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:11.503205 (+   672us) log.cc:844] Serialized 26 byte log entry
0709 12:10:11.673312 (+170107us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:11.681021 (+  7709us) log.cc:844] Serialized 14 byte log entry
0709 12:10:11.681385 (+   364us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:11.681453 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:11.691919 (+ 10466us) raft_consensus.cc:1685] finished
0709 12:10:11.691938 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:11.692062 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6641,"prepare.queue_time_us":7120,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":4964,"raft.run_cpu_time_us":334,"raft.run_wall_time_us":16803,"thread_start_us":118858,"threads_started":1,"wal-append.queue_time_us":167394}
W20260709 12:10:11.711117 14164 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.915s	user 0.006s	sys 0.037s
I20260709 12:10:11.678320 14233 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:11.722555 12427 catalog_manager.cc:5697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 1 to 3, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 3 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:11.716365 14258 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:11.732267 14234 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:11.741159 14234 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:11.724730 13987 consensus_queue.cc:1059] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.009s
I20260709 12:10:11.743083 14258 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:11.735548 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 68) took 1166 ms. Trace:
W20260709 12:10:11.744290 14262 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.390s	user 0.004s	sys 0.000s
I20260709 12:10:11.743871 12603 rpcz_store.cc:276] 0709 12:10:10.569000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.508234 (+939234us) service_pool.cc:224] Handling call
0709 12:10:11.520968 (+ 12734us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:11.521471 (+   503us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:11.521515 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:11.566676 (+ 45161us) log.cc:844] Serialized 26 byte log entry
0709 12:10:11.642774 (+ 76098us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:11.642841 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:11.642962 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:11.735315 (+ 92353us) raft_consensus.cc:1685] finished
0709 12:10:11.735344 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:11.735487 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":156,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":8949,"thread_start_us":43425,"threads_started":1,"wal-append.queue_time_us":136067}
W20260709 12:10:11.722160 14164 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.923s	user 0.006s	sys 0.038s
I20260709 12:10:11.745754 12427 catalog_manager.cc:5697] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 1 to 2, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 2 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:10:11.757071 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:11.758006 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.758342 14170 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:10:11.758857 12677 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:10:11.759069 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 62) took 1697 ms. Trace:
I20260709 12:10:11.759289 14170 raft_consensus.cc:3055] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:11.759728 14170 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:11.759887 14208 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:11.760414 14208 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:11.760615 14170 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:11.761828 14170 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:11.762638 14208 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:11.763655 14233 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:11.759505 12606 rpcz_store.cc:276] 0709 12:10:10.061401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:10.711349 (+649948us) service_pool.cc:224] Handling call
0709 12:10:10.726794 (+ 15445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:11.189938 (+463144us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:10:11.190249 (+   311us) log.cc:844] Serialized 14 byte log entry
0709 12:10:11.238728 (+ 48479us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:11.238786 (+    58us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:11.238849 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:11.758836 (+519987us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b5400374e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:11.758882 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:11.759012 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1535,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":96,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":91,"spinlock_wait_cycles":1063989632,"thread_start_us":47788,"threads_started":1,"wal-append.queue_time_us":48129}
I20260709 12:10:11.766469 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:11.751058 12427 catalog_manager.cc:5697] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 1 to 2, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 2 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:11.772138 12496 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:11.793119 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:11.794723 12427 catalog_manager.cc:5697] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 1 to 2, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 2 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:10:11.796648 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:11.797389 12427 catalog_manager.cc:5697] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 1 to 3, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 3 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:11.799774 14057 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:11.806242 12496 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:11.803273 13988 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:11.794142 14214 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 12:10:11.805431 14057 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:11.815726 14202 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:11.804711 14208 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260709 12:10:11.833652 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.811734 14140 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:11.832474 12677 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:11.817946 13988 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:11.843586 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.838933 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.845521 12672 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:11.847746 12671 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:10:11.812099 13304 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:11.837270 14164 raft_consensus.cc:3055] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:11.857224 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:11.859238 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:11.861294 12678 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 3: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:11.843182 14214 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:11.867341 14214 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:11.849924 13988 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:11.863150 14057 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:11.863569 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 56475.2us
I20260709 12:10:11.872296 12677 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:11.852324 13330 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: []
W20260709 12:10:11.861271 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 55203.6us
W20260709 12:10:11.859601 14266 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.479s	user 0.002s	sys 0.000s
I20260709 12:10:11.866360 14288 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:11.882840 12496 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:11.876652 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:11.878284 14202 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:11.880797 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.881248 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:11.876487 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:11.885797 12678 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
I20260709 12:10:11.884925 12671 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:11.868623 14214 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:11.889592 14288 raft_consensus.cc:697] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 3 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:11.889948 14214 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:11.890973 14288 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:11.893343 12530 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 69) took 1908 ms (client timeout 1764 ms). Trace:
W20260709 12:10:11.893792 12530 rpcz_store.cc:269] 0709 12:10:09.984637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:09.984852 (+   215us) service_pool.cc:224] Handling call
0709 12:10:11.893135 (+1908283us) spinlock_profiling.cc:243] Waited 970 ms on lock 0x7b540027fbd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:11.893268 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2230845184}
I20260709 12:10:11.898451 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:11.899747 12678 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 3: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:11.870010 14164 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:11.912889 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:11.902110 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:11.913057 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:11.902473 12643 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:11.889681 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:11.907169 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:11.910857 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:11.913914 12525 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:11.916536 12527 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
W20260709 12:10:11.915664 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:11.916540 14132 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.119s	user 0.000s	sys 0.011s
W20260709 12:10:11.918612 14132 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.121s	user 0.000s	sys 0.012s
I20260709 12:10:11.919530 13337 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 3 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:11.920918 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:11.921864 12527 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:11.929315 14131 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:11.930310 14131 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:11.932847 12527 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:11.937261 14131 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:11.941188 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:11.881759 14245 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:11.942437 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:11.942860 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:11.943502 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:11.943958 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:11.944533 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:11.939405 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:11.947454 12522 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:11.947824 12680 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:11.894788 14208 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:11.950716 12605 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:11.951428 12529 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:11.944527 12524 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:11.949327 12572 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:11.949539 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:11.949744 12527 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:11.953397 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:11.954726 12572 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:11.953523 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:11.949863 12671 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:11.958741 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:11.960760 12643 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:11.963654 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:11.958976 12605 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
mode: GRACEFUL
new_leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
 from {username='slave'} at 127.0.0.1:32832
I20260709 12:10:11.964416 12605 raft_consensus.cc:606] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Received request to transfer leadership to TS a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:11.964717 12605 raft_consensus.cc:614] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:11.966315 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:11.967648 12523 raft_consensus.cc:2410] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 5 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:10:11.893780 14202 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:11.970721 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:11.945864 14233 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:11.948804 14252 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:10:11.946249 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 56676.1us
I20260709 12:10:11.988056 12524 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:11.953544 14208 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:10:11.980237 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 95204us
I20260709 12:10:11.965690 13918 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:11.939348 14292 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.107s
I20260709 12:10:11.957633 14174 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:12.056670 12496 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:10:12.048974 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 78328.5us
I20260709 12:10:11.982484 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:11.959489 14012 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:11.958772 14132 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:11.980368 12680 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:11.969491 14240 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:12.056056 14208 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:11.982811 14138 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:11.950773 14245 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:12.041285 14057 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:12.062436 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 80042.9us
I20260709 12:10:12.044416 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:11.979179 14222 consensus_queue.cc:1059] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.083s
I20260709 12:10:12.065688 12672 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:12.048213 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:12.059903 14296 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:11.980707 14249 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:12.055102 14271 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:11.979803 14242 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.088s
I20260709 12:10:11.979995 14275 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:12.079573 12680 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "e56d487365394ac5ac5f1002926a5645"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:10:11.965296 14216 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.099s
I20260709 12:10:12.062774 14132 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:12.086565 12601 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:12.094800 14242 raft_consensus.cc:3055] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:12.095348 14242 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:12.100010 14240 raft_consensus.cc:3055] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:12.100498 14240 raft_consensus.cc:740] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:12.080500 14292 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:12.101516 14292 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:12.103798 14292 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:12.104331 14292 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:12.104684 14292 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:12.105108 14216 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:10:12.105455 14261 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.772s	user 0.000s	sys 0.003s
I20260709 12:10:12.106976 14202 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:12.108042 14297 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:12.110348 14297 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:12.111032 14297 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:12.108564 12601 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:12.114964 12672 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:12.124872 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1383) took 1283 ms. Trace:
I20260709 12:10:12.089177 13956 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:12.046664 14164 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:12.125330 12526 rpcz_store.cc:276] 0709 12:10:10.841665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.037461 (+195796us) service_pool.cc:224] Handling call
0709 12:10:11.041592 (+  4131us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:11.043527 (+  1935us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:11.043567 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:11.096928 (+ 53361us) log.cc:844] Serialized 26 byte log entry
0709 12:10:11.225716 (+128788us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:11.225752 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:11.225815 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:12.104752 (+878937us) spinlock_profiling.cc:243] Waited 2.69 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:12.113388 (+  8636us) raft_consensus.cc:1685] finished
0709 12:10:12.113412 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:12.124795 (+ 11383us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1503,"raft.queue_time_us":1639,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":9221,"spinlock_wait_cycles":6176000,"thread_start_us":114326,"threads_started":1,"wal-append.queue_time_us":188195}
I20260709 12:10:12.134111 14164 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:12.137235 14294 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:10:12.134270 13956 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:12.140875 12673 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:12.140293 13956 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:12.144223 14287 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.290s	user 0.001s	sys 0.000s
I20260709 12:10:12.121636 14275 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:12.151763 14275 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:12.123289 14234 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:12.160388 14220 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:10:12.161233 14220 consensus_queue.cc:1243] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 4.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:10:12.161666 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:12.162559 13330 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:12.163857 14220 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:12.164229 14220 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:12.120786 14132 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:12.166388 12680 raft_consensus.cc:606] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Received request to transfer leadership to TS e56d487365394ac5ac5f1002926a5645
I20260709 12:10:12.170482 12680 raft_consensus.cc:614] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:12.153946 13337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 323) took 1141 ms. Trace:
I20260709 12:10:12.145951 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1522) took 2193 ms. Trace:
I20260709 12:10:12.172508 12598 rpcz_store.cc:276] 0709 12:10:09.952143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:09.953773 (+  1630us) service_pool.cc:224] Handling call
0709 12:10:09.955133 (+  1360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:10.824430 (+869297us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:12.145672 (+1321242us) spinlock_profiling.cc:243] Waited 856 ms on lock 0x7b540007ba54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:12.145863 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1968124160}
I20260709 12:10:12.157900 14141 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:12.170110 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 91258.7us
W20260709 12:10:12.175247 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 1.06823e+06us
I20260709 12:10:12.171582 13337 rpcz_store.cc:276] 0709 12:10:11.012984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.014789 (+  1805us) service_pool.cc:224] Handling call
0709 12:10:11.015099 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:11.920124 (+905025us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:12.153712 (+233588us) spinlock_profiling.cc:243] Waited 904 ms on lock 0x7b54004439d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:12.153882 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2079503104}
I20260709 12:10:12.177942 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:12.176309 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:12.177028 14217 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 12:10:12.105552 14292 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:12.183550 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:12.182605 12494 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:12.082368 14057 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:12.184633 13337 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:12.187307 12572 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:12.188050 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:12.189409 12601 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:12.190639 14292 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:12.190737 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:12.188388 12572 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:12.193544 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:12.195281 14141 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:12.195894 14141 raft_consensus.cc:697] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:12.196873 14141 consensus_queue.cc:237] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:12.200466 13335 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:12.200678 13988 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:12.166846 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:12.201751 14245 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:12.165205 14220 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:12.202379 14245 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:12.202816 14245 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:12.203039 14220 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:12.203238 14245 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:12.187127 14200 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:12.088450 14296 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:12.184759 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:12.205993 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:12.200320 14213 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:12.185904 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:12.212684 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:12.215453 12494 leader_election.cc:400] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3.
I20260709 12:10:12.218400 12494 leader_election.cc:403] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:10:12.226065 12680 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e15d21db797e4a96b5732711a5379980"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "e56d487365394ac5ac5f1002926a5645"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:10:12.231782 13337 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:12.215778 14271 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:12.216368 14200 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:12.228123 14202 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3.
I20260709 12:10:12.216216 14268 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 12:10:12.226106 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:12.228346 14249 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:12.256557 12680 raft_consensus.cc:606] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Received request to transfer leadership to TS e56d487365394ac5ac5f1002926a5645
I20260709 12:10:12.291484 12680 raft_consensus.cc:614] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:12.302383 13337 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:10:12.235313 14012 consensus_queue.cc:1059] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
I20260709 12:10:12.290782 14200 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:12.289789 13918 raft_consensus.cc:697] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:12.290385 14138 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:10:12.361060 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 304448us
W20260709 12:10:12.356858 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:12.363340 12496 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:12.365808 12496 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:12.367348 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:12.257022 14214 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.064s
I20260709 12:10:12.306411 14202 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 12:10:12.306629 14249 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:12.299717 14302 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:12.313850 14174 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:12.360316 13988 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:12.391552 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:12.378309 14301 consensus_queue.cc:1059] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.073s
I20260709 12:10:12.424500 12531 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:12.449419 12673 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:12.443189 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:12.385171 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 154029us
I20260709 12:10:12.439144 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:12.437592 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:12.453540 12675 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:12.457759 12522 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
I20260709 12:10:12.401818 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:12.389052 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:12.462329 12530 raft_consensus.cc:2368] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3.
I20260709 12:10:12.464550 12528 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:12.462951 12526 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:12.464847 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:12.466374 12529 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
W20260709 12:10:12.479228 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 113476us
I20260709 12:10:12.480064 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:12.484203 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:12.480126 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:12.484973 12646 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:12.486785 12646 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:12.488169 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:12.454916 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:12.388314 14217 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:12.493541 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:12.494861 12671 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:12.496150 12646 leader_election.cc:400] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3.
I20260709 12:10:12.386993 14302 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:12.498409 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:12.373016 14213 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.101s
I20260709 12:10:12.378037 14288 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.101s
I20260709 12:10:12.386461 13956 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.107s
I20260709 12:10:12.508209 12522 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:12.498981 14268 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:12.503831 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:12.518600 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:12.562999 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:12.503557 14233 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:10:12.561257 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 191899us
I20260709 12:10:12.489435 14131 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.074s
I20260709 12:10:12.509528 12646 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:12.569317 12572 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:12.509860 14057 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:12.497431 14267 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 12:10:12.438428 14249 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:12.574476 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:12.476689 14208 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:12.572775 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 63380us
I20260709 12:10:12.573378 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:12.617244 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:12.617900 12531 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
W20260709 12:10:12.624342 14306 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:10:12.624092 14302 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:12.626176 12525 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:12.623183 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:12.500408 13918 consensus_queue.cc:237] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:12.630535 12531 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:12.569451 14296 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:12.631470 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:12.624342 14300 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
I20260709 12:10:12.637040 12522 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:12.637395 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:12.635901 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 180334us
I20260709 12:10:12.618800 14252 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:12.571270 14233 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:12.509231 14200 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:12.572466 13988 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:10:12.639194 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 159152us
I20260709 12:10:12.562736 14138 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:12.561311 14174 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:12.314978 14303 raft_consensus.cc:2955] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:10:12.625623 14208 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:12.644088 12496 leader_election.cc:400] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4.
I20260709 12:10:12.628872 14271 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:12.572747 14318 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:12.649114 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:12.695874 12528 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:12.564364 14305 raft_consensus.cc:3055] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:12.575794 14297 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:12.648582 12496 leader_election.cc:403] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260709 12:10:12.651785 14263 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.262s	user 0.002s	sys 0.002s
I20260709 12:10:12.652489 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:12.654481 14288 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:12.646584 14213 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:12.656929 14309 ts_tablet_manager.cc:933] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:12.642391 14233 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:12.660115 14057 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:12.660681 14132 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:12.666888 14321 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:12.668292 12531 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
W20260709 12:10:12.670157 14308 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 12:10:12.670534 13988 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:10:12.671640 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:12.675500 12530 raft_consensus.cc:2410] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:12.676380 12522 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:12.684008 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:12.687525 12524 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:12.631510 14313 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:12.571275 14312 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:10:12.707697 13305 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.693s (SENT) [suppressed 34 similar messages]
I20260709 12:10:12.716324 12523 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:10:12.725245 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 81289.4us
I20260709 12:10:12.693923 14131 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:10:12.729733 14220 consensus_meta.cc:287] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.526s	user 0.012s	sys 0.000s
I20260709 12:10:12.698983 14217 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:12.698769 14305 raft_consensus.cc:740] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:12.697566 14271 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:12.787912 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 116336us
I20260709 12:10:12.788388 12529 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:12.788728 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:12.786696 12496 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:12.791381 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 222158us
I20260709 12:10:12.795918 12496 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:12.781606 14314 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:10:12.711161 14318 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:12.806926 12496 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:12.802276 12523 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:12.738780 14213 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:12.802732 12531 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:12.793841 14321 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:12.811174 12496 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:12.790354 14131 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:12.798966 12524 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:12.792172 14214 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:12.732579 14319 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4.
I20260709 12:10:12.794392 14296 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:12.804874 12572 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:12.797452 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 223105us
I20260709 12:10:12.726552 14288 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:12.828774 12524 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:12.829712 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:12.793336 14217 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:12.832257 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 124628us
I20260709 12:10:12.810864 14316 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:12.826370 14305 consensus_queue.cc:260] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:12.836396 13335 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:12.837648 14213 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:12.838572 14214 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:10:12.838820 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:12.839185 14297 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:12.836531 14305 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:12.837718 14288 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:12.841737 14318 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:12.826645 14245 consensus_meta.cc:287] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.623s	user 0.014s	sys 0.000s
I20260709 12:10:12.817068 14313 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:12.846828 14132 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:12.852398 12496 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:12.822428 14131 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:12.826647 14321 raft_consensus.cc:697] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:12.896401 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 44181.1us
I20260709 12:10:12.874233 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:12.901161 12496 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:12.902777 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:12.904193 12496 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:12.906597 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52236.8us
I20260709 12:10:12.845623 14057 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:12.832883 14200 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:12.903304 12523 raft_consensus.cc:2955] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:10:12.856187 14317 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:12.853999 14313 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:12.856797 14328 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:12.908576 14316 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:12.899606 14249 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:12.911348 14214 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:12.917196 14057 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:12.958640 14312 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:12.863457 14245 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:12.976744 14317 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:10:13.052649 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:13.095356 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 137965us
I20260709 12:10:12.995599 14316 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:12.988664 14313 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:13.099669 13305 leader_election.cc:336] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 2.085s (SENT)
I20260709 12:10:13.135824 13305 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:12.968695 14200 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:13.141474 14200 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:13.074981 12528 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:10:12.983436 14202 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.147s
I20260709 12:10:13.138023 14309 tablet_copy_client.cc:323] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.130:39365
I20260709 12:10:13.138635 14328 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:13.139261 14141 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:13.145920 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 46310us
I20260709 12:10:13.000308 14214 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:13.148347 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 84306.9us
I20260709 12:10:13.003235 14057 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:13.012791 14312 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:13.149663 12496 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:13.094893 14317 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:13.135906 14316 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:13.162353 14312 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:13.169698 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 262164us
I20260709 12:10:13.147432 14288 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:13.011199 14249 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:13.177284 14320 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:13.211114 14320 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:13.170552 14141 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:13.212466 14249 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:13.161523 14214 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:13.213229 14320 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:13.213490 14141 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:13.206367 14200 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:13.216423 14200 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:13.207000 14271 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:13.217235 14200 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:13.177366 14339 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.048s
W20260709 12:10:13.222642 14327 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.239s	user 0.000s	sys 0.000s
I20260709 12:10:13.223590 14214 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:13.098086 14321 consensus_queue.cc:237] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:13.225733 14245 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:13.226111 14214 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:13.226559 12528 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:10:13.232211 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:13.233600 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:13.216480 14270 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:13.227211 14303 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.566s	user 0.012s	sys 0.000s
I20260709 12:10:13.238471 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:13.242894 12528 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:13.246304 14271 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:13.250547 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 81927.6us
I20260709 12:10:13.260686 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:13.261451 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:13.261440 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:13.262388 12675 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:13.262549 12678 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:13.267815 12678 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:10:13.246178 14334 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.085s	user 0.000s	sys 0.003s
I20260709 12:10:13.245893 14217 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:13.280285 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:13.281713 12672 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 3: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:13.235333 14305 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:10:13.285460 12679 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:13.287427 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:13.287345 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:13.237388 14317 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:13.252694 13920 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.040s
I20260709 12:10:13.253973 14271 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:13.325414 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 117669us
I20260709 12:10:13.329603 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:13.333626 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:13.323065 14233 consensus_meta.cc:287] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.577s	user 0.012s	sys 0.001s
I20260709 12:10:13.337234 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:13.343403 14335 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:10:13.348562 14234 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:13.345785 12571 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:13.358700 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:13.361537 12672 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:13.342355 14174 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:13.362920 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:39924 (request call id 14) took 1159 ms. Trace:
I20260709 12:10:13.364329 12680 rpcz_store.cc:276] 0709 12:10:12.200472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:12.210254 (+  9782us) service_pool.cc:224] Handling call
0709 12:10:13.359171 (+1148917us) spinlock_profiling.cc:243] Waited 29.6 ms on lock 0x7b54001acf54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1a853 00007f148d9e860a 00007f1486f6e7d4 00007f1486f6e762 00007f1486f6e691 00007f1486f6e60c 00007f1486f6d8c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:13.359310 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68088704}
I20260709 12:10:13.340752 14305 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:13.342545 14303 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:48482 (request call id 3) took 5613 ms. Trace:
I20260709 12:10:13.380087 12680 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:13.341710 14233 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:13.378194 14249 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:13.368264 14275 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:13.392326 14132 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:13.400081 12680 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:10:13.365430 14328 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:13.354548 13956 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.028s
I20260709 12:10:13.378605 14234 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:13.410779 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:13.419628 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:13.423064 12571 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:13.425856 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:13.429767 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:13.430042 12496 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:13.433243 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:13.436412 12672 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:13.439275 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:13.440409 12675 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:13.453500 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:10:13.451315 14320 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:13.415681 14245 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:13.457971 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:13.437741 14297 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:13.407369 14132 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:13.404462 14249 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:13.404400 14275 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:13.403378 14353 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.061s
I20260709 12:10:13.409849 14305 consensus_queue.cc:1059] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 12:10:13.440019 14328 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:13.500499 12675 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:13.497946 14180 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:13.516103 12598 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:13.543030 14346 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:13.448453 13988 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:13.543723 13988 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:13.543792 14347 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.093s
I20260709 12:10:13.544473 13988 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:13.415398 14234 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:13.546234 14352 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 12:10:13.547232 14351 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:10:13.547976 14348 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:13.534556 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:13.550235 14322 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:13.551460 14322 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:13.554332 14349 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
I20260709 12:10:13.554267 14234 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:13.537889 12597 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:13.557418 14234 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:13.558182 14234 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:13.538367 14271 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:13.511039 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:13.516868 12605 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:10:13.559870 14271 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:13.560451 14271 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:10:13.562693 14347 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:13.555593 14322 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:13.449786 14170 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.084s
I20260709 12:10:13.568452 12679 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:13.537623 12606 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:13.455143 14202 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.085s
W20260709 12:10:13.574180 14350 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:10:13.574563 14217 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:13.456555 14296 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.086s
I20260709 12:10:13.590445 14242 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:13.591817 14242 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:13.563896 14234 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:13.575138 14345 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.211s	user 0.000s	sys 0.002s
I20260709 12:10:13.542613 14180 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:13.534986 14141 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.049s
I20260709 12:10:13.528121 14297 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:13.581771 14359 consensus_queue.cc:1059] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:10:13.371274 14303 rpcz_store.cc:276] 0709 12:10:07.717768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:07.721446 (+  3678us) service_pool.cc:224] Handling call
0709 12:10:08.000001 (+278555us) log.cc:844] Serialized 506 byte log entry
0709 12:10:09.215921 (+1215920us) spinlock_profiling.cc:243] Waited 5.79 ms on lock 0x7b18000b1ac0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488979b0b 00007f14889feb63 00007f1488a26752 00007f148d9e6d29
0709 12:10:13.329853 (+4113932us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":487616,"raft.queue_time_us":609670,"raft.run_cpu_time_us":3050,"raft.run_wall_time_us":340936,"spinlock_wait_cycles":13308160,"thread_start_us":565685,"threads_started":1,"wal-append.queue_time_us":992341,"wal-append.run_cpu_time_us":1930,"wal-append.run_wall_time_us":1003695}
I20260709 12:10:13.612869 14132 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:13.606146 14271 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:13.613996 14354 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.156s	user 0.000s	sys 0.000s
I20260709 12:10:13.582361 14217 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:13.613664 14360 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:13.623339 14360 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:13.615880 14297 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:13.693440 14313 consensus_meta.cc:287] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.536s	user 0.011s	sys 0.000s
I20260709 12:10:13.693984 14313 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:13.567327 14240 consensus_queue.cc:260] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:13.699653 14240 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:13.609520 14200 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:10:13.708518 14200 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:13.691012 14316 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:13.712113 14316 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:13.689752 14360 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:13.713285 14249 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:13.714346 14180 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:13.716091 14180 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:13.727837 14275 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:13.729631 14217 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:13.742311 13988 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:13.745512 13988 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:13.753872 14270 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.519s	user 0.008s	sys 0.006s
W20260709 12:10:13.755021 14057 consensus_meta.cc:287] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.583s	user 0.013s	sys 0.001s
I20260709 12:10:13.792894 14057 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:13.797191 14057 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:13.758363 14348 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:10:13.763302 14131 consensus_meta.cc:287] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.845s	user 0.014s	sys 0.000s
I20260709 12:10:13.798926 14131 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:13.802237 14131 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:13.766139 14312 consensus_meta.cc:287] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.531s	user 0.010s	sys 0.001s
I20260709 12:10:13.803184 14312 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:13.806423 14312 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:13.698228 14313 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:13.825274 14222 raft_consensus.cc:3055] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:13.825829 13946 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:13.826097 14222 raft_consensus.cc:740] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:13.826651 13946 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:13.827471 14222 consensus_queue.cc:260] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:13.840494 14222 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:13.845538 14325 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.007s	user 0.000s	sys 0.007s
I20260709 12:10:13.848040 14307 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:13.848814 14307 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:13.850066 14307 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:13.863322 14307 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:13.836534 14317 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:13.838321 14233 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:13.834968 14245 raft_consensus.cc:3055] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:13.865541 14245 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:13.867676 14245 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:13.869510 14245 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:13.917393 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:13.921136 12527 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:13.923442 12523 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:10:13.925750 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:13.934711 12601 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:13.938573 12603 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:13.956682 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1463) took 1532 ms. Trace:
I20260709 12:10:13.959564 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1472) took 1492 ms. Trace:
W20260709 12:10:13.965830 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:13.976155 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1408) took 2051 ms. Trace:
I20260709 12:10:13.983536 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1421) took 1133 ms. Trace:
I20260709 12:10:13.985997 12523 rpcz_store.cc:276] 0709 12:10:12.850336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:13.160011 (+309675us) service_pool.cc:224] Handling call
0709 12:10:13.163063 (+  3052us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:13.924129 (+761066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:13.983032 (+ 58903us) spinlock_profiling.cc:243] Waited 759 ms on lock 0x7b5400407c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:13.983430 (+   398us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1745988992}
I20260709 12:10:13.832487 13918 raft_consensus.cc:3055] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:13.828063 13946 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:13.857070 14132 raft_consensus.cc:3055] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:14.094563 14132 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:14.096709 13918 raft_consensus.cc:740] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:13.837342 14140 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:14.098596 14140 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:14.098984 13918 consensus_queue.cc:260] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:13.976732 12525 rpcz_store.cc:276] 0709 12:10:11.927420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.928133 (+   713us) service_pool.cc:224] Handling call
0709 12:10:11.928491 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:12.626820 (+698329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:13.975397 (+1348577us) spinlock_profiling.cc:243] Waited 697 ms on lock 0x7b54003c0d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:13.975926 (+   529us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1603629312}
I20260709 12:10:14.059885 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:10:14.101749 12674 raft_consensus.cc:2410] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:13.973946 12531 rpcz_store.cc:276] 0709 12:10:12.467264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:12.738657 (+271393us) service_pool.cc:224] Handling call
0709 12:10:12.739019 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:12.810241 (+ 71222us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:13.958909 (+1148668us) spinlock_profiling.cc:243] Waited 63.5 ms on lock 0x7b5400053054. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:13.959349 (+   440us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":146102656}
I20260709 12:10:13.973471 12530 rpcz_store.cc:276] 0709 12:10:12.423585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:12.464504 (+ 40919us) service_pool.cc:224] Handling call
0709 12:10:13.956028 (+1491524us) spinlock_profiling.cc:243] Waited 202 ms on lock 0x7b54003cafd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:13.956392 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":463650048}
I20260709 12:10:14.103343 14339 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:14.104763 14339 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:14.105348 14292 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:14.106580 14292 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:14.093444 14143 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:14.101058 13918 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:14.114331 14057 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:14.114565 14318 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:14.121696 12429 catalog_manager.cc:5697] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 1 to 2, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:10:14.122282 13956 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:10:14.129739 12674 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1378) took 2287 ms (client timeout 1758 ms). Trace:
I20260709 12:10:14.133003 13946 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:14.137858 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:14.138888 14359 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:14.103471 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:14.101051 14140 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:14.146966 14143 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:14.097208 14132 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:14.143281 12674 rpcz_store.cc:269] 0709 12:10:11.842161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:11.842515 (+   354us) service_pool.cc:224] Handling call
0709 12:10:14.129239 (+2286724us) spinlock_profiling.cc:243] Waited 2.2 s on lock 0x7b54001e2154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:14.129562 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5059172736}
I20260709 12:10:14.156204 14140 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:14.156039 13331 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:14.199596 13956 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:14.202458 14057 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:14.207019 14318 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:14.211889 14143 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:14.208086 14240 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:14.206111 14057 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:14.211963 14318 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:14.230067 14132 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:14.231576 12427 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47524 (request call id 92) took 7108 ms. Trace:
I20260709 12:10:14.213097 14240 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:14.237305 14240 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:14.238772 12429 catalog_manager.cc:5697] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 1 to 2, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
W20260709 12:10:14.250240 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:14.207267 14359 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:14.234292 14292 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:14.256935 12494 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:14.259541 14319 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:14.259989 14319 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:14.260382 14319 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:14.271301 13331 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:10:14.274017 14339 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:14.279801 14292 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:14.288341 13339 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:10:14.208963 13956 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:14.241178 12429 catalog_manager.cc:5697] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 1 to 2, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:14.299796 14359 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:14.309693 14057 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.015s
W20260709 12:10:14.333711 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:14.340234 14320 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.851s	user 0.040s	sys 0.118s
W20260709 12:10:14.340849 14320 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.852s	user 0.040s	sys 0.118s
I20260709 12:10:14.341219 14320 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:14.342819 14320 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:14.423353 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:14.374626 12429 catalog_manager.cc:5697] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 2 to 4, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 4 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:14.232149 12427 rpcz_store.cc:276] 0709 12:10:07.170716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:07.183982 (+ 13266us) service_pool.cc:224] Handling call
0709 12:10:14.231231 (+7047249us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:12.044698 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:12.125151 (+ 80453us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:12.125206 (+    55us) write_op.cc:435] Acquired schema lock
0709 12:10:12.125254 (+    48us) tablet.cc:662] Decoding operations
0709 12:10:12.147387 (+ 22133us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:12.147472 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:12.147502 (+    30us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:10:12.148070 (+   568us) tablet.cc:701] Row locks acquired
0709 12:10:12.148091 (+    21us) write_op.cc:260] PREPARE: finished
0709 12:10:12.148238 (+   147us) write_op.cc:270] Start()
0709 12:10:12.148380 (+   142us) write_op.cc:276] Timestamp: P: 1783599012148191 usec, L: 0
0709 12:10:12.148410 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 12:10:12.149282 (+   872us) log.cc:844] Serialized 2068 byte log entry
0709 12:10:12.913292 (+764010us) op_driver.cc:464] REPLICATION: finished
0709 12:10:13.340896 (+427604us) write_op.cc:301] APPLY: starting
0709 12:10:13.343619 (+  2723us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:13.343874 (+   255us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:13.344075 (+   201us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:13.387554 (+ 43479us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:13.387766 (+   212us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=5
0709 12:10:13.387818 (+    52us) write_op.cc:312] APPLY: finished
0709 12:10:13.388076 (+   258us) log.cc:844] Serialized 46 byte log entry
0709 12:10:13.388273 (+   197us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:13.388401 (+   128us) write_op.cc:454] Released schema lock
0709 12:10:13.388464 (+    63us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:13.388897 (+   433us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:13.388932 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":71338,"reported_tablets":6,"child_traces":[["op",{"apply.queue_time_us":407636,"apply.run_cpu_time_us":3853,"apply.run_wall_time_us":57950,"num_ops":5,"prepare.queue_time_us":148413,"prepare.run_cpu_time_us":15344,"prepare.run_wall_time_us":408768,"raft.queue_time_us":190932,"raft.run_cpu_time_us":36721,"raft.run_wall_time_us":443731,"replication_time_us":684952,"thread_start_us":639912,"threads_started":4,"wal-append.queue_time_us":316996}]]}
I20260709 12:10:14.480777 14208 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:10:14.481539 13920 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.868s	user 0.036s	sys 0.062s
W20260709 12:10:14.482076 13920 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.977s	user 0.037s	sys 0.062s
I20260709 12:10:14.483619 14307 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:14.484525 14307 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:14.486186 12671 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.901s	user 0.003s	sys 0.028s
W20260709 12:10:14.486719 12671 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.957s	user 0.003s	sys 0.028s
I20260709 12:10:14.488062 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1555) took 1119 ms. Trace:
W20260709 12:10:14.490324 12496 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.906s	user 0.009s	sys 0.016s
W20260709 12:10:14.490869 12496 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.992s	user 0.009s	sys 0.017s
W20260709 12:10:14.491623 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:14.492236 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 1.03431e+06us
W20260709 12:10:14.494240 13304 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.907s	user 0.012s	sys 0.043s
W20260709 12:10:14.494836 13304 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.956s	user 0.013s	sys 0.043s
W20260709 12:10:14.495687 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 956479us
I20260709 12:10:14.498153 14288 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:10:14.499006 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:14.499805 14359 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:14.500490 14359 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:10:14.499955 12646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.909s	user 0.010s	sys 0.023s
W20260709 12:10:14.503186 12646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.974s	user 0.011s	sys 0.024s
I20260709 12:10:14.506338 14359 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:14.508831 14359 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:14.499867 14222 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:14.512081 14222 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:14.519996 12572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.007s	user 0.029s	sys 0.052s
W20260709 12:10:14.526361 12572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.014s	user 0.029s	sys 0.053s
I20260709 12:10:14.529146 14208 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:14.529995 14208 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:14.539666 12530 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
W20260709 12:10:14.540850 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:14.478783 12673 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.873s	user 0.036s	sys 0.072s
I20260709 12:10:14.470955 14057 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:14.488612 12671 rpcz_store.cc:276] 0709 12:10:13.368884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:13.382658 (+ 13774us) service_pool.cc:224] Handling call
0709 12:10:13.404539 (+ 21881us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:13.404794 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:13.529601 (+124807us) log.cc:844] Serialized 14 byte log entry
0709 12:10:14.487348 (+957747us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:14.487480 (+   132us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:14.487562 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:14.487693 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:14.487937 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":137050,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"thread_start_us":1068248,"threads_started":2}
W20260709 12:10:14.549569 12673 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.054s	user 0.036s	sys 0.072s
I20260709 12:10:14.537590 14132 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.010s
W20260709 12:10:14.561497 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 67340us
W20260709 12:10:14.551074 12644 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.956s	user 0.010s	sys 0.026s
W20260709 12:10:14.565351 12644 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.136s	user 0.011s	sys 0.026s
I20260709 12:10:14.566767 14364 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:14.556074 12646 connection.cc:256] RPC call timeout handler was delayed by -1.0935794199999691s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2142s for a response
I20260709 12:10:14.574483 14164 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:14.574693 14373 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:14.576082 14373 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:14.565099 12677 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.024s	user 0.004s	sys 0.021s
I20260709 12:10:14.567598 14364 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:14.582293 13338 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:14.582199 14307 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:14.589221 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1551) took 1287 ms. Trace:
I20260709 12:10:14.582363 14292 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:14.598327 12677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.187s	user 0.005s	sys 0.021s
I20260709 12:10:14.582715 14338 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:14.611063 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:14.611609 14338 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:14.613385 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:14.606005 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:14.614028 14338 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:14.585785 14057 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:14.610855 14371 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:14.622915 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:14.614168 14214 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.000s	sys 0.030s
W20260709 12:10:14.614679 14315 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.956s	user 0.004s	sys 0.027s
I20260709 12:10:14.603775 12673 rpcz_store.cc:276] 0709 12:10:13.301816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:13.301989 (+   173us) service_pool.cc:224] Handling call
0709 12:10:13.302509 (+   520us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:13.302818 (+   309us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:13.366641 (+ 63823us) log.cc:844] Serialized 14 byte log entry
0709 12:10:14.572816 (+1206175us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:14.572949 (+   133us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:14.573072 (+   123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:14.584107 (+ 11035us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:14.589075 (+  4968us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2590,"prepare.queue_time_us":150129,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":4580,"raft.queue_time_us":80741,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":276,"thread_start_us":1237851,"threads_started":3,"wal-append.queue_time_us":243803}
I20260709 12:10:14.623713 14222 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:14.651145 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:14.649729 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:14.649024 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:14.646968 14315 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.031s	user 0.004s	sys 0.028s
W20260709 12:10:14.682665 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:14.647186 14214 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.820s	user 0.000s	sys 0.031s
W20260709 12:10:14.681636 12600 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.086s	user 0.026s	sys 0.089s
I20260709 12:10:14.646530 14240 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:10:14.680419 12570 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.053s	user 0.008s	sys 0.043s
I20260709 12:10:14.668485 14348 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:14.669281 14373 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:14.691831 12570 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.196s	user 0.009s	sys 0.043s
W20260709 12:10:14.689595 12600 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.180s	user 0.028s	sys 0.089s
I20260709 12:10:14.694303 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:14.692312 13946 consensus_meta.cc:287] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.517s	user 0.016s	sys 0.000s
I20260709 12:10:14.700255 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:14.701362 12526 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:14.680351 14222 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:14.718462 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:14.675463 14292 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:10:14.693922 14138 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.770s	user 0.024s	sys 0.023s
W20260709 12:10:14.723690 14138 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.893s	user 0.024s	sys 0.024s
I20260709 12:10:14.696729 14371 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:14.702772 14240 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:14.726231 14321 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.095s	user 0.011s	sys 0.026s
I20260709 12:10:14.726147 14057 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:14.726745 12525 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:14.726889 14373 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:14.727458 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:14.728600 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:14.728472 14319 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:14.729460 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:14.724711 14348 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:14.749151 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:14.716269 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:14.683207 14313 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.098s
I20260709 12:10:14.738068 12526 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:14.739686 14321 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.157s	user 0.012s	sys 0.027s
I20260709 12:10:14.742519 12523 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 5: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:14.768136 12531 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: []
W20260709 12:10:14.741570 14220 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.811s	user 0.019s	sys 0.026s
I20260709 12:10:14.780383 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:14.775573 12527 raft_consensus.cc:2376] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3. Re-sending same reply.
W20260709 12:10:14.779119 13335 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.012s	sys 0.028s
W20260709 12:10:14.791908 14220 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.958s	user 0.020s	sys 0.027s
I20260709 12:10:14.765172 14222 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:14.795611 12571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.337s	user 0.016s	sys 0.045s
I20260709 12:10:14.801438 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:14.809419 12524 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:14.808245 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:10:14.785957 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 56668us
W20260709 12:10:14.802515 13335 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.012s	sys 0.028s
I20260709 12:10:14.773891 14373 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:14.834147 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1562) took 1293 ms. Trace:
I20260709 12:10:14.808579 12602 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:14.838351 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1412) took 2078 ms. Trace:
W20260709 12:10:14.803108 12680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.266s	user 0.024s	sys 0.063s
W20260709 12:10:14.845811 12571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.393s	user 0.016s	sys 0.049s
I20260709 12:10:14.805429 14319 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:14.803292 14292 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:14.807382 14338 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:14.804378 14348 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:14.806669 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:14.866456 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:14.876334 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:14.866561 12680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.337s	user 0.025s	sys 0.063s
I20260709 12:10:14.844594 12600 rpcz_store.cc:276] 0709 12:10:13.508536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:13.508651 (+   115us) service_pool.cc:224] Handling call
0709 12:10:13.508995 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:14.764046 (+1255051us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:14.769909 (+  5863us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:14.777169 (+  7260us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:14.777225 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:14.788000 (+ 10775us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:14.802204 (+ 14204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":58719,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":12070,"thread_start_us":1184552,"threads_started":1}
W20260709 12:10:14.885537 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:14.808988 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:14.886049 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 1.44323e+06us
I20260709 12:10:14.847084 12522 rpcz_store.cc:276] 0709 12:10:12.636584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:12.795204 (+158620us) service_pool.cc:224] Handling call
0709 12:10:12.805711 (+ 10507us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:12.809237 (+  3526us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:12.809288 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:13.971298 (+1162010us) spinlock_profiling.cc:243] Waited 6.7 ms on lock 0x7b2c00058028. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:10:13.972441 (+  1143us) log.cc:844] Serialized 26 byte log entry
0709 12:10:14.432040 (+459599us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:14.432089 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:14.439973 (+  7884us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:14.702619 (+262646us) raft_consensus.cc:1685] finished
0709 12:10:14.702665 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:14.714808 (+ 12143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3010,"raft.queue_time_us":3620,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":13383,"spinlock_wait_cycles":15417088,"thread_start_us":458860,"threads_started":1,"wal-append.queue_time_us":727738}
W20260709 12:10:14.931206 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:14.895262 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:14.936082 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:14.917817 14338 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:14.935902 14387 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:14.866823 14307 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:14.850564 12528 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
I20260709 12:10:15.005878 12525 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:14.944343 14312 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:14.956744 13333 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.027s	sys 0.023s
W20260709 12:10:15.019124 13339 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.017s	sys 0.005s
W20260709 12:10:15.047051 13339 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.018s	sys 0.005s
W20260709 12:10:15.035145 14320 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.022s	sys 0.056s
W20260709 12:10:15.035099 13331 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.000s	sys 0.022s
I20260709 12:10:15.043107 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:15.054921 14371 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:15.055968 14222 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:15.004351 14292 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:15.058355 12569 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:10:15.004640 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 141808us
W20260709 12:10:15.043898 14381 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.223s	user 0.001s	sys 0.001s
W20260709 12:10:15.047180 14359 consensus_meta.cc:287] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.537s	user 0.008s	sys 0.008s
I20260709 12:10:15.060009 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:15.085858 12530 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
W20260709 12:10:15.065167 14366 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.296s	user 0.001s	sys 0.005s
I20260709 12:10:15.066048 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:15.066725 14240 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:15.067950 14370 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.229s	user 0.001s	sys 0.001s
I20260709 12:10:15.068475 14307 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:15.068622 14371 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:10:15.050454 13333 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.029s	sys 0.023s
W20260709 12:10:15.073238 14368 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.535s	user 0.011s	sys 0.001s
W20260709 12:10:15.050201 14320 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.585s	user 0.023s	sys 0.056s
I20260709 12:10:15.077013 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:15.079438 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:15.165205 12528 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 3: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:10:15.060664 13331 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.000s	sys 0.022s
I20260709 12:10:15.088171 12526 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
W20260709 12:10:15.095777 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 400265us
I20260709 12:10:15.061244 14387 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:15.073586 14312 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:10:15.172132 14387 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:15.174577 14312 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:15.091099 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 502475us
I20260709 12:10:15.174873 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:15.175769 12523 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 2: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:15.115151 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:15.087792 14222 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:15.070088 14364 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:15.049612 14132 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
I20260709 12:10:15.137749 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:15.184571 12598 raft_consensus.cc:2410] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:15.137970 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:15.138885 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.139773 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.143028 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:15.143390 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:15.144384 14377 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:10:15.147064 13338 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
W20260709 12:10:15.149394 14208 consensus_meta.cc:287] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.619s	user 0.008s	sys 0.005s
I20260709 12:10:15.149737 14373 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:15.150641 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 114757us
I20260709 12:10:15.151388 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1431) took 1822 ms. Trace:
I20260709 12:10:15.151865 14371 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:15.152274 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1542) took 1893 ms. Trace:
I20260709 12:10:15.152877 14395 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:15.115375 14240 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:15.115909 14292 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:15.154892 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:15.155810 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:15.156193 14164 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:15.157233 14386 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:10:15.160563 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:15.169694 12604 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:10:15.179900 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:15.180852 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:15.129549 13946 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:15.191233 12571 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:15.207669 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:15.190666 14312 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:15.214332 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 204813us
I20260709 12:10:15.216209 12569 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:15.180650 14222 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:15.198315 14399 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:15.209822 13305 leader_election.cc:336] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.693s (SENT)
W20260709 12:10:15.223115 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 96469us
I20260709 12:10:15.207852 14315 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:10:15.223595 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:15.205643 12677 rpcz_store.cc:276] 0709 12:10:13.259101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:13.268740 (+  9639us) service_pool.cc:224] Handling call
0709 12:10:13.299131 (+ 30391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:13.328669 (+ 29538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:13.330352 (+  1683us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:13.377049 (+ 46697us) log.cc:844] Serialized 26 byte log entry
0709 12:10:14.693649 (+1316600us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:14.699433 (+  5784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:14.717375 (+ 17942us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:15.152016 (+434641us) raft_consensus.cc:1685] finished
0709 12:10:15.152036 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:15.152208 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2904,"raft.run_cpu_time_us":355,"raft.run_wall_time_us":19112,"thread_start_us":1201169,"threads_started":1,"wal-append.queue_time_us":1350696}
I20260709 12:10:15.234249 14371 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:15.266417 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:15.234606 14378 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.170s	user 0.001s	sys 0.000s
I20260709 12:10:15.230391 14164 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:15.266376 14371 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:15.273069 14371 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:15.220736 12601 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
W20260709 12:10:15.275403 14389 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.065s	user 0.001s	sys 0.002s
I20260709 12:10:15.237648 12569 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:15.234735 14240 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:15.239890 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:15.241096 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1561) took 1732 ms. Trace:
I20260709 12:10:15.242977 12522 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:10:15.244696 14217 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:15.248291 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:15.248503 14057 raft_consensus.cc:3055] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:15.223308 14380 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:15.201570 12529 rpcz_store.cc:276] 0709 12:10:13.329025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:13.394660 (+ 65635us) service_pool.cc:224] Handling call
0709 12:10:13.411097 (+ 16437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:13.934701 (+523604us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:13.934789 (+    88us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:13.936123 (+  1334us) log.cc:844] Serialized 26 byte log entry
0709 12:10:14.429452 (+493329us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:14.429517 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:14.512528 (+ 83011us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b5400408654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:14.512594 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:15.150913 (+638319us) raft_consensus.cc:1685] finished
0709 12:10:15.150932 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:15.151187 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":146435,"raft.run_cpu_time_us":403,"raft.run_wall_time_us":40574,"spinlock_wait_cycles":1201302400,"thread_start_us":492370,"threads_started":1,"wal-append.queue_time_us":680307}
I20260709 12:10:15.218659 14321 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:15.249356 14310 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:15.218479 14214 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:15.255259 14222 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:15.255716 13305 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:15.256394 14315 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:15.257974 14383 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:10:15.258517 14400 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:15.231221 14313 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:15.259636 14373 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:15.259858 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.260510 14392 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:15.261039 14338 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:15.261690 14141 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:15.265226 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:15.261817 14364 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:15.266289 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.300331 14057 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:15.204879 14208 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:15.235420 12599 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:10:15.303031 14380 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:15.230109 14292 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:15.283052 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:15.218952 14395 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:15.276525 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.280109 12602 rpcz_store.cc:276] 0709 12:10:13.508199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:13.509389 (+  1190us) service_pool.cc:224] Handling call
0709 12:10:13.519328 (+  9939us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:14.868728 (+1349400us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:15.240769 (+372041us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b5400079c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:15.241020 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2963592448}
I20260709 12:10:15.307017 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:15.307986 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 70477.2us
I20260709 12:10:15.316804 12603 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:15.280573 13330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 333) took 1067 ms. Trace:
I20260709 12:10:15.233760 14312 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:15.329314 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.277534 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:15.332129 14373 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:15.316781 13946 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:15.330593 12599 pending_rounds.cc:85] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 5
W20260709 12:10:15.345335 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:15.327386 13335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 334) took 1193 ms. Trace:
I20260709 12:10:15.347471 12599 pending_rounds.cc:107] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:10:15.356441 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.360208 12597 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:15.361130 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1556) took 1957 ms. Trace:
I20260709 12:10:15.322646 14399 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:15.362644 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 110955us
I20260709 12:10:15.362771 12604 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:15.364171 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.337684 14380 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:15.358808 12599 raft_consensus.cc:2889] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:15.361611 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.313563 14214 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:15.368543 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.378839 14214 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:15.378978 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.379544 14395 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:15.379904 14315 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:15.380190 14395 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:15.380498 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.369675 14240 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:15.371121 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.371399 12527 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:15.377725 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:15.371237 14400 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:10:15.372260 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 337971us
I20260709 12:10:15.372771 14369 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:15.373103 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.373776 12531 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:15.374650 14373 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:15.375013 14057 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:15.376168 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:15.350555 14392 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:15.375783 12604 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:15.333767 14313 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:15.336484 14321 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:15.354348 14364 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:15.364571 14307 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:15.362856 14363 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:15.357824 13335 rpcz_store.cc:276] 0709 12:10:14.131828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:14.134802 (+  2974us) service_pool.cc:224] Handling call
0709 12:10:14.135493 (+   691us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:14.140128 (+  4635us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:14.140194 (+    66us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:14.141258 (+  1064us) log.cc:844] Serialized 26 byte log entry
0709 12:10:14.850590 (+709332us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:14.867952 (+ 17362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:14.876828 (+  8876us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:15.313753 (+436925us) raft_consensus.cc:1685] finished
0709 12:10:15.313795 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:15.325471 (+ 11676us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3371,"raft.queue_time_us":67546,"raft.run_cpu_time_us":331,"raft.run_wall_time_us":332,"thread_start_us":667455,"threads_started":1,"wal-append.queue_time_us":881823}
I20260709 12:10:15.397096 14312 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:15.352404 14310 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:15.382766 14220 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:15.398303 12600 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:15.398319 14164 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:15.399333 13918 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:15.400025 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.400416 14400 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:15.401409 14399 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:15.401595 14390 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
I20260709 12:10:15.364871 14141 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:15.402444 14395 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:15.390372 14371 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:15.403518 14208 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:15.361717 12680 rpcz_store.cc:276] 0709 12:10:13.403286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:13.403917 (+   631us) service_pool.cc:224] Handling call
0709 12:10:13.414511 (+ 10594us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:13.528328 (+113817us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:13.528359 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:13.528880 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 12:10:14.946391 (+1417511us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:15.180962 (+234571us) log.cc:844] Serialized 14 byte log entry
0709 12:10:15.199711 (+ 18749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:15.199791 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:15.360818 (+161027us) raft_consensus.cc:1685] finished
0709 12:10:15.360848 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:15.361024 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15360,"prepare.queue_time_us":268595,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":171701,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":7242,"thread_start_us":1603806,"threads_started":3,"wal-append.queue_time_us":1587134}
I20260709 12:10:15.403944 14320 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:15.397766 12599 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
W20260709 12:10:15.408176 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 75047.2us
I20260709 12:10:15.413568 12523 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:15.414350 12598 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:10:15.450562 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.451169 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:15.425839 14057 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:15.416788 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:15.417292 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:15.461760 12677 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:10:15.461905 12679 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:15.463042 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:15.463649 14141 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:15.463673 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:15.464309 14141 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:15.464673 12680 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:15.417896 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:15.463770 12677 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:15.466513 12675 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:15.467633 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:15.469060 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:15.469937 12677 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 2: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:15.420351 12597 raft_consensus.cc:2410] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:15.414541 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:15.408820 13305 leader_election.cc:336] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.659s (SENT)
I20260709 12:10:15.427291 12526 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:15.422901 12525 pending_rounds.cc:85] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 4
W20260709 12:10:15.430320 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 73085.7us
I20260709 12:10:15.434873 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.435003 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1415) took 2789 ms. Trace:
I20260709 12:10:15.436100 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:15.447340 12673 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:10:15.447491 14321 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:15.448173 14395 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:15.450099 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:15.451427 12599 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 2: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:15.451854 12672 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:15.451999 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.414647 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:15.415061 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.454421 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:15.457401 14310 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:15.457404 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:15.459120 12643 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:15.459661 14313 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:15.460263 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 62242.4us
I20260709 12:10:15.460520 13956 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:15.461184 14364 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:15.419961 14392 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:15.471403 14220 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:15.509044 12602 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 2: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:15.472674 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:15.473541 14400 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:15.475194 14373 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:15.426003 14363 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:15.435397 13946 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 12:10:15.485339 12525 pending_rounds.cc:107] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:10:15.486034 13305 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:15.479843 12524 rpcz_store.cc:276] 0709 12:10:12.637484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:12.839973 (+202489us) service_pool.cc:224] Handling call
0709 12:10:12.840290 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:14.838770 (+1998480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:15.421617 (+582847us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b5400054954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:15.427179 (+  5562us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4527416192}
I20260709 12:10:15.531919 12525 raft_consensus.cc:2889] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:15.494314 12675 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:15.503412 12678 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:15.504815 12676 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:15.507612 14321 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:15.525137 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.522948 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:15.526736 12674 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 2: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:10:15.539072 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 80067.9us
I20260709 12:10:15.539964 12646 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:15.541131 14364 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:10:15.529631 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 58319us
I20260709 12:10:15.531090 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:15.542228 14364 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:15.543278 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:15.543589 14364 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:15.486864 12604 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:15.546375 12646 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:15.544190 12673 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
I20260709 12:10:15.533787 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:15.549420 12605 raft_consensus.cc:2410] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:15.529243 14392 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:15.551852 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.549855 14164 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:15.553475 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:15.548225 14141 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:10:15.550103 14395 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:10:15.555086 14222 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:10:15.555971 14383 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.195s	user 0.005s	sys 0.000s
I20260709 12:10:15.556725 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:15.562296 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 153526us
I20260709 12:10:15.562379 12675 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:15.563572 12601 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:15.566354 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:15.571112 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:15.572566 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:15.572778 14373 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:15.573280 12675 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
W20260709 12:10:15.574605 14405 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:10:15.574854 12675 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260709 12:10:15.575285 13946 raft_consensus.cc:3055] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:15.575405 12646 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:15.575816 13946 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:15.576629 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.577222 12646 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:15.577006 13946 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:15.577939 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.579241 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.579880 14363 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:15.598441 12680 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:15.549732 14400 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:15.603580 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.553934 14321 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:15.569046 13335 raft_consensus.cc:2410] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:10:15.603032 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:15.606048 12679 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:15.609391 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:15.609589 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.610302 12675 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
I20260709 12:10:15.553812 14418 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:15.613651 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.569859 14222 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:15.580540 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:15.573285 14373 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:15.565869 14392 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:15.616568 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:15.636075 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:15.748064 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 116108us
W20260709 12:10:15.720808 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:15.748986 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.665591 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.670677 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:15.770685 12598 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:15.753507 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.771720 12601 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
W20260709 12:10:15.754213 14413 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:10:15.756246 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:15.757522 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:15.342403 13330 rpcz_store.cc:276] 0709 12:10:14.131107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:14.132135 (+  1028us) service_pool.cc:224] Handling call
0709 12:10:14.133021 (+   886us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:14.133742 (+   721us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:10:14.631815 (+498073us) log.cc:844] Serialized 14 byte log entry
0709 12:10:15.129758 (+497943us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:15.130736 (+   978us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:15.162216 (+ 31480us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:15.176464 (+ 14248us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:15.195959 (+ 19495us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":641,"prepare.queue_time_us":667100,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":64,"raft.queue_time_us":27301,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":26365,"thread_start_us":885070,"threads_started":2,"wal-append.queue_time_us":620256}
I20260709 12:10:15.770993 13338 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:15.706954 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:15.621953 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:15.667471 14423 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:15.671753 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:15.791432 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 214323us
W20260709 12:10:15.708073 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:15.706415 14411 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 12:10:15.614259 14395 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:10:15.797405 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:15.812917 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:15.653815 13331 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260709 12:10:15.803123 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.804966 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:15.816116 12675 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:15.614905 14418 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:15.806011 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.809813 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:15.706405 14369 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:15.632169 14371 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:15.811904 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:15.654086 14164 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:15.708475 14313 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:15.665021 14412 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:10:15.867761 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 250924us
I20260709 12:10:15.815467 13331 pending_rounds.cc:85] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 5
I20260709 12:10:15.872874 14418 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:15.706099 14420 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.198s
I20260709 12:10:15.821543 14423 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:15.804049 14422 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:15.831719 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:15.815238 12677 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:15.877925 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.889392 13946 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:15.879595 14371 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:15.869032 14369 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:15.668164 14380 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:15.882263 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 74013.8us
I20260709 12:10:15.931346 12603 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:10:15.882715 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:15.883141 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 88) took 1044 ms. Trace:
I20260709 12:10:15.821764 14057 consensus_queue.cc:1059] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.101s
I20260709 12:10:15.930588 12600 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:15.883293 14220 raft_consensus.cc:3055] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:15.894039 12676 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:15.814705 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:15.896328 12678 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:15.892983 14164 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:15.900715 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:15.905315 14412 raft_consensus.cc:697] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:15.876062 14364 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:15.913408 12672 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:15.878822 12674 raft_consensus.cc:2376] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4. Re-sending same reply.
I20260709 12:10:15.899336 13331 pending_rounds.cc:107] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:10:15.928005 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:15.932209 12530 rpcz_store.cc:276] 0709 12:10:14.840678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.180551 (+339873us) service_pool.cc:224] Handling call
0709 12:10:15.199862 (+ 19311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:15.327317 (+127455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:15.327358 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:15.339976 (+ 12618us) log.cc:844] Serialized 26 byte log entry
0709 12:10:15.439310 (+ 99334us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:15.439354 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:15.439421 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:15.882810 (+443389us) raft_consensus.cc:1685] finished
0709 12:10:15.882843 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:15.883058 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17410,"raft.queue_time_us":39965,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":155,"thread_start_us":160097,"threads_started":2,"wal-append.queue_time_us":154900}
I20260709 12:10:15.945784 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:15.952363 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:15.960829 12606 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:15.907232 14313 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:15.992347 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46684.3us
I20260709 12:10:16.034965 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:16.038408 12678 raft_consensus.cc:2410] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:16.042135 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:16.043195 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 91720.8us
I20260709 12:10:16.036624 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:16.052141 12526 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 3: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:15.942900 14220 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:15.955276 13333 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:10:16.050395 12674 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:16.071130 12524 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260709 12:10:16.088016 12524 pending_rounds.cc:85] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 4
I20260709 12:10:16.088338 12524 pending_rounds.cc:107] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260709 12:10:16.088600 12524 raft_consensus.cc:2889] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:16.071251 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:15.944759 14414 raft_consensus.cc:2764] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260709 12:10:16.109877 12678 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:15.965677 14364 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:16.117246 14364 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:16.120234 12675 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:16.122375 12605 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:16.122795 12676 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:10:15.990124 13331 raft_consensus.cc:2889] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:10:16.129113 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 166818us
I20260709 12:10:16.131168 12672 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:16.134204 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:16.045814 14321 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:16.076620 14392 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:16.038069 13332 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 1.
W20260709 12:10:16.127722 14427 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.244s	user 0.000s	sys 0.001s
I20260709 12:10:16.045241 14348 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.090s
W20260709 12:10:16.151611 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:16.140843 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 215900us
I20260709 12:10:15.958962 14422 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:16.135355 14388 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:10:16.123814 14240 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:16.122591 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:15.947494 14373 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:15.955561 14222 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:16.170372 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 235638us
W20260709 12:10:16.180969 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 247786us
I20260709 12:10:16.200819 14421 consensus_queue.cc:1059] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 12:10:16.190359 14132 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:16.164439 12677 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:16.211555 14220 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:16.177682 14240 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:16.238811 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 87) took 1439 ms. Trace:
I20260709 12:10:16.239518 12525 rpcz_store.cc:276] 0709 12:10:14.799369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.168831 (+369462us) service_pool.cc:224] Handling call
0709 12:10:15.409526 (+240695us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b5800002a98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:10:15.409755 (+   229us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:15.546109 (+136354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:15.546162 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:15.552531 (+  6369us) log.cc:844] Serialized 26 byte log entry
0709 12:10:15.800809 (+248278us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:15.800860 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:15.800932 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:16.235228 (+434296us) raft_consensus.cc:1685] finished
0709 12:10:16.235258 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:16.238715 (+  3457us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":24964480,"thread_start_us":186733,"threads_started":1,"wal-append.queue_time_us":280931}
I20260709 12:10:16.207348 14420 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.027s
I20260709 12:10:16.219259 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:16.251195 14430 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:10:16.219719 14132 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:16.195073 14412 consensus_queue.cc:237] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:16.276080 13305 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:16.248713 14220 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:16.285348 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:16.286415 12606 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:16.294612 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:16.294855 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 58619.1us
I20260709 12:10:16.251289 14423 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:16.298250 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:16.299844 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:16.304409 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:16.302817 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:16.247568 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:16.306324 12646 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:16.294661 14422 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:16.290187 14369 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:16.311707 12616 tablet_copy_service.cc:140] P f5fc807f7ecc4d1ca0fd6be524f734ce: Received BeginTabletCopySession request for tablet 70b8c9b89e3d43fbb70422110d76702c from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:48492)
I20260709 12:10:16.293483 14380 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:16.314752 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:16.316367 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:16.298774 14364 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:16.319140 12616 tablet_copy_service.cc:161] P f5fc807f7ecc4d1ca0fd6be524f734ce: Beginning new tablet copy session on tablet 70b8c9b89e3d43fbb70422110d76702c from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:48492: session id = e56d487365394ac5ac5f1002926a5645-70b8c9b89e3d43fbb70422110d76702c
I20260709 12:10:16.321599 14364 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:16.313393 14431 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.261s	user 0.000s	sys 0.001s
I20260709 12:10:16.294528 14371 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:16.297770 14438 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:16.331871 12646 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:16.312665 14434 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
W20260709 12:10:16.317513 14426 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.242s	user 0.001s	sys 0.001s
I20260709 12:10:16.343210 12677 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:10:16.332152 12496 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:16.326998 12603 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:16.345353 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:16.336804 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 94283.4us
I20260709 12:10:16.310972 14313 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:16.351063 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:16.352334 12600 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260709 12:10:16.354215 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50078.8us
I20260709 12:10:16.360611 12496 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:16.362807 12524 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:16.366906 12604 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:16.367362 12525 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:10:16.368198 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 281598us
I20260709 12:10:16.346958 14371 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:16.370249 12603 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260709 12:10:16.375535 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:16.376724 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:16.372761 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:16.361196 14310 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:16.335289 14369 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
W20260709 12:10:16.402808 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 78318.5us
I20260709 12:10:16.390254 12604 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 2. Current term is 3. Ops: []
W20260709 12:10:16.388235 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 80963.3us
I20260709 12:10:16.408028 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:16.390317 12525 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:16.402540 12530 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: []
W20260709 12:10:16.414458 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:16.415247 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:16.418619 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:16.432164 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:16.426496 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:16.432873 12604 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:16.433646 12523 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:16.413764 12494 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:16.437872 12529 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:16.429898 12496 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:16.444594 12616 tablet_copy_source_session.cc:215] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:10:16.445377 14450 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:10:16.444653 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:16.447606 12530 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:16.447700 12528 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:16.407245 14364 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:16.454401 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:16.387506 14392 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:16.455237 12604 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
I20260709 12:10:16.391278 14310 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:16.387686 14449 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:16.455909 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:16.385815 13304 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:16.446352 14420 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:10:16.412102 14448 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.080s
W20260709 12:10:16.455183 13946 consensus_meta.cc:287] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.525s	user 0.008s	sys 0.004s
I20260709 12:10:16.502475 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:16.498430 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:16.463454 12496 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:16.493882 14240 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:16.759275 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 301014us
I20260709 12:10:16.761638 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:16.580087 12671 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
W20260709 12:10:16.763079 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 349387us
I20260709 12:10:16.462256 12646 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:16.484879 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:16.765172 12680 raft_consensus.cc:2410] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:16.760743 12604 raft_consensus.cc:2376] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4. Re-sending same reply.
I20260709 12:10:16.766458 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:16.768100 12494 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:16.760202 12496 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:16.446727 14292 raft_consensus.cc:3055] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:10:16.790944 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:16.454304 14364 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:10:16.470234 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 97582.6us
W20260709 12:10:16.808014 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47902.9us
I20260709 12:10:16.812845 12679 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: [4.2-4.2]
W20260709 12:10:16.481873 12477 auto_rebalancer.cc:629] Failed to schedule move for tablet 33b7d46307bc4c33acd6c9a58055c7af: Illegal state: Replica f5fc807f7ecc4d1ca0fd6be524f734ce is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 4 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:10:16.813287 12496 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:16.813376 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:16.809794 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:16.787963 13337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 346) took 1021 ms. Trace:
I20260709 12:10:16.486641 14313 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:16.460573 14449 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:16.808375 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 401474us
I20260709 12:10:16.483631 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:16.462190 14447 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:16.831215 14451 raft_consensus.cc:2764] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election decision vote started in defunct term 1: won
I20260709 12:10:16.832880 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:16.822252 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:16.822438 12680 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:10:16.579063 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:16.487584 14164 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:16.846526 12671 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:16.456775 14442 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.326s
I20260709 12:10:16.819160 13337 rpcz_store.cc:276] 0709 12:10:15.766688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.793878 (+ 27190us) service_pool.cc:224] Handling call
0709 12:10:16.043987 (+250109us) spinlock_profiling.cc:243] Waited 11.4 ms on lock 0x7b5800163298. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:10:16.155345 (+111358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:16.155639 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:16.155683 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:16.155749 (+    66us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:16.155801 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:16.155880 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:16.787758 (+631878us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b5400145fd0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:16.787900 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":281509504}
I20260709 12:10:16.757339 14456 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:16.579293 14320 raft_consensus.cc:3055] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:16.460127 14310 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:16.782392 14348 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:16.591140 14369 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:16.841406 12597 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
W20260709 12:10:16.595352 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 229005us
I20260709 12:10:16.862504 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:16.862089 12606 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:10:16.863158 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:16.864100 12679 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:16.783190 14292 raft_consensus.cc:740] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:16.865258 14292 consensus_queue.cc:260] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:16.866532 14445 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:16.867004 14292 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:16.867121 14445 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:16.866671 14313 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:16.867738 14445 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:16.868040 14313 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:16.868489 14313 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:16.869015 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:16.811998 14217 raft_consensus.cc:3055] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:16.870297 14217 raft_consensus.cc:740] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:16.870699 12671 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 12:10:16.871008 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 411788us
I20260709 12:10:16.871328 14217 consensus_queue.cc:260] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:16.871979 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:16.872437 13918 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:16.872771 14217 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:16.873075 13918 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:16.873956 14459 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:16.866642 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:16.877326 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:16.880007 12679 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:16.881984 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:16.827709 14240 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:16.457532 14141 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.386s
I20260709 12:10:16.884325 14240 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:16.885334 14240 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:16.885474 14457 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.440s	user 0.001s	sys 0.009s
I20260709 12:10:16.859747 14369 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:16.886790 14369 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:16.860345 14348 raft_consensus.cc:697] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:16.891284 14348 consensus_queue.cc:237] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:10:16.892387 12674 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:10:16.893641 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:16.860702 14442 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:16.827171 14449 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:16.647846 14448 consensus_queue.cc:1243] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 2.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:10:16.894250 14442 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:16.895411 14442 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:16.873342 12646 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:16.896647 14442 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:16.896958 14448 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:16.481634 14454 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.427s
I20260709 12:10:16.898051 14439 raft_consensus.cc:3055] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:16.898584 14439 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:16.899297 12646 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:16.875020 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:16.899578 14439 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:16.901283 14439 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:16.903604 12572 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:16.905650 12572 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:16.898180 14454 raft_consensus.cc:3055] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:16.883522 14132 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:16.907001 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:16.908720 12679 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:16.908787 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 80185.9us
I20260709 12:10:16.922870 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:16.871954 12671 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:16.825881 14364 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:16.911449 12531 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:16.855139 14456 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:16.914907 12674 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:16.845826 14412 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:10:16.932369 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:16.935319 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:16.936056 12530 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:10:16.938601 14456 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:16.939090 14220 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:16.939549 14220 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:16.939917 14220 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:16.939944 14400 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:16.940527 14400 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:16.941176 14400 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:16.941807 12528 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:16.854408 14320 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:16.909916 13956 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:10:16.952644 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 139485us
W20260709 12:10:16.960940 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:16.911720 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:16.909662 14454 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:16.965196 12674 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:16.966935 12676 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:16.967239 12671 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:16.927773 14448 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:16.934792 14395 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:10:16.977087 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 77934.6us
I20260709 12:10:16.979842 12676 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:16.965497 13956 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:16.986773 14132 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:16.987528 14448 raft_consensus.cc:697] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:16.987845 12524 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:16.989327 12646 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:16.991173 14425 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:16.991878 14459 consensus_queue.cc:1059] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:10:16.994923 14433 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.747s	user 0.008s	sys 0.008s
I20260709 12:10:16.996554 14310 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:16.997061 14373 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:17.001358 12429 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58980 (request call id 181) took 10527 ms. Trace:
I20260709 12:10:17.003633 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:17.006891 14418 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:17.006603 14320 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.018205 12679 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:16.989591 14313 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:17.032045 14458 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:10:17.032802 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 43550.1us
I20260709 12:10:17.094288 12646 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:17.016691 13956 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:16.977633 14395 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:10:16.980932 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 119018us
I20260709 12:10:17.040524 14132 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:17.098634 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 153627us
W20260709 12:10:17.085206 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 275467us
W20260709 12:10:17.082044 14438 consensus_meta.cc:287] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.730s	user 0.011s	sys 0.000s
W20260709 12:10:17.082333 14371 consensus_meta.cc:287] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.710s	user 0.011s	sys 0.000s
I20260709 12:10:17.097627 12527 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:17.103211 12494 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:17.090687 14392 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:17.122390 12530 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:17.123579 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:17.127362 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:17.082868 12429 rpcz_store.cc:276] 0709 12:10:06.473709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:06.483271 (+  9562us) service_pool.cc:224] Handling call
0709 12:10:17.001201 (+10517930us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:15.056564 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:15.057575 (+  1011us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:15.057633 (+    58us) write_op.cc:435] Acquired schema lock
0709 12:10:15.057655 (+    22us) tablet.cc:662] Decoding operations
0709 12:10:15.058524 (+   869us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:15.058620 (+    96us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:15.058647 (+    27us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:10:15.059163 (+   516us) tablet.cc:701] Row locks acquired
0709 12:10:15.059211 (+    48us) write_op.cc:260] PREPARE: finished
0709 12:10:15.059379 (+   168us) write_op.cc:270] Start()
0709 12:10:15.059531 (+   152us) write_op.cc:276] Timestamp: P: 1783599015059315 usec, L: 0
0709 12:10:15.059582 (+    51us) op_driver.cc:348] REPLICATION: starting
0709 12:10:15.060321 (+   739us) log.cc:844] Serialized 1705 byte log entry
0709 12:10:15.706979 (+646658us) op_driver.cc:464] REPLICATION: finished
0709 12:10:16.074969 (+367990us) write_op.cc:301] APPLY: starting
0709 12:10:16.159829 (+ 84860us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:16.202195 (+ 42366us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:16.231917 (+ 29722us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:16.252506 (+ 20589us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:16.252940 (+   434us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=4
0709 12:10:16.253020 (+    80us) write_op.cc:312] APPLY: finished
0709 12:10:16.322644 (+ 69624us) log.cc:844] Serialized 40 byte log entry
0709 12:10:16.350666 (+ 28022us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:16.360810 (+ 10144us) write_op.cc:454] Released schema lock
0709 12:10:16.370501 (+  9691us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:16.449073 (+ 78572us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:16.458321 (+  9248us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7288,"reported_tablets":10,"child_traces":[["op",{"apply.queue_time_us":243318,"apply.run_cpu_time_us":13727,"apply.run_wall_time_us":443835,"num_ops":4,"prepare.queue_time_us":534117,"prepare.run_cpu_time_us":66030,"prepare.run_wall_time_us":164047,"raft.queue_time_us":200823,"raft.run_cpu_time_us":26380,"raft.run_wall_time_us":301498,"replication_time_us":571473,"thread_start_us":873982,"threads_started":4,"wal-append.queue_time_us":251096}]]}
I20260709 12:10:17.098958 14395 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:17.109776 14454 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:17.136511 12597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 1559) took 1690 ms (client timeout 1961 ms). Trace:
I20260709 12:10:17.146394 12523 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:10:17.157059 12523 pending_rounds.cc:85] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 2
I20260709 12:10:17.157361 12523 pending_rounds.cc:107] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:10:17.157671 12523 raft_consensus.cc:2889] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:17.145109 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:17.164227 12524 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
W20260709 12:10:17.154336 12597 rpcz_store.cc:269] 0709 12:10:15.445801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.602740 (+156939us) service_pool.cc:224] Handling call
0709 12:10:17.136303 (+1533563us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54002f1554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:17.136417 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2714621184}
W20260709 12:10:17.166148 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 212483us
I20260709 12:10:17.088971 14240 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.181746 12496 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:17.123416 14438 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:10:17.137948 14464 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.308s	user 0.000s	sys 0.001s
I20260709 12:10:17.092072 14369 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:17.121438 14371 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:10:17.139216 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:17.086149 14449 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.141448 13918 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:17.218094 14459 raft_consensus.cc:3055] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:17.142544 14222 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:17.134589 14132 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:17.235620 12672 raft_consensus.cc:2410] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:17.136859 14435 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:17.165851 14412 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:17.226984 12496 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:17.216883 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 122748us
I20260709 12:10:17.097047 14445 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.144096 14217 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.083s
I20260709 12:10:17.128333 14467 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.093s
I20260709 12:10:17.228261 12494 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:17.237042 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:17.237371 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 97) took 1696 ms. Trace:
I20260709 12:10:17.149865 14395 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:17.239837 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 236280us
I20260709 12:10:17.165979 14439 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.072s
I20260709 12:10:17.242508 12522 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:17.167990 14292 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:17.238181 14475 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.093s
I20260709 12:10:17.247483 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:17.277571 12531 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:17.277091 12673 rpcz_store.cc:276] 0709 12:10:15.540603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.612368 (+ 71765us) service_pool.cc:224] Handling call
0709 12:10:15.750787 (+138419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:15.751546 (+   759us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:15.751579 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:15.752183 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 12:10:15.811378 (+ 59195us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:15.811428 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:15.811506 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:16.994451 (+1182945us) spinlock_profiling.cc:243] Waited 1.34 ms on lock 0x7b2400201878. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14857b99c4 00007f14857b7acb 00007f1485972dc9 00007f1485972a55 00007f148597209d 00007f14885a410e 00007f14885bb88c 00007f14885babb6 00007f14885ba7d8 00007f14885c4221
0709 12:10:17.227078 (+232627us) raft_consensus.cc:1685] finished
0709 12:10:17.227101 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:17.237280 (+ 10179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":106,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"spinlock_wait_cycles":3090816,"thread_start_us":58790,"threads_started":1,"wal-append.queue_time_us":454071}
I20260709 12:10:17.239676 14435 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:10:17.320447 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 167197us
I20260709 12:10:17.205864 14364 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:17.243575 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:17.324349 12678 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:10:17.232249 14459 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:10:17.320014 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:17.349148 12646 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:10:17.330224 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 176087us
I20260709 12:10:17.331200 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:17.350577 12674 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:17.331255 12524 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:17.339161 12678 pending_rounds.cc:85] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 1
I20260709 12:10:17.351673 12678 pending_rounds.cc:107] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:10:17.351974 12678 raft_consensus.cc:2889] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:10:17.339591 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 111438us
I20260709 12:10:17.247640 14483 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:17.243027 14057 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:17.342497 14466 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:17.343321 14460 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:17.343833 14348 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:17.346544 14141 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:17.347254 14292 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:17.346606 14435 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.347404 12675 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:10:17.153366 14448 consensus_queue.cc:237] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.390733 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:17.329893 14392 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:17.397660 12494 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:17.358012 14442 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:17.404138 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 55116.1us
I20260709 12:10:17.390960 14292 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:17.414026 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:17.415053 12597 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 4
W20260709 12:10:17.410526 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 190302us
I20260709 12:10:17.417527 13302 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:17.405859 14447 consensus_meta.cc:287] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.575s	user 0.013s	sys 0.003s
I20260709 12:10:17.421948 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:17.407392 14435 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:17.401939 14057 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:17.424587 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:17.426108 12646 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:17.423081 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:17.436463 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:17.410204 14348 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:17.416231 14310 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:17.405800 14217 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:10:17.446455 14486 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:17.446928 14470 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 12:10:17.447880 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:17.447602 14392 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.449359 14364 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:17.488731 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:17.410797 14400 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:17.412817 14483 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:17.421348 14292 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:17.413815 14456 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:17.452873 14493 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:17.396710 14141 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.460673 12599 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:10:17.474583 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:17.444837 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:17.517580 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 127852us
I20260709 12:10:17.435830 14447 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:17.486631 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:17.483647 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 66250us
I20260709 12:10:17.400130 14461 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:10:17.451140 14395 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:17.415359 14442 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.442911 14439 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:17.505776 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:17.508240 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:17.511313 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:17.399367 14487 consensus_queue.cc:1059] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.080s
W20260709 12:10:17.520200 14220 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.580s	user 0.013s	sys 0.001s
W20260709 12:10:17.517912 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:17.510090 14483 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:17.491801 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:17.518697 14491 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:10:17.522466 13302 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:17.455977 14371 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
W20260709 12:10:17.546564 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 309744us
I20260709 12:10:17.547801 12571 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:10:17.523307 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 104443us
I20260709 12:10:17.517014 14369 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:17.482429 14240 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:17.557457 12496 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:17.517966 12597 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:17.534976 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:17.535264 12601 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:17.536033 12674 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
I20260709 12:10:17.389879 14460 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:17.538158 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49514.5us
W20260709 12:10:17.566787 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 44423.6us
W20260709 12:10:17.566726 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 145764us
I20260709 12:10:17.512889 14445 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:17.518162 14364 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:17.436582 14459 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.540944 12604 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:17.446979 14057 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.424989 14164 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.098s
I20260709 12:10:17.555313 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:17.568162 12646 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:17.557040 14481 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.228s	user 0.000s	sys 0.002s
I20260709 12:10:17.546949 14449 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:17.589223 12571 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:17.546615 14483 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:17.590567 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:17.589187 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 66693.5us
I20260709 12:10:17.592515 12496 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:17.577106 14217 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:17.491590 14486 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.600252 12494 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:17.571334 14480 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:17.547152 14220 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.608582 12674 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:17.586154 14240 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:17.611770 12496 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:17.576376 14501 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:17.589723 14491 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:17.633862 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:17.634577 12680 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:17.614233 12604 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:17.576445 14348 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.618057 14505 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:17.612772 14476 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:17.618659 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:17.618098 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:10:17.619786 14492 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:10:17.585791 14364 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.620702 14309 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 70b8c9b89e3d43fbb70422110d76702c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:17.621497 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:17.624549 14499 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:10:17.625577 14373 consensus_meta.cc:287] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.535s	user 0.007s	sys 0.003s
I20260709 12:10:17.613271 14461 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:10:17.626384 14502 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:10:17.627453 14503 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:17.630873 14369 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:17.631762 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:17.641613 12675 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
I20260709 12:10:17.642251 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:17.648095 13337 raft_consensus.cc:2410] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260709 12:10:17.595383 13956 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.655602 12496 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:17.657155 14310 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
W20260709 12:10:17.650177 12569 consensus_peers.cc:597] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 3c9b5315396948b2af3fa5480c9bee87. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:10:17.676930 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:17.635026 14501 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:17.642311 14449 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:17.644761 14476 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:17.683254 14476 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:17.643712 14491 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:17.659873 12494 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:17.668754 12671 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:17.675148 13336 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:17.665904 14220 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:17.665539 14499 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:17.686492 14132 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:17.671556 14364 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:17.672356 14240 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:17.694446 14057 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:17.695186 14500 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:17.695312 14491 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.696110 14441 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:17.671731 14373 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:17.698357 12601 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:17.706763 14057 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:17.703382 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 71699us
I20260709 12:10:17.703354 14449 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:17.734728 14449 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:17.708038 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1548) took 2290 ms. Trace:
W20260709 12:10:17.736301 14418 consensus_meta.cc:287] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.648s	user 0.004s	sys 0.006s
I20260709 12:10:17.737032 14479 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:17.710300 14494 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 12:10:17.737447 14240 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.727102 14364 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:17.736021 12598 rpcz_store.cc:276] 0709 12:10:15.417727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.907010 (+489283us) service_pool.cc:224] Handling call
0709 12:10:15.915724 (+  8714us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:16.192631 (+276907us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:10:16.192662 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:16.200382 (+  7720us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:16.203070 (+  2688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:17.707466 (+1504396us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b5400231c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:17.707631 (+   165us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:17.707881 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18826,"raft.queue_time_us":27395,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":7837,"spinlock_wait_cycles":627680128}
I20260709 12:10:17.739259 14364 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.712157 13956 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:17.713610 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58114.7us
I20260709 12:10:17.719277 14460 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:17.720937 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 61123.7us
I20260709 12:10:17.659487 14480 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:10:17.679097 14501 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:17.726982 14498 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:17.693199 14132 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:17.737597 14479 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.742565 12680 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:17.743400 12496 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:17.693718 14499 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:17.751730 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:17.701769 14141 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:17.715271 14441 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:10:17.752933 13335 consensus_meta.cc:287] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.513s	user 0.006s	sys 0.006s
I20260709 12:10:17.792989 14057 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.794061 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:17.795102 12674 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:17.795686 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:17.796548 14498 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:17.799402 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:17.805882 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:17.762822 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:17.781711 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:17.995445 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 406246us
W20260709 12:10:17.987568 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 244230us
I20260709 12:10:17.996863 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:17.821663 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:17.997898 12597 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:17.996994 12598 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:17.922416 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:17.723277 14500 raft_consensus.cc:697] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:17.900388 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:18.061918 12531 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:17.997485 12496 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:17.840184 12572 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:18.071692 12599 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:18.079020 12597 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:17.762877 14132 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:18.088685 12597 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:17.989456 14442 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:17.932284 14310 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:17.985447 14486 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:18.092123 12597 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
W20260709 12:10:18.092423 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52358.4us
W20260709 12:10:18.092661 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 252602us
I20260709 12:10:18.058076 14400 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:18.082240 14364 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:17.759975 14460 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:18.090222 14510 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:17.821478 13335 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:18.113989 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:18.114801 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:17.894697 14488 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:17.929548 14487 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:18.000027 14479 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:18.076335 14425 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:18.079277 14445 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:18.082449 14478 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:17.895797 12677 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:18.111588 14496 consensus_queue.cc:1059] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 12:10:18.123989 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:18.085667 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:17.890507 14387 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.221s
I20260709 12:10:17.851161 13332 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:10:18.094343 14501 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:18.182462 14387 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:18.183063 14387 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:18.184265 14387 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:18.187379 14455 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:18.188779 14455 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:17.752561 14483 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:18.133466 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 136076us
I20260709 12:10:18.190834 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:18.114959 13918 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:17.904666 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:17.822502 14476 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:18.150695 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:18.199369 12569 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.132:45801 timed out after 1.801s (SENT) [suppressed 18 similar messages]
W20260709 12:10:18.155457 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 72399.8us
I20260709 12:10:18.158536 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:17.852977 14441 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:18.120627 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:18.172824 12597 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:18.006749 14449 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:18.167821 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:18.162237 14495 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.026s
I20260709 12:10:18.155175 14510 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:17.948962 14292 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:18.202421 12600 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:18.204555 12598 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:18.194077 13918 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:18.195711 14387 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:18.144313 14460 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:18.165411 13946 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:10:18.196555 14505 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:18.195930 14310 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:18.230474 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 109950us
I20260709 12:10:18.174322 14487 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:18.165746 14488 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:18.210879 14501 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:18.210467 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 347767us
I20260709 12:10:18.197022 14445 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:18.369220 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 169852us
I20260709 12:10:18.231649 13918 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:18.371204 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 170863us
I20260709 12:10:18.225800 14461 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:18.544608 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:18.456480 14460 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:18.240916 14483 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:18.448913 14525 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:10:18.393514 14523 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:18.338020 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:18.307215 14425 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:18.239820 14505 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:18.570263 14448 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:18.552254 14320 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:18.450599 14476 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:18.324568 14453 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:18.553607 14292 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:18.232491 14217 consensus_queue.cc:1059] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.340s
I20260709 12:10:18.314075 14527 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:18.563838 13946 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:18.625453 13946 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:18.628274 13946 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:18.570591 14502 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:18.628634 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:18.570950 14461 raft_consensus.cc:697] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:18.460801 14310 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:18.630829 14487 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:18.631258 14369 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:18.632032 14369 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:18.230613 14500 consensus_queue.cc:237] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:18.583204 14495 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.051s
I20260709 12:10:18.631031 14461 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:18.574105 14479 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260709 12:10:18.632035 14510 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:18.602540 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 411765us
I20260709 12:10:18.637362 12496 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:18.594969 14448 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:18.639601 14448 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:18.640244 14369 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:18.641501 14217 consensus_queue.cc:1243] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 2.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 12:10:18.589874 14528 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.248s	user 0.000s	sys 0.002s
I20260709 12:10:18.587898 13330 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:18.643688 14538 ts_tablet_manager.cc:933] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:18.644336 13956 raft_consensus.cc:3055] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:18.565460 14526 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:18.647205 12496 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:18.651012 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:18.651371 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:18.653324 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:18.676245 12598 raft_consensus.cc:2410] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:18.677882 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:18.681463 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 104707us
W20260709 12:10:18.683140 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 106021us
I20260709 12:10:18.679605 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:18.686199 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:18.680325 14527 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:18.692030 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:18.695684 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:18.596779 14453 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:18.696810 12526 raft_consensus.cc:2393] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:18.697261 12600 raft_consensus.cc:2410] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 6 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 1.
I20260709 12:10:18.702426 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:18.644795 13956 raft_consensus.cc:740] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:18.713088 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:18.692488 14483 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:18.692332 14510 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:18.684077 14292 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:10:18.714529 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 67416.5us
I20260709 12:10:18.717482 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:18.720485 12496 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:18.721303 12571 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:18.743353 12526 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:10:18.725553 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:18.698863 14512 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:18.726938 12530 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:10:18.766511 12530 pending_rounds.cc:85] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 1
I20260709 12:10:18.766935 12530 pending_rounds.cc:107] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:10:18.703085 14453 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:18.695991 14222 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:10:18.735131 14309 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.548s	user 0.013s	sys 0.000s
I20260709 12:10:18.734952 14445 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:18.735855 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:18.723860 14533 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:18.738571 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:18.693356 14526 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:18.740207 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:18.704239 14364 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:18.769888 14387 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.535s	user 0.006s	sys 0.006s
I20260709 12:10:18.726622 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:18.762565 12602 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.028s	sys 0.032s
I20260709 12:10:18.763160 12531 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:18.723281 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:18.810343 12602 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.030s	sys 0.032s
I20260709 12:10:18.767274 12530 raft_consensus.cc:2889] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:18.729197 12528 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 2: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:18.742555 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:18.778173 12525 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:18.812800 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:18.813948 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:18.778023 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 57627.8us
I20260709 12:10:18.815317 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:18.816020 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:18.778435 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 61436.4us
I20260709 12:10:18.780690 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:18.817718 12523 raft_consensus.cc:2393] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 3: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:18.700146 14460 consensus_queue.cc:1059] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 12:10:18.710927 14488 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:18.719187 14495 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
I20260709 12:10:18.790215 12524 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:18.674101 14425 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.041s
I20260709 12:10:18.728332 14292 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:18.784564 14523 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:10:18.824513 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:18.851301 12522 raft_consensus.cc:2368] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3.
I20260709 12:10:18.785403 14486 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:18.826578 14538 tablet_copy_client.cc:323] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645: tablet copy: Beginning tablet copy session from remote peer at address 127.11.13.130:39365
I20260709 12:10:18.839610 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:18.836062 14487 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:18.814716 13333 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:18.803256 14533 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:18.852366 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:18.853394 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:18.839236 13332 raft_consensus.cc:2410] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:10:18.856308 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:18.849601 13302 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:18.843176 14488 raft_consensus.cc:3055] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:18.837323 14292 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:18.813131 13956 consensus_queue.cc:260] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:10:18.865906 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:18.874748 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:18.883441 12673 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:10:18.863710 14483 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:18.837977 14523 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:18.853662 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:18.803323 14222 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:18.868839 12674 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:10:18.892397 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:18.822914 13946 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260709 12:10:18.808959 14364 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:18.865239 14445 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:18.894863 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:18.893149 12676 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:18.866784 14487 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:18.876466 14533 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:18.946138 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:18.877470 14488 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:18.957546 12523 raft_consensus.cc:2410] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:18.876669 14460 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:19.039026 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:18.943615 14487 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:18.940701 14523 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:18.942401 14545 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 12:10:19.025871 14510 raft_consensus.cc:697] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:10:19.063107 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:19.063463 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:10:19.038311 14478 consensus_meta.cc:287] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.864s	user 0.005s	sys 0.010s
W20260709 12:10:19.061904 14531 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.271s	user 0.000s	sys 0.011s
W20260709 12:10:19.085860 14549 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
W20260709 12:10:19.086529 14550 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 12:10:19.087332 14527 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:19.276021 14499 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.391s	user 0.011s	sys 0.003s
W20260709 12:10:19.088304 14540 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.221s	user 0.000s	sys 0.000s
W20260709 12:10:19.098042 12672 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1410) took 2516 ms (client timeout 2918 ms). Trace:
I20260709 12:10:19.028369 13918 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:19.277601 12672 rpcz_store.cc:269] 0709 12:10:16.581213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.766168 (+184955us) service_pool.cc:224] Handling call
0709 12:10:19.097735 (+2331567us) spinlock_profiling.cc:243] Waited 467 ms on lock 0x7b54000ac8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:19.097952 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1074667904}
I20260709 12:10:19.144521 14536 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:19.126217 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1574) took 1370 ms. Trace:
I20260709 12:10:19.138039 14526 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:19.083819 14309 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:38472 (request call id 144) took 6916 ms. Trace:
W20260709 12:10:19.357653 12643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.000s	sys 0.011s
I20260709 12:10:19.152478 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:19.358841 12643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.000s	sys 0.011s
I20260709 12:10:19.175806 12673 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:19.171809 14445 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:19.343209 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:19.119585 12674 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
W20260709 12:10:19.367473 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:19.344628 12531 consensus_meta.cc:287] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.520s	user 0.009s	sys 0.001s
W20260709 12:10:19.359247 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 673134us
I20260709 12:10:19.383750 12531 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:19.342432 14455 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:19.293386 14478 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:19.276528 14499 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:19.331099 12602 rpcz_store.cc:276] 0709 12:10:17.756082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.756223 (+   141us) service_pool.cc:224] Handling call
0709 12:10:17.792962 (+ 36739us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:17.793420 (+   458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:17.793454 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:18.068303 (+274849us) log.cc:844] Serialized 26 byte log entry
0709 12:10:18.810912 (+742609us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:18.848426 (+ 37514us) log.cc:844] Serialized 14 byte log entry
0709 12:10:18.972766 (+124340us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:18.972904 (+   138us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:19.102993 (+130089us) raft_consensus.cc:1685] finished
0709 12:10:19.103013 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:19.126091 (+ 23078us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":54236,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":11832,"raft.queue_time_us":43634,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":31774,"spinlock_wait_cycles":5622528,"thread_start_us":818040,"threads_started":2,"wal-append.queue_time_us":847873}
I20260709 12:10:19.431077 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:19.342789 14380 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:19.142588 14483 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:19.434681 14455 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:19.357584 14425 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:19.565922 14132 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.449s	user 0.012s	sys 0.004s
I20260709 12:10:19.644846 14559 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:19.655503 14132 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:19.655815 14559 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:19.655620 14425 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:19.478042 14486 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.004s	sys 0.004s
I20260709 12:10:19.642429 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 112) took 1798 ms. Trace:
I20260709 12:10:19.645367 14560 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:19.633558 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 741200us
I20260709 12:10:19.658104 14559 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:19.436282 14488 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:19.664058 14488 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:19.447808 14554 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:19.666831 14380 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:19.667388 14557 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:19.667543 14554 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:19.668583 14556 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:19.668164 14557 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:19.551301 12529 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 4: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:19.670017 14554 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:19.671006 14556 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:19.671756 14556 raft_consensus.cc:697] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:19.671830 14557 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:19.673152 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 242405us
I20260709 12:10:19.567090 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:19.670352 14565 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:19.679396 12496 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:19.341454 14292 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:19.657863 14132 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:19.658704 14486 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.006s	sys 0.004s
I20260709 12:10:19.658774 12680 rpcz_store.cc:276] 0709 12:10:17.844155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.859198 (+ 15043us) service_pool.cc:224] Handling call
0709 12:10:17.859554 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:17.862726 (+  3172us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:17.862777 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:17.863439 (+   662us) log.cc:844] Serialized 26 byte log entry
0709 12:10:18.543207 (+679768us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:18.686295 (+143088us) log.cc:844] Serialized 14 byte log entry
0709 12:10:18.943917 (+257622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:18.944046 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:19.642093 (+698047us) raft_consensus.cc:1685] finished
0709 12:10:19.642150 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:19.642345 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15996,"prepare.queue_time_us":274099,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":2765,"raft.queue_time_us":246290,"raft.run_cpu_time_us":274,"raft.run_wall_time_us":281,"thread_start_us":845223,"threads_started":3,"wal-append.queue_time_us":905289}
I20260709 12:10:19.674088 13956 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:19.691588 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:19.699388 12523 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:19.660562 14455 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:19.682448 14565 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:19.712410 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:19.717204 14553 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.008s	sys 0.006s
W20260709 12:10:19.717648 14553 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.008s	sys 0.006s
I20260709 12:10:19.704521 14364 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:19.720588 14369 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.080s	user 0.010s	sys 0.000s
I20260709 12:10:19.723223 14499 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:19.714154 14559 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:19.726755 14559 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:19.727385 13918 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:19.729113 14559 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:19.691751 14510 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:19.734560 14425 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:19.709085 12680 pending_rounds.cc:85] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 1
I20260709 12:10:19.735920 12680 pending_rounds.cc:107] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:10:19.660046 14560 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:19.736434 12680 raft_consensus.cc:2889] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:19.721305 14369 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:19.690754 14486 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:19.661293 14445 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:19.708662 13946 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:19.743896 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:19.766680 14561 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.103s	user 0.000s	sys 0.001s
I20260709 12:10:19.805583 12531 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:19.694756 14556 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:19.775195 14560 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:19.778455 14486 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:19.794445 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:19.779059 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1430) took 2534 ms. Trace:
I20260709 12:10:19.786067 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:19.828796 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:19.829965 12524 raft_consensus.cc:2393] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 2: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:19.710987 14554 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:19.828092 12569 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:19.846218 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:19.827080 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:19.841671 12494 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.814s	user 0.006s	sys 0.003s
I20260709 12:10:19.768429 13334 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:19.860805 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:19.914386 12494 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.006s	sys 0.003s
I20260709 12:10:19.861235 12598 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 12:10:19.914871 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 887334us
I20260709 12:10:19.915431 12601 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 4: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:19.916564 13337 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 6: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:19.851643 12604 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 12:10:19.855464 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 176149us
I20260709 12:10:19.840912 14553 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:19.916676 12671 rpcz_store.cc:276] 0709 12:10:17.245040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.278854 (+ 33814us) service_pool.cc:224] Handling call
0709 12:10:17.318317 (+ 39463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:17.687051 (+368734us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:19.778689 (+2091638us) spinlock_profiling.cc:243] Waited 350 ms on lock 0x7b540016eed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:19.778952 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":804958976}
I20260709 12:10:19.855860 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:19.827732 14499 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:19.358666 14309 rpcz_store.cc:276] 0709 12:10:12.160879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:12.164609 (+  3730us) service_pool.cc:224] Handling call
0709 12:10:12.730041 (+565432us) ts_tablet_manager.cc:934] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
0709 12:10:17.589990 (+4859949us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:10:18.186979 (+596989us) tablet_metadata.cc:665] Metadata flushed
0709 12:10:18.187136 (+   157us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:10:18.820469 (+633333us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:10:18.866297 (+ 45828us) tablet_replica.cc:206] Creating consensus instance
0709 12:10:18.938849 (+ 72552us) ts_tablet_manager.cc:989] Replica registered.
0709 12:10:19.077213 (+138364us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":7353,"tablet-copy.queue_time_us":455919,"thread_start_us":487770,"threads_started":2}
W20260709 12:10:19.861304 14453 consensus_meta.cc:287] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.058s	user 0.015s	sys 0.000s
W20260709 12:10:19.863078 14320 consensus_meta.cc:287] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.267s	user 0.013s	sys 0.002s
I20260709 12:10:19.768733 13946 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:19.921411 12522 raft_consensus.cc:2410] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 5 because replica has last-logged OpId of term: 4 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:10:19.861865 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:19.921802 12671 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:19.918474 12616 tablet_copy_service.cc:140] P f5fc807f7ecc4d1ca0fd6be524f734ce: Received BeginTabletCopySession request for tablet 3c9b5315396948b2af3fa5480c9bee87 from peer e56d487365394ac5ac5f1002926a5645 ({username='slave'} at 127.0.0.1:48492)
I20260709 12:10:19.942085 12531 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:10:19.734815 14565 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:19.867164 14387 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.018s	sys 0.009s
I20260709 12:10:19.929505 12525 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:10:19.845566 13918 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:19.919756 14533 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.857s	user 0.010s	sys 0.003s
I20260709 12:10:19.843498 14558 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:19.914155 14164 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:19.936803 14313 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:19.939669 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:19.925489 14571 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:19.980270 12531 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:10:19.977437 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:19.982157 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:19.975771 14309 tablet_copy_client.cc:806] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:10:19.929682 14554 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:10:19.983732 12616 tablet_copy_service.cc:161] P f5fc807f7ecc4d1ca0fd6be524f734ce: Beginning new tablet copy session on tablet 3c9b5315396948b2af3fa5480c9bee87 from peer e56d487365394ac5ac5f1002926a5645 at {username='slave'} at 127.0.0.1:48492: session id = e56d487365394ac5ac5f1002926a5645-3c9b5315396948b2af3fa5480c9bee87
I20260709 12:10:20.028509 14533 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:20.055337 14571 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:20.082595 14309 tablet_copy_client.cc:670] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
I20260709 12:10:19.940642 14453 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:20.036089 14543 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:19.917987 14553 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:20.049937 12671 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 2: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:20.103173 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:20.098315 14571 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:20.041129 14558 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:20.053256 14445 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:10:20.054512 14581 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.063s	user 0.003s	sys 0.004s
I20260709 12:10:20.043823 14560 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:10:20.023677 12643 leader_election.cc:336] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.673s (SENT)
I20260709 12:10:20.097401 14504 raft_consensus.cc:3055] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:20.104455 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:20.121243 12605 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
W20260709 12:10:20.023954 14387 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.021s	sys 0.014s
I20260709 12:10:20.106735 12530 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:20.108168 14533 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:20.109239 14460 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:20.118225 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 94586us
I20260709 12:10:20.056783 14164 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:20.058183 14565 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:20.121878 12671 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:20.054524 14576 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:20.073733 14292 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:20.029755 13918 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:20.152485 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:20.149163 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:20.121805 14455 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:20.164857 12671 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:20.151993 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:20.176985 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:20.177660 12671 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:10:20.131731 14568 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.276s	user 0.000s	sys 0.001s
W20260709 12:10:20.182236 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:20.150516 13335 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 345) took 4390 ms (client timeout 2888 ms). Trace:
W20260709 12:10:20.185221 13335 rpcz_store.cc:269] 0709 12:10:15.760404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:15.765776 (+  5372us) service_pool.cc:224] Handling call
0709 12:10:16.483470 (+717694us) spinlock_profiling.cc:243] Waited 5.54 ms on lock 0x7b5800163298. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fbeee 000000000036eb39 00007f148d96f1e0 00007f148da380c4 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:10:20.150240 (+3666770us) spinlock_profiling.cc:243] Waited 394 ms on lock 0x7b5400478454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:20.150387 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":919741184}
I20260709 12:10:20.151046 14164 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:20.194447 12527 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:20.135228 14504 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:20.131129 14369 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:20.124091 14543 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:20.154857 14460 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:20.209342 14453 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:20.179551 14455 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:20.219135 14579 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 12:10:20.220600 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:20.202656 13330 consensus_meta.cc:287] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.559s	user 0.006s	sys 0.008s
I20260709 12:10:20.251854 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:20.130836 14445 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:20.255914 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1441) took 1400 ms. Trace:
I20260709 12:10:20.188890 14576 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:20.242027 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 413784us
I20260709 12:10:20.254742 13336 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:20.198393 13946 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:20.256345 12672 rpcz_store.cc:276] 0709 12:10:18.855659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:19.409882 (+554223us) service_pool.cc:224] Handling call
0709 12:10:19.410325 (+   443us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:19.410776 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:19.410819 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:19.434938 (+ 24119us) log.cc:844] Serialized 26 byte log entry
0709 12:10:19.702914 (+267976us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:19.702957 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:19.703024 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:20.255683 (+552659us) raft_consensus.cc:1685] finished
0709 12:10:20.255709 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:20.255852 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1120,"raft.queue_time_us":258804,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":107,"thread_start_us":267539,"threads_started":1,"wal-append.queue_time_us":322525}
I20260709 12:10:20.258097 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:20.257619 12496 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:10:20.261996 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:20.260386 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:20.266469 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:20.254298 14533 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:20.259660 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:20.275022 14456 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:20.277387 14578 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:10:20.260371 14292 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.018s
I20260709 12:10:20.279096 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:20.298605 12679 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:20.279431 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:20.299234 12672 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:20.281565 14558 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:20.267385 14573 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:20.269002 14504 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:20.284184 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 1476) took 1574 ms. Trace:
I20260709 12:10:20.304498 12529 rpcz_store.cc:276] 0709 12:10:18.709845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:18.709969 (+   124us) service_pool.cc:224] Handling call
0709 12:10:20.283847 (+1573878us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b54003f7354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:20.284067 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1678061440}
I20260709 12:10:20.287369 12494 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:20.288316 12671 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:20.288461 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:20.290804 13956 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.588s	user 0.009s	sys 0.003s
I20260709 12:10:20.291654 14533 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:20.312793 14460 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:20.312876 14454 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:20.265964 14553 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:20.293110 14532 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.733s	user 0.000s	sys 0.012s
I20260709 12:10:20.256948 14554 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:20.252051 13330 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:20.296118 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:20.295125 14526 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:20.297293 14445 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:20.302598 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:20.285607 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1435) took 1431 ms. Trace:
I20260709 12:10:20.307005 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:20.336087 12671 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:20.306331 14527 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:20.304454 14576 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:20.311679 13918 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:20.330020 12679 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:20.333849 12672 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:20.345309 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:20.345613 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1442) took 1489 ms. Trace:
I20260709 12:10:20.301796 14573 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:20.346227 14573 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:20.345983 12674 rpcz_store.cc:276] 0709 12:10:18.855929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:19.418219 (+562290us) service_pool.cc:224] Handling call
0709 12:10:19.420072 (+  1853us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:19.716790 (+296718us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:19.716838 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:19.717483 (+   645us) log.cc:844] Serialized 26 byte log entry
0709 12:10:19.831539 (+114056us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:19.831597 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:19.831693 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:20.345396 (+513703us) raft_consensus.cc:1685] finished
0709 12:10:20.345417 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:20.345556 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":249275,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":119,"thread_start_us":161543,"threads_started":2,"wal-append.queue_time_us":437689}
I20260709 12:10:20.348035 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:20.348650 12674 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:20.349565 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:20.334327 12675 rpcz_store.cc:276] 0709 12:10:18.853937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:18.864508 (+ 10571us) service_pool.cc:224] Handling call
0709 12:10:18.865042 (+   534us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:18.865456 (+   414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:18.865496 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:19.565974 (+700478us) log.cc:844] Serialized 26 byte log entry
0709 12:10:19.844238 (+278264us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:19.844358 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:19.857312 (+ 12954us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:20.285291 (+427979us) raft_consensus.cc:1685] finished
0709 12:10:20.285326 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:20.285527 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":801632,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":129,"thread_start_us":276483,"threads_started":1,"wal-append.queue_time_us":347492}
I20260709 12:10:20.351518 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:20.352363 12675 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 1.
I20260709 12:10:20.346004 12672 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:20.362635 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:20.363196 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:20.364109 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:20.364924 12679 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 3
W20260709 12:10:20.333278 14523 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.984s	user 0.015s	sys 0.002s
I20260709 12:10:20.370250 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:20.373299 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:20.339196 14504 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:20.375221 12531 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:20.378765 12598 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
I20260709 12:10:20.379769 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:20.380211 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1608) took 2258 ms. Trace:
I20260709 12:10:20.380350 12598 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:20.380676 12599 rpcz_store.cc:276] 0709 12:10:18.121457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:18.121701 (+   244us) service_pool.cc:224] Handling call
0709 12:10:18.133829 (+ 12128us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:18.134342 (+   513us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:18.134387 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:18.142590 (+  8203us) log.cc:844] Serialized 26 byte log entry
0709 12:10:18.313213 (+170623us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:18.313264 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:18.313357 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:20.292227 (+1978870us) spinlock_profiling.cc:243] Waited 4.36 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:20.375115 (+ 82888us) raft_consensus.cc:1685] finished
0709 12:10:20.375147 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:20.380110 (+  4963us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":931,"raft.queue_time_us":693,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":13579,"spinlock_wait_cycles":10024704,"thread_start_us":107555,"threads_started":1,"wal-append.queue_time_us":293908}
W20260709 12:10:20.363013 13330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 283) took 2034 ms (client timeout 1669 ms). Trace:
I20260709 12:10:20.383064 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:20.386590 12599 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 4: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:20.324461 14590 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:20.339718 14527 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:20.344352 14526 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:20.374982 14533 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:20.396674 12672 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:20.397933 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:20.398993 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:20.396382 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:20.403815 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:20.404018 12599 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:20.378238 14523 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:20.451972 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:20.404875 12679 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:20.432543 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:20.449286 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:20.466986 12606 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:20.439837 12675 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:20.467669 12679 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260709 12:10:20.468771 12528 raft_consensus.cc:2410] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:20.469013 12672 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:20.395354 14222 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:20.425621 12671 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:20.413278 12616 tablet_copy_source_session.cc:215] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Tablet Copy: opened 0 blocks and 1 log segments
W20260709 12:10:20.438506 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 151253us
I20260709 12:10:20.472086 12496 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:20.453763 14487 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.115s	user 0.010s	sys 0.003s
W20260709 12:10:20.430404 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 104516us
W20260709 12:10:20.540395 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:20.484490 12598 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
W20260709 12:10:20.431702 14572 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.131s	user 0.001s	sys 0.000s
I20260709 12:10:20.504941 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:20.549626 12671 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 1.
I20260709 12:10:20.390195 14590 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:20.450686 14583 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 12:10:20.507458 12523 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 2: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:10:20.509685 14512 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.707s	user 0.009s	sys 0.017s
I20260709 12:10:20.538545 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:20.538765 12528 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 4. Current term is 5. Ops: []
W20260709 12:10:20.542625 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 405823us
W20260709 12:10:20.386263 13330 rpcz_store.cc:269] 0709 12:10:18.308138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:18.310470 (+  2332us) service_pool.cc:224] Handling call
0709 12:10:20.341448 (+2030978us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:20.551260 14487 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:20.552793 12675 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:20.553352 12606 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260709 12:10:20.557963 12599 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:20.566335 12671 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:20.564448 12603 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:20.538693 14141 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:20.570281 12494 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:20.574774 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 102810us
W20260709 12:10:20.599664 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:20.600131 12496 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:20.575472 14589 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:20.562757 14590 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:20.562947 14512 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.774s	user 0.009s	sys 0.017s
I20260709 12:10:20.587990 12671 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:20.594790 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:20.601230 12678 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 4: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:20.601914 12674 raft_consensus.cc:2410] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:20.602308 12675 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:10:20.432317 14527 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:20.604094 12599 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:20.580958 14595 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:20.605491 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:20.600630 14141 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:20.603623 14348 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:20.652228 14595 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:20.653064 14348 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:20.652875 14141 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:20.653580 14348 raft_consensus.cc:697] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:20.654029 14141 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:20.654671 14348 consensus_queue.cc:237] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:20.655699 14498 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
W20260709 12:10:20.616500 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:20.648940 12671 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:10:20.653450 14595 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:20.659353 14445 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:20.659806 14445 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:20.660719 12671 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:20.450371 14526 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:20.660207 14445 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:20.638077 14456 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:20.663295 14456 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:10:20.664245 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:20.665143 12673 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:20.633231 14371 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:20.596611 14553 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
W20260709 12:10:20.685344 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:20.636963 14512 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:10:20.696422 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:20.699360 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 93956.8us
W20260709 12:10:20.669622 14449 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.860s	user 0.029s	sys 0.020s
I20260709 12:10:20.704188 12496 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:20.705096 14589 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:20.705814 14589 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:20.638134 14387 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:10:20.664309 14456 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:20.703035 14449 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.916s	user 0.030s	sys 0.021s
I20260709 12:10:20.701853 14371 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:20.660455 14596 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.139s
I20260709 12:10:20.756973 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:20.673204 12672 raft_consensus.cc:2410] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:21.130985 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:21.147368 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:21.206094 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:21.063719 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:21.207587 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:21.208293 12671 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
I20260709 12:10:21.212527 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1599) took 1289 ms. Trace:
I20260709 12:10:21.129740 12496 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:21.215727 12672 rpcz_store.cc:276] 0709 12:10:19.923117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:20.397626 (+474509us) service_pool.cc:224] Handling call
0709 12:10:21.212434 (+814808us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:21.217114 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:21.217530 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:21.220721 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:21.221655 12528 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
W20260709 12:10:21.221961 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 92298.1us
I20260709 12:10:21.210034 14400 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:21.228448 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:21.229174 12495 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.008s	sys 0.021s
W20260709 12:10:20.883059 14573 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.536s	user 0.008s	sys 0.004s
W20260709 12:10:20.947494 14454 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.634s	user 0.013s	sys 0.000s
W20260709 12:10:21.229619 12495 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.008s	sys 0.021s
I20260709 12:10:21.226784 12672 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:21.231465 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1610) took 1050 ms. Trace:
W20260709 12:10:20.775238 14594 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:10:21.232002 12672 rpcz_store.cc:276] 0709 12:10:20.181200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:21.226026 (+1044826us) service_pool.cc:224] Handling call
0709 12:10:21.226520 (+   494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:21.231198 (+  4678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:21.231401 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:21.233682 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:20.808212 14516 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:21.160899 14453 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:21.227766 14599 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:20.773622 14589 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:20.750833 14387 consensus_queue.cc:1243] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 5.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:10:20.776736 14478 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:21.262290 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:21.249027 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:20.951644 14487 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:20.808969 14523 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:21.256690 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1611) took 1011 ms. Trace:
I20260709 12:10:21.266718 12672 rpcz_store.cc:276] 0709 12:10:20.245124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:21.233322 (+988198us) service_pool.cc:224] Handling call
0709 12:10:21.255959 (+ 22637us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:21.256212 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:21.256251 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:21.256315 (+    64us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:21.256360 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:21.256435 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:21.256592 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:20.745590 14369 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.489s
W20260709 12:10:21.212407 14595 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.005s	sys 0.020s
I20260709 12:10:21.238996 14425 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:21.254349 14608 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:21.235275 14400 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:21.278437 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 133993us
W20260709 12:10:21.264701 14504 consensus_meta.cc:287] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.878s	user 0.011s	sys 0.003s
I20260709 12:10:21.256946 14516 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:21.250512 14598 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:10:21.314821 12524 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:21.315516 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1517) took 1396 ms. Trace:
I20260709 12:10:21.315912 12524 rpcz_store.cc:276] 0709 12:10:19.919137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:19.930219 (+ 11082us) service_pool.cc:224] Handling call
0709 12:10:21.314355 (+1384136us) spinlock_profiling.cc:243] Waited 7.14 ms on lock 0x7b5400107128. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e7fd5f4 00007f148d9e6227 00007f148d9e5ab8 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:21.315444 (+  1089us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16419072}
I20260709 12:10:21.319116 14499 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:21.319864 14499 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:21.321066 14400 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:21.280213 14425 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:21.279940 14608 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:21.323406 14608 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:21.236604 14573 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:20.882612 14512 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:21.264070 14478 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:21.299280 12671 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
I20260709 12:10:21.299712 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:21.278469 14595 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s	user 0.005s	sys 0.021s
I20260709 12:10:21.269208 14371 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:21.347399 12528 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:10:21.285820 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:21.257313 14453 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:21.327559 12427 catalog_manager.cc:5697] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:10:21.358336 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:21.359519 12672 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:21.357224 12427 catalog_manager.cc:5697] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:10:21.363500 12497 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.013s	sys 0.023s
W20260709 12:10:21.363924 12497 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.602s	user 0.013s	sys 0.023s
I20260709 12:10:21.346388 14478 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:21.364029 12427 catalog_manager.cc:5697] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 1 to 3, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 3 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:10:21.364944 14526 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.007s	sys 0.031s
I20260709 12:10:21.374313 14369 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:21.370921 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:21.375342 14369 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:21.372006 12427 catalog_manager.cc:5697] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:10:21.371809 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 167693us
I20260709 12:10:21.378773 12427 catalog_manager.cc:5697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:21.379303 14369 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:21.365732 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1450) took 1330 ms. Trace:
W20260709 12:10:21.382864 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:21.385064 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:21.364085 14590 consensus_meta.cc:287] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.725s	user 0.013s	sys 0.002s
W20260709 12:10:21.374629 14526 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.007s	sys 0.031s
I20260709 12:10:21.384742 12675 rpcz_store.cc:276] 0709 12:10:20.030766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:20.658779 (+628013us) service_pool.cc:224] Handling call
0709 12:10:20.659086 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:20.659415 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:20.659442 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:20.659905 (+   463us) log.cc:844] Serialized 26 byte log entry
0709 12:10:20.872841 (+212936us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:21.234753 (+361912us) log.cc:844] Serialized 14 byte log entry
0709 12:10:21.360423 (+125670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:21.360545 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:21.360620 (+    75us) raft_consensus.cc:1685] finished
0709 12:10:21.360640 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:21.360857 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":413808,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":7711,"raft.queue_time_us":74093,"raft.run_cpu_time_us":477,"raft.run_wall_time_us":27582,"thread_start_us":621951,"threads_started":3,"wal-append.queue_time_us":601804}
I20260709 12:10:21.381173 12427 catalog_manager.cc:5697] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 1 to 3, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 3 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:21.390290 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1612) took 1137 ms. Trace:
I20260709 12:10:21.396644 12531 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:21.393436 12672 rpcz_store.cc:276] 0709 12:10:20.253231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:21.365619 (+1112388us) service_pool.cc:224] Handling call
0709 12:10:21.389476 (+ 23857us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:21.389766 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:21.389809 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:21.389874 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:21.389929 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:21.390010 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:21.390203 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:21.373896 14598 raft_consensus.cc:3055] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:10:21.382882 14499 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:21.363801 14516 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:21.402663 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:21.394812 12427 catalog_manager.cc:5697] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 1 to 4, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 4 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:21.407830 12672 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 2: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:21.401273 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:21.409394 12427 catalog_manager.cc:5697] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:10:21.401170 14598 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:21.416244 12427 catalog_manager.cc:5697] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 2 to 5. New cstate: current_term: 5 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:10:21.392397 14590 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:21.473136 14614 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
I20260709 12:10:21.438738 14516 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:21.544755 14505 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:21.437354 14618 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:21.546432 14618 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:21.546882 14618 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:21.474025 14612 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.145s	user 0.000s	sys 0.001s
I20260709 12:10:21.539134 14573 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:21.429849 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:21.551911 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:21.552500 12675 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:21.447804 14596 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:21.417753 12427 catalog_manager.cc:5697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 2 to 3, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 3 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:10:21.422324 14599 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:21.534544 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:21.542897 14590 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:21.535280 14606 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
I20260709 12:10:21.557025 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1443) took 2694 ms. Trace:
I20260709 12:10:21.566530 12680 rpcz_store.cc:276] 0709 12:10:18.862976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:19.707585 (+844609us) service_pool.cc:224] Handling call
0709 12:10:19.708800 (+  1215us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:19.755134 (+ 46334us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:19.755180 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:19.755790 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 12:10:19.943528 (+187738us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:20.197281 (+253753us) log.cc:844] Serialized 14 byte log entry
0709 12:10:20.197634 (+   353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:20.197763 (+   129us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:21.541918 (+1344155us) spinlock_profiling.cc:243] Waited 1.4 ms on lock 0x7b180039a420. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:10:21.556777 (+ 14859us) raft_consensus.cc:1685] finished
0709 12:10:21.556873 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:21.556978 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":312697,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":1994,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":3881,"spinlock_wait_cycles":3229440,"thread_start_us":340025,"threads_started":2,"wal-append.queue_time_us":441842}
I20260709 12:10:21.568938 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:21.555043 12427 catalog_manager.cc:5697] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 1 to 4. New cstate: current_term: 4 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:10:21.557739 14459 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:21.559731 14599 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:21.560742 14445 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.898s	user 0.008s	sys 0.003s
I20260709 12:10:21.562431 14453 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:21.545951 14505 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:21.582242 12675 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:21.575728 13334 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 356) took 2811 ms. Trace:
I20260709 12:10:21.574049 14449 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:21.583238 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:21.584383 12675 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:21.585111 12680 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:10:21.585813 12677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1403) took 5082 ms (client timeout 2094 ms). Trace:
I20260709 12:10:21.586567 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1617) took 1115 ms. Trace:
I20260709 12:10:21.586743 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1613) took 1332 ms. Trace:
W20260709 12:10:21.586578 12677 rpcz_store.cc:269] 0709 12:10:16.500447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.521074 (+ 20627us) service_pool.cc:224] Handling call
0709 12:10:21.582535 (+5061461us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b54003be054. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:21.582676 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1993296256}
I20260709 12:10:21.587046 12675 rpcz_store.cc:276] 0709 12:10:20.471595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:21.583598 (+1112003us) service_pool.cc:224] Handling call
0709 12:10:21.584085 (+   487us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:21.586341 (+  2256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:21.586516 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:21.587273 12680 rpcz_store.cc:276] 0709 12:10:20.254480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:21.573696 (+1319216us) service_pool.cc:224] Handling call
0709 12:10:21.584897 (+ 11201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:21.586444 (+  1547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:21.586669 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:21.588845 12677 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:21.589389 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1619) took 1117 ms. Trace:
I20260709 12:10:21.589597 12677 rpcz_store.cc:276] 0709 12:10:20.472174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:21.588364 (+1116190us) service_pool.cc:224] Handling call
0709 12:10:21.588667 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:21.589187 (+   520us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:21.589346 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:21.584573 13334 rpcz_store.cc:276] 0709 12:10:18.764708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:18.769310 (+  4602us) service_pool.cc:224] Handling call
0709 12:10:18.840195 (+ 70885us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:19.923811 (+1083616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:21.575376 (+1651565us) spinlock_profiling.cc:243] Waited 916 ms on lock 0x7b54001b49d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:21.575615 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2107678080}
I20260709 12:10:21.582880 14445 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:21.556356 14596 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:21.597350 12672 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:10:21.598800 12677 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:21.590842 14449 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:21.614735 12529 raft_consensus.cc:3055] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:21.615182 12529 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:21.615962 14476 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:10:21.616570 14601 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.783s	user 0.000s	sys 0.009s
I20260709 12:10:21.616839 14476 consensus_queue.cc:1243] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:10:21.616858 14573 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:21.617899 14625 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:21.618427 14625 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:21.619107 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1440) took 2763 ms. Trace:
I20260709 12:10:21.619417 14625 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:21.620855 14625 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:21.619525 12673 rpcz_store.cc:276] 0709 12:10:18.855376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:19.412360 (+556984us) service_pool.cc:224] Handling call
0709 12:10:19.461228 (+ 48868us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:20.665599 (+1204371us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:21.618816 (+953217us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54002c5454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:21.619036 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2758980480}
I20260709 12:10:21.623085 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1618) took 1151 ms. Trace:
I20260709 12:10:21.623490 12672 rpcz_store.cc:276] 0709 12:10:20.471918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:21.586589 (+1114671us) service_pool.cc:224] Handling call
0709 12:10:21.597091 (+ 10502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:21.622811 (+ 25720us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:21.623020 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:21.626606 12680 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:21.630005 12675 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:10:21.633280 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:21.636685 12672 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:21.638500 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:21.638734 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:21.646591 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:21.647017 14543 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:21.648144 14445 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:21.648661 14478 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:21.634483 14557 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:21.650449 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:21.651168 12528 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:21.650928 14557 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:21.655494 14292 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:21.656190 14292 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:21.658535 12672 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:21.661255 12680 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:21.670974 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1478) took 2957 ms. Trace:
W20260709 12:10:21.676844 12523 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1514) took 1894 ms (client timeout 1943 ms). Trace:
W20260709 12:10:21.677345 12523 rpcz_store.cc:269] 0709 12:10:19.782012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:19.828453 (+ 46441us) service_pool.cc:224] Handling call
0709 12:10:21.676604 (+1848151us) spinlock_profiling.cc:243] Waited 592 ms on lock 0x7b54003c0854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:21.676775 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1360812288}
I20260709 12:10:21.679992 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:21.680851 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 119) took 1015 ms. Trace:
I20260709 12:10:21.682523 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:21.683194 12523 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
I20260709 12:10:21.671523 12526 rpcz_store.cc:276] 0709 12:10:18.713864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:18.714091 (+   227us) service_pool.cc:224] Handling call
0709 12:10:18.739004 (+ 24913us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:18.762759 (+ 23755us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:21.670605 (+2907846us) spinlock_profiling.cc:243] Waited 3.95 ms on lock 0x7b5400408654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:21.670867 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9091328}
I20260709 12:10:21.681063 12676 rpcz_store.cc:276] 0709 12:10:20.665220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:20.757682 (+ 92462us) service_pool.cc:224] Handling call
0709 12:10:21.679834 (+922152us) spinlock_profiling.cc:243] Waited 1.75 ms on lock 0x7b58001f2c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:10:21.680788 (+   954us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4032896}
W20260709 12:10:21.686733 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 71644.4us
I20260709 12:10:21.688110 12523 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:10:21.688511 14453 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:10:21.692021 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:21.692461 12523 raft_consensus.cc:606] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:21.692888 13334 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:21.693776 14522 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:21.694780 12496 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:21.695025 14522 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:21.688874 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:21.691030 14369 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:21.705163 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:21.705863 13331 raft_consensus.cc:2376] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2. Re-sending same reply.
I20260709 12:10:21.691468 13336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38482 (request call id 281) took 3762 ms. Trace:
W20260709 12:10:21.691793 12646 connection.cc:256] RPC call timeout handler was delayed by -2.5829354290001447s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1663s for a response
I20260709 12:10:21.697125 12496 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:21.697636 12523 raft_consensus.cc:614] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:10:21.687714 13305 connection.cc:256] RPC call timeout handler was delayed by -1.8708735259999685s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18530000000000002s for a response
I20260709 12:10:21.699342 14599 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:21.700395 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:21.700673 14573 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:21.701200 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:21.706735 13336 rpcz_store.cc:276] 0709 12:10:17.928915 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.932590 (+  3675us) service_pool.cc:224] Handling call
0709 12:10:20.151031 (+2218441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:20.261091 (+110060us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:21.689563 (+1428472us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b5400478454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:21.691270 (+  1707us) spinlock_profiling.cc:243] Waited 2.16 s on lock 0x7b5400478450. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:21.691411 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5201701504}
I20260709 12:10:21.700435 14526 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:21.702001 13335 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:10:21.703083 14478 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:21.703882 14455 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:21.703893 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:10:21.707262 12646 connection.cc:256] RPC call timeout handler was delayed by -2.5748354290001316s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16720000000000002s for a response
W20260709 12:10:21.717291 12646 connection.cc:256] RPC call timeout handler was delayed by -2.5703354290001243s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16770000000000002s for a response
W20260709 12:10:21.717675 12646 connection.cc:256] RPC call timeout handler was delayed by -2.5046354290000181s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17500000000000002s for a response
I20260709 12:10:21.708894 12496 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:10:21.717994 12646 connection.cc:256] RPC call timeout handler was delayed by -2.5001354290000108s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1755s for a response
I20260709 12:10:21.712750 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:21.716192 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:21.709718 13305 connection.cc:256] RPC call timeout handler was delayed by -1.6329758080000829s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1976s for a response
W20260709 12:10:21.718465 12646 connection.cc:256] RPC call timeout handler was delayed by -2.4757004350001353s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15460000000000002s for a response
W20260709 12:10:21.718772 13305 connection.cc:256] RPC call timeout handler was delayed by -1.5874286729999767s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1595s for a response
W20260709 12:10:21.718818 12646 connection.cc:256] RPC call timeout handler was delayed by -2.4236354290001145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18400000000000002s for a response
W20260709 12:10:21.719280 13305 connection.cc:256] RPC call timeout handler was delayed by -1.4026154639998367s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19490000000000002s for a response
W20260709 12:10:21.719350 12646 connection.cc:256] RPC call timeout handler was delayed by -2.343535428999985s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19290000000000002s for a response
W20260709 12:10:21.719980 12646 connection.cc:256] RPC call timeout handler was delayed by -2.3381354289999763s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1935s for a response
I20260709 12:10:21.720546 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:21.721186 13339 raft_consensus.cc:2376] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3. Re-sending same reply.
W20260709 12:10:21.720361 12646 connection.cc:256] RPC call timeout handler was delayed by -2.3309354289999646s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19430000000000003s for a response
I20260709 12:10:21.722611 13338 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:21.723057 13338 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:10:21.723549 12646 connection.cc:256] RPC call timeout handler was delayed by -2.2283354290000261s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2057s for a response
W20260709 12:10:21.723805 12646 connection.cc:256] RPC call timeout handler was delayed by -2.21843542900001s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2068s for a response
W20260709 12:10:21.724042 12646 connection.cc:256] RPC call timeout handler was delayed by -2.212135429s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20750000000000002s for a response
W20260709 12:10:21.724283 12646 connection.cc:256] RPC call timeout handler was delayed by -2.1950354289999723s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2094s for a response
W20260709 12:10:21.724507 12646 connection.cc:256] RPC call timeout handler was delayed by -2.1788354290001735s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21120000000000003s for a response
W20260709 12:10:21.724726 12646 connection.cc:256] RPC call timeout handler was delayed by -2.1752354290001676s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2116s for a response
W20260709 12:10:21.724958 12646 connection.cc:256] RPC call timeout handler was delayed by -2.15813542900014s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2135s for a response
W20260709 12:10:21.725169 12646 connection.cc:256] RPC call timeout handler was delayed by -2.1554354290001356s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2138s for a response
W20260709 12:10:21.725417 12646 connection.cc:256] RPC call timeout handler was delayed by -2.1353764119999141s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2089s for a response
W20260709 12:10:21.725659 12646 connection.cc:256] RPC call timeout handler was delayed by -2.0105354290001287s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2299s for a response
W20260709 12:10:21.725873 12646 connection.cc:256] RPC call timeout handler was delayed by -1.981406214000117s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16690000000000002s for a response
I20260709 12:10:21.726502 13333 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:21.726677 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:21.727026 13331 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:21.727389 13339 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:21.711998 14057 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:21.729245 13334 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:21.729466 14057 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:21.729928 14057 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:21.705580 14369 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:21.733243 14369 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:21.738651 13338 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:21.740198 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:21.740917 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:21.741762 13956 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:21.742373 13956 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
W20260709 12:10:21.744711 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:21.745078 13956 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:21.745873 12601 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
mode: GRACEFUL
new_leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
 from {username='slave'} at 127.0.0.1:32832
I20260709 12:10:21.746548 12601 raft_consensus.cc:606] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Received request to transfer leadership to TS a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:21.746898 12601 raft_consensus.cc:614] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:21.748407 13338 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:21.751983 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:21.754688 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1489) took 1776 ms. Trace:
I20260709 12:10:21.755115 12525 rpcz_store.cc:276] 0709 12:10:19.978564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:19.981928 (+  3364us) service_pool.cc:224] Handling call
0709 12:10:20.151967 (+170039us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:20.268261 (+116294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:20.268295 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:20.268343 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:20.268377 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:21.754465 (+1486088us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b5400015754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:21.754516 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:21.754630 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":266934912}
I20260709 12:10:21.721987 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:21.767619 13334 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:21.767835 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:21.734041 14425 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:21.773402 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:21.775879 12646 connection.cc:256] RPC call timeout handler was delayed by -1.7098043670000607s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15400000000000003s for a response
I20260709 12:10:21.757225 14292 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:21.769981 12529 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:21.754189 14478 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:21.783006 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:21.793361 13305 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:21.793571 12529 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:21.796061 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:21.797773 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1583) took 1862 ms. Trace:
W20260709 12:10:21.784257 12646 connection.cc:256] RPC call timeout handler was delayed by -1.2932354290001058s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30960000000000004s for a response
I20260709 12:10:21.705197 14618 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:21.799319 13336 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:21.779160 12523 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:10:21.795673 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 65256.8us
I20260709 12:10:21.802081 14292 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:10:21.804667 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 45448.5us
I20260709 12:10:21.805874 12600 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:21.810858 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:21.811533 12601 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:21.808706 12646 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:21.808998 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:21.714953 14608 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:21.813612 12523 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:21.815902 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:21.817060 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:21.818373 12646 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:21.817627 12571 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:21.796123 14453 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:21.824800 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:21.798296 12606 rpcz_store.cc:276] 0709 12:10:19.934981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:20.567810 (+632829us) service_pool.cc:224] Handling call
0709 12:10:20.596963 (+ 29153us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:20.597225 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:20.633948 (+ 36723us) log.cc:844] Serialized 14 byte log entry
0709 12:10:21.797391 (+1163443us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:21.797448 (+    57us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:21.797499 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:21.797572 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:21.797689 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1147802,"prepare.queue_time_us":37690,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"spinlock_wait_cycles":6100736,"thread_start_us":15031,"threads_started":1,"wal-append.queue_time_us":168756}
I20260709 12:10:21.823036 13337 raft_consensus.cc:2368] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 3. Current term is 4.
I20260709 12:10:21.822592 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:21.822364 12646 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:21.827436 12523 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:21.823726 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:21.828819 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:21.829895 12646 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:21.832620 12646 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:21.804611 14479 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:10:21.843331 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 48456.9us
I20260709 12:10:21.784982 14533 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:10:21.804690 14478 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:21.848009 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 86) took 2003 ms. Trace:
I20260709 12:10:21.813184 13335 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:21.846690 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:21.791612 14590 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:10:21.847185 12646 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:21.846141 12527 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 3: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:21.850049 13305 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:21.804865 14456 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:10:21.855444 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:21.853946 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:21.864656 12606 raft_consensus.cc:2410] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:21.827348 14292 consensus_queue.cc:1243] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:10:21.823873 14240 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:21.823745 14395 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:21.820907 14593 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:21.866647 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:21.867686 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:21.860517 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:21.828317 14454 raft_consensus.cc:2764] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260709 12:10:21.844568 14504 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:21.828140 13946 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:21.872915 12600 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:21.847432 14620 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:21.882638 12646 leader_election.cc:400] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3.
I20260709 12:10:21.883050 12646 leader_election.cc:403] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260709 12:10:21.877388 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:21.851262 14364 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:21.828974 14425 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.051s
I20260709 12:10:21.907541 14637 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:10:21.907953 14633 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:10:21.853585 13335 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:21.908206 14637 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:21.908829 14637 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:21.884277 14371 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:10:21.886812 14593 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:21.887143 13331 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:21.889573 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:21.889595 13946 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:21.890544 13338 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:10:21.878584 14619 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:21.909457 13335 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:21.910444 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:21.911068 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:21.908226 14240 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3.
I20260709 12:10:21.911737 14420 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:21.851778 14348 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.037s
I20260709 12:10:21.848021 14610 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.039s
I20260709 12:10:21.856560 14479 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:21.863507 14526 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 12:10:21.829213 14613 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:10:21.848097 14621 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.040s
I20260709 12:10:21.877905 13956 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:10:21.944003 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54501.6us
W20260709 12:10:21.945048 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 95207.7us
I20260709 12:10:21.886694 14615 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.071s
I20260709 12:10:21.884848 14475 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:21.957511 12572 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:21.911219 14618 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:21.970582 12494 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:21.976459 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:21.978029 14292 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:21.978916 14589 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:21.979410 14589 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:21.979826 14589 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:21.981036 14348 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:21.970643 14420 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:21.982837 14420 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:21.958792 14454 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:21.987049 14454 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:21.987608 14454 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:21.987733 14610 raft_consensus.cc:3055] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:21.988451 14610 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:21.966006 14478 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:21.960529 14057 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:22.001147 14057 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:22.001907 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:21.969367 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 98067.5us
I20260709 12:10:22.003110 12646 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:22.004530 12646 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:22.005690 12646 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:21.968510 13335 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:22.006398 14240 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:22.007504 14240 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:22.008325 14240 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:22.011230 12646 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:22.014219 12646 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:21.972708 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:22.017697 12523 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 3: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:10:22.017591 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 44970.9us
I20260709 12:10:22.003279 12525 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:10:22.022302 12525 raft_consensus.cc:606] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:22.020226 12571 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:22.036479 12528 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:22.039055 12528 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:22.016824 14527 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:22.055378 12571 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:22.059020 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 44941.9us
I20260709 12:10:22.063748 14593 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:22.063934 13335 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:22.064107 14442 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:22.064347 14613 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:10:22.065042 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:22.064960 14613 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:22.065559 14364 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:22.066608 14240 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:22.067333 14613 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:21.923545 14566 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.144s
I20260709 12:10:22.067879 12646 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:22.068140 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:22.069216 12646 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:22.069602 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:22.070292 12600 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:22.071623 14608 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:22.071823 14504 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:22.072234 14495 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:22.072481 14504 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:22.072662 14495 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:22.073082 14495 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:22.073205 14504 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:22.067368 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:22.075629 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:22.076629 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:22.076740 12646 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:22.076767 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 119360us
I20260709 12:10:22.077533 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:22.078526 12529 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:22.028998 14475 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:21.990299 14610 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:22.082232 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:22.081694 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 104113us
I20260709 12:10:22.083384 12646 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:22.083998 12494 consensus_peers.cc:597] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 -> Peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Couldn't send request to peer e56d487365394ac5ac5f1002926a5645. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:10:22.085284 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:22.085233 12646 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:22.060563 14642 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:22.060184 14425 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:22.091770 12494 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:22.083072 14476 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:10:22.078033 13918 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:22.084880 14442 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:22.081444 14593 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:22.093518 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:22.082859 14475 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:22.083240 14619 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:22.085752 14516 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:22.076061 14502 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.017s
I20260709 12:10:22.108358 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:22.091748 14057 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260709 12:10:22.080871 14620 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260709 12:10:22.089002 14364 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:22.114393 14608 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:22.085606 14292 raft_consensus.cc:697] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:22.121637 12646 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:22.133021 12646 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:22.127174 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:22.138573 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:22.130170 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:22.139503 12604 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:22.140568 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:22.140777 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:22.130848 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:22.142712 12679 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:22.133489 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:22.144279 12676 raft_consensus.cc:2393] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 3: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:22.133737 12572 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:22.134709 14502 raft_consensus.cc:3055] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:22.134768 14442 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:22.092775 14425 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:22.136885 14449 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:22.136962 14292 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:22.138290 14240 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:22.119386 14593 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:22.154927 14593 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:22.087163 14590 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:10:22.089246 14625 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:10:22.115949 14603 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:22.100039 14526 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:22.102988 14504 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:22.150735 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:22.132474 14516 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:22.149421 14502 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:22.178876 14502 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:22.180155 12605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1596) took 4768 ms (client timeout 1673 ms). Trace:
W20260709 12:10:22.180644 12605 rpcz_store.cc:269] 0709 12:10:17.412181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:17.412364 (+   183us) service_pool.cc:224] Handling call
0709 12:10:22.179964 (+4767600us) spinlock_profiling.cc:243] Waited 2.51 s on lock 0x7b54003110d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:22.180063 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5783905536}
I20260709 12:10:22.181216 12673 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:22.157900 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:22.182713 14567 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:22.183012 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:22.183473 14625 consensus_queue.cc:1243] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:10:22.186798 14486 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:22.187260 14486 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:22.187644 14486 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:22.164752 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:22.158735 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:22.181833 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:22.162828 12646 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:22.167706 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:22.169279 12494 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:22.192257 12677 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
W20260709 12:10:22.134932 14460 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260709 12:10:22.106173 13918 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:22.156538 14425 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:22.172144 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:22.191387 12676 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:22.194680 12680 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:22.172212 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:22.195662 12675 raft_consensus.cc:2376] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2. Re-sending same reply.
I20260709 12:10:22.176913 14557 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:22.179376 12678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1438) took 3324 ms (client timeout 1549 ms). Trace:
I20260709 12:10:22.173877 14643 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:22.180270 14502 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:22.203917 13918 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:22.204186 14643 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:22.204663 14643 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:22.164916 14613 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260709 12:10:22.203380 12678 rpcz_store.cc:269] 0709 12:10:18.854881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:18.894549 (+ 39668us) service_pool.cc:224] Handling call
0709 12:10:22.179154 (+3284605us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b54001414d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:22.179298 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3827377536}
I20260709 12:10:22.165057 14576 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:22.170275 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:22.166052 14619 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:22.196287 12526 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:22.166204 14603 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:22.206188 12606 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:22.191403 12605 raft_consensus.cc:2376] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3. Re-sending same reply.
I20260709 12:10:22.196043 14449 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:22.214111 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 123) took 1512 ms. Trace:
I20260709 12:10:22.222489 12671 rpcz_store.cc:276] 0709 12:10:20.701981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:21.216707 (+514726us) service_pool.cc:224] Handling call
0709 12:10:22.213900 (+997193us) spinlock_profiling.cc:243] Waited 70.3 ms on lock 0x7b54000c26d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:22.214044 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161601920}
I20260709 12:10:22.227106 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 97) took 2041 ms. Trace:
I20260709 12:10:22.227766 12531 rpcz_store.cc:276] 0709 12:10:20.185888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:20.385660 (+199772us) service_pool.cc:224] Handling call
0709 12:10:20.398485 (+ 12825us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:21.397155 (+998670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:22.226589 (+829434us) spinlock_profiling.cc:243] Waited 998 ms on lock 0x7b54003c1254. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:22.227010 (+   421us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2294951296}
I20260709 12:10:22.125169 12524 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:22.205526 14448 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:22.207803 14445 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:22.207026 14567 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.028s
I20260709 12:10:22.235889 14505 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:22.236270 14567 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:22.236518 14505 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:22.237782 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:22.238561 12605 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:22.236840 14567 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:22.232470 14449 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:22.231427 14536 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:22.237550 14420 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:22.264762 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1485) took 3399 ms. Trace:
I20260709 12:10:22.269019 12530 rpcz_store.cc:276] 0709 12:10:18.864894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:18.876402 (+ 11508us) service_pool.cc:224] Handling call
0709 12:10:18.931965 (+ 55563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:20.129987 (+1198022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:22.262944 (+2132957us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400210cd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:22.264687 (+  1743us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2701164928}
I20260709 12:10:22.270064 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:22.240609 14445 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:22.276013 12571 connection.cc:256] RPC call timeout handler was delayed by -1.8805816250001044s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1549s for a response
I20260709 12:10:22.266045 14363 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:22.264793 13956 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:22.288988 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:22.289717 12605 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:22.265470 14449 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:22.284790 14548 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 3.246s	user 0.009s	sys 0.005s
I20260709 12:10:22.280176 14380 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:22.279956 14141 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:22.266686 14576 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:22.266498 14536 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:22.301792 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:22.471880 12522 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:22.382470 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:22.285583 14505 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:22.298058 14609 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.484s	user 0.005s	sys 0.000s
I20260709 12:10:22.472576 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1584) took 2537 ms. Trace:
I20260709 12:10:21.848564 12602 rpcz_store.cc:276] 0709 12:10:19.844523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:19.844702 (+   179us) service_pool.cc:224] Handling call
0709 12:10:19.858683 (+ 13981us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:19.913466 (+ 54783us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:19.913499 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:19.923303 (+  9804us) log.cc:844] Serialized 52 byte log entry
0709 12:10:19.983091 (+ 59788us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:20.051270 (+ 68179us) log.cc:844] Serialized 14 byte log entry
0709 12:10:21.729952 (+1678682us) spinlock_profiling.cc:243] Waited 28 ms on lock 0x7b18002950c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:10:21.783454 (+ 53502us) log.cc:844] Serialized 14 byte log entry
0709 12:10:21.825291 (+ 41837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:21.825368 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:21.847761 (+ 22393us) raft_consensus.cc:1685] finished
0709 12:10:21.847784 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:21.847928 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":95941,"prepare.queue_time_us":72643,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":45,"raft.queue_time_us":1748017,"raft.run_cpu_time_us":605,"raft.run_wall_time_us":28153,"spinlock_wait_cycles":64378752,"thread_start_us":150580,"threads_started":3,"wal-append.queue_time_us":311470,"wal-append.run_cpu_time_us":2448,"wal-append.run_wall_time_us":1039151}
I20260709 12:10:22.483415 12598 rpcz_store.cc:276] 0709 12:10:19.935360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:20.583575 (+648215us) service_pool.cc:224] Handling call
0709 12:10:20.631449 (+ 47874us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:21.809797 (+1178348us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:10:22.083333 (+273536us) log.cc:844] Serialized 14 byte log entry
0709 12:10:22.191961 (+108628us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:22.214146 (+ 22185us) log.cc:844] Serialized 26 byte log entry
0709 12:10:22.214352 (+   206us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:22.220518 (+  6166us) log.cc:844] Serialized 14 byte log entry
0709 12:10:22.235460 (+ 14942us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:22.235570 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:22.472349 (+236779us) raft_consensus.cc:1685] finished
0709 12:10:22.472368 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:22.472513 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1192391,"prepare.queue_time_us":62461,"prepare.run_cpu_time_us":109,"prepare.run_wall_time_us":97,"raft.queue_time_us":1182707,"raft.run_cpu_time_us":447,"raft.run_wall_time_us":27884,"thread_start_us":353591,"threads_started":2,"wal-append.queue_time_us":108447}
I20260709 12:10:22.472623 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1490) took 2493 ms. Trace:
I20260709 12:10:22.473560 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:22.291399 14603 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:22.478806 14653 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:10:22.485867 12597 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:22.496654 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:22.485337 12522 rpcz_store.cc:276] 0709 12:10:19.978838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:20.040608 (+ 61770us) service_pool.cc:224] Handling call
0709 12:10:22.469321 (+2428713us) spinlock_profiling.cc:243] Waited 3.4 ms on lock 0x7b5800002a98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:10:22.469565 (+   244us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:22.472361 (+  2796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:22.472563 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7815424}
I20260709 12:10:22.488610 12605 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:22.297925 14363 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:22.298895 13956 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:22.488718 12606 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:22.499439 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:22.486681 14589 consensus_meta.cc:287] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.506s	user 0.012s	sys 0.000s
I20260709 12:10:22.500100 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:22.497696 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:22.490367 14652 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.195s	user 0.000s	sys 0.000s
I20260709 12:10:22.485884 14380 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:10:22.671172 14655 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 12:10:22.471664 14141 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:22.701093 12598 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:22.477929 14576 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:22.700229 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:22.800794 12606 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:22.671819 14651 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:10:22.506611 14538 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3c9b5315396948b2af3fa5480c9bee87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:10:22.476691 14475 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:22.496299 14596 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:22.723955 14527 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:22.588586 14649 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:10:22.529122 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:22.506906 14603 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:22.538805 14589 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:22.799584 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:22.805215 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:22.817718 12603 pending_rounds.cc:85] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 4
I20260709 12:10:22.845682 12603 pending_rounds.cc:107] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 3.5
W20260709 12:10:22.823623 14486 consensus_meta.cc:287] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.636s	user 0.012s	sys 0.000s
W20260709 12:10:22.824247 14495 consensus_meta.cc:287] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.751s	user 0.010s	sys 0.005s
W20260709 12:10:22.792778 14454 consensus_meta.cc:287] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.804s	user 0.009s	sys 0.009s
I20260709 12:10:22.798658 14380 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:22.870754 12598 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:22.825430 14371 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:22.846289 12603 raft_consensus.cc:2889] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:22.825459 14442 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:22.845768 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1621) took 1025 ms. Trace:
I20260709 12:10:22.887066 12597 rpcz_store.cc:276] 0709 12:10:21.820546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:22.799237 (+978691us) service_pool.cc:224] Handling call
0709 12:10:22.845667 (+ 46430us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:22.852229 14441 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:22.889539 12606 raft_consensus.cc:2410] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:22.896016 12597 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:10:22.896570 12597 pending_rounds.cc:85] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 1
I20260709 12:10:22.896811 12597 pending_rounds.cc:107] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:10:22.897150 12597 raft_consensus.cc:2889] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:22.900998 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1602) took 1121 ms. Trace:
I20260709 12:10:22.901636 12597 rpcz_store.cc:276] 0709 12:10:21.779963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:22.894958 (+1114995us) service_pool.cc:224] Handling call
0709 12:10:22.895312 (+   354us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:22.900292 (+  4980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:22.900894 (+   602us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:22.904002 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 1613) took 1084 ms. Trace:
I20260709 12:10:22.904570 12606 rpcz_store.cc:276] 0709 12:10:21.819255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:22.506488 (+687233us) service_pool.cc:224] Handling call
0709 12:10:22.903933 (+397445us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:22.905756 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:22.906631 12606 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 4
W20260709 12:10:22.912549 12674 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1608) took 2740 ms (client timeout 2014 ms). Trace:
I20260709 12:10:22.878662 14505 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:22.919842 12674 rpcz_store.cc:269] 0709 12:10:20.172411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:20.349329 (+176918us) service_pool.cc:224] Handling call
0709 12:10:22.912253 (+2562924us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b540008eb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:22.912445 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":577910144}
I20260709 12:10:22.893143 13335 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 4: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:22.925503 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:22.854254 14527 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:22.874840 14454 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:22.863051 14486 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:22.868945 14495 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:22.948001 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:22.912137 14660 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:22.893841 14371 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:22.955649 14527 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:22.894773 14442 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:22.964114 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:22.975987 12602 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:22.970661 14313 raft_consensus.cc:3055] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:22.976845 12606 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:22.972333 14412 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:22.938190 14505 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:22.969368 14369 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:22.981128 12605 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:22.950253 14657 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.009514 12597 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:23.017419 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:22.982821 14313 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:22.983896 14412 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:23.010794 14596 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.015097 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:22.985638 14363 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:23.029632 12525 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:22.988448 14369 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:22.991321 14292 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:23.032577 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:23.036329 12528 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:22.967468 14660 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:23.036123 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 98572.2us
I20260709 12:10:23.038162 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:23.043296 12529 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
W20260709 12:10:23.032788 14240 consensus_meta.cc:287] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.879s	user 0.014s	sys 0.005s
I20260709 12:10:23.055397 12572 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:23.057518 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:22.991103 14658 consensus_queue.cc:1059] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.055s
W20260709 12:10:23.039471 14593 consensus_meta.cc:287] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.884s	user 0.011s	sys 0.007s
I20260709 12:10:23.058554 12525 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 4: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:10:23.039374 14536 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.551s	user 0.009s	sys 0.002s
W20260709 12:10:23.052896 14502 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.848s	user 0.012s	sys 0.002s
I20260709 12:10:23.068384 14479 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:23.069705 14412 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:23.070142 14217 consensus_queue.cc:1059] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.007s
W20260709 12:10:23.098757 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:23.071388 14442 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:23.072378 12525 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:10:23.054068 14141 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:23.026202 14596 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.073035 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:23.073369 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:23.062390 14240 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:23.096040 14363 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:23.074568 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:23.074851 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:23.048616 14292 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.078430 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1618) took 1267 ms. Trace:
I20260709 12:10:23.082616 14369 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:23.060760 14380 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:23.087697 12674 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: [4.3-4.4]
I20260709 12:10:23.088529 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:23.090770 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1631) took 1461 ms. Trace:
I20260709 12:10:23.188704 12672 rpcz_store.cc:276] 0709 12:10:21.629384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:21.637879 (+  8495us) service_pool.cc:224] Handling call
0709 12:10:23.090469 (+1452590us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b54002c6854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:23.090643 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42391424}
I20260709 12:10:23.112529 14412 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:23.189324 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 250819us
I20260709 12:10:23.191799 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:10:23.140218 14619 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.924s	user 0.012s	sys 0.002s
I20260709 12:10:23.211930 14619 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:10:23.213470 12673 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:23.217417 14445 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:23.098745 14313 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.218900 14313 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:23.219884 14573 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.220093 12522 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:23.220446 14573 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:23.222767 14533 raft_consensus.cc:3055] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:23.223026 14369 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:23.223191 14533 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:23.224005 12526 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1527) took 1538 ms (client timeout 1581 ms). Trace:
I20260709 12:10:23.224159 14533 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:23.093185 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:23.225296 14533 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:23.226082 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:23.226908 14610 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:23.227301 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:23.227892 12600 raft_consensus.cc:2376] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6. Re-sending same reply.
I20260709 12:10:23.228709 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:23.229585 12600 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 4: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:23.229332 14573 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:23.098649 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:10:23.231499 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 132995us
I20260709 12:10:23.101040 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:23.234485 12675 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
W20260709 12:10:23.107793 14425 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.899s	user 0.012s	sys 0.001s
I20260709 12:10:23.005614 14657 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.110919 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:23.130784 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:23.249162 12679 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:23.074637 14454 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:23.138757 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:23.075273 14536 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.158151 12602 rpcz_store.cc:276] 0709 12:10:21.811395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:22.483957 (+672562us) service_pool.cc:224] Handling call
0709 12:10:23.078091 (+594134us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b540022fbd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:23.078332 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1099691264}
I20260709 12:10:23.107942 14660 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:23.108935 14589 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:23.092072 14495 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:23.194577 14486 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:23.152189 14363 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:23.096931 14659 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.133s
I20260709 12:10:23.145648 14141 raft_consensus.cc:3055] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:23.099649 14593 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:23.195531 12524 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 106) took 1571 ms (client timeout 1950 ms). Trace:
I20260709 12:10:23.150534 14575 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:10:23.265888 12524 rpcz_store.cc:269] 0709 12:10:21.623954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:21.624081 (+   127us) service_pool.cc:224] Handling call
0709 12:10:23.195345 (+1571264us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b5400053554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:23.195466 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1004229376}
I20260709 12:10:23.196431 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:23.197548 14371 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:23.198196 14598 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:23.199398 14643 consensus_meta.cc:287] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.994s	user 0.013s	sys 0.003s
I20260709 12:10:23.200320 14559 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.201637 14505 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:23.186820 13330 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
W20260709 12:10:23.202783 12527 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 1496) took 2533 ms (client timeout 2197 ms). Trace:
I20260709 12:10:23.203676 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1533) took 1462 ms. Trace:
I20260709 12:10:23.284190 12523 rpcz_store.cc:276] 0709 12:10:21.741046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:21.828543 (+ 87497us) service_pool.cc:224] Handling call
0709 12:10:23.203499 (+1374956us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b540038e0d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:23.203619 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":423556352}
I20260709 12:10:23.285866 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:23.203986 14516 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:23.283212 12527 rpcz_store.cc:269] 0709 12:10:20.668913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:21.253278 (+584365us) service_pool.cc:224] Handling call
0709 12:10:23.202559 (+1949281us) spinlock_profiling.cc:243] Waited 583 ms on lock 0x7b54003ca5d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:23.202709 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1340447488}
I20260709 12:10:23.204818 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:23.290346 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:23.291595 12524 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 2.
I20260709 12:10:23.205919 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:23.207244 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:23.211174 13918 raft_consensus.cc:3055] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Stepping down as leader of term 1
W20260709 12:10:23.224429 12526 rpcz_store.cc:269] 0709 12:10:21.685911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:21.688577 (+  2666us) service_pool.cc:224] Handling call
0709 12:10:23.223830 (+1535253us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b5400052b54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:23.223947 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1146563840}
W20260709 12:10:23.055689 14637 consensus_meta.cc:287] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.146s	user 0.018s	sys 0.012s
I20260709 12:10:23.233983 12531 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:23.236480 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:23.237581 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:23.234993 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:23.249522 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 57836.5us
I20260709 12:10:23.262002 12646 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:23.296665 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 62017.7us
I20260709 12:10:23.251480 14502 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:23.234429 13946 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:23.264693 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1606) took 1483 ms. Trace:
I20260709 12:10:23.272785 12605 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:10:23.300586 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1603) took 1520 ms. Trace:
I20260709 12:10:23.255278 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:23.271945 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:23.276793 14449 consensus_meta.cc:287] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.808s	user 0.011s	sys 0.000s
I20260709 12:10:23.304693 12606 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:23.293970 13918 raft_consensus.cc:740] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:23.277570 14665 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:23.278002 14643 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:23.280046 12631 heartbeater.cc:646] Failed to heartbeat to 127.11.13.190:46629 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.11.13.190:46629 timed out after 15.000s (SENT)
I20260709 12:10:23.280400 14536 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:23.266347 14141 raft_consensus.cc:740] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:23.262732 14363 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:23.204476 14412 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:23.263123 14660 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.274209 14575 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.304872 14516 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:23.297462 12571 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:23.300141 12602 rpcz_store.cc:276] 0709 12:10:21.781115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.253662 (+1472547us) service_pool.cc:224] Handling call
0709 12:10:23.263962 (+ 10300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:23.264220 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:23.264259 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:23.264326 (+    67us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:23.264372 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:23.264445 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:23.264600 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:23.301005 12605 rpcz_store.cc:276] 0709 12:10:21.780226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.253477 (+1473251us) service_pool.cc:224] Handling call
0709 12:10:23.269535 (+ 16058us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:23.300278 (+ 30743us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:23.300524 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:23.299405 12600 raft_consensus.cc:2410] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:10:23.301245 12569 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:23.303032 12646 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:23.338519 12571 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:23.305379 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1608) took 1513 ms. Trace:
I20260709 12:10:23.340301 12606 rpcz_store.cc:276] 0709 12:10:21.791984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.291848 (+1499864us) service_pool.cc:224] Handling call
0709 12:10:23.301981 (+ 10133us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:23.305106 (+  3125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:23.305323 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:23.276865 14593 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:23.252194 14425 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.312633 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:23.292402 14559 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.351101 12602 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
W20260709 12:10:23.343354 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 330086us
I20260709 12:10:23.352006 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1610) took 1552 ms. Trace:
I20260709 12:10:23.314688 13335 raft_consensus.cc:2410] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:10:23.315724 14141 consensus_queue.cc:260] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.307698 14371 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:10:23.294267 12601 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1620) took 1477 ms (client timeout 1858 ms). Trace:
I20260709 12:10:23.336362 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:23.307766 14449 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.348906 12600 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:23.361773 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1623) took 1530 ms. Trace:
I20260709 12:10:23.362291 12600 rpcz_store.cc:276] 0709 12:10:21.831764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.348247 (+1516483us) service_pool.cc:224] Handling call
0709 12:10:23.348588 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:23.361409 (+ 12821us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:23.361690 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:23.359117 12601 rpcz_store.cc:269] 0709 12:10:21.816988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.206964 (+1389976us) service_pool.cc:224] Handling call
0709 12:10:23.294200 (+ 87236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:23.348327 14459 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:23.349574 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:23.370342 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:23.372237 12571 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:23.352371 12602 rpcz_store.cc:276] 0709 12:10:21.799498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.337604 (+1538106us) service_pool.cc:224] Handling call
0709 12:10:23.350916 (+ 13312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:23.351588 (+   672us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:23.351940 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:23.355433 12605 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:23.373854 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1615) took 1563 ms. Trace:
I20260709 12:10:23.345858 14502 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:23.299805 14637 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:23.336377 14516 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:23.359885 12527 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
W20260709 12:10:23.347075 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 45928.2us
I20260709 12:10:23.337224 13331 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:23.345289 12646 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:10:23.364104 13337 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 289) took 1665 ms (client timeout 1727 ms). Trace:
I20260709 12:10:23.322947 14369 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:10:23.365023 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:23.366541 12599 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:10:23.348834 14659 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:23.374233 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:23.384443 13339 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 364) took 1599 ms (client timeout 1840 ms). Trace:
I20260709 12:10:23.385807 12646 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:23.374297 12605 rpcz_store.cc:276] 0709 12:10:21.810546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.337707 (+1527161us) service_pool.cc:224] Handling call
0709 12:10:23.355257 (+ 17550us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:23.373510 (+ 18253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:23.373781 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:23.387452 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 154538us
I20260709 12:10:23.388448 12571 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:23.376192 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:23.384078 13337 rpcz_store.cc:269] 0709 12:10:21.698402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:21.698564 (+   162us) service_pool.cc:224] Handling call
0709 12:10:23.363901 (+1665337us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54004781d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:23.364033 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":241001856}
I20260709 12:10:23.347275 14512 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:10:23.391278 13305 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:23.398954 12602 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:23.343585 14660 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.399727 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1616) took 1509 ms. Trace:
I20260709 12:10:23.404093 12602 rpcz_store.cc:276] 0709 12:10:21.890211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.373597 (+1483386us) service_pool.cc:224] Handling call
0709 12:10:23.396523 (+ 22926us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:23.399429 (+  2906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:23.399656 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:23.366971 14420 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:23.354774 14656 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.022s
I20260709 12:10:23.400413 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:23.367906 14459 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:23.385862 13339 rpcz_store.cc:269] 0709 12:10:21.785328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:21.785493 (+   165us) service_pool.cc:224] Handling call
0709 12:10:23.364864 (+1579371us) spinlock_profiling.cc:243] Waited 6.6 ms on lock 0x7b5800163298. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:10:23.384378 (+ 19514us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15181568}
I20260709 12:10:23.376585 14502 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:23.377141 14371 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:23.377359 14593 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:23.423627 12646 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:23.396503 14516 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:23.398540 13946 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:23.433911 12602 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:23.444103 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1631) took 1487 ms. Trace:
W20260709 12:10:23.444170 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 43881.6us
I20260709 12:10:23.444550 12602 rpcz_store.cc:276] 0709 12:10:21.957170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.405022 (+1447852us) service_pool.cc:224] Handling call
0709 12:10:23.433637 (+ 28615us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:23.443790 (+ 10153us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:23.444028 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:23.438908 14679 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:23.439435 14680 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:10:23.440418 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 64364.2us
W20260709 12:10:23.455782 13302 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.690s (SENT) [suppressed 22 similar messages]
I20260709 12:10:23.434355 14502 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:23.398680 14505 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:23.431346 14671 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:23.451313 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:23.456776 12671 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:23.457335 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:23.451702 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:23.452517 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:23.459192 12527 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:23.459417 12676 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:23.398320 14664 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:23.461498 12602 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:23.357101 13918 consensus_queue.cc:260] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.436844 14593 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:23.434631 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:23.437927 14678 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.460681 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:23.454612 12679 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:23.447669 14454 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:23.462780 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:23.469007 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1619) took 1474 ms. Trace:
I20260709 12:10:23.475387 12675 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:23.475234 12602 rpcz_store.cc:276] 0709 12:10:21.998411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.445602 (+1447191us) service_pool.cc:224] Handling call
0709 12:10:23.461307 (+ 15705us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:23.468492 (+  7185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:23.468877 (+   385us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:23.475028 12527 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:23.460430 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:23.470947 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:23.476543 12672 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:23.443068 13946 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:23.429343 14665 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:23.432832 14459 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:23.460144 14505 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:23.491050 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:23.492220 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:23.493010 12676 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:23.498679 12671 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:23.477581 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:23.480370 13946 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:23.480789 14371 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:23.458982 14671 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:23.462177 14662 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.455834 14680 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:23.490217 12672 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:23.443408 14657 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:23.486706 14575 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:23.462656 14675 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:23.461877 14664 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:23.471422 14593 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:23.457497 14495 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.034s
I20260709 12:10:23.518345 14680 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:23.507503 14412 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:23.502206 14505 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:23.543275 12677 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 4: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:10:23.543382 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 87662.5us
I20260709 12:10:23.547533 13302 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:23.479710 14678 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:23.519269 14454 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:23.505658 14671 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:23.596076 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:23.596989 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:23.602459 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:23.591792 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:23.603857 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:23.509938 12522 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:23.606899 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:23.606298 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 182814us
I20260709 12:10:23.610513 12676 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:23.541486 14664 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:23.612946 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:23.619139 14369 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:23.619536 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:23.615124 12600 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:10:23.615628 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:23.621905 13302 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:23.544872 14675 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:23.622232 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1622) took 1622 ms. Trace:
I20260709 12:10:23.625887 12600 rpcz_store.cc:276] 0709 12:10:21.999783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.614540 (+1614757us) service_pool.cc:224] Handling call
0709 12:10:23.614839 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:23.621931 (+  7092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:23.622175 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:23.613771 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:23.616482 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:23.623570 13302 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:23.624809 13337 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:23.631119 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:23.542305 14363 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.611954 14412 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:23.636199 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:23.638471 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:23.518719 14662 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.598224 14677 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:23.611634 14425 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:23.614737 14364 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:23.613000 14643 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:23.632475 14369 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:23.629925 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:23.672073 12676 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:23.616564 14460 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:23.675001 14637 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:23.676888 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:23.679597 12528 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:10:23.625586 14371 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:23.633749 13302 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:23.639595 14309 tablet_copy_client.cc:538] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:10:23.621204 14619 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:10:23.634905 14674 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:23.652118 14412 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:23.627583 14512 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:10:23.662572 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:23.635372 14449 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:23.666699 12524 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:23.685051 12528 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:10:23.641222 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:23.721087 12600 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:23.688144 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:23.690538 12526 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 5: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:10:23.692874 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 59250.6us
I20260709 12:10:23.700838 12604 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:23.634761 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:23.708491 12676 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
W20260709 12:10:23.710294 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:23.663779 14454 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.699826 14637 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:23.658372 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:23.725373 13302 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:23.730185 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:23.731730 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:23.738042 12494 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:23.703881 14677 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:10:23.740859 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 114622us
I20260709 12:10:23.757118 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:23.682734 14675 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:23.767463 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:23.771613 12529 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:23.626405 14665 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:23.775812 13337 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:23.744609 14656 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.781390 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:23.754405 14449 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:23.728155 14371 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:10:23.787209 13302 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:23.792908 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:23.795723 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 105583us
I20260709 12:10:23.781400 12494 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:23.759168 14674 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:23.794029 12600 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:23.804474 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:23.806157 12494 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:23.758507 14683 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:10:23.887257 14420 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:23.887996 14420 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:23.888198 14683 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:10:23.827977 14313 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:10:23.831311 14681 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:10:23.891400 14141 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:23.831458 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:23.894788 14371 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:23.895243 14371 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:23.896117 14371 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:23.897401 14371 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:23.832422 14662 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:23.833038 14677 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:23.842641 14512 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.905649 14516 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.906993 12605 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:23.906563 14512 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.909164 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1630) took 1976 ms. Trace:
I20260709 12:10:23.847363 14664 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.911443 14516 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:23.916540 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 99) took 1013 ms. Trace:
I20260709 12:10:23.917213 12597 rpcz_store.cc:276] 0709 12:10:22.902901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:22.929419 (+ 26518us) service_pool.cc:224] Handling call
0709 12:10:23.916273 (+986854us) spinlock_profiling.cc:243] Waited 52.2 ms on lock 0x7b54001171d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:23.916454 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":120012672}
I20260709 12:10:23.811028 14559 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:23.920269 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:23.922891 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:23.925283 12494 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:23.850628 14650 consensus_queue.cc:1059] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:23.850459 13946 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.851433 14363 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:23.853541 14610 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.626s	user 0.008s	sys 0.002s
I20260709 12:10:23.809852 14608 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:23.858773 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1601) took 2079 ms. Trace:
I20260709 12:10:23.948146 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:23.948107 12598 rpcz_store.cc:276] 0709 12:10:21.779408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:22.812333 (+1032925us) service_pool.cc:224] Handling call
0709 12:10:22.862880 (+ 50547us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:22.871563 (+  8683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:23.858491 (+986928us) spinlock_profiling.cc:243] Waited 7.52 ms on lock 0x7b54002f1a54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:23.858705 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17298176}
I20260709 12:10:23.860873 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1591) took 3921 ms. Trace:
I20260709 12:10:23.862862 12527 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: [4.4-4.4]
I20260709 12:10:23.949563 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:23.951416 12679 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:23.806464 14637 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:23.794401 14656 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.808378 14522 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:23.872102 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:23.979090 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:23.979467 12674 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:23.980342 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:23.980990 12674 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:23.873526 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1595) took 3932 ms. Trace:
I20260709 12:10:23.950558 14680 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.982292 12599 rpcz_store.cc:276] 0709 12:10:19.941170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:20.638091 (+696921us) service_pool.cc:224] Handling call
0709 12:10:20.656710 (+ 18619us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:23.384783 (+2728073us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:23.873078 (+488295us) spinlock_profiling.cc:243] Waited 2.71 s on lock 0x7b54002f2454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:23.873442 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6231018880}
I20260709 12:10:23.979144 14522 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:23.985481 14522 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:23.875059 14687 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:10:23.988296 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 107) took 1052 ms. Trace:
I20260709 12:10:23.988582 14660 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:23.988773 12600 rpcz_store.cc:276] 0709 12:10:22.936016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.987243 (+1051227us) service_pool.cc:224] Handling call
0709 12:10:23.987652 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:23.987870 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:23.987902 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:23.987948 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:23.987984 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:23.988044 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:23.988187 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:23.875253 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:23.878998 12524 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:23.990550 12672 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:23.991729 12524 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:23.880610 14502 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.881278 13956 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:23.884145 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:23.980342 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:23.993350 13337 raft_consensus.cc:2410] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:10:23.882889 14292 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:23.886864 14412 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:23.908461 14420 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:23.929071 12601 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:23.913374 14664 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:23.928881 14369 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 6 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:23.934093 14526 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.943146 14459 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.944567 14613 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:23.946951 14686 raft_consensus.cc:3055] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:23.946856 13946 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:23.924428 14538 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.564s	user 0.011s	sys 0.000s
I20260709 12:10:23.953018 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:23.864985 14505 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:10:23.949407 12603 rpcz_store.cc:276] 0709 12:10:19.939894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:20.596935 (+657041us) service_pool.cc:224] Handling call
0709 12:10:20.638962 (+ 42027us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:22.891351 (+2252389us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:22.891400 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:22.892020 (+   620us) log.cc:844] Serialized 26 byte log entry
0709 12:10:23.187586 (+295566us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:23.391153 (+203567us) log.cc:844] Serialized 14 byte log entry
0709 12:10:23.391810 (+   657us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:23.860015 (+468205us) spinlock_profiling.cc:243] Waited 5.91 ms on lock 0x7b18002c7e20. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419
0709 12:10:23.860532 (+   517us) spinlock_profiling.cc:243] Waited 2.18 s on lock 0x7b5400231c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:23.860582 (+    50us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:23.860645 (+    63us) raft_consensus.cc:1685] finished
0709 12:10:23.860657 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:23.860808 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10072,"prepare.queue_time_us":283008,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":147,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":8094,"spinlock_wait_cycles":5023714944,"thread_start_us":430759,"threads_started":2,"wal-append.queue_time_us":371591}
I20260709 12:10:23.924618 14449 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:23.909912 14512 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:23.952123 14364 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:23.985054 12677 raft_consensus.cc:2368] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5.
I20260709 12:10:23.986074 14596 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:23.991936 12672 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:23.993577 12598 raft_consensus.cc:2410] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:23.909673 12605 rpcz_store.cc:276] 0709 12:10:21.933076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.387909 (+1454833us) service_pool.cc:224] Handling call
0709 12:10:23.418362 (+ 30453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:23.907434 (+489072us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:23.908805 (+  1371us) spinlock_profiling.cc:243] Waited 488 ms on lock 0x7b54003110d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:23.909081 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1123304704}
W20260709 12:10:24.001308 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 249997us
I20260709 12:10:24.045574 12597 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:23.996114 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1619) took 2184 ms. Trace:
I20260709 12:10:24.046523 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1634) took 1044 ms. Trace:
I20260709 12:10:24.046339 12606 rpcz_store.cc:276] 0709 12:10:21.811638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.341504 (+1529866us) service_pool.cc:224] Handling call
0709 12:10:23.344040 (+  2536us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:23.344254 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:23.344301 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:23.344366 (+    65us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:23.344405 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:23.344475 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:23.995933 (+651458us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b54001167d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:23.996048 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4763392}
I20260709 12:10:24.047214 12597 rpcz_store.cc:276] 0709 12:10:23.002053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.028181 (+1026128us) service_pool.cc:224] Handling call
0709 12:10:24.045125 (+ 16944us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:24.046191 (+  1066us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.046446 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:24.048748 12643 leader_election.cc:400] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 3. Current term is 4.
I20260709 12:10:24.036171 14449 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:24.038410 14369 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:24.051682 14505 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:24.052898 14309 tablet_bootstrap.cc:492] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
I20260709 12:10:24.009213 12599 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:10:24.001948 12678 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:23.997720 12569 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:24.023937 14691 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:23.968007 14608 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:23.975483 14679 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.512s	user 0.009s	sys 0.005s
I20260709 12:10:23.983043 14486 raft_consensus.cc:3055] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:23.944128 14593 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.984349 14680 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:23.998924 14674 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:10:24.049252 12643 leader_election.cc:403] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:10:24.038380 14502 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:24.049749 14449 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:24.024140 14522 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:24.008872 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:24.011323 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1633) took 1009 ms. Trace:
I20260709 12:10:24.077524 12600 rpcz_store.cc:276] 0709 12:10:23.001767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.009993 (+1008226us) service_pool.cc:224] Handling call
0709 12:10:24.010579 (+   586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:24.010838 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:24.010880 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:24.010946 (+    66us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:24.010998 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.011074 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:24.011238 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:24.079869 12600 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:23.965355 14567 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:24.024869 14686 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:24.092891 14686 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:24.063544 14486 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:24.094375 14686 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:24.095741 14522 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:24.095232 14486 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:24.096441 14420 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:24.053539 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:24.097404 12672 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:24.056649 12605 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:24.023767 14526 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:23.995601 13302 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:24.118687 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 123197us
W20260709 12:10:24.077983 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 69275.3us
W20260709 12:10:24.054759 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 57088.6us
I20260709 12:10:24.081228 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:24.120939 12571 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:24.121336 12600 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:24.059231 14505 raft_consensus.cc:2764] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 12:10:24.067829 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:24.090221 13946 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:24.086537 14680 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 3. Current term is 4.
I20260709 12:10:24.097189 14486 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:24.105530 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:24.145103 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:24.145249 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:24.145690 12523 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 3: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:24.106885 12602 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:24.145975 12524 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:24.124039 12672 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:24.124989 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:24.068339 14679 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:24.124080 12571 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:24.136188 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:24.157058 13337 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:24.141479 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:24.142338 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:24.093528 14671 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:24.158215 12524 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:24.148037 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:24.129506 14665 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:24.126518 14502 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:24.149822 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:24.142493 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:24.260921 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47238.1us
I20260709 12:10:24.146261 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:24.264617 12600 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:24.264729 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:24.158275 12522 raft_consensus.cc:2410] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:24.265486 12672 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 3: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:24.260531 14412 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:24.262925 14680 consensus_queue.cc:1059] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:10:24.142068 14660 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:24.281584 12526 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:10:24.136845 14688 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:24.063710 14691 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:24.267549 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:24.172147 14538 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.StartTabletCopy from 127.0.0.1:38472 (request call id 159) took 5952 ms. Trace:
I20260709 12:10:24.269572 12675 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:10:24.172448 14690 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.118s
I20260709 12:10:24.274643 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:24.267148 14576 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:24.268234 14665 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:24.276371 14659 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:24.277196 14526 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:24.277340 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:24.266822 14412 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:24.291949 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:24.317173 12672 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 2.
I20260709 12:10:24.117033 14512 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:24.257648 13946 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:24.263587 14505 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:24.290851 14619 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.008s	sys 0.033s
I20260709 12:10:24.289801 14688 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:24.306010 14659 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:24.347664 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:24.314368 14656 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:24.329831 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:24.359043 12672 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:24.360566 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:24.294529 14420 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.025s
I20260709 12:10:24.315289 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:24.342741 14688 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:24.344367 14691 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:24.332669 14512 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:24.336645 14619 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.009s	sys 0.034s
I20260709 12:10:24.333457 14459 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:24.350430 14593 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:24.352147 14671 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:24.326557 14412 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:24.263705 14608 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:24.295046 14660 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:24.331159 14576 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:24.329764 13946 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:24.325531 12680 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:24.357745 14659 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:24.340029 14505 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:24.389492 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 306930us
I20260709 12:10:24.372596 14656 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:10:24.399888 13304 leader_election.cc:336] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.950s (SENT)
I20260709 12:10:24.374749 14619 raft_consensus.cc:697] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:24.367148 14449 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:24.387712 14576 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:24.396550 13946 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:24.294836 14538 rpcz_store.cc:276] 0709 12:10:18.211573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:18.229897 (+ 18324us) service_pool.cc:224] Handling call
0709 12:10:18.644069 (+414172us) ts_tablet_manager.cc:934] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
0709 12:10:22.190217 (+3546148us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:10:23.281337 (+1091120us) tablet_metadata.cc:665] Metadata flushed
0709 12:10:23.281471 (+   134us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:10:24.039443 (+757972us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:10:24.074903 (+ 35460us) tablet_replica.cc:206] Creating consensus instance
0709 12:10:24.144937 (+ 70034us) ts_tablet_manager.cc:989] Replica registered.
0709 12:10:24.159524 (+ 14587us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":606,"tablet-copy.queue_time_us":305233,"thread_start_us":404319,"threads_started":2}
I20260709 12:10:24.408968 14576 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:24.419437 14694 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.063s	user 0.001s	sys 0.001s
I20260709 12:10:24.422755 14665 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:24.422197 14619 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:24.430039 14688 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:24.433486 14688 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:24.435205 14659 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:24.437652 14659 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:24.324457 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 399287us
W20260709 12:10:24.386608 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:24.510666 13332 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260709 12:10:24.477638 12531 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: []
W20260709 12:10:24.512275 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:24.384347 12672 raft_consensus.cc:2955] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:10:24.418452 14538 tablet_copy_client.cc:806] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 0 data blocks...
I20260709 12:10:24.521868 14538 tablet_copy_client.cc:670] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645: tablet copy: Starting download of 1 WAL segments...
W20260709 12:10:24.387953 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 302728us
I20260709 12:10:24.371212 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:24.481200 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 149868us
I20260709 12:10:24.476208 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1542) took 1300 ms. Trace:
I20260709 12:10:24.497375 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:24.502667 14505 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:24.506598 14512 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:24.506997 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:10:24.368846 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 53698.2us
I20260709 12:10:24.527422 12605 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 5: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:24.536021 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:24.536479 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:24.539645 12643 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:24.527338 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:24.543179 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1621) took 2543 ms. Trace:
I20260709 12:10:24.543762 12602 rpcz_store.cc:276] 0709 12:10:21.999401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.476183 (+1476782us) service_pool.cc:224] Handling call
0709 12:10:23.486517 (+ 10334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:24.146213 (+659696us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.542468 (+396255us) spinlock_profiling.cc:243] Waited 620 ms on lock 0x7b54003a0cd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:24.542906 (+   438us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1426026368}
I20260709 12:10:24.549810 12603 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260709 12:10:24.550999 14313 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:10:24.552654 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:24.553367 12606 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:24.553593 12496 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:24.560340 14659 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:24.560981 12571 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:24.561429 14659 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:24.562592 14656 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:24.563498 14656 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:24.566679 14420 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:24.567420 14420 raft_consensus.cc:697] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:24.570252 14420 consensus_queue.cc:237] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:24.571784 13946 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:24.575230 13946 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:24.576493 14449 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:24.559166 12602 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:10:24.558509 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1488) took 1014 ms. Trace:
I20260709 12:10:24.582540 12605 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:24.527815 12529 rpcz_store.cc:276] 0709 12:10:23.175750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.775030 (+599280us) service_pool.cc:224] Handling call
0709 12:10:23.792892 (+ 17862us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:23.793170 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:23.793209 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:23.793770 (+   561us) log.cc:844] Serialized 26 byte log entry
0709 12:10:24.211736 (+417966us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:24.211783 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.211858 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:24.475170 (+263312us) raft_consensus.cc:1685] finished
0709 12:10:24.475249 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:24.475770 (+   521us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":336704,"threads_started":1,"wal-append.queue_time_us":539140}
I20260709 12:10:24.584303 14313 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:24.584827 14475 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:24.587714 14619 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:24.594537 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1636) took 1567 ms. Trace:
I20260709 12:10:24.565013 14656 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:24.596308 12528 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:24.595448 12603 rpcz_store.cc:276] 0709 12:10:23.014049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.029052 (+1015003us) service_pool.cc:224] Handling call
0709 12:10:24.548659 (+519607us) spinlock_profiling.cc:243] Waited 1.79 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fbeee 000000000036eb39 00007f148d96f1e0 00007f148da380c4 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:10:24.548994 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:24.556127 (+  7133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.581270 (+ 25143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4136064}
W20260709 12:10:24.602949 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 76988us
I20260709 12:10:24.589447 12600 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:24.605465 14522 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:24.605886 14313 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:24.607260 14619 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:24.607254 14523 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:24.636840 14691 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:24.637519 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 137) took 1042 ms. Trace:
I20260709 12:10:24.605553 14533 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260709 12:10:24.638566 14691 raft_consensus.cc:3055] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:24.607476 14512 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:10:24.639128 14691 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:24.639822 14512 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:24.640290 14512 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:24.638216 12674 rpcz_store.cc:276] 0709 12:10:23.595476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.981690 (+386214us) service_pool.cc:224] Handling call
0709 12:10:23.981995 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:23.998403 (+ 16408us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:10:24.306233 (+307830us) log.cc:844] Serialized 14 byte log entry
0709 12:10:24.615440 (+309207us) log.cc:844] Serialized 14 byte log entry
0709 12:10:24.636753 (+ 21313us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:24.636826 (+    73us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:24.636925 (+    99us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.637038 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:24.637299 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41535,"prepare.queue_time_us":370016,"prepare.run_cpu_time_us":204,"prepare.run_wall_time_us":201,"raft.queue_time_us":101050,"raft.run_cpu_time_us":398,"raft.run_wall_time_us":400,"thread_start_us":508353,"threads_started":2,"wal-append.queue_time_us":124966}
I20260709 12:10:24.640689 14691 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:24.641916 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1483) took 1173 ms. Trace:
W20260709 12:10:24.611163 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 57921.2us
I20260709 12:10:24.565040 14659 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:24.615144 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:24.616787 14678 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:24.620251 14449 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:10:24.622051 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:24.622566 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:24.580542 12675 rpcz_store.cc:276] 0709 12:10:23.541038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.599812 (+ 58774us) service_pool.cc:224] Handling call
0709 12:10:23.633775 (+ 33963us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:24.294968 (+661193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.554559 (+259591us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b54000ab9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:24.555270 (+   711us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1461577088}
I20260709 12:10:24.629017 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1625) took 2770 ms. Trace:
I20260709 12:10:24.630553 12598 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:24.642429 12678 rpcz_store.cc:276] 0709 12:10:23.468691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.469501 (+   810us) service_pool.cc:224] Handling call
0709 12:10:24.641430 (+1171929us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b54003bcc54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:24.641608 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1148079488}
I20260709 12:10:24.645515 14660 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:24.646368 14533 consensus_queue.cc:1243] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:10:24.641501 14512 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:24.625909 14576 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:24.647575 12496 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:24.648180 14659 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:24.650252 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1626) took 2558 ms. Trace:
I20260709 12:10:24.651744 14691 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:24.652346 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:24.653105 14560 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:24.654659 14313 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:24.654412 14475 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:24.654428 14502 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260709 12:10:24.657239 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:24.657673 14371 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:24.683900 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:24.659402 14608 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:24.684532 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:24.685029 14608 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:24.686425 14660 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:24.687306 12675 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:24.688126 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1641) took 1082 ms. Trace:
I20260709 12:10:24.688216 14560 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:24.688395 14608 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:24.660056 14522 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:24.688571 12675 rpcz_store.cc:276] 0709 12:10:23.605502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.686472 (+1080970us) service_pool.cc:224] Handling call
0709 12:10:24.686967 (+   495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:24.687814 (+   847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.688032 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:24.689471 14313 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:24.690173 14522 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:24.691422 12526 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:24.661162 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1639) took 1055 ms. Trace:
I20260709 12:10:24.692803 14313 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:24.692992 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:24.664753 13335 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:24.693291 14657 raft_consensus.cc:3055] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:24.665670 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:24.706240 12604 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1635) took 1703 ms (client timeout 1828 ms). Trace:
I20260709 12:10:24.670233 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:24.670958 12646 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:24.672092 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:24.679334 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1637) took 1663 ms. Trace:
I20260709 12:10:24.722893 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:24.679581 12599 rpcz_store.cc:276] 0709 12:10:22.091283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.983451 (+1892168us) service_pool.cc:224] Handling call
0709 12:10:23.983834 (+   383us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:24.053912 (+ 70078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.649526 (+595614us) spinlock_profiling.cc:243] Waited 25 ms on lock 0x7b5400232154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:24.650010 (+   484us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57515776}
W20260709 12:10:24.705034 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 82660.3us
I20260709 12:10:24.673929 12601 rpcz_store.cc:276] 0709 12:10:21.858750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.366741 (+1507991us) service_pool.cc:224] Handling call
0709 12:10:23.926708 (+559967us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:24.001194 (+ 74486us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.628491 (+627297us) spinlock_profiling.cc:243] Waited 531 ms on lock 0x7b54001171d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:24.628826 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1220477696}
I20260709 12:10:24.686470 14533 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:24.724633 14533 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:24.694052 14657 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:24.711480 14449 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:24.755267 14449 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:24.758258 14449 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:24.702335 14502 consensus_queue.cc:1243] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 5.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:10:24.722474 12527 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:24.704006 14512 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:24.687311 14624 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:24.723145 12598 rpcz_store.cc:276] 0709 12:10:23.014490 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.045018 (+1030528us) service_pool.cc:224] Handling call
0709 12:10:24.629793 (+584775us) spinlock_profiling.cc:243] Waited 3.99 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:10:24.630174 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:24.674487 (+ 44313us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.678014 (+  3527us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9175424}
I20260709 12:10:24.725440 14533 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:24.703030 14454 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:24.726933 14598 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:24.727023 12529 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 3: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:24.803709 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:24.694037 12526 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 3: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:24.805742 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:24.807049 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:24.808612 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:24.808657 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:24.809674 12526 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 3: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:24.714505 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:24.811167 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 125052us
I20260709 12:10:24.812032 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:24.813113 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:24.709542 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1642) took 1103 ms. Trace:
I20260709 12:10:24.702214 14576 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:24.684556 14371 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:24.814234 14683 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:24.815343 14371 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:24.693210 12679 rpcz_store.cc:276] 0709 12:10:23.605190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.363616 (+758426us) service_pool.cc:224] Handling call
0709 12:10:24.364020 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:24.369171 (+  5151us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:24.369210 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:24.369815 (+   605us) log.cc:844] Serialized 26 byte log entry
0709 12:10:24.583577 (+213762us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:24.583728 (+   151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.583896 (+   168us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:24.660575 (+ 76679us) raft_consensus.cc:1685] finished
0709 12:10:24.660596 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:24.660892 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1604,"raft.queue_time_us":1602,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":15879,"thread_start_us":20957,"threads_started":1,"wal-append.queue_time_us":196803}
I20260709 12:10:24.730309 13918 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:10:24.643260 12674 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:24.821755 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1640) took 1216 ms. Trace:
I20260709 12:10:24.814280 12678 rpcz_store.cc:276] 0709 12:10:23.605650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.707580 (+1101930us) service_pool.cc:224] Handling call
0709 12:10:24.708238 (+   658us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:24.708939 (+   701us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:24.708996 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:24.709073 (+    77us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:24.709133 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.709262 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:24.709455 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:24.822527 12674 rpcz_store.cc:276] 0709 12:10:23.605345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.642016 (+1036671us) service_pool.cc:224] Handling call
0709 12:10:24.642722 (+   706us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:24.821430 (+178708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.821663 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:24.824723 12678 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: [3.2-4.3]
I20260709 12:10:24.825618 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1645) took 1210 ms. Trace:
I20260709 12:10:24.669570 14412 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:24.705435 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:24.692922 14522 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:24.733076 14364 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:24.701442 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1546) took 1092 ms. Trace:
I20260709 12:10:24.702832 12671 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:24.704298 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1649) took 1087 ms. Trace:
I20260709 12:10:24.707561 14292 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:24.736368 13335 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:24.688946 14560 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:24.700052 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:24.742295 14486 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.562s	user 0.006s	sys 0.012s
W20260709 12:10:24.742858 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 56815.5us
I20260709 12:10:24.743609 14595 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:24.744009 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:24.742898 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:24.713719 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:24.745548 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:24.747081 12597 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:10:24.747023 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1633) took 1142 ms. Trace:
I20260709 12:10:24.748652 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:24.719414 14240 raft_consensus.cc:3055] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:24.749046 12425 catalog_manager.cc:5697] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 1 to 2, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 2 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:24.750810 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 113) took 1573 ms. Trace:
I20260709 12:10:24.752009 14523 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:24.754225 14505 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:10:24.722477 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 51950.5us
I20260709 12:10:24.724628 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:24.865267 12606 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:24.865309 12600 raft_consensus.cc:2376] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5. Re-sending same reply.
I20260709 12:10:24.770295 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:24.866397 12603 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:24.771633 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:24.867020 14595 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:24.771523 12427 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47524 (request call id 93) took 8855 ms. Trace:
I20260709 12:10:24.773950 12602 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:10:24.790481 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:24.870421 12528 rpcz_store.cc:276] 0709 12:10:23.177494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.928697 (+751203us) service_pool.cc:224] Handling call
0709 12:10:23.966781 (+ 38084us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:24.596913 (+630132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.750472 (+153559us) spinlock_profiling.cc:243] Waited 628 ms on lock 0x7b540027fbd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:24.750730 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1444924672}
I20260709 12:10:24.871083 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1487) took 1363 ms. Trace:
I20260709 12:10:24.871953 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:24.872483 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:24.791424 14657 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
W20260709 12:10:24.794170 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:24.731683 12604 rpcz_store.cc:269] 0709 12:10:23.002388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.205683 (+203295us) service_pool.cc:224] Handling call
0709 12:10:24.705474 (+1499791us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b54001176d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:24.705890 (+   416us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":938876800}
I20260709 12:10:24.865039 14292 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:24.857113 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:24.873116 12524 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 3: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:24.760787 12673 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
W20260709 12:10:24.869992 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:24.871709 12671 rpcz_store.cc:276] 0709 12:10:23.507270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.525612 (+ 18342us) service_pool.cc:224] Handling call
0709 12:10:24.701819 (+1176207us) spinlock_profiling.cc:243] Waited 5.33 ms on lock 0x7b58001f2c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:10:24.702221 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:24.870645 (+168424us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.870952 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12249344}
I20260709 12:10:24.825987 12678 rpcz_store.cc:276] 0709 12:10:23.615212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.823390 (+1208178us) service_pool.cc:224] Handling call
0709 12:10:24.824252 (+   862us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:24.825248 (+   996us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.825518 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":36736}
I20260709 12:10:24.867554 14595 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:24.874949 12646 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:24.873570 12680 rpcz_store.cc:276] 0709 12:10:23.604040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.831143 (+227103us) service_pool.cc:224] Handling call
0709 12:10:24.746514 (+915371us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b5400091fd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:24.746767 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":639052160}
I20260709 12:10:24.814392 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:24.873756 12677 rpcz_store.cc:276] 0709 12:10:23.616318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.756756 (+140438us) service_pool.cc:224] Handling call
0709 12:10:24.703798 (+947042us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b540008eb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:24.704064 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":521356032}
I20260709 12:10:24.872399 12531 rpcz_store.cc:276] 0709 12:10:23.608615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.984921 (+376306us) service_pool.cc:224] Handling call
0709 12:10:23.985374 (+   453us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:24.512873 (+527499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.701101 (+188228us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b540007ce54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:24.701373 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1130481920}
I20260709 12:10:24.892345 12527 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:24.867997 14412 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:24.896835 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 172393us
I20260709 12:10:24.898079 14657 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:24.898976 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:24.876673 14505 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:24.900820 14502 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:24.906155 14522 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:24.874373 14523 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:24.873356 14240 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:24.924604 14412 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:24.932116 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:24.934413 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:24.935441 12529 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 3: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:10:24.945639 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:24.868028 12427 rpcz_store.cc:276] 0709 12:10:15.990350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.050609 (+ 60259us) service_pool.cc:224] Handling call
0709 12:10:24.771171 (+8720562us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:21.734852 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:21.747451 (+ 12599us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:21.747487 (+    36us) write_op.cc:435] Acquired schema lock
0709 12:10:21.747504 (+    17us) tablet.cc:662] Decoding operations
0709 12:10:21.918708 (+171204us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:21.918771 (+    63us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:21.918798 (+    27us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:10:22.015256 (+ 96458us) tablet.cc:701] Row locks acquired
0709 12:10:22.015281 (+    25us) write_op.cc:260] PREPARE: finished
0709 12:10:22.015515 (+   234us) write_op.cc:270] Start()
0709 12:10:22.015688 (+   173us) write_op.cc:276] Timestamp: P: 1783599022015456 usec, L: 0
0709 12:10:22.015713 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:10:22.016653 (+   940us) log.cc:844] Serialized 4366 byte log entry
0709 12:10:22.958817 (+942164us) op_driver.cc:464] REPLICATION: finished
0709 12:10:23.398267 (+439450us) write_op.cc:301] APPLY: starting
0709 12:10:23.406308 (+  8041us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:23.476585 (+ 70277us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:23.486298 (+  9713us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:23.536631 (+ 50333us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:23.536922 (+   291us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0709 12:10:23.542413 (+  5491us) write_op.cc:312] APPLY: finished
0709 12:10:23.542793 (+   380us) log.cc:844] Serialized 82 byte log entry
0709 12:10:23.542985 (+   192us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:23.543140 (+   155us) write_op.cc:454] Released schema lock
0709 12:10:23.544264 (+  1124us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:23.597108 (+ 52844us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:23.613884 (+ 16776us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":18,"child_traces":[["op",{"apply.queue_time_us":409324,"apply.run_cpu_time_us":4832,"apply.run_wall_time_us":190493,"num_ops":11,"prepare.queue_time_us":115980,"prepare.run_cpu_time_us":21995,"prepare.run_wall_time_us":499051,"raft.queue_time_us":80888,"raft.run_cpu_time_us":13855,"raft.run_wall_time_us":378891,"replication_time_us":924702,"thread_start_us":627388,"threads_started":4,"wal-append.queue_time_us":241114,"wal-append.run_cpu_time_us":26121,"wal-append.run_wall_time_us":2371330}]]}
I20260709 12:10:24.951001 12671 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:10:24.952160 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1647) took 1336 ms. Trace:
I20260709 12:10:24.952271 14560 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:24.952728 12671 rpcz_store.cc:276] 0709 12:10:23.615755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.917623 (+1301868us) service_pool.cc:224] Handling call
0709 12:10:24.948736 (+ 31113us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:24.951716 (+  2980us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.952049 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:24.954687 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:24.955552 14292 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:24.957237 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:24.958689 12671 raft_consensus.cc:2393] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 5: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:24.914803 14683 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.045s
I20260709 12:10:24.959661 12528 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:24.959780 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:24.960791 12671 raft_consensus.cc:2410] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:24.960958 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:24.961651 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:24.962577 12671 raft_consensus.cc:2410] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:24.963864 12671 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:10:24.964383 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1654) took 1346 ms. Trace:
I20260709 12:10:24.964669 12671 rpcz_store.cc:276] 0709 12:10:23.618467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.963129 (+1344662us) service_pool.cc:224] Handling call
0709 12:10:24.963472 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:24.964159 (+   687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.964338 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:24.966022 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 99985.4us
I20260709 12:10:24.950831 14643 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:24.941838 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:24.965992 12671 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:24.981875 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:24.968556 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1655) took 1349 ms. Trace:
I20260709 12:10:24.941395 14364 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:24.997520 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:24.999524 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:24.986334 12671 rpcz_store.cc:276] 0709 12:10:23.618926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.965372 (+1346446us) service_pool.cc:224] Handling call
0709 12:10:24.965746 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:24.968259 (+  2513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.968506 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:25.019805 12571 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:24.933132 14502 raft_consensus.cc:697] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:24.922272 14660 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.070s
I20260709 12:10:25.161520 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:25.138919 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:25.028481 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 133750us
I20260709 12:10:25.133920 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:25.023025 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:25.165930 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 163196us
W20260709 12:10:25.181607 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:25.140756 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:25.089152 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:25.082443 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:25.084210 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 147247us
I20260709 12:10:25.194962 12528 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 3: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:25.226802 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:25.202549 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 165581us
I20260709 12:10:24.985915 14576 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:25.246531 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1644) took 1640 ms. Trace:
I20260709 12:10:25.284766 13330 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 2. Current term is 3. Ops: []
W20260709 12:10:25.246299 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:25.256621 14523 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:25.052155 14420 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:25.260499 14665 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:25.285300 14486 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:25.127461 14683 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:10:25.242654 12673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 136) took 1693 ms (client timeout 1950 ms). Trace:
I20260709 12:10:25.295125 14637 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:25.027735 14412 consensus_queue.cc:1059] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.211s
I20260709 12:10:25.266510 14613 consensus_queue.cc:1059] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:25.297926 12678 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:25.285492 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:25.286656 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:25.266937 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1646) took 1651 ms. Trace:
I20260709 12:10:25.255371 14454 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:25.053035 14533 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:25.237241 14505 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:25.286494 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:25.043862 14643 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:25.256158 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1638) took 1650 ms. Trace:
W20260709 12:10:25.323838 12675 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.009s	sys 0.053s
I20260709 12:10:25.049911 14595 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:25.127993 14619 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:25.420452 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:25.383100 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:25.162621 13335 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 373) took 1562 ms (client timeout 1719 ms). Trace:
I20260709 12:10:25.372447 12527 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:25.348938 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:25.325548 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:25.448112 12675 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.673s	user 0.015s	sys 0.058s
W20260709 12:10:25.437057 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 46399.2us
I20260709 12:10:25.437332 12526 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:25.411957 12523 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 2. (term mismatch)
I20260709 12:10:25.447115 12528 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:25.448753 12674 rpcz_store.cc:276] 0709 12:10:23.615488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.823421 (+1207933us) service_pool.cc:224] Handling call
0709 12:10:24.824231 (+   810us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:24.824937 (+   706us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:24.824967 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:24.825683 (+   716us) log.cc:844] Serialized 26 byte log entry
0709 12:10:25.004497 (+178814us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:25.004538 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:25.004626 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:25.266675 (+262049us) raft_consensus.cc:1685] finished
0709 12:10:25.266696 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:25.266872 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":728,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":103,"spinlock_wait_cycles":29568,"thread_start_us":178326,"threads_started":1,"wal-append.queue_time_us":422945}
W20260709 12:10:25.448779 12673 rpcz_store.cc:269] 0709 12:10:23.549106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.581796 (+ 32690us) service_pool.cc:224] Handling call
0709 12:10:25.223404 (+1641608us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54003d4fd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:25.242518 (+ 19114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2542903680}
I20260709 12:10:25.324522 12679 rpcz_store.cc:276] 0709 12:10:23.605977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.818612 (+1212635us) service_pool.cc:224] Handling call
0709 12:10:24.819124 (+   512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:24.819969 (+   845us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:24.820011 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:24.820987 (+   976us) log.cc:844] Serialized 26 byte log entry
0709 12:10:25.068060 (+247073us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:25.068127 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:25.068233 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:25.246253 (+178020us) raft_consensus.cc:1685] finished
0709 12:10:25.246281 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:25.246449 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":877,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":154,"thread_start_us":141658,"threads_started":1,"wal-append.queue_time_us":335210}
I20260709 12:10:25.128432 14686 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:25.464046 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1491) took 1651 ms. Trace:
I20260709 12:10:25.220336 14560 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:25.132809 13918 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:25.163151 14460 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:25.454892 12523 pending_rounds.cc:85] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 1
I20260709 12:10:25.448537 12530 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:25.449306 12529 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:10:25.048839 14512 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:10:25.472491 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 152209us
I20260709 12:10:25.482074 12572 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:25.483471 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:25.366439 12571 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:25.471843 12678 rpcz_store.cc:276] 0709 12:10:23.812475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:25.091551 (+1279076us) service_pool.cc:224] Handling call
0709 12:10:25.215872 (+124321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:25.463648 (+247776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:25.463956 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:25.449991 12672 rpcz_store.cc:276] 0709 12:10:23.605041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.361919 (+756878us) service_pool.cc:224] Handling call
0709 12:10:24.364327 (+  2408us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:24.370082 (+  5755us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:24.631523 (+261441us) log.cc:844] Serialized 14 byte log entry
0709 12:10:24.789239 (+157716us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:24.790138 (+   899us) log.cc:844] Serialized 26 byte log entry
0709 12:10:24.790344 (+   206us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:24.790398 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:24.790668 (+   270us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:25.255607 (+464939us) raft_consensus.cc:1685] finished
0709 12:10:25.255641 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:25.255909 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11557,"raft.queue_time_us":8400,"raft.run_cpu_time_us":391,"raft.run_wall_time_us":15851,"thread_start_us":143608,"threads_started":1,"wal-append.queue_time_us":608076}
I20260709 12:10:25.323524 14576 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:25.479343 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:25.023342 14364 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:25.325857 14371 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:25.488001 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:25.493110 12531 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:25.480249 12523 pending_rounds.cc:107] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 2.2
I20260709 12:10:25.496958 12523 raft_consensus.cc:2889] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:25.491963 14454 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:25.501147 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 306793us
I20260709 12:10:25.501987 12674 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 3: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:25.503532 12597 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:10:25.504447 12528 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:10:25.508240 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 159583us
I20260709 12:10:25.509675 12529 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:10:25.479877 13918 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:25.503938 12597 pending_rounds.cc:85] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 3
I20260709 12:10:25.515867 12525 raft_consensus.cc:606] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:25.516417 12530 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:25.516743 14454 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:25.517889 12523 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:25.528398 12525 raft_consensus.cc:614] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:25.518908 12604 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3. Ops: [2.3-2.3]
I20260709 12:10:25.522320 14412 consensus_queue.cc:1059] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:25.523135 13332 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 1. Current term is 2. Ops: []
I20260709 12:10:25.524266 13330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 172) took 1664 ms. Trace:
I20260709 12:10:25.528024 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:25.530500 14691 consensus_queue.cc:1059] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.008s
I20260709 12:10:25.522307 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:25.531030 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 49101.8us
I20260709 12:10:25.532915 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:25.483590 14449 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:25.493743 14576 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:25.533597 14683 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:25.530361 12524 raft_consensus.cc:2410] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:25.539101 12522 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 2.
I20260709 12:10:25.509359 14523 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:25.548370 12529 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:10:25.530638 13918 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:25.527748 12597 pending_rounds.cc:107] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 4.4
I20260709 12:10:25.531032 14505 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:25.555186 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:25.466674 13335 rpcz_store.cc:269] 0709 12:10:23.600526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.611846 (+ 11320us) service_pool.cc:224] Handling call
0709 12:10:25.162291 (+1550445us) spinlock_profiling.cc:243] Waited 916 ms on lock 0x7b54006419d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:25.162491 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2106667264}
I20260709 12:10:25.479444 14637 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:25.561080 12597 raft_consensus.cc:2917] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Aborting config change with OpId 4.4: Aborted: Op aborted by new leader
I20260709 12:10:25.483455 14619 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:25.536129 14454 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:25.498863 14486 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:25.561592 12597 raft_consensus.cc:2889] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:25.565541 12679 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:25.560544 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:25.566694 12602 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:25.550493 14624 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:25.546177 14613 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:25.557991 14595 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:25.500792 14502 consensus_queue.cc:237] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:25.573701 12676 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:25.501462 14665 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:25.577394 12672 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 3: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:25.568312 12678 raft_consensus.cc:2376] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4. Re-sending same reply.
I20260709 12:10:25.577574 12646 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:25.536935 14420 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:25.579983 12604 raft_consensus.cc:2410] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:25.582226 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:25.584950 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:25.526443 14686 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:25.631168 12605 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:25.588361 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:25.625130 12602 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
W20260709 12:10:25.627729 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 76882.9us
I20260709 12:10:25.550700 14449 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:25.621644 12571 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:25.566491 14660 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:10:25.584812 14309 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.191s	user 0.001s	sys 0.003s
I20260709 12:10:25.531730 14460 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:25.631705 12600 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:25.631567 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:25.633351 12674 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:25.569061 14512 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:25.537463 13330 rpcz_store.cc:276] 0709 12:10:23.859678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.860129 (+   451us) service_pool.cc:224] Handling call
0709 12:10:25.284312 (+1424183us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:25.482678 (+198366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:25.523901 (+ 41223us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b54006419d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:25.524156 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2995720960}
I20260709 12:10:25.638931 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:25.644110 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 68639.6us
I20260709 12:10:25.641885 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.BulkChangeConfig from 127.0.0.1:48482 (request call id 5) took 8777 ms. Trace:
I20260709 12:10:25.635190 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:25.650516 12671 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:25.650776 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:25.649778 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:25.586592 14613 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:25.655958 12606 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:25.661715 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:25.646366 14292 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.012s
I20260709 12:10:25.641572 14412 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:25.666281 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48476 (request call id 8) took 2615 ms. Trace:
I20260709 12:10:25.666658 12604 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:10:25.667636 14619 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:25.671729 12525 rpcz_store.cc:276] 0709 12:10:23.051247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.072149 (+ 20902us) service_pool.cc:224] Handling call
0709 12:10:25.662361 (+2590212us) spinlock_profiling.cc:243] Waited 2.38 s on lock 0x7b5400106dd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1a853 00007f148d9e860a 00007f1486f6e7d4 00007f1486f6e762 00007f1486f6e691 00007f1486f6e60c 00007f1486f6d8c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:25.666148 (+  3787us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5463795968}
W20260709 12:10:25.681334 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 59765.7us
I20260709 12:10:25.673197 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:25.645188 12597 rpcz_store.cc:276] 0709 12:10:16.859993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:16.860162 (+   169us) service_pool.cc:224] Handling call
0709 12:10:16.937354 (+ 77192us) log.cc:844] Serialized 506 byte log entry
0709 12:10:17.612503 (+675149us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b54005102a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488999142 00007f1488979eaa 00007f1488980d53 00007f1488980cd9 00007f1488980c69 00007f1488980c31 00007f148897fdfd 0000000000388634 0000000000388479 00007f1485933475
0709 12:10:17.851590 (+239087us) spinlock_profiling.cc:243] Waited 19.6 ms on lock 0x7b54005102a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488999142 00007f1488979eaa 00007f1488980d53 00007f1488980cd9 00007f1488980c69 00007f1488980c31 00007f148897fdfd 0000000000388634 0000000000388479 00007f1485933475
0709 12:10:25.633103 (+7781513us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36533,"raft.queue_time_us":23977,"raft.run_cpu_time_us":10271,"raft.run_wall_time_us":1686581,"spinlock_wait_cycles":107418368}
I20260709 12:10:25.672693 14292 consensus_queue.cc:1243] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:10:25.649336 14657 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:25.686586 12527 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:25.697311 12672 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:25.675091 14660 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:25.677063 14533 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:25.692493 14726 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.244s	user 0.000s	sys 0.002s
I20260709 12:10:25.723210 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:25.689327 14657 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:25.733153 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:25.738777 14559 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:10:25.739393 14731 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:10:25.739645 14559 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:25.741190 14683 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:25.742543 14683 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:25.743176 14683 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:25.743701 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:25.745519 14683 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:25.693583 14560 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260709 12:10:25.753574 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1643) took 2147 ms. Trace:
I20260709 12:10:25.776976 12571 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:25.753217 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:10:25.783226 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:25.786437 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:25.760495 12675 rpcz_store.cc:276] 0709 12:10:23.605803 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.766350 (+1160547us) service_pool.cc:224] Handling call
0709 12:10:24.766904 (+   554us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:24.773016 (+  6112us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:24.773068 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:24.774222 (+  1154us) log.cc:844] Serialized 52 byte log entry
0709 12:10:25.453142 (+678920us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:25.453174 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:25.453228 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:25.753303 (+300075us) raft_consensus.cc:1685] finished
0709 12:10:25.753327 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:25.753501 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5101,"raft.run_cpu_time_us":199,"raft.run_wall_time_us":202,"thread_start_us":678419,"threads_started":1,"wal-append.queue_time_us":769849}
I20260709 12:10:25.688288 14454 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:25.749892 14686 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:25.751920 14449 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:25.797821 14734 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:25.798382 14733 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:10:25.799561 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 111526us
I20260709 12:10:25.805876 14624 raft_consensus.cc:697] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:25.758536 14598 raft_consensus.cc:3055] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:25.763375 14559 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:25.843829 12678 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:25.779062 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:25.761073 14657 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:25.811813 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 118) took 1012 ms. Trace:
I20260709 12:10:25.812258 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1490) took 2128 ms. Trace:
I20260709 12:10:25.818895 14486 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:25.816244 13918 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:25.802294 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 159472us
I20260709 12:10:25.850287 12599 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3. Ops: [2.2-2.2]
I20260709 12:10:25.847262 12522 rpcz_store.cc:276] 0709 12:10:24.799027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:25.243665 (+444638us) service_pool.cc:224] Handling call
0709 12:10:25.286276 (+ 42611us) spinlock_profiling.cc:243] Waited 41.4 ms on lock 0x7b5800002a98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:10:25.811595 (+525319us) spinlock_profiling.cc:243] Waited 90.8 ms on lock 0x7b5400106dd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:25.811728 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":304092544}
W20260709 12:10:25.829665 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 292311us
I20260709 12:10:25.827705 14449 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:10:25.835461 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 272328us
I20260709 12:10:25.835299 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:25.840337 12672 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 5. Ops: []
I20260709 12:10:25.836004 14512 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:25.864607 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:25.865653 12675 raft_consensus.cc:2410] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:10:25.865751 14732 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:10:25.848151 12677 rpcz_store.cc:276] 0709 12:10:23.683769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:25.056103 (+1372334us) service_pool.cc:224] Handling call
0709 12:10:25.261596 (+205493us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:25.262035 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:25.576407 (+314372us) log.cc:844] Serialized 14 byte log entry
0709 12:10:25.811869 (+235462us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:25.811939 (+    70us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:25.811993 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:25.812083 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:25.812198 (+   115us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":476260,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":8997,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":10193,"thread_start_us":484264,"threads_started":2}
I20260709 12:10:25.870596 14613 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:25.871172 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:10:25.872136 14660 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:25.872761 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:25.874672 14745 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:25.875568 14745 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:25.876948 14643 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:25.846333 14420 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:25.859843 14637 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:25.878508 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1652) took 2261 ms. Trace:
I20260709 12:10:25.879005 12680 rpcz_store.cc:276] 0709 12:10:23.617601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:24.902883 (+1285282us) service_pool.cc:224] Handling call
0709 12:10:24.903434 (+   551us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:24.966043 (+ 62609us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:24.966074 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:25.085027 (+118953us) log.cc:844] Serialized 52 byte log entry
0709 12:10:25.544318 (+459291us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:25.544357 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:25.544521 (+   164us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:25.878206 (+333685us) raft_consensus.cc:1685] finished
0709 12:10:25.878237 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:25.878426 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21563,"raft.queue_time_us":29625,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":75210,"thread_start_us":381087,"threads_started":1,"wal-append.queue_time_us":556855}
I20260709 12:10:25.882261 14741 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:25.883198 13918 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:25.870428 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:25.864835 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:25.884951 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:25.881757 12680 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:25.894359 12671 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:25.841306 14598 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:25.896538 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 174516us
I20260709 12:10:25.899868 12525 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:25.903379 12672 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:25.904302 12522 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260709 12:10:25.903857 14240 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:25.904969 12672 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:25.905448 14240 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:25.906038 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:25.864850 14624 consensus_queue.cc:237] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:25.906502 14738 consensus_queue.cc:1059] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:25.907137 12526 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 5: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:25.907990 14738 raft_consensus.cc:3055] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Stepping down as leader of term 3
W20260709 12:10:25.908042 12496 leader_election.cc:336] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.014s (SENT)
I20260709 12:10:25.908548 14738 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:25.908707 12496 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:25.909683 14613 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260709 12:10:25.910722 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:25.915174 12496 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:25.956082 12674 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:25.984496 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:25.862174 14486 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:26.006255 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:25.854768 14449 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:25.916088 12572 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:26.007022 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:25.914410 14643 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.099s
I20260709 12:10:25.884284 14741 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.137s
I20260709 12:10:26.017298 14533 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:26.009616 14486 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:26.018648 14412 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:10:26.044453 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 177619us
I20260709 12:10:25.848600 14734 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:26.053685 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 156604us
I20260709 12:10:26.055666 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:25.851485 14733 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:26.037797 14460 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:26.040897 14745 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:26.061005 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:26.068677 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 154782us
I20260709 12:10:26.070493 14748 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:26.093660 14449 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.094028 14748 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.094995 14449 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:26.095660 14734 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:26.097242 14748 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:26.070214 14743 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.017s
I20260709 12:10:26.097654 14739 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:26.072495 14624 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:26.073459 14741 raft_consensus.cc:3055] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:10:26.098726 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 77601.9us
I20260709 12:10:26.099092 14741 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:26.100294 14741 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.101773 14741 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:26.039333 14637 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:26.102900 14733 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:26.030572 14738 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.103287 14637 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:26.104241 14738 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:26.047638 14691 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:26.105063 14691 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.051937 14486 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:26.107580 14691 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:26.108790 14665 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:26.109018 12680 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:26.054525 14560 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:26.085940 13918 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:26.078480 14523 consensus_meta.cc:287] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.522s	user 0.012s	sys 0.001s
I20260709 12:10:26.058558 14364 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:26.098268 14739 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.110559 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:26.101827 12572 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:26.084316 14683 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:26.111898 12496 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:26.118609 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:26.119067 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1664) took 1029 ms. Trace:
W20260709 12:10:26.119772 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 62177.7us
I20260709 12:10:26.119495 12679 rpcz_store.cc:276] 0709 12:10:25.135351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:25.472327 (+336976us) service_pool.cc:224] Handling call
0709 12:10:26.118892 (+646565us) spinlock_profiling.cc:243] Waited 71.9 ms on lock 0x7b54001e2154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:26.118997 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165401344}
I20260709 12:10:26.119330 12529 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:26.116225 14740 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:26.125728 13918 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:26.125584 14740 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:26.125492 14733 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:10:26.127831 12477 auto_rebalancer.cc:629] Failed to schedule move for tablet 3c9b5315396948b2af3fa5480c9bee87: Aborted: Op aborted by new leader
I20260709 12:10:26.128064 14740 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:26.112968 14598 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:26.130098 14598 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:26.134446 12673 raft_consensus.cc:2410] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:10:26.135756 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:26.113826 14691 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:26.090605 14460 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:10:26.119266 14748 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.047s
I20260709 12:10:26.171664 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:26.123503 14309 tablet_bootstrap.cc:492] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:10:26.175639 14309 tablet_bootstrap.cc:492] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
I20260709 12:10:26.177315 14309 ts_tablet_manager.cc:1403] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 2.125s	user 0.027s	sys 0.021s
I20260709 12:10:26.171259 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:26.181674 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:26.188689 12674 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:26.188380 13305 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:10:26.199734 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 88139.5us
I20260709 12:10:26.191013 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:26.200685 12496 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:26.121129 14683 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.207209 12678 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:26.150105 14743 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:26.197899 14512 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:26.205721 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:10:26.206547 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 58768.4us
I20260709 12:10:26.207896 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:26.172974 14523 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 12:10:26.211762 12674 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:26.215422 14657 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.215874 14560 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:26.170277 14733 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:10:26.216418 14560 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:26.217376 14560 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:26.215792 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 92427.2us
I20260709 12:10:26.210474 14512 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:26.212414 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 110688us
I20260709 12:10:26.201354 14595 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:26.217813 14657 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:26.152155 14637 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:26.157455 14691 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.232725 12494 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:26.140110 14364 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.220890 14740 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:26.226015 14748 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:26.244812 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:26.246770 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:26.248631 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48018.4us
I20260709 12:10:26.250336 12496 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:26.229837 14595 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:26.200465 14412 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.254841 12679 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:26.248792 14748 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:26.254914 14757 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:10:26.263059 14560 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:26.263679 14364 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:26.266851 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 45389us
I20260709 12:10:26.241040 14740 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:26.303956 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58616.8us
W20260709 12:10:26.305052 12676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1636) took 2689 ms (client timeout 1865 ms). Trace:
I20260709 12:10:26.271874 14309 raft_consensus.cc:359] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:10:26.273988 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:26.273749 14613 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:26.243086 14643 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:26.283105 14683 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:26.284646 14739 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:26.286306 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:26.305640 12676 rpcz_store.cc:269] 0709 12:10:23.604709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.727680 (+122971us) service_pool.cc:224] Handling call
0709 12:10:24.705039 (+977359us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b58001f2c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:10:26.293908 (+1588869us) spinlock_profiling.cc:243] Waited 642 ms on lock 0x7b54000ac8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:26.294291 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1481954304}
I20260709 12:10:26.311318 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:26.293062 14371 raft_consensus.cc:3055] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:26.323356 14371 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:26.324417 14371 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.324882 14757 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:26.307644 14740 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:26.326303 14364 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:26.326874 14364 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.329133 14364 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:26.330758 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1666) took 1086 ms. Trace:
I20260709 12:10:26.331224 14691 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:26.311349 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:26.325691 14757 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:26.335321 14512 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:26.252046 14502 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.070s
I20260709 12:10:26.338703 14657 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:26.339202 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1663) took 1340 ms. Trace:
I20260709 12:10:26.339668 14657 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:26.340737 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:26.342618 14657 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:26.344388 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:26.320760 14738 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.032s
I20260709 12:10:26.350374 12494 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:26.350769 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:26.351871 13332 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:26.331171 12673 rpcz_store.cc:276] 0709 12:10:25.244173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:25.478956 (+234783us) service_pool.cc:224] Handling call
0709 12:10:26.330553 (+851597us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b5400013bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:26.330689 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1486749952}
W20260709 12:10:26.312513 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 62471.6us
I20260709 12:10:26.286504 14595 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:26.315088 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 72262.5us
I20260709 12:10:26.316810 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:26.318004 13334 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:26.312696 14613 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:26.319540 14743 raft_consensus.cc:3055] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:26.293885 14748 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:26.291524 14523 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:26.312553 14533 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:26.339715 12678 rpcz_store.cc:276] 0709 12:10:24.998580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:25.851848 (+853268us) service_pool.cc:224] Handling call
0709 12:10:26.201495 (+349647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:26.209311 (+  7816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:26.338948 (+129637us) spinlock_profiling.cc:243] Waited 349 ms on lock 0x7b54000c26d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:26.339118 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":802797952}
W20260709 12:10:26.370880 14761 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:10:26.406790 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:26.363386 12569 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:26.408567 14576 consensus_meta.cc:287] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.844s	user 0.014s	sys 0.000s
W20260709 12:10:26.408700 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 53256.8us
I20260709 12:10:26.408849 13918 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:26.325681 14371 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:26.310223 14309 raft_consensus.cc:740] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
I20260709 12:10:26.373131 14757 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:26.383561 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 132266us
I20260709 12:10:26.372560 12675 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260709 12:10:26.401958 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:26.402962 12676 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:10:26.352276 14760 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.065s	user 0.002s	sys 0.000s
W20260709 12:10:26.404493 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:26.405254 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 181524us
I20260709 12:10:26.409858 13918 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:26.416901 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 53685.6us
I20260709 12:10:26.417302 13918 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:26.409498 12678 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:26.373764 14743 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:26.413555 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 59208.6us
I20260709 12:10:26.427208 14757 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:26.430932 14743 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.426857 14533 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260709 12:10:26.425531 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:26.406409 14502 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:26.415642 12675 pending_rounds.cc:85] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 5
W20260709 12:10:26.433890 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:26.438273 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:26.438301 12675 pending_rounds.cc:107] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260709 12:10:26.438928 12643 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:26.439191 12675 raft_consensus.cc:2889] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:26.443183 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:26.443933 12676 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:26.418723 14767 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:26.450789 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:26.451655 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:26.406401 14486 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.415251 14523 raft_consensus.cc:697] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:26.409085 14576 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.451309 14454 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:26.463760 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:26.442749 14412 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:26.483498 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:26.481805 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:26.482169 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:26.484157 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:26.454484 14683 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:10:26.486088 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 171533us
I20260709 12:10:26.455560 14767 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:26.491622 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:26.492393 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:26.493059 12572 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:26.498450 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:26.500283 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:26.501549 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:26.445182 14613 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.503789 14613 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:26.504783 14740 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:26.506349 14454 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:26.490255 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:26.497186 12477 auto_rebalancer.cc:629] Failed to schedule move for tablet 317be0a176ee473b9b3631bdc1030bcc: Illegal state: Replica 193a40da6e364ead8ab561e14c3cc8d1 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:10:26.510017 13918 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:26.510003 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:26.511813 12673 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:26.518472 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:26.518385 12676 raft_consensus.cc:2410] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 3 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
I20260709 12:10:26.520239 13918 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:26.483520 14691 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:26.526404 12601 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:26.527423 13918 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:10:26.528743 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 65878.9us
I20260709 12:10:26.529878 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:26.460083 14749 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.068s
W20260709 12:10:26.532974 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 44106.8us
I20260709 12:10:26.507617 14739 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:26.532915 13918 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.458725 14502 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.534924 12496 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:26.535951 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:26.542905 12673 raft_consensus.cc:2410] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:26.546456 14454 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:26.546854 14765 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.086s	user 0.001s	sys 0.000s
I20260709 12:10:26.547327 14746 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:10:26.454107 14309 consensus_queue.cc:260] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:10:26.559255 13338 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:26.487112 14412 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.566988 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:26.567601 13918 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:26.567889 12674 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:26.568533 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 76827.1us
W20260709 12:10:26.572620 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 92164.6us
I20260709 12:10:26.573782 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:26.574496 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:26.581996 12679 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:26.557641 14454 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:26.584245 12676 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: [3.2-5.3]
W20260709 12:10:26.586529 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:26.587563 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:26.559923 14240 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:26.590755 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:26.569963 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:26.594872 13334 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:26.595047 12600 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:10:26.597385 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:26.598163 13332 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:26.598654 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:26.599359 14770 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:10:26.600699 12524 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:26.599601 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:26.613149 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 78308.7us
I20260709 12:10:26.601778 12674 raft_consensus.cc:2410] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:26.616163 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:26.551357 14691 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.571712 14643 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:26.619697 12526 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:26.621953 12604 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:26.570652 14502 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:26.582895 14739 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:26.602624 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:26.623980 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:26.624400 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 51964us
I20260709 12:10:26.624938 14454 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:26.625595 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:26.585335 14598 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:26.614063 12496 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:26.602811 14449 consensus_meta.cc:287] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.507s	user 0.014s	sys 0.000s
I20260709 12:10:26.638432 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:26.644407 12679 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:26.638976 12569 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:26.637720 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:26.639467 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:26.646277 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:26.646658 12522 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:26.640669 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:26.647408 12674 raft_consensus.cc:2376] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4. Re-sending same reply.
I20260709 12:10:26.650179 12678 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:26.654911 12675 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:26.602598 14240 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:26.658564 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:26.660073 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46094.1us
W20260709 12:10:26.663049 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 110036us
I20260709 12:10:26.656831 12523 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
W20260709 12:10:26.665475 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 74851.8us
I20260709 12:10:26.668193 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:26.661147 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:26.620951 14746 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:26.673712 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:26.675668 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:26.670513 12524 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:26.653678 14512 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:26.678140 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:26.578742 14523 consensus_queue.cc:237] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.679217 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:26.561157 14767 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.601583 14769 consensus_queue.cc:1059] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.080s
I20260709 12:10:26.683073 12526 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:26.680625 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:26.656047 14502 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:26.667033 14749 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:26.708793 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:26.727298 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:26.728718 12673 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: [4.3-4.4]
I20260709 12:10:26.670506 14643 raft_consensus.cc:3055] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Stepping down as leader of term 2
W20260709 12:10:26.745812 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 66696.2us
I20260709 12:10:26.676220 14449 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:26.673199 14740 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:26.735260 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:26.658993 14739 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.685352 14512 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:26.678660 14683 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:26.753588 12604 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:26.682644 14486 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:26.685470 14598 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:10:26.668041 14747 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.101s
I20260709 12:10:26.732472 14502 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:26.789314 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 143181us
I20260709 12:10:26.746466 14767 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:26.738687 14576 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:26.798426 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:26.807693 12646 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:26.753096 14643 raft_consensus.cc:740] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:26.809628 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:26.810591 12604 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 6: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:10:26.801668 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54202.6us
W20260709 12:10:26.769611 14771 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.182s	user 0.000s	sys 0.001s
I20260709 12:10:26.727085 14746 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.774135 14512 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:26.805760 12673 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 2. Current term is 3. Ops: []
W20260709 12:10:26.762912 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 79182.6us
I20260709 12:10:26.811151 12646 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:26.817718 12600 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:26.793630 14598 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:26.812227 14502 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:26.823342 14781 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:26.803462 14683 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:26.835582 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 195457us
W20260709 12:10:26.831430 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 150976us
I20260709 12:10:26.806195 14449 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:26.850744 13335 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:26.851302 14412 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:26.851972 14683 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.853363 14777 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:26.803273 14364 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:10:26.854624 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:26.857237 14746 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:26.754235 14749 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:26.847361 12646 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:26.878290 14309 ts_tablet_manager.cc:1434] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.700s	user 0.018s	sys 0.000s
I20260709 12:10:26.796382 14678 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:10:26.828908 14598 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:26.912798 12601 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:10:26.816354 14619 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:26.860719 14449 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:27.062729 13331 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: [5.8-5.8]
W20260709 12:10:27.062965 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 215684us
I20260709 12:10:26.852437 13334 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
W20260709 12:10:27.011647 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 180017us
W20260709 12:10:26.899230 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50250.7us
I20260709 12:10:27.068423 12494 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:10:26.867450 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 100182us
I20260709 12:10:27.036433 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:27.039553 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1653) took 1568 ms. Trace:
I20260709 12:10:27.008121 14691 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:27.006731 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 148658us
W20260709 12:10:27.098625 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:27.007877 14739 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:27.061092 14686 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:26.921527 14783 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:26.852732 14733 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.194s
I20260709 12:10:27.071496 12599 rpcz_store.cc:276] 0709 12:10:25.470918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:25.471088 (+   170us) service_pool.cc:224] Handling call
0709 12:10:25.483023 (+ 11935us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:25.850684 (+367661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:27.039154 (+1188470us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b540023fd54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:27.039462 (+   308us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":843972480}
W20260709 12:10:27.072690 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:27.104151 12600 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:10:27.057621 14595 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.649s	user 0.012s	sys 0.000s
W20260709 12:10:27.044548 14309 ts_tablet_manager.cc:1467] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Tablet startup took 866ms
I20260709 12:10:27.037036 14460 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.069s
I20260709 12:10:27.107728 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:27.057897 14454 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.062794 14449 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:27.120576 14691 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.107494 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:27.215859 12601 raft_consensus.cc:2376] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6. Re-sending same reply.
W20260709 12:10:27.126374 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:27.216413 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 91791.3us
I20260709 12:10:27.106895 14746 raft_consensus.cc:3055] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:27.110610 14412 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:27.221521 12601 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:27.184304 12602 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3. Ops: []
W20260709 12:10:27.112792 14560 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.805s	user 0.011s	sys 0.002s
I20260709 12:10:27.224210 14788 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.180s
I20260709 12:10:26.878736 14240 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:27.225601 14371 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.813s	user 0.013s	sys 0.005s
I20260709 12:10:27.226262 14790 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.101s
I20260709 12:10:27.226657 14798 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:27.230337 14683 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:27.188773 12606 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
W20260709 12:10:27.192749 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 73641.3us
W20260709 12:10:27.194053 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 86407.7us
W20260709 12:10:27.204530 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:27.209558 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 83515.9us
I20260709 12:10:27.185153 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:27.111028 14783 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:27.130241 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:27.122284 14738 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:10:27.237697 12675 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.573s	user 0.005s	sys 0.005s
W20260709 12:10:27.239481 12671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1662) took 2296 ms (client timeout 2171 ms). Trace:
I20260709 12:10:27.239741 12675 raft_consensus.cc:2410] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 12:10:27.222457 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:27.240566 12673 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: []
W20260709 12:10:27.239946 12671 rpcz_store.cc:269] 0709 12:10:24.973412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:25.022665 (+ 49253us) service_pool.cc:224] Handling call
0709 12:10:27.239255 (+2216590us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b54002c4a54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:27.239404 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1031607680}
I20260709 12:10:27.221580 14412 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:27.235282 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:27.214993 14678 raft_consensus.cc:3055] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:27.249066 14678 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:27.250411 14678 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:27.226302 14619 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:10:27.244141 14559 raft_consensus.cc:3055] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:27.248214 14412 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:27.242417 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:27.220566 14746 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:10:27.256759 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 192798us
I20260709 12:10:27.074096 14643 consensus_queue.cc:260] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:27.122066 14576 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.108s
I20260709 12:10:27.122792 14595 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:27.265007 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:27.106832 14781 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.131s
I20260709 12:10:27.132333 14686 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:27.221426 14454 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:27.297478 12671 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:27.245564 14738 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:27.270787 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:27.244274 14637 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.096s
I20260709 12:10:27.308923 12646 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:27.230975 14789 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.161s
I20260709 12:10:27.270015 12496 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:27.254851 12526 consensus_meta.cc:287] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.521s	user 0.011s	sys 0.000s
I20260709 12:10:27.220407 14691 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:27.360837 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:27.379036 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 193822us
W20260709 12:10:27.350504 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:27.458650 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 149939us
I20260709 12:10:27.456542 12679 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:27.460273 12646 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:27.374296 12680 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:27.457379 12672 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:27.457989 12526 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
W20260709 12:10:27.464820 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 196413us
I20260709 12:10:27.432479 12571 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:27.462167 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1514) took 1035 ms. Trace:
I20260709 12:10:27.268922 14559 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:27.468612 13338 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: []
W20260709 12:10:27.344192 14785 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
W20260709 12:10:27.398262 14778 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.335s	user 0.000s	sys 0.002s
W20260709 12:10:27.389808 14794 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:10:27.506978 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:27.507920 12674 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:27.311822 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 210401us
I20260709 12:10:27.511590 12496 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:27.469867 12680 rpcz_store.cc:276] 0709 12:10:26.428974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:27.358060 (+929086us) service_pool.cc:224] Handling call
0709 12:10:27.366866 (+  8806us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:27.461848 (+ 94982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:27.462066 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:27.477468 12646 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:27.246589 14783 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:27.509797 12529 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:27.297880 14678 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:27.343937 14797 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.390550 14749 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:27.310376 14805 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:27.363931 14643 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:27.468029 14364 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:27.464151 14738 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:27.520231 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 87869.9us
I20260709 12:10:27.344270 14798 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:10:27.535207 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 67814.6us
I20260709 12:10:27.535786 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:27.527973 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 50618.1us
I20260709 12:10:27.519093 14796 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:27.539819 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:27.543162 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:27.504493 14741 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:27.545636 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:27.530009 14364 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:27.562024 13333 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 3: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:27.505452 14804 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.062s
I20260709 12:10:27.562582 13338 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 1 LEARNER]: Advancing to term 2
I20260709 12:10:27.563603 12680 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:10:27.564412 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:27.564666 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:27.568372 12526 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:27.563378 12530 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:27.554909 14371 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:27.369653 14746 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:27.566656 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:27.567731 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 332514us
I20260709 12:10:27.570660 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:27.571604 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:27.571519 12674 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:27.520905 14745 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:10:27.566861 12672 raft_consensus.cc:2410] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:27.599311 12672 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:27.582978 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:27.576942 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:27.565886 14364 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:27.565923 14749 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:27.603508 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:27.555236 14660 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:27.587355 12524 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:27.605206 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:27.602840 12522 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:27.564216 14683 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:27.609298 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:27.610970 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 56755.1us
I20260709 12:10:27.577943 14371 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:27.612536 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 129) took 1123 ms. Trace:
I20260709 12:10:27.597057 14624 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:27.578264 14805 raft_consensus.cc:697] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:10:27.616750 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50240.9us
I20260709 12:10:27.629367 12571 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:27.626869 14814 consensus_queue.cc:1059] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:27.630069 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:10:27.609580 14598 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.547s	user 0.013s	sys 0.001s
I20260709 12:10:27.631250 13302 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:27.619966 12527 rpcz_store.cc:276] 0709 12:10:26.486529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:26.487454 (+   925us) service_pool.cc:224] Handling call
0709 12:10:26.497390 (+  9936us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:26.526908 (+ 29518us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:26.526955 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:26.532049 (+  5094us) log.cc:844] Serialized 26 byte log entry
0709 12:10:26.725876 (+193827us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:26.725929 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:26.726011 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:27.609481 (+883470us) raft_consensus.cc:1685] finished
0709 12:10:27.609502 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:27.609709 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38985,"raft.queue_time_us":33635,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":21324,"thread_start_us":158549,"threads_started":1,"wal-append.queue_time_us":264016}
W20260709 12:10:27.634104 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 110115us
I20260709 12:10:27.635416 13302 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:27.638038 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:27.607263 14576 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:27.537463 14797 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:27.638867 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:27.646166 12496 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:27.620761 14613 consensus_meta.cc:287] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.116s	user 0.015s	sys 0.000s
I20260709 12:10:27.627951 14371 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:27.564313 14796 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:27.652312 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:27.628827 14624 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:27.648217 13302 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:27.611708 14766 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:10:27.654070 12525 raft_consensus.cc:2410] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:27.642907 14798 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:10:27.663524 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49773.6us
I20260709 12:10:27.638331 14812 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:27.620589 14660 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:27.644557 14733 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:27.674306 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:27.631189 14683 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:27.681833 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52563.8us
I20260709 12:10:27.682626 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:27.683663 12571 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:27.647595 14801 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:27.662168 14624 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:27.639015 14598 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:27.687723 12599 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:10:27.652137 14806 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:27.659845 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:27.709401 12522 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: [2.5-2.5]
I20260709 12:10:27.710870 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1503) took 1363 ms. Trace:
I20260709 12:10:27.719213 14769 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:10:27.681589 14793 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.175s	user 0.001s	sys 0.001s
I20260709 12:10:27.668375 14798 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:27.733554 12675 rpcz_store.cc:276] 0709 12:10:26.347748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:26.484706 (+136958us) service_pool.cc:224] Handling call
0709 12:10:27.710625 (+1225919us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b540016f3d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:27.710798 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":328988544}
I20260709 12:10:27.735795 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:27.655817 14757 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:10:27.662379 14613 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.030s
W20260709 12:10:27.730497 14767 consensus_meta.cc:287] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.914s	user 0.008s	sys 0.006s
I20260709 12:10:27.752804 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:27.753403 12606 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:27.685653 14686 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:27.685514 14739 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.679927 14595 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:27.685860 14560 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:27.696208 14801 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:27.741883 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1515) took 1311 ms. Trace:
I20260709 12:10:27.678824 14576 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.070s
I20260709 12:10:27.695935 14454 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:27.758760 12602 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:10:27.751929 14807 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:10:27.747534 14748 consensus_meta.cc:287] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.335s	user 0.012s	sys 0.004s
I20260709 12:10:27.685259 14811 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:27.763831 12599 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:27.736713 14769 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:27.777782 14766 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.005s
I20260709 12:10:27.750650 14798 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:27.757903 14637 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:27.761622 14806 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:27.745075 13334 raft_consensus.cc:2376] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3. Re-sending same reply.
I20260709 12:10:27.705739 14559 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:27.752099 14770 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:27.780843 14810 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.211s	user 0.002s	sys 0.000s
W20260709 12:10:27.785221 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 113878us
I20260709 12:10:27.787786 14595 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.800779 12494 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:27.782334 12677 rpcz_store.cc:276] 0709 12:10:26.432409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:27.438502 (+1006093us) service_pool.cc:224] Handling call
0709 12:10:27.439029 (+   527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:27.463093 (+ 24064us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:27.466407 (+  3314us) log.cc:844] Serialized 14 byte log entry
0709 12:10:27.709375 (+242968us) spinlock_profiling.cc:243] Waited 1.48 ms on lock 0x7b180039b620. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a 00007f1488a2aa2a
0709 12:10:27.741455 (+ 32080us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:27.741531 (+    76us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:27.741592 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:27.741672 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:27.741811 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":3253,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":2579,"raft.queue_time_us":7885,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":7800,"spinlock_wait_cycles":3407104,"thread_start_us":24990,"threads_started":1}
I20260709 12:10:27.772311 14816 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:10:27.776636 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 111254us
I20260709 12:10:27.777508 14767 raft_consensus.cc:3055] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:27.778654 14801 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:27.821723 12527 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:27.790267 14748 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:27.828408 14748 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:27.830454 14797 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:27.830251 14816 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:27.830891 14743 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:27.832041 14512 consensus_meta.cc:287] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.974s	user 0.007s	sys 0.007s
I20260709 12:10:27.832787 14816 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:27.785851 14807 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:27.808833 14240 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:27.833374 14807 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:27.833453 14595 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:27.785494 14486 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.021s
W20260709 12:10:27.814272 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:27.818961 14598 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:27.820215 14502 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.939s	user 0.014s	sys 0.000s
W20260709 12:10:27.821429 14449 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.606s	user 0.012s	sys 0.000s
I20260709 12:10:27.832507 14512 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:27.811548 14683 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:27.795390 14820 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.794200 14769 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:27.788933 14811 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:27.807106 12496 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:27.838755 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1504) took 1490 ms. Trace:
I20260709 12:10:27.836943 14559 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:27.851738 12606 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:27.776203 14739 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:27.838205 14818 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:10:27.821898 14767 raft_consensus.cc:740] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:27.798529 14637 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:27.837091 14595 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:27.857827 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 57119.6us
I20260709 12:10:27.890455 14637 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:27.891034 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1513) took 1468 ms. Trace:
I20260709 12:10:27.891424 12673 rpcz_store.cc:276] 0709 12:10:26.423801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:26.838410 (+414609us) service_pool.cc:224] Handling call
0709 12:10:27.240369 (+401959us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:27.240962 (+   593us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:27.890828 (+649866us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b54002c4a50. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:27.890967 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":527680384}
I20260709 12:10:27.843848 14757 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:27.894526 14757 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:27.890014 14660 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:27.895478 14757 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:27.897205 14757 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:27.864302 12676 rpcz_store.cc:276] 0709 12:10:26.348130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:26.524493 (+176363us) service_pool.cc:224] Handling call
0709 12:10:26.583977 (+ 59484us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:26.589251 (+  5274us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:27.838438 (+1249187us) spinlock_profiling.cc:243] Waited 57.2 ms on lock 0x7b54000aafd0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:27.838656 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131557632}
I20260709 12:10:27.899847 12600 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:10:27.868793 12672 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:10:27.852356 14502 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:27.869540 14827 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:27.901686 14827 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:27.870661 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 63633.2us
I20260709 12:10:27.871892 14829 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:27.903128 14502 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:27.869423 14766 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:27.754531 14805 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:27.902940 12604 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:27.883276 14796 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:27.885645 14454 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:27.886303 14512 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:27.796756 14774 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.206s
W20260709 12:10:27.123008 14309 ts_tablet_manager.cc:1469] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Trace:
0709 12:10:12.160879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:12.164609 (+  3730us) service_pool.cc:224] Handling call
0709 12:10:12.730041 (+565432us) ts_tablet_manager.cc:934] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
0709 12:10:17.589990 (+4859949us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:10:18.186979 (+596989us) tablet_metadata.cc:665] Metadata flushed
0709 12:10:18.187136 (+   157us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:10:18.820469 (+633333us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:10:18.866297 (+ 45828us) tablet_replica.cc:206] Creating consensus instance
0709 12:10:18.938849 (+ 72552us) ts_tablet_manager.cc:989] Replica registered.
0709 12:10:19.077213 (+138364us) inbound_call.cc:177] Queueing success response
0709 12:10:24.052397 (+4975184us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:10:26.177975 (+2125578us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:10:26.178998 (+  1023us) tablet_replica.cc:261] Starting instrumentation
0709 12:10:26.198747 (+ 19749us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":7353,"raft.queue_time_us":1643,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":16312,"tablet-copy.queue_time_us":455919,"tablet-download-pool.queue_time_us":232,"tablet-download-pool.run_cpu_time_us":2458,"tablet-download-pool.run_wall_time_us":3543864,"thread_start_us":487770,"threads_started":2}
I20260709 12:10:27.862715 14820 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:27.955771 13333 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:27.956328 14820 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:27.908645 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1523) took 1349 ms. Trace:
I20260709 12:10:27.957223 12671 rpcz_store.cc:276] 0709 12:10:26.559339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:27.469430 (+910091us) service_pool.cc:224] Handling call
0709 12:10:27.505122 (+ 35692us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:27.505504 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:27.505534 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:27.506168 (+   634us) log.cc:844] Serialized 26 byte log entry
0709 12:10:27.638808 (+132640us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:27.638855 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:27.638949 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:27.908334 (+269385us) raft_consensus.cc:1685] finished
0709 12:10:27.908362 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:27.908554 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10020,"raft.queue_time_us":1320,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":15043,"thread_start_us":72373,"threads_started":1,"wal-append.queue_time_us":140292}
I20260709 12:10:27.917083 12496 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:27.958731 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1574) took 1363 ms. Trace:
I20260709 12:10:27.959105 12525 rpcz_store.cc:276] 0709 12:10:26.595214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:26.598341 (+  3127us) service_pool.cc:224] Handling call
0709 12:10:27.958501 (+1360160us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54003cc3d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:27.958656 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2424854912}
I20260709 12:10:27.959753 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:27.866422 14811 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:27.807233 14770 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:27.962076 14746 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:27.967883 14770 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:27.968673 14770 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:27.885083 14826 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.855201 14449 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:27.973053 14826 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:27.926041 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:27.975481 14789 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:27.975664 12680 raft_consensus.cc:2410] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 3 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
W20260709 12:10:27.975872 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 49918.3us
I20260709 12:10:27.976110 14789 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:27.977381 14613 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:27.977147 14789 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:27.977914 14613 raft_consensus.cc:697] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:27.978260 14770 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:27.980295 14815 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.292s	user 0.000s	sys 0.001s
I20260709 12:10:27.981557 12677 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:27.982376 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:27.923112 14783 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:27.984697 14783 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:27.903043 14827 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:27.936043 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1570) took 1795 ms. Trace:
I20260709 12:10:27.985093 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:27.979400 14613 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:27.986506 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:27.928872 14819 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:27.997466 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:27.997466 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:27.998062 14819 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:27.998837 12675 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:27.943327 14767 consensus_queue.cc:260] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:10:27.999572 14819 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:27.945155 14739 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:28.004382 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:28.005242 12678 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:27.959439 14738 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:27.985924 14827 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:28.019763 14819 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:27.974561 14449 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:28.021898 14738 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:27.982532 12676 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:27.985530 12529 rpcz_store.cc:276] 0709 12:10:26.140875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:27.218390 (+1077515us) service_pool.cc:224] Handling call
0709 12:10:27.242789 (+ 24399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:27.526279 (+283490us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:27.935666 (+409387us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b54003f8c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:27.935949 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":613580672}
W20260709 12:10:27.942579 14828 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
W20260709 12:10:27.991099 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 307474us
I20260709 12:10:27.926968 14798 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:27.985771 12600 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:28.032421 12646 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:28.032792 12571 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:27.997380 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:28.034498 12597 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:27.997797 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:28.035283 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:27.999706 14364 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:27.949827 14801 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:27.918269 14454 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:28.035794 12571 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:28.042579 12677 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
W20260709 12:10:28.043906 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 395751us
I20260709 12:10:28.048307 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:28.006301 14767 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:28.055598 12526 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:28.007460 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:28.056723 12569 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:28.008610 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 336130us
I20260709 12:10:28.008759 12673 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:28.058503 12571 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:28.025903 13305 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:28.061681 14788 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:28.062379 14788 raft_consensus.cc:697] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 3 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:28.026803 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:27.941980 14412 consensus_meta.cc:287] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.611s	user 0.010s	sys 0.001s
I20260709 12:10:28.037686 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:28.050531 12646 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:27.964154 14691 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:28.075234 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:28.075376 12672 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 5: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:28.074795 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:28.079743 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:27.989400 14826 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:28.001854 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:28.068696 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:28.069394 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.024303 14292 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:28.070946 12680 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:28.071084 12678 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:28.072227 14502 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:28.072970 14748 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:28.073429 12571 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:28.082870 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:28.083029 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:28.109747 12522 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:28.108906 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:28.111089 14560 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:10:28.085795 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 413732us
I20260709 12:10:28.111797 14560 raft_consensus.cc:697] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:28.112782 14560 consensus_queue.cc:237] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.087678 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:27.953966 14637 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:28.116722 14637 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:28.092443 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:28.117838 12527 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:28.040591 14774 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:28.119616 14774 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:28.120124 14774 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:28.120615 12522 raft_consensus.cc:2410] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:10:28.040971 14240 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:28.122825 14240 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:28.123389 14240 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:28.093814 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 52689.9us
W20260709 12:10:28.100142 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:28.102914 12643 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.858s (SENT)
I20260709 12:10:27.965698 14371 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:28.121081 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.125382 12646 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:28.079037 12530 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:28.125382 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:28.128489 14746 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:28.132844 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:10:28.138301 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 53056.3us
I20260709 12:10:28.154340 12646 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:28.155620 14820 raft_consensus.cc:2764] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260709 12:10:28.155859 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.156585 14820 raft_consensus.cc:2764] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:10:28.158854 14746 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:28.160668 14820 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:28.161175 14820 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:28.162278 14820 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.164577 12646 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:10:28.166265 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.166201 14512 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:28.203197 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 123) took 1788 ms. Trace:
I20260709 12:10:28.169488 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:28.063382 14788 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:27.958020 14624 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.063796 14809 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260709 12:10:28.184814 13333 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:28.064349 14738 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:27.978372 14769 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:27.964089 14807 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:28.190857 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 120) took 1372 ms. Trace:
I20260709 12:10:28.091495 14412 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.084197 14749 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:28.191519 12600 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:28.191478 12598 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:28.051209 14595 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.132s
I20260709 12:10:28.102455 14806 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:28.093142 14745 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:27.979579 14770 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:28.206636 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 109769us
I20260709 12:10:28.207212 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:28.105186 14691 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:28.108315 14826 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:28.208595 13337 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:28.185127 14538 tablet_copy_client.cc:538] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260709 12:10:28.071261 14783 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.118s
I20260709 12:10:28.323081 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:28.323534 12599 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 7: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:10:28.226080 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.250458 12604 rpcz_store.cc:276] 0709 12:10:26.822155 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:26.831932 (+  9777us) service_pool.cc:224] Handling call
0709 12:10:28.190601 (+1358669us) spinlock_profiling.cc:243] Waited 889 ms on lock 0x7b5400231c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:28.190792 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2045113728}
W20260709 12:10:28.252310 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 164707us
I20260709 12:10:28.327235 12494 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:28.084707 14739 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:28.328399 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1669) took 1203 ms. Trace:
I20260709 12:10:28.328593 14739 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.329025 12598 rpcz_store.cc:276] 0709 12:10:27.124506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:27.125101 (+   595us) service_pool.cc:224] Handling call
0709 12:10:28.191289 (+1066188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:28.276572 (+ 85283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:28.328125 (+ 51553us) spinlock_profiling.cc:243] Waited 977 ms on lock 0x7b5400231c50. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:28.328308 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2247013760}
I20260709 12:10:28.330560 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:28.331518 14739 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:28.255733 13332 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260709 12:10:28.332310 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:28.333092 12597 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:28.203685 12528 rpcz_store.cc:276] 0709 12:10:26.414353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:26.414480 (+   127us) service_pool.cc:224] Handling call
0709 12:10:26.421640 (+  7160us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:26.424892 (+  3252us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:10:26.480069 (+ 55177us) log.cc:844] Serialized 14 byte log entry
0709 12:10:27.746413 (+1266344us) spinlock_profiling.cc:243] Waited 3.45 ms on lock 0x7b180030bc20. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a
0709 12:10:28.202758 (+456345us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:28.202833 (+    75us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:28.202886 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:28.202972 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:28.203125 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53421,"prepare.queue_time_us":178898,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":56070,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"spinlock_wait_cycles":7940992,"thread_start_us":682020,"threads_started":3,"wal-append.queue_time_us":308755,"wal-append.run_cpu_time_us":1443,"wal-append.run_wall_time_us":1052048}
I20260709 12:10:28.104579 14292 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.208055 14512 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:28.335361 14512 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:28.335970 12526 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:28.336232 14748 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:28.337241 14748 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:28.306190 12525 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:28.348284 12526 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:28.192407 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:28.357051 12597 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
W20260709 12:10:28.362785 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 250529us
I20260709 12:10:28.365607 12572 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:28.322748 14806 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:28.373018 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:28.317271 14836 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:28.317446 14768 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:10:28.365878 12643 leader_election.cc:336] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.244s (SENT)
I20260709 12:10:28.278653 14829 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:28.314863 14576 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:28.355798 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:10:28.404752 14743 consensus_meta.cc:287] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.573s	user 0.006s	sys 0.005s
I20260709 12:10:28.345103 14691 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:28.406209 14847 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:10:28.405635 14806 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.406594 14691 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:28.406977 14806 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:28.408202 14839 log.cc:927] Time spent T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:10:28.410923 12523 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:28.411890 14811 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:28.413595 14834 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.157s
I20260709 12:10:28.343787 14826 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.415966 14826 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:28.417590 14807 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:28.418205 14613 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:28.399582 14598 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:10:28.368294 14741 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:28.418711 14613 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.420605 14364 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:28.307946 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:28.250181 14738 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.399480 12494 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:28.225502 14454 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.565953 14613 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:28.534253 14822 raft_consensus.cc:3055] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:10:28.545441 14820 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:28.520596 14559 consensus_meta.cc:287] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.652s	user 0.011s	sys 0.000s
I20260709 12:10:28.544267 14292 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:28.526199 14576 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:28.423298 14801 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:28.474260 14796 consensus_queue.cc:1059] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.123s
I20260709 12:10:28.597573 14576 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:28.544111 14827 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.524s	user 0.010s	sys 0.000s
I20260709 12:10:28.368409 14523 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:28.473505 14811 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:28.483445 14598 raft_consensus.cc:697] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:28.598902 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.489356 14678 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:28.570572 14774 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:28.602903 14678 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:28.603884 14678 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:28.604912 14774 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:28.605237 14678 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:28.599318 14559 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:28.609464 14559 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:28.425820 14843 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:28.613088 14559 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:28.595956 14613 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:28.594141 12646 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:28.615757 12646 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:10:28.616926 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 251716us
I20260709 12:10:28.616937 14820 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:28.617834 12643 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:28.618014 14820 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:28.618842 14746 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:28.619124 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 270002us
W20260709 12:10:28.619925 12643 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.270s (SENT) [suppressed 23 similar messages]
I20260709 12:10:28.617861 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:28.625415 14824 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:28.627000 14864 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:28.628834 14864 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:28.630044 14864 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.436334 14798 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:28.628839 14576 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.633610 14738 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:28.529827 14757 consensus_meta.cc:287] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.632s	user 0.001s	sys 0.012s
I20260709 12:10:28.425856 14836 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:28.425649 12425 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58992 (request call id 169) took 21052 ms (client timeout 15000 ms). Trace:
I20260709 12:10:28.650326 14796 raft_consensus.cc:3055] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:28.650787 14796 raft_consensus.cc:740] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:28.651686 14796 consensus_queue.cc:260] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:28.653527 14796 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:28.654327 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:28.551455 14829 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:28.655442 14829 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:28.424527 14741 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:28.662480 14741 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:28.468079 14412 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:28.575948 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.577732 14371 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:28.540199 14502 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:28.666540 14502 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.667604 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:28.668107 14613 raft_consensus.cc:697] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:28.668339 12678 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:28.668831 14502 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:28.476693 14807 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:28.669250 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:28.550915 14819 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.530s	user 0.006s	sys 0.005s
I20260709 12:10:28.518160 14691 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.670456 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:28.670733 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:28.670907 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.671108 12672 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 4
I20260709 12:10:28.671535 12527 raft_consensus.cc:2410] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:28.671793 14691 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:28.594385 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 194992us
I20260709 12:10:28.673393 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:28.675486 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:28.676334 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:28.676966 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:28.677194 12677 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:28.677790 12523 raft_consensus.cc:2410] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:10:28.678704 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:28.678872 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:28.679476 12598 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:10:28.680127 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:28.680722 12604 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:28.681140 12601 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:28.681258 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:28.681614 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:28.682315 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:28.683578 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:28.683725 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:28.684337 12523 raft_consensus.cc:2410] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:10:28.684347 12600 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:28.595901 14822 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:28.669248 14613 consensus_queue.cc:237] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.686992 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:28.686798 14822 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.687844 14822 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:10:28.574849 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.688287 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:28.688797 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:28.689165 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.690821 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:28.691365 12522 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:28.439885 14769 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:28.627003 14797 raft_consensus.cc:3055] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:28.703928 14797 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:28.704151 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:28.704911 12678 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 5: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:28.704792 14797 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:28.705770 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:28.693094 14777 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:28.599773 14598 consensus_queue.cc:237] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.706636 14777 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:28.706346 12678 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:28.710642 12529 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:28.711274 12496 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:28.634496 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 57194.3us
I20260709 12:10:28.635628 14746 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:10:28.546227 14768 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:28.635985 12571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.002s	sys 0.021s
I20260709 12:10:28.617417 14801 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:28.636518 14820 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:28.718982 12643 leader_election.cc:336] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.828s (SENT)
I20260709 12:10:28.719326 14801 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:28.718017 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:28.721863 14801 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:28.722325 14769 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:28.724257 14836 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:28.640538 14576 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:28.640694 13338 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.000s	sys 0.011s
I20260709 12:10:28.726426 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:28.717250 14819 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:28.598526 14523 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:28.727276 12527 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 3: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:28.727526 14523 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:28.645779 12425 rpcz_store.cc:269] 0709 12:10:07.373022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:07.373370 (+   348us) service_pool.cc:224] Handling call
0709 12:10:28.425516 (+21052146us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:25.738852 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:25.739623 (+   771us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:25.739691 (+    68us) write_op.cc:435] Acquired schema lock
0709 12:10:25.739718 (+    27us) tablet.cc:662] Decoding operations
0709 12:10:25.740283 (+   565us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:25.740368 (+    85us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:25.740419 (+    51us) tablet.cc:685] Acquiring locks for 1 operations
0709 12:10:25.740817 (+   398us) tablet.cc:701] Row locks acquired
0709 12:10:25.740868 (+    51us) write_op.cc:260] PREPARE: finished
0709 12:10:25.741090 (+   222us) write_op.cc:270] Start()
0709 12:10:25.741273 (+   183us) write_op.cc:276] Timestamp: P: 1783599025741006 usec, L: 0
0709 12:10:25.741308 (+    35us) op_driver.cc:348] REPLICATION: starting
0709 12:10:25.742392 (+  1084us) log.cc:844] Serialized 540 byte log entry
0709 12:10:26.149341 (+406949us) op_driver.cc:464] REPLICATION: finished
0709 12:10:26.352031 (+202690us) write_op.cc:301] APPLY: starting
0709 12:10:26.352176 (+   145us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:26.352331 (+   155us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:26.352506 (+   175us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:26.352936 (+   430us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:26.353210 (+   274us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0709 12:10:26.353256 (+    46us) write_op.cc:312] APPLY: finished
0709 12:10:26.416073 (+ 62817us) log.cc:844] Serialized 22 byte log entry
0709 12:10:26.421333 (+  5260us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:26.421810 (+   477us) write_op.cc:454] Released schema lock
0709 12:10:26.422775 (+   965us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:26.428129 (+  5354us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:26.428747 (+   618us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":114728,"reported_tablets":10,"child_traces":[["op",{"apply.queue_time_us":202075,"apply.run_cpu_time_us":3497,"apply.run_wall_time_us":75106,"num_ops":1,"prepare.queue_time_us":433495,"prepare.run_cpu_time_us":12935,"prepare.run_wall_time_us":117034,"raft.queue_time_us":270149,"raft.run_cpu_time_us":29722,"raft.run_wall_time_us":105588,"replication_time_us":401336,"thread_start_us":628301,"threads_started":4,"wal-append.queue_time_us":123856,"wal-append.run_cpu_time_us":53137,"wal-append.run_wall_time_us":1559134}]]}
I20260709 12:10:28.730738 14768 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.733911 14746 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:28.734591 14746 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:28.627261 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:28.735970 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.736984 14691 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:28.737099 14746 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:28.737062 12643 leader_election.cc:336] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.723s (SENT)
I20260709 12:10:28.697526 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:28.700088 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:28.741425 12530 raft_consensus.cc:2410] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:28.742377 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:28.742444 12494 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:28.567590 14512 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.703455 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:28.705927 14797 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:28.743922 12677 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:28.744305 12494 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:28.743887 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:28.745292 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:28.577724 14817 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:28.746096 14746 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:28.746295 14769 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:28.746414 14817 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:28.747000 14827 raft_consensus.cc:2764] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260709 12:10:28.746924 14746 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:28.748631 14817 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:28.716135 12522 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:28.749512 14449 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:28.750345 14449 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.718875 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:28.720543 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:28.720822 12525 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:10:28.751575 12525 pending_rounds.cc:85] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 2
I20260709 12:10:28.751806 12525 pending_rounds.cc:107] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:10:28.752044 12525 raft_consensus.cc:2889] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:10:28.639408 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.753082 14449 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:28.753335 12496 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:28.726696 13338 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.000s	sys 0.012s
I20260709 12:10:28.682410 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:28.718287 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:28.755182 12677 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 4: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:28.751166 14769 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:28.755997 14769 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:28.713325 12571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.002s	sys 0.022s
W20260709 12:10:28.757941 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 625209us
I20260709 12:10:28.749239 14412 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:28.593149 14624 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:28.737612 14691 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.753891 14449 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:28.742372 14819 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:28.702800 14741 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:28.745975 14811 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:10:28.750928 12522 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:10:28.696621 12672 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:28.734578 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:28.654487 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 441136us
I20260709 12:10:28.762416 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:28.764187 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:28.765344 12678 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:28.765895 12571 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:28.776127 12524 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:28.775486 12496 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:28.776666 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:28.736752 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.637648 14454 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:28.776085 12528 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:28.749284 14817 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:28.779580 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:28.783195 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:28.784485 12530 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 7: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:10:28.789873 12676 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.003s	sys 0.012s
I20260709 12:10:28.785698 12496 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:28.790234 12600 raft_consensus.cc:2410] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 12:10:28.790699 12676 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.606s	user 0.003s	sys 0.012s
I20260709 12:10:28.785679 12643 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:28.786373 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:28.786460 12646 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:28.740429 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:28.820489 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:28.771231 14449 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:28.821882 12527 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:28.824101 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:28.825189 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:28.842517 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 56140.6us
W20260709 12:10:28.862852 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.863025 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:28.863656 12646 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:10:28.858889 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 51121.1us
I20260709 12:10:28.859012 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:28.820880 12529 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:28.825758 12678 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260709 12:10:28.865253 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 79931.6us
I20260709 12:10:28.554145 14240 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.865477 12571 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:28.874532 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:28.744246 14829 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.791879 14819 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:10:28.877081 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 91449.4us
I20260709 12:10:28.788341 14595 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:28.903010 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:28.918365 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:28.792294 14807 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.113s
W20260709 12:10:28.928828 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54401.3us
I20260709 12:10:28.927930 12604 raft_consensus.cc:2410] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
W20260709 12:10:28.904351 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:28.905560 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:28.840868 14806 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:28.858682 14739 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:28.858700 14502 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:28.792138 14412 consensus_queue.cc:1059] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.136s
I20260709 12:10:28.867024 14449 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:28.934763 13337 raft_consensus.cc:2410] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:10:28.927210 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:28.950964 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:28.878754 14560 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:28.970973 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:28.826692 14817 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.987217 12597 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:28.867699 14371 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:28.792935 14836 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260709 12:10:28.990293 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 45386.4us
I20260709 12:10:28.975179 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:10:28.877269 14748 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.003s	sys 0.003s
I20260709 12:10:28.923964 13335 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:28.880717 14783 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:10:29.041586 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 114429us
I20260709 12:10:29.044080 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:29.005532 12527 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
W20260709 12:10:28.950256 14789 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.001s	sys 0.021s
I20260709 12:10:29.019678 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.022902 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:28.792151 14777 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:28.879429 14854 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:28.910105 14822 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:29.049774 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:28.939273 14595 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:29.039577 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:29.097349 12526 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:28.936695 14512 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:28.935556 14741 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:29.099220 12525 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
W20260709 12:10:29.053310 14789 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.648s	user 0.001s	sys 0.022s
I20260709 12:10:29.052491 14836 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:10:29.082062 14872 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.262s	user 0.000s	sys 0.000s
I20260709 12:10:28.997049 14502 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.870427 14559 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.170s
I20260709 12:10:28.990959 14806 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:28.929874 14857 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:29.097954 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:29.044487 14748 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.006s	sys 0.012s
I20260709 12:10:29.105135 14595 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:29.024011 14371 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:28.932547 14801 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:29.111785 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:29.139309 12678 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:29.140653 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:29.141466 12678 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 5: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:29.142166 12523 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:29.144253 12523 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:29.145618 12523 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:29.043458 14560 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:29.146888 14560 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:28.872080 14486 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.169s
W20260709 12:10:29.150940 14867 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:10:29.152674 14801 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:29.153437 14801 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:29.154677 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1586) took 1108 ms. Trace:
I20260709 12:10:29.060274 14811 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:29.155368 12531 rpcz_store.cc:276] 0709 12:10:28.046168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:28.046299 (+   131us) service_pool.cc:224] Handling call
0709 12:10:28.083895 (+ 37596us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:28.217242 (+133347us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:28.217280 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:28.659469 (+442189us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b2c00058028. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:10:28.659987 (+   518us) log.cc:844] Serialized 52 byte log entry
0709 12:10:28.873492 (+213505us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:28.873535 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:28.873612 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:29.154233 (+280621us) raft_consensus.cc:1685] finished
0709 12:10:29.154263 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:29.154598 (+   335us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":120604,"raft.queue_time_us":163239,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":32869,"spinlock_wait_cycles":292106112,"thread_start_us":213103,"threads_started":1,"wal-append.queue_time_us":419218}
I20260709 12:10:29.157298 14769 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:29.130429 12597 raft_consensus.cc:2368] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 3. Current term is 4.
I20260709 12:10:29.128131 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:29.163378 14240 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:29.164827 14870 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:10:29.165989 14852 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:29.105754 14854 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:29.166790 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.166756 14852 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:29.168257 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.134069 12529 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:29.169454 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.170338 14733 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:29.170445 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:29.171526 14733 raft_consensus.cc:3055] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:29.172048 14733 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:29.160660 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.115710 14836 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:29.173566 12597 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:29.167459 12604 raft_consensus.cc:2410] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:29.173028 14733 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:29.154752 12674 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:29.175633 14733 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:29.133213 14768 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:29.180415 12646 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:29.171622 12529 raft_consensus.cc:2368] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6.
I20260709 12:10:29.107100 12524 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:29.182003 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:29.115239 12672 raft_consensus.cc:2410] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:10:29.174531 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.209633 12597 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:29.129196 14512 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:10:29.198844 14848 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.728s	user 0.008s	sys 0.009s
I20260709 12:10:29.129344 14827 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:29.205405 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:29.181998 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:29.153868 14801 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:29.203624 13302 leader_election.cc:336] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.010s (SENT)
I20260709 12:10:29.207844 12528 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:29.234360 13302 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:29.186623 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:29.235781 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:29.215531 12600 raft_consensus.cc:2410] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 3 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:29.236114 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:29.236505 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1695) took 1240 ms. Trace:
I20260709 12:10:29.216900 12646 leader_election.cc:400] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6.
I20260709 12:10:29.217465 14806 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:29.138278 14292 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:29.221870 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:29.252724 12672 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 5: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:29.137961 14820 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:29.178373 14371 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:29.226035 12606 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:10:29.232647 12678 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:29.257474 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:29.261441 14801 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:29.264382 14801 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:29.265046 14801 raft_consensus.cc:697] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:29.215382 12526 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
W20260709 12:10:29.278867 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 75278.6us
I20260709 12:10:29.237067 13333 raft_consensus.cc:2410] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 7 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:10:29.236927 12676 rpcz_store.cc:276] 0709 12:10:27.996288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:28.041939 (+ 45651us) service_pool.cc:224] Handling call
0709 12:10:28.080816 (+ 38877us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:28.081000 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:28.791141 (+710141us) log.cc:844] Serialized 14 byte log entry
0709 12:10:29.236144 (+445003us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:29.236213 (+    69us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:29.236264 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:29.236333 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:29.236449 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":388410,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"thread_start_us":1045539,"threads_started":2}
I20260709 12:10:29.237134 12646 leader_election.cc:403] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:10:29.187886 14624 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:29.244092 12673 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 4: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:10:29.282754 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 74730.1us
I20260709 12:10:29.283277 13332 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:29.283650 12646 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:29.237692 12496 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:29.285677 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48089.7us
W20260709 12:10:29.287117 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:29.287864 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:29.254004 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:29.290337 12672 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 4: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:29.291953 12672 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:29.293354 12672 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:10:29.294751 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:29.223665 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.295773 12597 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:29.211607 14768 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:29.250430 14827 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:29.266286 14292 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:10:29.269301 14862 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 12:10:29.235955 12529 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:29.224476 14819 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:29.275807 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:29.206806 12524 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:29.299685 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.308687 12606 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:29.300843 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.302031 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1681) took 1320 ms. Trace:
I20260709 12:10:29.282318 14624 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:29.232474 14512 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:29.350965 14624 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:29.351351 14512 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:29.352563 14512 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:29.353132 14624 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:29.353698 14624 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:29.299998 14811 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:29.312711 14820 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:29.355337 14820 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:29.356069 14820 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:29.314523 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.357544 12606 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:29.314621 14836 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:29.358649 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.316062 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.359303 12606 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:29.309897 12605 rpcz_store.cc:276] 0709 12:10:27.981904 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:27.987782 (+  5878us) service_pool.cc:224] Handling call
0709 12:10:27.988144 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:27.988539 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:27.988628 (+    89us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:27.989106 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 12:10:28.280989 (+291883us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:28.281069 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:28.281174 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:29.198238 (+917064us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:29.298595 (+100357us) raft_consensus.cc:1685] finished
0709 12:10:29.298626 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:29.301959 (+  3333us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1410,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":20042,"spinlock_wait_cycles":527659264,"thread_start_us":283971,"threads_started":1,"wal-append.queue_time_us":481105}
I20260709 12:10:29.359748 12597 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:29.316291 14292 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:29.325721 14801 consensus_queue.cc:237] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:29.299774 14789 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:29.334647 14768 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:29.335439 12529 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:29.344918 14852 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:29.347275 14827 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:29.303267 14678 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:29.348063 14819 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:29.355531 14850 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.974s	user 0.000s	sys 0.010s
I20260709 12:10:29.316531 14538 tablet_bootstrap.cc:492] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645: Bootstrap starting.
W20260709 12:10:29.318387 14523 consensus_meta.cc:287] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.590s	user 0.006s	sys 0.004s
I20260709 12:10:29.365960 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.357265 14820 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:29.376176 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:29.382562 12527 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:29.383724 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:29.384809 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.360824 14624 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:29.433005 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1694) took 1437 ms. Trace:
I20260709 12:10:29.436131 12606 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:29.447759 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:29.436874 12675 rpcz_store.cc:276] 0709 12:10:27.995962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:27.999729 (+  3767us) service_pool.cc:224] Handling call
0709 12:10:28.000091 (+   362us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:28.000494 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:28.000523 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:28.001198 (+   675us) log.cc:844] Serialized 52 byte log entry
0709 12:10:28.139061 (+137863us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:28.335988 (+196927us) log.cc:844] Serialized 14 byte log entry
0709 12:10:28.341346 (+  5358us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:28.341424 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:29.354864 (+1013440us) spinlock_profiling.cc:243] Waited 229 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:29.376859 (+ 21995us) raft_consensus.cc:1685] finished
0709 12:10:29.432743 (+ 55884us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:29.432922 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45538,"prepare.queue_time_us":329598,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":17137,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":10108,"spinlock_wait_cycles":527156480,"thread_start_us":162327,"threads_started":2,"wal-append.queue_time_us":198237}
I20260709 12:10:29.452508 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:29.453239 12606 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:29.457549 12572 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:29.458226 12494 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:29.463654 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.464708 12597 raft_consensus.cc:2368] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 3. Current term is 4.
I20260709 12:10:29.440850 14807 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:29.435878 14827 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:29.468775 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.480940 12605 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:29.485065 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.486176 12597 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:29.487257 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.488067 12597 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:29.489151 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:29.490849 12599 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.021s	sys 0.016s
I20260709 12:10:29.489840 12597 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:29.462595 14454 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:29.462385 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:29.467206 14857 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:29.370942 14789 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:29.448254 14523 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:29.465699 14807 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:29.518990 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.530195 12605 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:29.533193 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.534430 12605 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:29.535444 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.536413 12605 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:29.518736 13331 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
W20260709 12:10:29.496096 12599 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.022s	sys 0.016s
I20260709 12:10:29.509086 14852 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:29.541347 12569 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:29.550681 14576 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:29.556926 12597 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:29.484601 14827 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:29.552467 14807 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:29.554337 12531 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:29.509603 14836 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:29.592104 12606 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:29.555687 14819 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:29.564540 12572 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:29.573045 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.551429 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.667186 12597 raft_consensus.cc:2368] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6.
I20260709 12:10:29.668215 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:29.668972 12597 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:29.533087 12616 tablet_copy_service.cc:342] P f5fc807f7ecc4d1ca0fd6be524f734ce: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-70b8c9b89e3d43fbb70422110d76702c received from {username='slave'} at 127.0.0.1:48492
I20260709 12:10:29.549324 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.670867 12599 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:29.672657 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:29.673643 12675 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 4: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:10:29.675748 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 111283us
W20260709 12:10:29.621565 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:29.581218 14811 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:29.677929 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:29.682396 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:29.684162 12677 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: [3.2-3.2]
W20260709 12:10:29.685098 12601 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.019s	sys 0.042s
I20260709 12:10:29.684569 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:29.688876 12601 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.519s	user 0.019s	sys 0.042s
W20260709 12:10:29.688478 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 400709us
I20260709 12:10:29.689473 12606 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 6: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:29.690954 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.692085 12606 raft_consensus.cc:2376] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4. Re-sending same reply.
W20260709 12:10:29.682140 14449 consensus_meta.cc:287] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.663s	user 0.009s	sys 0.003s
I20260709 12:10:29.686623 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.561167 14852 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:29.590444 14857 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.108s
I20260709 12:10:29.771021 12678 raft_consensus.cc:2410] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 7 because replica has last-logged OpId of term: 5 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:10:29.683449 14836 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.088s
I20260709 12:10:29.768867 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1697) took 1104 ms. Trace:
I20260709 12:10:29.772094 12674 rpcz_store.cc:276] 0709 12:10:28.664550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:28.678631 (+ 14081us) service_pool.cc:224] Handling call
0709 12:10:29.768584 (+1089953us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b54000ac3d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:29.768754 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1093130624}
I20260709 12:10:29.773356 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:29.773558 14854 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
W20260709 12:10:29.773878 14838 log.cc:927] Time spent T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.305s	user 0.000s	sys 0.011s
I20260709 12:10:29.774174 14854 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:29.681435 14807 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:29.772836 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.776324 12605 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:29.777066 12599 raft_consensus.cc:2393] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:10:29.776994 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 94689.2us
W20260709 12:10:29.683573 14861 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 1.046s	user 0.001s	sys 0.004s
I20260709 12:10:29.777446 12597 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:10:29.778028 12572 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:29.778923 14857 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:29.779364 14624 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:29.774663 14854 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:29.779852 14857 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:29.683797 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:29.828262 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:29.804217 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:29.845184 13338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 386) took 2310 ms. Trace:
W20260709 12:10:29.775396 14856 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.165s	user 0.004s	sys 0.014s
W20260709 12:10:29.765247 14844 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 1.288s	user 0.007s	sys 0.005s
I20260709 12:10:29.848282 13338 rpcz_store.cc:276] 0709 12:10:27.530634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:27.530789 (+   155us) service_pool.cc:224] Handling call
0709 12:10:27.562418 (+ 31629us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:27.917090 (+354672us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:27.917139 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:27.917758 (+   619us) log.cc:844] Serialized 26 byte log entry
0709 12:10:28.754636 (+836878us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:28.754686 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:28.754753 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:29.664006 (+909253us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:29.826633 (+162627us) raft_consensus.cc:1685] finished
0709 12:10:29.826663 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:29.841136 (+ 14473us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1842,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":114,"spinlock_wait_cycles":520602880,"thread_start_us":753719,"threads_started":1,"wal-append.queue_time_us":719262}
I20260709 12:10:29.850875 14576 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:29.782646 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.819635 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:29.807225 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:29.842525 12497 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.009s	sys 0.021s
I20260709 12:10:29.856288 12677 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:29.826210 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:29.793980 14770 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.211s	user 0.006s	sys 0.026s
W20260709 12:10:29.752085 14748 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.534s	user 0.031s	sys 0.038s
I20260709 12:10:29.862843 12676 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
W20260709 12:10:29.683727 14783 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.018s	sys 0.018s
I20260709 12:10:29.692603 14240 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:29.856945 12497 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.010s	sys 0.021s
I20260709 12:10:29.686801 14806 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:10:29.864977 14770 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.282s	user 0.006s	sys 0.034s
W20260709 12:10:29.857427 12528 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.016s	sys 0.014s
I20260709 12:10:29.860106 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.917909 12599 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:29.918936 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:29.919778 12599 raft_consensus.cc:2410] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:10:29.920572 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.921649 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:29.863749 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 395631us
I20260709 12:10:29.922528 13337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38472 (request call id 187) took 1140 ms. Trace:
I20260709 12:10:29.922940 13337 rpcz_store.cc:276] 0709 12:10:28.781707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:28.781856 (+   149us) service_pool.cc:224] Handling call
0709 12:10:29.922332 (+1140476us) spinlock_profiling.cc:243] Waited 111 ms on lock 0x7b54006450d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:29.922473 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256369920}
I20260709 12:10:29.857151 14576 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:29.692979 14811 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:29.872514 12674 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:29.780756 14449 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:29.781695 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.698858 14877 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.229s
W20260709 12:10:29.842747 12495 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.020s	sys 0.010s
W20260709 12:10:29.933552 12495 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.021s	sys 0.010s
I20260709 12:10:29.934059 12677 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: []
W20260709 12:10:29.760310 14876 log.cc:927] Time spent T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
W20260709 12:10:29.760756 14875 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
I20260709 12:10:29.819247 14733 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:29.943480 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.944959 12599 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:10:29.842991 12646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.007s	sys 0.023s
I20260709 12:10:29.681478 14822 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:29.946171 12677 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:10:29.841092 14824 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:29.946802 14824 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:29.946908 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:29.948866 14865 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.333s	user 0.004s	sys 0.000s
I20260709 12:10:29.949110 14824 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:29.949918 14824 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:29.950438 14824 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:29.951040 14769 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:29.951167 14854 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:29.951726 14769 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:29.952225 14769 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:10:29.906107 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.952986 14502 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:29.953868 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.954542 12604 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
W20260709 12:10:29.954803 14866 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.315s	user 0.002s	sys 0.003s
I20260709 12:10:29.955523 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:29.955955 14883 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:10:29.829128 14746 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:29.864784 14748 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.031s	sys 0.045s
W20260709 12:10:29.894748 14783 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.019s	sys 0.022s
I20260709 12:10:29.854592 14789 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:29.796725 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1672) took 2267 ms. Trace:
W20260709 12:10:29.927141 12528 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.016s	sys 0.015s
I20260709 12:10:29.827116 14852 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:29.896914 14371 consensus_meta.cc:287] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.615s	user 0.012s	sys 0.000s
I20260709 12:10:29.865321 14523 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:29.751556 14879 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:29.932067 12606 raft_consensus.cc:2393] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 3: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:29.932296 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:29.945477 14733 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:29.952572 14824 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:29.698644 14454 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:29.686869 12616 tablet_copy_service.cc:434] P f5fc807f7ecc4d1ca0fd6be524f734ce: ending tablet copy session e56d487365394ac5ac5f1002926a5645-70b8c9b89e3d43fbb70422110d76702c on tablet 70b8c9b89e3d43fbb70422110d76702c with peer e56d487365394ac5ac5f1002926a5645
I20260709 12:10:29.863919 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:29.985816 12646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.007s	sys 0.024s
I20260709 12:10:29.916239 14240 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:30.004819 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 721252us
I20260709 12:10:29.989317 12605 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:30.006456 12597 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:30.009177 12603 rpcz_store.cc:276] 0709 12:10:27.509521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:27.519033 (+  9512us) service_pool.cc:224] Handling call
0709 12:10:27.519494 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:27.529582 (+ 10088us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:27.529625 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:27.534891 (+  5266us) log.cc:844] Serialized 26 byte log entry
0709 12:10:27.840294 (+305403us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b18002c2780. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:10:28.076499 (+236205us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:28.076689 (+   190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:28.076984 (+   295us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:29.773272 (+1696288us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:29.776201 (+  2929us) raft_consensus.cc:1685] finished
0709 12:10:29.776220 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:29.776350 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10391,"raft.run_cpu_time_us":280,"raft.run_wall_time_us":27095,"spinlock_wait_cycles":403198976,"thread_start_us":235937,"threads_started":1,"wal-append.queue_time_us":921913}
I20260709 12:10:29.990296 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.957753 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:29.936486 14827 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:29.956352 12672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.658s	user 0.007s	sys 0.006s
W20260709 12:10:30.022748 12672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.724s	user 0.008s	sys 0.006s
I20260709 12:10:30.012856 12646 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:30.025004 12646 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:30.018460 12599 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
W20260709 12:10:29.992461 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 1.12611e+06us
I20260709 12:10:30.055768 12646 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:30.056468 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1699) took 1294 ms. Trace:
I20260709 12:10:29.991882 14783 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:29.992194 14371 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:30.056860 12598 rpcz_store.cc:276] 0709 12:10:28.762404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:28.762544 (+   140us) service_pool.cc:224] Handling call
0709 12:10:28.762903 (+   359us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:28.763176 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:28.763203 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:28.771700 (+  8497us) log.cc:844] Serialized 26 byte log entry
0709 12:10:29.116149 (+344449us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:29.116190 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:29.116270 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:30.056247 (+939977us) raft_consensus.cc:1685] finished
0709 12:10:30.056268 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:30.056408 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":249126,"threads_started":1,"wal-append.queue_time_us":684859}
W20260709 12:10:30.273015 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 365818us
I20260709 12:10:30.283193 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:30.065764 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:30.161036 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:30.208287 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:30.058398 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:30.289090 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:30.294350 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:30.297999 13338 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
W20260709 12:10:30.298760 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:30.291030 12597 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:30.323817 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:30.328775 12598 raft_consensus.cc:2410] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:10:30.339761 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 131623us
I20260709 12:10:30.341156 12571 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:30.101784 14512 consensus_meta.cc:287] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.748s	user 0.013s	sys 0.000s
I20260709 12:10:30.343356 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:30.353730 14769 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:30.354048 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:30.106787 14748 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:30.342901 14502 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:30.072309 14894 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:30.357807 14894 raft_consensus.cc:697] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:30.346140 13337 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:10:30.346601 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1685) took 2359 ms. Trace:
I20260709 12:10:30.096305 14843 raft_consensus.cc:3055] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:30.371518 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:30.111645 14880 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.247s	user 0.000s	sys 0.000s
I20260709 12:10:30.040359 14746 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:30.350832 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:30.332358 14879 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:30.013103 14824 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.324s
I20260709 12:10:30.072381 14801 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:30.357585 14783 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:30.358863 13331 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260709 12:10:30.358518 14748 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:30.055336 13334 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:30.351862 12597 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:30.112421 14595 consensus_meta.cc:287] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.924s	user 0.005s	sys 0.005s
W20260709 12:10:30.102653 14884 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.145s	user 0.001s	sys 0.000s
I20260709 12:10:30.312142 14559 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:30.155028 14891 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:30.363265 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:30.111112 14449 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:30.363807 14733 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:30.351096 14852 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:30.083823 14624 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:30.055833 14890 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:30.040462 14796 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:30.109920 14240 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:30.350380 14889 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:10:30.411836 14598 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.702s	user 0.030s	sys 0.044s
I20260709 12:10:30.012698 14857 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:30.355077 12523 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 4: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:30.385546 12598 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:30.451364 14882 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.408s	user 0.006s	sys 0.006s
I20260709 12:10:30.452927 14894 consensus_queue.cc:237] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:30.455363 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:30.429756 14891 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:30.459107 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1598) took 1694 ms. Trace:
I20260709 12:10:30.459586 12522 rpcz_store.cc:276] 0709 12:10:28.764402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:29.056070 (+291668us) service_pool.cc:224] Handling call
0709 12:10:29.056477 (+   407us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:29.100850 (+ 44373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:29.100884 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:29.199560 (+ 98676us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b2c00058028. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:10:29.200001 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 12:10:29.679937 (+479936us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:29.679976 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:29.680073 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:30.458746 (+778673us) raft_consensus.cc:1685] finished
0709 12:10:30.458771 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:30.459024 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":43890,"raft.queue_time_us":44032,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":7627,"spinlock_wait_cycles":8804224,"thread_start_us":479525,"threads_started":1,"wal-append.queue_time_us":705462}
I20260709 12:10:30.395102 14879 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:10:30.459589 14891 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:30.443387 14796 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:30.432140 14852 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:30.465135 14852 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:30.386029 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 230657us
I20260709 12:10:30.337735 14789 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:30.387704 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46642.3us
W20260709 12:10:30.294404 12643 connection.cc:256] RPC call timeout handler was delayed by -1.1218056000000161s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1542s for a response
I20260709 12:10:30.482924 14796 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:30.483505 12525 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1595) took 1720 ms (client timeout 1835 ms). Trace:
I20260709 12:10:30.483625 14240 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:30.484175 14240 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:30.485214 14898 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:30.485785 14855 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 2.015s	user 0.003s	sys 0.028s
I20260709 12:10:30.485816 14898 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:30.486986 14885 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:30.487701 14885 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:10:30.488524 14898 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:30.489142 14898 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:30.393175 12572 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:30.490537 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 159) took 2350 ms. Trace:
W20260709 12:10:30.483989 12525 rpcz_store.cc:269] 0709 12:10:28.763311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:28.763474 (+   163us) service_pool.cc:224] Handling call
0709 12:10:30.483327 (+1719853us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b54003f7d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:30.483440 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":636490112}
I20260709 12:10:30.398880 12526 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:30.370523 14512 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6.
I20260709 12:10:30.382192 14843 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:30.412650 14905 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:10:30.489619 14898 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:30.419925 12524 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 4: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:30.499029 12571 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:10:30.497731 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 104665us
I20260709 12:10:30.507367 12598 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:30.427853 12605 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:30.508208 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1685) took 1181 ms. Trace:
I20260709 12:10:30.508476 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1693) took 1172 ms. Trace:
I20260709 12:10:30.508855 12605 rpcz_store.cc:276] 0709 12:10:29.336638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:29.897878 (+561240us) service_pool.cc:224] Handling call
0709 12:10:30.508418 (+610540us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:30.510345 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:30.511134 12605 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:30.107096 14811 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:30.394969 14746 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:30.408744 14801 raft_consensus.cc:697] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:30.411159 14769 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:30.367936 12671 rpcz_store.cc:276] 0709 12:10:27.971818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:28.009633 (+ 37815us) service_pool.cc:224] Handling call
0709 12:10:28.010075 (+   442us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:28.037454 (+ 27379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:28.037484 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:28.065781 (+ 28297us) log.cc:844] Serialized 52 byte log entry
0709 12:10:28.356192 (+290411us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:28.356264 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:28.356400 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:29.774742 (+1418342us) spinlock_profiling.cc:243] Waited 212 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:30.004789 (+230047us) raft_consensus.cc:1685] finished
0709 12:10:30.004812 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:30.331131 (+326319us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4505,"raft.queue_time_us":22760,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":16282,"spinlock_wait_cycles":486776704,"thread_start_us":161283,"threads_started":1,"wal-append.queue_time_us":427178}
I20260709 12:10:30.470850 12597 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:30.522291 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 129) took 1031 ms. Trace:
I20260709 12:10:30.522799 12597 rpcz_store.cc:276] 0709 12:10:29.490585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.343071 (+852486us) service_pool.cc:224] Handling call
0709 12:10:30.522199 (+179128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:30.471848 14371 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:30.472935 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:30.429257 14559 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:30.434023 14904 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:30.477048 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1709) took 1365 ms. Trace:
I20260709 12:10:30.444695 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:30.457422 14889 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:30.440482 14890 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:30.486344 14820 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:30.486892 12523 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:30.337323 14836 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:30.336030 12494 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:30.541165 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 205346us
I20260709 12:10:30.521126 14877 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:30.490962 12680 rpcz_store.cc:276] 0709 12:10:28.140140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:28.140299 (+   159us) service_pool.cc:224] Handling call
0709 12:10:28.140700 (+   401us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:28.141192 (+   492us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:28.141219 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:28.141760 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 12:10:28.312551 (+170791us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:28.312615 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:28.312719 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:30.485162 (+2172443us) spinlock_profiling.cc:243] Waited 178 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:30.490205 (+  5043us) raft_consensus.cc:1685] finished
0709 12:10:30.490231 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:30.490469 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1001,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":124,"spinlock_wait_cycles":408882048,"thread_start_us":170315,"threads_started":1,"wal-append.queue_time_us":329000}
I20260709 12:10:30.499110 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1602) took 1559 ms. Trace:
I20260709 12:10:30.543977 12527 rpcz_store.cc:276] 0709 12:10:28.939815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:29.127685 (+187870us) service_pool.cc:224] Handling call
0709 12:10:30.498931 (+1371246us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54003ca5d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:30.499047 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":460681472}
W20260709 12:10:30.503018 14832 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 2.032s	user 0.007s	sys 0.015s
I20260709 12:10:30.502684 14733 consensus_queue.cc:1059] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:30.546336 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:30.547549 12524 raft_consensus.cc:2368] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7.
I20260709 12:10:30.513397 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:30.514278 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:30.446825 14598 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.738s	user 0.031s	sys 0.044s
I20260709 12:10:30.532748 14904 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:30.508607 12598 rpcz_store.cc:276] 0709 12:10:29.327047 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.350535 (+1023488us) service_pool.cc:224] Handling call
0709 12:10:30.508127 (+157592us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:30.434260 14576 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:30.528915 12522 raft_consensus.cc:2393] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:30.526186 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:30.530197 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:30.524962 14901 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.006s
I20260709 12:10:30.521028 14560 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:30.505918 14789 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:30.533288 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:30.533329 12672 rpcz_store.cc:276] 0709 12:10:29.111393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:29.293966 (+182573us) service_pool.cc:224] Handling call
0709 12:10:29.298061 (+  4095us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:30.023590 (+725529us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:30.023651 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:30.084369 (+ 60718us) log.cc:844] Serialized 26 byte log entry
0709 12:10:30.367604 (+283235us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:30.367638 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:30.367715 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:30.476790 (+109075us) raft_consensus.cc:1685] finished
0709 12:10:30.476812 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:30.476972 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":147109,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":9846,"thread_start_us":979664,"threads_started":2,"wal-append.queue_time_us":321636}
I20260709 12:10:30.528630 14769 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:30.521348 14502 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:30.542304 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:30.414575 14595 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:30.544591 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:30.546151 12646 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:30.516940 14893 consensus_queue.cc:1059] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260709 12:10:30.712602 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:30.592094 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 306563us
I20260709 12:10:30.720537 12680 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:30.548312 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:30.720923 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:30.721474 12527 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:30.721503 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:30.721716 12522 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 3: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:30.580853 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 156) took 3022 ms. Trace:
I20260709 12:10:30.684854 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:30.722241 12680 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
W20260709 12:10:30.662461 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 145438us
I20260709 12:10:30.542186 14900 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:30.727464 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 185262us
I20260709 12:10:30.731264 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:30.725826 12571 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:30.544232 14885 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:30.543994 14292 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:30.762820 12528 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:30.723443 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:30.723382 12597 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:30.795938 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 1710) took 1350 ms. Trace:
I20260709 12:10:30.723142 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:30.726888 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 213643us
I20260709 12:10:30.797410 12572 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:30.543041 14820 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:30.751557 14894 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:30.722389 12679 rpcz_store.cc:276] 0709 12:10:27.558325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:27.636325 (+ 78000us) service_pool.cc:224] Handling call
0709 12:10:27.645113 (+  8788us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:27.689789 (+ 44676us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:27.689834 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:27.760851 (+ 71017us) log.cc:844] Serialized 26 byte log entry
0709 12:10:27.991745 (+230894us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:27.991794 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:27.992986 (+  1192us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:30.502299 (+2509313us) spinlock_profiling.cc:243] Waited 190 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:30.580328 (+ 78029us) raft_consensus.cc:1685] finished
0709 12:10:30.580351 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:30.580768 (+   417us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":41961,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":53136,"spinlock_wait_cycles":437556608,"thread_start_us":244852,"threads_started":2,"wal-append.queue_time_us":688557}
I20260709 12:10:30.726581 12605 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:10:30.752265 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:30.719870 12677 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:30.668576 14906 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:30.556341 14877 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.225s
I20260709 12:10:30.538956 14801 consensus_queue.cc:237] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:30.763044 14292 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:30.824993 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:30.825489 14292 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:30.709895 14863 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 2.097s	user 0.006s	sys 0.016s
I20260709 12:10:30.762351 12603 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:30.829039 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1701) took 1489 ms. Trace:
I20260709 12:10:30.829434 12603 rpcz_store.cc:276] 0709 12:10:29.339468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.316456 (+976988us) service_pool.cc:224] Handling call
0709 12:10:30.828819 (+512363us) spinlock_profiling.cc:243] Waited 390 ms on lock 0x7b5400230d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:30.828971 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":895976832}
I20260709 12:10:30.832150 14739 raft_consensus.cc:3055] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:30.832309 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:30.832667 14739 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:30.748149 14900 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:30.662971 14240 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:30.711609 14890 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:30.556137 14843 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:30.669780 14846 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 2.268s	user 0.012s	sys 0.015s
I20260709 12:10:30.553658 14733 consensus_queue.cc:1243] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:10:30.834136 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:30.768054 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 222002us
W20260709 12:10:30.756242 14864 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.121s	user 0.015s	sys 0.015s
W20260709 12:10:30.839491 14864 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.205s	user 0.017s	sys 0.017s
I20260709 12:10:30.586793 14748 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:30.839781 12679 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:30.774188 12494 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:30.836038 14889 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:30.840667 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 66583.8us
I20260709 12:10:30.836826 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:30.841284 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:30.841696 12606 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:30.841866 12679 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:30.839488 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:30.842795 14925 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:30.843257 14925 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:30.756484 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 272243us
I20260709 12:10:30.843501 12646 leader_election.cc:400] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7.
I20260709 12:10:30.843935 12646 leader_election.cc:403] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:10:30.844753 14877 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:30.845189 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:30.845258 14877 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:30.796322 12597 rpcz_store.cc:276] 0709 12:10:29.445113 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.529775 (+1084662us) service_pool.cc:224] Handling call
0709 12:10:30.795868 (+266093us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:30.725874 14783 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:30.847931 14864 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:30.848500 14783 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:30.850957 14783 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:30.752611 14738 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:30.730670 12525 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:30.753629 14819 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:30.814842 14595 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:30.819216 12600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1645) took 1995 ms (client timeout 1535 ms). Trace:
I20260709 12:10:30.819334 14769 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:30.819648 14576 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:30.820784 14691 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:30.821420 14454 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:30.821648 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:30.821918 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:30.796475 14559 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:30.837831 14843 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:30.763244 14502 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 12:10:30.774688 14598 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:30.712296 14745 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:30.775148 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:30.839517 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1678) took 2858 ms. Trace:
I20260709 12:10:30.928324 12672 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:30.854454 12671 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:30.856495 12679 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:30.929522 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:30.930249 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:30.930814 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1684) took 1604 ms. Trace:
I20260709 12:10:30.931056 12679 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:30.931382 12597 rpcz_store.cc:276] 0709 12:10:29.326721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.874291 (+1547570us) service_pool.cc:224] Handling call
0709 12:10:30.930745 (+ 56454us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:30.544525 14836 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:30.862419 12600 rpcz_store.cc:269] 0709 12:10:28.823734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:28.823858 (+   124us) service_pool.cc:224] Handling call
0709 12:10:30.818980 (+1995122us) spinlock_profiling.cc:243] Waited 352 ms on lock 0x7b54002f2954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:30.819129 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":810709760}
W20260709 12:10:30.888242 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 90932.1us
I20260709 12:10:30.935638 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:30.723692 14560 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:30.795465 14885 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election won for term 4
W20260709 12:10:30.891266 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46161us
I20260709 12:10:30.843612 14925 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:30.797780 14898 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:30.863781 14894 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:30.947482 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:30.892668 14788 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.319s	user 0.008s	sys 0.013s
I20260709 12:10:30.895035 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:30.948722 12603 raft_consensus.cc:2368] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6.
I20260709 12:10:30.948810 14877 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:30.900295 14796 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:10:30.949836 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:30.950489 12603 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:30.951427 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:30.951756 14894 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:30.952277 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1702) took 1612 ms. Trace:
I20260709 12:10:30.952620 12603 rpcz_store.cc:276] 0709 12:10:29.339801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.951176 (+1611375us) service_pool.cc:224] Handling call
0709 12:10:30.952216 (+  1040us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:30.953205 14820 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:30.953552 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:30.927974 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:30.954290 12680 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:30.928602 12602 rpcz_store.cc:276] 0709 12:10:27.981096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:27.982999 (+  1903us) service_pool.cc:224] Handling call
0709 12:10:27.983348 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:27.983710 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:27.983737 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:27.984302 (+   565us) log.cc:844] Serialized 26 byte log entry
0709 12:10:28.054976 (+ 70674us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:28.056257 (+  1281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:28.056328 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:30.580263 (+2523935us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:30.839225 (+258962us) raft_consensus.cc:1685] finished
0709 12:10:30.839245 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:30.839458 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5999,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":20357,"spinlock_wait_cycles":489293184,"thread_start_us":70271,"threads_started":1,"wal-append.queue_time_us":322108}
I20260709 12:10:30.955837 14836 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:30.956001 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:30.956885 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1678) took 1636 ms. Trace:
I20260709 12:10:30.957512 14502 raft_consensus.cc:3055] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:30.957939 14502 raft_consensus.cc:740] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:10:30.958719 14917 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:10:30.934417 14240 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7.
I20260709 12:10:30.959421 14502 consensus_queue.cc:260] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:30.890479 14738 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:30.961287 12527 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:30.931718 14576 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:30.962064 14576 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:30.964419 14576 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:30.894739 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:30.603058 14746 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:30.894316 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:30.967114 12606 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:30.851603 14783 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:10:30.835695 14900 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:30.975288 14885 raft_consensus.cc:697] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:30.976365 14885 consensus_queue.cc:237] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:30.861811 14891 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:30.901700 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:30.929945 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:30.972190 12527 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:30.903115 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:30.917773 14924 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:30.859320 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:30.957269 12602 rpcz_store.cc:276] 0709 12:10:29.320320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.955716 (+1635396us) service_pool.cc:224] Handling call
0709 12:10:30.956823 (+  1107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.590318 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:31.590929 12602 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:10:30.948195 12597 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:31.595685 14449 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:31.596143 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:31.596294 14449 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:31.597350 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:31.598645 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:31.599427 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1706) took 1326 ms. Trace:
I20260709 12:10:31.599763 12604 rpcz_store.cc:276] 0709 12:10:30.272749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.598412 (+1325663us) service_pool.cc:224] Handling call
0709 12:10:31.599369 (+   957us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.600597 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:31.601344 12604 raft_consensus.cc:2410] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 4 because replica has last-logged OpId of term: 2 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:10:30.934847 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:30.984411 12672 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 2.
I20260709 12:10:30.894618 12674 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:10:30.973564 12680 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:30.978070 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:30.942325 12679 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:31.701889 14741 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:31.758510 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1725) took 1213 ms. Trace:
I20260709 12:10:31.759122 12679 rpcz_store.cc:276] 0709 12:10:30.545176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.929853 (+384677us) service_pool.cc:224] Handling call
0709 12:10:31.758443 (+828590us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.760713 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:30.960906 14502 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:30.900461 14454 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:30.902308 14819 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:31.770514 14819 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:31.772186 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1716) took 1270 ms. Trace:
I20260709 12:10:31.772725 12674 rpcz_store.cc:276] 0709 12:10:30.501604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.503806 (+  2202us) service_pool.cc:224] Handling call
0709 12:10:30.522666 (+ 18860us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:31.753110 (+1230444us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.771888 (+ 18778us) spinlock_profiling.cc:243] Waited 372 ms on lock 0x7b54000933d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:31.772117 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":854496640}
W20260709 12:10:30.901630 14804 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.345s	user 0.011s	sys 0.039s
W20260709 12:10:31.777238 14804 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.220s	user 0.017s	sys 0.041s
I20260709 12:10:30.862087 14852 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:31.784698 14852 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:30.983578 14891 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:30.935832 14892 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.829s	user 0.015s	sys 0.032s
W20260709 12:10:30.968098 14788 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.394s	user 0.010s	sys 0.013s
I20260709 12:10:30.962610 14738 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:30.852782 14904 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:30.889273 14906 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:31.038861 14920 log.cc:927] Time spent T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.146s	user 0.001s	sys 0.001s
I20260709 12:10:31.069392 14925 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:31.249032 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:31.802995 12676 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:31.803560 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1718) took 1292 ms. Trace:
I20260709 12:10:31.803872 12676 rpcz_store.cc:276] 0709 12:10:30.510719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.248667 (+737948us) service_pool.cc:224] Handling call
0709 12:10:31.803488 (+554821us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.804267 14925 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:10:31.804917 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:31.805533 12676 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:31.805977 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1732) took 1050 ms. Trace:
I20260709 12:10:31.806336 12676 rpcz_store.cc:276] 0709 12:10:30.755334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.804616 (+1049282us) service_pool.cc:224] Handling call
0709 12:10:31.805927 (+  1311us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.806864 14906 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:31.807585 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:31.808386 12676 raft_consensus.cc:2410] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 4 because replica has last-logged OpId of term: 2 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260709 12:10:30.985143 12605 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
W20260709 12:10:31.329088 14932 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.330s	user 0.002s	sys 0.004s
I20260709 12:10:31.343209 14864 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:31.809583 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:31.385550 14789 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:31.810858 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:31.811978 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:31.813319 12605 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: [4.7-5.8]
W20260709 12:10:31.391310 13334 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 398) took 2631 ms (client timeout 1819 ms). Trace:
W20260709 12:10:31.814311 13334 rpcz_store.cc:269] 0709 12:10:28.760402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:28.762963 (+  2561us) service_pool.cc:224] Handling call
0709 12:10:29.946733 (+1183770us) spinlock_profiling.cc:243] Waited 34.7 ms on lock 0x7b5800163298. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:10:31.390985 (+1444252us) spinlock_profiling.cc:243] Waited 89.9 ms on lock 0x7b540055c8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:31.391180 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":286681344}
I20260709 12:10:31.814919 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1733) took 1059 ms. Trace:
W20260709 12:10:31.413439 14292 consensus_meta.cc:287] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.587s	user 0.014s	sys 0.003s
I20260709 12:10:31.815294 12676 rpcz_store.cc:276] 0709 12:10:30.764655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.807315 (+1042660us) service_pool.cc:224] Handling call
0709 12:10:31.814856 (+  7541us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.815697 14292 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:31.816650 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:31.817360 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1731) took 1062 ms. Trace:
I20260709 12:10:31.813891 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1656) took 2673 ms. Trace:
I20260709 12:10:31.414425 12606 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
W20260709 12:10:31.818455 12606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1711) took 1525 ms (client timeout 1865 ms). Trace:
W20260709 12:10:31.818874 12606 rpcz_store.cc:269] 0709 12:10:30.293226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.858481 (+565255us) service_pool.cc:224] Handling call
0709 12:10:31.818397 (+959916us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.818027 12605 rpcz_store.cc:276] 0709 12:10:29.140853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.812780 (+2671927us) service_pool.cc:224] Handling call
0709 12:10:31.813090 (+   310us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:31.813644 (+   554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.813841 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.820125 12606 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:31.820540 12605 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 2. Current term is 3. Ops: []
I20260709 12:10:31.820700 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1661) took 2678 ms. Trace:
W20260709 12:10:31.437006 14851 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 2.961s	user 0.000s	sys 0.025s
I20260709 12:10:31.821136 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1663) took 2678 ms. Trace:
I20260709 12:10:31.440641 14926 raft_consensus.cc:3055] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:10:31.453611 14886 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.343s	user 0.000s	sys 0.012s
I20260709 12:10:31.469192 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1702) took 2472 ms. Trace:
I20260709 12:10:31.818962 14454 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:31.822441 12601 rpcz_store.cc:276] 0709 12:10:28.997237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:28.997414 (+   177us) service_pool.cc:224] Handling call
0709 12:10:29.039461 (+ 42047us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:29.169295 (+129834us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:29.169329 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:29.169921 (+   592us) log.cc:844] Serialized 26 byte log entry
0709 12:10:29.689401 (+519480us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:29.689443 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.468232 (+1778789us) spinlock_profiling.cc:243] Waited 129 ms on lock 0x7b5400115dd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:31.468315 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:31.468570 (+   255us) raft_consensus.cc:1685] finished
0709 12:10:31.468592 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:31.469102 (+   510us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33265,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"spinlock_wait_cycles":297593600,"thread_start_us":519048,"threads_started":1,"wal-append.queue_time_us":761832,"wal-append.run_cpu_time_us":20145,"wal-append.run_wall_time_us":1618925}
I20260709 12:10:31.821543 12605 rpcz_store.cc:276] 0709 12:10:29.143225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.820103 (+2676878us) service_pool.cc:224] Handling call
0709 12:10:31.820392 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:31.820930 (+   538us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.821089 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.471969 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1607) took 1060 ms. Trace:
I20260709 12:10:31.825134 12527 rpcz_store.cc:276] 0709 12:10:30.411866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.796834 (+384968us) service_pool.cc:224] Handling call
0709 12:10:31.471697 (+674863us) spinlock_profiling.cc:243] Waited 66.7 ms on lock 0x7b54003c1754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:31.471885 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":153351936}
I20260709 12:10:31.474426 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1572) took 1124 ms. Trace:
I20260709 12:10:31.827200 12529 rpcz_store.cc:276] 0709 12:10:30.353473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.374601 (+ 21128us) service_pool.cc:224] Handling call
0709 12:10:30.375041 (+   440us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:30.794268 (+419227us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:10:30.794301 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:30.794353 (+    52us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:30.794397 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.473246 (+678849us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b18000620e0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419
0709 12:10:31.474067 (+   821us) spinlock_profiling.cc:243] Waited 389 ms on lock 0x7b540027fbd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:31.474162 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:31.474346 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19760,"raft.queue_time_us":19947,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":14819,"spinlock_wait_cycles":902210688}
W20260709 12:10:31.526947 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:31.535984 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:31.834079 12605 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:31.834870 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1671) took 2530 ms. Trace:
I20260709 12:10:31.835183 12605 rpcz_store.cc:276] 0709 12:10:29.304337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.825122 (+2520785us) service_pool.cc:224] Handling call
0709 12:10:31.833753 (+  8631us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:31.834601 (+   848us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.834809 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.836460 12603 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:31.537408 14749 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:31.538393 12530 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 1566) took 2822 ms (client timeout 3147 ms). Trace:
I20260709 12:10:31.538990 14905 raft_consensus.cc:3055] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:31.539218 14822 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:31.541663 14879 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:31.544747 14902 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.965s	user 0.004s	sys 0.001s
I20260709 12:10:31.548044 14817 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:31.853401 12605 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:31.854185 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1703) took 2473 ms. Trace:
I20260709 12:10:31.854550 12605 rpcz_store.cc:276] 0709 12:10:29.380905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.843878 (+2462973us) service_pool.cc:224] Handling call
0709 12:10:31.853066 (+  9188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:31.853907 (+   841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.854109 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:31.549736 14899 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.970s	user 0.001s	sys 0.004s
I20260709 12:10:30.943527 12643 leader_election.cc:400] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 3. Current term is 4.
I20260709 12:10:31.567616 14486 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:31.570493 14811 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:31.575752 12677 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:31.577893 14560 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:31.580894 14777 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:31.584708 13302 leader_election.cc:336] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.564s (SENT)
I20260709 12:10:31.878229 13302 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:31.869243 14788 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:31.587272 12525 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:10:31.879208 14486 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:31.879595 14788 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:31.880069 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 146) took 1473 ms. Trace:
I20260709 12:10:31.880577 12525 rpcz_store.cc:276] 0709 12:10:30.406765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.870996 (+464231us) service_pool.cc:224] Handling call
0709 12:10:31.879986 (+1008990us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.881800 14486 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:31.588519 12526 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:31.882472 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1611) took 1373 ms. Trace:
I20260709 12:10:31.882846 12526 rpcz_store.cc:276] 0709 12:10:30.509340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.826033 (+316693us) service_pool.cc:224] Handling call
0709 12:10:31.882407 (+1056374us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.879232 14811 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:31.609299 14901 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:31.884123 14733 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:31.885160 14811 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:31.615499 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:31.886173 12522 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 5: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:31.886796 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 1573) took 1511 ms. Trace:
I20260709 12:10:31.887236 12522 rpcz_store.cc:276] 0709 12:10:30.375653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.614847 (+1239194us) service_pool.cc:224] Handling call
0709 12:10:31.886711 (+271864us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.591904 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:31.618185 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1567) took 1296 ms. Trace:
W20260709 12:10:31.618707 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:31.888712 12528 rpcz_store.cc:276] 0709 12:10:30.322205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.322386 (+   181us) service_pool.cc:224] Handling call
0709 12:10:30.322810 (+   424us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:30.763518 (+440708us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.617865 (+854347us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b540038ead4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:31.618087 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1011516672}
I20260709 12:10:31.598658 14449 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:31.853504 14822 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:31.636996 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 1584) took 1244 ms. Trace:
W20260709 12:10:31.647240 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:31.649842 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:31.891198 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:10:31.658310 14853 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 3.181s	user 0.006s	sys 0.023s
I20260709 12:10:31.890604 12523 rpcz_store.cc:276] 0709 12:10:30.392382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.454959 (+ 62577us) service_pool.cc:224] Handling call
0709 12:10:31.636768 (+1181809us) spinlock_profiling.cc:243] Waited 30.3 ms on lock 0x7b5400054454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:31.636919 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69684608}
I20260709 12:10:31.896767 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:31.897712 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1613) took 1358 ms. Trace:
I20260709 12:10:31.898674 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1543) took 2226 ms. Trace:
I20260709 12:10:31.896692 14822 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:31.900117 14733 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:31.658677 14796 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:31.900847 14733 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:31.901387 14733 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:31.901273 14796 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:31.904286 14796 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:30.936429 14598 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:31.907105 14598 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:31.667677 12599 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5. Ops: [3.5-4.6]
I20260709 12:10:31.908207 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1655) took 2767 ms. Trace:
I20260709 12:10:31.908641 12599 rpcz_store.cc:276] 0709 12:10:29.140561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.666953 (+2526392us) service_pool.cc:224] Handling call
0709 12:10:31.667368 (+   415us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:31.907841 (+240473us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.908126 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.681918 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:31.910531 12672 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:31.911288 12523 rpcz_store.cc:276] 0709 12:10:30.539371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.896438 (+1357067us) service_pool.cc:224] Handling call
0709 12:10:31.897639 (+  1201us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.912338 12672 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:31.912492 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:31.913364 12523 raft_consensus.cc:2368] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6.
W20260709 12:10:31.697419 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 897170us
I20260709 12:10:31.704455 14939 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:10:31.713677 14896 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.539s	user 0.004s	sys 0.004s
I20260709 12:10:31.719038 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1719) took 1205 ms. Trace:
I20260709 12:10:31.913211 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1720) took 1395 ms. Trace:
I20260709 12:10:31.915184 12671 rpcz_store.cc:276] 0709 12:10:30.513329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.523400 (+ 10071us) service_pool.cc:224] Handling call
0709 12:10:31.718804 (+1195404us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54002c4f54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:31.718959 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":233815296}
W20260709 12:10:31.724246 14895 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.628s	user 0.004s	sys 0.008s
I20260709 12:10:31.916157 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1709) took 1623 ms. Trace:
I20260709 12:10:31.727218 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1535) took 2679 ms. Trace:
I20260709 12:10:31.923553 12602 rpcz_store.cc:276] 0709 12:10:30.292571 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.591633 (+1299062us) service_pool.cc:224] Handling call
0709 12:10:31.916083 (+324450us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.924031 12678 rpcz_store.cc:276] 0709 12:10:29.047313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:29.275365 (+228052us) service_pool.cc:224] Handling call
0709 12:10:31.726847 (+2451482us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b5400092ed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:31.727111 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1064274176}
I20260709 12:10:31.925830 12678 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:31.926506 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1730) took 1171 ms. Trace:
I20260709 12:10:31.926667 12671 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:31.926887 12678 rpcz_store.cc:276] 0709 12:10:30.754919 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.925119 (+1170200us) service_pool.cc:224] Handling call
0709 12:10:31.925547 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:31.926264 (+   717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.926454 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.927260 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1554) took 1166 ms. Trace:
W20260709 12:10:31.728690 14913 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.147s	user 0.004s	sys 0.001s
I20260709 12:10:31.733217 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:31.737639 14512 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:31.739920 14801 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:31.745922 14931 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.763s	user 0.004s	sys 0.000s
I20260709 12:10:31.936813 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1709) took 2496 ms. Trace:
W20260709 12:10:31.748498 14843 consensus_meta.cc:287] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.827s	user 0.014s	sys 0.000s
I20260709 12:10:31.693912 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:31.757617 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1722) took 1229 ms. Trace:
I20260709 12:10:31.938935 12680 rpcz_store.cc:276] 0709 12:10:30.528458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.723209 (+194751us) service_pool.cc:224] Handling call
0709 12:10:31.757443 (+1034234us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54000c26d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:31.757556 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":376286592}
I20260709 12:10:31.759043 14741 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:31.881806 14788 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:31.942626 14741 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:31.760980 12429 catalog_manager.cc:5697] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 2 to 5, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 5 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:31.943336 14788 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:31.943987 14788 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:31.945402 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:31.944824 12429 catalog_manager.cc:5697] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 1 to 2, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 2 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:10:31.947023 12429 catalog_manager.cc:5697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:10:31.948501 12429 catalog_manager.cc:5697] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 4 to 5, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 5 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:10:30.833808 14739 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:31.776225 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:31.778028 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:31.960707 13337 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:31.778541 12597 raft_consensus.cc:2410] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 4 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
W20260709 12:10:31.942687 14946 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:10:31.962581 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1683) took 2640 ms. Trace:
I20260709 12:10:31.781935 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:31.786854 14891 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:31.963681 12679 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:10:31.963284 12429 catalog_manager.cc:5697] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 1 to 4, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 4 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:10:31.790385 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:31.966566 12674 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:31.790977 14892 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.684s	user 0.018s	sys 0.032s
I20260709 12:10:31.788074 14852 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:31.799292 14904 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:31.440038 14678 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:31.821030 12606 rpcz_store.cc:276] 0709 12:10:29.142489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.819652 (+2677163us) service_pool.cc:224] Handling call
0709 12:10:31.819948 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:31.820472 (+   524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.820646 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.829947 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:31.817668 12676 rpcz_store.cc:276] 0709 12:10:30.755161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.816382 (+1061221us) service_pool.cc:224] Handling call
0709 12:10:31.817304 (+   922us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.821781 14926 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:10:31.971124 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 133452us
I20260709 12:10:31.843080 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1700) took 2504 ms. Trace:
W20260709 12:10:31.844638 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 64478us
I20260709 12:10:31.845837 12601 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:10:31.971952 12601 pending_rounds.cc:85] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 2
I20260709 12:10:31.972287 12601 pending_rounds.cc:107] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:10:31.972535 12601 raft_consensus.cc:2889] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:31.973464 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1677) took 2653 ms. Trace:
I20260709 12:10:31.817869 14292 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:31.852739 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:31.961685 14733 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:31.835918 14454 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:31.844033 14749 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:31.840579 14905 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:31.983034 14901 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:31.983830 14901 consensus_queue.cc:1243] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:10:31.984258 14905 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:31.985347 14905 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:31.986394 14749 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:31.987002 14749 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:31.987491 14749 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:31.987052 14926 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:31.876690 12643 leader_election.cc:403] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:10:31.988292 14926 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:31.988853 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 1.0501e+06us
I20260709 12:10:31.989732 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:31.587380 14924 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:31.602073 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:31.613075 14829 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:31.656736 12600 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 12:10:31.891812 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:31.993577 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1654) took 2853 ms. Trace:
I20260709 12:10:31.993989 12600 rpcz_store.cc:276] 0709 12:10:29.140144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.656004 (+2515860us) service_pool.cc:224] Handling call
0709 12:10:31.656448 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:31.993252 (+336804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.993515 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.995645 12600 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:10:31.996336 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 124) took 2507 ms. Trace:
I20260709 12:10:31.996645 12600 rpcz_store.cc:276] 0709 12:10:29.488643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.994837 (+2506194us) service_pool.cc:224] Handling call
0709 12:10:31.995361 (+   524us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:31.996076 (+   715us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.996278 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.997144 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:31.893970 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:31.914518 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:31.997648 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:31.891173 12646 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:31.915570 12672 rpcz_store.cc:276] 0709 12:10:30.517497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.911462 (+1393965us) service_pool.cc:224] Handling call
0709 12:10:31.911833 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:31.912900 (+  1067us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.913144 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.998735 12606 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:10:31.999197 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:31.999891 12600 raft_consensus.cc:2376] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6. Re-sending same reply.
I20260709 12:10:32.000218 12643 leader_election.cc:400] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6.
W20260709 12:10:31.879508 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 294822us
I20260709 12:10:31.998004 12674 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:32.000988 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:31.916406 12496 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:32.001376 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1557) took 1083 ms. Trace:
I20260709 12:10:31.927572 12671 rpcz_store.cc:276] 0709 12:10:30.761078 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.926235 (+1165157us) service_pool.cc:224] Handling call
0709 12:10:31.926501 (+   266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:31.927036 (+   535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.927211 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:32.001791 12674 rpcz_store.cc:276] 0709 12:10:30.917678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.773948 (+856270us) service_pool.cc:224] Handling call
0709 12:10:32.001316 (+227368us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.928198 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.002830 12643 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:31.930730 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.008322 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:32.009166 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 118110us
I20260709 12:10:32.009429 12528 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 5: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:31.925140 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.010102 12646 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:32.010469 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 145) took 1604 ms. Trace:
I20260709 12:10:32.011020 12522 rpcz_store.cc:276] 0709 12:10:30.406410 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.924770 (+1518360us) service_pool.cc:224] Handling call
0709 12:10:32.010377 (+ 85607us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:32.012396 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.008778 12604 raft_consensus.cc:2410] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:10:31.899317 12675 rpcz_store.cc:276] 0709 12:10:29.672553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:29.674440 (+  1887us) service_pool.cc:224] Handling call
0709 12:10:29.674768 (+   328us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:29.682329 (+  7561us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:29.682363 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:29.691713 (+  9350us) log.cc:844] Serialized 52 byte log entry
0709 12:10:30.056881 (+365168us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:30.343083 (+286202us) log.cc:844] Serialized 14 byte log entry
0709 12:10:30.343284 (+   201us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:30.343363 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:31.544131 (+1200768us) spinlock_profiling.cc:243] Waited 1.2 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:31.898205 (+354074us) raft_consensus.cc:1685] finished
0709 12:10:31.898374 (+   169us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:31.898592 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3463,"prepare.queue_time_us":285975,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":442,"raft.queue_time_us":3648,"raft.run_cpu_time_us":747,"raft.run_wall_time_us":82400,"spinlock_wait_cycles":2753920,"thread_start_us":294675,"threads_started":1,"wal-append.queue_time_us":777258}
I20260709 12:10:32.014611 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1710) took 1721 ms. Trace:
I20260709 12:10:32.015166 12604 rpcz_store.cc:276] 0709 12:10:30.292902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.601866 (+1308964us) service_pool.cc:224] Handling call
0709 12:10:32.014507 (+412641us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:32.015547 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:32.017191 12675 raft_consensus.cc:2410] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 7 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:10:31.914060 14939 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:31.936028 14801 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:31.937367 12602 rpcz_store.cc:276] 0709 12:10:29.440139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.924297 (+2484158us) service_pool.cc:224] Handling call
0709 12:10:31.936133 (+ 11836us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:31.936449 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:31.936476 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:31.936540 (+    64us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:31.936571 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.936626 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:31.936755 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1533,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":114}
I20260709 12:10:31.978582 14741 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:31.951169 14512 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:31.935421 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1552) took 1175 ms. Trace:
I20260709 12:10:32.028223 12673 rpcz_store.cc:276] 0709 12:10:30.760422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.732890 (+972468us) service_pool.cc:224] Handling call
0709 12:10:31.935359 (+202469us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.938953 12524 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:32.029287 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1609) took 1598 ms. Trace:
I20260709 12:10:31.953575 12680 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:31.957062 14739 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:31.963020 12597 rpcz_store.cc:276] 0709 12:10:29.326207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.597105 (+2270898us) service_pool.cc:224] Handling call
0709 12:10:31.962369 (+365264us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b54002ff654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:31.962515 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":414741888}
I20260709 12:10:31.965410 12429 catalog_manager.cc:5697] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:10:31.838716 12530 rpcz_store.cc:269] 0709 12:10:28.716270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:28.742110 (+ 25840us) service_pool.cc:224] Handling call
0709 12:10:31.533812 (+2791702us) spinlock_profiling.cc:243] Waited 41.3 ms on lock 0x7b5400407754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:31.538268 (+  4456us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95036672}
I20260709 12:10:31.973795 12601 rpcz_store.cc:276] 0709 12:10:29.319960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.833697 (+2513737us) service_pool.cc:224] Handling call
0709 12:10:31.845340 (+ 11643us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:31.973187 (+127847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.973410 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.974563 12527 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:31.857214 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1704) took 2476 ms. Trace:
I20260709 12:10:31.971771 12603 rpcz_store.cc:276] 0709 12:10:29.339122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.973450 (+1634328us) service_pool.cc:224] Handling call
0709 12:10:31.842848 (+869398us) spinlock_profiling.cc:243] Waited 53.9 ms on lock 0x7b54001176d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:31.842994 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":123911552}
I20260709 12:10:31.979205 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:31.981624 14486 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:10:31.988413 14749 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:31.892349 12531 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 4: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:31.998026 12525 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:32.000042 12672 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:31.920526 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1708) took 2538 ms. Trace:
I20260709 12:10:32.001807 12600 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 4: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:32.003664 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:31.929528 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1553) took 1168 ms. Trace:
W20260709 12:10:32.001976 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 87930us
I20260709 12:10:32.002995 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:32.038728 12674 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:32.021010 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:32.021370 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.040329 12528 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 3.
I20260709 12:10:32.008458 14454 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.021395 12571 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:32.022961 12604 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:10:32.025672 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1712) took 2566 ms. Trace:
I20260709 12:10:32.014212 14885 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 3. Current term is 4.
I20260709 12:10:32.025991 12646 leader_election.cc:400] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6.
I20260709 12:10:32.049526 12646 leader_election.cc:403] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:10:32.039407 12496 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:32.028358 14889 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:32.030857 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1560) took 1087 ms. Trace:
I20260709 12:10:32.054586 12680 rpcz_store.cc:276] 0709 12:10:30.943438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.939924 (+996486us) service_pool.cc:224] Handling call
0709 12:10:31.953254 (+ 13330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:32.030416 (+ 77162us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.030771 (+   355us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1920}
I20260709 12:10:32.032596 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.029762 12524 rpcz_store.cc:276] 0709 12:10:30.430504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.693461 (+1262957us) service_pool.cc:224] Handling call
0709 12:10:32.029222 (+335761us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:31.978523 14678 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:31.990846 14777 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.057828 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:32.040956 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1612) took 1525 ms. Trace:
I20260709 12:10:32.013559 12494 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:32.039968 12671 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:32.059713 12528 rpcz_store.cc:276] 0709 12:10:30.515593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.021144 (+1505551us) service_pool.cc:224] Handling call
0709 12:10:32.040897 (+ 19753us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:32.039939 12677 rpcz_store.cc:276] 0709 12:10:30.760812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.821337 (+ 60525us) service_pool.cc:224] Handling call
0709 12:10:31.929301 (+1107964us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b54003be054. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:31.929445 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1566988672}
I20260709 12:10:32.038295 12599 rpcz_store.cc:276] 0709 12:10:29.382206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.909652 (+2527446us) service_pool.cc:224] Handling call
0709 12:10:31.919740 (+ 10088us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:31.920019 (+   279us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:31.920063 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:31.920131 (+    68us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:31.920182 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.920263 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:31.920437 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:32.061218 13337 raft_consensus.cc:2410] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:10:32.061296 12599 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:32.062041 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1720) took 1214 ms. Trace:
I20260709 12:10:32.062372 12599 rpcz_store.cc:276] 0709 12:10:30.847393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.060798 (+1213405us) service_pool.cc:224] Handling call
0709 12:10:32.061123 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:32.061802 (+   679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.061977 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:32.014539 14783 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:32.044651 12606 rpcz_store.cc:276] 0709 12:10:29.459268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.971053 (+2511785us) service_pool.cc:224] Handling call
0709 12:10:31.998401 (+ 27348us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:31.999138 (+   737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.025383 (+ 26245us) spinlock_profiling.cc:243] Waited 7.64 ms on lock 0x7b54002e6dd0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:32.025585 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":17563904}
I20260709 12:10:31.938310 12598 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
W20260709 12:10:32.066006 12598 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1712) took 1772 ms (client timeout 1706 ms). Trace:
I20260709 12:10:32.041651 12605 rpcz_store.cc:276] 0709 12:10:29.381220 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.855369 (+2474149us) service_pool.cc:224] Handling call
0709 12:10:31.855689 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:31.856871 (+  1182us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 12:10:31.856903 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:31.856949 (+    46us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:31.856980 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:31.857039 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:31.857157 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":858,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":5619}
W20260709 12:10:32.066731 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 53292.5us
W20260709 12:10:32.067140 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 45203.5us
I20260709 12:10:32.067524 12605 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:32.021590 14292 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:32.024724 14560 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:31.969817 14852 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:32.008543 14412 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:32.029570 14240 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:10:32.048079 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:32.048458 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 125) took 2559 ms. Trace:
I20260709 12:10:32.084841 12604 rpcz_store.cc:276] 0709 12:10:29.489042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.016273 (+2527231us) service_pool.cc:224] Handling call
0709 12:10:32.016739 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:32.048169 (+ 31430us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.048399 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:32.044032 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1714) took 2115 ms. Trace:
I20260709 12:10:32.050733 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:32.086192 13334 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:32.051604 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 1575) took 1673 ms. Trace:
I20260709 12:10:32.087028 12531 rpcz_store.cc:276] 0709 12:10:30.378510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.649510 (+1271000us) service_pool.cc:224] Handling call
0709 12:10:32.051540 (+402030us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:32.051864 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:32.088068 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.028606 14746 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:32.085906 12600 rpcz_store.cc:276] 0709 12:10:29.929044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.043225 (+2114181us) service_pool.cc:224] Handling call
0709 12:10:32.043528 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:32.043717 (+   189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:32.043738 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:32.043778 (+    40us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:32.043806 (+    28us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.043856 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:32.043952 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:32.027246 14857 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:10:32.051348 12496 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:10:32.095002 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 43738us
I20260709 12:10:32.058072 12602 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:32.095795 12496 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:32.096143 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 127) took 2606 ms. Trace:
I20260709 12:10:32.058247 14449 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:32.096720 12602 rpcz_store.cc:276] 0709 12:10:29.489815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.037244 (+2547429us) service_pool.cc:224] Handling call
0709 12:10:32.057877 (+ 20633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:32.095752 (+ 37875us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.096064 (+   312us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:32.057776 12429 catalog_manager.cc:5697] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:10:32.098219 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:32.058820 12672 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:10:32.098914 13332 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:32.028510 14788 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:32.035575 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:32.100409 12676 raft_consensus.cc:2410] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 4 because replica has last-logged OpId of term: 2 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 2.
W20260709 12:10:32.066499 12598 rpcz_store.cc:269] 0709 12:10:30.293593 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:30.722885 (+429292us) service_pool.cc:224] Handling call
0709 12:10:32.065787 (+1342902us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54002f1554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:32.065923 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2625017088}
I20260709 12:10:32.076246 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.079682 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.101920 12530 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:32.083280 13302 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:32.027107 14925 consensus_queue.cc:1059] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260709 12:10:32.088621 13338 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:32.087021 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.052946 14749 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:32.090895 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.050348 14948 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.028913 12643 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:32.057922 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:32.072075 14877 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.111491 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:32.085879 14691 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:10:32.114351 12496 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:32.080138 14486 consensus_queue.cc:1243] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:10:32.123270 12525 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 5: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:32.072275 14292 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:32.148627 14925 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:32.148910 14292 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:32.149250 14925 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:32.150223 14486 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:32.150679 14486 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:32.131053 12671 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:32.080135 14523 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:32.150911 12527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:10:32.151783 14523 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:32.151973 12527 raft_consensus.cc:606] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:32.093318 14412 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:32.121162 12429 catalog_manager.cc:5697] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 2 to 4, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 4 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:32.152443 12527 raft_consensus.cc:614] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:32.138696 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.080845 14560 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:32.051252 14454 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:32.153010 14486 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:32.153532 14560 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:32.154500 14486 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:32.112046 14449 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:32.155864 14449 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:32.156287 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:32.076728 14777 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:32.157094 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1574) took 1779 ms. Trace:
I20260709 12:10:32.157203 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1558) took 1228 ms. Trace:
I20260709 12:10:32.157817 12678 rpcz_store.cc:276] 0709 12:10:30.928562 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.934373 (+1005811us) service_pool.cc:224] Handling call
0709 12:10:31.953195 (+ 18822us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:31.953568 (+   373us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:31.953598 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:31.954065 (+   467us) log.cc:844] Serialized 26 byte log entry
0709 12:10:31.974793 (+ 20728us) spinlock_profiling.cc:243] Waited 4.01 ms on lock 0x7b180039b980. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:10:32.030164 (+ 55371us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:32.030228 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.030516 (+   288us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:32.156863 (+126347us) raft_consensus.cc:1685] finished
0709 12:10:32.156887 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:32.157131 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":467,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"spinlock_wait_cycles":9213312,"thread_start_us":55134,"threads_started":1,"wal-append.queue_time_us":138948}
I20260709 12:10:32.159854 14901 raft_consensus.cc:3055] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:32.160341 14901 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:32.161208 14901 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:32.140705 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:32.093813 14746 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:32.162495 14746 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:32.125466 14598 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:32.163928 14598 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:32.164402 14598 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:32.146540 14804 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:10:32.146951 12460 debug-util.cc:398] Leaking SignalData structure 0x7b08002ae600 after lost signal to thread 11321
I20260709 12:10:32.165913 14804 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:32.166510 14804 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:32.166793 12460 debug-util.cc:398] Leaking SignalData structure 0x7b080022a4c0 after lost signal to thread 12463
I20260709 12:10:32.153735 12530 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:32.168273 12524 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:10:32.169047 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1578) took 1789 ms. Trace:
I20260709 12:10:32.171669 12527 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:32.148309 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.142699 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.036849 14741 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:32.093585 14819 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.147819 14857 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:32.149479 14678 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:32.157613 12528 rpcz_store.cc:276] 0709 12:10:30.378106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.156145 (+1778039us) service_pool.cc:224] Handling call
0709 12:10:32.156504 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:32.156701 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:32.156735 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:32.156780 (+    45us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:32.156813 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.156877 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:32.157008 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:32.169448 12524 rpcz_store.cc:276] 0709 12:10:30.390215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.167495 (+1777280us) service_pool.cc:224] Handling call
0709 12:10:32.167921 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:32.168705 (+   784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.168966 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:32.133858 14817 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:32.134470 14538 tablet_bootstrap.cc:492] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:10:32.202555 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1606) took 1791 ms. Trace:
I20260709 12:10:32.276885 12528 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:10:32.276332 12525 rpcz_store.cc:276] 0709 12:10:30.411558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.147997 (+1736439us) service_pool.cc:224] Handling call
0709 12:10:32.202474 (+ 54477us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:32.204882 12531 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:10:32.206442 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:32.233690 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 93092.1us
I20260709 12:10:32.280138 12496 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:32.233649 12460 debug-util.cc:398] Leaking SignalData structure 0x7b08002c4280 after lost signal to thread 12555
W20260709 12:10:32.245282 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:32.281826 12496 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:10:32.282383 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 199238us
I20260709 12:10:32.257872 12599 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:32.292363 12599 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:10:32.260977 13304 leader_election.cc:336] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.531s (SENT)
I20260709 12:10:32.220991 14822 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:32.220320 14843 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.254083 14788 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:32.277853 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1583) took 1885 ms. Trace:
I20260709 12:10:32.237161 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1579) took 1786 ms. Trace:
I20260709 12:10:32.283664 12524 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:10:32.304091 12528 rpcz_store.cc:276] 0709 12:10:30.392149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.276252 (+1884103us) service_pool.cc:224] Handling call
0709 12:10:32.276645 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:32.277470 (+   825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.277772 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:32.304806 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1585) took 1912 ms. Trace:
I20260709 12:10:32.305192 12524 rpcz_store.cc:276] 0709 12:10:30.392587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.278328 (+1885741us) service_pool.cc:224] Handling call
0709 12:10:32.283340 (+  5012us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:32.304491 (+ 21151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.304741 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:32.285897 12522 raft_consensus.cc:2410] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:32.293794 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:32.283262 12460 debug-util.cc:398] Leaking SignalData structure 0x7b080020dca0 after lost signal to thread 12630
I20260709 12:10:32.306998 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:32.307610 12460 debug-util.cc:398] Leaking SignalData structure 0x7b080034c1a0 after lost signal to thread 12704
W20260709 12:10:32.308090 12460 debug-util.cc:398] Leaking SignalData structure 0x7b0800394620 after lost signal to thread 12705
I20260709 12:10:32.283996 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1581) took 1892 ms. Trace:
I20260709 12:10:32.308524 12531 rpcz_store.cc:276] 0709 12:10:30.391320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.173214 (+1781894us) service_pool.cc:224] Handling call
0709 12:10:32.173617 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:32.283652 (+110035us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.283923 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:32.276216 14901 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:32.298548 12599 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:32.145241 14877 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:32.296568 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 267834us
I20260709 12:10:32.296645 13302 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:32.231936 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 210673us
I20260709 12:10:32.316202 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:32.320022 12599 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:32.320894 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:32.271705 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 249171us
I20260709 12:10:32.302603 12496 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:32.246969 14857 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:32.324030 14857 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:32.307703 12529 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
I20260709 12:10:32.327219 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:32.327998 12529 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:10:32.233500 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 166025us
I20260709 12:10:32.292578 14892 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.006s
I20260709 12:10:32.278395 14817 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:32.335601 14817 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:32.308046 12527 rpcz_store.cc:276] 0709 12:10:30.390837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.170943 (+1780106us) service_pool.cc:224] Handling call
0709 12:10:32.171397 (+   454us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:32.177064 (+  5667us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.177389 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:32.153568 12429 catalog_manager.cc:5697] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 1 to 2, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 2 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
W20260709 12:10:32.310557 12460 debug-util.cc:398] Leaking SignalData structure 0x7b080032fbc0 after lost signal to thread 13363
W20260709 12:10:32.336621 12460 debug-util.cc:398] Leaking SignalData structure 0x7b08002c7300 after lost signal to thread 13364
I20260709 12:10:32.336791 12527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:10:32.337445 12527 raft_consensus.cc:606] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:32.337114 12429 catalog_manager.cc:5697] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 2 to 3, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 3 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
W20260709 12:10:32.296191 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:32.341607 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:32.322849 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:32.339042 12429 catalog_manager.cc:5697] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 4 to 5, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 5 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:32.345021 12429 catalog_manager.cc:5697] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 1 to 2, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 2 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:10:32.347512 12429 catalog_manager.cc:5697] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 3 to 4, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 4 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:32.320713 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 149) took 1489 ms. Trace:
I20260709 12:10:32.350592 12530 rpcz_store.cc:276] 0709 12:10:30.831568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.306097 (+1474529us) service_pool.cc:224] Handling call
0709 12:10:32.320227 (+ 14130us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:32.320408 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:32.320433 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:32.320476 (+    43us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:32.320503 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.320554 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:32.320653 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:32.351246 12524 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:32.352268 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 148) took 1835 ms. Trace:
I20260709 12:10:32.352690 12524 rpcz_store.cc:276] 0709 12:10:30.516510 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.306024 (+1789514us) service_pool.cc:224] Handling call
0709 12:10:32.351043 (+ 45019us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:32.351895 (+   852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.352196 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:32.355189 12525 raft_consensus.cc:1217] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[2.5-4.6]   Dedup: 2.5->[4.6-4.6]
I20260709 12:10:32.355816 12525 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 4. Current term is 5. Ops: [2.5-4.6]
W20260709 12:10:32.356261 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 53750.6us
I20260709 12:10:32.357664 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:10:32.358392 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:32.349620 12429 catalog_manager.cc:5697] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:32.144455 14948 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:32.301329 14796 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:32.253439 14819 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:32.317111 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:32.288130 14889 raft_consensus.cc:697] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:32.364461 12646 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:32.344851 14857 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:32.364737 12530 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:10:32.303226 14738 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:32.366587 12569 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:32.376087 12572 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:32.389360 12569 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:32.391005 12530 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:32.313408 14864 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.391665 12529 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:10:32.309746 14843 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:32.312384 14822 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:32.396924 12599 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:32.398205 12646 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:10:32.405802 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:32.405962 12429 catalog_manager.cc:5697] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:10:32.367676 14892 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:32.385963 14538 tablet_bootstrap.cc:492] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645: Bootstrap complete.
W20260709 12:10:32.379400 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:32.415748 12530 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:32.376463 14502 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:32.385826 14926 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:32.367055 14796 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:32.417932 12646 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:10:32.419875 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:32.456902 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:32.457273 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:32.413039 12525 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:10:32.366042 14789 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:32.387641 14412 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:32.377539 14954 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.072s
I20260709 12:10:32.433486 12524 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:32.379295 14952 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:32.376945 14924 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:32.354007 14523 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:32.365732 14829 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260709 12:10:32.447444 14733 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:32.459322 12525 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:10:32.459457 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 137411us
I20260709 12:10:32.416602 14783 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:32.414274 14962 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:32.412290 14512 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:32.410807 12429 catalog_manager.cc:5697] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 1 to 3, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 3 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:10:32.411911 14925 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:32.411847 14801 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:32.458215 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 68946.8us
I20260709 12:10:32.444681 14502 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:32.472692 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 54924us
W20260709 12:10:32.473093 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:32.408893 14864 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:32.460999 14968 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:32.495333 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:32.453687 14926 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:32.463079 14796 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:32.486305 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 144) took 2080 ms. Trace:
I20260709 12:10:32.455010 14777 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:32.455089 14939 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:32.523152 14240 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:10:32.524063 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 148093us
I20260709 12:10:32.516852 14454 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:32.435950 14738 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
W20260709 12:10:32.563773 11323 debug-util.cc:398] Leaking SignalData structure 0x7b08003d0200 after lost signal to thread 12643
I20260709 12:10:32.460783 14789 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:32.537549 14925 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:32.433092 14959 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:32.532101 12569 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:32.484315 14885 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:32.602566 14885 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:32.603183 14885 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:32.545198 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50015.1us
I20260709 12:10:32.471673 14741 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:32.533802 12646 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:32.510835 14852 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:32.509905 14412 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:32.453008 14956 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.088s
I20260709 12:10:32.555696 12528 rpcz_store.cc:276] 0709 12:10:30.405997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.304919 (+1898922us) service_pool.cc:224] Handling call
0709 12:10:32.305235 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:32.415351 (+110116us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:32.415387 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:32.415450 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:32.415503 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.485997 (+ 70494us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b5400106dd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:32.486043 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:32.486225 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3302,"spinlock_wait_cycles":244771072}
I20260709 12:10:32.523444 12524 pending_rounds.cc:85] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 6
I20260709 12:10:32.613906 12524 pending_rounds.cc:107] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 5.7
I20260709 12:10:32.614250 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:32.614360 12524 raft_consensus.cc:2889] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:32.522976 14879 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.484861 14958 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:10:32.524722 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 164947us
I20260709 12:10:32.511689 14889 consensus_queue.cc:237] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:32.511027 14957 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:32.525630 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 208625us
I20260709 12:10:32.540361 14502 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:32.529016 14971 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:32.567899 12525 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:10:32.548485 14891 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.585s	user 0.013s	sys 0.000s
I20260709 12:10:32.468997 14538 ts_tablet_manager.cc:1403] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645: Time spent bootstrapping tablet: real 3.161s	user 0.063s	sys 0.072s
I20260709 12:10:32.467327 14953 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.096s
I20260709 12:10:32.639545 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.608695 14974 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.571194 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.547930 14783 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:32.556196 14926 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:32.585587 14691 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6.
W20260709 12:10:32.590724 12460 debug-util.cc:398] Leaking SignalData structure 0x7b080029a020 after lost signal to thread 14768
I20260709 12:10:32.574391 14843 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:32.570708 14789 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:32.619946 14958 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:32.548717 14822 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:32.608083 12569 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:32.609076 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:32.731679 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 123719us
W20260709 12:10:32.732360 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 123390us
W20260709 12:10:32.700088 14966 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.289s	user 0.002s	sys 0.000s
I20260709 12:10:32.615559 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.628758 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:32.744650 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:32.626597 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:32.628305 12672 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:32.745904 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:32.631778 12673 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:10:32.629078 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:32.641501 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:32.616387 14905 consensus_meta.cc:287] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.624s	user 0.011s	sys 0.003s
I20260709 12:10:32.643000 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:32.529474 14924 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:32.644562 12525 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:32.752493 12671 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:32.522455 14892 raft_consensus.cc:697] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:32.646592 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:32.571118 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1614) took 1001 ms. Trace:
I20260709 12:10:32.631464 14879 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:32.669771 14739 consensus_meta.cc:287] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.628s	user 0.006s	sys 0.006s
I20260709 12:10:32.756076 12675 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:32.700368 14829 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.756070 12531 rpcz_store.cc:276] 0709 12:10:31.569435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.314761 (+745326us) service_pool.cc:224] Handling call
0709 12:10:32.315097 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:32.315324 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:10:32.369078 (+ 53754us) log.cc:844] Serialized 14 byte log entry
0709 12:10:32.569734 (+200656us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:32.569885 (+   151us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:32.570011 (+   126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.570660 (+   649us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:32.570942 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":54362,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":51,"raft.queue_time_us":41747,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":98,"thread_start_us":191232,"threads_started":1,"wal-append.queue_time_us":373260}
I20260709 12:10:32.661549 14977 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:32.670513 14804 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.504s	user 0.004s	sys 0.013s
W20260709 12:10:32.671566 14292 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.522s	user 0.009s	sys 0.010s
W20260709 12:10:32.672257 14486 consensus_meta.cc:287] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.517s	user 0.012s	sys 0.004s
I20260709 12:10:32.619737 14968 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:32.642488 14857 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:32.674350 14948 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:32.675370 14969 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:10:32.679337 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.669308 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:32.646602 14904 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.677s	user 0.012s	sys 0.000s
I20260709 12:10:32.617991 14412 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:32.638064 14971 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:32.699327 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 58717.5us
I20260709 12:10:32.755882 14924 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:32.774781 14957 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:32.724798 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:32.622182 14976 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260709 12:10:32.611578 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 77884.9us
W20260709 12:10:32.814028 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 89372.4us
I20260709 12:10:32.814271 12646 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:32.735666 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:32.662563 14955 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.720155 14817 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:10:32.695111 14974 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:32.740823 14598 consensus_meta.cc:287] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.576s	user 0.004s	sys 0.015s
I20260709 12:10:32.837750 14691 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:10:32.838984 14901 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.522s	user 0.011s	sys 0.000s
I20260709 12:10:32.838655 14598 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:32.725028 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 388861us
I20260709 12:10:32.843250 12494 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:32.844977 12494 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:32.845996 14738 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:32.821128 14739 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:32.846664 14739 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:32.749045 12674 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
W20260709 12:10:32.753662 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 112255us
I20260709 12:10:32.847350 14739 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:32.847980 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:32.848311 12571 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:32.849061 14877 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:32.849314 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:32.849730 14877 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:32.768851 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:32.855602 12671 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:32.755267 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1717) took 2811 ms. Trace:
I20260709 12:10:32.856840 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.846570 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.857746 12522 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:32.857417 12531 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:10:32.734516 14958 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:32.774679 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1619) took 1203 ms. Trace:
I20260709 12:10:32.863555 12527 rpcz_store.cc:276] 0709 12:10:31.571369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.338865 (+767496us) service_pool.cc:224] Handling call
0709 12:10:32.680404 (+341539us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:32.680729 (+   325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:32.680768 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:32.680837 (+    69us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:32.680889 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.680977 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:32.774391 (+ 93414us) spinlock_profiling.cc:243] Waited 341 ms on lock 0x7b54003f7350. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:32.774580 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":784618880}
W20260709 12:10:32.734858 14774 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260709 12:10:32.785895 12680 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:32.786631 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:32.734345 14925 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:32.663655 14891 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:32.770532 14954 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.807340 12572 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:32.702606 12460 debug-util.cc:398] Leaking SignalData structure 0x7b08002980c0 after lost signal to thread 14914
I20260709 12:10:32.817693 14976 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:32.880450 12460 debug-util.cc:398] Leaking SignalData structure 0x7b0800298080 after lost signal to thread 14915
W20260709 12:10:32.881055 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 73881.1us
I20260709 12:10:32.801239 12606 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:32832
I20260709 12:10:32.882452 12606 raft_consensus.cc:606] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:32.719444 14970 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.101s
W20260709 12:10:32.815948 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 80401.9us
I20260709 12:10:32.775627 14804 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:32.884341 12643 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:32.738585 14864 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:32.816740 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1561) took 1871 ms. Trace:
I20260709 12:10:32.801065 12675 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:32.782451 14968 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:32.822978 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:32.828485 13305 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:10:32.893705 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 65311us
I20260709 12:10:32.831168 14962 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:32.801234 14965 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:32.833487 14512 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:10:32.835753 14449 consensus_meta.cc:287] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.679s	user 0.011s	sys 0.004s
I20260709 12:10:32.804237 14486 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.846786 14738 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:32.694998 14819 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:32.850040 12674 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:32.696373 14741 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.850926 14240 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:10:32.850054 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:32.858495 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.856668 12601 rpcz_store.cc:276] 0709 12:10:29.944201 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.043354 (+2099153us) service_pool.cc:224] Handling call
0709 12:10:32.051099 (+  7745us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:32.051317 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:10:32.289679 (+238362us) log.cc:844] Serialized 14 byte log entry
0709 12:10:32.754600 (+464921us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b18002b8ca0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287
0709 12:10:32.754734 (+   134us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:32.754796 (+    62us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:32.754920 (+   124us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.755008 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:32.755178 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39645,"prepare.queue_time_us":301347,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":57,"raft.queue_time_us":146077,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":42230,"spinlock_wait_cycles":44620032,"thread_start_us":468215,"threads_started":3,"wal-append.queue_time_us":243925}
I20260709 12:10:32.856688 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:32.861527 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:32.957285 12525 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 5: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:32.778010 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:32.956169 12673 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:32.866878 12680 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:10:32.958205 12672 raft_consensus.cc:2410] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
W20260709 12:10:32.867887 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:32.808265 14412 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:32.774703 14829 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:32.812206 14924 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:32.958993 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:32.959537 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:32.783797 14292 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:32.962466 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 116359us
I20260709 12:10:32.815119 14971 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:32.819206 14957 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:32.877327 12523 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 4: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:32.827397 14948 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:32.885586 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.775029 14977 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:32.898890 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:32.887329 12679 rpcz_store.cc:276] 0709 12:10:30.945485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.964666 (+1019181us) service_pool.cc:224] Handling call
0709 12:10:31.965071 (+   405us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:32.052894 (+ 87823us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:32.052926 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:32.080768 (+ 27842us) log.cc:844] Serialized 52 byte log entry
0709 12:10:32.282842 (+202074us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:32.292163 (+  9321us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.397089 (+104926us) spinlock_profiling.cc:243] Waited 83.5 ms on lock 0x7b54003bc754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:32.397173 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:32.800337 (+403164us) raft_consensus.cc:1685] finished
0709 12:10:32.800377 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:32.816503 (+ 16126us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":137,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":7704,"spinlock_wait_cycles":191938432,"thread_start_us":142617,"threads_started":1,"wal-append.queue_time_us":425989}
I20260709 12:10:32.863938 14598 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:32.901719 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:32.911631 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:32.925937 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:32.829102 14955 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:32.873245 14877 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:10:32.940672 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:32.886956 14954 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:32.920974 14959 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:32.900668 14691 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:32.904165 14953 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:32.903208 14968 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:32.957389 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 107425us
I20260709 12:10:32.919389 14864 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:32.973582 14819 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:32.824911 14783 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:10:32.853179 14964 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.284s	user 0.001s	sys 0.000s
W20260709 12:10:32.960139 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:32.939653 14560 consensus_meta.cc:287] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.785s	user 0.009s	sys 0.012s
I20260709 12:10:32.964268 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:33.017460 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:32.967626 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:33.018193 12675 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:32.950270 14738 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:32.979769 12531 raft_consensus.cc:2368] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4.
I20260709 12:10:32.981527 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:32.888157 14976 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:32.986809 12528 raft_consensus.cc:2393] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 4: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:10:32.988961 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 77439.2us
I20260709 12:10:33.022480 12643 leader_election.cc:400] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6.
I20260709 12:10:33.023376 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:32.990247 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:32.936028 14885 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:32.935376 14449 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:10:32.976387 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:32.999565 12571 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:32.868917 14892 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:33.005383 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:33.030797 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:33.007761 14877 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:33.008644 14955 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:32.977314 14957 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:32.958753 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:32.964769 14986 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:32.949862 14486 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:33.014484 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:33.041433 12522 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:33.000451 14959 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:32.999795 14948 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 8 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:32.961159 14741 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:32.990711 14777 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.042s
I20260709 12:10:33.032089 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:33.030982 14995 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:33.019361 14783 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:33.077371 14783 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:33.079154 14977 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:33.083762 14978 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:33.084262 14978 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:33.085134 14978 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:33.085706 14958 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:33.088716 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 65415us
I20260709 12:10:33.038380 12675 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:33.039072 12674 raft_consensus.cc:2410] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:33.026407 14738 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:33.044179 12526 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 4: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:10:33.040870 14983 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
W20260709 12:10:33.031313 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:33.033113 14962 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:33.032796 14953 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:33.012490 14975 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 12:10:33.060341 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:33.120276 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 60028.8us
I20260709 12:10:33.121480 12571 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:33.014611 14796 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:33.058669 14974 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:33.055120 14959 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:33.068346 14995 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:33.070521 14804 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:33.071210 14986 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:33.071682 14970 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:33.009807 14973 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 12:10:33.074704 14292 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:33.042621 14819 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:10:33.026225 14560 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:33.039426 14879 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:33.014148 14864 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:33.017241 14538 raft_consensus.cc:359] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645 [term 4 NON_PARTICIPANT]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:33.115851 12529 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 12:10:33.115868 13302 leader_election.cc:336] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.531s (SENT)
I20260709 12:10:33.153998 14977 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:33.194600 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:33.200742 12602 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:10:33.201201 14538 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Initialized, Role: LEARNER
I20260709 12:10:33.202138 14538 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:10:33.138774 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:33.204155 14538 ts_tablet_manager.cc:1434] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645: Time spent starting tablet: real 0.552s	user 0.007s	sys 0.005s
I20260709 12:10:33.139633 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:33.101227 14990 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:33.204998 12599 raft_consensus.cc:2376] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5. Re-sending same reply.
I20260709 12:10:33.142429 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:33.205986 12598 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:33.207166 12597 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:33.143390 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:33.168105 14691 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:33.155220 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 62174us
I20260709 12:10:33.212782 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:33.156697 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:33.118669 14978 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:33.161752 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:33.163091 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:33.172140 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 174471us
I20260709 12:10:33.173821 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:33.224664 12678 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:33.175220 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 55987.4us
I20260709 12:10:33.118011 14891 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:33.180751 12530 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260709 12:10:33.179842 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:33.182824 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:33.226934 12672 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:33.183109 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:33.227918 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:33.137758 14843 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:33.228648 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:33.179713 13331 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:33.055223 14817 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:33.153636 14992 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:33.204569 14538 ts_tablet_manager.cc:1467] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645: Tablet startup took 566ms
I20260709 12:10:33.140398 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:33.208415 12679 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:33.144506 12496 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:33.218771 14980 raft_consensus.cc:3055] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:33.219297 14990 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:33.195608 13302 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:33.230063 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:33.217715 14829 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:33.229931 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:33.248587 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 132759us
I20260709 12:10:33.249120 12571 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:33.249358 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:33.243870 14980 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:33.250537 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:33.174476 14512 consensus_queue.cc:1059] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:10:33.233141 12598 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:10:33.176738 14739 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:33.236846 12676 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:33.264456 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:33.235110 14733 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:33.121114 14952 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.074s
W20260709 12:10:33.271940 14981 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.406s	user 0.000s	sys 0.004s
I20260709 12:10:33.273077 14891 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:33.241210 12675 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 4: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:33.168305 14986 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:33.284333 14986 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:33.231680 14926 consensus_meta.cc:287] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.002s
I20260709 12:10:33.242306 14843 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:33.218259 14977 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:33.253654 12571 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:33.222980 14958 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:33.287034 14843 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:33.231567 14962 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260709 12:10:33.292997 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:33.300635 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:33.290970 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:33.301704 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:33.305961 12571 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:33.237761 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:33.263140 14774 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:33.307840 14774 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:33.263283 14925 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:10:33.308252 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 70625.2us
I20260709 12:10:33.308727 14925 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:33.308284 14774 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:33.269320 14741 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:33.270445 14240 raft_consensus.cc:3055] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:10:33.271123 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 126781us
I20260709 12:10:33.276510 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:33.340971 12603 raft_consensus.cc:2410] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:33.340854 12496 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:33.341974 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:33.342274 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 61138.7us
I20260709 12:10:33.342656 12603 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:33.276813 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:33.343365 12600 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:33.344511 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:33.345240 12600 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:33.277372 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:33.277478 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:33.264472 14904 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:33.348542 12599 raft_consensus.cc:2410] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:10:33.278427 14955 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:33.279557 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:33.286432 14926 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:33.352355 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:33.353158 12598 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:33.285843 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:33.210261 14789 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:33.254208 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:33.197340 14948 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:33.222114 14691 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:33.363725 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:33.303864 14970 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:33.291829 14989 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:33.303257 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:33.286741 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:33.265218 12676 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 5: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:33.273730 14891 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:33.375145 12496 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:33.375783 14691 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:33.376583 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:33.377372 12672 raft_consensus.cc:2410] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:10:33.379046 12600 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:33.380043 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:33.380271 15005 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:33.380609 12600 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:33.380688 15005 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:33.381206 15005 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:33.305989 12675 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:10:33.330248 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:33.383519 12601 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:33.331879 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:33.349797 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:33.384618 12599 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:33.385532 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:33.386341 12599 raft_consensus.cc:2410] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:33.349047 14958 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:33.246315 14992 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:33.350664 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:33.335832 14996 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.252s	user 0.002s	sys 0.000s
I20260709 12:10:33.275002 14885 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:33.262423 12679 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:33.368765 13333 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:33.368803 12605 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:33.337961 14817 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:33.368789 12603 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:33.387053 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:33.279357 12678 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:33.388077 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:33.377722 12496 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:33.388557 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:33.464341 12599 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:33.464524 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 76031.8us
I20260709 12:10:33.340205 14240 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:33.372213 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:33.298291 14954 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:33.289038 14733 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:33.392882 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:33.475889 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:33.362437 14819 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:33.441752 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:33.448202 12598 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
W20260709 12:10:33.482779 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 139570us
I20260709 12:10:33.443153 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:33.367161 14975 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.101s
I20260709 12:10:33.465869 12675 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:33.381727 14739 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:33.470170 12677 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:33.382757 14904 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 12:10:33.477151 13333 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:10:33.484683 12605 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:33.496341 12601 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:33.496842 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:33.465173 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:33.497779 12605 raft_consensus.cc:2410] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:10:33.497912 12603 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:33.395170 14958 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:33.500232 12601 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:33.486382 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:33.470853 14970 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:33.346868 14990 consensus_queue.cc:1059] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.120s
I20260709 12:10:33.351480 14993 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.140s
I20260709 12:10:33.463598 14817 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:33.492136 14560 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:33.476617 12678 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:33.493088 12604 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:33.492740 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:33.494030 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 116368us
W20260709 12:10:33.484215 14749 consensus_meta.cc:287] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.350s	user 0.011s	sys 0.013s
I20260709 12:10:33.498347 12600 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:33.483737 13302 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:33.240531 14538 ts_tablet_manager.cc:1469] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645: Trace:
0709 12:10:18.211573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:18.229897 (+ 18324us) service_pool.cc:224] Handling call
0709 12:10:18.644069 (+414172us) ts_tablet_manager.cc:934] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645: Initiating tablet copy from peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
0709 12:10:22.190217 (+3546148us) tablet_copy_client.cc:368] Tablet copy session begun
0709 12:10:23.281337 (+1091120us) tablet_metadata.cc:665] Metadata flushed
0709 12:10:23.281471 (+   134us) tablet_copy_client.cc:497] Wrote new tablet metadata
0709 12:10:24.039443 (+757972us) tablet_copy_client.cc:418] Wrote new consensus metadata
0709 12:10:24.074903 (+ 35460us) tablet_replica.cc:206] Creating consensus instance
0709 12:10:24.144937 (+ 70034us) ts_tablet_manager.cc:989] Replica registered.
0709 12:10:24.159524 (+ 14587us) inbound_call.cc:177] Queueing success response
0709 12:10:29.254626 (+5095102us) ts_tablet_manager.cc:1379] Bootstrapping tablet
0709 12:10:32.651716 (+3397090us) ts_tablet_manager.cc:1435] Starting tablet replica
0709 12:10:32.698516 (+ 46800us) tablet_replica.cc:261] Starting instrumentation
0709 12:10:32.699200 (+   684us) tablet_replica.cc:280] Starting consensus
Metrics: {"dns_us":606,"raft.queue_time_us":123,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":105,"tablet-copy.queue_time_us":305233,"tablet-download-pool.queue_time_us":693,"tablet-download-pool.run_cpu_time_us":2150,"tablet-download-pool.run_wall_time_us":3581761,"thread_start_us":404319,"threads_started":2}
W20260709 12:10:33.553447 14957 consensus_meta.cc:287] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.513s	user 0.006s	sys 0.004s
I20260709 12:10:33.553609 14995 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:33.499641 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:33.380636 14783 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:33.556017 14783 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:33.500958 12679 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
W20260709 12:10:33.487571 14746 consensus_meta.cc:287] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.324s	user 0.005s	sys 0.018s
I20260709 12:10:33.505795 12676 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:10:33.520335 12605 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:10:33.434862 15000 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:33.286684 14980 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:33.382082 14976 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:33.523206 14777 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:33.452232 14968 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
W20260709 12:10:33.531314 14412 consensus_meta.cc:287] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.559s	user 0.005s	sys 0.005s
I20260709 12:10:33.485994 14796 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:33.535840 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:33.490669 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:33.540261 12599 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
W20260709 12:10:33.521050 14924 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.548s	user 0.011s	sys 0.000s
W20260709 12:10:33.595904 15002 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:10:33.596156 14992 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:33.596796 14999 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:10:33.597329 12528 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:33.543429 15011 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
W20260709 12:10:33.544845 15004 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:10:33.390980 14997 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.153s
W20260709 12:10:33.546337 15007 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:10:33.558907 12675 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 7: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:33.521348 14958 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:33.391794 14989 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:33.563760 12496 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:33.522321 14970 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:33.567683 12679 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:10:33.569164 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 107661us
I20260709 12:10:33.573539 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:33.538422 14817 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:33.591087 15000 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:33.590293 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 106642us
I20260709 12:10:33.628365 13302 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:33.595461 14924 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:33.520676 14864 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:33.599900 12673 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:33.600917 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:33.558159 14879 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:33.616376 12674 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 4: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:33.579030 14777 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:33.573099 14746 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:33.622938 12679 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:33.624122 13333 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:33.642900 12679 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:10:33.624842 12571 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:33.553980 14957 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:33.468504 14977 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:33.633009 12494 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:33.571684 14829 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:10:33.581403 14560 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:33.637282 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 73648.9us
I20260709 12:10:33.647843 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:33.639750 12678 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:33.650708 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:33.604365 14412 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:33.655409 12494 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:33.629043 14970 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:33.653642 12496 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:33.656651 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:33.655891 14891 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:33.638285 14777 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:33.663200 12673 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:33.663419 12494 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:33.689383 12494 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:33.690387 12494 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:33.691452 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:33.664168 12496 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:33.665789 12601 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: [6.2-6.2]
I20260709 12:10:33.665971 14979 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:33.694267 14292 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:33.694897 14292 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:33.695411 14292 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:33.648969 14926 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:33.667570 15023 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:33.635464 14978 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.028s
W20260709 12:10:33.709932 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 81659.5us
I20260709 12:10:33.681082 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 133) took 4014 ms. Trace:
I20260709 12:10:33.682981 15019 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:33.655692 15010 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:33.661614 15008 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:33.654505 14997 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:33.706564 15023 raft_consensus.cc:697] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 5 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:33.707157 14746 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:33.664930 14924 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:33.703627 14959 consensus_meta.cc:287] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.527s	user 0.009s	sys 0.001s
I20260709 12:10:33.699848 14789 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:33.668952 14982 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.672s	user 0.002s	sys 0.000s
W20260709 12:10:33.667332 12523 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 1594) took 1827 ms (client timeout 2396 ms). Trace:
I20260709 12:10:33.647248 14843 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260709 12:10:33.728410 12523 rpcz_store.cc:269] 0709 12:10:31.839818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.122636 (+282818us) service_pool.cc:224] Handling call
0709 12:10:32.606649 (+484013us) spinlock_profiling.cc:243] Waited 6.63 ms on lock 0x7b54003f67a8. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e7fd5f4 00007f148d9e6227 00007f148d9e5ab8 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:33.667107 (+1060458us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b54003f6954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:33.667271 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":636723328}
W20260709 12:10:33.721637 14788 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.412s	user 0.010s	sys 0.005s
I20260709 12:10:33.712530 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:33.638506 14879 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:33.665128 14892 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:33.712014 12597 rpcz_store.cc:276] 0709 12:10:29.666543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.042612 (+2376069us) service_pool.cc:224] Handling call
0709 12:10:32.047618 (+  5006us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:33.207747 (+1160129us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:33.680844 (+473097us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54002e54d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:33.681012 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2666165120}
I20260709 12:10:33.686720 14975 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:33.724102 14811 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:33.718102 15019 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:33.721781 14804 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:33.656536 14560 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:33.722739 14997 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:33.748220 15023 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:33.725440 14986 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:33.750542 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1601) took 1444 ms. Trace:
I20260709 12:10:33.739104 14788 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:33.757757 15011 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:33.758886 13334 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:33.759028 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:33.759812 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:33.783525 13339 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
W20260709 12:10:33.740970 14738 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.621s	user 0.010s	sys 0.002s
I20260709 12:10:33.729616 14979 raft_consensus.cc:697] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:33.756127 15019 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:33.767025 14804 raft_consensus.cc:697] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:33.717986 14774 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:33.792075 13338 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
W20260709 12:10:33.770756 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 79389.2us
I20260709 12:10:33.751449 14292 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:33.728652 14959 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:33.739311 15010 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:33.773576 12524 rpcz_store.cc:276] 0709 12:10:32.306246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.469426 (+163180us) service_pool.cc:224] Handling call
0709 12:10:32.523245 (+ 53819us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:32.619860 (+ 96615us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:32.619894 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:32.620363 (+   469us) log.cc:844] Serialized 26 byte log entry
0709 12:10:32.870531 (+250168us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:32.870667 (+   136us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:32.870783 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:33.667982 (+797199us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:33.750231 (+ 82249us) raft_consensus.cc:1685] finished
0709 12:10:33.750253 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:33.750476 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23044480,"thread_start_us":196822,"threads_started":1,"wal-append.queue_time_us":309676}
I20260709 12:10:33.793184 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:33.796065 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:33.686111 15000 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:33.753208 14975 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:33.756250 14892 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:33.766521 14811 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:33.770327 14997 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:33.761544 15022 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:33.796968 12494 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:33.788645 14976 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:33.798255 14989 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:33.798188 14957 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:33.798559 14986 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:33.787930 15011 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:33.798753 14292 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:33.825858 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:33.834657 12494 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:33.807818 12677 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:33.816517 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:33.811513 14955 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:33.792434 14738 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:33.822082 13332 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 6: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:33.795425 14968 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:33.825825 14412 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:33.842763 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:33.845237 13333 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:10:33.817839 15010 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:33.812361 14975 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:33.809849 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:33.832660 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 168554us
I20260709 12:10:33.835862 13334 raft_consensus.cc:2410] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:10:33.786926 12616 tablet_copy_service.cc:342] P f5fc807f7ecc4d1ca0fd6be524f734ce: Request end of tablet copy session e56d487365394ac5ac5f1002926a5645-3c9b5315396948b2af3fa5480c9bee87 received from {username='slave'} at 127.0.0.1:48492
I20260709 12:10:33.827490 14892 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:33.862406 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:33.831100 14789 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:33.866283 12496 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:33.833480 14502 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.172s	user 0.012s	sys 0.003s
I20260709 12:10:33.869095 13337 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:33.866784 12616 tablet_copy_service.cc:434] P f5fc807f7ecc4d1ca0fd6be524f734ce: ending tablet copy session e56d487365394ac5ac5f1002926a5645-3c9b5315396948b2af3fa5480c9bee87 on tablet 3c9b5315396948b2af3fa5480c9bee87 with peer e56d487365394ac5ac5f1002926a5645
I20260709 12:10:33.871835 14774 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:33.872284 14486 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:33.871989 15005 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:33.827883 14804 consensus_queue.cc:237] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:10:33.873772 12525 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:33.878988 15010 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:33.881120 15034 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:33.833662 15006 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.054s
W20260709 12:10:33.909355 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 72849.4us
I20260709 12:10:33.840996 14819 consensus_queue.cc:1059] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.059s
I20260709 12:10:33.873557 14789 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:10:33.915117 14971 consensus_meta.cc:287] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.931s	user 0.011s	sys 0.002s
W20260709 12:10:33.946306 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 59448.3us
I20260709 12:10:33.916567 15034 raft_consensus.cc:697] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:33.928184 14789 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:33.898792 14502 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:33.981436 12598 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:33.984310 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:33.886327 14811 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:33.867303 14979 consensus_queue.cc:237] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:33.985148 12598 raft_consensus.cc:2368] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7.
I20260709 12:10:33.995782 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:33.996156 12602 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 6: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:33.996598 12598 raft_consensus.cc:2368] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5.
I20260709 12:10:33.988102 12599 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:33.995771 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:33.967564 14774 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:34.009898 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:34.011883 14774 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:34.012872 15032 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:10:34.013404 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:34.014343 12600 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:33.979604 14486 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:33.970623 15027 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:10:34.015954 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 103280us
W20260709 12:10:34.024027 15031 log.cc:927] Time spent T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 12:10:34.022859 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:33.985554 14560 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:34.028187 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:34.031419 14774 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:34.029585 12600 raft_consensus.cc:2368] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 3. Current term is 4.
I20260709 12:10:34.035166 12597 raft_consensus.cc:2368] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7.
W20260709 12:10:34.036917 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:34.037479 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:34.038075 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:34.038363 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:34.039453 12600 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:34.041307 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:34.041944 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:34.042670 13330 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:10:34.044425 14733 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:34.043689 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:34.045361 12600 raft_consensus.cc:2410] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:34.047010 15008 raft_consensus.cc:993] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: : Instructing follower 193a40da6e364ead8ab561e14c3cc8d1 to start an election
I20260709 12:10:34.047178 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:34.048036 12600 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:34.049103 12600 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:10:34.050064 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:34.051010 12600 raft_consensus.cc:2393] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 6: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:34.051472 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:34.051538 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:34.053264 12599 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:34.053702 12597 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260709 12:10:34.055522 14502 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:34.056685 15005 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:34.051827 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:34.059770 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:34.060746 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:34.061982 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:34.020200 14959 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:34.063900 12598 raft_consensus.cc:2376] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5. Re-sending same reply.
I20260709 12:10:34.063424 12600 raft_consensus.cc:2410] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:34.065754 14904 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:34.066242 14904 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:34.066896 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:33.947526 14971 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:34.068756 12599 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:10:34.071240 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:34.074649 12599 pending_rounds.cc:85] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 2
I20260709 12:10:34.074976 12599 pending_rounds.cc:107] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:10:34.075325 12599 raft_consensus.cc:2889] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:34.077544 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:10:34.079052 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 110048us
I20260709 12:10:34.082701 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:34.081144 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:34.086330 14738 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:34.060297 15019 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:34.085795 12531 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:34.072438 14292 raft_consensus.cc:1081] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Signalling peer 193a40da6e364ead8ab561e14c3cc8d1 to start an election
I20260709 12:10:34.025187 14486 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:34.096051 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:34.100180 12679 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 4: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:34.102788 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:34.107067 12678 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:10:34.039736 14970 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:34.067205 14904 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:34.104382 14777 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:34.129583 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:34.130283 12679 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:34.130859 14486 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:34.131592 14777 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:34.133190 15037 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:10:34.133669 14971 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:34.118239 14733 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:10:34.140525 14970 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:34.141278 14733 raft_consensus.cc:3055] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:34.141666 12529 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:34.116883 15011 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:34.142794 12599 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260709 12:10:34.144719 12676 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:10:34.129249 12572 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.845s (SENT) [suppressed 27 similar messages]
I20260709 12:10:34.141769 14733 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:34.176244 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:34.177367 13339 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:34.179211 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:34.190834 12679 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:34.081965 15034 consensus_queue.cc:237] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:34.208544 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:34.209717 12676 raft_consensus.cc:2393] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 5: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:10:34.220614 14817 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.578s	user 0.012s	sys 0.001s
W20260709 12:10:34.252614 13304 leader_election.cc:336] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.795s (SENT)
I20260709 12:10:34.253580 13304 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:34.291271 12599 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:10:34.297101 15042 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:10:34.315254 14993 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:34.315954 14993 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:34.319483 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:34.320338 12599 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:34.323331 14993 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:34.287109 14959 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:34.239090 14817 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:34.372144 12599 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
W20260709 12:10:34.359522 14958 consensus_meta.cc:287] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.730s	user 0.014s	sys 0.000s
I20260709 12:10:34.376246 12599 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:34.234846 14733 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:34.389155 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:34.740186 14975 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.871s	user 0.014s	sys 0.000s
W20260709 12:10:34.482327 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 203134us
I20260709 12:10:34.800639 14975 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:34.801692 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:34.803563 14975 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:34.560230 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:34.550922 14993 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:34.821290 14993 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:34.725092 13339 consensus_meta.cc:287] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.542s	user 0.012s	sys 0.000s
I20260709 12:10:34.848624 13339 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:10:34.850561 12569 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:34.381816 15019 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:34.847540 14973 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:34.884125 14973 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:34.885071 14796 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:34.885586 14796 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:34.886497 14796 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:34.887857 14796 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:34.895005 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:34.896531 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:34.824168 14993 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:34.900590 12598 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.043s	sys 0.052s
W20260709 12:10:34.901315 12598 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.043s	sys 0.052s
I20260709 12:10:34.847810 15010 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:34.897028 14973 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:34.905753 14502 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:34.906464 15016 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:34.906383 14502 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:34.906925 15016 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:34.907864 15016 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:34.908989 15016 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:34.908926 15010 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:34.911567 15010 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:34.912464 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1752) took 1475 ms. Trace:
I20260709 12:10:34.912919 12601 rpcz_store.cc:276] 0709 12:10:33.436689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:33.502243 (+ 65554us) service_pool.cc:224] Handling call
0709 12:10:33.534540 (+ 32297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:33.693225 (+158685us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:34.912153 (+1218928us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54001617d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:34.912368 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":301195648}
I20260709 12:10:34.917564 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:34.918846 12528 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:34.380301 14958 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:34.930781 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:34.931128 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:34.913911 14993 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:34.933569 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:34.913486 14817 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:34.939348 12671 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:34.943174 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:34.945801 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1610) took 2061 ms. Trace:
I20260709 12:10:34.946465 12530 rpcz_store.cc:276] 0709 12:10:32.884671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.884792 (+   121us) service_pool.cc:224] Handling call
0709 12:10:32.885124 (+   332us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:33.231706 (+346582us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:34.945399 (+1713693us) spinlock_profiling.cc:243] Waited 295 ms on lock 0x7b5400053a54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:34.945704 (+   305us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":679256320}
I20260709 12:10:34.948338 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:34.949241 12530 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:10:34.950902 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1768) took 1807 ms. Trace:
I20260709 12:10:34.951409 12677 rpcz_store.cc:276] 0709 12:10:33.143766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:33.143902 (+   136us) service_pool.cc:224] Handling call
0709 12:10:34.950681 (+1806779us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b54000ab4d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:34.950817 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":556901632}
I20260709 12:10:34.956789 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:34.967944 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:34.975641 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 172) took 1648 ms. Trace:
I20260709 12:10:34.976208 12673 rpcz_store.cc:276] 0709 12:10:33.326888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:33.630568 (+303680us) service_pool.cc:224] Handling call
0709 12:10:33.630881 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:33.688300 (+ 57419us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:34.975283 (+1286983us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b54003d45d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:34.975542 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73871104}
I20260709 12:10:34.986339 15023 consensus_queue.cc:1059] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:34.988063 15023 raft_consensus.cc:3055] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:34.988605 15023 raft_consensus.cc:740] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:35.003624 14980 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:35.007633 15029 raft_consensus.cc:3055] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:35.008428 15029 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:35.018765 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 179) took 1678 ms. Trace:
I20260709 12:10:35.019516 12675 rpcz_store.cc:276] 0709 12:10:33.341568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:33.492493 (+150925us) service_pool.cc:224] Handling call
0709 12:10:35.018474 (+1525981us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b5400092ed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:35.018630 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42611584}
W20260709 12:10:35.021777 12602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 1734) took 1746 ms (client timeout 1699 ms). Trace:
W20260709 12:10:35.022455 12602 rpcz_store.cc:269] 0709 12:10:33.275818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:33.277093 (+  1275us) service_pool.cc:224] Handling call
0709 12:10:35.021425 (+1744332us) spinlock_profiling.cc:243] Waited 648 ms on lock 0x7b5400231c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:35.021656 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1490616192}
W20260709 12:10:35.024693 13338 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 412) took 1693 ms (client timeout 1699 ms). Trace:
W20260709 12:10:35.025312 13338 rpcz_store.cc:269] 0709 12:10:33.331412 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:33.331612 (+   200us) service_pool.cc:224] Handling call
0709 12:10:35.024423 (+1692811us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b54000d7854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:35.024590 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":937433856}
I20260709 12:10:35.029691 14523 raft_consensus.cc:3055] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:35.030443 14523 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:35.031697 14523 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:35.033208 14523 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:35.037083 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:35.039714 15054 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.189s	user 0.003s	sys 0.001s
I20260709 12:10:35.001554 14240 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:35.042546 15055 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.192s	user 0.002s	sys 0.000s
I20260709 12:10:35.044569 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:35.045753 13338 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 7: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:10:35.046587 12671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1576) took 1880 ms (client timeout 2185 ms). Trace:
W20260709 12:10:35.047094 12525 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1641) took 1662 ms (client timeout 1974 ms). Trace:
W20260709 12:10:35.047880 12525 rpcz_store.cc:269] 0709 12:10:33.384886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:33.471857 (+ 86971us) service_pool.cc:224] Handling call
0709 12:10:35.017854 (+1545997us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b540042a9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:35.046983 (+ 29129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":641621248}
I20260709 12:10:34.957899 12531 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:35.051828 14560 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:35.052646 14560 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:34.969694 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:35.056857 12529 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 5: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:35.058166 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:34.973961 14879 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:35.060581 14879 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:35.047222 12671 rpcz_store.cc:269] 0709 12:10:33.166395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:33.253914 (+ 87519us) service_pool.cc:224] Handling call
0709 12:10:35.046340 (+1792426us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54003d40d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:35.046484 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3643349888}
I20260709 12:10:35.061740 14560 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:35.009682 15029 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:35.066886 14449 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:35.067137 15029 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:34.990000 15023 consensus_queue.cc:260] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:35.074149 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:35.075377 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:35.052342 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:35.080384 12531 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 4: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:35.084721 14733 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:35.090930 12674 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1764) took 1952 ms (client timeout 1856 ms). Trace:
W20260709 12:10:35.091471 12674 rpcz_store.cc:269] 0709 12:10:33.138309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:33.138477 (+   168us) service_pool.cc:224] Handling call
0709 12:10:35.090683 (+1952206us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b54002c4554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:35.090836 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":944519424}
I20260709 12:10:35.084656 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:35.094260 13338 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 7: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:35.077086 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1737) took 1794 ms. Trace:
I20260709 12:10:35.100596 14959 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:35.100778 12604 rpcz_store.cc:276] 0709 12:10:33.282617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:33.282815 (+   198us) service_pool.cc:224] Handling call
0709 12:10:33.283193 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:33.539621 (+256428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:35.076794 (+1537173us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b54002ffb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:35.076986 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":482211072}
I20260709 12:10:35.102154 14968 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:35.102717 14968 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:35.103201 13302 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:35.104915 14968 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:35.105393 15011 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:10:35.107923 14691 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:35.108541 14691 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:35.069851 15023 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:35.110553 14801 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:35.061309 14879 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:35.111188 14801 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:35.113304 14879 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:35.113538 14801 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:35.049769 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:35.114110 14801 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:35.114647 14801 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:35.116909 14801 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:35.118259 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:35.120570 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:35.121402 12674 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 6: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:35.121583 14691 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:35.123036 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:35.123741 12674 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:10:35.129787 14240 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:35.134186 12671 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:51308
I20260709 12:10:35.135177 12671 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:10:35.135655 12671 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:35.137748 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:35.144562 14691 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:35.145421 14885 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:35.145830 12671 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:35.149394 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:35.153348 12671 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:35.155396 12672 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:10:35.155575 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:35.156610 14959 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:35.157711 14959 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:35.158792 14959 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:35.164788 12671 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:10:35.173585 14997 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.352s	user 0.011s	sys 0.004s
I20260709 12:10:35.174301 14997 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:35.175912 14502 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:35.181941 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:35.182783 13332 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 5 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5. Ops: []
W20260709 12:10:35.183079 15053 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.325s	user 0.000s	sys 0.000s
I20260709 12:10:35.183712 13337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 410) took 1884 ms. Trace:
I20260709 12:10:35.184108 13337 rpcz_store.cc:276] 0709 12:10:33.299356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:33.299613 (+   257us) service_pool.cc:224] Handling call
0709 12:10:33.345382 (+ 45769us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:33.888128 (+542746us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:35.183488 (+1295360us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b54006419d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:35.183654 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1200127872}
I20260709 12:10:35.185591 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1772) took 1098 ms. Trace:
I20260709 12:10:35.186048 12680 rpcz_store.cc:276] 0709 12:10:34.087472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:34.087704 (+   232us) service_pool.cc:224] Handling call
0709 12:10:34.088119 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:34.088396 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:34.088418 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:34.102953 (+ 14535us) log.cc:844] Serialized 26 byte log entry
0709 12:10:34.367025 (+264072us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:34.367077 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:34.367178 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:35.185331 (+818153us) raft_consensus.cc:1685] finished
0709 12:10:35.185358 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:35.185512 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":190482,"threads_started":1,"wal-append.queue_time_us":596405}
I20260709 12:10:35.181442 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:35.196672 14502 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:35.197420 14502 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:35.198010 15027 raft_consensus.cc:3055] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:35.198475 14958 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:35.189875 13331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 409) took 3414 ms (client timeout 1797 ms). Trace:
I20260709 12:10:35.219190 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:35.220224 13333 raft_consensus.cc:2410] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:10:35.220387 12605 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260709 12:10:35.220777 12605 pending_rounds.cc:85] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 1
I20260709 12:10:35.221005 12605 pending_rounds.cc:107] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:10:35.221264 12605 raft_consensus.cc:2889] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:10:35.221673 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:35.176836 14997 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:35.224882 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1734) took 1174 ms. Trace:
I20260709 12:10:35.182590 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:35.244443 12605 rpcz_store.cc:276] 0709 12:10:34.049940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:34.060430 (+ 10490us) service_pool.cc:224] Handling call
0709 12:10:35.219815 (+1159385us) spinlock_profiling.cc:243] Waited 1.72 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:10:35.220030 (+   215us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:35.221982 (+  1952us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:35.224793 (+  2811us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3953024}
I20260709 12:10:35.231256 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:35.242018 14502 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:35.199779 13331 rpcz_store.cc:269] 0709 12:10:31.775717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.775924 (+   207us) service_pool.cc:224] Handling call
0709 12:10:35.189644 (+3413720us) spinlock_profiling.cc:243] Waited 858 ms on lock 0x7b54001b49d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:35.189778 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1973604224}
I20260709 12:10:34.938601 14993 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:35.246225 13333 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:35.246500 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:35.180124 15050 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.627s	user 0.000s	sys 0.001s
W20260709 12:10:35.252045 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:35.277968 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:35.280519 13332 raft_consensus.cc:2410] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 7 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:10:35.244266 14971 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:35.248683 15012 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
W20260709 12:10:35.252485 14789 consensus_meta.cc:287] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.265s	user 0.016s	sys 0.000s
I20260709 12:10:35.198478 15027 raft_consensus.cc:740] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:35.293395 15012 consensus_queue.cc:1243] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:10:35.289242 14971 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:35.140455 12531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:48476
W20260709 12:10:35.373559 12676 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.857s	user 0.051s	sys 0.094s
W20260709 12:10:35.374605 12679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.954s	user 0.051s	sys 0.080s
I20260709 12:10:35.374856 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:35.376353 12572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.039s	sys 0.056s
I20260709 12:10:35.379323 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 163) took 1049 ms. Trace:
I20260709 12:10:35.176740 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:35.243613 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1751) took 1892 ms. Trace:
I20260709 12:10:35.387750 12673 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
W20260709 12:10:35.393028 12645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.786s	user 0.000s	sys 0.061s
I20260709 12:10:35.395608 12671 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 6: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:35.276365 12674 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:35.373649 12525 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:10:35.429445 12572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.584s	user 0.040s	sys 0.056s
W20260709 12:10:35.476819 12679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.056s	user 0.052s	sys 0.080s
I20260709 12:10:35.488441 14993 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:35.480494 12676 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.964s	user 0.051s	sys 0.096s
I20260709 12:10:35.491066 14789 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:35.504817 12523 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:35.506801 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1778) took 1411 ms. Trace:
I20260709 12:10:35.504987 12603 rpcz_store.cc:276] 0709 12:10:33.351123 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:33.499012 (+147889us) service_pool.cc:224] Handling call
0709 12:10:33.499989 (+   977us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:33.500324 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:33.500353 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:33.534090 (+ 33737us) log.cc:844] Serialized 26 byte log entry
0709 12:10:34.492691 (+958601us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b1800336340. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:10:34.924106 (+431415us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:34.924161 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:34.924250 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:35.177169 (+252919us) raft_consensus.cc:1685] finished
0709 12:10:35.177192 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:35.243520 (+ 66328us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":20658,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":4139,"spinlock_wait_cycles":38526720,"thread_start_us":431019,"threads_started":1,"wal-append.queue_time_us":1625973}
W20260709 12:10:35.526269 12645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.919s	user 0.000s	sys 0.062s
W20260709 12:10:35.536108 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:35.511580 15027 consensus_queue.cc:260] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:35.513805 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:35.426811 12524 rpcz_store.cc:276] 0709 12:10:34.329375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:34.343031 (+ 13656us) service_pool.cc:224] Handling call
0709 12:10:34.343492 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:34.343777 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:34.343814 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:34.344472 (+   658us) log.cc:844] Serialized 26 byte log entry
0709 12:10:34.344636 (+   164us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:34.344690 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:34.344850 (+   160us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:35.378529 (+1033679us) raft_consensus.cc:1685] finished
0709 12:10:35.378610 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:35.379021 (+   411us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:35.544369 15012 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:35.515093 12528 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:35.587745 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:35.517572 12526 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 1597) took 3664 ms (client timeout 1845 ms). Trace:
I20260709 12:10:35.545780 15012 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:35.536891 12676 rpcz_store.cc:276] 0709 12:10:34.095087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:34.214105 (+119018us) service_pool.cc:224] Handling call
0709 12:10:34.224494 (+ 10389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:34.224806 (+   312us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:10:34.512320 (+287514us) log.cc:844] Serialized 14 byte log entry
0709 12:10:35.505776 (+993456us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:35.505937 (+   161us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:35.506169 (+   232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:35.506346 (+   177us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:35.506643 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":704327,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":43,"thread_start_us":1276283,"threads_started":2}
W20260709 12:10:35.595250 12526 rpcz_store.cc:269] 0709 12:10:31.853297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:32.047826 (+194529us) service_pool.cc:224] Handling call
0709 12:10:35.517084 (+3469258us) spinlock_profiling.cc:243] Waited 959 ms on lock 0x7b540038e0d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:35.517349 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2206008192}
I20260709 12:10:35.534603 14958 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:35.605935 12531 raft_consensus.cc:606] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:35.639611 12531 raft_consensus.cc:614] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:10:35.651013 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:35.638676 14939 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.007s	sys 0.008s
W20260709 12:10:35.658583 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:35.661496 12677 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.000s	sys 0.011s
W20260709 12:10:35.661369 15059 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.717s	user 0.019s	sys 0.046s
W20260709 12:10:35.662061 12677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.000s	sys 0.011s
W20260709 12:10:35.662410 14979 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.007s	sys 0.006s
W20260709 12:10:35.662997 14979 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.594s	user 0.007s	sys 0.006s
W20260709 12:10:35.650343 14741 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.007s	sys 0.005s
W20260709 12:10:35.650352 14804 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.505s	user 0.001s	sys 0.011s
I20260709 12:10:35.653788 15027 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:35.661408 14412 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.000s	sys 0.014s
W20260709 12:10:35.662457 15059 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.778s	user 0.020s	sys 0.046s
W20260709 12:10:35.664175 14953 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.068s	sys 0.081s
W20260709 12:10:35.671243 14939 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.008s	sys 0.008s
W20260709 12:10:35.710688 12675 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.006s	sys 0.009s
W20260709 12:10:35.711401 12675 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.569s	user 0.006s	sys 0.009s
W20260709 12:10:35.717373 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 271220us
W20260709 12:10:35.719563 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 270302us
W20260709 12:10:35.721822 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:35.666733 14993 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:35.694895 14741 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.007s	sys 0.006s
W20260709 12:10:35.691584 14804 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.001s	sys 0.012s
W20260709 12:10:35.719616 12570 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.015s	sys 0.036s
W20260709 12:10:35.723186 14412 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.000s	sys 0.015s
W20260709 12:10:35.751986 12570 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.015s	sys 0.037s
W20260709 12:10:35.757406 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:35.726323 14953 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.820s	user 0.068s	sys 0.082s
W20260709 12:10:35.764611 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:35.764848 14502 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:35.766373 14502 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:35.777153 12572 leader_election.cc:400] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4.
I20260709 12:10:35.777839 12572 leader_election.cc:403] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260709 12:10:35.677151 14958 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:35.795735 14412 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:35.799778 15012 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:35.800092 14412 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:35.801678 14796 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.913s	user 0.013s	sys 0.016s
W20260709 12:10:35.802680 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:35.803653 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:35.804792 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:35.810807 12603 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
mode: GRACEFUL
new_leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
 from {username='slave'} at 127.0.0.1:32832
I20260709 12:10:35.811630 12603 raft_consensus.cc:606] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Received request to transfer leadership to TS a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:35.812187 12603 raft_consensus.cc:614] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:35.815737 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:10:35.822032 15072 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:35.823411 14789 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:35.823288 15072 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:35.822392 15073 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:35.827236 15073 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:35.872265 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 95500.6us
I20260709 12:10:35.785214 14502 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:35.763291 14993 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:10:35.904412 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:35.907693 14980 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.903s	user 0.020s	sys 0.000s
I20260709 12:10:35.908893 12529 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:10:35.911235 14412 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:35.944859 12523 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:35.924167 12678 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:10:35.981020 12528 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:10:35.974407 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:10:35.974529 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 176055us
I20260709 12:10:35.903185 12496 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:36.031311 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 1633) took 1123 ms. Trace:
I20260709 12:10:36.032028 12523 rpcz_store.cc:276] 0709 12:10:34.903461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:34.944328 (+ 40867us) service_pool.cc:224] Handling call
0709 12:10:36.002229 (+1057901us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:35.973330 15023 consensus_meta.cc:287] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.864s	user 0.011s	sys 0.012s
W20260709 12:10:35.971200 15029 consensus_meta.cc:287] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.903s	user 0.016s	sys 0.004s
I20260709 12:10:36.030449 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 1631) took 1122 ms. Trace:
I20260709 12:10:35.912267 14733 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:35.974447 15094 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:35.974102 14240 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.840s	user 0.001s	sys 0.019s
I20260709 12:10:36.037902 12528 rpcz_store.cc:276] 0709 12:10:34.902371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:34.943960 (+ 41589us) service_pool.cc:224] Handling call
0709 12:10:36.003270 (+1059310us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:36.038563 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 93285.8us
I20260709 12:10:36.040126 14560 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:36.041106 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:36.042909 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:36.046001 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 142984us
I20260709 12:10:36.046962 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 1621) took 1153 ms. Trace:
I20260709 12:10:35.971509 15072 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:36.060655 15035 raft_consensus.cc:3055] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:36.062683 15092 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:36.063336 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:36.064378 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:36.066748 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:36.066711 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:36.098243 12525 rpcz_store.cc:276] 0709 12:10:34.893631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:35.049343 (+155712us) service_pool.cc:224] Handling call
0709 12:10:36.046651 (+997308us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b54003caad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:36.046867 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":593446784}
I20260709 12:10:36.067800 14993 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:10:36.071612 14971 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:36.072116 12676 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:36.073632 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:36.074729 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:36.075621 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:36.076651 15066 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.431s	user 0.004s	sys 0.000s
I20260709 12:10:36.076788 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:36.079301 15016 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.170s	user 0.023s	sys 0.006s
I20260709 12:10:36.079897 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:36.080332 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 442119us
W20260709 12:10:36.081961 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.074790 14733 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:36.086345 14958 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:36.087337 15073 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:35.970640 14879 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4.
I20260709 12:10:36.091246 15093 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:36.092165 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 1629) took 1190 ms. Trace:
I20260709 12:10:36.094817 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:36.100296 12643 leader_election.cc:400] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7.
I20260709 12:10:36.100559 14560 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:36.101347 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:36.102679 12572 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:36.129107 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:36.129854 12672 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:36.104511 12605 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 4: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:36.067615 12677 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:36.116076 12527 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:36.116546 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:36.116567 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:36.119479 12674 raft_consensus.cc:2410] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:10:36.089699 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:36.123453 12602 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260709 12:10:36.108124 15035 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:36.126580 12529 rpcz_store.cc:276] 0709 12:10:34.901802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:35.057779 (+155977us) service_pool.cc:224] Handling call
0709 12:10:36.091922 (+1034143us) spinlock_profiling.cc:243] Waited 850 ms on lock 0x7b5400408654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:36.092094 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1953850112}
I20260709 12:10:36.128903 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:36.138684 15093 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:36.139528 12643 leader_election.cc:400] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5.
I20260709 12:10:36.139961 12643 leader_election.cc:403] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:10:36.141573 15093 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:36.141995 14560 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:36.143281 12526 raft_consensus.cc:2393] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:10:36.143424 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.109875 15094 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:36.144250 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:36.144491 14905 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:36.142901 15035 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:36.142823 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:36.145598 14905 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:36.146664 15035 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:36.147020 12527 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:36.148645 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:36.148816 14905 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:36.149677 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.151690 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:36.144299 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:36.153745 12525 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:36.154522 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:36.155673 12525 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:36.156066 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:36.154043 12526 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:36.149966 12527 raft_consensus.cc:2410] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:10:36.143424 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:36.157783 13332 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:36.163008 12671 raft_consensus.cc:2393] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 4: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:36.166929 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:36.167212 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:36.166683 14486 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:36.131081 14971 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:36.178038 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:36.180064 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:36.185021 13338 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:36.187166 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:36.188895 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:36.192126 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 82361.2us
I20260709 12:10:36.175740 14560 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:36.179281 14486 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:36.202040 12524 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:36.199579 12526 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:36.200336 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:36.203694 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:36.207041 12676 raft_consensus.cc:2368] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4.
I20260709 12:10:36.204628 12674 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:36.208591 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:36.209349 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.209568 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:36.210489 12531 raft_consensus.cc:3055] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:10:36.210911 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.210913 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:36.187721 14240 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5.
I20260709 12:10:36.210860 12531 raft_consensus.cc:740] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:36.206929 12528 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:36.204540 14486 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:36.226648 13338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 327) took 1045 ms. Trace:
W20260709 12:10:36.226962 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 346988us
I20260709 12:10:36.227205 13338 rpcz_store.cc:276] 0709 12:10:35.181245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:35.182202 (+   957us) service_pool.cc:224] Handling call
0709 12:10:36.226285 (+1044083us) spinlock_profiling.cc:243] Waited 794 ms on lock 0x7b540047b8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:36.226476 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1825607808}
I20260709 12:10:36.228628 15029 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:36.229120 14741 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:36.230106 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:36.223477 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 150) took 1019 ms. Trace:
I20260709 12:10:36.231474 12598 rpcz_store.cc:276] 0709 12:10:35.204235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:35.204399 (+   164us) service_pool.cc:224] Handling call
0709 12:10:35.241650 (+ 37251us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:10:35.377746 (+136096us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:35.377826 (+    80us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:10:35.394578 (+ 16752us) log.cc:844] Serialized 78 byte log entry
0709 12:10:35.625512 (+230934us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:35.642693 (+ 17181us) log.cc:844] Serialized 14 byte log entry
0709 12:10:35.785849 (+143156us) log.cc:844] Serialized 14 byte log entry
0709 12:10:35.796897 (+ 11048us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:35.797089 (+   192us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:36.222920 (+425831us) raft_consensus.cc:1685] finished
0709 12:10:36.222944 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:36.223197 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":154046,"prepare.queue_time_us":11555,"prepare.run_cpu_time_us":120,"prepare.run_wall_time_us":121,"raft.queue_time_us":460051,"raft.run_cpu_time_us":2216,"raft.run_wall_time_us":128796,"spinlock_wait_cycles":29056,"thread_start_us":230202,"threads_started":1,"wal-append.queue_time_us":778618}
I20260709 12:10:36.219292 14891 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:36.224874 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:36.236460 12528 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:36.224042 12606 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
W20260709 12:10:36.239964 12494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.214s	user 0.029s	sys 0.041s
I20260709 12:10:36.212172 14965 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:36.225548 12524 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:36.251353 14965 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:36.223934 14733 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:36.250551 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:36.253829 12526 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:36.201742 15097 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:36.231194 12605 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:36.232988 12572 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:36.229346 12496 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:36.255795 12678 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:36.320264 15029 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:36.322113 12675 raft_consensus.cc:2410] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260709 12:10:36.237797 14891 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:36.334407 15029 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:36.365869 12527 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 7: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:36.366227 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1632) took 3260 ms. Trace:
I20260709 12:10:36.366866 12522 rpcz_store.cc:276] 0709 12:10:33.105918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:33.106151 (+   233us) service_pool.cc:224] Handling call
0709 12:10:33.106504 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:33.136282 (+ 29778us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 12:10:33.136323 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:33.154783 (+ 18460us) log.cc:844] Serialized 26 byte log entry
0709 12:10:36.044069 (+2889286us) spinlock_profiling.cc:243] Waited 2.26 ms on lock 0x7b18003c2c80. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:10:36.192714 (+148645us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:36.197019 (+  4305us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:36.203657 (+  6638us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:36.365556 (+161899us) raft_consensus.cc:1685] finished
0709 12:10:36.365586 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:36.365916 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25361,"raft.queue_time_us":62614,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":9527,"spinlock_wait_cycles":5209344,"thread_start_us":116038,"threads_started":1,"wal-append.queue_time_us":3193323}
I20260709 12:10:36.370229 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1776) took 2279 ms. Trace:
I20260709 12:10:36.203429 15094 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:36.370896 12679 rpcz_store.cc:276] 0709 12:10:34.090943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:34.192562 (+101619us) service_pool.cc:224] Handling call
0709 12:10:34.193228 (+   666us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:34.193850 (+   622us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:10:34.419954 (+226104us) log.cc:844] Serialized 14 byte log entry
0709 12:10:35.669631 (+1249677us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:35.712918 (+ 43287us) log.cc:844] Serialized 26 byte log entry
0709 12:10:35.713181 (+   263us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:35.713223 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:35.713375 (+   152us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:36.075550 (+362175us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:36.369710 (+294160us) raft_consensus.cc:1685] finished
0709 12:10:36.369847 (+   137us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:36.370072 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":733557,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":81998,"raft.run_cpu_time_us":696,"raft.run_wall_time_us":56926,"spinlock_wait_cycles":61976832,"thread_start_us":1422640,"threads_started":3,"wal-append.queue_time_us":1127661}
I20260709 12:10:36.346493 15076 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:36.348839 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:36.376642 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:36.378420 12494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.355s	user 0.029s	sys 0.041s
W20260709 12:10:36.378976 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 2.35788e+06us
I20260709 12:10:36.291805 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:36.400843 15100 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:10:36.402601 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 263300us
I20260709 12:10:36.407416 12677 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:36.410995 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:36.412209 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.341243 12671 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:36.209425 15093 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.101s
W20260709 12:10:36.439721 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.446065 14486 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:36.383494 14891 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:36.447906 14733 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:36.450194 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 217403us
I20260709 12:10:36.450595 15011 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:36.450899 12531 consensus_queue.cc:260] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:36.218295 14971 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:36.458894 12672 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
W20260709 12:10:36.459443 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 79423us
W20260709 12:10:36.477571 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.452721 12676 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:36.473683 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:36.481092 13305 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:36.502943 12678 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:36.511374 12674 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:10:36.512562 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.509572 12675 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
W20260709 12:10:36.514355 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.508169 12646 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:36.518208 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:36.520843 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 45103.9us
I20260709 12:10:36.501149 14741 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:36.524317 12671 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:10:36.525475 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 296282us
I20260709 12:10:36.529002 12679 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:10:36.530203 12678 raft_consensus.cc:2410] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:36.535544 14788 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:36.535655 14885 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.056s
I20260709 12:10:36.536208 14788 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:36.536628 14788 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:36.535822 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:36.539922 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:36.529771 15094 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:10:36.545775 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 86920.8us
I20260709 12:10:36.541358 14939 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 12:10:36.548895 12643 leader_election.cc:400] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 3. Current term is 4.
I20260709 12:10:36.549284 12643 leader_election.cc:403] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:10:36.550204 14939 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:36.550866 15026 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:36.551507 15026 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:36.552786 12531 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:36.553457 14801 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 3. Current term is 4.
I20260709 12:10:36.554419 12526 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:10:36.544504 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 341274us
I20260709 12:10:36.556355 14971 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:36.558838 12572 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:36.560362 15076 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:36.561384 15076 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:36.561724 15076 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:36.563843 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:36.564325 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.545279 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:36.529452 15011 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:36.562244 15029 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:36.562311 12572 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:36.573236 14486 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:36.575692 15027 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.849s	user 0.013s	sys 0.001s
I20260709 12:10:36.544972 15094 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:36.583343 12496 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:36.562106 15076 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:36.585752 12569 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:36.586513 14980 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:36.587157 14980 raft_consensus.cc:697] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:36.587760 15073 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:36.588145 14971 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:36.588784 14971 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:36.588214 14980 consensus_queue.cc:237] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:36.589191 14971 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:36.591128 13338 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:10:36.580054 13305 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:36.579586 15094 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:36.593947 14486 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:36.596796 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:36.598889 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:36.601747 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:36.604075 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 55392.8us
I20260709 12:10:36.608582 14891 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:36.613040 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:36.597781 15035 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:36.602984 15023 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:36.611364 15072 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.014s
I20260709 12:10:36.626592 12523 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:36.630582 13305 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:36.632637 15093 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:36.633481 14891 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:36.633083 14885 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:10:36.633807 15093 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:36.635179 15093 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:36.634334 14885 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:36.643121 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:36.610196 15076 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:36.638291 15088 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 12:10:36.630079 15011 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:36.650996 15027 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260709 12:10:36.654907 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:36.649360 15083 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.498s	user 0.001s	sys 0.019s
I20260709 12:10:36.640574 14885 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:10:36.656656 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 123039us
I20260709 12:10:36.660753 15026 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:36.662993 12646 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:36.663506 14955 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:36.647672 14788 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:36.687706 14955 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:36.689604 14788 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:36.690833 14891 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:10:36.691429 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:36.692169 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:36.692463 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.692619 14891 raft_consensus.cc:2764] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:10:36.692788 12522 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:36.654753 15073 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:36.671381 14691 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:36.698251 12496 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:36.676527 12531 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
W20260709 12:10:36.678184 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50533.7us
I20260709 12:10:36.694911 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:36.699404 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:36.699760 12598 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:10:36.701228 14993 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:36.728873 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:36.729034 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:36.709254 14691 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:36.642333 14971 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:36.729941 12679 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:10:36.731709 15092 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:36.734534 14885 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:36.735839 14240 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:36.736316 14885 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:36.737017 14801 consensus_queue.cc:1059] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:36.708413 12677 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:10:36.747136 12522 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:36.751714 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:36.752498 12678 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
W20260709 12:10:36.754683 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 65854.1us
I20260709 12:10:36.755008 13305 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:36.733623 14971 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:36.749752 14733 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:36.708039 15072 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:36.706471 15073 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:36.764869 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 70601.6us
I20260709 12:10:36.706651 15088 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:36.707440 14955 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:36.775421 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:36.774967 13333 raft_consensus.cc:2376] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5. Re-sending same reply.
W20260709 12:10:36.791209 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 162688us
I20260709 12:10:36.785005 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:36.792641 12598 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:36.764245 15011 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:36.784075 12643 leader_election.cc:400] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7.
I20260709 12:10:36.771453 14741 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:36.776212 15073 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:36.799583 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:36.768957 14939 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:36.786549 15088 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:36.804831 12605 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
W20260709 12:10:36.786741 15016 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.013s	sys 0.037s
I20260709 12:10:36.810652 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:36.810586 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:36.812816 12572 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:36.813738 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:36.814604 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:36.814864 15072 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:36.815482 15072 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:36.815907 15072 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:36.818683 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:36.819641 12679 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:36.804497 15011 raft_consensus.cc:697] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 5 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:36.820943 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:36.821638 12671 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:36.826309 14904 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:36.829313 12606 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:36.834028 12572 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:36.834447 15114 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:10:36.835795 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:36.837091 15011 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:36.839087 14240 raft_consensus.cc:3055] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:36.840513 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:36.839247 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:36.842615 12597 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 5. Ops: []
I20260709 12:10:36.844082 12603 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
W20260709 12:10:36.834501 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 63855.6us
I20260709 12:10:36.846881 12643 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:36.847234 15068 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.605s	user 0.000s	sys 0.004s
I20260709 12:10:36.838181 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:36.809981 14741 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:36.848397 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:36.842660 12602 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:36.850075 13302 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:36.838697 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:36.851409 12606 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:36.809419 15088 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:36.857306 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:36.862619 14788 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:36.863369 14788 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:36.864532 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:36.864825 12678 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 3.
W20260709 12:10:36.808678 15016 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.013s	sys 0.038s
I20260709 12:10:36.848135 15092 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:36.807299 14691 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:36.894531 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:36.887738 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:36.872515 12600 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:36.825699 15094 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:36.886191 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:36.694732 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:36.898299 12531 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 6: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:10:36.977728 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:36.929023 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.897979 12671 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:36.885149 13336 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:10:36.974668 12602 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
W20260709 12:10:36.993377 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:36.995267 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:36.873910 14788 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:36.998436 12672 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:36.998919 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:37.001147 13333 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 6: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:36.998564 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 149) took 1665 ms. Trace:
I20260709 12:10:36.854951 14240 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:10:36.868395 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 129203us
I20260709 12:10:36.887296 13334 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 2 LEARNER]: Advancing to term 4
I20260709 12:10:37.000015 15094 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:37.010208 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:37.010596 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 174901us
I20260709 12:10:37.011212 12677 raft_consensus.cc:2376] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4. Re-sending same reply.
W20260709 12:10:36.888736 15098 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.738s	user 0.016s	sys 0.007s
I20260709 12:10:36.869448 15129 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:36.879865 15035 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:36.869692 14971 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:36.886351 14486 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:36.878224 14955 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:37.018316 12671 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:10:36.882488 14953 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:37.013239 14891 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260709 12:10:37.023469 14953 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:37.023607 15008 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:37.027760 15117 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:10:36.983361 15093 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:37.030221 15012 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:37.030396 13332 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:37.030687 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:37.030843 15012 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:37.032011 13331 raft_consensus.cc:2368] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6.
I20260709 12:10:37.033485 14971 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:37.034379 13334 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 4 LEARNER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:10:37.038513 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:37.038820 12605 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:37.039337 12676 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 4: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:36.849054 14939 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:37.039860 12572 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:37.041400 12572 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:37.041982 14939 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:37.011662 14885 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:37.021806 14691 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:37.044713 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 196444us
I20260709 12:10:37.007321 15026 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:37.001919 12604 rpcz_store.cc:276] 0709 12:10:35.203900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:35.204060 (+   160us) service_pool.cc:224] Handling call
0709 12:10:35.204464 (+   404us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:35.204904 (+   440us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:35.204932 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:35.241805 (+ 36873us) log.cc:844] Serialized 52 byte log entry
0709 12:10:36.557333 (+1315528us) spinlock_profiling.cc:243] Waited 4.64 ms on lock 0x7b18001e3dc0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:10:36.648429 (+ 91096us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:36.769854 (+121425us) log.cc:844] Serialized 14 byte log entry
0709 12:10:36.771296 (+  1442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:36.797228 (+ 25932us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:36.850649 (+ 53421us) raft_consensus.cc:1685] finished
0709 12:10:36.865944 (+ 15295us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:36.868864 (+  2920us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":149505,"prepare.queue_time_us":169240,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":18909,"raft.run_cpu_time_us":277,"raft.run_wall_time_us":16331,"spinlock_wait_cycles":10661120,"thread_start_us":205095,"threads_started":2,"wal-append.queue_time_us":1558875}
I20260709 12:10:37.031740 15093 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:37.033929 14971 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:37.061372 15023 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:37.061262 14240 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:37.063093 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1619) took 2169 ms. Trace:
I20260709 12:10:37.067265 15126 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:37.069389 14741 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:37.072340 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1792) took 1908 ms. Trace:
I20260709 12:10:37.073671 12680 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:37.037748 15088 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:37.060042 15026 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:37.079509 14968 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:37.038543 13330 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:37.086537 12600 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:37.043177 14955 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:37.114516 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:10:37.098429 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 227565us
I20260709 12:10:37.058301 14486 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:37.102989 12673 rpcz_store.cc:276] 0709 12:10:35.165213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:35.165788 (+   575us) service_pool.cc:224] Handling call
0709 12:10:35.166092 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:35.527082 (+360990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:37.071800 (+1544718us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b54000131d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:37.072146 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":507963264}
I20260709 12:10:37.125082 14965 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:37.119225 14955 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:37.126765 14971 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:37.127851 14971 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:37.129631 14240 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:37.132282 15073 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:37.129693 12494 connection.cc:256] RPC call timeout handler was delayed by -1.890151355000171s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16990000000000002s for a response
I20260709 12:10:37.124516 15126 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:37.144539 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 46331.7us
I20260709 12:10:37.109028 14968 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:37.159777 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:37.143738 14891 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:37.146847 14939 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:37.101050 12530 rpcz_store.cc:276] 0709 12:10:34.893070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:35.150803 (+257733us) service_pool.cc:224] Handling call
0709 12:10:35.151227 (+   424us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:35.151724 (+   497us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:35.675223 (+523499us) log.cc:844] Serialized 14 byte log entry
0709 12:10:35.783791 (+108568us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:36.248647 (+464856us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b2c00058028. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:10:36.249468 (+   821us) log.cc:844] Serialized 52 byte log entry
0709 12:10:36.249637 (+   169us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:36.249673 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:36.249754 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:36.887932 (+638178us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:37.062690 (+174758us) raft_consensus.cc:1685] finished
0709 12:10:37.062715 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:37.062903 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4635,"prepare.queue_time_us":796434,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":65,"raft.queue_time_us":288,"raft.run_cpu_time_us":492,"raft.run_wall_time_us":68119,"spinlock_wait_cycles":80295552,"thread_start_us":523969,"threads_started":2,"wal-append.queue_time_us":436447}
I20260709 12:10:37.158042 12646 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:37.166829 12678 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:37.143534 14980 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:37.177699 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 86687.2us
I20260709 12:10:37.147150 15072 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:37.183058 12600 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260709 12:10:37.186486 12606 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
W20260709 12:10:37.189118 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:37.194514 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:37.201031 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:37.202488 14968 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:37.203518 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:37.204476 15076 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:37.204828 12603 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:37.205261 15076 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:37.205982 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:37.206470 14885 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:37.206719 12597 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:37.168761 15132 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:37.210783 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:37.219614 12601 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:10:37.202507 14240 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:37.221302 14240 raft_consensus.cc:697] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:10:37.222205 14958 consensus_queue.cc:1175] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" responder_term: 5 status { last_received { term: 3 index: 3 } last_committed_idx: 3 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:10:37.222252 14240 consensus_queue.cc:237] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:37.223929 15137 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:37.227550 12597 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:37.214586 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:37.215346 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:37.216297 12600 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:37.201126 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:37.216982 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:37.180791 15073 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:37.242861 12602 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:37.243257 12604 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:37.243480 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:37.217882 13330 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:37.178979 14939 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:37.182303 14891 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:37.299281 15134 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.123s	user 0.002s	sys 0.000s
I20260709 12:10:37.300225 15026 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:37.306927 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:37.310187 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:37.310483 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:37.311245 12677 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:37.311254 12671 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:37.309434 12604 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:37.286265 14885 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:37.285969 15139 consensus_queue.cc:1059] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:37.194087 14980 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:37.313153 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:37.316681 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 112350us
I20260709 12:10:37.320811 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:37.323192 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:37.324026 12530 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:37.334187 12526 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:10:37.315811 15092 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:37.340658 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:37.341266 12524 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:37.338059 12522 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:37.338290 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:37.338892 12531 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:37.346923 12671 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
W20260709 12:10:37.286021 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 186780us
W20260709 12:10:37.348742 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 212612us
I20260709 12:10:37.322860 15076 raft_consensus.cc:697] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:37.359628 12672 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:10:37.353575 12571 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:37.355543 15126 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:37.334543 15117 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:37.317490 14412 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 12:10:37.340092 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:37.382412 12524 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:10:37.384644 15076 consensus_queue.cc:237] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:37.386337 13304 leader_election.cc:336] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.944s (SENT)
I20260709 12:10:37.392225 14879 raft_consensus.cc:3055] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:37.392544 15143 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:37.393502 12572 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:37.348312 15088 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:10:37.346203 15132 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:10:37.400112 13330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38472 (request call id 207) took 3355 ms (client timeout 3036 ms). Trace:
I20260709 12:10:37.394550 12427 catalog_manager.cc:5697] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:37.401376 15141 raft_consensus.cc:3055] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:37.408530 12602 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:37.424954 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:37.425410 12671 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:10:37.395714 14486 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:10:37.427361 14891 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:37.427567 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1599) took 1326 ms. Trace:
I20260709 12:10:37.427660 14939 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:37.400166 15093 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:10:37.397295 15117 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:37.474730 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:37.511204 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:37.518333 13305 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:10:37.519160 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 147550us
I20260709 12:10:37.443724 14879 raft_consensus.cc:740] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:37.512943 12680 rpcz_store.cc:276] 0709 12:10:36.101673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:36.489320 (+387647us) service_pool.cc:224] Handling call
0709 12:10:37.073192 (+583872us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:37.103543 (+ 30351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:37.427363 (+323820us) spinlock_profiling.cc:243] Waited 575 ms on lock 0x7b54000131d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:37.427500 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1322244864}
I20260709 12:10:37.552742 14486 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:37.561309 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:37.563094 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 226698us
I20260709 12:10:37.474961 15139 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:37.534814 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:37.512902 15073 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:37.530912 15072 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:37.512111 15141 raft_consensus.cc:740] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:10:38.125241 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 568851us
W20260709 12:10:37.901398 12429 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59006 (request call id 174) took 14834 ms (client timeout 15000 ms). Trace:
W20260709 12:10:37.987352 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 426909us
W20260709 12:10:38.129750 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:38.042508 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:38.131091 12671 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:37.527535 15143 consensus_queue.cc:1243] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 5.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:10:38.125669 14240 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 1, Time since last communication: 0.007s
W20260709 12:10:38.127283 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 740013us
I20260709 12:10:37.642683 12672 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 7: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:10:37.617044 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:37.763978 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:38.142421 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:38.050350 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:37.677618 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 334401us
W20260709 12:10:37.798815 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 424447us
W20260709 12:10:37.912572 12530 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.588s	user 0.011s	sys 0.000s
I20260709 12:10:38.146833 12530 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
W20260709 12:10:38.135938 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:37.703670 14955 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.577s	user 0.008s	sys 0.004s
W20260709 12:10:37.864787 14971 consensus_meta.cc:287] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.736s	user 0.009s	sys 0.005s
I20260709 12:10:37.902143 12427 catalog_manager.cc:5697] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 3 to 4, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 4 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:38.210595 14955 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:38.211863 12427 catalog_manager.cc:5697] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 3 to 4, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 4 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:10:38.176753 14879 consensus_queue.cc:260] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:38.190908 14939 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.024s
I20260709 12:10:38.213666 12427 catalog_manager.cc:5697] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 3 to 4, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 4 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:38.214722 14939 consensus_queue.cc:1243] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:10:38.214941 14955 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:38.214501 14879 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:38.215314 12427 catalog_manager.cc:5697] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 3 to 4, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 4 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:38.194975 15141 consensus_queue.cc:260] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:38.128695 12429 rpcz_store.cc:269] 0709 12:10:23.067223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:23.067491 (+   268us) service_pool.cc:224] Handling call
0709 12:10:23.584601 (+517110us) spinlock_profiling.cc:243] Waited 98.1 ms on lock 0x7b54000104d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148898a090 00007f1488a28f3a 00007f148e7dee9c 00007f148e8dd04e 00007f148cd9e2a4 00007f148cd9e232 00007f148cd9e161 00007f148cd9e0dc 00007f148cd9d392 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:37.901067 (+14316466us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:32.967847 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:33.058959 (+ 91112us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:33.086206 (+ 27247us) write_op.cc:435] Acquired schema lock
0709 12:10:33.086226 (+    20us) tablet.cc:662] Decoding operations
0709 12:10:33.206609 (+120383us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:33.218424 (+ 11815us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:33.223136 (+  4712us) tablet.cc:685] Acquiring locks for 16 operations
0709 12:10:33.632881 (+409745us) tablet.cc:701] Row locks acquired
0709 12:10:33.636091 (+  3210us) write_op.cc:260] PREPARE: finished
0709 12:10:33.653610 (+ 17519us) write_op.cc:270] Start()
0709 12:10:33.662340 (+  8730us) write_op.cc:276] Timestamp: P: 1783599033647478 usec, L: 0
0709 12:10:33.664875 (+  2535us) op_driver.cc:348] REPLICATION: starting
0709 12:10:33.777028 (+112153us) log.cc:844] Serialized 6336 byte log entry
0709 12:10:34.884440 (+1107412us) op_driver.cc:464] REPLICATION: finished
0709 12:10:35.045625 (+161185us) spinlock_profiling.cc:243] Waited 12.9 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14859160ac 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f148c456ef0
0709 12:10:35.086921 (+ 41296us) spinlock_profiling.cc:243] Waited 22.8 ms on lock 0x7b54000104d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1be28 00007f148c456581 00007f148c455d58 00007f148c45a9f1 00007f148c45a9a9 00007f148c45a939 00007f148c45a901 00007f148c459b2d 0000000000388634 0000000000388479
0709 12:10:35.721103 (+634182us) write_op.cc:301] APPLY: starting
0709 12:10:35.721323 (+   220us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:35.721814 (+   491us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:35.721874 (+    60us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:36.059462 (+337588us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:36.070686 (+ 11224us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=16
0709 12:10:36.070816 (+   130us) write_op.cc:312] APPLY: finished
0709 12:10:36.071509 (+   693us) log.cc:844] Serialized 112 byte log entry
0709 12:10:36.197158 (+125649us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:36.231846 (+ 34688us) spinlock_profiling.cc:243] Waited 2.54 ms on lock 0x7b18000778c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f148c456ef0 00007f148c457489 00007f148c45971a
0709 12:10:36.370317 (+138471us) write_op.cc:454] Released schema lock
0709 12:10:36.370448 (+   131us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:36.532902 (+162454us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:36.538954 (+  6052us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1119794,"reported_tablets":30,"spinlock_wait_cycles":225716992,"child_traces":[["op",{"apply.queue_time_us":814126,"apply.run_cpu_time_us":38682,"apply.run_wall_time_us":804648,"num_ops":16,"prepare.queue_time_us":257176,"prepare.run_cpu_time_us":42394,"prepare.run_wall_time_us":2137020,"raft.queue_time_us":689298,"raft.run_cpu_time_us":73837,"raft.run_wall_time_us":1348305,"replication_time_us":1219382,"spinlock_wait_cycles":88021504,"thread_start_us":1504363,"threads_started":5,"wal-append.queue_time_us":441338,"wal-append.run_cpu_time_us":43978,"wal-append.run_wall_time_us":2872103}]]}
I20260709 12:10:38.182320 15093 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:37.832316 14691 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.785s	user 0.013s	sys 0.000s
I20260709 12:10:38.217692 12605 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
W20260709 12:10:37.867226 15129 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.840s	user 0.011s	sys 0.001s
I20260709 12:10:38.217764 15093 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:38.218995 15129 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:38.017673 14968 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:38.173524 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:38.131891 15116 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.522s	user 0.007s	sys 0.009s
I20260709 12:10:37.835863 15117 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:37.848517 15012 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:38.172667 14788 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:38.131630 14486 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:38.047827 14891 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:38.268574 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:38.136443 15073 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:38.242697 14885 raft_consensus.cc:3055] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:38.132385 15142 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:38.278471 12531 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 3.
I20260709 12:10:38.279016 15142 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:38.280102 15008 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:38.280619 15008 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:38.280934 13305 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:38.186311 12572 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:38.281872 15012 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:38.278537 14885 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:38.182066 12643 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:38.282523 15012 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:38.283018 15012 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:38.283577 15088 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:38.283191 14885 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:38.284009 15088 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:38.284466 15088 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:38.284734 14885 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:38.285665 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:38.283241 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 101305us
I20260709 12:10:38.286576 15093 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:37.908969 15140 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.482s	user 0.002s	sys 0.000s
I20260709 12:10:38.287313 14971 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:38.287793 14971 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:38.187801 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:38.288173 14971 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:38.288765 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:38.194397 13304 leader_election.cc:336] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.102s (SENT)
I20260709 12:10:38.287130 12643 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:38.194432 12494 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:10:38.282526 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 109444us
I20260709 12:10:38.217095 15141 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:37.562562 15023 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:38.217078 12427 catalog_manager.cc:5697] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 5 to 6, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 6 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:37.750008 14980 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:38.243968 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:37.753319 15146 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:38.260293 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:38.263583 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:38.298938 12523 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:38.178378 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:38.299829 12680 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:38.267791 14788 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:38.275315 14486 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:38.276413 14891 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:38.268947 14993 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:37.643350 15132 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:38.195401 15027 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.070s
I20260709 12:10:38.309938 14691 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:38.188530 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:38.209414 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:38.221091 12671 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:38.290112 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:38.294874 15023 raft_consensus.cc:697] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 6 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:38.323788 12672 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:38.297114 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1809) took 1017 ms. Trace:
W20260709 12:10:37.513134 13330 rpcz_store.cc:269] 0709 12:10:34.044845 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:34.077227 (+ 32382us) service_pool.cc:224] Handling call
0709 12:10:37.399826 (+3322599us) spinlock_profiling.cc:243] Waited 2.96 s on lock 0x7b540061f654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:37.400013 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6807512832}
I20260709 12:10:38.327630 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:38.281708 15142 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:38.323133 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 129110us
I20260709 12:10:38.345415 12494 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:10:38.350267 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:38.350440 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 63424us
I20260709 12:10:38.351969 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:38.353096 14891 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:38.353048 12672 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:38.355283 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:38.356114 12672 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:38.356253 15150 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.232s	user 0.000s	sys 0.001s
I20260709 12:10:38.345793 12680 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 7: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:10:38.363003 15155 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:10:38.281646 15008 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.2, Last appended by leader: 2, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:38.364746 15008 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:38.315032 14993 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:38.371531 14993 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:38.371907 15162 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.248s	user 0.000s	sys 0.002s
I20260709 12:10:38.377239 12672 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:10:38.377504 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1805) took 1280 ms. Trace:
W20260709 12:10:38.379462 15147 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.335s	user 0.001s	sys 0.001s
I20260709 12:10:38.380558 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1808) took 1182 ms. Trace:
I20260709 12:10:38.380909 15023 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:38.385110 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 1769) took 1208 ms. Trace:
W20260709 12:10:38.386879 15161 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.262s	user 0.000s	sys 0.002s
I20260709 12:10:38.315989 15139 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:38.463758 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:38.324694 12675 rpcz_store.cc:276] 0709 12:10:37.280202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:37.280416 (+   214us) service_pool.cc:224] Handling call
0709 12:10:37.291667 (+ 11251us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:37.314325 (+ 22658us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:37.314359 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:37.319664 (+  5305us) log.cc:844] Serialized 26 byte log entry
0709 12:10:37.532280 (+212616us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:37.532322 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:37.532536 (+   214us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:38.296767 (+764231us) raft_consensus.cc:1685] finished
0709 12:10:38.296802 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:38.297021 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":381,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":12414,"thread_start_us":177587,"threads_started":1,"wal-append.queue_time_us":931038}
I20260709 12:10:38.359684 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:38.347059 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:38.398972 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:38.310824 15117 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:38.402523 15073 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:38.406100 14788 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:38.396299 14486 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:38.409008 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 214635us
W20260709 12:10:38.421227 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 75884.3us
I20260709 12:10:38.400934 14891 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:38.466528 12671 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
W20260709 12:10:38.476549 13304 leader_election.cc:336] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.635s (SENT)
I20260709 12:10:38.476897 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:38.477078 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:38.348158 15093 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:38.440670 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 250223us
I20260709 12:10:38.477669 12671 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 3.
I20260709 12:10:38.448659 12679 rpcz_store.cc:276] 0709 12:10:37.097246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:37.097381 (+   135us) service_pool.cc:224] Handling call
0709 12:10:37.126243 (+ 28862us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:37.126778 (+   535us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 12:10:37.126814 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:37.127354 (+   540us) log.cc:844] Serialized 26 byte log entry
0709 12:10:37.383104 (+255750us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:37.383149 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:37.383220 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:38.377169 (+993949us) raft_consensus.cc:1685] finished
0709 12:10:38.377197 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:38.377424 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12116,"raft.queue_time_us":13064,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":4812,"thread_start_us":247207,"threads_started":1,"wal-append.queue_time_us":1046772}
I20260709 12:10:38.458185 12646 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:38.459290 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:38.448904 12678 rpcz_store.cc:276] 0709 12:10:37.197940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:37.198111 (+   171us) service_pool.cc:224] Handling call
0709 12:10:37.277401 (+ 79290us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:37.396433 (+119032us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:10:37.564223 (+167790us) log.cc:844] Serialized 14 byte log entry
0709 12:10:38.208374 (+644151us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:38.210357 (+  1983us) log.cc:844] Serialized 26 byte log entry
0709 12:10:38.210533 (+   176us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:38.210566 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:38.210644 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:38.380295 (+169651us) raft_consensus.cc:1685] finished
0709 12:10:38.380317 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:38.380467 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59532,"prepare.queue_time_us":31690,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":97421,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":23002,"thread_start_us":647993,"threads_started":3,"wal-append.queue_time_us":587242}
I20260709 12:10:38.437494 15125 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:38.485438 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:38.462217 12523 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:38.457785 12605 rpcz_store.cc:276] 0709 12:10:37.176944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:37.515636 (+338692us) service_pool.cc:224] Handling call
0709 12:10:38.384876 (+869240us) spinlock_profiling.cc:243] Waited 89.1 ms on lock 0x7b5400375354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:38.385006 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":205031680}
W20260709 12:10:38.393006 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 186030us
I20260709 12:10:38.467376 12672 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:38.480093 12494 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:38.488328 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:38.489203 12675 raft_consensus.cc:2410] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:10:38.476035 15126 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:38.497277 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1804) took 1474 ms. Trace:
I20260709 12:10:38.503633 12676 rpcz_store.cc:276] 0709 12:10:37.054696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:37.058216 (+  3520us) service_pool.cc:224] Handling call
0709 12:10:37.071118 (+ 12902us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:37.072366 (+  1248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:37.072409 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:37.493357 (+420948us) spinlock_profiling.cc:243] Waited 9.1 ms on lock 0x7b2c00037868. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:10:37.554197 (+ 60840us) log.cc:844] Serialized 52 byte log entry
0709 12:10:38.040725 (+486528us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:38.040787 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:38.040912 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:38.386181 (+345269us) spinlock_profiling.cc:243] Waited 24.9 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:38.496883 (+110702us) raft_consensus.cc:1685] finished
0709 12:10:38.496930 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:38.497166 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2960,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":1143,"spinlock_wait_cycles":78311552,"thread_start_us":297725,"threads_started":1,"wal-append.queue_time_us":445749}
W20260709 12:10:38.508489 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 109641us
I20260709 12:10:38.509888 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:38.511096 12672 raft_consensus.cc:2410] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:10:38.512645 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:38.485356 14691 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:38.485576 15117 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:38.515646 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:38.512960 12646 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:38.522738 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:38.524067 12679 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:38.490468 14891 raft_consensus.cc:697] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:38.525079 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48555us
I20260709 12:10:38.512454 14788 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:38.508085 14980 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:38.549894 14240 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:38.546880 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 65770.8us
I20260709 12:10:38.550833 14240 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:38.490633 15125 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:38.559203 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58493.8us
I20260709 12:10:38.549661 14691 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:38.570951 14691 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:38.524884 15093 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:38.566679 14891 consensus_queue.cc:237] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:38.560048 15094 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:38.564453 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:38.585723 12679 raft_consensus.cc:2410] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260709 12:10:38.577140 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 59633.6us
I20260709 12:10:38.500283 14971 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:38.581545 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:38.560482 15073 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:38.547472 15152 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:38.584138 15093 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:38.550020 14980 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:38.596778 14240 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:38.604413 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:38.612324 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:38.613444 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:38.614265 12679 raft_consensus.cc:2393] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:38.615060 12571 leader_election.cc:400] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 3 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4.
I20260709 12:10:38.599011 14980 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:38.596187 15094 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:38.617682 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:38.584302 14939 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.030s
I20260709 12:10:38.592950 15143 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:10:38.619485 15152 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:38.601563 15027 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.046s
W20260709 12:10:38.658804 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 353825us
I20260709 12:10:38.645455 15129 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:38.615061 14965 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.034s	sys 0.053s
I20260709 12:10:38.678074 12571 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:38.671571 12672 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:38.663923 14993 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:38.674563 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 72511.9us
I20260709 12:10:38.674613 12675 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
W20260709 12:10:38.674607 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 138666us
I20260709 12:10:38.640452 15168 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:38.669927 14741 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:38.684818 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 188) took 1156 ms. Trace:
I20260709 12:10:38.690876 12675 rpcz_store.cc:276] 0709 12:10:37.528646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:38.618251 (+1089605us) service_pool.cc:224] Handling call
0709 12:10:38.662092 (+ 43841us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:38.683902 (+ 21810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:38.684759 (+   857us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:38.686966 15125 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:38.685673 13304 leader_election.cc:336] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.703s (SENT)
I20260709 12:10:38.690325 15132 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:38.678444 14965 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.035s	sys 0.054s
I20260709 12:10:38.645143 15117 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:38.701936 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 309154us
W20260709 12:10:38.704025 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 122603us
I20260709 12:10:38.704098 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:38.704499 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:38.705349 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:38.690594 14741 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:38.709203 13304 leader_election.cc:336] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.794s (SENT)
I20260709 12:10:38.690653 15168 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:38.713758 12523 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:38.714282 13304 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:38.680351 15129 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:38.722800 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:38.721370 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:10:38.721318 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 109051us
I20260709 12:10:38.727789 15023 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:38.712334 15073 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:38.729207 15073 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:38.717537 14741 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:38.730928 15166 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:10:38.731760 15073 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:38.732559 13330 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:38.736460 15125 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:38.736730 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:38.737694 14965 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:38.738241 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:38.739094 12524 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:10:38.706959 15094 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:38.740664 15093 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:38.693135 14993 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:38.743943 14993 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:38.746884 15169 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:10:38.748903 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1815) took 1221 ms. Trace:
W20260709 12:10:38.749572 15167 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:10:38.749364 12677 rpcz_store.cc:276] 0709 12:10:37.527598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:37.622332 (+ 94734us) service_pool.cc:224] Handling call
0709 12:10:37.704252 (+ 81920us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:38.191295 (+487043us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:38.191623 (+   328us) log.cc:844] Serialized 14 byte log entry
0709 12:10:38.447259 (+255636us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:38.468662 (+ 21403us) log.cc:844] Serialized 26 byte log entry
0709 12:10:38.468864 (+   202us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:38.468898 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:38.468980 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:38.748636 (+279656us) raft_consensus.cc:1685] finished
0709 12:10:38.748658 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:38.748820 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":486503,"prepare.queue_time_us":133,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":489027,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":7106,"thread_start_us":250548,"threads_started":1,"wal-append.queue_time_us":453000}
I20260709 12:10:38.751489 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:38.752024 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1813) took 1338 ms. Trace:
I20260709 12:10:38.753059 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:38.752393 12673 rpcz_store.cc:276] 0709 12:10:37.413893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:37.414068 (+   175us) service_pool.cc:224] Handling call
0709 12:10:37.414533 (+   465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:38.041996 (+627463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:38.042032 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:38.148326 (+106294us) log.cc:844] Serialized 26 byte log entry
0709 12:10:38.439894 (+291568us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:38.439963 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:38.440080 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:38.751695 (+311615us) raft_consensus.cc:1685] finished
0709 12:10:38.751727 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:38.751947 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":61203,"raft.queue_time_us":627263,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":60740,"thread_start_us":253292,"threads_started":1,"wal-append.queue_time_us":469129}
I20260709 12:10:38.753912 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:38.757056 15175 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:38.757906 15175 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:38.759352 15175 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:38.760828 15072 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:38.764004 12571 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:38.718251 15168 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:38.766494 15072 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:38.770241 15132 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:38.784997 12525 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:38.726425 15152 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:38.787120 12643 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:10:38.784886 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:38.807761 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:38.818830 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 91998.2us
W20260709 12:10:38.818173 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 112911us
W20260709 12:10:38.819653 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:38.820468 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:38.825816 15126 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:38.826395 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:38.788213 14486 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:38.809326 15149 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:38.802772 15143 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:38.822785 14891 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:10:38.818799 15173 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.122s	user 0.001s	sys 0.001s
I20260709 12:10:38.865250 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1814) took 1416 ms. Trace:
I20260709 12:10:38.865648 12674 rpcz_store.cc:276] 0709 12:10:37.449194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:37.449407 (+   213us) service_pool.cc:224] Handling call
0709 12:10:37.490952 (+ 41545us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:38.202716 (+711764us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:38.202767 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:38.382480 (+179713us) spinlock_profiling.cc:243] Waited 3.61 ms on lock 0x7b2c00037868. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:10:38.382921 (+   441us) log.cc:844] Serialized 26 byte log entry
0709 12:10:38.642312 (+259391us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:38.642906 (+   594us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:38.643014 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:38.864892 (+221878us) raft_consensus.cc:1685] finished
0709 12:10:38.864917 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:38.865177 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":649069,"raft.queue_time_us":682949,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":99,"spinlock_wait_cycles":8297856,"thread_start_us":258916,"threads_started":1,"wal-append.queue_time_us":286351}
I20260709 12:10:38.869451 15129 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:38.838572 15076 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:38.871600 15076 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:38.877924 15174 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:38.878872 15174 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:38.881800 15174 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:38.882576 15126 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:38.883550 15126 raft_consensus.cc:697] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:38.884936 15126 consensus_queue.cc:237] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:38.809128 15125 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:38.839377 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:38.808713 15132 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:38.842711 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:38.842543 15073 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:38.844326 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:38.835425 14971 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:38.845208 12526 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:38.845811 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:38.841943 15149 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:38.846546 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:38.846997 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:38.847793 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:38.848598 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:38.848774 15012 consensus_meta.cc:287] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.565s	user 0.011s	sys 0.001s
I20260709 12:10:38.849359 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:38.833693 15027 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 12:10:38.857184 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:38.858678 12601 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260709 12:10:38.859552 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:38.838686 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 70686.4us
W20260709 12:10:38.831673 15088 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.547s	user 0.013s	sys 0.000s
W20260709 12:10:38.834102 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 124946us
W20260709 12:10:38.941210 13304 leader_election.cc:336] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.824s (SENT)
I20260709 12:10:38.941854 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:38.895866 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:38.899001 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:38.904734 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:38.905992 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 86350.7us
W20260709 12:10:38.912853 12643 connection.cc:256] RPC call timeout handler was delayed by -3.6416926570002488s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19740000000000002s for a response
I20260709 12:10:38.834282 15141 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:38.913987 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 189) took 1291 ms. Trace:
I20260709 12:10:38.845049 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:38.953474 12597 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:38.837181 14486 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:38.920598 12522 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:38.846259 15139 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:38.921748 12606 raft_consensus.cc:2393] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:38.924499 12600 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:38.855945 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:38.862423 15143 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:38.926800 15012 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:38.927803 15073 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:38.928874 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:38.940690 15125 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:38.924708 13331 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:38.944929 12605 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:10:38.918093 15132 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:38.953842 12679 rpcz_store.cc:276] 0709 12:10:37.622855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:38.642056 (+1019201us) service_pool.cc:224] Handling call
0709 12:10:38.659630 (+ 17574us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:38.680265 (+ 20635us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:10:38.697839 (+ 17574us) log.cc:844] Serialized 14 byte log entry
0709 12:10:38.913494 (+215655us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:38.913571 (+    77us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:38.913653 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:38.913752 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:38.913906 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28829,"prepare.queue_time_us":14186,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":3780,"raft.queue_time_us":33871,"raft.run_cpu_time_us":343,"raft.run_wall_time_us":12962,"thread_start_us":165255,"threads_started":1}
I20260709 12:10:38.925858 15149 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:38.957634 12599 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 4: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:38.959342 14691 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.020s
I20260709 12:10:38.978896 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:38.980173 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:38.973169 15125 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:38.981693 12599 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:38.958792 12602 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:38.955411 14788 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:38.930625 13330 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:10:38.969458 12646 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:38.989990 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:38.990962 12603 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
W20260709 12:10:38.970044 13304 leader_election.cc:336] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.843s (SENT)
I20260709 12:10:38.944203 15180 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:38.967700 12496 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:38.972996 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:38.974381 12571 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:38.957543 14486 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:38.976197 15182 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
I20260709 12:10:38.972934 15183 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:38.976989 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:38.963229 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:38.980901 12598 raft_consensus.cc:1217] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[3.5-4.6]   Dedup: 4.6->[]
I20260709 12:10:39.012732 12598 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5. Ops: [3.5-4.6]
I20260709 12:10:38.988276 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:38.997080 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 177220us
I20260709 12:10:38.972723 13335 raft_consensus.cc:2368] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6.
I20260709 12:10:39.001750 12601 raft_consensus.cc:2368] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5.
I20260709 12:10:38.943996 15117 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:10:39.004308 12496 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:39.005527 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:39.006691 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:39.007189 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:39.007400 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:39.003827 12571 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:38.950512 15088 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:39.015239 12602 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:39.028785 12494 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:39.030244 12605 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 6: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:39.031111 14971 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:10:39.031260 12571 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:39.031554 14971 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:39.031260 12606 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:39.032130 14971 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:39.018956 12600 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:38.964553 15141 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:39.044785 15023 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:39.048086 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:39.049006 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:39.008672 15175 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:39.050911 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:39.052089 12603 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5. Ops: [2.2-4.3]
I20260709 12:10:39.052711 15143 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:39.005577 14691 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:39.055578 14741 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:39.056743 15088 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:38.991869 14891 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:39.069296 15146 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:39.065961 12597 raft_consensus.cc:2393] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 7: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:39.066663 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:39.050062 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:39.070976 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:39.113245 12600 raft_consensus.cc:2393] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:39.113809 12598 raft_consensus.cc:2410] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:10:39.078768 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:39.068794 12599 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:39.008451 15027 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:39.085031 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:39.114475 12602 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
W20260709 12:10:39.085753 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 81550.3us
I20260709 12:10:39.081048 15175 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:39.120162 12525 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:39.091089 15072 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:39.095271 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:39.123131 12597 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:10:39.123162 12603 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
W20260709 12:10:39.063263 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 107490us
I20260709 12:10:39.092173 14691 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:39.108300 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 93810.4us
I20260709 12:10:39.068113 14788 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:38.982774 15139 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:39.090955 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:39.119325 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:39.119536 12569 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:39.082388 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:39.119560 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:39.118068 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:39.135236 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:39.122052 12671 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:39.125736 12494 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:39.136112 12599 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:39.126006 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:39.126771 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:39.137204 12600 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:39.126777 12524 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:10:39.138610 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 168595us
I20260709 12:10:39.128656 12603 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:39.131455 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:39.135537 12673 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:39.121701 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:39.137311 12528 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:39.127920 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:39.017593 15183 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:39.141709 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:39.140065 13304 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.809s (SENT) [suppressed 21 similar messages]
I20260709 12:10:39.013017 15180 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:39.118270 15027 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:39.199288 12602 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
W20260709 12:10:39.199508 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 59479.8us
W20260709 12:10:39.219154 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:39.231527 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:39.132484 15143 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:39.122779 15175 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:39.223317 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:39.259217 12604 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 7: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:39.266170 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:39.278302 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 152653us
I20260709 12:10:39.278790 12676 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:39.277227 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:39.196029 15073 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:39.256677 15027 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:39.133814 15076 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:39.299341 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:39.126597 14691 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:39.300633 12643 connection.cc:256] RPC call timeout handler was delayed by -3.4463926570001604s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2191s for a response
W20260709 12:10:39.371304 12643 connection.cc:256] RPC call timeout handler was delayed by -3.4220926570001211s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2218s for a response
W20260709 12:10:39.371877 12643 connection.cc:256] RPC call timeout handler was delayed by -3.4130926570001066s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22280000000000003s for a response
W20260709 12:10:39.372290 12643 connection.cc:256] RPC call timeout handler was delayed by -3.0800926570002503s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2598s for a response
W20260709 12:10:39.372740 12643 connection.cc:256] RPC call timeout handler was delayed by -2.522992657000259s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.32170000000000004s for a response
W20260709 12:10:39.373133 12643 connection.cc:256] RPC call timeout handler was delayed by -2.3183249920002709s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1502s for a response
I20260709 12:10:39.280855 14958 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:39.303103 15188 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
W20260709 12:10:39.303325 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:39.303573 15174 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:10:39.304289 15186 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.275s	user 0.000s	sys 0.001s
I20260709 12:10:39.375149 14958 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:39.304617 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:39.376696 15190 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:39.305645 14891 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:39.306571 15193 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:39.306865 15141 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:39.310820 15129 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:39.377871 14958 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:39.379096 15129 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:39.379890 15190 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:39.312368 15139 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:39.312865 12676 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:10:39.264581 15125 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:39.320185 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:39.321168 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:39.322984 12525 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:39.323975 12527 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:39.324276 14971 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:39.325073 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:39.407904 12528 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:39.331875 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52719.7us
I20260709 12:10:39.408788 12494 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:10:39.357369 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:39.377163 12674 raft_consensus.cc:2410] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 6 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:39.304997 15157 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:39.310869 15185 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
I20260709 12:10:39.381510 15129 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:39.435503 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:39.438375 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:39.384028 15073 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:39.451666 12643 connection.cc:256] RPC call timeout handler was delayed by -2.1230249920001825s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17190000000000003s for a response
W20260709 12:10:39.452306 12643 connection.cc:256] RPC call timeout handler was delayed by -2.0060249920002207s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1849s for a response
W20260709 12:10:39.452709 12643 connection.cc:256] RPC call timeout handler was delayed by -1.8512249920001977s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2021s for a response
W20260709 12:10:39.453070 12643 connection.cc:256] RPC call timeout handler was delayed by -1.6775249920001443s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2214s for a response
I20260709 12:10:39.402452 15076 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:39.454830 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:39.406750 15012 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:39.521517 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:39.620378 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 500909us
W20260709 12:10:39.560386 14486 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.509s	user 0.010s	sys 0.000s
I20260709 12:10:39.516711 15076 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:39.812932 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:39.650473 12531 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.026s	sys 0.071s
I20260709 12:10:39.818254 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:39.818559 12530 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:39.650768 15072 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:39.513463 15073 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:39.635672 15146 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.012s	sys 0.019s
W20260709 12:10:39.821452 15146 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.694s	user 0.019s	sys 0.027s
I20260709 12:10:39.821975 15146 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:39.434478 15023 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:39.825007 15023 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:39.827036 12530 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:10:39.827317 15023 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:39.604408 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:39.828418 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 252218us
W20260709 12:10:39.671377 12496 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.011s	sys 0.022s
W20260709 12:10:39.812768 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 404037us
I20260709 12:10:39.465809 15157 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:39.570573 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:39.816912 14486 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:39.829445 12496 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.017s	sys 0.028s
W20260709 12:10:39.842108 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 726642us
W20260709 12:10:39.816829 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:39.843122 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:39.843747 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:39.844254 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:39.830312 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:39.820129 15072 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:10:39.706234 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:39.817972 12528 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:39.381417 15190 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:39.847028 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:39.847846 12601 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:39.837200 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:39.823415 15146 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:39.828450 15076 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:39.844964 12673 raft_consensus.cc:2376] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6. Re-sending same reply.
I20260709 12:10:39.851271 12599 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:10:39.839797 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:39.818545 12531 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.032s	sys 0.077s
I20260709 12:10:39.859908 12671 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:10:39.849156 12603 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:39.860773 12604 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:39.861370 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:39.862059 12672 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:39.828032 15023 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.132s
I20260709 12:10:39.956314 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:40.056900 13333 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
W20260709 12:10:39.962956 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 96724.5us
I20260709 12:10:39.960029 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:40.044502 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:39.963891 13338 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:39.864526 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:40.059502 12522 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.653s	user 0.009s	sys 0.036s
I20260709 12:10:39.865319 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:39.868902 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:40.066967 13332 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
W20260709 12:10:40.067252 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:40.062790 12531 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 5
I20260709 12:10:40.064941 13339 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:39.942739 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:39.866564 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:39.999768 15132 consensus_meta.cc:287] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.024s	user 0.010s	sys 0.001s
I20260709 12:10:40.072436 13334 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260709 12:10:40.066186 12528 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:39.769825 15199 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:10:40.066187 12522 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.010s	sys 0.036s
I20260709 12:10:40.073004 12671 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:10:39.842391 14741 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:39.956841 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:39.853595 15008 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:39.852960 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:39.993287 15139 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:40.086462 13336 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:10:39.834940 15073 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:40.089926 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:39.993815 13330 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260709 12:10:40.085294 15194 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.030s	sys 0.041s
I20260709 12:10:40.083123 14412 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:40.224639 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:39.849951 15142 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.288s
I20260709 12:10:40.296396 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:39.850359 14958 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.287s
I20260709 12:10:40.087224 15199 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:40.146863 14939 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.017s	sys 0.033s
I20260709 12:10:39.997530 15076 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:40.375409 12605 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.048s	user 0.011s	sys 0.016s
W20260709 12:10:40.375901 12605 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.049s	user 0.011s	sys 0.016s
W20260709 12:10:40.351476 12679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.791s	user 0.011s	sys 0.014s
W20260709 12:10:40.377792 12679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.011s	sys 0.016s
W20260709 12:10:40.380069 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:40.367345 13304 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.550s	user 0.008s	sys 0.024s
W20260709 12:10:40.381742 13304 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.008s	sys 0.024s
W20260709 12:10:40.382171 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 731137us
W20260709 12:10:40.384745 12601 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.526s	user 0.008s	sys 0.000s
I20260709 12:10:40.384802 13304 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:40.084183 15072 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:10:40.418403 12680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.202s	user 0.025s	sys 0.033s
W20260709 12:10:40.419132 12680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.203s	user 0.025s	sys 0.033s
W20260709 12:10:40.363147 15196 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.854s	user 0.003s	sys 0.000s
W20260709 12:10:40.330930 15198 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.894s	user 0.000s	sys 0.002s
I20260709 12:10:40.493028 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:40.374987 15194 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.049s	user 0.036s	sys 0.047s
I20260709 12:10:40.180249 15183 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:40.377930 14939 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.767s	user 0.017s	sys 0.035s
I20260709 12:10:40.385175 12601 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:10:40.178786 15088 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:40.419009 14980 consensus_meta.cc:287] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.768s	user 0.014s	sys 0.004s
W20260709 12:10:40.431959 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:40.427914 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 362965us
I20260709 12:10:40.533970 12602 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:40.091902 15132 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:40.418322 15146 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.564s	user 0.029s	sys 0.031s
W20260709 12:10:40.417271 15195 log.cc:927] Time spent T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.939s	user 0.002s	sys 0.000s
I20260709 12:10:40.361261 15180 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:40.540985 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 156273us
I20260709 12:10:40.371081 15211 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.209s
I20260709 12:10:40.299106 15139 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:40.377631 15199 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:40.413060 15073 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:40.574458 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 486900us
W20260709 12:10:40.303685 12572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.192s	user 0.007s	sys 0.018s
W20260709 12:10:40.691550 12572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.580s	user 0.017s	sys 0.034s
W20260709 12:10:40.692076 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 1.58016e+06us
I20260709 12:10:40.439324 12531 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:40.315662 14971 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:40.628537 15146 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.029s	sys 0.032s
I20260709 12:10:40.363636 15208 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:10:40.746219 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47416.9us
I20260709 12:10:40.306274 14741 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:40.365006 15205 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:40.416069 14412 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:40.781437 12572 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:40.699232 15198 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
W20260709 12:10:40.740772 14891 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.012s	sys 0.040s
W20260709 12:10:40.802744 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 228869us
I20260709 12:10:40.692359 15220 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:40.573318 15088 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:40.584749 14980 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:40.804220 13305 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:40.744629 15008 raft_consensus.cc:697] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:40.834554 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:40.837209 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:40.807545 14891 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.013s	sys 0.040s
W20260709 12:10:40.813046 15027 consensus_meta.cc:287] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.409s	user 0.007s	sys 0.007s
I20260709 12:10:40.808763 15157 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:40.803399 15210 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:40.845427 12526 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: []
W20260709 12:10:40.828733 14691 consensus_meta.cc:287] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.422s	user 0.006s	sys 0.007s
I20260709 12:10:40.743407 15208 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:40.744758 15073 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:10:40.831672 14958 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:40.834199 15142 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:40.838685 15219 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.049s
I20260709 12:10:40.807699 15205 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:40.825019 15220 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:40.937104 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:40.939067 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1693) took 1106 ms. Trace:
I20260709 12:10:40.939548 12530 rpcz_store.cc:276] 0709 12:10:39.832887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:40.936824 (+1103937us) service_pool.cc:224] Handling call
0709 12:10:40.938987 (+  2163us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:40.886632 14891 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:40.857676 15027 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:40.846235 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 64928.5us
I20260709 12:10:40.809258 15223 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:40.952788 15142 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:10:40.955971 15201 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.186s	user 0.011s	sys 0.000s
I20260709 12:10:40.959036 14412 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:40.958700 14891 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:40.960309 14980 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:40.960285 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:40.961102 13339 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.891s	user 0.011s	sys 0.000s
W20260709 12:10:40.961838 15187 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.893s	user 0.026s	sys 0.038s
I20260709 12:10:40.962380 15008 consensus_queue.cc:237] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.016461 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1620) took 1898 ms. Trace:
I20260709 12:10:40.962827 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:40.944763 15076 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:40.963331 15149 consensus_meta.cc:287] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.985s	user 0.008s	sys 0.006s
I20260709 12:10:40.906642 15211 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:40.964918 15168 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 2.196s	user 0.014s	sys 0.000s
W20260709 12:10:40.966006 15222 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
W20260709 12:10:40.967391 15175 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.653s	user 0.012s	sys 0.000s
I20260709 12:10:40.875198 14691 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:40.967482 15073 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:40.968502 15214 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.164s	user 0.001s	sys 0.001s
I20260709 12:10:40.969128 15208 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:40.969451 15218 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.153s	user 0.000s	sys 0.001s
I20260709 12:10:40.982244 15157 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:41.014469 13339 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:41.021157 14412 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:41.019341 12646 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:40.969730 15215 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:40.931313 14958 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:40.998324 15193 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.008s	sys 0.000s
I20260709 12:10:41.021562 12680 rpcz_store.cc:276] 0709 12:10:39.118336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:39.118913 (+   577us) service_pool.cc:224] Handling call
0709 12:10:39.133204 (+ 14291us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:39.133447 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:10:39.216120 (+ 82673us) log.cc:844] Serialized 14 byte log entry
0709 12:10:41.000450 (+1784330us) log.cc:844] Serialized 14 byte log entry
0709 12:10:41.016017 (+ 15567us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:41.016082 (+    65us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:41.016143 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:41.016231 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:41.016387 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15269,"prepare.queue_time_us":115262,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":380329,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":10976,"spinlock_wait_cycles":9172224,"thread_start_us":1783190,"threads_started":3,"wal-append.queue_time_us":1601122}
I20260709 12:10:41.044581 12677 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 6: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:10:41.044385 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 81645.7us
I20260709 12:10:41.046299 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:41.048177 12571 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:41.050239 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:41.051185 12530 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
W20260709 12:10:41.011070 15143 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.730s	user 0.014s	sys 0.000s
W20260709 12:10:41.027668 12494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.885s	user 0.008s	sys 0.015s
W20260709 12:10:40.998517 15191 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.005s	sys 0.008s
I20260709 12:10:40.806878 14741 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:41.033351 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 110925us
I20260709 12:10:41.034170 13339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 334) took 1162 ms. Trace:
I20260709 12:10:40.980890 15223 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.023496 15232 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:10:41.054737 12494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.913s	user 0.008s	sys 0.015s
I20260709 12:10:41.056354 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:41.113752 13304 leader_election.cc:336] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.615s (SENT)
I20260709 12:10:41.113876 12601 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:41.034317 15076 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:41.113196 13339 rpcz_store.cc:276] 0709 12:10:39.871526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:39.956576 (+ 85050us) service_pool.cc:224] Handling call
0709 12:10:41.034042 (+1077466us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:41.119735 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:41.121101 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 312848us
W20260709 12:10:41.113312 15191 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.296s	user 0.017s	sys 0.011s
I20260709 12:10:41.126598 12524 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:41.027212 15149 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:41.042236 15235 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:41.034718 15175 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.034852 15168 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:41.061219 15215 raft_consensus.cc:697] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 5 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:41.114733 15208 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:41.114450 15073 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:10:41.044256 15193 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.746s	user 0.009s	sys 0.000s
I20260709 12:10:41.042135 15157 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:41.151413 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 164) took 1581 ms. Trace:
I20260709 12:10:41.042343 14412 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.155308 12528 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:41.042685 15132 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:41.031889 15211 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:41.180801 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:41.127738 15088 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:41.184413 15027 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:41.212147 15157 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:41.214022 15216 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.325s	user 0.000s	sys 0.001s
I20260709 12:10:41.214402 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1762) took 2167 ms. Trace:
I20260709 12:10:41.148326 15142 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:41.215217 15142 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.214915 12605 rpcz_store.cc:276] 0709 12:10:39.047198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:39.060338 (+ 13140us) service_pool.cc:224] Handling call
0709 12:10:39.078962 (+ 18624us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:39.082245 (+  3283us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:10:39.090961 (+  8716us) log.cc:844] Serialized 14 byte log entry
0709 12:10:39.303078 (+212117us) spinlock_profiling.cc:243] Waited 3.48 ms on lock 0x7b1800260500. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:10:40.376386 (+1073308us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:40.416409 (+ 40023us) log.cc:844] Serialized 26 byte log entry
0709 12:10:40.416652 (+   243us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:40.841444 (+424792us) log.cc:844] Serialized 14 byte log entry
0709 12:10:40.853681 (+ 12237us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:40.853823 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:40.955320 (+101497us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:41.214135 (+258815us) raft_consensus.cc:1685] finished
0709 12:10:41.214163 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:41.214308 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3304,"prepare.queue_time_us":615535,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":7556,"raft.queue_time_us":262445,"raft.run_cpu_time_us":639,"raft.run_wall_time_us":52318,"spinlock_wait_cycles":749634560,"thread_start_us":1665341,"threads_started":3,"wal-append.queue_time_us":318291}
I20260709 12:10:41.195099 14891 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:41.217492 15226 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:10:41.201045 13335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 426) took 1343 ms. Trace:
I20260709 12:10:41.219270 15142 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:41.195022 15231 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:41.220054 15231 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:41.195705 12530 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:41.195871 12606 raft_consensus.cc:2393] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:41.205345 12604 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:41.221747 12528 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:10:41.202708 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:41.196938 12600 rpcz_store.cc:276] 0709 12:10:39.569795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:39.620163 (+ 50368us) service_pool.cc:224] Handling call
0709 12:10:39.814725 (+194562us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:39.814917 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:39.814944 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:39.814997 (+    53us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:10:39.815048 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:39.815103 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:41.151235 (+1336132us) spinlock_profiling.cc:243] Waited 50.4 ms on lock 0x7b54002e68d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:41.151352 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":115953920}
I20260709 12:10:41.206158 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1623) took 1951 ms. Trace:
I20260709 12:10:41.207090 15168 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:41.208662 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 186) took 1270 ms. Trace:
I20260709 12:10:41.229218 12524 rpcz_store.cc:276] 0709 12:10:39.938329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:40.091660 (+153331us) service_pool.cc:224] Handling call
0709 12:10:41.208465 (+1116805us) spinlock_profiling.cc:243] Waited 751 ms on lock 0x7b54003c1254. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:41.208591 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1727501696}
I20260709 12:10:41.161202 15215 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:41.219508 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1619) took 2101 ms. Trace:
I20260709 12:10:41.218331 13335 rpcz_store.cc:276] 0709 12:10:39.854839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:39.854995 (+   156us) service_pool.cc:224] Handling call
0709 12:10:39.855372 (+   377us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:39.855823 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:39.855860 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:40.056472 (+200612us) log.cc:844] Serialized 26 byte log entry
0709 12:10:40.738441 (+681969us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:40.783987 (+ 45546us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:40.801373 (+ 17386us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:41.161558 (+360185us) raft_consensus.cc:1685] finished
0709 12:10:41.161580 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:41.198028 (+ 36448us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25722,"raft.queue_time_us":682377,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":166,"thread_start_us":361445,"threads_started":1,"wal-append.queue_time_us":636791}
I20260709 12:10:41.151304 15076 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:41.224180 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:41.220502 15231 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:41.186419 15073 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:41.258368 15073 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:41.260138 15072 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:41.223191 12679 rpcz_store.cc:276] 0709 12:10:39.255178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:39.258649 (+  3471us) service_pool.cc:224] Handling call
0709 12:10:39.299509 (+ 40860us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:39.405414 (+105905us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:39.405455 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:39.463311 (+ 57856us) log.cc:844] Serialized 52 byte log entry
0709 12:10:40.378315 (+915004us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:40.378367 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:40.378444 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:41.205906 (+827462us) raft_consensus.cc:1685] finished
0709 12:10:41.205927 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:41.206070 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28411,"raft.queue_time_us":69453,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":27696,"thread_start_us":818270,"threads_started":1,"wal-append.queue_time_us":1188750}
I20260709 12:10:41.236805 12677 rpcz_store.cc:276] 0709 12:10:39.117997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:39.122175 (+  4178us) service_pool.cc:224] Handling call
0709 12:10:40.492834 (+1370659us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b58001f2c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:10:41.219263 (+726429us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b54002c6854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:41.219405 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1296868608}
W20260709 12:10:41.245551 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 226330us
I20260709 12:10:41.265877 12646 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:41.268316 15149 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:41.269160 15239 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:10:41.172999 15235 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:41.126240 15232 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.222177 15208 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:41.250954 14486 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:41.252970 15141 raft_consensus.cc:3055] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:41.254597 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1760) took 2208 ms. Trace:
I20260709 12:10:41.256145 15139 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:41.290181 12606 rpcz_store.cc:276] 0709 12:10:39.046336 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:39.050599 (+  4263us) service_pool.cc:224] Handling call
0709 12:10:41.254302 (+2203703us) spinlock_profiling.cc:243] Waited 2.11 s on lock 0x7b5400310bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:41.254503 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4862921088}
W20260709 12:10:41.291792 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 99559.8us
W20260709 12:10:41.273268 12494 connection.cc:256] RPC call timeout handler was delayed by -1.7517472170000019s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15660000000000002s for a response
W20260709 12:10:41.292626 12494 connection.cc:256] RPC call timeout handler was delayed by -1.5591472169999179s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17800000000000002s for a response
W20260709 12:10:41.293002 12494 connection.cc:256] RPC call timeout handler was delayed by -1.5438472170001205s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1797s for a response
W20260709 12:10:41.293259 12494 connection.cc:256] RPC call timeout handler was delayed by -1.3044472169999608s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20630000000000004s for a response
W20260709 12:10:41.271950 15225 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:10:41.299791 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:41.300830 12677 raft_consensus.cc:2410] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 5 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:41.302676 12671 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:41.300861 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:41.271845 15211 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:41.269193 15149 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:41.272591 15076 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.306551 12672 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:41.305509 12643 leader_election.cc:400] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6.
I20260709 12:10:41.311254 12643 leader_election.cc:403] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:10:41.315716 15175 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:41.315614 15208 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.316689 14691 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:41.290874 15141 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:41.317497 15247 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:41.317955 14958 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:41.318145 15247 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.319159 15223 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:41.320888 15247 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:41.290195 14891 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:41.289958 15205 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:41.336143 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:41.341537 12522 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.026s	sys 0.019s
W20260709 12:10:41.341904 12522 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.026s	sys 0.019s
I20260709 12:10:41.345786 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:41.308619 15149 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:41.321985 15220 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:41.356515 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:41.361750 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:41.334676 15247 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6.
I20260709 12:10:41.362428 12677 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:41.317886 15141 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.368768 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1648) took 2050 ms. Trace:
I20260709 12:10:41.355803 15149 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:41.369573 12522 rpcz_store.cc:276] 0709 12:10:39.318609 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:39.318795 (+   186us) service_pool.cc:224] Handling call
0709 12:10:39.319213 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:39.319605 (+   392us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:10:40.087400 (+767795us) log.cc:844] Serialized 14 byte log entry
0709 12:10:41.348859 (+1261459us) log.cc:844] Serialized 14 byte log entry
0709 12:10:41.368364 (+ 19505us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:41.368417 (+    53us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:41.368502 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:41.368592 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:41.368711 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44050,"prepare.queue_time_us":164603,"prepare.run_cpu_time_us":100,"prepare.run_wall_time_us":104,"raft.queue_time_us":260747,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":33309,"thread_start_us":1649817,"threads_started":3,"wal-append.queue_time_us":899039}
I20260709 12:10:41.350589 14741 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:41.338583 14891 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.363924 15200 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:41.381219 12494 leader_election.cc:400] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6.
I20260709 12:10:41.381629 12494 leader_election.cc:403] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:10:41.381814 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:41.382587 12678 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
W20260709 12:10:41.387662 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 121872us
I20260709 12:10:41.390780 12677 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:41.379900 15008 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.013s
I20260709 12:10:41.402184 15191 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6.
I20260709 12:10:41.373721 15209 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:41.406112 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:41.406468 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:41.408047 12677 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:41.410526 14412 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:41.415169 15076 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:41.418550 15141 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:41.390517 14939 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
W20260709 12:10:41.421845 15246 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:10:41.419044 15232 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:41.424026 14891 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:41.428351 12671 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:41.435644 12529 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:41.435451 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:41.436345 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1661) took 1982 ms. Trace:
I20260709 12:10:41.436753 12529 rpcz_store.cc:276] 0709 12:10:39.454203 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.435096 (+1980893us) service_pool.cc:224] Handling call
0709 12:10:41.435451 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:41.436098 (+   647us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:41.436286 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:41.433889 15073 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:41.444772 12496 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:41.446074 15152 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:10:41.446528 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:41.446065 15073 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:41.454970 14891 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:41.458318 14958 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:41.458819 15205 raft_consensus.cc:3055] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:41.449757 13305 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:10:41.459750 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:41.447446 12496 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:41.472684 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:41.448954 15215 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:41.480873 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:41.481926 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:41.447028 15152 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.029s
I20260709 12:10:41.471398 14412 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:41.466207 15205 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:41.472532 14891 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:41.470831 14788 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:41.471232 15232 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:41.457063 15247 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.027s
I20260709 12:10:41.475041 15157 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:41.475129 15211 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:10:41.474012 15199 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.751s	user 0.012s	sys 0.001s
I20260709 12:10:41.500237 15255 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:41.473438 14958 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:41.507915 12643 connection.cc:256] RPC call timeout handler was delayed by -1.0559499260000393s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
I20260709 12:10:41.490764 15237 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:10:41.511119 12643 connection.cc:256] RPC call timeout handler was delayed by -1.0010499259999506s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1593s for a response
I20260709 12:10:41.493103 15180 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.020s
I20260709 12:10:41.500346 14788 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:41.516103 14788 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:41.518080 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:41.493628 15088 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:41.478785 15219 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:10:41.492807 15241 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:10:41.508648 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:41.524914 15241 consensus_queue.cc:1243] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:10:41.525214 13335 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:41.500789 15157 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:41.525651 15248 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:10:41.526023 15157 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:41.526764 15205 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:41.509464 14691 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:41.544157 14891 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.503523 15211 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:41.546540 15211 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:41.546998 15088 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.507206 15227 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:41.548108 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:41.551578 12678 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:10:41.551602 12671 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:41.496411 14412 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.523183 15255 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:41.512466 15199 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.565562 15227 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:41.566979 12523 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 5: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:41.568286 15199 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:41.570928 15259 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:10:41.483246 15215 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:41.572327 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:41.572845 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 48731.8us
I20260709 12:10:41.587782 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:41.591295 15267 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:41.573318 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:41.572955 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:41.592794 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 52417us
I20260709 12:10:41.519639 15237 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:41.600353 15267 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:41.602241 14691 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.605681 14691 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:41.607048 15217 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.411s	user 0.003s	sys 0.000s
I20260709 12:10:41.607332 12643 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:41.608179 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 197953us
I20260709 12:10:41.569130 15227 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:41.610235 12675 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:10:41.504259 15232 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.623838 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1621) took 2505 ms. Trace:
I20260709 12:10:41.626804 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:41.611654 15237 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:41.635592 15215 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:41.638291 15088 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:41.582894 14958 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:41.624388 12675 rpcz_store.cc:276] 0709 12:10:39.118652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:39.122393 (+  3741us) service_pool.cc:224] Handling call
0709 12:10:39.437079 (+314686us) spinlock_profiling.cc:243] Waited 26.6 ms on lock 0x7b58001f2c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:10:39.437296 (+   217us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:41.609941 (+2172645us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b5400285620. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488997dc0 00007f1488a221c0 00007f1488a23231 00007f1488a1fd8f 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c
0709 12:10:41.623457 (+ 13516us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:41.623756 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126131712}
I20260709 12:10:41.640982 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:41.608266 15199 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:41.614708 15076 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:41.664476 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:41.665567 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:41.671833 15264 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:10:41.674000 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:41.674569 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:41.675411 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 63509us
I20260709 12:10:41.654745 15199 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:41.651902 15247 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:41.680797 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:41.681680 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:41.665896 15076 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:41.694439 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:41.686383 15269 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:41.686600 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:41.699097 12675 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:41.688205 15149 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.689059 14691 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:41.668731 14958 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:41.682638 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:41.702831 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:41.703455 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:41.697947 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:41.706772 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:41.707744 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 1657) took 2386 ms. Trace:
I20260709 12:10:41.708128 12527 rpcz_store.cc:276] 0709 12:10:39.321467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.706414 (+2384947us) service_pool.cc:224] Handling call
0709 12:10:41.707666 (+  1252us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:41.709313 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:41.712337 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:41.623991 15231 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.718637 15209 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:41.719308 15209 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:41.720536 15231 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:41.693715 15199 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:41.722635 12671 raft_consensus.cc:2393] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 7: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:41.723608 13302 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:41.725558 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:41.696877 15088 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:41.726611 15141 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:41.694005 15247 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:41.727324 15247 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:41.727161 15141 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:41.727644 15023 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:10:41.729020 15023 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:41.702320 14958 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:41.725422 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:41.732165 12680 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:41.727177 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:41.742617 12672 raft_consensus.cc:2410] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:10:41.743134 12571 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:41.743420 12643 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:41.745859 13302 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:41.697290 15076 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:41.753710 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:41.754696 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:41.755344 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:41.705359 15269 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:41.731084 14958 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:41.756810 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:41.756283 13302 connection.cc:256] RPC call timeout handler was delayed by -1.5642065729998649s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16080000000000003s for a response
I20260709 12:10:41.761404 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:41.756078 12679 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 7: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:41.771054 12675 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:41.771267 12524 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:10:41.730067 15141 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:41.776217 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:41.777118 13330 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:41.752099 14891 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:41.779372 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:41.780159 12600 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:10:41.783234 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:41.784195 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:41.785416 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:41.778688 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:41.786654 12606 raft_consensus.cc:2376] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6. Re-sending same reply.
I20260709 12:10:41.788151 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:41.788122 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:41.752815 15231 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:41.762048 15088 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:41.793509 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:41.757828 14788 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:41.772535 15072 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:41.762199 15194 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:41.805387 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:41.813010 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:41.752022 14691 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
I20260709 12:10:41.797966 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:41.810948 12605 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:41.769513 15269 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:41.796360 12675 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:41.829489 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:41.730182 15272 consensus_queue.cc:1059] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.214s
W20260709 12:10:41.818257 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 107629us
I20260709 12:10:41.779839 15210 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:41.819449 12524 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:41.836663 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:41.833972 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:41.838939 12531 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:41.807302 15088 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:41.800877 15231 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:41.841648 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:41.847769 12673 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:41.848347 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:41.808532 15072 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:41.816329 15194 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:41.849612 12675 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:10:41.856331 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:41.857897 12675 raft_consensus.cc:2376] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6. Re-sending same reply.
W20260709 12:10:41.862820 15277 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:10:41.865046 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:41.865767 12603 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:41.866851 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:41.867333 15208 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:41.867676 12603 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 7: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:41.868148 15208 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.868737 13331 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:41.868799 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:41.869877 12603 raft_consensus.cc:2410] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:10:41.870790 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:41.873108 15141 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:41.795825 14891 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.873729 15141 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.876402 15141 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:41.876129 15157 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:41.876964 15141 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:41.877532 14891 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:41.878293 15232 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:41.879387 12600 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:41.880795 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:41.881450 12597 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:41.851087 15231 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:41.847841 15088 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:41.887887 12600 pending_rounds.cc:85] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 1
I20260709 12:10:41.888249 12600 pending_rounds.cc:107] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:10:41.888535 14412 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:41.877475 15141 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.847613 15210 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:41.896160 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:41.896868 12673 raft_consensus.cc:2376] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6. Re-sending same reply.
W20260709 12:10:41.897897 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:41.909992 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:41.898653 12602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 169) took 2056 ms (client timeout 1608 ms). Trace:
I20260709 12:10:41.879025 15232 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:41.901935 15255 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:41.904043 13336 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:41.906277 12605 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 5. Ops: []
I20260709 12:10:41.901553 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1685) took 2970 ms. Trace:
I20260709 12:10:41.934351 12526 rpcz_store.cc:276] 0709 12:10:38.931455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:38.931635 (+   180us) service_pool.cc:224] Handling call
0709 12:10:38.950786 (+ 19151us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:40.870346 (+1919560us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:41.901327 (+1030981us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b54003f7854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:41.901494 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4356898560}
I20260709 12:10:41.912461 12569 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:41.901485 15149 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:41.936990 15073 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:41.937208 15157 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:41.937453 15073 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:41.937947 15073 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:41.939091 15209 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.940552 12527 raft_consensus.cc:2410] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:10:41.940948 13330 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:41.917187 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:41.910933 12602 rpcz_store.cc:269] 0709 12:10:39.842101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:39.843460 (+  1359us) service_pool.cc:224] Handling call
0709 12:10:41.898408 (+2054948us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b540007b554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:41.898551 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":545855360}
I20260709 12:10:41.888545 12600 raft_consensus.cc:2889] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:41.941493 15269 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.943363 15141 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:41.944810 15219 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:41.946581 15208 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:41.933085 15220 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:41.948009 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1691) took 2133 ms. Trace:
I20260709 12:10:41.971195 12525 rpcz_store.cc:276] 0709 12:10:39.815072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.947135 (+2132063us) service_pool.cc:224] Handling call
0709 12:10:41.947471 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:41.947687 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:41.947714 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:41.947757 (+    43us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:41.947790 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:41.947850 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:41.947953 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:41.948558 15211 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:41.974191 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:41.949831 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:41.976559 12602 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:41.976989 12677 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:10:41.941115 15175 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:10:41.956528 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:41.957723 12531 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:10:41.859086 15194 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:41.847195 15152 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:41.854506 15072 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:41.987049 15152 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.961460 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:41.964141 15073 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:41.966341 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 201729us
I20260709 12:10:41.967296 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:41.978189 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:41.992877 12643 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:41.988909 13333 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:41.867866 15237 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:41.990396 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 223201us
I20260709 12:10:41.993013 12673 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:41.873814 15247 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:41.997364 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:41.998059 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:42.000557 12677 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:10:42.003603 15211 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:42.006856 15073 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:42.009202 15231 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:42.012593 15231 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:42.015357 13337 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 6: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:42.034960 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:42.038026 12673 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:42.040681 12602 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:42.041766 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:42.043592 13333 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:41.985998 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:42.049598 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:41.982908 15210 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:42.057260 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:42.058983 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:42.059819 12602 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
W20260709 12:10:42.054428 12643 leader_election.cc:336] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.647s (SENT)
I20260709 12:10:41.989539 15209 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:42.065729 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:42.067559 12643 leader_election.cc:336] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.716s (SENT)
I20260709 12:10:42.069885 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:41.978317 15220 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:42.077454 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:42.080363 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:42.080395 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:42.051517 15231 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:42.068248 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:42.092216 12643 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.786s (SENT)
I20260709 12:10:42.092857 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:42.094565 12643 leader_election.cc:336] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.918s (SENT)
I20260709 12:10:42.055698 15149 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:42.056126 15255 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:42.102877 15076 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:42.105366 15076 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:42.107749 15247 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:42.075403 15143 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:42.108744 15143 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:42.111073 15143 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:42.112407 12494 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:42.113054 15199 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:42.113709 15143 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:42.116223 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:42.116454 12494 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:42.117060 12602 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 6: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:42.121726 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:42.132615 15220 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:10:42.132902 14958 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:42.128575 15209 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:42.133654 15194 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:42.134682 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:42.135293 15280 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:42.134465 15237 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:42.135339 12602 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:42.139187 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:42.140007 12602 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 7: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:42.139827 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:42.140585 12601 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:42.140825 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:42.141429 12602 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:42.129586 15255 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:42.142585 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:42.143328 15237 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:10:42.143946 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:42.144635 12598 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:42.141367 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:42.146628 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:42.145550 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:42.147826 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:42.146780 12601 raft_consensus.cc:2410] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 6 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:10:42.145766 15199 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:42.149153 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:42.079802 15088 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:42.124704 15247 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:42.150653 15194 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:42.148118 12680 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:10:42.130148 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:42.160167 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:42.123997 15149 raft_consensus.cc:697] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:42.168282 12601 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:42.170405 12673 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:42.174391 12598 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:42.166834 12603 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 7: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:42.124053 15143 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:10:42.187567 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:42.188215 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:42.191588 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:42.192677 12602 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 6: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:42.192812 12598 raft_consensus.cc:2410] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 6 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:42.197065 12601 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:42.206550 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:42.207438 12602 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:42.195964 15272 consensus_queue.cc:1243] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:10:42.217864 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:42.187002 15072 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:42.240585 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 189) took 1113 ms. Trace:
I20260709 12:10:42.241066 12530 rpcz_store.cc:276] 0709 12:10:41.126812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.126995 (+   183us) service_pool.cc:224] Handling call
0709 12:10:41.127378 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:41.222525 (+ 95147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:42.240303 (+1017778us) spinlock_profiling.cc:243] Waited 68.1 ms on lock 0x7b5400407754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:42.240502 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156730752}
I20260709 12:10:42.246618 15251 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:42.247177 15251 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:42.248107 15251 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:42.249488 15251 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
W20260709 12:10:42.253803 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 246165us
I20260709 12:10:42.260736 12604 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
W20260709 12:10:42.264564 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 142945us
I20260709 12:10:42.265398 12494 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:42.266280 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 147369us
I20260709 12:10:42.260758 12569 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:42.266342 15209 raft_consensus.cc:697] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:42.268731 15076 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:42.269129 12602 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:10:42.269322 15076 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:42.269584 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:42.270215 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:42.270448 14939 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:42.269954 15209 consensus_queue.cc:237] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:42.270920 12602 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:42.270969 14939 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:42.270516 15076 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:42.271500 14939 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:42.273478 15117 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:42.280464 12494 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:42.280599 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:42.285915 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:42.287622 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:42.313627 15143 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:42.313886 15157 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:42.314363 15143 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:42.314841 15210 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:42.314901 15143 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:42.315402 15157 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 6 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:42.319933 15157 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:42.322155 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:42.288929 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:42.323357 13337 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 6: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:42.323948 12602 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:42.326453 12604 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:42.326453 13304 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:42.338455 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 48316.8us
I20260709 12:10:42.339428 12494 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:42.350368 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:42.333267 14412 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:42.337920 15211 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:42.356433 14412 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:42.361655 15088 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:42.363087 14958 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:42.364030 14412 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:42.359624 15008 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:42.365075 14741 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:42.365151 15008 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:42.365823 15008 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:42.367014 12604 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:42.340008 15210 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:42.374054 12597 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:10:42.380293 12597 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:42.259816 15149 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:42.382073 12604 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:10:42.386689 12597 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:42.391335 12604 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
W20260709 12:10:42.392163 12523 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 1644) took 3275 ms (client timeout 2184 ms). Trace:
W20260709 12:10:42.392851 12523 rpcz_store.cc:269] 0709 12:10:39.117117 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:39.119069 (+  1952us) service_pool.cc:224] Handling call
0709 12:10:42.391863 (+3272794us) spinlock_profiling.cc:243] Waited 2.43 s on lock 0x7b54003c0854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:42.392058 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5594170624}
I20260709 12:10:42.397495 15152 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:42.376636 15210 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:42.402078 12597 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:10:42.393864 12604 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
W20260709 12:10:42.461547 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 111252us
I20260709 12:10:42.403218 15210 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:42.464335 12494 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:42.469633 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:42.471478 15229 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:42.475633 12494 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:42.452623 15072 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:42.491071 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1843) took 1176 ms. Trace:
I20260709 12:10:42.492455 12494 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:42.491972 12673 rpcz_store.cc:276] 0709 12:10:41.315086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:42.196407 (+881321us) service_pool.cc:224] Handling call
0709 12:10:42.196858 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:42.205783 (+  8925us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:42.205828 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:42.209011 (+  3183us) log.cc:844] Serialized 26 byte log entry
0709 12:10:42.384746 (+175735us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:42.384788 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:42.384864 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:42.490759 (+105895us) raft_consensus.cc:1685] finished
0709 12:10:42.490792 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:42.490986 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":94,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":5640,"thread_start_us":104692,"threads_started":1,"wal-append.queue_time_us":218143}
I20260709 12:10:42.494167 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:42.480252 15152 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:42.504398 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:42.525976 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:42.495978 15200 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:42.500398 15209 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:42.546554 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:42.496158 15293 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:42.560703 15293 raft_consensus.cc:697] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:42.547297 12674 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:42.500741 15193 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:10:42.505460 15291 consensus_queue.cc:1059] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.056s
I20260709 12:10:42.558389 15117 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:42.558396 15200 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:10:42.559809 15294 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:10:42.500309 15245 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 12:10:42.562654 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:42.571991 12673 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:10:42.513795 15235 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:42.561694 15293 consensus_queue.cc:237] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:42.593940 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:42.598671 12674 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:10:42.599561 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1630) took 1166 ms. Trace:
I20260709 12:10:42.599972 12674 rpcz_store.cc:276] 0709 12:10:41.432808 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:42.546186 (+1113378us) service_pool.cc:224] Handling call
0709 12:10:42.599473 (+ 53287us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:42.601651 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:42.602430 12674 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:42.618820 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:42.619839 12673 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:10:42.608305 15235 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:42.635989 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:42.650282 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:42.651712 12673 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 7: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:10:42.642102 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 123129us
I20260709 12:10:42.657954 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:42.659662 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:42.660225 12673 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:42.617411 15193 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 12:10:42.655951 15152 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:42.679152 12674 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:42.641848 15235 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:42.684917 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:42.685840 12674 raft_consensus.cc:2393] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 7: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:42.687292 12673 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:42.688308 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:42.689173 12673 raft_consensus.cc:2410] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:10:42.695590 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:42.697180 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:42.697775 12673 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 3 FOLLOWER]: Advancing to term 5
W20260709 12:10:42.682998 15300 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:10:42.671417 15298 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.013s
I20260709 12:10:42.704495 12496 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:42.713294 12673 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:42.715452 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:42.717195 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:42.717885 12673 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:42.719525 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:42.720137 12673 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:42.720836 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1634) took 1287 ms. Trace:
I20260709 12:10:42.721310 12679 rpcz_store.cc:276] 0709 12:10:41.433669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.675219 (+241550us) service_pool.cc:224] Handling call
0709 12:10:42.720567 (+1045348us) spinlock_profiling.cc:243] Waited 57.8 ms on lock 0x7b54003bc754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:42.720728 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":132934016}
I20260709 12:10:42.723040 15227 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:42.698606 15303 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:42.725965 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:42.728026 12673 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:42.728619 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:42.729571 12674 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:42.729723 12673 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:42.685220 15304 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.041s
W20260709 12:10:42.737097 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 80762.3us
I20260709 12:10:42.736909 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1846) took 1405 ms. Trace:
W20260709 12:10:42.935992 14939 consensus_meta.cc:287] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.664s	user 0.013s	sys 0.002s
W20260709 12:10:43.131505 15210 consensus_meta.cc:287] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.650s	user 0.004s	sys 0.009s
W20260709 12:10:43.037510 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 271479us
I20260709 12:10:43.137305 14939 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:43.138031 15210 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:43.139997 14939 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:43.136700 15231 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.008s	user 0.018s	sys 0.023s
I20260709 12:10:43.140225 12598 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:43.140796 15210 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:42.880537 15209 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:43.051131 15191 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:43.141783 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:42.796034 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 91618.4us
W20260709 12:10:42.871380 15251 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.621s	user 0.014s	sys 0.004s
I20260709 12:10:43.142163 15191 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:42.946008 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1781) took 1082 ms. Trace:
I20260709 12:10:43.143818 12605 rpcz_store.cc:276] 0709 12:10:41.863928 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.878400 (+ 14472us) service_pool.cc:224] Handling call
0709 12:10:41.906002 (+ 27602us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:41.939014 (+ 33012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:42.945548 (+1006534us) spinlock_profiling.cc:243] Waited 27.2 ms on lock 0x7b540007b550. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:42.945897 (+   349us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62527744}
I20260709 12:10:43.142045 12673 rpcz_store.cc:276] 0709 12:10:41.331247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:42.729177 (+1397930us) service_pool.cc:224] Handling call
0709 12:10:42.729518 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:42.730312 (+   794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:42.736775 (+  6463us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:43.145180 15191 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:43.142445 15231 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.018s	user 0.018s	sys 0.023s
I20260709 12:10:43.029973 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1711) took 1277 ms. Trace:
I20260709 12:10:43.146173 12601 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:10:43.146283 15231 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:42.835124 15267 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:42.729964 15303 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:43.147583 15267 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:43.147864 15231 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:43.148941 15267 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:42.916429 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:43.146394 12527 rpcz_store.cc:276] 0709 12:10:41.752948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.753442 (+   494us) service_pool.cc:224] Handling call
0709 12:10:43.029589 (+1276147us) spinlock_profiling.cc:243] Waited 185 ms on lock 0x7b54003f6e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:43.029862 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":426637568}
W20260709 12:10:42.870942 12678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1639) took 1401 ms (client timeout 1720 ms). Trace:
W20260709 12:10:43.151463 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:43.152082 12678 rpcz_store.cc:269] 0709 12:10:41.469417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.517618 (+ 48201us) service_pool.cc:224] Handling call
0709 12:10:42.870552 (+1352934us) spinlock_profiling.cc:243] Waited 32.6 ms on lock 0x7b54000f00d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:42.870837 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":74898304}
W20260709 12:10:43.153206 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:42.885077 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1642) took 1154 ms. Trace:
I20260709 12:10:43.146256 12673 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 4. Ops: []
I20260709 12:10:42.794730 13334 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:43.156754 12675 rpcz_store.cc:276] 0709 12:10:41.730927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:42.058483 (+327556us) service_pool.cc:224] Handling call
0709 12:10:42.881977 (+823494us) spinlock_profiling.cc:243] Waited 7.11 ms on lock 0x7b5400092328. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e7fd5f4 00007f148d9e6227 00007f148d9e5ab8 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:42.885005 (+  3028us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16345344}
W20260709 12:10:42.825219 12643 leader_election.cc:336] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.502s (SENT)
I20260709 12:10:43.159322 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1852) took 1750 ms. Trace:
I20260709 12:10:43.159997 13334 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 435) took 1665 ms. Trace:
W20260709 12:10:42.919576 15143 consensus_meta.cc:287] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.604s	user 0.009s	sys 0.004s
I20260709 12:10:43.159494 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:43.135761 13302 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:43.162026 12496 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:43.159726 12673 rpcz_store.cc:276] 0709 12:10:41.408634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.145520 (+1736886us) service_pool.cc:224] Handling call
0709 12:10:43.146016 (+   496us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.159077 (+ 13061us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.159268 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:43.161015 13334 rpcz_store.cc:276] 0709 12:10:41.494607 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.494845 (+   238us) service_pool.cc:224] Handling call
0709 12:10:42.794310 (+1299465us) spinlock_profiling.cc:243] Waited 5.42 ms on lock 0x7b5800163298. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:10:42.794536 (+   226us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.159747 (+365211us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.159934 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12471040}
W20260709 12:10:43.173659 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 348440us
I20260709 12:10:43.177275 15191 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:43.177830 15308 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:43.177351 14939 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260709 12:10:43.160279 15149 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:43.180045 15149 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:43.157231 15072 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:43.182905 15072 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:43.183568 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:43.183666 15200 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:43.185539 15072 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:43.186314 15251 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:43.186790 15251 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:43.187757 15251 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:43.171272 15157 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:43.189025 15251 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:43.188936 15157 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:43.190342 13332 pending_rounds.cc:85] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 5
I20260709 12:10:43.190712 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:43.191033 13332 pending_rounds.cc:107] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:10:43.191395 12673 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:10:43.191438 13332 raft_consensus.cc:2889] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:43.193253 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:43.194345 12603 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:43.194489 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:43.194648 12597 raft_consensus.cc:2410] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 7 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:10:43.189203 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:43.196875 12603 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:43.169620 12571 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:43.198899 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:43.202316 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:43.204242 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:43.208927 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:43.209010 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:43.209919 12673 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:10:42.838790 15199 raft_consensus.cc:3055] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:10:43.211562 12673 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:43.212080 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:43.212301 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:43.222280 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 60327.9us
I20260709 12:10:43.222667 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:43.223191 12496 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:43.223563 12526 raft_consensus.cc:2410] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:10:43.223212 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:43.221643 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:43.225225 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1632) took 1792 ms. Trace:
I20260709 12:10:43.214655 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:43.214789 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:43.149619 15303 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:43.215430 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:43.218561 12597 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:43.213527 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:43.208410 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:43.229837 12605 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:43.220707 15199 raft_consensus.cc:740] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:43.232589 15199 consensus_queue.cc:260] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:43.225664 12673 rpcz_store.cc:276] 0709 12:10:41.433237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.210986 (+1777749us) service_pool.cc:224] Handling call
0709 12:10:43.211366 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.224980 (+ 13614us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.225169 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:43.228719 12528 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:10:43.253391 12604 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:43.258752 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:42.795037 15008 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:43.156337 15235 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:43.236164 12606 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:43.174203 15210 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:10:42.883026 15232 consensus_queue.cc:1059] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.335s
I20260709 12:10:43.240085 14691 raft_consensus.cc:3055] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:43.241557 13336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 339) took 1454 ms. Trace:
I20260709 12:10:43.243228 15180 raft_consensus.cc:3055] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:43.243606 13337 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 12:10:43.281872 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58767us
I20260709 12:10:43.246543 12524 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:10:43.247218 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:43.282794 12496 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:43.247143 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:43.283860 12598 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:43.247995 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:43.211611 12571 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:43.264765 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:43.284799 12530 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
W20260709 12:10:43.266880 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:43.206147 15076 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:43.304090 12531 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:43.299605 12496 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:43.300817 15076 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:43.308920 15076 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:43.201583 15143 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:42.738615 15152 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:43.207453 15072 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:10:43.244561 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:43.341212 12529 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:43.283164 12597 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:43.280846 13336 rpcz_store.cc:276] 0709 12:10:41.787432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.787879 (+   447us) service_pool.cc:224] Handling call
0709 12:10:43.241277 (+1453398us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b540046a5d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:43.241465 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":263850240}
W20260709 12:10:43.344564 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 58045.1us
W20260709 12:10:43.348412 12643 leader_election.cc:336] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.719s (SENT)
I20260709 12:10:43.349313 12494 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:43.319092 12675 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:10:43.352380 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 204) took 1603 ms. Trace:
I20260709 12:10:43.352948 12675 rpcz_store.cc:276] 0709 12:10:41.748702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.304916 (+1556214us) service_pool.cc:224] Handling call
0709 12:10:43.305381 (+   465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.351905 (+ 46524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.352289 (+   384us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:43.357050 12675 pending_rounds.cc:85] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 7
I20260709 12:10:43.357677 12675 pending_rounds.cc:107] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260709 12:10:43.358320 12675 raft_consensus.cc:2889] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:43.274351 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:43.287181 14691 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:10:43.361402 12643 leader_election.cc:336] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.849s (SENT)
I20260709 12:10:43.366053 12643 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:43.362434 12527 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:43.287402 15180 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:43.299624 15241 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:10:43.372133 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 72592us
W20260709 12:10:43.373661 12643 leader_election.cc:336] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.974s (SENT)
I20260709 12:10:43.377532 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:43.315501 15200 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:43.317356 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:43.287494 15312 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:43.384969 13333 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 3. Current term is 5. Ops: []
I20260709 12:10:43.343387 14741 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election won for term 5
W20260709 12:10:43.494544 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 282995us
I20260709 12:10:43.498498 12496 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:43.508630 12597 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:10:43.516920 12571 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:43.476963 15293 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:10:43.505239 15227 consensus_meta.cc:287] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.782s	user 0.010s	sys 0.005s
I20260709 12:10:43.570902 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1848) took 2216 ms. Trace:
W20260709 12:10:43.562930 15314 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.425s	user 0.000s	sys 0.002s
I20260709 12:10:43.380241 15291 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.146s
W20260709 12:10:43.613481 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:43.657025 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:43.518771 15320 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:43.380438 15229 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.140s
I20260709 12:10:43.483685 15210 raft_consensus.cc:2764] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:10:43.627377 15306 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:43.598517 14741 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:43.621593 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 247969us
I20260709 12:10:43.557490 15312 raft_consensus.cc:3055] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:43.617337 12671 rpcz_store.cc:276] 0709 12:10:41.354715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:42.878303 (+1523588us) service_pool.cc:224] Handling call
0709 12:10:43.133223 (+254920us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.133488 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:10:43.347177 (+213689us) log.cc:844] Serialized 14 byte log entry
0709 12:10:43.569395 (+222218us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:43.569636 (+   241us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:43.569854 (+   218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.570254 (+   400us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:43.570823 (+   569us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16793,"prepare.queue_time_us":131579,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"thread_start_us":391536,"threads_started":3}
I20260709 12:10:43.572172 15293 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:43.563645 15304 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:43.627956 15149 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:43.675913 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 177470us
I20260709 12:10:43.564201 15232 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:10:43.685359 12643 leader_election.cc:336] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.021s (SENT)
I20260709 12:10:43.659045 13331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 336) took 1853 ms. Trace:
I20260709 12:10:43.662631 15227 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:10:43.680665 15318 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 12:10:43.611454 15331 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:43.674877 15306 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:43.619736 15072 raft_consensus.cc:697] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:43.676502 15312 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:43.682700 15293 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:43.683784 15210 raft_consensus.cc:3055] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:10:43.690034 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:43.735675 12671 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:10:43.737793 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 216) took 1456 ms. Trace:
I20260709 12:10:43.738351 12671 rpcz_store.cc:276] 0709 12:10:42.281504 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.693317 (+1411813us) service_pool.cc:224] Handling call
0709 12:10:43.730604 (+ 37287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.737428 (+  6824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.737718 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:43.745894 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 491130us
I20260709 12:10:43.622510 15180 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:43.751658 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 67584.2us
I20260709 12:10:43.750243 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1847) took 2402 ms. Trace:
I20260709 12:10:43.736593 15210 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:43.659799 14691 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:43.710762 15333 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:43.756441 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1790) took 1869 ms. Trace:
I20260709 12:10:43.706437 15331 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:43.763815 12680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1640) took 2291 ms (client timeout 2166 ms). Trace:
I20260709 12:10:43.753115 12674 rpcz_store.cc:276] 0709 12:10:41.348839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:42.730485 (+1381646us) service_pool.cc:224] Handling call
0709 12:10:43.133283 (+402798us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.133536 (+   253us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:10:43.499793 (+366257us) log.cc:844] Serialized 14 byte log entry
0709 12:10:43.748943 (+249150us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:43.749158 (+   215us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:43.749254 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.749435 (+   181us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:43.749941 (+   506us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":111043,"prepare.queue_time_us":205510,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":5426,"spinlock_wait_cycles":117504,"thread_start_us":587551,"threads_started":3}
I20260709 12:10:43.764487 12672 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:10:43.764174 12600 rpcz_store.cc:276] 0709 12:10:41.887230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.887352 (+   122us) service_pool.cc:224] Handling call
0709 12:10:41.887686 (+   334us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:41.983518 (+ 95832us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:41.983565 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:41.984332 (+   767us) log.cc:844] Serialized 52 byte log entry
0709 12:10:42.884918 (+900586us) spinlock_profiling.cc:243] Waited 8.75 ms on lock 0x7b18001cbf40. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:10:43.096906 (+211988us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:43.157853 (+ 60947us) log.cc:844] Serialized 14 byte log entry
0709 12:10:43.301136 (+143283us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.301230 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:43.755996 (+454766us) raft_consensus.cc:1685] finished
0709 12:10:43.756139 (+   143us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:43.756370 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":63360,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":21480,"raft.run_cpu_time_us":237,"raft.run_wall_time_us":9303,"spinlock_wait_cycles":20131712,"thread_start_us":354566,"threads_started":2,"wal-append.queue_time_us":1131237}
W20260709 12:10:43.768687 12680 rpcz_store.cc:269] 0709 12:10:41.471923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.702498 (+230575us) service_pool.cc:224] Handling call
0709 12:10:43.763496 (+2060998us) spinlock_profiling.cc:243] Waited 28.5 ms on lock 0x7b54003bdb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:43.763704 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":65558400}
I20260709 12:10:43.728891 15232 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:43.771231 12680 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:10:43.774574 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1854) took 1168 ms. Trace:
I20260709 12:10:43.736467 15320 raft_consensus.cc:697] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:43.774971 12680 rpcz_store.cc:276] 0709 12:10:42.605735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.770499 (+1164764us) service_pool.cc:224] Handling call
0709 12:10:43.770919 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:43.774197 (+  3278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.774492 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:43.777134 12680 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:43.777884 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1856) took 1166 ms. Trace:
I20260709 12:10:43.778628 12680 rpcz_store.cc:276] 0709 12:10:42.611549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.776493 (+1164944us) service_pool.cc:224] Handling call
0709 12:10:43.776871 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.777590 (+   719us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.777818 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:43.755311 15143 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:43.784250 12674 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:10:43.803532 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1855) took 1197 ms. Trace:
I20260709 12:10:43.804394 12674 rpcz_store.cc:276] 0709 12:10:42.606037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.770958 (+1164921us) service_pool.cc:224] Handling call
0709 12:10:43.771252 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.802640 (+ 31388us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.803334 (+   694us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:43.804669 12671 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:10:43.805995 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1650) took 1241 ms. Trace:
I20260709 12:10:43.798810 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1838) took 2496 ms. Trace:
I20260709 12:10:43.806500 12671 rpcz_store.cc:276] 0709 12:10:42.565167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.774311 (+1209144us) service_pool.cc:224] Handling call
0709 12:10:43.804236 (+ 29925us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.805331 (+  1095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.805942 (+   611us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:43.767441 15335 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:43.690742 13331 rpcz_store.cc:276] 0709 12:10:41.774697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.778542 (+  3845us) service_pool.cc:224] Handling call
0709 12:10:43.611852 (+1833310us) spinlock_profiling.cc:243] Waited 70.3 ms on lock 0x7b540061f654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:43.628159 (+ 16307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":161632640}
I20260709 12:10:43.816223 12674 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:43.817065 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1860) took 1199 ms. Trace:
I20260709 12:10:43.775398 15333 raft_consensus.cc:697] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:43.817478 12674 rpcz_store.cc:276] 0709 12:10:42.617878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.805288 (+1187410us) service_pool.cc:224] Handling call
0709 12:10:43.815993 (+ 10705us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.816764 (+   771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.816988 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:43.801994 15313 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.423s	user 0.000s	sys 0.002s
I20260709 12:10:43.772959 15331 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:43.812558 13332 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.027s	sys 0.002s
W20260709 12:10:43.832376 13332 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.028s	sys 0.003s
I20260709 12:10:43.762995 14891 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260709 12:10:43.797326 15337 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:10:43.827450 15329 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.220s	user 0.001s	sys 0.000s
W20260709 12:10:43.811982 15317 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.311s	user 0.000s	sys 0.001s
I20260709 12:10:43.806720 12676 rpcz_store.cc:276] 0709 12:10:41.302224 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.425421 (+123197us) service_pool.cc:224] Handling call
0709 12:10:41.425866 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:41.426091 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:10:41.426483 (+   392us) log.cc:844] Serialized 14 byte log entry
0709 12:10:43.497751 (+2071268us) spinlock_profiling.cc:243] Waited 2 ms on lock 0x7b18002c6ec0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a
0709 12:10:43.726451 (+228700us) log.cc:844] Serialized 14 byte log entry
0709 12:10:43.798337 (+ 71886us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:43.798451 (+   114us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:43.798507 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.798586 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:43.798736 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5999,"prepare.queue_time_us":48786,"prepare.run_cpu_time_us":242,"prepare.run_wall_time_us":1880,"raft.queue_time_us":269056,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":13511,"spinlock_wait_cycles":4594944,"thread_start_us":345200,"threads_started":3,"wal-append.queue_time_us":227690,"wal-append.run_cpu_time_us":1757,"wal-append.run_wall_time_us":1102579}
W20260709 12:10:43.852802 15314 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:10:43.800369 15248 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:43.855163 12674 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: []
W20260709 12:10:43.856540 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 84384us
I20260709 12:10:43.822690 15117 raft_consensus.cc:3055] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:43.890487 15117 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:43.891593 15345 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:10:43.891551 15117 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:43.892760 15117 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:43.867535 12671 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:43.894663 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1862) took 1254 ms. Trace:
I20260709 12:10:43.895089 12671 rpcz_store.cc:276] 0709 12:10:42.641589 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.832528 (+1190939us) service_pool.cc:224] Handling call
0709 12:10:43.867326 (+ 34798us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.894345 (+ 27019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.894585 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:43.899168 12527 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.527s	user 0.006s	sys 0.005s
W20260709 12:10:43.849490 12529 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.000s
I20260709 12:10:43.871734 12676 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:10:43.900444 12529 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:10:43.900786 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1864) took 1246 ms. Trace:
I20260709 12:10:43.900005 12527 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:10:43.901247 12676 rpcz_store.cc:276] 0709 12:10:42.654419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.862795 (+1208376us) service_pool.cc:224] Handling call
0709 12:10:43.863202 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.900491 (+ 37289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.900731 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:43.884994 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1861) took 1258 ms. Trace:
I20260709 12:10:43.903200 12674 rpcz_store.cc:276] 0709 12:10:42.627015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.820972 (+1193957us) service_pool.cc:224] Handling call
0709 12:10:43.854761 (+ 33789us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.884650 (+ 29889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.884910 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:43.904736 12674 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:43.818037 15335 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:43.772765 14741 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:43.823226 15338 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:43.907039 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1871) took 1221 ms. Trace:
W20260709 12:10:43.868242 15323 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 12:10:43.859740 15337 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:43.925323 12646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.752s	user 0.010s	sys 0.005s
W20260709 12:10:43.928195 12646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.010s	sys 0.005s
W20260709 12:10:43.928795 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 769036us
I20260709 12:10:43.910548 12674 rpcz_store.cc:276] 0709 12:10:42.685116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.904204 (+1219088us) service_pool.cc:224] Handling call
0709 12:10:43.904502 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.905275 (+   773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.906644 (+  1369us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:43.813251 15076 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:43.929723 12646 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:43.819329 15210 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:43.821401 15072 consensus_queue.cc:237] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:43.916347 12604 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.002s	sys 0.013s
I20260709 12:10:43.798970 15306 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.2, Last appended by leader: 2, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:43.853305 15320 consensus_queue.cc:237] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:43.915827 13304 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.716s	user 0.015s	sys 0.019s
W20260709 12:10:44.227406 12602 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.008s	sys 0.004s
I20260709 12:10:44.185117 15346 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:44.085737 15319 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:44.261382 15346 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:44.261534 15319 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:44.024395 13302 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.850s	user 0.000s	sys 0.018s
W20260709 12:10:43.925761 15313 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:10:44.185302 15322 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:44.000484 12675 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.004s	sys 0.004s
W20260709 12:10:44.293946 12675 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.933s	user 0.005s	sys 0.011s
I20260709 12:10:43.992719 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1641) took 2316 ms. Trace:
W20260709 12:10:43.998226 12494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.004s	sys 0.004s
I20260709 12:10:44.119361 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:44.269457 13304 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.070s	user 0.019s	sys 0.019s
I20260709 12:10:44.123696 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:44.205330 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:44.113106 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:44.141376 12571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.000s	sys 0.009s
W20260709 12:10:44.260418 12602 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.645s	user 0.010s	sys 0.004s
W20260709 12:10:44.316943 12571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.750s	user 0.003s	sys 0.013s
W20260709 12:10:44.308720 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:44.317807 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 800934us
I20260709 12:10:44.319437 12571 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:10:44.322530 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 1.12391e+06us
I20260709 12:10:44.324534 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.325287 12601 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
W20260709 12:10:43.914513 15209 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.000s	sys 0.003s
I20260709 12:10:43.961100 12671 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
W20260709 12:10:44.280048 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 350415us
W20260709 12:10:44.053036 15343 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
W20260709 12:10:44.280225 12604 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.915s	user 0.009s	sys 0.020s
I20260709 12:10:44.335003 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.066076 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1845) took 2750 ms. Trace:
I20260709 12:10:44.335831 12597 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
W20260709 12:10:43.997278 15330 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.291s	user 0.000s	sys 0.001s
W20260709 12:10:44.326205 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:44.326581 12494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.932s	user 0.010s	sys 0.011s
I20260709 12:10:44.257625 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.323362 12603 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:44.252224 15335 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:44.279556 13302 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.105s	user 0.002s	sys 0.031s
W20260709 12:10:44.344722 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 1.20901e+06us
I20260709 12:10:44.336885 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:44.338531 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.346014 13302 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:44.254699 15076 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:44.336416 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 179) took 2548 ms. Trace:
I20260709 12:10:44.345840 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:44.295117 12673 rpcz_store.cc:276] 0709 12:10:41.676549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.264746 (+1588197us) service_pool.cc:224] Handling call
0709 12:10:43.265473 (+   727us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.265878 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:10:43.298602 (+ 32724us) log.cc:844] Serialized 14 byte log entry
0709 12:10:43.662652 (+364050us) log.cc:844] Serialized 14 byte log entry
0709 12:10:43.992279 (+329627us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:43.992344 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:43.992409 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.992490 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:43.992642 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75006,"prepare.queue_time_us":210668,"prepare.run_cpu_time_us":126,"prepare.run_wall_time_us":4759,"raft.queue_time_us":460073,"raft.run_cpu_time_us":366,"raft.run_wall_time_us":20342,"thread_start_us":624173,"threads_started":3,"wal-append.queue_time_us":374277}
I20260709 12:10:44.347149 12597 raft_consensus.cc:2368] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5.
I20260709 12:10:44.362174 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.184958 15357 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.238s
I20260709 12:10:44.250929 15352 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:44.348084 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1865) took 1681 ms. Trace:
I20260709 12:10:44.370518 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:44.370910 12671 rpcz_store.cc:276] 0709 12:10:42.667136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.895993 (+1228857us) service_pool.cc:224] Handling call
0709 12:10:43.960822 (+ 64829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:44.347811 (+386989us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.348008 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:44.344587 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 995272us
I20260709 12:10:44.047118 15334 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:43.960887 15337 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:44.346612 12601 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:44.264209 15319 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:44.286455 15210 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:44.350749 13302 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:44.350867 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:44.327557 15209 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.022s	user 0.002s	sys 0.015s
I20260709 12:10:44.363122 12597 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:44.335744 12679 rpcz_store.cc:276] 0709 12:10:41.324273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:42.727291 (+1403018us) service_pool.cc:224] Handling call
0709 12:10:42.727623 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:42.727856 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:10:43.108065 (+380209us) log.cc:844] Serialized 14 byte log entry
0709 12:10:44.065248 (+957183us) spinlock_profiling.cc:243] Waited 967 us on lock 0x7b18002d4d80. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a
0709 12:10:44.065573 (+   325us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:44.065639 (+    66us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:44.065717 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.065814 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:44.065993 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41829,"prepare.queue_time_us":473976,"prepare.run_cpu_time_us":137,"prepare.run_wall_time_us":85,"raft.queue_time_us":21159,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":9885,"spinlock_wait_cycles":2224896,"thread_start_us":530406,"threads_started":2,"wal-append.queue_time_us":233416}
I20260709 12:10:44.319733 15341 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:44.369287 12646 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:44.387341 12603 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:44.347913 15293 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.604s	user 0.010s	sys 0.003s
I20260709 12:10:44.373519 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:44.374428 12600 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:44.407724 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:43.909225 15248 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:44.299026 15346 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:44.414675 12527 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:44.416399 12529 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:44.347084 15347 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:44.401970 15326 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.715s	user 0.003s	sys 0.002s
I20260709 12:10:44.429770 15347 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:44.432137 15347 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:44.318190 15322 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:44.409416 15334 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:44.434962 13331 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:10:44.435482 15322 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:44.436365 15334 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:44.436239 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:44.436830 15331 consensus_meta.cc:287] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.577s	user 0.008s	sys 0.003s
I20260709 12:10:44.437404 15331 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:44.358832 12599 rpcz_store.cc:276] 0709 12:10:41.788066 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:41.788448 (+   382us) service_pool.cc:224] Handling call
0709 12:10:41.805248 (+ 16800us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:41.808463 (+  3215us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:10:42.178623 (+370160us) spinlock_profiling.cc:243] Waited 5.99 ms on lock 0x7b1800400720. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:10:42.840071 (+661448us) spinlock_profiling.cc:243] Waited 1.57 ms on lock 0x7b1800400720. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf
0709 12:10:42.840287 (+   216us) log.cc:844] Serialized 14 byte log entry
0709 12:10:43.560246 (+719959us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:43.612242 (+ 51996us) log.cc:844] Serialized 26 byte log entry
0709 12:10:43.612496 (+   254us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:43.612810 (+   314us) log.cc:844] Serialized 14 byte log entry
0709 12:10:43.712811 (+100001us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.712945 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:44.336179 (+623234us) raft_consensus.cc:1685] finished
0709 12:10:44.336205 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:44.336351 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2661,"prepare.queue_time_us":43781,"prepare.run_cpu_time_us":121,"prepare.run_wall_time_us":32226,"raft.queue_time_us":212986,"raft.run_cpu_time_us":476,"raft.run_wall_time_us":23606,"spinlock_wait_cycles":17379840,"thread_start_us":1115418,"threads_started":4,"wal-append.queue_time_us":414705}
I20260709 12:10:44.384713 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.403406 13338 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:44.388042 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:44.418008 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 44565.4us
I20260709 12:10:44.499850 12494 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:44.380900 15350 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:44.428325 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.440629 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:44.441085 15251 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.251s	user 0.012s	sys 0.005s
I20260709 12:10:44.348807 15354 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:44.367394 15361 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 12:10:44.442862 15341 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:44.376156 15328 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.074s
I20260709 12:10:44.443825 14741 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:44.523430 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:44.411300 15293 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:44.523715 15364 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:10:44.524271 15328 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:44.524688 15328 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:44.525727 15293 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:44.525467 15328 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:44.446015 15367 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:44.526755 15328 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:44.526629 15367 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:44.528916 15367 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:44.529711 15269 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:44.516435 15334 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:44.519366 15350 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 8 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:44.449193 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1629) took 3016 ms. Trace:
I20260709 12:10:44.535902 15205 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:44.536069 15331 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:44.512017 15360 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.075s	user 0.001s	sys 0.001s
I20260709 12:10:44.512758 15322 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260709 12:10:44.534853 12678 rpcz_store.cc:276] 0709 12:10:41.432603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.153395 (+1720792us) service_pool.cc:224] Handling call
0709 12:10:43.153843 (+   448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:43.154108 (+   265us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:43.154178 (+    70us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:43.154737 (+   559us) log.cc:844] Serialized 26 byte log entry
0709 12:10:43.347776 (+193039us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:43.380590 (+ 32814us) log.cc:844] Serialized 14 byte log entry
0709 12:10:43.523967 (+143377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:43.524061 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:44.187559 (+663498us) spinlock_profiling.cc:243] Waited 8.14 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:44.448832 (+261273us) raft_consensus.cc:1685] finished
0709 12:10:44.448946 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:44.449125 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12244,"prepare.queue_time_us":149877,"prepare.run_cpu_time_us":6399,"prepare.run_wall_time_us":31737,"raft.queue_time_us":309761,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":6116,"spinlock_wait_cycles":18713856,"thread_start_us":247454,"threads_started":2,"wal-append.queue_time_us":197857}
I20260709 12:10:44.538504 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 208) took 2788 ms. Trace:
I20260709 12:10:44.539139 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1876) took 1366 ms. Trace:
I20260709 12:10:44.539523 12678 rpcz_store.cc:276] 0709 12:10:43.172761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.538053 (+1365292us) service_pool.cc:224] Handling call
0709 12:10:44.538398 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:44.538583 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:44.538609 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:44.538866 (+   257us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:44.538914 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.538980 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:44.539078 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:10:44.450335 15356 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 12:10:44.386421 15352 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:44.024327 15333 consensus_queue.cc:237] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:44.538862 12675 rpcz_store.cc:276] 0709 12:10:41.750092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.355214 (+1605122us) service_pool.cc:224] Handling call
0709 12:10:43.356764 (+  1550us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:43.359580 (+  2816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:43.359652 (+    72us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:43.360642 (+   990us) log.cc:844] Serialized 26 byte log entry
0709 12:10:44.294511 (+933869us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:10:44.316343 (+ 21832us) log.cc:844] Serialized 14 byte log entry
0709 12:10:44.329215 (+ 12872us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.329297 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:44.537925 (+208628us) raft_consensus.cc:1685] finished
0709 12:10:44.537948 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:44.538087 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12526,"prepare.queue_time_us":2282,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":12817,"spinlock_wait_cycles":14458368,"thread_start_us":933436,"threads_started":1,"wal-append.queue_time_us":1007938}
I20260709 12:10:44.543563 15352 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:44.454843 12603 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:44.457428 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:44.459431 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:44.466181 15368 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:44.386458 15362 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:44.523438 15251 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.023s
I20260709 12:10:44.546844 15362 raft_consensus.cc:697] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:44.535271 15350 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:44.548009 15362 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:44.386438 15335 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.093s
I20260709 12:10:44.546301 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:44.555027 15341 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:44.566013 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: true dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:44.566987 12606 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 6: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:44.567888 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:44.569041 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.569804 12606 raft_consensus.cc:2368] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8.
I20260709 12:10:44.506799 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1869) took 1828 ms. Trace:
W20260709 12:10:44.504762 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 136168us
I20260709 12:10:44.516530 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.517799 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:44.518707 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.576257 12601 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 7: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:44.519757 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.577333 12597 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 7: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:44.515331 15346 consensus_queue.cc:1059] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260709 12:10:44.545020 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:44.574378 12676 rpcz_store.cc:276] 0709 12:10:42.678011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.902037 (+1224026us) service_pool.cc:224] Handling call
0709 12:10:43.902449 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.902710 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:10:43.925696 (+ 22986us) log.cc:844] Serialized 14 byte log entry
0709 12:10:44.505972 (+580276us) log.cc:844] Serialized 14 byte log entry
0709 12:10:44.506292 (+   320us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:44.506365 (+    73us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:44.506445 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.506546 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:44.506717 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":143408,"prepare.run_cpu_time_us":218,"prepare.run_wall_time_us":219,"raft.queue_time_us":34629,"raft.run_cpu_time_us":342,"raft.run_wall_time_us":28265,"spinlock_wait_cycles":11685632,"thread_start_us":549423,"threads_started":2,"wal-append.queue_time_us":516506}
I20260709 12:10:44.578926 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:44.574972 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1873) took 1872 ms. Trace:
I20260709 12:10:44.579165 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.458554 15358 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:10:44.625878 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 47042us
I20260709 12:10:44.580096 12603 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:44.627591 15377 raft_consensus.cc:3055] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:44.625931 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:44.628027 15377 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:44.631038 12606 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:44.615900 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1872) took 1921 ms. Trace:
I20260709 12:10:44.631776 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:44.614578 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 91807.4us
I20260709 12:10:44.587283 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.633468 12603 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 7: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:44.635897 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:44.635991 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.637194 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:44.636343 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:44.634846 12600 raft_consensus.cc:2368] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5.
I20260709 12:10:44.637960 12603 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:44.632843 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:44.640201 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:44.640991 12675 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:44.641988 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:44.642813 12678 raft_consensus.cc:2376] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7. Re-sending same reply.
I20260709 12:10:44.643990 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:44.645080 12678 raft_consensus.cc:2393] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 7: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:44.631458 15205 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.015s
W20260709 12:10:44.648623 13304 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 3.214s (SENT) [suppressed 39 similar messages]
I20260709 12:10:44.641988 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.653733 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.654661 12597 raft_consensus.cc:2410] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:10:44.546394 15368 raft_consensus.cc:697] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 5 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:44.655176 12606 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:10:44.657418 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:44.664955 12600 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:44.649649 15357 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:10:44.629364 15377 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:44.665796 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.666232 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:44.666719 12676 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:10:44.671238 12606 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:44.666762 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:44.670605 12603 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:10:44.632579 12674 rpcz_store.cc:276] 0709 12:10:42.698662 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.279280 (+1580618us) service_pool.cc:224] Handling call
0709 12:10:44.298162 (+ 18882us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:44.298468 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:10:44.375906 (+ 77438us) log.cc:844] Serialized 14 byte log entry
0709 12:10:44.615332 (+239426us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:44.615443 (+   111us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:44.615520 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.615618 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:44.615816 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":55846,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":95,"raft.run_cpu_time_us":138,"raft.run_wall_time_us":3591,"spinlock_wait_cycles":3328,"thread_start_us":314844,"threads_started":2}
I20260709 12:10:44.669620 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:44.673873 12678 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:44.677188 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.677837 12675 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 7: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:10:44.661304 15117 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.761s	user 0.008s	sys 0.008s
I20260709 12:10:44.650393 15358 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:44.681388 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.689654 12600 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:10:44.692675 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:44.694329 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.683195 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.630439 15352 consensus_queue.cc:1059] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.031s
I20260709 12:10:44.695372 12601 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:44.697666 12603 raft_consensus.cc:2393] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:44.697563 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:44.680091 15346 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:44.705112 15346 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:44.705937 15008 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:44.707527 15346 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:44.710948 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:44.712103 12643 leader_election.cc:336] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.191s (SENT)
I20260709 12:10:44.712577 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:44.713719 12643 leader_election.cc:336] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.214s (SENT)
W20260709 12:10:44.694365 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:44.714530 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1874) took 1542 ms. Trace:
I20260709 12:10:44.710091 15368 consensus_queue.cc:237] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:44.695478 12571 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:44.714964 12671 rpcz_store.cc:276] 0709 12:10:43.172080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.371849 (+1199769us) service_pool.cc:224] Handling call
0709 12:10:44.407173 (+ 35324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:44.442389 (+ 35216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:44.442423 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:44.442925 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 12:10:44.523232 (+ 80307us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:44.523277 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.523356 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:44.714273 (+190917us) raft_consensus.cc:1685] finished
0709 12:10:44.714296 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:44.714453 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34683,"raft.queue_time_us":35776,"raft.run_cpu_time_us":94,"raft.run_wall_time_us":95,"thread_start_us":79887,"threads_started":1,"wal-append.queue_time_us":208688}
W20260709 12:10:44.719460 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:44.720062 12603 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260709 12:10:44.720991 12603 pending_rounds.cc:85] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 2
W20260709 12:10:44.720112 12643 leader_election.cc:336] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.218s (SENT)
I20260709 12:10:44.721650 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:44.722255 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:44.725627 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:44.728220 12571 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:44.721314 12603 pending_rounds.cc:107] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:10:44.723846 15346 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:10:44.729882 12643 leader_election.cc:336] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.228s (SENT)
I20260709 12:10:44.717619 15377 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:44.733642 12603 raft_consensus.cc:2889] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:44.729162 12606 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:10:44.736256 15346 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:10:44.739243 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:44.722543 15367 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:44.724596 15251 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:44.745445 15365 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:44.747023 12606 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5. Ops: [4.2-4.2]
I20260709 12:10:44.749593 15365 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:44.749698 15234 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:44.750048 15365 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:44.727016 15335 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:10:44.718641 15338 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:44.748988 15251 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:44.751605 15338 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:44.753561 15251 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:44.744009 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:44.754206 15337 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:44.723255 15269 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:44.745288 12603 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 4. (term mismatch)
I20260709 12:10:44.756420 12603 pending_rounds.cc:85] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 3
I20260709 12:10:44.756769 12603 pending_rounds.cc:107] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 4.4
I20260709 12:10:44.757048 12603 raft_consensus.cc:2889] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:44.757428 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:44.750721 12606 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:44.758869 15344 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260709 12:10:44.756096 12601 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 7: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:44.739516 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:44.734974 14741 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:44.768956 15251 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:10:44.786595 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:44.729024 15333 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.043s
I20260709 12:10:44.734238 15322 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:10:44.768896 15344 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:44.794564 15346 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:44.764763 15338 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:44.792623 15334 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:44.771065 15367 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:44.791556 15251 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:44.789440 14741 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:44.839246 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:44.840145 12671 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:44.846093 15251 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:44.850442 12671 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:10:44.851979 12571 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:44.853106 15333 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:10:44.853515 15384 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:10:44.853721 15333 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:10:44.864023 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:44.835539 15337 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:44.873412 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 62426.3us
I20260709 12:10:44.854952 15333 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:44.863664 15354 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:44.757000 15358 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:44.900312 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:44.873229 12671 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260709 12:10:44.915531 12677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 209) took 3160 ms (client timeout 3214 ms). Trace:
W20260709 12:10:44.916047 12677 rpcz_store.cc:269] 0709 12:10:41.756844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:42.033791 (+276947us) service_pool.cc:224] Handling call
0709 12:10:44.915297 (+2881506us) spinlock_profiling.cc:243] Waited 851 ms on lock 0x7b54000924d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:44.915467 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1957633920}
I20260709 12:10:44.919473 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:44.925004 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:44.939543 13334 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 5 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
W20260709 12:10:44.940639 12643 connection.cc:256] RPC call timeout handler was delayed by -1.2248434799998904s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2143s for a response
W20260709 12:10:44.955698 15373 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.512s	user 0.000s	sys 0.003s
W20260709 12:10:44.957865 15372 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.515s	user 0.001s	sys 0.000s
I20260709 12:10:44.939136 15357 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:10:44.965312 12646 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:44.974392 15344 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:44.975098 15344 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:44.975504 15344 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:44.977694 14741 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:44.970275 15357 consensus_queue.cc:1243] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:10:44.982630 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:44.983584 13333 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
W20260709 12:10:44.990152 15388 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:10:44.963856 15354 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 12:10:44.996965 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:45.000078 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 207) took 3250 ms. Trace:
I20260709 12:10:45.490727 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1718) took 2277 ms. Trace:
I20260709 12:10:44.579833 12673 rpcz_store.cc:276] 0709 12:10:42.702960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.361492 (+1658532us) service_pool.cc:224] Handling call
0709 12:10:44.361901 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:44.362178 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:10:44.380403 (+ 18225us) log.cc:844] Serialized 14 byte log entry
0709 12:10:44.574447 (+194044us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:44.574519 (+    72us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:44.574569 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.574645 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:44.574896 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":81041,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":108,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":45046,"thread_start_us":193476,"threads_started":1}
W20260709 12:10:45.215497 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:45.985445 12529 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.017s	sys 0.031s
I20260709 12:10:45.515980 12601 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 12:10:46.049701 12529 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.741s	user 0.020s	sys 0.032s
I20260709 12:10:46.051805 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1816) took 1479 ms. Trace:
I20260709 12:10:46.052883 12601 rpcz_store.cc:276] 0709 12:10:44.572695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:45.489671 (+916976us) service_pool.cc:224] Handling call
0709 12:10:45.515550 (+ 25879us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:46.051381 (+535831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:46.051718 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:45.120232 13337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 440) took 1931 ms. Trace:
I20260709 12:10:45.316826 12599 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:46.048048 12531 rpcz_store.cc:276] 0709 12:10:43.213309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.213427 (+   118us) service_pool.cc:224] Handling call
0709 12:10:43.213714 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.307431 (+ 93717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:45.490408 (+2182977us) spinlock_profiling.cc:243] Waited 90.1 ms on lock 0x7b54003c1754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:45.490643 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":207229696}
I20260709 12:10:46.047772 12672 rpcz_store.cc:276] 0709 12:10:41.749766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.340783 (+1591017us) service_pool.cc:224] Handling call
0709 12:10:43.764243 (+423460us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:43.770014 (+  5771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.999718 (+1229704us) spinlock_profiling.cc:243] Waited 388 ms on lock 0x7b54003bdb50. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:44.999963 (+   245us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":893180288}
I20260709 12:10:46.060153 13337 rpcz_store.cc:276] 0709 12:10:43.188468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.188627 (+   159us) service_pool.cc:224] Handling call
0709 12:10:43.243032 (+ 54405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:43.282054 (+ 39022us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:45.119952 (+1837898us) spinlock_profiling.cc:243] Waited 54 ms on lock 0x7b540046a5d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:45.120148 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124274432}
W20260709 12:10:45.499692 15398 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.345s	user 0.000s	sys 0.001s
W20260709 12:10:46.050215 15320 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.606s	user 0.006s	sys 0.011s
W20260709 12:10:45.615062 15377 consensus_meta.cc:287] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.855s	user 0.007s	sys 0.007s
W20260709 12:10:45.623410 15358 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.611s	user 0.019s	sys 0.000s
W20260709 12:10:46.083828 15358 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.087s	user 0.021s	sys 0.000s
W20260709 12:10:45.369771 15365 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.619s	user 0.011s	sys 0.002s
I20260709 12:10:46.084914 15365 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:45.274863 15210 raft_consensus.cc:3055] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:10:45.120288 15370 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.682s	user 0.002s	sys 0.000s
I20260709 12:10:46.079569 13335 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:10:46.081316 15320 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.644s	user 0.006s	sys 0.012s
I20260709 12:10:45.709741 14691 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:45.608024 15319 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.002s	sys 0.019s
I20260709 12:10:46.075255 15395 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.084309 15367 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:45.482139 15241 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:46.113770 15241 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:46.113610 15395 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:46.117417 15365 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:45.190526 15357 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:10:45.428812 15396 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.455s	user 0.002s	sys 0.000s
I20260709 12:10:46.118559 15357 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:45.485134 15328 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:46.121838 15328 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:46.122107 15233 raft_consensus.cc:1064] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201: attempting to promote NON_VOTER e56d487365394ac5ac5f1002926a5645 to VOTER
I20260709 12:10:46.122754 15328 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:46.123929 15241 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:46.124521 15233 consensus_queue.cc:237] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:46.123980 15328 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:46.126525 15241 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:46.127295 15395 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:46.121420 15357 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:46.093633 15322 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.753s	user 0.025s	sys 0.014s
I20260709 12:10:45.498102 15312 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:46.130987 15312 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:46.133317 15344 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.157s	user 0.014s	sys 0.002s
I20260709 12:10:46.133850 15344 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:46.113051 15319 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.230s	user 0.004s	sys 0.030s
I20260709 12:10:46.074491 12601 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:46.223829 15357 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:46.244541 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 189) took 1481 ms. Trace:
I20260709 12:10:46.244709 13330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 234) took 3989 ms. Trace:
I20260709 12:10:46.185359 12673 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:46.188603 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1665) took 1293 ms. Trace:
W20260709 12:10:46.185330 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:46.207365 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:45.556260 15157 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:46.074895 15180 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:46.209929 15199 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:10:46.092025 15210 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:46.211584 15241 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.276219 15241 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:46.221357 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:46.281342 12600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 188) took 1601 ms (client timeout 1885 ms). Trace:
I20260709 12:10:46.227090 15354 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:46.282043 12600 rpcz_store.cc:269] 0709 12:10:44.680112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.692386 (+ 12274us) service_pool.cc:224] Handling call
0709 12:10:46.220901 (+1528515us) spinlock_profiling.cc:243] Waited 6.69 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:10:46.281182 (+ 60281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15381376}
I20260709 12:10:46.282739 15354 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:46.284245 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:46.244903 15357 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:46.246317 12675 rpcz_store.cc:276] 0709 12:10:44.895173 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.895296 (+   123us) service_pool.cc:224] Handling call
0709 12:10:44.895692 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:44.896120 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:44.896163 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:44.896708 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 12:10:46.051260 (+1154552us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:46.051325 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:46.051424 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:46.187653 (+136229us) raft_consensus.cc:1685] finished
0709 12:10:46.187730 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:46.188212 (+   482us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2380,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":5948,"thread_start_us":122437,"threads_started":1,"wal-append.queue_time_us":1195736}
I20260709 12:10:46.250272 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1857) took 3638 ms. Trace:
I20260709 12:10:46.245407 13330 rpcz_store.cc:276] 0709 12:10:42.255063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:42.256868 (+  1805us) service_pool.cc:224] Handling call
0709 12:10:43.627988 (+1371120us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:43.687247 (+ 59259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:43.690098 (+  2851us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:43.752889 (+ 62791us) log.cc:844] Serialized 26 byte log entry
0709 12:10:44.259581 (+506692us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:44.259638 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.259749 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:44.460359 (+200610us) raft_consensus.cc:1685] finished
0709 12:10:44.460403 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:46.243971 (+1783568us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b540061f650. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:46.244410 (+   439us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":414809,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":153,"spinlock_wait_cycles":3069992192,"thread_start_us":338436,"threads_started":1,"wal-append.queue_time_us":670347,"wal-append.run_cpu_time_us":1702,"wal-append.run_wall_time_us":1001753}
I20260709 12:10:46.255203 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:46.286031 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:46.260254 15322 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.920s	user 0.026s	sys 0.015s
I20260709 12:10:46.245146 12601 rpcz_store.cc:276] 0709 12:10:44.763062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:46.070382 (+1307320us) service_pool.cc:224] Handling call
0709 12:10:46.074282 (+  3900us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:46.243833 (+169551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:46.244270 (+   437us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:46.264081 12672 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:46.269465 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:46.259389 15344 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:46.277238 15210 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:46.286614 12680 rpcz_store.cc:276] 0709 12:10:42.611922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.779652 (+1167730us) service_pool.cc:224] Handling call
0709 12:10:43.780074 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:43.801980 (+ 21906us) raft_consensus.cc:1534] Early marking committed up to index 1
0709 12:10:43.802010 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:43.811093 (+  9083us) log.cc:844] Serialized 26 byte log entry
0709 12:10:43.868437 (+ 57344us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:43.868514 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:46.249222 (+2380708us) spinlock_profiling.cc:243] Waited 18.7 ms on lock 0x7b540016f3d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:46.249345 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:46.249436 (+    91us) raft_consensus.cc:1685] finished
0709 12:10:46.249453 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:46.249975 (+   522us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6090,"raft.queue_time_us":67449,"raft.run_cpu_time_us":250,"raft.run_wall_time_us":76552,"spinlock_wait_cycles":43033984,"thread_start_us":41138,"threads_started":1,"wal-append.queue_time_us":351155,"wal-append.run_cpu_time_us":3056,"wal-append.run_wall_time_us":1372802}
I20260709 12:10:46.286789 15354 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:46.327148 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:46.317795 15371 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.873s	user 0.007s	sys 0.000s
I20260709 12:10:46.335378 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:46.337018 15241 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:46.317886 15357 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:46.341668 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:46.345768 12601 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:46.355834 15210 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:46.346364 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:41.900736 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:10:46.363246 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:46.380476 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:46.380962 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:46.386278 15359 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 2.062s	user 0.000s	sys 0.014s
I20260709 12:10:46.389246 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1839) took 3148 ms. Trace:
I20260709 12:10:46.389786 12604 rpcz_store.cc:276] 0709 12:10:43.240737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.304252 (+ 63515us) service_pool.cc:224] Handling call
0709 12:10:43.365083 (+ 60831us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:44.348358 (+983275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:44.348391 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:44.356020 (+  7629us) log.cc:844] Serialized 52 byte log entry
0709 12:10:44.520049 (+164029us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:44.639901 (+119852us) log.cc:844] Serialized 14 byte log entry
0709 12:10:44.640342 (+   441us) log.cc:844] Serialized 14 byte log entry
0709 12:10:46.282505 (+1642163us) spinlock_profiling.cc:243] Waited 8.96 ms on lock 0x7b18004045c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485933522 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:10:46.387827 (+105322us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b18004045c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f14885d6d70 00007f14885ca293 00007f14885bfcea 00007f14885c023a 00007f1488a2aa2a 00007f1488a3a0bf 00007f1488a3a038
0709 12:10:46.388485 (+   658us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:46.388633 (+   148us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:46.388732 (+    99us) raft_consensus.cc:1685] finished
0709 12:10:46.388747 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:46.389033 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1697317,"prepare.queue_time_us":1761272,"prepare.run_cpu_time_us":152,"prepare.run_wall_time_us":154,"raft.queue_time_us":368265,"raft.run_cpu_time_us":481,"raft.run_wall_time_us":15603,"spinlock_wait_cycles":26338432,"thread_start_us":1084323,"threads_started":2,"wal-append.queue_time_us":185328}
W20260709 12:10:46.393719 15369 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 1.949s	user 0.000s	sys 0.011s
W20260709 12:10:46.390012 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:46.397212 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:46.398701 15248 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:46.400388 15241 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.401510 15241 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:46.403057 13332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 441) took 3214 ms. Trace:
I20260709 12:10:46.404502 15241 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:46.404867 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:46.403584 13332 rpcz_store.cc:276] 0709 12:10:43.188806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.189741 (+   935us) service_pool.cc:224] Handling call
0709 12:10:43.190081 (+   340us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:43.832810 (+642729us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:43.832842 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:43.855769 (+ 22927us) log.cc:844] Serialized 52 byte log entry
0709 12:10:44.349993 (+494224us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:44.350060 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.350159 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:46.392566 (+2042407us) spinlock_profiling.cc:243] Waited 3.69 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:46.402432 (+  9866us) raft_consensus.cc:1685] finished
0709 12:10:46.402464 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:46.402847 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":365043,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":11302,"spinlock_wait_cycles":8492288,"thread_start_us":1014895,"threads_started":2,"wal-append.queue_time_us":538898}
I20260709 12:10:46.406267 15241 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.407235 15241 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:46.393594 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:46.413861 12604 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:10:46.414805 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1837) took 3174 ms. Trace:
I20260709 12:10:46.415337 15229 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.416365 15229 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:46.417503 15354 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.405821 12600 raft_consensus.cc:2376] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8. Re-sending same reply.
I20260709 12:10:46.420643 12600 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:46.423635 15229 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:46.424113 13331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38472 (request call id 240) took 1931 ms (client timeout 1934 ms). Trace:
W20260709 12:10:46.457312 13331 rpcz_store.cc:269] 0709 12:10:44.492268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.493513 (+  1245us) service_pool.cc:224] Handling call
0709 12:10:46.375679 (+1882166us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b5400469954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:46.423919 (+ 48240us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":358444288}
I20260709 12:10:46.424825 12597 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:10:46.456179 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:46.436257 12675 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:10:46.437319 15223 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.438068 15193 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:46.439168 12673 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:10:46.444053 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:46.449421 12643 connection.cc:256] RPC call timeout handler was delayed by -1.0833368639998753s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2786s for a response
I20260709 12:10:46.431157 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1805) took 2042 ms. Trace:
I20260709 12:10:46.460937 12672 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
W20260709 12:10:46.470258 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:46.457340 15298 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.472697 15298 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:46.447642 15303 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.466305 15344 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.474406 15303 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:46.474850 15344 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:46.477012 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:46.486438 12643 connection.cc:256] RPC call timeout handler was delayed by -1.0833368639998753s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2786s for a response
I20260709 12:10:46.489118 13334 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38482 (request call id 342) took 2043 ms. Trace:
I20260709 12:10:46.491747 13334 rpcz_store.cc:276] 0709 12:10:44.430805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.438922 (+  8117us) service_pool.cc:224] Handling call
0709 12:10:44.439351 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:44.940174 (+500823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:46.473496 (+1533322us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b540047b8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:46.474033 (+   537us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1149453568}
I20260709 12:10:46.418538 15354 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:46.502182 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1811) took 1880 ms. Trace:
I20260709 12:10:46.502924 12606 rpcz_store.cc:276] 0709 12:10:44.569923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.759613 (+189690us) service_pool.cc:224] Handling call
0709 12:10:44.760117 (+   504us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:44.819461 (+ 59344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:44.819494 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:44.828902 (+  9408us) log.cc:844] Serialized 26 byte log entry
0709 12:10:44.940241 (+111339us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:45.025111 (+ 84870us) log.cc:844] Serialized 14 byte log entry
0709 12:10:45.029445 (+  4334us) spinlock_profiling.cc:243] Waited 6.01 ms on lock 0x7b480024c258. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:10:46.354386 (+1324941us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:46.370331 (+ 15945us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:46.382347 (+ 12016us) raft_consensus.cc:1685] finished
0709 12:10:46.450296 (+ 67949us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:46.450537 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2111797,"prepare.queue_time_us":84477,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":5096,"raft.queue_time_us":1065132,"raft.run_cpu_time_us":1765,"raft.run_wall_time_us":263555,"spinlock_wait_cycles":13832576,"thread_start_us":389520,"threads_started":3,"wal-append.queue_time_us":177415}
W20260709 12:10:46.506242 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:46.506587 15193 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:46.509649 15193 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:46.510800 12600 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260709 12:10:46.513805 12600 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:46.466604 15248 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.415298 12602 rpcz_store.cc:276] 0709 12:10:43.240128 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.250372 (+ 10244us) service_pool.cc:224] Handling call
0709 12:10:43.280678 (+ 30306us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:43.388001 (+107323us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:43.388220 (+   219us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:43.389207 (+   987us) log.cc:844] Serialized 52 byte log entry
0709 12:10:44.321098 (+931891us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:44.347447 (+ 26349us) log.cc:844] Serialized 14 byte log entry
0709 12:10:44.347661 (+   214us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.347730 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:46.315999 (+1968269us) spinlock_profiling.cc:243] Waited 3.69 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:46.414233 (+ 98234us) raft_consensus.cc:1685] finished
0709 12:10:46.414362 (+   129us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:46.414601 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14925,"prepare.queue_time_us":26620,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":420019,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":31311,"spinlock_wait_cycles":8498432,"thread_start_us":802824,"threads_started":2,"wal-append.queue_time_us":919443}
I20260709 12:10:46.514739 15248 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:46.516741 15298 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:46.518532 15414 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:46.520139 12672 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:46.519615 15414 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } }
I20260709 12:10:46.507061 14691 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:46.502424 15241 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:46.536837 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:46.510846 15303 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:46.480769 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 69092.2us
I20260709 12:10:46.529827 15248 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:46.545346 15414 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:46.546380 15363 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 2.132s	user 0.009s	sys 0.007s
I20260709 12:10:46.550503 12601 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260709 12:10:46.553432 15344 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:46.554541 15237 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.555811 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1840) took 3314 ms. Trace:
I20260709 12:10:46.555457 15237 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:46.559598 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:46.561640 12606 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: [4.2-5.3]
I20260709 12:10:46.464754 12603 rpcz_store.cc:276] 0709 12:10:44.389032 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.758157 (+369125us) service_pool.cc:224] Handling call
0709 12:10:44.758604 (+   447us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:44.759213 (+   609us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:44.759245 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:44.760115 (+   870us) log.cc:844] Serialized 52 byte log entry
0709 12:10:44.886200 (+126085us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:44.962673 (+ 76473us) log.cc:844] Serialized 14 byte log entry
0709 12:10:46.075932 (+1113259us) log.cc:844] Serialized 14 byte log entry
0709 12:10:46.373122 (+297190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:46.430442 (+ 57320us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:46.430537 (+    95us) raft_consensus.cc:1685] finished
0709 12:10:46.430564 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:46.430873 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1121060,"prepare.queue_time_us":178400,"prepare.run_cpu_time_us":130,"prepare.run_wall_time_us":133,"raft.queue_time_us":1128391,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":24719,"thread_start_us":404403,"threads_started":3,"wal-append.queue_time_us":148600}
I20260709 12:10:46.562250 15354 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:46.562855 14691 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:46.474421 15223 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:46.564527 14412 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:10:46.566226 15366 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 2.128s	user 0.017s	sys 0.000s
I20260709 12:10:46.566737 12675 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:46.566994 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:46.570305 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:46.570794 15076 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:46.571682 15414 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.572281 15344 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:46.573000 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:46.573263 15337 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:46.556269 12598 rpcz_store.cc:276] 0709 12:10:43.240975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.342676 (+101701us) service_pool.cc:224] Handling call
0709 12:10:43.379820 (+ 37144us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:43.521938 (+142118us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:43.522086 (+   148us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:43.615294 (+ 93208us) log.cc:844] Serialized 26 byte log entry
0709 12:10:44.177295 (+562001us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:44.250831 (+ 73536us) log.cc:844] Serialized 14 byte log entry
0709 12:10:44.254398 (+  3567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.254512 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:46.458610 (+2204098us) spinlock_profiling.cc:243] Waited 29.9 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:46.555134 (+ 96524us) raft_consensus.cc:1685] finished
0709 12:10:46.555254 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:46.555576 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11749,"prepare.queue_time_us":63767,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":45,"raft.queue_time_us":380314,"raft.run_cpu_time_us":408,"raft.run_wall_time_us":69201,"spinlock_wait_cycles":68748160,"thread_start_us":580624,"threads_started":2,"wal-append.queue_time_us":675775}
I20260709 12:10:46.495895 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 190) took 1726 ms. Trace:
I20260709 12:10:46.580678 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:46.581606 15248 consensus_queue.cc:1059] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:46.583657 12571 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:46.585069 12600 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:10:46.585160 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:46.585860 15365 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:10:46.593839 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:46.610206 12604 rpcz_store.cc:276] 0709 12:10:44.763992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:46.414900 (+1650908us) service_pool.cc:224] Handling call
0709 12:10:46.415444 (+   544us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:46.416165 (+   721us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:46.416199 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:46.417079 (+   880us) log.cc:844] Serialized 52 byte log entry
0709 12:10:46.483841 (+ 66762us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:46.483880 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:46.483955 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:46.490106 (+  6151us) raft_consensus.cc:1685] finished
0709 12:10:46.490152 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:46.490371 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":625,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":165,"thread_start_us":66224,"threads_started":1,"wal-append.queue_time_us":66605}
I20260709 12:10:46.577943 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:46.629443 14412 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:46.630772 12603 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: [5.3-6.4]
I20260709 12:10:46.633289 12527 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 7: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:46.634341 12672 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:46.634646 12530 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 9: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:10:46.635007 15312 consensus_meta.cc:287] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.503s	user 0.020s	sys 0.001s
W20260709 12:10:46.635277 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 56880.6us
I20260709 12:10:46.635840 15344 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:46.637291 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:46.633776 14412 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:46.642959 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 84948.8us
I20260709 12:10:46.653740 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:46.670107 12523 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:46.670583 15338 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.654863 15298 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.671919 15338 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:46.672128 15298 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:46.674544 15344 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:46.674952 15223 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:46.656879 15248 raft_consensus.cc:3055] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:46.675670 15248 raft_consensus.cc:740] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:46.652036 15414 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:46.676599 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:46.676877 15248 consensus_queue.cc:260] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:46.659866 12679 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:10:46.678335 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 87489.3us
I20260709 12:10:46.678762 12606 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:10:46.680135 12602 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:10:46.680954 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:46.683357 12597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 187) took 2007 ms (client timeout 2053 ms). Trace:
W20260709 12:10:46.684185 12597 rpcz_store.cc:269] 0709 12:10:44.676312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.996546 (+320234us) service_pool.cc:224] Handling call
0709 12:10:46.683103 (+1686557us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b54001167d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:46.683293 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2742671616}
I20260709 12:10:46.686838 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:46.688383 12675 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:10:46.681937 12606 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:10:46.656097 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1728) took 2208 ms. Trace:
I20260709 12:10:46.691754 12604 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:46.691231 12676 raft_consensus.cc:2410] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:46.687178 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:46.653256 13331 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:10:46.710206 12673 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:46.710012 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:46.711153 12672 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:10:46.713894 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:46.714358 12673 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 2. (index mismatch)
I20260709 12:10:46.714910 12676 raft_consensus.cc:2410] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:10:46.715466 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:46.715914 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:46.716168 12673 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:46.716727 12523 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:46.718536 12527 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:46.720000 15338 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:46.720645 12525 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:46.721825 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:46.722566 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:46.722482 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:46.723663 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:46.724363 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:46.725118 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1835) took 3485 ms. Trace:
I20260709 12:10:46.642289 15337 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.049s
I20260709 12:10:46.725481 12605 rpcz_store.cc:276] 0709 12:10:43.239526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:43.275214 (+ 35688us) service_pool.cc:224] Handling call
0709 12:10:43.276880 (+  1666us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:43.623721 (+346841us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:43.623785 (+    64us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:43.625984 (+  2199us) log.cc:844] Serialized 52 byte log entry
0709 12:10:43.993113 (+367129us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:44.095845 (+102732us) log.cc:844] Serialized 14 byte log entry
0709 12:10:44.096276 (+   431us) log.cc:844] Serialized 14 byte log entry
0709 12:10:44.096482 (+   206us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:44.096585 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:46.390093 (+2293508us) spinlock_profiling.cc:243] Waited 8.02 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:46.565662 (+175569us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b240033f7b8. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14857b99c4 00007f14857b7acb 00007f1485972dc9 00007f1485972a55 00007f148597209d 00007f14885a410e 00007f14885bb88c 00007f14885babb6 00007f14885ba7d8 00007f14885c4221
0709 12:10:46.724744 (+159082us) raft_consensus.cc:1685] finished
0709 12:10:46.724766 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:46.724957 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17054,"prepare.queue_time_us":105182,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":75,"raft.queue_time_us":390868,"raft.run_cpu_time_us":324,"raft.run_wall_time_us":9418,"spinlock_wait_cycles":76179584,"thread_start_us":638235,"threads_started":2,"wal-append.queue_time_us":711613}
I20260709 12:10:46.727969 15076 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:46.728444 15076 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:46.728832 15076 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:46.659313 14691 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:46.729843 12678 raft_consensus.cc:2393] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 8: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:46.721873 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:46.678355 15328 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:46.733567 12673 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:46.689407 15414 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:46.656983 15237 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:46.742870 12530 raft_consensus.cc:2410] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260709 12:10:46.691365 12529 rpcz_store.cc:276] 0709 12:10:44.447819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.447973 (+   154us) service_pool.cc:224] Handling call
0709 12:10:44.448335 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:45.308395 (+860060us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:10:46.050992 (+742597us) log.cc:844] Serialized 14 byte log entry
0709 12:10:46.461649 (+410657us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:46.489680 (+ 28031us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:46.537706 (+ 48026us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:46.655551 (+117845us) spinlock_profiling.cc:243] Waited 860 ms on lock 0x7b540038e0d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:46.655603 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:46.655838 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":796590,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":13958,"raft.run_cpu_time_us":286,"raft.run_wall_time_us":1244,"spinlock_wait_cycles":1977190656,"thread_start_us":1012897,"threads_started":2,"wal-append.queue_time_us":325309}
I20260709 12:10:46.744571 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:46.731345 15241 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:46.747030 15248 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:46.764825 12643 connection.cc:256] RPC call timeout handler was delayed by -1.0143410409998523s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17020000000000002s for a response
W20260709 12:10:46.768844 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:46.755367 15241 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:10:46.770139 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 108939us
I20260709 12:10:46.755040 15328 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:46.777166 15298 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:46.777329 15319 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
W20260709 12:10:46.777984 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:46.780712 12677 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:10:46.782990 15328 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:46.772370 15349 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.194s	user 0.015s	sys 0.020s
W20260709 12:10:46.784308 15349 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.339s	user 0.015s	sys 0.020s
I20260709 12:10:46.809360 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:46.774639 15241 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:46.812326 12496 connection.cc:256] RPC call timeout handler was delayed by -1.3230593879998196s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1585s for a response
W20260709 12:10:46.814952 12496 connection.cc:256] RPC call timeout handler was delayed by -1.2807593879997512s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1632s for a response
W20260709 12:10:46.815461 12496 connection.cc:256] RPC call timeout handler was delayed by -1.0476593879998291s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18910000000000002s for a response
I20260709 12:10:46.817657 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:46.818964 12672 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260709 12:10:46.821663 12530 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 6: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:46.826581 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:46.813664 12643 leader_election.cc:336] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.598s (SENT)
I20260709 12:10:46.833053 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:46.835534 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:46.841928 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:46.843199 15413 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:46.843755 15413 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:46.844136 15413 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:46.847560 15338 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:46.848860 15357 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:10:46.850095 12599 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1797) took 2545 ms (client timeout 2032 ms). Trace:
W20260709 12:10:46.850572 12599 rpcz_store.cc:269] 0709 12:10:44.294371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.496402 (+202031us) service_pool.cc:224] Handling call
0709 12:10:46.827877 (+2331475us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b54002f1054. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:46.839338 (+ 11461us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":757453696}
I20260709 12:10:46.853008 15076 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:46.849222 12530 raft_consensus.cc:2410] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:10:46.854331 15430 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.854907 15430 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:46.856182 15076 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:46.857529 15430 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:46.867216 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 65289us
I20260709 12:10:46.869254 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:46.871438 12678 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:10:46.873487 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 51013.3us
I20260709 12:10:46.875720 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:46.876600 12672 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:46.877575 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:46.878762 12524 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:46.879669 15429 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:46.880476 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:46.880316 15429 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:46.880734 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:46.881387 12531 raft_consensus.cc:2410] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260709 12:10:46.882354 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:46.882496 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:46.882532 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:46.875803 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 63697.8us
I20260709 12:10:46.885358 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:46.878912 15303 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:10:46.887823 15303 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:46.877642 12530 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:46.878021 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:46.896354 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:46.897043 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:46.909646 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:46.923153 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:46.927316 12529 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
W20260709 12:10:46.986783 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:46.988698 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:46.989706 12523 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:10:46.910032 15303 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:10:47.011497 12680 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 5. Current term is 6. Ops: [3.5-5.6]
W20260709 12:10:47.014595 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 84601.5us
I20260709 12:10:47.024760 15241 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:47.027139 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:47.027959 12673 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:47.028142 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:47.029731 15357 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:47.030401 15357 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:47.031458 12672 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:10:47.032336 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:47.032967 15357 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:47.033524 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:47.028890 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:47.036986 12673 raft_consensus.cc:3055] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:47.037451 12673 raft_consensus.cc:740] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:47.035245 15241 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:47.038976 12522 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:10:47.039194 12672 raft_consensus.cc:2376] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7. Re-sending same reply.
I20260709 12:10:47.028918 12529 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 7: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:47.040177 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:47.045260 12525 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:47.037755 15422 consensus_queue.cc:1059] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.007s
I20260709 12:10:47.049496 15426 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:10:47.063712 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 202133us
I20260709 12:10:47.050714 15229 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:47.052685 15241 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:47.038522 15233 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 1, Time since last communication: 0.015s
I20260709 12:10:47.053913 15411 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:47.058105 15354 raft_consensus.cc:3055] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:47.039948 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:47.038614 12673 consensus_queue.cc:260] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:47.049896 15441 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:47.073207 15199 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 12:10:47.074287 15336 raft_consensus.cc:2764] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 12:10:47.073813 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 48445.3us
I20260709 12:10:47.076318 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:47.077358 15336 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:47.078334 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:47.080699 12529 raft_consensus.cc:2376] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6. Re-sending same reply.
I20260709 12:10:47.081192 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:47.084105 12524 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260709 12:10:47.099311 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:47.037285 15237 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.044s
W20260709 12:10:47.105530 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 45931.1us
I20260709 12:10:47.107900 12522 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:10:47.108196 15380 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.689s	user 0.008s	sys 0.006s
I20260709 12:10:47.080367 15336 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:47.113747 12673 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:47.082480 15354 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:10:47.135742 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 46020.9us
I20260709 12:10:47.069110 15430 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:10:47.142831 12680 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:10:47.143998 12674 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:47.146003 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:47.133878 15336 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:47.153106 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 93086.8us
I20260709 12:10:47.157979 12522 raft_consensus.cc:2376] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8. Re-sending same reply.
I20260709 12:10:47.164829 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:47.161078 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:47.171240 15354 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:47.172767 15354 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:47.148883 15429 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:47.181797 12524 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:47.202483 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:47.206172 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:47.207918 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:47.210088 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:47.210891 12530 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:47.211862 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:47.213024 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:47.213652 12530 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:47.220942 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:47.222617 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:47.222815 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 47458.9us
I20260709 12:10:47.232296 12527 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7. Ops: []
W20260709 12:10:47.232612 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:47.233188 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:47.233075 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:47.225107 12530 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:47.234767 12524 raft_consensus.cc:2393] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 8: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:47.238723 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:47.237231 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:47.234645 15422 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Stepping down as leader of term 4
W20260709 12:10:47.246027 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 103337us
I20260709 12:10:47.254357 12531 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:47.229041 15426 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:47.182586 15180 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 12:10:47.262241 12523 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:47.262487 15448 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:47.226801 15413 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:47.178041 15334 consensus_queue.cc:1059] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.079s
I20260709 12:10:47.289398 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:47.292307 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:47.291246 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1875) took 4118 ms. Trace:
I20260709 12:10:47.294298 12598 raft_consensus.cc:2376] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8. Re-sending same reply.
I20260709 12:10:47.300932 12673 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
W20260709 12:10:47.302690 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:47.247902 15422 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:47.304509 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:47.297645 12679 rpcz_store.cc:276] 0709 12:10:43.172430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:44.388429 (+1215999us) service_pool.cc:224] Handling call
0709 12:10:44.415396 (+ 26967us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:46.678302 (+2262906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:47.271362 (+593060us) spinlock_profiling.cc:243] Waited 2.22 s on lock 0x7b5400012cd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:47.291152 (+ 19790us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5112484096}
I20260709 12:10:47.304685 12605 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:10:47.270696 15426 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:47.277642 15199 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:10:47.310873 15442 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:10:47.311757 15426 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:47.313163 15426 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:47.301636 12673 pending_rounds.cc:85] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 5
I20260709 12:10:47.315575 12673 pending_rounds.cc:107] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260709 12:10:47.315936 12673 raft_consensus.cc:2889] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:47.318248 15303 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:47.322214 12522 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:10:47.322968 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50914.8us
I20260709 12:10:47.325055 15241 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:47.325646 15241 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:47.309522 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:47.331640 12526 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:47.277560 15237 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.052s
W20260709 12:10:47.350427 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:47.329680 15303 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:47.363960 15422 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:47.385607 15422 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:10:47.387208 15444 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:10:47.388486 15447 raft_consensus.cc:3055] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:47.389385 15447 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:47.380209 15267 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:47.363458 13305 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:47.293255 15448 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:47.395136 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 111249us
I20260709 12:10:47.399137 15412 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
I20260709 12:10:47.401192 15448 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:47.404816 15447 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:47.407157 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:47.366422 15449 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 12:10:47.416699 15447 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:47.416939 15413 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:47.418447 12601 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 7. Current term is 9. Ops: []
I20260709 12:10:47.418740 15429 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:47.419236 15429 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:47.420471 15429 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:47.380848 15199 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.031s
I20260709 12:10:47.413365 15210 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:10:47.415086 15458 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:10:47.458698 13302 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:10:47.464177 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 217581us
I20260709 12:10:47.463284 15210 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:47.539472 15210 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:47.548082 13305 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:47.539829 15441 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:10:47.558561 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 376617us
W20260709 12:10:47.548959 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:47.551530 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:47.553565 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:47.543550 15459 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.145s	user 0.002s	sys 0.000s
W20260709 12:10:47.565580 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:47.553879 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:47.571986 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 129901us
I20260709 12:10:47.576108 15426 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:47.574419 15442 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:47.574606 15377 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:47.546092 15324 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:47.543848 15312 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:47.585953 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 260174us
W20260709 12:10:47.591657 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 129309us
I20260709 12:10:47.583593 13302 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:47.575434 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 419261us
I20260709 12:10:47.550964 15429 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:47.584525 15210 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:47.582783 15357 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:10:47.605870 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:47.608700 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 81222.7us
I20260709 12:10:47.605986 15454 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:10:47.593668 15441 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 8 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:47.593087 15426 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:47.616032 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:47.584235 15267 consensus_queue.cc:1059] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.034s
I20260709 12:10:47.611694 12643 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:47.620664 12678 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:47.614346 12646 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:47.625506 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:47.541361 15241 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:47.631459 15441 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:47.611999 15357 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:47.619256 15454 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:47.626740 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:47.604882 15324 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:47.641523 12679 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:47.654457 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:47.641283 15357 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:47.656692 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 44426.2us
W20260709 12:10:47.659636 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 57722.2us
I20260709 12:10:47.653311 12676 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:47.660238 12646 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:47.612584 15377 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:47.619082 15340 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.030s
I20260709 12:10:47.658207 12425 catalog_manager.cc:5697] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 4 to 5, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 5 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
W20260709 12:10:47.719239 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 59112.4us
I20260709 12:10:47.676095 15241 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:47.676947 15426 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:47.678092 12427 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47524 (request call id 94) took 19586 ms (client timeout 15000 ms). Trace:
I20260709 12:10:47.686882 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:47.688112 15229 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:47.689275 15411 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:10:47.691243 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:47.694414 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:47.696097 15336 consensus_meta.cc:287] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.512s	user 0.012s	sys 0.001s
W20260709 12:10:47.710834 15458 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:10:47.645037 15454 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:10:47.723383 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 72050.7us
I20260709 12:10:47.668666 15324 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:47.724802 12646 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:47.652395 15149 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:10:47.728996 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:47.729997 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:47.742179 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:47.717422 15312 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:47.734616 15411 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:47.765300 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:47.767431 12604 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 3. (term mismatch)
I20260709 12:10:47.746717 15426 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:47.767982 12604 pending_rounds.cc:85] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 2
W20260709 12:10:47.773965 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 87157.5us
I20260709 12:10:47.776963 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:47.777822 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:47.775918 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 192448us
I20260709 12:10:47.765700 12496 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:47.780197 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:47.747406 15312 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:47.774643 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:47.788529 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:47.789968 12600 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:47.782110 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:47.791177 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 1893) took 1063 ms. Trace:
I20260709 12:10:47.767884 15324 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:47.802675 12602 raft_consensus.cc:2410] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:10:47.798700 12604 pending_rounds.cc:107] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:10:47.803642 12604 raft_consensus.cc:2889] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:47.742058 15336 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:47.800927 12601 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:47.791589 12676 rpcz_store.cc:276] 0709 12:10:46.714087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:46.715623 (+  1536us) service_pool.cc:224] Handling call
0709 12:10:47.777275 (+1061652us) spinlock_profiling.cc:243] Waited 937 ms on lock 0x7b54000c2bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:47.777435 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2154057472}
I20260709 12:10:47.789613 15312 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:47.836035 15341 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:47.836503 15411 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:47.837178 15341 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:47.838303 15231 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:47.838431 15341 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:47.839030 15231 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:47.839592 15231 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:47.842589 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:47.844393 15149 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:47.845515 12597 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:47.846103 15149 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:10:47.846982 15149 consensus_queue.cc:1243] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 4.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:10:47.848973 12602 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:47.852718 12425 catalog_manager.cc:5697] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 2 to 3, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 3 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:47.866055 12606 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:47.865034 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:47.889147 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:47.889956 12597 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:47.893661 12600 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 12:10:47.896355 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:47.816409 15463 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:10:47.915982 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 46410.6us
I20260709 12:10:47.924621 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:47.917697 15447 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:47.921795 15463 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:47.950382 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:47.952929 15469 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:10:47.965752 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 157062us
W20260709 12:10:47.966338 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 278130us
W20260709 12:10:47.971238 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 157277us
W20260709 12:10:47.731690 12427 rpcz_store.cc:269] 0709 12:10:28.318281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:28.368519 (+ 50238us) service_pool.cc:224] Handling call
0709 12:10:47.677719 (+19309200us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:38.858194 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:38.858864 (+   670us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:38.858920 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:10:38.858939 (+    19us) tablet.cc:662] Decoding operations
0709 12:10:38.859827 (+   888us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:38.859909 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:38.859937 (+    28us) tablet.cc:685] Acquiring locks for 6 operations
0709 12:10:38.860593 (+   656us) tablet.cc:701] Row locks acquired
0709 12:10:38.860636 (+    43us) write_op.cc:260] PREPARE: finished
0709 12:10:38.860820 (+   184us) write_op.cc:270] Start()
0709 12:10:38.860976 (+   156us) write_op.cc:276] Timestamp: P: 1783599038860741 usec, L: 0
0709 12:10:38.861001 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:10:38.961791 (+100790us) log.cc:844] Serialized 2451 byte log entry
0709 12:10:41.313806 (+2352015us) op_driver.cc:464] REPLICATION: finished
0709 12:10:41.516632 (+202826us) write_op.cc:301] APPLY: starting
0709 12:10:41.516749 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:41.516935 (+   186us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:41.517069 (+   134us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:41.517820 (+   751us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:41.518089 (+   269us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=6
0709 12:10:41.524139 (+  6050us) write_op.cc:312] APPLY: finished
0709 12:10:41.542638 (+ 18499us) log.cc:844] Serialized 52 byte log entry
0709 12:10:41.542839 (+   201us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:41.542966 (+   127us) write_op.cc:454] Released schema lock
0709 12:10:41.543017 (+    51us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:41.543508 (+   491us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:41.543536 (+    28us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":636684,"reported_tablets":15,"spinlock_wait_cycles":9600,"child_traces":[["op",{"apply.queue_time_us":186984,"apply.run_cpu_time_us":2786,"apply.run_wall_time_us":26817,"num_ops":6,"prepare.queue_time_us":418353,"prepare.run_cpu_time_us":17206,"prepare.run_wall_time_us":265020,"raft.queue_time_us":343893,"raft.run_cpu_time_us":66841,"raft.run_wall_time_us":247536,"replication_time_us":2411411,"thread_start_us":662549,"threads_started":4,"wal-append.queue_time_us":245900,"wal-append.run_cpu_time_us":48036,"wal-append.run_wall_time_us":3327264}]]}
I20260709 12:10:47.944473 15411 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:47.989079 12496 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:47.990878 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:47.961896 15463 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:47.991743 12528 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:10:47.996323 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:48.008229 12425 catalog_manager.cc:5697] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 1 to 5, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 5 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:48.022725 15413 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:48.023406 15413 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:48.025112 12425 catalog_manager.cc:5697] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 1 to 5, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 5 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:48.012888 15312 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:48.033082 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54615.5us
I20260709 12:10:48.014271 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:48.034447 12603 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:10:48.015110 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:48.015633 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.012715 15210 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:48.026209 15422 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:48.027035 12494 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:48.049201 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:48.027371 15377 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:48.030225 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:48.034605 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:48.039563 12679 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
W20260709 12:10:48.040048 12643 connection.cc:256] RPC call timeout handler was delayed by -2.9142908520000219s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20390000000000003s for a response
W20260709 12:10:48.053983 12643 connection.cc:256] RPC call timeout handler was delayed by -2.9069567589999679s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.29700000000000004s for a response
I20260709 12:10:48.051640 12673 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
W20260709 12:10:48.044513 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.058686 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:48.059155 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:48.061199 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:48.061935 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.018684 15336 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:48.051636 12525 pending_rounds.cc:85] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 1
I20260709 12:10:48.064957 12525 pending_rounds.cc:107] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 4.2
I20260709 12:10:48.065295 12525 raft_consensus.cc:2889] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:48.069130 12597 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:48.072763 12606 raft_consensus.cc:2410] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 8 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:10:48.073817 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:48.082227 12494 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:48.096019 15341 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:48.100526 12425 catalog_manager.cc:5697] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 1 to 2, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 2 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:48.102545 15312 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:48.055989 15241 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:48.103549 15241 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:48.104044 15241 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:48.104619 15467 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.214s	user 0.003s	sys 0.001s
I20260709 12:10:48.105033 15223 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:10:48.109174 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 74630.4us
I20260709 12:10:48.110266 12496 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:48.055708 15453 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:10:48.137063 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.052392 15193 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.060s
I20260709 12:10:48.055282 15229 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.078s
I20260709 12:10:48.148294 12646 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:48.153422 12605 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:48.144199 12494 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:48.159055 12425 catalog_manager.cc:5697] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 3 to 4, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 4 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:48.130782 15298 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:48.131388 15453 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:48.155293 15442 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.556s	user 0.014s	sys 0.000s
I20260709 12:10:48.171191 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:48.172098 12526 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:48.174273 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:48.175208 12530 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:48.178687 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 68562.1us
I20260709 12:10:48.180095 12496 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:48.164539 15312 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:10:48.196556 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 405436us
I20260709 12:10:48.168308 15453 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:48.165264 15298 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:48.209218 12425 catalog_manager.cc:5697] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:48.162407 15223 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.036s
I20260709 12:10:48.185999 15442 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:48.162689 15462 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:48.258734 12603 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:10:48.199842 15312 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:48.174297 15422 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.065s
W20260709 12:10:48.200568 15429 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.593s	user 0.010s	sys 0.005s
I20260709 12:10:48.212030 15430 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:48.256093 15223 consensus_queue.cc:1243] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 3.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:10:48.288154 12528 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:10:48.291661 12526 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
W20260709 12:10:48.264972 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 278634us
I20260709 12:10:48.238108 15461 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:48.296326 12646 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:48.241844 15298 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:48.261121 15341 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:48.265722 15469 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 6 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:48.259985 15480 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:48.306675 12425 catalog_manager.cc:5697] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: config changed from index -1 to 2, term changed from 1 to 3, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to <none>, NON_VOTER e56d487365394ac5ac5f1002926a5645 (127.11.13.132) added. New cstate: current_term: 3 committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:10:48.303175 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:48.266719 15312 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:48.264041 15193 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:10:48.325395 12524 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:10:48.276533 15430 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:48.326074 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:48.387770 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:48.423696 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.296281 15461 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:48.358285 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.270754 15465 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 12:10:48.355432 12425 catalog_manager.cc:5697] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 1 to 3, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 3 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:10:48.430261 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.345758 15441 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:48.349015 15478 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:48.305464 15422 consensus_queue.cc:1243] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:10:48.426513 15475 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.197s
I20260709 12:10:48.429821 15461 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:48.354000 15231 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:48.479709 12425 catalog_manager.cc:5697] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:10:48.506155 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:48.487344 15478 raft_consensus.cc:697] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:10:48.514984 13304 leader_election.cc:336] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 3.214s (SENT)
W20260709 12:10:48.525447 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:48.525386 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.361698 15480 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:48.534392 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 347345us
I20260709 12:10:48.484302 15441 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:48.540407 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:48.548137 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:48.534715 15474 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.357s	user 0.001s	sys 0.001s
W20260709 12:10:48.588871 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 386122us
W20260709 12:10:48.598503 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 58185.4us
I20260709 12:10:48.599083 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:48.604688 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.599772 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:48.550314 15454 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.787s	user 0.013s	sys 0.000s
W20260709 12:10:48.618664 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 113569us
W20260709 12:10:48.617565 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 69375.4us
I20260709 12:10:48.623402 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:48.618652 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:48.626319 15430 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:48.631579 15481 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
W20260709 12:10:48.632984 15477 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.373s	user 0.000s	sys 0.001s
W20260709 12:10:48.643061 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 462934us
W20260709 12:10:48.632884 15426 consensus_meta.cc:287] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.833s	user 0.012s	sys 0.001s
I20260709 12:10:48.648923 12496 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:48.686677 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:48.641376 15193 raft_consensus.cc:2764] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 12:10:48.691836 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.691161 12602 pending_rounds.cc:85] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 1
I20260709 12:10:48.704918 12602 pending_rounds.cc:107] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260709 12:10:48.705345 12602 raft_consensus.cc:2889] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:48.705717 13335 raft_consensus.cc:2410] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:10:48.622256 15454 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:48.713845 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:48.659320 15500 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:10:48.725135 12643 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.003s	sys 0.010s
W20260709 12:10:48.725533 12643 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.003s	sys 0.010s
W20260709 12:10:48.725811 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 645041us
I20260709 12:10:48.549113 15485 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.109s
I20260709 12:10:48.694269 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:48.726796 13332 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
W20260709 12:10:48.710175 15476 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.413s	user 0.000s	sys 0.001s
W20260709 12:10:48.774782 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 126062us
W20260709 12:10:48.782023 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.607499 15469 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:48.788981 12530 consensus_meta.cc:287] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.605s	user 0.008s	sys 0.004s
I20260709 12:10:48.805578 12530 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:48.727833 15430 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:48.814458 12529 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.013s	sys 0.058s
W20260709 12:10:48.814833 12529 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.013s	sys 0.058s
W20260709 12:10:48.716387 15210 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.010s	sys 0.000s
I20260709 12:10:48.812789 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:48.734536 15500 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:48.811385 15426 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:48.810170 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:48.817581 15210 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.011s	sys 0.000s
W20260709 12:10:48.862455 12643 leader_election.cc:336] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.109s (SENT)
I20260709 12:10:48.867691 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:48.868374 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:48.870388 15472 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:48.871309 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:48.875013 15484 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.207s	user 0.001s	sys 0.000s
W20260709 12:10:48.878902 15491 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
W20260709 12:10:48.880028 15487 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.161s	user 0.000s	sys 0.000s
I20260709 12:10:48.715512 15478 consensus_queue.cc:237] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:48.882611 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1679) took 1266 ms. Trace:
I20260709 12:10:48.883252 12674 rpcz_store.cc:276] 0709 12:10:47.616112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:47.620730 (+  4618us) service_pool.cc:224] Handling call
0709 12:10:47.622790 (+  2060us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:47.923470 (+300680us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:47.923618 (+   148us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:47.942716 (+ 19098us) log.cc:844] Serialized 52 byte log entry
0709 12:10:48.266729 (+324013us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:48.675030 (+408301us) log.cc:844] Serialized 14 byte log entry
0709 12:10:48.675376 (+   346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:48.675510 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:48.882259 (+206749us) raft_consensus.cc:1685] finished
0709 12:10:48.882293 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:48.882519 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2550,"raft.queue_time_us":22447,"raft.run_cpu_time_us":394,"raft.run_wall_time_us":11200,"thread_start_us":916391,"threads_started":3,"wal-append.queue_time_us":636263}
I20260709 12:10:48.886773 15479 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:10:48.888196 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 62580.9us
I20260709 12:10:48.888518 15498 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:48.889362 15498 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:48.890586 15498 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:48.891623 15231 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:48.890522 15493 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:10:48.894223 15495 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:48.891206 15454 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:48.894986 15495 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:48.896098 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:48.897600 15495 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:48.899247 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 230300us
I20260709 12:10:48.896190 15413 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:48.901022 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.913916 12496 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:48.924930 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 54935.2us
W20260709 12:10:48.934321 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:48.941300 12598 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:10:48.863075 15357 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.142s	user 0.013s	sys 0.002s
I20260709 12:10:48.960948 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:48.891484 15490 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 12:10:48.963207 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 63124.4us
I20260709 12:10:48.891743 13330 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 4 LEARNER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260709 12:10:48.899295 15344 raft_consensus.cc:3055] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:48.836668 15223 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.123s
I20260709 12:10:48.894589 15441 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:48.978893 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 170547us
I20260709 12:10:48.977788 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:48.913321 15426 raft_consensus.cc:697] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:49.005426 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:48.912727 13331 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:10:49.015929 12674 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
W20260709 12:10:49.060868 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:49.051508 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:49.109660 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:49.041929 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:49.059541 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46446.9us
W20260709 12:10:49.031445 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 47741.2us
I20260709 12:10:49.102882 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:49.111063 12494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.011s	sys 0.046s
W20260709 12:10:49.111557 12494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.011s	sys 0.046s
I20260709 12:10:49.111657 12601 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:49.049953 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:49.112414 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:49.114169 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:49.110057 12676 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
W20260709 12:10:48.996523 15411 consensus_meta.cc:287] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.014s	user 0.003s	sys 0.010s
W20260709 12:10:49.102555 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 51725.7us
W20260709 12:10:49.119458 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 205186us
W20260709 12:10:49.119824 12643 connection.cc:256] RPC call timeout handler was delayed by -2.7775431110001136s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1656s for a response
I20260709 12:10:49.126539 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:49.129806 15453 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.855s	user 0.012s	sys 0.000s
I20260709 12:10:49.130409 15453 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:49.130841 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1748) took 1154 ms. Trace:
I20260709 12:10:49.094036 15426 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:49.131973 13331 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:10:49.131420 12529 rpcz_store.cc:276] 0709 12:10:47.976318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:47.976497 (+   179us) service_pool.cc:224] Handling call
0709 12:10:47.976928 (+   431us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:47.977215 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:47.977254 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:47.983123 (+  5869us) log.cc:844] Serialized 26 byte log entry
0709 12:10:48.815189 (+832066us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:48.815226 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:48.815298 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:49.129909 (+314611us) raft_consensus.cc:1685] finished
0709 12:10:49.129928 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:49.130597 (+   669us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":743711,"threads_started":1,"wal-append.queue_time_us":1013196}
I20260709 12:10:49.132874 15453 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:49.137116 15447 consensus_meta.cc:287] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.190s	user 0.013s	sys 0.000s
I20260709 12:10:49.058787 15509 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:48.999750 15480 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:49.136969 15241 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.032s	user 0.009s	sys 0.004s
I20260709 12:10:49.157986 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:49.158735 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:49.137854 15447 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:48.988162 15344 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:49.167778 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:49.169739 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:49.171314 12677 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260709 12:10:49.186566 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:49.167677 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:49.099191 15193 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:48.978708 15210 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.178s
I20260709 12:10:49.198393 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:48.973146 15357 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:49.202608 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 83822.5us
W20260709 12:10:49.213392 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54291.8us
I20260709 12:10:49.212596 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:49.304379 13302 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:49.049765 15494 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:10:49.164920 15461 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.652s	user 0.011s	sys 0.003s
W20260709 12:10:49.294651 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 84826.2us
W20260709 12:10:49.210743 12531 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.788s	user 0.008s	sys 0.004s
I20260709 12:10:49.058529 15450 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.252s
I20260709 12:10:49.057619 15475 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.139s
I20260709 12:10:49.050405 15508 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.260s
I20260709 12:10:49.150604 15509 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:10:49.322767 12643 leader_election.cc:336] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 3.217s (SENT)
I20260709 12:10:49.159386 15453 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:49.307710 12531 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:49.298856 15480 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:49.315269 13304 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:49.298673 15199 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:49.299901 12522 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:49.323240 12679 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 2. Preceding OpId from leader: term: 8 index: 3. (index mismatch)
I20260709 12:10:49.323769 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 1697) took 1036 ms. Trace:
I20260709 12:10:49.312247 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:49.329319 12531 rpcz_store.cc:276] 0709 12:10:48.287451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:48.287969 (+   518us) service_pool.cc:224] Handling call
0709 12:10:49.323700 (+1035731us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:49.329563 12672 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:49.325089 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:49.309881 15463 consensus_meta.cc:287] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.291s	user 0.006s	sys 0.009s
I20260709 12:10:49.302381 15411 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:49.331894 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:49.206202 15241 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:49.344184 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:49.114063 15500 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:49.323890 15485 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:10:49.326866 15475 consensus_queue.cc:1243] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:10:49.362383 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:49.324652 15193 raft_consensus.cc:697] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:49.327644 15480 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:49.365818 12672 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:49.328339 15509 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:49.392145 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:49.335564 15453 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:49.367249 15312 consensus_meta.cc:287] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.888s	user 0.015s	sys 0.000s
I20260709 12:10:49.401859 15480 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:49.417214 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54971.3us
I20260709 12:10:49.341662 15199 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:49.361938 15469 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:49.422245 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 125053us
I20260709 12:10:49.313773 15461 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:49.376754 12572 connection.cc:256] RPC call timeout handler was delayed by -5.3023556900000131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.28700000000000003s for a response
I20260709 12:10:49.423352 15324 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:49.343596 15463 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:49.321157 15494 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:49.416028 12675 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260709 12:10:49.349383 15411 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:49.434396 12678 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:10:49.427464 13333 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:49.429518 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:49.424599 15478 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:49.429692 15525 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:49.448331 12572 connection.cc:256] RPC call timeout handler was delayed by -5.2024556900000789s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2981s for a response
W20260709 12:10:49.449800 12572 connection.cc:256] RPC call timeout handler was delayed by -4.9513556900001277s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.326s for a response
W20260709 12:10:49.450171 12572 connection.cc:256] RPC call timeout handler was delayed by -4.0115684159998182s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17820000000000003s for a response
W20260709 12:10:49.450438 12572 connection.cc:256] RPC call timeout handler was delayed by -4.0061684159998094s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17880000000000001s for a response
W20260709 12:10:49.450688 12572 connection.cc:256] RPC call timeout handler was delayed by -3.8387684159999935s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19740000000000002s for a response
W20260709 12:10:49.450969 12572 connection.cc:256] RPC call timeout handler was delayed by -3.7346407469999576s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.43570000000000003s for a response
W20260709 12:10:49.451231 12572 connection.cc:256] RPC call timeout handler was delayed by -3.4265684160000092s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2432s for a response
W20260709 12:10:49.451475 12572 connection.cc:256] RPC call timeout handler was delayed by -3.1196684159999677s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.27730000000000005s for a response
W20260709 12:10:49.451718 12572 connection.cc:256] RPC call timeout handler was delayed by -3.0269684159998178s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2876s for a response
W20260709 12:10:49.451956 12572 connection.cc:256] RPC call timeout handler was delayed by -2.7211556900001597s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.57380000000000009s for a response
W20260709 12:10:49.452209 12572 connection.cc:256] RPC call timeout handler was delayed by -2.428468415999987s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3541s for a response
W20260709 12:10:49.441099 15505 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:10:49.459137 12679 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:49.435803 15522 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:49.371809 15344 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:49.468515 12496 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:49.466336 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 153776us
W20260709 12:10:49.456303 15486 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.394s	user 0.002s	sys 0.000s
I20260709 12:10:49.486631 15525 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:49.487571 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1882) took 1492 ms. Trace:
I20260709 12:10:49.487735 15199 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:49.488029 12604 rpcz_store.cc:276] 0709 12:10:47.995217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:47.995363 (+   146us) service_pool.cc:224] Handling call
0709 12:10:48.010579 (+ 15216us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:48.011017 (+   438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:48.011044 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:48.011582 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 12:10:48.450867 (+439285us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:48.450931 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:48.451033 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:49.487366 (+1036333us) raft_consensus.cc:1685] finished
0709 12:10:49.487384 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:49.487509 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7332,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"thread_start_us":438846,"threads_started":1,"wal-append.queue_time_us":854773}
I20260709 12:10:49.476229 15241 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:49.477221 15534 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:10:49.479652 15537 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:10:49.482933 15529 log.cc:927] Time spent T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:10:49.483521 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1887) took 1471 ms. Trace:
I20260709 12:10:49.531484 12606 rpcz_store.cc:276] 0709 12:10:48.011872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:48.087309 (+ 75437us) service_pool.cc:224] Handling call
0709 12:10:48.135892 (+ 48583us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:48.554274 (+418382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:48.554325 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:48.554976 (+   651us) log.cc:844] Serialized 26 byte log entry
0709 12:10:48.966897 (+411921us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:48.974279 (+  7382us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:48.974465 (+   186us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:49.483284 (+508819us) raft_consensus.cc:1685] finished
0709 12:10:49.483308 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:49.483457 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3312,"raft.queue_time_us":508775,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":13671,"thread_start_us":727614,"threads_started":2,"wal-append.queue_time_us":866485}
W20260709 12:10:49.536638 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:49.475076 15411 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:49.433949 15312 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:49.489145 15525 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:49.507004 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:49.508288 12496 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:49.508667 15500 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:49.441262 15324 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:49.553586 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:49.554100 12643 connection.cc:256] RPC call timeout handler was delayed by -2.4138908520001223s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2595s for a response
W20260709 12:10:49.554482 12643 connection.cc:256] RPC call timeout handler was delayed by -2.3272591979998651s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.4289s for a response
W20260709 12:10:49.558172 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49949.9us
I20260709 12:10:49.559214 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:49.457563 15519 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
I20260709 12:10:49.569926 12496 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:49.571993 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:49.575371 13335 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:49.542979 15522 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:49.571288 15525 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:49.575098 15426 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:49.577911 15478 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:49.667635 15478 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:49.560840 15511 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.108s
I20260709 12:10:49.668910 15511 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:49.669315 15511 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:49.670070 15478 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:49.635468 13304 leader_election.cc:336] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.776s (SENT)
I20260709 12:10:49.670324 15511 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:49.671053 13304 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:49.672461 15511 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:49.672951 15430 consensus_meta.cc:287] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.854s	user 0.010s	sys 0.000s
I20260709 12:10:49.673019 15312 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:49.581156 15344 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:49.546725 15193 consensus_queue.cc:237] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:49.677008 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 166956us
I20260709 12:10:49.677881 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1749) took 1683 ms. Trace:
I20260709 12:10:49.578457 15411 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.105s
I20260709 12:10:49.678387 12525 rpcz_store.cc:276] 0709 12:10:47.994428 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:47.994563 (+   135us) service_pool.cc:224] Handling call
0709 12:10:48.041200 (+ 46637us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:10:48.212049 (+170849us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:48.212079 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:10:48.264589 (+ 52510us) log.cc:844] Serialized 78 byte log entry
0709 12:10:48.723758 (+459169us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:48.723883 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:48.724103 (+   220us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:49.677662 (+953559us) raft_consensus.cc:1685] finished
0709 12:10:49.677684 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:49.677822 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13411,"raft.queue_time_us":74951,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":20419,"thread_start_us":396912,"threads_started":1,"wal-append.queue_time_us":818386}
I20260709 12:10:49.680684 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:49.691809 15430 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:49.693429 15426 raft_consensus.cc:3055] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:49.693957 15426 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:49.695410 15426 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:49.699819 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:49.706084 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 143226us
I20260709 12:10:49.708956 12494 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:49.710274 15537 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:49.710807 15525 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:49.685259 15539 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:49.720979 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:49.730086 15501 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:49.719305 15461 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:49.730669 15522 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:49.734943 15532 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 12:10:49.735772 15411 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:10:49.751631 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:49.758637 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1870) took 2009 ms. Trace:
I20260709 12:10:49.759295 12605 rpcz_store.cc:276] 0709 12:10:47.749705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:47.782037 (+ 32332us) service_pool.cc:224] Handling call
0709 12:10:47.814802 (+ 32765us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:48.152636 (+337834us) spinlock_profiling.cc:243] Waited 4.65 ms on lock 0x7b540002d120. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488997dc0 00007f1488a221c0 00007f1488a23231 00007f1488a1fd8f 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c
0709 12:10:48.155840 (+  3204us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:49.758225 (+1602385us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b5400375354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:49.758521 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":410398848}
I20260709 12:10:49.761926 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:49.762840 12673 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:49.763782 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:49.764353 12673 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:10:49.764784 15324 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:49.765376 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:49.772811 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:49.729336 15465 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:49.786525 15453 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:49.790606 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:49.790681 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:49.792136 12680 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:49.793542 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:49.793673 12673 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:49.794548 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:49.765426 15463 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:49.794679 12643 connection.cc:256] RPC call timeout handler was delayed by -1.9053908519999823s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31600000000000006s for a response
I20260709 12:10:49.798473 12604 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:10:49.795342 12675 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
W20260709 12:10:49.821388 12643 connection.cc:256] RPC call timeout handler was delayed by -1.666715718000205s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
I20260709 12:10:49.789397 15229 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:49.795212 15199 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:49.830595 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:49.795214 15525 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:49.806155 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:49.808022 12598 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:10:49.810225 13332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 251) took 1031 ms. Trace:
I20260709 12:10:49.793084 15537 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:49.813565 13333 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38482 (request call id 351) took 1105 ms. Trace:
I20260709 12:10:49.817740 15509 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:49.840301 13333 rpcz_store.cc:276] 0709 12:10:48.708311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:48.779698 (+ 71387us) service_pool.cc:224] Handling call
0709 12:10:48.835295 (+ 55597us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:49.443292 (+607997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:49.813297 (+370005us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b540055ee54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:49.813492 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1128521984}
I20260709 12:10:49.841084 12597 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:49.818400 15480 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:49.821898 12643 connection.cc:256] RPC call timeout handler was delayed by -1.565915718000042s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16310000000000002s for a response
I20260709 12:10:49.822788 12680 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:49.825361 13335 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:10:49.838711 13332 rpcz_store.cc:276] 0709 12:10:48.779401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:48.783565 (+  4164us) service_pool.cc:224] Handling call
0709 12:10:49.809442 (+1025877us) spinlock_profiling.cc:243] Waited 5.24 ms on lock 0x7b5800163298. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:10:49.809652 (+   210us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:49.809853 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:49.809877 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:49.809918 (+    41us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:49.809955 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:49.810011 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:49.810160 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12044032}
W20260709 12:10:49.796764 15483 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 1.265s	user 0.003s	sys 0.007s
I20260709 12:10:49.755906 15500 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260709 12:10:49.787802 15324 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:49.854641 12673 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
W20260709 12:10:49.845189 12643 connection.cc:256] RPC call timeout handler was delayed by -1.3121157180000864s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19130000000000003s for a response
I20260709 12:10:49.858868 12671 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: [5.2-5.2]
W20260709 12:10:49.861493 12643 connection.cc:256] RPC call timeout handler was delayed by -1.2743157180002527s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1955s for a response
W20260709 12:10:49.861910 12643 connection.cc:256] RPC call timeout handler was delayed by -1.2050157180001406s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20320000000000002s for a response
I20260709 12:10:49.843530 15229 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:49.862615 15324 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:49.862319 12643 connection.cc:256] RPC call timeout handler was delayed by -1.0997157180001977s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2149s for a response
I20260709 12:10:49.864682 15324 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:49.864837 12643 connection.cc:256] RPC call timeout handler was delayed by -1.0862157180001759s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21640000000000004s for a response
W20260709 12:10:49.865041 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 59052.9us
I20260709 12:10:49.864718 15229 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:49.840328 15537 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:10:49.877349 15298 consensus_meta.cc:287] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.524s	user 0.016s	sys 0.000s
I20260709 12:10:49.877862 15298 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:49.879086 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:49.880213 15298 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:49.886168 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:49.888069 12572 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:49.830920 15494 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:49.894996 12673 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:49.902292 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:49.903148 12673 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 8: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:49.845335 15199 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:49.907464 12673 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:49.871189 15337 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:49.896378 15498 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:49.902916 13331 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:49.868650 15525 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:49.918857 12673 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
W20260709 12:10:49.928411 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 61018.4us
I20260709 12:10:49.930810 12673 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:49.912410 15324 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:49.948979 15503 raft_consensus.cc:3055] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:49.949805 15503 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:49.950219 15537 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:49.951220 15503 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:49.953301 15503 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:49.953946 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:49.946447 15324 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:49.963579 15324 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:49.967567 15552 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.170s	user 0.000s	sys 0.002s
W20260709 12:10:49.971347 15554 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:10:49.972482 15426 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:49.972977 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1889) took 1349 ms. Trace:
I20260709 12:10:49.980574 12602 rpcz_store.cc:276] 0709 12:10:48.636015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:48.647206 (+ 11191us) service_pool.cc:224] Handling call
0709 12:10:48.688825 (+ 41619us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:48.767058 (+ 78233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:48.767089 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:49.743669 (+976580us) spinlock_profiling.cc:243] Waited 18.2 ms on lock 0x7b2c00000e98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:10:49.744756 (+  1087us) log.cc:844] Serialized 52 byte log entry
0709 12:10:49.835840 (+ 91084us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:49.837497 (+  1657us) log.cc:844] Serialized 14 byte log entry
0709 12:10:49.847386 (+  9889us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:49.848158 (+   772us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:49.972660 (+124502us) raft_consensus.cc:1685] finished
0709 12:10:49.972762 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:49.972905 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":983,"prepare.run_wall_time_us":5166,"raft.queue_time_us":194881,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":13050,"spinlock_wait_cycles":56545920,"thread_start_us":90497,"threads_started":1,"wal-append.queue_time_us":91003}
I20260709 12:10:49.971439 12496 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:49.986907 12496 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:49.987654 15480 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:49.988144 12496 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:49.978718 15199 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:49.991001 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 112026us
I20260709 12:10:49.987753 15537 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:49.976486 15525 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:49.993197 15430 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:49.994469 15229 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:49.994729 12496 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:49.995009 15229 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:49.995445 15229 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:49.996315 12572 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:49.997253 15489 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:49.998669 12572 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:49.999467 15489 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:50.000233 15298 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:50.000677 12496 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:50.001808 15462 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:50.001916 15538 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:50.000792 15298 raft_consensus.cc:697] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:50.002252 15462 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:50.003144 12496 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:50.003401 15462 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:50.005553 15465 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:50.005978 15465 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:50.006382 15465 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:50.003196 15298 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:50.014087 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:50.014232 15430 raft_consensus.cc:697] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:50.015347 15430 consensus_queue.cc:237] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:50.015690 15193 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:10:50.016424 15193 consensus_queue.cc:1243] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 2.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:10:50.019860 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:50.020594 12602 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:50.021335 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:50.022420 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:50.022936 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:50.023178 12603 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:50.023216 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:50.025488 13334 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 10: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:50.025982 13334 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 456) took 1203 ms. Trace:
I20260709 12:10:50.028422 13334 rpcz_store.cc:276] 0709 12:10:48.823100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:50.022896 (+1199796us) service_pool.cc:224] Handling call
0709 12:10:50.025878 (+  2982us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:50.028952 12646 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:50.036576 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:50.038826 12601 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:50.038903 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:50.039820 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:50.040027 12604 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:50.041260 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:50.041934 12605 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:50.045938 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:50.046933 12603 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:49.993747 15537 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:50.049484 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 50893.3us
I20260709 12:10:50.058270 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:50.059521 12675 raft_consensus.cc:2410] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:10:50.057019 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:50.064059 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:50.053364 12672 pending_rounds.cc:85] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 4
I20260709 12:10:50.065080 12674 raft_consensus.cc:2410] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:10:50.065310 12672 pending_rounds.cc:107] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 2.5
I20260709 12:10:50.065668 12672 raft_consensus.cc:2889] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:10:50.068049 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 45328.1us
I20260709 12:10:50.036484 15478 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:50.069957 12572 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:50.070658 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:50.043673 15344 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:50.046485 15509 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:50.077812 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:50.055676 15537 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:50.085302 15489 raft_consensus.cc:2764] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:10:50.122375 12603 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
W20260709 12:10:50.131807 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 46742.7us
I20260709 12:10:50.100077 15422 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:50.094604 15465 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:50.180491 12494 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:50.181407 15324 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:50.187331 15324 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:50.196058 12602 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:50.170496 15422 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:50.207625 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:50.196867 15478 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:50.220157 15478 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:50.221386 15561 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:10:50.221899 13339 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:50.222664 13339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 253) took 1327 ms. Trace:
I20260709 12:10:50.222973 13339 rpcz_store.cc:276] 0709 12:10:48.895074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:48.941798 (+ 46724us) service_pool.cc:224] Handling call
0709 12:10:50.221445 (+1279647us) spinlock_profiling.cc:243] Waited 8.09 ms on lock 0x7b5800163298. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:10:50.221695 (+   250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:50.222368 (+   673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:50.222603 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":18608384}
W20260709 12:10:50.224679 15511 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.552s	user 0.010s	sys 0.002s
I20260709 12:10:50.207446 15324 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:50.234393 15324 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:50.240546 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 43704.5us
I20260709 12:10:50.196460 15538 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election won for term 6
W20260709 12:10:50.217917 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:50.251432 12572 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:50.268998 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:50.261214 15442 raft_consensus.cc:3055] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:50.270634 15442 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:50.271051 15465 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:50.271833 15442 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:50.267208 15511 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:50.262590 15538 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:50.275421 15538 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:50.279022 15563 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
W20260709 12:10:50.281594 15565 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:10:50.293426 15357 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:50.294332 15510 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:50.244721 15567 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:10:50.298890 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:50.299492 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 213) took 1174 ms. Trace:
I20260709 12:10:50.300938 15422 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:50.293787 12572 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:50.299925 12524 rpcz_store.cc:276] 0709 12:10:49.125219 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:49.125405 (+   186us) service_pool.cc:224] Handling call
0709 12:10:49.169250 (+ 43845us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:49.446682 (+277432us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:10:49.576407 (+129725us) log.cc:844] Serialized 14 byte log entry
0709 12:10:49.831750 (+255343us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:49.853960 (+ 22210us) log.cc:844] Serialized 26 byte log entry
0709 12:10:49.854213 (+   253us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:49.854249 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:50.299034 (+444785us) spinlock_profiling.cc:243] Waited 272 ms on lock 0x7b540027fbd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:50.299087 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:50.299141 (+    54us) raft_consensus.cc:1685] finished
0709 12:10:50.299245 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:50.299412 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26197,"prepare.queue_time_us":302728,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":1476,"raft.run_cpu_time_us":361,"raft.run_wall_time_us":17509,"spinlock_wait_cycles":626297088,"thread_start_us":274879,"threads_started":2,"wal-append.queue_time_us":172548}
I20260709 12:10:50.273430 15229 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:50.316021 12601 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:50.273465 15442 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:50.323920 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:50.324637 12603 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:50.327399 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:50.329423 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:50.330518 12599 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:50.332549 12604 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
W20260709 12:10:50.334067 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 256398us
I20260709 12:10:50.322412 15510 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:50.322566 15567 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:50.336144 12572 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:50.341348 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:50.346246 12494 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:50.350786 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:50.350988 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:50.351310 12643 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:50.351564 12679 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:50.361804 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:50.359091 15508 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:50.363312 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:50.364874 12606 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:10:50.350741 15429 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:50.316576 15462 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:50.367287 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1751) took 2220 ms. Trace:
I20260709 12:10:50.374923 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:50.375543 12531 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:50.373739 12522 rpcz_store.cc:276] 0709 12:10:48.147018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:48.148127 (+  1109us) service_pool.cc:224] Handling call
0709 12:10:48.165777 (+ 17650us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:49.334822 (+1169045us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:50.367005 (+1032183us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b54003ca5d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:50.367203 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2607606016}
I20260709 12:10:50.386273 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:50.386715 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:50.369839 12603 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:10:50.388760 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:50.389432 12603 raft_consensus.cc:2410] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:10:50.390688 12603 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:10:50.393409 12525 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:50.397015 15229 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:50.369167 15429 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:50.398589 15429 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.2, Last appended by leader: 2, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:50.400112 15324 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:50.364775 15517 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:50.402797 12599 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:10:50.400961 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:50.404156 12522 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 8: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:50.364491 15508 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:50.362195 15578 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:50.406214 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1750) took 2402 ms. Trace:
W20260709 12:10:50.404430 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 62785.2us
I20260709 12:10:50.409060 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:50.411000 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 78866.6us
I20260709 12:10:50.410177 15508 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:50.420341 15422 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:50.413543 12494 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:50.427645 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:50.427676 12572 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:50.426535 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:50.430548 12522 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:50.432723 12523 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:50.435011 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:50.435663 12604 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:50.406638 12527 rpcz_store.cc:276] 0709 12:10:48.003632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:48.003777 (+   145us) service_pool.cc:224] Handling call
0709 12:10:48.050127 (+ 46350us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:48.063603 (+ 13476us) raft_consensus.cc:1534] Early marking committed up to index 2
0709 12:10:48.094078 (+ 30475us) log.cc:844] Serialized 14 byte log entry
0709 12:10:48.724523 (+630445us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:48.779584 (+ 55061us) log.cc:844] Serialized 52 byte log entry
0709 12:10:48.781250 (+  1666us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:48.781567 (+   317us) log.cc:844] Serialized 14 byte log entry
0709 12:10:48.781787 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:48.781885 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:50.370999 (+1589114us) spinlock_profiling.cc:243] Waited 4.26 ms on lock 0x7b180042bec0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:10:50.405962 (+ 34963us) raft_consensus.cc:1685] finished
0709 12:10:50.405986 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:50.406154 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18098,"prepare.queue_time_us":17429,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":88,"raft.queue_time_us":152360,"raft.run_cpu_time_us":582,"raft.run_wall_time_us":84109,"spinlock_wait_cycles":9801600,"thread_start_us":629780,"threads_started":2,"wal-append.queue_time_us":512246}
I20260709 12:10:50.437196 12529 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:50.437543 12494 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:50.439419 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:50.439357 12494 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:50.440938 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:50.443933 15199 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:10:50.446656 15576 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:10:50.446765 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:50.447814 15538 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:10:50.462563 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:50.463833 13334 raft_consensus.cc:2410] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 12:10:50.464479 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:50.469054 12529 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260709 12:10:50.456845 12531 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:50.476560 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:50.450058 15580 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
W20260709 12:10:50.479878 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:50.444479 15501 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:50.444938 15517 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:50.449625 15241 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:50.428038 15324 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260709 12:10:50.485004 13332 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
W20260709 12:10:50.491636 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 79871.6us
W20260709 12:10:50.492906 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 99634.5us
I20260709 12:10:50.450881 15575 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
W20260709 12:10:50.504138 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 82440.7us
I20260709 12:10:50.481688 15422 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:50.509169 12603 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: [5.2-5.2]
I20260709 12:10:50.509421 12599 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:50.486795 15501 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:50.493028 15461 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:50.518926 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:50.527158 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:50.497747 15241 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:50.515367 15575 consensus_queue.cc:1243] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 6.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:10:50.520082 15501 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:50.535935 15530 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.793s	user 0.000s	sys 0.006s
W20260709 12:10:50.556560 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 60808.5us
I20260709 12:10:50.450160 15567 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:50.526640 15422 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:50.517632 15324 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
W20260709 12:10:50.570192 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 77324.7us
I20260709 12:10:50.532253 15461 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:50.570417 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 51558.7us
I20260709 12:10:50.528738 15500 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:10:50.557422 15199 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:50.562316 15575 raft_consensus.cc:3055] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:10:50.619776 12599 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:10:50.620167 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 53714.7us
I20260709 12:10:50.551492 15357 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
W20260709 12:10:50.629433 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 103208us
I20260709 12:10:50.569931 15462 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:50.632467 12572 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:50.644645 15556 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.651s	user 0.003s	sys 0.000s
I20260709 12:10:50.646318 15510 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:50.647504 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:50.650354 13302 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:50.651739 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:50.652122 13334 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 6. (term mismatch)
I20260709 12:10:50.652519 12679 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:50.652659 13334 pending_rounds.cc:85] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 5
I20260709 12:10:50.654316 13334 pending_rounds.cc:107] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 5.6
I20260709 12:10:50.654640 13334 raft_consensus.cc:2889] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:50.658595 15241 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:50.660012 15591 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:50.661001 15357 consensus_queue.cc:1243] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:10:50.661588 15357 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:50.661903 15357 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:50.660356 15442 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:10:50.662437 15357 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:50.665076 12676 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:50.621655 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:50.587896 15587 raft_consensus.cc:3055] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:50.564651 15377 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.067s
I20260709 12:10:50.673013 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:50.689205 12672 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:10:50.677582 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 55990.8us
I20260709 12:10:50.679306 12571 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:50.683306 12679 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:10:50.692790 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:50.622334 15575 raft_consensus.cc:740] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:50.684548 12522 raft_consensus.cc:2410] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:10:50.696658 15422 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:50.696712 15511 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:50.697785 15510 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:50.569409 15517 consensus_queue.cc:1059] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.102s
I20260709 12:10:50.721007 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:50.686215 15587 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:50.872004 15587 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:50.873163 15587 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:50.875481 15429 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:50.704492 13337 raft_consensus.cc:3055] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:50.883276 13337 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:50.708920 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 204) took 1736 ms. Trace:
W20260709 12:10:50.884732 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:50.884294 13337 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:50.885599 13337 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:50.883858 12597 rpcz_store.cc:276] 0709 12:10:48.972874 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:48.973280 (+   406us) service_pool.cc:224] Handling call
0709 12:10:48.973707 (+   427us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:49.841757 (+868050us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:50.708590 (+866833us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b54002e54d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:50.708832 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1994231680}
I20260709 12:10:50.856532 12679 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:50.788781 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:50.891400 12605 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:50.623342 15584 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.098s
I20260709 12:10:50.796396 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:50.894829 12599 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:50.896067 12523 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 6. Current term is 7. Ops: []
W20260709 12:10:50.797931 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 170190us
W20260709 12:10:50.798601 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 269785us
W20260709 12:10:50.804129 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 105607us
I20260709 12:10:50.705737 15511 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:50.716452 15510 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:10:50.901384 12606 consensus_meta.cc:287] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.536s	user 0.005s	sys 0.005s
I20260709 12:10:50.855985 15500 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:50.857787 12672 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:10:50.859817 15517 raft_consensus.cc:3055] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:50.959643 15500 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:50.960106 15517 raft_consensus.cc:740] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:50.960139 12605 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:10:50.961237 15517 consensus_queue.cc:260] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:50.962620 15517 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:50.861559 15461 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:50.964180 15357 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:50.864415 12680 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:10:50.968044 12680 raft_consensus.cc:606] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Received request to transfer leadership to TS a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:50.968490 12680 raft_consensus.cc:614] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:50.931939 15601 raft_consensus.cc:2955] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index 2 to 5, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:10:50.972401 12680 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:10:50.973093 12680 raft_consensus.cc:606] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Received request to transfer leadership to TS a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:50.973455 12680 raft_consensus.cc:614] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:50.689226 15430 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:50.637459 15480 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:50.974794 15480 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:50.977308 15480 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:50.897634 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:50.789839 15462 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.106s
I20260709 12:10:50.901232 15511 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:50.919023 12676 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:50.901973 12606 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:50.920123 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:50.913693 15510 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:10:50.924527 15537 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.830s	user 0.007s	sys 0.005s
I20260709 12:10:50.927896 12604 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:10:50.930519 12569 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:10:50.939090 15508 consensus_meta.cc:287] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.519s	user 0.007s	sys 0.004s
W20260709 12:10:50.939502 15582 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.368s	user 0.002s	sys 0.000s
I20260709 12:10:50.942485 15575 consensus_queue.cc:260] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:50.948357 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:50.949128 15411 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:10:50.951318 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54016.7us
I20260709 12:10:50.980906 13337 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:10:50.981961 13304 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.523s (SENT) [suppressed 34 similar messages]
W20260709 12:10:50.996544 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 66154.5us
I20260709 12:10:50.997428 12496 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:50.996068 12646 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:51.002664 12496 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:50.995815 15328 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:50.981246 15357 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:50.981433 15500 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:51.006618 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 86642us
W20260709 12:10:51.007763 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:51.030303 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:51.031044 12672 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
W20260709 12:10:50.999557 15519 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 7 current term: 7 is higher.
W20260709 12:10:51.008926 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 111407us
I20260709 12:10:50.986778 15549 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:50.993009 15411 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:51.056627 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:51.006086 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:51.066133 12572 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:51.001664 15461 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:50.992419 15508 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:51.004356 15587 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:50.993419 15537 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:51.087993 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:51.045320 15199 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:51.093868 12524 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:51.024104 15430 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:51.099921 12672 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:51.077210 15461 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:51.121373 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:51.097837 15587 raft_consensus.cc:697] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:51.079005 15357 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:51.152103 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:51.160789 15357 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:51.095088 15517 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:10:51.154913 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:51.129014 15501 consensus_meta.cc:287] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.566s	user 0.010s	sys 0.004s
I20260709 12:10:51.106395 15199 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:51.120504 15461 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:51.121150 12676 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:10:51.164819 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:51.165149 12494 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:51.167479 15567 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:51.111351 12680 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:51.106801 15480 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.050s
I20260709 12:10:51.174677 12674 raft_consensus.cc:2410] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:10:51.177886 12678 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:10:51.106998 15430 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:51.201967 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:51.204540 12496 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:51.211678 12680 raft_consensus.cc:1217] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Deduplicated request from leader. Original: 1.4->[3.5-5.6]   Dedup: 5.6->[]
I20260709 12:10:51.213759 12680 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 5. Current term is 6. Ops: [3.5-5.6]
I20260709 12:10:51.221640 12679 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:10:51.209110 15241 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:51.226513 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:51.227684 12599 raft_consensus.cc:2410] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:10:51.228080 12605 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:10:51.230512 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:51.233232 15606 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:10:51.236197 15602 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
W20260709 12:10:51.236901 12571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.033s	sys 0.043s
W20260709 12:10:51.237253 12571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.536s	user 0.033s	sys 0.043s
I20260709 12:10:51.237311 15578 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:51.237563 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 541882us
I20260709 12:10:51.237980 15578 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:51.240937 15578 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:51.140815 15411 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:51.181586 15517 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:51.245213 15517 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:51.245784 15357 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:51.246977 15199 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:51.207015 15480 consensus_queue.cc:1243] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:10:51.251360 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:51.253746 12672 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260709 12:10:51.254035 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 256501us
I20260709 12:10:51.258522 12680 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
W20260709 12:10:51.259963 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 278011us
W20260709 12:10:51.266362 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 75858.6us
I20260709 12:10:51.274569 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:51.171509 15501 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:51.272995 12678 raft_consensus.cc:2955] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:10:51.338907 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:51.339824 12674 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:10:51.342437 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:51.344805 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:10:51.347185 15588 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.726s	user 0.002s	sys 0.001s
I20260709 12:10:51.349486 15587 consensus_queue.cc:237] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:51.259987 15561 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:10:51.342528 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:51.255164 15241 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:51.279894 15522 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.589s	user 0.010s	sys 0.012s
I20260709 12:10:51.354161 15453 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:51.378306 12677 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:51.357376 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:51.373245 15522 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.011s	sys 0.012s
I20260709 12:10:51.361022 15604 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:51.419132 15377 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:51.424475 12679 raft_consensus.cc:606] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Received request to transfer leadership to TS a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:51.446946 12679 raft_consensus.cc:614] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:51.428742 15210 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:51.434654 15241 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:51.439487 15579 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:51.381677 15537 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:51.381883 15596 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:10:51.486868 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:51.484416 15462 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:10:51.489917 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:51.398597 15453 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:51.492300 12676 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:10:51.490075 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:51.494486 12671 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:51.493906 15453 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:51.504912 12643 leader_election.cc:336] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.519s (SENT)
I20260709 12:10:51.505537 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:51.507473 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:51.508402 13331 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 6: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:51.508756 15328 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:51.510911 15501 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:51.511935 15573 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:51.513763 12643 leader_election.cc:336] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.631s (SENT)
I20260709 12:10:51.514276 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:51.514155 15596 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:51.515116 15573 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:10:51.516542 12643 leader_election.cc:336] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.913s (SENT)
I20260709 12:10:51.516765 15596 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:51.516981 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:51.488129 15465 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:51.518476 15596 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:51.462069 15210 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:51.521318 15210 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:51.522397 15210 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:51.528369 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:51.528798 12671 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:10:51.529153 12672 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:51.501972 15461 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:51.535392 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 289988us
W20260709 12:10:51.557793 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:51.534641 12496 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:51.576900 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:51.577895 12676 raft_consensus.cc:2410] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:10:51.578317 12672 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:51.441526 15604 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:51.581315 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:51.440696 15229 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.091s
I20260709 12:10:51.578907 12598 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
W20260709 12:10:51.603060 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 346784us
I20260709 12:10:51.613119 12531 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:51.614074 12524 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:10:51.600713 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:51.630002 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:51.601164 15604 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:51.584117 15579 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:51.667924 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:51.669903 12604 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:51.643988 15606 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:51.645274 15517 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:51.615819 12597 raft_consensus.cc:2955] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:10:51.598534 15522 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:51.667924 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:51.686920 12524 raft_consensus.cc:2410] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 7 because replica has last-logged OpId of term: 5 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:51.672356 12529 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:51.561909 15537 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 12:10:51.518500 15465 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:51.677323 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:51.681572 12600 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:51.691604 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:51.691931 12530 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:10:51.692293 12531 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:10:51.696698 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:51.701176 12601 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:51.702688 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:51.703511 12600 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:51.707862 15606 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:51.715616 15453 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:51.712231 15199 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:51.706159 15592 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:10:51.718771 15508 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:51.688784 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:51.742533 13305 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:10:51.738034 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 691431us
W20260709 12:10:51.739006 15510 consensus_meta.cc:287] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.743s	user 0.013s	sys 0.000s
I20260709 12:10:51.772465 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:51.749478 12604 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:51.641029 15426 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.089s
I20260709 12:10:51.689226 15579 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:51.638613 15612 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.100s
I20260709 12:10:51.766170 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:51.776950 12605 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 6
I20260709 12:10:51.778573 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:51.743741 15480 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.044s
I20260709 12:10:51.787727 15596 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:51.788147 15573 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:10:51.788579 15621 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:51.788738 15596 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:51.788914 15573 raft_consensus.cc:3055] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:51.789204 15596 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:51.789338 15573 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:51.789745 15517 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:51.790210 15573 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:51.790392 15596 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:51.791692 15596 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:51.792605 12531 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:10:51.791692 15573 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:51.734854 15590 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:10:51.831698 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:51.832791 12530 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 7: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:51.760411 15453 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:51.698338 15522 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:51.852593 15453 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:51.854562 15522 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:51.856114 15430 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:51.857148 12679 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "816b6f34d124434882148e507cf5b612"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:10:51.857779 12679 raft_consensus.cc:606] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:51.860960 12679 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:10:51.861583 12679 raft_consensus.cc:606] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:51.861964 12679 raft_consensus.cc:614] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:10:51.862527 15579 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:51.863300 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:10:51.834018 15578 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
W20260709 12:10:51.844741 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 76567.3us
I20260709 12:10:51.865247 12572 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:51.758234 15357 consensus_queue.cc:1059] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
I20260709 12:10:51.851263 15591 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:51.852249 15465 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:51.865454 12527 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:51.866714 15357 raft_consensus.cc:3055] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:51.867071 15357 raft_consensus.cc:740] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:51.850195 15584 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:51.871035 15480 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:51.871483 15480 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:51.781963 15510 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:51.872406 15480 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:51.873600 15510 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:51.875075 15357 consensus_queue.cc:260] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:51.876456 15607 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.198s	user 0.003s	sys 0.000s
I20260709 12:10:51.876763 15519 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:51.782249 15442 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:51.878173 15517 raft_consensus.cc:3055] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:51.879074 15461 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:51.880225 15511 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.888s	user 0.011s	sys 0.003s
I20260709 12:10:51.873663 15480 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:51.764505 15199 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:51.842582 15598 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:10:51.803915 15538 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:51.821476 12605 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:51.876987 15357 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:51.781781 15595 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:51.900938 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:51.908660 12531 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:51.910648 13305 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:10:51.913499 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 48330.9us
I20260709 12:10:51.899941 15522 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:51.892279 15517 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:51.890164 15442 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:51.921079 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:51.916728 15613 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:51.886987 15621 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:51.931182 15442 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:10:51.941087 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 342099us
I20260709 12:10:51.951977 15622 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:51.907346 15511 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:51.964696 15511 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:51.967262 15511 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:51.967878 15511 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:51.970422 15511 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:51.891793 15519 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:51.922559 15624 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:51.915237 15538 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:51.991712 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 75814us
I20260709 12:10:51.994316 12572 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:51.998353 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:51.929239 15522 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:51.914666 15595 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:52.009315 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:52.010109 12672 raft_consensus.cc:2393] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 7: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:52.012625 12604 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:52.013962 12605 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260709 12:10:51.983702 15624 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:52.014873 15377 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:52.026451 15590 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:52.027421 15199 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:52.027554 12572 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:52.028519 15575 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:52.029229 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:52.029881 15519 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:52.031877 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:52.036796 15621 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:52.042569 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:52.050834 12679 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:52.052263 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:52.053115 12604 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:51.976415 15604 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:52.037631 15442 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:52.060693 12605 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 8. Current term is 9. Ops: []
W20260709 12:10:52.055203 12643 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.019s	sys 0.015s
W20260709 12:10:52.062072 12643 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.542s	user 0.020s	sys 0.015s
W20260709 12:10:52.062541 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 542710us
I20260709 12:10:52.052315 12527 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:10:52.046909 15624 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:52.070415 15510 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:52.072492 12572 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:52.015519 15210 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:52.075836 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.076767 12604 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:10:52.083555 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:52.084352 12677 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.512s	user 0.011s	sys 0.008s
I20260709 12:10:52.084528 12605 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
W20260709 12:10:52.084872 12677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.012s	sys 0.008s
I20260709 12:10:52.097730 12679 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
W20260709 12:10:52.102572 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:52.103545 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:52.103499 12600 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:10:52.102202 12496 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:52.053959 15377 raft_consensus.cc:697] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:52.110001 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.111159 12600 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:52.087136 15603 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:52.106377 15622 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:10:52.129781 15614 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.394s	user 0.002s	sys 0.000s
W20260709 12:10:52.130621 12643 leader_election.cc:336] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.032s (SENT)
I20260709 12:10:52.102226 15621 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260709 12:10:52.144330 15604 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:52.145860 15522 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:52.146538 15522 consensus_queue.cc:1059] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:10:52.117674 15601 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:52.148110 15601 raft_consensus.cc:697] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:52.149246 15442 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:52.150838 12643 leader_election.cc:336] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.149s (SENT)
I20260709 12:10:52.151081 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:52.151764 12529 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:10:52.151930 12643 leader_election.cc:336] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.164s (SENT)
I20260709 12:10:52.152376 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:52.153609 12643 leader_election.cc:336] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.372s (SENT)
I20260709 12:10:52.154634 12572 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:52.155669 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:52.054106 15590 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:52.194433 12495 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.000s	sys 0.066s
W20260709 12:10:52.195981 12495 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.000s	sys 0.067s
I20260709 12:10:52.113230 15511 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:10:52.205019 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:52.204808 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.099089 15210 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:52.205516 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 411510us
W20260709 12:10:52.218894 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:52.220360 12494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.008s	sys 0.024s
W20260709 12:10:52.220760 12494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.008s	sys 0.024s
W20260709 12:10:52.234324 12674 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.015s	sys 0.011s
W20260709 12:10:52.237248 12674 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.015s	sys 0.012s
I20260709 12:10:52.196270 15626 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:10:52.238284 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 84697.2us
W20260709 12:10:52.244526 12643 connection.cc:256] RPC call timeout handler was delayed by -1.8927371190000031s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16670000000000001s for a response
W20260709 12:10:52.245126 12643 connection.cc:256] RPC call timeout handler was delayed by -1.8864371189999929s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1674s for a response
W20260709 12:10:52.245553 12643 connection.cc:256] RPC call timeout handler was delayed by -1.8864371189999929s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1674s for a response
W20260709 12:10:52.246243 12643 connection.cc:256] RPC call timeout handler was delayed by -1.8864371189999929s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1674s for a response
W20260709 12:10:52.246749 12643 connection.cc:256] RPC call timeout handler was delayed by -1.8756371189999754s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1686s for a response
W20260709 12:10:52.247187 12643 connection.cc:256] RPC call timeout handler was delayed by -1.7820371189998241s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17900000000000002s for a response
W20260709 12:10:52.247557 12643 connection.cc:256] RPC call timeout handler was delayed by -1.7820371189998241s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17900000000000002s for a response
W20260709 12:10:52.248018 12643 connection.cc:256] RPC call timeout handler was delayed by -1.7253371189999598s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18530000000000002s for a response
W20260709 12:10:52.248378 12643 connection.cc:256] RPC call timeout handler was delayed by -1.6425371189998259s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1945s for a response
W20260709 12:10:52.248831 12643 connection.cc:256] RPC call timeout handler was delayed by -1.6137371190000067s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19770000000000001s for a response
W20260709 12:10:52.249253 12643 connection.cc:256] RPC call timeout handler was delayed by -1.5840371189999587s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20099999999999998s for a response
W20260709 12:10:52.249653 12643 connection.cc:256] RPC call timeout handler was delayed by -1.4373371189999489s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21730000000000002s for a response
W20260709 12:10:52.245484 12678 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.799s	user 0.008s	sys 0.026s
I20260709 12:10:52.248086 12529 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
W20260709 12:10:52.250082 12643 connection.cc:256] RPC call timeout handler was delayed by -1.2942371189999449s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2332s for a response
I20260709 12:10:52.206740 15603 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:52.202224 15501 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:52.202467 15604 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:52.149483 15601 consensus_queue.cc:237] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:52.255702 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:52.204797 15621 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:52.258975 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 104418us
I20260709 12:10:52.265149 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:52.266934 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:52.267656 12676 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:52.271479 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:52.273736 12675 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:52.278838 15621 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:52.251775 15587 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:52.251035 12643 connection.cc:256] RPC call timeout handler was delayed by -1.1223371189998943s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2523s for a response
I20260709 12:10:52.281396 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:52.297125 12643 connection.cc:256] RPC call timeout handler was delayed by -1.0332371189999776s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2622s for a response
W20260709 12:10:52.284339 12569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.024s	sys 0.012s
I20260709 12:10:52.285121 12675 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
W20260709 12:10:52.298033 12569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.692s	user 0.024s	sys 0.012s
W20260709 12:10:52.250456 12678 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.804s	user 0.008s	sys 0.026s
I20260709 12:10:52.301497 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1967) took 1031 ms. Trace:
I20260709 12:10:52.302057 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.303072 12604 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 7: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:52.304683 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.305510 12600 raft_consensus.cc:2376] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7. Re-sending same reply.
I20260709 12:10:52.302261 12678 rpcz_store.cc:276] 0709 12:10:51.269939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:51.272129 (+  2190us) service_pool.cc:224] Handling call
0709 12:10:51.272517 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:51.272824 (+   307us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:10:51.446048 (+173224us) log.cc:844] Serialized 14 byte log entry
0709 12:10:52.301001 (+854953us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:52.301096 (+    95us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:52.301173 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:52.301267 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:52.301419 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":852736,"threads_started":1}
I20260709 12:10:52.306262 12643 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:52.314251 12529 pending_rounds.cc:85] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 5
I20260709 12:10:52.314356 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.314793 12529 pending_rounds.cc:107] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260709 12:10:52.314837 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:52.315286 12529 raft_consensus.cc:2889] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:52.325788 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:52.326947 12673 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:52.316794 12494 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:52.329502 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.305697 15587 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:52.342655 15633 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:10:52.332427 15624 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:52.335050 15210 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:52.262456 15501 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:52.267149 15604 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:52.268931 15377 consensus_queue.cc:237] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:52.340951 12678 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:10:52.317399 15357 raft_consensus.cc:3055] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:10:52.344512 12671 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:52.346344 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:52.353983 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:52.305645 15594 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:52.409296 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 55381.4us
I20260709 12:10:52.309842 15538 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:52.343776 15626 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:52.294178 15622 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.112s
I20260709 12:10:52.300246 15510 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:10:52.435801 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:52.436650 12604 raft_consensus.cc:2376] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6. Re-sending same reply.
I20260709 12:10:52.343969 15575 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:52.419478 12643 leader_election.cc:336] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.674s (SENT)
I20260709 12:10:52.486541 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:52.495051 12604 raft_consensus.cc:2368] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7.
I20260709 12:10:52.494179 12643 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:52.416155 12597 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.010s	sys 0.032s
W20260709 12:10:52.496158 12597 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.010s	sys 0.032s
I20260709 12:10:52.467839 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:52.417213 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 305726us
I20260709 12:10:52.498313 13302 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:52.499660 13302 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:52.501243 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:52.465230 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.502331 12600 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
W20260709 12:10:52.487463 15606 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.770s	user 0.007s	sys 0.031s
I20260709 12:10:52.489733 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.357051 15357 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:52.415769 15641 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:52.530546 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 213808us
I20260709 12:10:52.526717 12605 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:52.311347 15517 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:52.529479 12673 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:10:52.415719 15626 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:52.484913 15210 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:52.625757 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:52.511906 15606 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.795s	user 0.007s	sys 0.032s
I20260709 12:10:52.434386 15324 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:10:52.630234 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 98030.3us
I20260709 12:10:52.501089 15612 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:52.543074 15298 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.093s
I20260709 12:10:52.505306 15519 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:10:52.677830 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:52.679335 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 246614us
I20260709 12:10:52.544102 15199 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:52.686038 12522 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:52.609308 15641 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:52.681962 15480 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.772s	user 0.009s	sys 0.004s
I20260709 12:10:52.687304 15587 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:52.687150 15357 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:52.688127 15612 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:52.635232 15324 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:52.431471 15594 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:52.690338 15649 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:52.691481 15501 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:52.690665 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 271185us
W20260709 12:10:52.692022 15632 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.382s	user 0.004s	sys 0.000s
I20260709 12:10:52.692500 15517 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:52.692046 15633 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:52.695103 15650 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:52.640492 15210 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:52.623204 15626 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:52.696973 12604 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
W20260709 12:10:52.697721 15627 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.388s	user 0.000s	sys 0.001s
I20260709 12:10:52.698088 15621 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:52.415138 15573 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:52.706446 15629 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.396s	user 0.002s	sys 0.000s
I20260709 12:10:52.708524 15442 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:52.709422 15604 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:52.631139 15638 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:52.711000 15590 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:52.711949 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:52.713231 15624 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:52.722163 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.775151 12600 raft_consensus.cc:2410] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 9 because replica has last-logged OpId of term: 8 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 2.
I20260709 12:10:52.723341 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:52.775719 12531 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:10:52.725031 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:52.725584 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 89464.6us
I20260709 12:10:52.725826 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1763) took 1114 ms. Trace:
I20260709 12:10:52.777339 12523 rpcz_store.cc:276] 0709 12:10:51.611678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:51.612433 (+   755us) service_pool.cc:224] Handling call
0709 12:10:51.612725 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:51.612922 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:10:52.158937 (+546015us) log.cc:844] Serialized 14 byte log entry
0709 12:10:52.622723 (+463786us) log.cc:844] Serialized 14 byte log entry
0709 12:10:52.725148 (+102425us) log.cc:844] Serialized 14 byte log entry
0709 12:10:52.725398 (+   250us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:52.725464 (+    66us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:52.725518 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:52.725615 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:52.725766 (+   151us) inbound_call.cc:177] Queueing success response
0709 12:10:52.734910 (+  9144us) spinlock_profiling.cc:243] Waited 5.91 ms on lock 0x7b1800336ee0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
Metrics: {"mutex_wait_us":238792,"prepare.queue_time_us":521296,"prepare.run_cpu_time_us":258,"prepare.run_wall_time_us":261,"raft.queue_time_us":337307,"raft.run_cpu_time_us":264,"raft.run_wall_time_us":10471,"spinlock_wait_cycles":23349760,"thread_start_us":709002,"threads_started":2,"wal-append.queue_time_us":540010}
I20260709 12:10:52.726094 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.728389 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.729348 15641 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:52.731815 15199 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:52.732409 15595 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:52.769323 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:52.740823 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 239623us
I20260709 12:10:52.736748 15411 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:52.754184 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.746383 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.744715 12605 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:10:52.761556 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.742636 15324 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:52.741786 15357 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:52.757544 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:10:52.758140 12427 catalog_manager.cc:5697] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 2 to 5, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 5 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:52.780630 12598 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:52.753890 15650 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:52.735541 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:52.763733 15328 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:52.763358 15649 raft_consensus.cc:697] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:10:52.761665 12643 leader_election.cc:336] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.853s (SENT)
I20260709 12:10:52.805308 12643 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:52.790933 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.794004 12531 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:10:52.787346 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 1978) took 1261 ms. Trace:
I20260709 12:10:52.775913 15621 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:52.801164 15650 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:52.809275 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:52.808355 12677 rpcz_store.cc:276] 0709 12:10:51.526194 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:51.527277 (+  1083us) service_pool.cc:224] Handling call
0709 12:10:51.567393 (+ 40116us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:51.567931 (+   538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:51.567977 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:51.568622 (+   645us) log.cc:844] Serialized 26 byte log entry
0709 12:10:52.085413 (+516791us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:52.085475 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:52.085571 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:52.411508 (+325937us) spinlock_profiling.cc:243] Waited 6.89 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:52.782991 (+371483us) raft_consensus.cc:1685] finished
0709 12:10:52.783020 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:52.787261 (+  4241us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1304,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":14398,"spinlock_wait_cycles":15846784,"thread_start_us":516362,"threads_started":1,"wal-append.queue_time_us":709709}
I20260709 12:10:52.809850 15595 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:52.811393 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:52.784344 15199 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:52.814267 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:52.818801 13335 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:52.820472 15660 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:52.821954 13337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 469) took 1609 ms. Trace:
W20260709 12:10:52.823534 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 61905.1us
W20260709 12:10:52.829057 12643 leader_election.cc:336] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.977s (SENT)
W20260709 12:10:52.822467 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:52.829694 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:52.828063 13337 rpcz_store.cc:276] 0709 12:10:51.212257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:51.212422 (+   165us) service_pool.cc:224] Handling call
0709 12:10:51.212721 (+   299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:51.306974 (+ 94253us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:51.307007 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:51.360020 (+ 53013us) log.cc:844] Serialized 26 byte log entry
0709 12:10:51.893111 (+533091us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:10:51.893171 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:51.893270 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:52.821731 (+928461us) raft_consensus.cc:1685] finished
0709 12:10:52.821751 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:52.821888 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93810,"raft.queue_time_us":95276,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":15889,"spinlock_wait_cycles":1564032,"thread_start_us":478948,"threads_started":1,"wal-append.queue_time_us":757334}
I20260709 12:10:52.834028 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.839023 15199 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:52.827535 15638 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:10:52.843606 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.845851 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.846853 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.848481 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:52.848647 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:52.849268 12602 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:52.830996 15660 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:52.852615 15660 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:52.855523 13334 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:10:52.831772 15643 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:52.858489 12569 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:52.860222 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.806923 15328 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:52.860509 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:52.867790 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:52.835609 15312 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:52.866616 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:52.870407 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:52.864950 12598 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:10:52.874451 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:52.875622 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:52.876060 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:52.876881 12676 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 6: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:10:52.870731 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 404577us
I20260709 12:10:52.830034 15621 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:52.878950 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:52.860898 15655 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:10:52.847090 15659 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:52.882164 12679 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:10:52.829978 15595 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:52.887625 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:52.889838 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:52.866888 15622 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:52.889313 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:52.892201 12674 raft_consensus.cc:2410] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:10:52.889174 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.895547 12598 raft_consensus.cc:2410] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 6 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:10:52.897881 12602 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:10:52.822736 15633 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:52.899005 15442 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:52.903329 15661 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:52.903810 15661 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:52.904791 15661 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:52.899732 13302 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:52.907648 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:52.910811 15442 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:52.898936 15638 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:52.887225 15624 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:52.883582 15649 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:52.893384 15659 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:52.914348 13304 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:52.915952 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:52.919229 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:52.964241 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49981.4us
I20260709 12:10:52.918107 15442 raft_consensus.cc:697] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 8 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:52.964931 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:52.924021 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.965487 15442 consensus_queue.cc:237] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:52.927165 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:52.966482 15522 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:52.967147 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:52.967335 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.968355 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:52.968583 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.924664 12598 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 8. Ops: [6.3-6.3]
I20260709 12:10:52.927793 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:52.969259 15591 consensus_queue.cc:1059] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:10:52.868564 15643 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:52.928135 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:52.970639 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:52.971320 12600 raft_consensus.cc:2376] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8. Re-sending same reply.
I20260709 12:10:52.972445 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.973512 12600 raft_consensus.cc:2410] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 7 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:10:52.965932 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:52.930816 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 153171us
I20260709 12:10:52.975445 13304 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:52.977042 12494 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:52.930948 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 195491us
I20260709 12:10:52.979133 12496 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:52.940225 12525 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:10:52.928756 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:52.980938 12604 raft_consensus.cc:2368] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8.
I20260709 12:10:52.925308 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:52.942957 12523 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:52.944656 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:52.945317 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:52.982734 12494 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:52.919528 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:52.970191 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.983884 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.984977 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.926192 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:52.985612 12598 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:10:52.986171 12606 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:52.941681 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:52.986974 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:52.987844 12531 raft_consensus.cc:2410] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 6 because replica has last-logged OpId of term: 5 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:10:52.944227 12524 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:52.965538 12529 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:10:52.991854 12528 raft_consensus.cc:2368] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6.
I20260709 12:10:52.906113 15661 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:52.995805 12598 raft_consensus.cc:2368] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5.
I20260709 12:10:52.999511 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:53.001914 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:53.002530 12603 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:10:53.002694 12599 raft_consensus.cc:2376] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7. Re-sending same reply.
I20260709 12:10:53.003626 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:53.003818 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:53.004705 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:53.004920 12599 raft_consensus.cc:2410] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:10:53.008301 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:53.009248 12605 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:10:52.916301 15622 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:53.012341 12600 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 7: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:10:53.013739 12600 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
W20260709 12:10:52.938247 15654 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 12:10:53.020326 12599 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:52.912062 15612 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:52.931819 15199 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:52.961148 15659 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:53.033908 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:53.033486 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:53.026084 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:53.044893 12604 raft_consensus.cc:2368] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8.
I20260709 12:10:53.045094 13337 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 6: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:53.037765 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:52.950213 15328 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:53.035344 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:52.933007 15624 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:52.997661 15660 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:10:53.050756 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:52.930967 15573 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:53.051636 12600 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:53.039897 13339 raft_consensus.cc:2955] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index 2 to 5, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:10:52.997102 15590 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:52.939954 15621 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:52.957768 15210 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:53.052502 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:53.053261 12606 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:10:53.023314 12605 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:53.073132 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:53.067939 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:53.022994 15229 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:53.076256 12598 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:53.042207 15592 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:53.017017 15608 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:53.050416 15193 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.066187 15624 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:53.050839 15199 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:53.079928 12605 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:10:53.034730 15643 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:53.005683 15501 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.069s
I20260709 12:10:53.063831 15612 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:53.086347 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:53.091789 12604 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:53.086380 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:53.078336 12525 raft_consensus.cc:2368] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6.
I20260709 12:10:53.089061 12606 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:10:53.093510 13330 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:53.069180 15511 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:53.096513 12600 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:52.934809 15638 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:53.039779 15639 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.047s
I20260709 12:10:53.103348 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:53.103834 13335 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:53.104275 12606 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:53.041100 15650 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:53.090461 15199 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:53.091002 15604 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.097836 15643 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.080703 15324 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:53.100538 15511 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:53.120393 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:53.122363 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:53.122802 15650 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:53.123436 13332 raft_consensus.cc:2368] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 6. Current term is 7.
I20260709 12:10:53.129405 15603 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:53.131990 15638 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:53.136190 15626 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:53.146889 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 51865.1us
I20260709 12:10:53.099006 15612 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:53.088550 15193 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:53.151348 12673 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:10:53.138926 15604 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:53.156222 12606 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
W20260709 12:10:53.151706 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 115167us
I20260709 12:10:53.156973 12597 raft_consensus.cc:2410] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:10:53.156601 12604 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:10:53.159791 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:53.179283 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:53.186020 15612 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:53.187824 15324 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:53.105064 15622 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:53.189850 15604 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:53.156307 15229 raft_consensus.cc:697] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:53.142737 15643 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:53.147895 15603 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:53.199126 15626 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:53.194963 15670 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.054s	user 0.000s	sys 0.001s
W20260709 12:10:53.201316 15665 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:10:53.204213 15643 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:53.205590 15575 raft_consensus.cc:3055] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:53.206254 15193 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:53.210633 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:53.211437 15511 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:53.211822 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:53.213258 12606 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:10:53.217518 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:53.218601 15603 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:53.221297 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:53.232873 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:53.233958 13337 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:53.241412 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:53.242414 12530 raft_consensus.cc:2368] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8.
I20260709 12:10:53.243265 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:53.243384 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:53.244063 12530 raft_consensus.cc:2410] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:10:53.246835 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:53.250228 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:53.206032 15575 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:53.256281 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:53.257736 12530 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:10:53.259407 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:53.223789 15511 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:53.286862 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:53.288581 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:53.290309 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:53.291764 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:53.296192 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 81442.7us
I20260709 12:10:53.297266 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:53.302042 12530 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:53.251482 15592 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:53.302177 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:53.301883 15422 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:53.303798 15461 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:10:53.307778 15640 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.995s	user 0.005s	sys 0.000s
I20260709 12:10:53.257388 15522 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:53.321514 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1945) took 1595 ms. Trace:
I20260709 12:10:53.321873 13331 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:10:53.322053 12601 rpcz_store.cc:276] 0709 12:10:51.725932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:51.726093 (+   161us) service_pool.cc:224] Handling call
0709 12:10:51.756109 (+ 30016us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:51.764962 (+  8853us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:51.764990 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:51.774416 (+  9426us) log.cc:844] Serialized 26 byte log entry
0709 12:10:52.215209 (+440793us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:52.215276 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:52.215449 (+   173us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:53.307177 (+1091728us) spinlock_profiling.cc:243] Waited 5.61 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:53.310393 (+  3216us) raft_consensus.cc:1685] finished
0709 12:10:53.310418 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:53.321395 (+ 10977us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33065,"raft.run_cpu_time_us":54,"raft.run_wall_time_us":54,"spinlock_wait_cycles":12909312,"thread_start_us":372154,"threads_started":1,"wal-append.queue_time_us":497195}
W20260709 12:10:53.328563 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:53.303095 15229 consensus_queue.cc:237] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:53.341197 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:53.341471 15411 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.010s
I20260709 12:10:53.332650 15594 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:53.332849 15575 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:53.348443 13338 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:10:53.327009 12494 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:53.349177 15592 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:53.325009 15673 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:10:53.374925 12603 raft_consensus.cc:2410] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260709 12:10:53.376303 12527 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:10:53.376844 12527 pending_rounds.cc:85] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 3
I20260709 12:10:53.377148 12527 pending_rounds.cc:107] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 6.4
I20260709 12:10:53.377446 12527 raft_consensus.cc:2889] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:53.381805 13305 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:53.313324 15604 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:53.331355 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:53.388367 12674 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:53.396028 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:53.348349 15575 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:53.397401 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 70480us
I20260709 12:10:53.403463 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:53.405359 12672 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:10:53.294318 15511 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:53.416575 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:53.416911 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:53.395231 15442 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:53.424824 12530 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 5 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:53.396591 15604 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:53.423199 12646 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:53.440335 15604 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:53.440585 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:53.441579 12680 raft_consensus.cc:2410] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:10:53.442130 15638 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:53.427068 15674 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:53.443650 15674 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:53.430591 15630 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.124s	user 0.000s	sys 0.005s
I20260709 12:10:53.422998 15643 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.009s
I20260709 12:10:53.408103 15193 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:53.446368 15193 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:53.447294 15193 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:53.448660 15595 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:53.431674 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1764) took 1819 ms. Trace:
I20260709 12:10:53.450281 12522 rpcz_store.cc:276] 0709 12:10:51.611994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:51.613075 (+  1081us) service_pool.cc:224] Handling call
0709 12:10:51.613346 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:52.718488 (+1105142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:53.431307 (+712819us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54003f7354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:53.431586 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2466434816}
I20260709 12:10:53.437235 15594 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.452104 15594 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:53.438786 12496 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:53.454674 15633 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:10:53.455224 15241 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:53.455452 15511 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:53.455770 15241 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:53.439600 15626 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.456631 15442 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 6 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:53.456763 15626 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:53.446394 12524 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 12:10:53.459898 12527 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260709 12:10:53.459818 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:53.460597 12672 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:53.461074 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:53.428320 12530 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260709 12:10:53.445865 15674 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:53.456209 15241 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:53.463303 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:53.466610 12526 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:53.468611 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:53.469066 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:53.470590 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:53.474294 12598 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:10:53.473007 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:53.475126 12601 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:10:53.474653 12604 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:10:53.474709 12671 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:10:53.479140 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:53.480019 12678 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:53.481868 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:53.482252 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:53.482772 12676 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:53.475056 12524 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:10:53.484819 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:53.489123 12604 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:10:53.457778 15442 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:53.400902 15199 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:53.494376 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:53.498656 12677 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:10:53.497196 12676 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 7: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:53.506603 12674 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:10:53.509534 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:53.478853 15522 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.540679 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:53.543639 12674 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
W20260709 12:10:53.518703 15681 log.cc:927] Time spent T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:10:53.550045 12525 raft_consensus.cc:2368] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6.
I20260709 12:10:53.497593 15638 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:53.484469 15587 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 12:10:53.563941 12677 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:10:53.520596 15594 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:53.564647 15675 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:10:53.559563 15638 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:53.581302 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:53.582738 12674 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:53.550670 15522 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:53.604964 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:53.590243 15587 consensus_queue.cc:1243] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 8.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:10:53.591307 15638 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:10:53.635416 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:53.658347 12677 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:53.605909 15626 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
W20260709 12:10:53.659956 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:53.633488 15324 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:53.688216 15683 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
W20260709 12:10:53.691092 15678 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 12:10:53.691490 13338 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260709 12:10:53.693347 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:53.694219 15603 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:10:53.694226 15199 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:53.698422 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:53.703297 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:53.703502 12674 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260709 12:10:53.704447 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:53.705647 12672 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:10:53.693389 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:53.707506 12678 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:10:53.672053 15587 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:53.715507 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:53.739776 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:53.763643 12572 leader_election.cc:400] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6.
I20260709 12:10:53.768813 12569 leader_election.cc:400] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 6. Current term is 7.
I20260709 12:10:53.769332 12569 leader_election.cc:403] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:10:53.787844 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:53.788244 12496 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:53.788929 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:53.797147 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:53.797508 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 68119.8us
I20260709 12:10:53.797994 12674 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:53.720355 15650 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:53.684767 15324 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:53.727185 15613 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:53.786335 15661 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:53.712774 15603 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.114s
I20260709 12:10:53.719923 15649 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.116s
I20260709 12:10:53.766101 15511 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.074s
I20260709 12:10:53.825018 12572 leader_election.cc:400] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6.
W20260709 12:10:53.873302 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 48543.5us
I20260709 12:10:53.785435 15587 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:53.848418 15650 raft_consensus.cc:697] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:53.907120 15650 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:53.848141 15661 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:53.910403 15661 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:53.847133 15517 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 6. Current term is 7.
I20260709 12:10:53.876367 12572 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:53.878140 12674 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:53.884366 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1840) took 1094 ms. Trace:
I20260709 12:10:53.919400 12597 rpcz_store.cc:276] 0709 12:10:52.790491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:52.790671 (+   180us) service_pool.cc:224] Handling call
0709 12:10:53.884104 (+1093433us) spinlock_profiling.cc:243] Waited 350 ms on lock 0x7b54002f1554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:53.884291 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":805916416}
W20260709 12:10:53.885332 15688 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 12:10:53.886271 15684 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:10:53.889226 12599 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260709 12:10:53.921201 12599 pending_rounds.cc:85] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 2
I20260709 12:10:53.921509 12599 pending_rounds.cc:107] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260709 12:10:53.921635 12676 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:53.921761 12599 raft_consensus.cc:2889] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:53.922168 12676 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:53.849253 15604 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:53.923071 12676 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:53.924001 12676 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:53.924122 15604 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:53.933094 15587 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:53.934264 15522 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:53.936496 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1864) took 1002 ms. Trace:
I20260709 12:10:53.936960 12602 rpcz_store.cc:276] 0709 12:10:52.933983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:52.968074 (+ 34091us) service_pool.cc:224] Handling call
0709 12:10:53.936249 (+968175us) spinlock_profiling.cc:243] Waited 517 ms on lock 0x7b5400374e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:53.936421 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1190218496}
I20260709 12:10:53.937621 15324 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:53.860381 15633 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.079s
W20260709 12:10:53.947081 15624 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.851s	user 0.013s	sys 0.000s
I20260709 12:10:53.947626 15624 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:53.950490 13334 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:53.951804 15517 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:53.952363 15517 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:53.956761 15603 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:10:53.957013 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 80737.1us
I20260709 12:10:53.957855 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:53.959139 12572 leader_election.cc:400] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8.
W20260709 12:10:53.894595 15666 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.809s	user 0.008s	sys 0.000s
I20260709 12:10:53.869962 15674 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:53.902557 15573 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:53.929939 15575 consensus_meta.cc:287] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.514s	user 0.006s	sys 0.002s
I20260709 12:10:53.966677 15603 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:53.973672 12599 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:10:53.974143 15573 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:53.977664 13304 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:53.979040 13304 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:53.981407 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:53.985284 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:53.985886 12600 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:53.987743 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:53.992166 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:53.998946 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:53.999500 12600 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:53.999708 13339 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:10:54.010891 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:54.016904 12599 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:53.997814 15660 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:54.012212 15649 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:10:54.019789 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 60771us
I20260709 12:10:54.020545 12676 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:10:54.021023 12572 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:54.024021 12572 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:54.025475 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:53.975539 15668 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:53.966885 15324 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:54.014463 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:10:54.028472 12527 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.016s	sys 0.012s
W20260709 12:10:54.027171 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 231221us
W20260709 12:10:54.033949 12527 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.016s	sys 0.012s
I20260709 12:10:54.033979 13338 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:54.032076 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 273731us
I20260709 12:10:54.035946 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:54.037467 12496 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:54.011510 15522 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:10:54.028378 15594 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260709 12:10:53.975740 15674 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:54.026473 15612 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:54.043370 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:10:54.094002 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50731.1us
I20260709 12:10:54.047659 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:54.037094 15241 consensus_meta.cc:287] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.564s	user 0.009s	sys 0.005s
I20260709 12:10:54.025199 15649 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:54.086216 15517 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:54.039923 15324 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:54.040583 15633 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:54.063397 13333 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 473) took 1027 ms. Trace:
I20260709 12:10:54.042516 15511 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:54.064921 12496 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:54.036075 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 327909us
I20260709 12:10:54.047240 15603 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:10:54.108770 13333 rpcz_store.cc:276] 0709 12:10:53.035951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:53.038740 (+  2789us) service_pool.cc:224] Handling call
0709 12:10:53.039059 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:53.950517 (+911458us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:53.950574 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:53.950619 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:53.950661 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:54.063069 (+112408us) spinlock_profiling.cc:243] Waited 911 ms on lock 0x7b5400478454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:54.063131 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:54.063313 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2095276416}
I20260709 12:10:54.086279 15643 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:54.123474 12524 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:10:54.127775 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:54.132313 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 67508.5us
I20260709 12:10:54.134785 12496 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:54.098980 15649 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:10:54.136370 12528 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:10:54.136129 12496 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:54.135318 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:54.143400 13337 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:10:54.145478 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:54.146358 15639 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:54.146878 15639 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:54.147315 15639 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:54.149813 15357 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
W20260709 12:10:54.151101 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 103534us
I20260709 12:10:54.151996 12572 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:54.159051 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:54.160732 12527 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 7: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:10:54.168442 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:54.169893 13331 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:10:54.170497 13330 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:54.171816 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:54.111132 15511 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:54.178704 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:54.186249 12522 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.645s	user 0.002s	sys 0.014s
I20260709 12:10:54.190284 12606 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:10:54.192195 12600 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 7: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:10:54.192796 12522 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.002s	sys 0.015s
I20260709 12:10:54.135975 15643 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:54.138303 15592 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:54.204312 12601 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:10:54.103379 15241 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:54.108510 15517 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.075s
W20260709 12:10:54.204317 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:54.185846 15694 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:10:54.171445 15229 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:54.226719 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:10:54.227892 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 214923us
I20260709 12:10:54.231066 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:54.232045 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:54.232585 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:54.242486 12643 leader_election.cc:336] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.702s (SENT)
I20260709 12:10:54.242944 12643 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:54.243651 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:54.243821 15693 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:10:54.245651 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:54.141031 15612 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 12:10:54.194389 15511 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:54.249238 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:54.250218 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:54.267122 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 124872us
I20260709 12:10:54.269093 12571 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:54.271626 12494 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:10:54.250547 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:54.150813 15633 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 12:10:54.156033 15587 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.056s
W20260709 12:10:54.252369 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 80633us
I20260709 12:10:54.197253 15643 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:54.254565 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:54.254966 12529 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:54.257865 15324 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:54.258633 15699 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:10:54.254675 15616 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:54.261132 15241 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:54.261920 15603 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260709 12:10:54.282239 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:54.301523 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 149635us
I20260709 12:10:54.322778 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:54.328927 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 57442.1us
W20260709 12:10:54.349536 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 80521us
W20260709 12:10:54.350250 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 65754.7us
I20260709 12:10:54.357995 12676 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:10:54.335882 15573 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:54.497073 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:54.498749 15517 consensus_queue.cc:1243] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 12:10:54.494482 15697 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
I20260709 12:10:54.354856 15608 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.147s
W20260709 12:10:54.504789 15710 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
W20260709 12:10:54.506296 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 155154us
W20260709 12:10:54.573179 13338 consensus_meta.cc:287] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.535s	user 0.014s	sys 0.000s
W20260709 12:10:54.507513 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 149159us
W20260709 12:10:54.510928 15704 log.cc:927] Time spent T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
W20260709 12:10:54.519066 15705 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
I20260709 12:10:54.353312 15703 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:54.532994 15612 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Stepping down as leader of term 5
W20260709 12:10:54.537065 15638 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.876s	user 0.012s	sys 0.001s
I20260709 12:10:54.353438 15717 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:54.553388 15699 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:54.553828 13334 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 471) took 1529 ms. Trace:
W20260709 12:10:54.499994 15695 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.403s	user 0.001s	sys 0.001s
I20260709 12:10:52.810856 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:54.557047 15718 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:54.557245 12427 catalog_manager.cc:5697] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 4 to 8. New cstate: current_term: 8 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:54.570070 12530 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:10:54.573702 13338 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:10:54.577580 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:54.580416 13334 rpcz_store.cc:276] 0709 12:10:53.024457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:53.024636 (+   179us) service_pool.cc:224] Handling call
0709 12:10:53.034508 (+  9872us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:53.950952 (+916444us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:54.553530 (+602578us) spinlock_profiling.cc:243] Waited 916 ms on lock 0x7b54004439d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:54.553757 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2105785088}
W20260709 12:10:54.582165 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:54.583050 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 280485us
I20260709 12:10:54.581199 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:54.594328 15717 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:54.595603 15673 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:54.597278 15608 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:10:54.597888 15608 raft_consensus.cc:697] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:54.599166 15608 consensus_queue.cc:237] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:54.599836 15579 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:54.600704 15579 consensus_queue.cc:1243] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 7.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:10:54.597793 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1773) took 1265 ms. Trace:
I20260709 12:10:54.602903 12674 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:10:54.604651 15661 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.694s	user 0.014s	sys 0.000s
I20260709 12:10:54.605103 15661 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:54.607461 15661 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:54.610265 15357 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:54.613083 15624 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:54.587770 15638 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:54.614821 15639 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:54.615404 15638 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:54.580470 15703 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:54.616156 15703 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:54.579905 15612 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:54.617056 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1723) took 1657 ms. Trace:
I20260709 12:10:54.617849 15612 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:54.619174 15612 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:54.626348 12672 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:54.602422 12522 rpcz_store.cc:276] 0709 12:10:53.332417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:53.451219 (+118802us) service_pool.cc:224] Handling call
0709 12:10:53.462357 (+ 11138us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:53.473689 (+ 11332us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:53.473720 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:53.501078 (+ 27358us) log.cc:844] Serialized 52 byte log entry
0709 12:10:54.193279 (+692201us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:54.223895 (+ 30616us) log.cc:844] Serialized 14 byte log entry
0709 12:10:54.283217 (+ 59322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:54.283331 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:54.530264 (+246933us) raft_consensus.cc:1685] finished
0709 12:10:54.530299 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:54.597623 (+ 67324us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b54003c0d50. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:54.597729 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":116012,"prepare.queue_time_us":29272,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":2997,"raft.queue_time_us":86913,"raft.run_cpu_time_us":332,"raft.run_wall_time_us":16851,"spinlock_wait_cycles":24802688,"thread_start_us":651666,"threads_started":1,"wal-append.queue_time_us":984168}
I20260709 12:10:54.617514 12523 rpcz_store.cc:276] 0709 12:10:52.960147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:53.436742 (+476595us) service_pool.cc:224] Handling call
0709 12:10:53.459396 (+ 22654us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:53.467109 (+  7713us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:53.467159 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:53.472955 (+  5796us) log.cc:844] Serialized 26 byte log entry
0709 12:10:53.966490 (+493535us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:53.972778 (+  6288us) log.cc:844] Serialized 14 byte log entry
0709 12:10:53.973309 (+   531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:53.973394 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:54.616839 (+643445us) raft_consensus.cc:1685] finished
0709 12:10:54.616861 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:54.616993 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8586,"prepare.queue_time_us":5722,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"raft.queue_time_us":25225,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":7190,"thread_start_us":493050,"threads_started":1,"wal-append.queue_time_us":565594}
I20260709 12:10:54.617072 15639 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:54.651187 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 57882.9us
I20260709 12:10:54.653810 12494 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:10:54.655817 15193 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:54.656241 15193 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:54.656616 15193 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:54.664953 15411 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:10:54.666961 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:54.679270 12675 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:10:54.679925 12679 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:10:54.683827 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 91670.3us
W20260709 12:10:54.686303 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 92925.2us
I20260709 12:10:54.688706 12572 leader_election.cc:400] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6.
I20260709 12:10:54.642030 15702 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:54.701157 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:54.703370 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:54.709666 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 43558us
I20260709 12:10:54.689142 12572 leader_election.cc:403] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260709 12:10:54.594313 15618 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.094s
I20260709 12:10:54.719146 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:54.677125 15624 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:10:54.729694 12675 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:54.699223 15720 raft_consensus.cc:3055] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:54.735405 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:54.736027 12679 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:54.682663 15511 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:54.740736 13332 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:54.743834 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:54.744181 12602 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:10:54.744501 12599 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:54.751592 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:54.756203 12606 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:10:54.721248 15650 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:54.712824 15622 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:54.759023 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:54.759931 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:54.727370 15199 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:10:54.763801 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:54.766018 15241 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:10:54.766819 15241 consensus_queue.cc:1243] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 6.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:10:54.767087 15538 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:10:54.768041 15461 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:54.768236 12600 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:10:54.768452 15461 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:54.769407 15461 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:54.770335 12600 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:10:54.770890 15461 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:54.772512 12600 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260709 12:10:54.729895 15604 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:54.783174 15674 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:54.734547 15720 raft_consensus.cc:740] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:54.785395 15601 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:10:54.785248 15622 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:54.787963 15622 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:54.791553 12494 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:54.793390 15601 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:54.793768 15601 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:54.704610 15661 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
I20260709 12:10:54.727643 15727 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.082s
I20260709 12:10:54.795187 15601 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:54.795949 15594 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:54.796545 15594 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:54.799793 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1725) took 1096 ms. Trace:
I20260709 12:10:54.800269 12680 rpcz_store.cc:276] 0709 12:10:53.703396 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:53.706323 (+  2927us) service_pool.cc:224] Handling call
0709 12:10:53.706580 (+   257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:53.706767 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:10:54.046505 (+339738us) log.cc:844] Serialized 14 byte log entry
0709 12:10:54.798741 (+752236us) log.cc:844] Serialized 14 byte log entry
0709 12:10:54.799189 (+   448us) log.cc:844] Serialized 14 byte log entry
0709 12:10:54.799336 (+   147us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:54.799381 (+    45us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:54.799446 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:54.799558 (+   112us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:54.799716 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":499738,"prepare.queue_time_us":517806,"prepare.run_cpu_time_us":152,"prepare.run_wall_time_us":2398,"thread_start_us":530191,"threads_started":2,"wal-append.queue_time_us":481316}
I20260709 12:10:54.805330 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2022) took 1016 ms. Trace:
I20260709 12:10:54.797744 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:54.805848 12673 rpcz_store.cc:276] 0709 12:10:53.789260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:53.790460 (+  1200us) service_pool.cc:224] Handling call
0709 12:10:53.790798 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:53.791068 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:10:54.111563 (+320495us) log.cc:844] Serialized 14 byte log entry
0709 12:10:54.785791 (+674228us) spinlock_profiling.cc:243] Waited 8.69 ms on lock 0x7b180000f580. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:10:54.785964 (+   173us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:54.786015 (+    51us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:54.786075 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:54.805150 (+ 19075us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:54.805268 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39524,"prepare.queue_time_us":464419,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":528609,"raft.run_cpu_time_us":56,"raft.run_wall_time_us":56,"spinlock_wait_cycles":19987840,"thread_start_us":441509,"threads_started":2,"wal-append.queue_time_us":408724}
W20260709 12:10:54.752727 15649 consensus_meta.cc:287] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.581s	user 0.012s	sys 0.000s
W20260709 12:10:54.823478 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46791.8us
I20260709 12:10:54.811187 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:54.824572 12673 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 8
I20260709 12:10:54.789500 15193 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:54.747366 15442 raft_consensus.cc:3055] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:54.815737 12680 pending_rounds.cc:85] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 4
I20260709 12:10:54.817458 12427 catalog_manager.cc:5697] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:54.837383 12680 pending_rounds.cc:107] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:10:54.837592 12643 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:54.837774 12680 raft_consensus.cc:2889] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:54.727809 15725 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.126s
I20260709 12:10:54.843607 12526 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260709 12:10:54.840488 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 151973us
I20260709 12:10:54.852479 12572 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:54.841552 12679 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:10:54.826642 15601 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:54.808838 15587 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:54.829627 15517 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6.
I20260709 12:10:54.757761 15411 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.091s
I20260709 12:10:54.842583 15613 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:54.876945 12673 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:10:54.849865 15573 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:54.855595 15693 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:54.857057 15725 consensus_queue.cc:1243] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 5.2 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 1
I20260709 12:10:54.836635 15442 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:54.908784 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:10:54.931511 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 79229.5us
I20260709 12:10:54.816874 15522 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.059s
I20260709 12:10:54.817044 15357 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:10:54.829402 15672 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.045s
I20260709 12:10:54.939062 12677 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
W20260709 12:10:54.933338 15714 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
W20260709 12:10:54.953125 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 100729us
I20260709 12:10:54.954102 12572 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:54.809795 15650 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:54.952328 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 114816us
I20260709 12:10:54.958472 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:54.960171 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:54.950414 15693 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:54.965185 15693 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:54.969430 15624 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:10:54.970575 15723 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:10:54.971695 15643 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.668s	user 0.012s	sys 0.000s
I20260709 12:10:54.972056 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:54.972188 15643 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:54.973620 12522 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 8: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:54.974615 15622 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:54.974964 15199 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:54.975222 15622 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:54.975754 15199 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:54.976363 15622 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:54.977108 15511 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:54.978778 15594 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:54.979511 15594 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:10:54.704160 15718 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:10:54.873601 15587 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:54.981853 12572 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:54.982714 15718 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:54.983201 15587 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:54.983531 15613 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:54.983985 15613 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:54.984642 12572 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:54.985141 15650 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:54.985440 15718 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:54.836977 15649 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:54.987151 12572 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:54.935798 15703 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:54.991997 12673 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:10:54.991073 12677 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:10:54.991241 15661 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.017s
I20260709 12:10:55.009699 15601 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:55.015645 15729 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:10:55.010531 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:55.015980 15725 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:55.016547 15725 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:55.016950 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:55.017839 12677 raft_consensus.cc:2410] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 9 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:10:55.013238 12522 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:10:55.013895 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:55.018041 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:55.018908 15725 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:55.019178 12527 raft_consensus.cc:2410] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260709 12:10:55.019649 12528 pending_rounds.cc:85] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 1
I20260709 12:10:55.019999 12528 pending_rounds.cc:107] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 5.2
I20260709 12:10:55.020081 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:55.015247 15732 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:55.020936 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:55.018139 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:55.014742 15643 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:55.020823 12524 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:55.023753 12643 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:55.020313 12528 raft_consensus.cc:2889] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:55.026995 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:55.027642 13336 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:55.027563 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:55.028391 12677 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:10:55.032070 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:55.032680 12677 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:55.032531 12571 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:55.052632 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:55.064818 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:55.064939 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:55.065883 12606 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:55.070717 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:55.072450 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:55.074666 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:55.075628 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:55.076805 12679 raft_consensus.cc:2410] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:10:54.963364 15717 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:55.122354 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:55.124607 12600 raft_consensus.cc:2376] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7. Re-sending same reply.
I20260709 12:10:55.105727 12524 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:10:54.991271 15718 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:55.020932 15732 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:55.129608 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:55.139957 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:55.070185 15725 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:55.141264 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:55.127358 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:55.143041 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:55.145123 12600 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260709 12:10:55.134508 12679 raft_consensus.cc:1217] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Deduplicated request from leader. Original: 2.2->[3.3-5.4]   Dedup: 3.3->[5.4-5.4]
W20260709 12:10:55.141889 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 67325.9us
I20260709 12:10:55.147603 12679 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 12:10:55.148414 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:55.092793 15587 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:55.121773 15650 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:55.135702 15324 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260709 12:10:55.179397 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 376028us
I20260709 12:10:55.181071 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:55.185555 15703 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:55.140892 15725 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:55.172418 15594 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:55.217509 15587 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:55.308382 15726 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:55.297307 15732 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:55.246503 15357 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:55.287500 15700 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.015s	sys 0.017s
I20260709 12:10:55.092588 15442 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:55.193408 15717 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.103s
I20260709 12:10:55.316473 13336 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:10:55.321589 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:55.321979 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:55.327564 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:55.413129 12604 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:10:55.413501 12523 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:10:55.217144 15650 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:55.219784 15324 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:55.380503 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:55.405599 15700 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.015s	sys 0.017s
I20260709 12:10:55.423102 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:55.388545 12572 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:55.407097 12427 catalog_manager.cc:5697] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:55.414268 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:55.368243 15725 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:55.416282 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:55.425977 12601 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.022s	sys 0.060s
I20260709 12:10:55.430770 12572 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:55.431118 12601 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.022s	sys 0.060s
W20260709 12:10:55.432056 12599 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.049s	sys 0.058s
W20260709 12:10:55.432588 12599 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.050s	sys 0.058s
I20260709 12:10:55.421635 15324 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:55.414141 15594 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:55.454672 15461 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.000s	sys 0.019s
I20260709 12:10:55.419791 15717 consensus_queue.cc:1243] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 6.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
W20260709 12:10:55.453536 13304 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.018s	sys 0.058s
W20260709 12:10:55.458053 12675 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.012s	sys 0.008s
I20260709 12:10:55.437184 15718 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:10:55.497334 13304 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.642s	user 0.018s	sys 0.059s
I20260709 12:10:55.414808 15357 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:55.498884 12675 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.012s	sys 0.009s
I20260709 12:10:55.450490 15649 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:55.462266 15700 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:55.405850 15726 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:55.466991 12674 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.590s	user 0.004s	sys 0.017s
W20260709 12:10:55.504492 12674 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.627s	user 0.004s	sys 0.018s
W20260709 12:10:55.501720 15461 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.001s	sys 0.022s
I20260709 12:10:55.470438 15723 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:55.522861 12677 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:10:55.507130 15693 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:55.508196 15700 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:55.469551 15594 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:55.512673 15649 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:55.528108 15743 raft_consensus.cc:2764] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
I20260709 12:10:55.508746 15717 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.046s
W20260709 12:10:55.596562 15730 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.308s	user 0.000s	sys 0.002s
W20260709 12:10:55.618595 13332 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.807s	user 0.032s	sys 0.069s
W20260709 12:10:55.592972 15737 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:10:55.574632 15740 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:55.637244 13332 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.825s	user 0.033s	sys 0.070s
I20260709 12:10:55.641876 12524 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:10:55.519196 15587 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:55.630502 15744 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:55.616972 15649 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:55.662542 15752 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:10:55.649410 15700 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:55.671375 15752 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:55.671994 15752 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:55.672410 15649 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:55.568280 15693 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:55.684409 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:55.690629 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:55.663883 15733 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:10:55.671376 15700 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:55.730445 12523 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:55.684338 15743 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:55.731503 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:55.733282 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:55.734432 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1972) took 1774 ms. Trace:
I20260709 12:10:55.735052 12603 rpcz_store.cc:276] 0709 12:10:53.959762 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:53.964874 (+  5112us) service_pool.cc:224] Handling call
0709 12:10:53.965282 (+   408us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:53.967088 (+  1806us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:53.967119 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:54.604003 (+636884us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b18002c8720. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a
0709 12:10:54.604448 (+   445us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b2c00000e98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:10:54.605126 (+   678us) log.cc:844] Serialized 52 byte log entry
0709 12:10:54.859589 (+254463us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:54.859651 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:54.859753 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:55.733975 (+874222us) raft_consensus.cc:1685] finished
0709 12:10:55.734005 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:55.734237 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1348,"raft.queue_time_us":4135,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":1060,"spinlock_wait_cycles":73421568,"thread_start_us":253999,"threads_started":1,"wal-append.queue_time_us":573159}
W20260709 12:10:55.704743 15739 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.269s	user 0.001s	sys 0.001s
I20260709 12:10:55.656060 15740 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:55.665529 15723 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.044s
I20260709 12:10:55.749991 12524 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:55.701678 15601 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:55.692847 15742 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:55.783751 12522 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:10:55.703320 15733 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:10:55.790553 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:55.726015 15638 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:10:55.796005 15755 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
W20260709 12:10:55.796124 15747 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 12:10:55.796329 15752 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:55.842803 15741 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.277s	user 0.000s	sys 0.001s
I20260709 12:10:55.843623 15726 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:55.844440 15594 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:55.756440 15650 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:55.846542 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1788) took 1140 ms. Trace:
I20260709 12:10:55.847152 12529 rpcz_store.cc:276] 0709 12:10:54.706621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.706813 (+   192us) service_pool.cc:224] Handling call
0709 12:10:54.707974 (+  1161us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:54.725136 (+ 17162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:54.725187 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:54.725804 (+   617us) log.cc:844] Serialized 52 byte log entry
0709 12:10:55.308108 (+582304us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:55.308176 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:55.308286 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:55.846200 (+537914us) raft_consensus.cc:1685] finished
0709 12:10:55.846232 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:55.846449 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21287,"raft.queue_time_us":30217,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":14106,"thread_start_us":482295,"threads_started":1,"wal-append.queue_time_us":757645}
W20260709 12:10:55.797308 15746 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:10:55.799615 15748 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
W20260709 12:10:55.851401 15613 consensus_meta.cc:287] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.867s	user 0.015s	sys 0.000s
I20260709 12:10:55.851894 15613 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:55.763447 15743 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:55.854537 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1994) took 1104 ms. Trace:
I20260709 12:10:55.855207 12599 rpcz_store.cc:276] 0709 12:10:54.750185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.752950 (+  2765us) service_pool.cc:224] Handling call
0709 12:10:54.759974 (+  7024us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:54.760190 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:54.760214 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:54.812180 (+ 51966us) log.cc:844] Serialized 26 byte log entry
0709 12:10:55.433216 (+621036us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:55.433283 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:55.433368 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:55.854235 (+420867us) raft_consensus.cc:1685] finished
0709 12:10:55.854266 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:55.854446 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":574920,"threads_started":1,"wal-append.queue_time_us":826886}
I20260709 12:10:55.857370 15743 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:55.857685 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1731) took 1316 ms. Trace:
I20260709 12:10:55.858646 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 232) took 1102 ms. Trace:
I20260709 12:10:55.803813 15740 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:55.860920 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2041) took 1126 ms. Trace:
I20260709 12:10:55.812779 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:55.813216 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:55.695822 15744 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:55.838750 15753 log.cc:927] Time spent T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:10:55.839989 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2032) took 1215 ms. Trace:
I20260709 12:10:55.873718 12678 rpcz_store.cc:276] 0709 12:10:54.624999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.634237 (+  9238us) service_pool.cc:224] Handling call
0709 12:10:54.787760 (+153523us) spinlock_profiling.cc:243] Waited 1.31 ms on lock 0x7b58001f2c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:10:54.787979 (+   219us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:54.788344 (+   365us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:54.788372 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:54.788883 (+   511us) log.cc:844] Serialized 26 byte log entry
0709 12:10:55.122280 (+333397us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:55.122319 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:55.122394 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:55.839671 (+717277us) raft_consensus.cc:1685] finished
0709 12:10:55.839696 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:55.839925 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7478,"raft.run_cpu_time_us":55,"raft.run_wall_time_us":55,"spinlock_wait_cycles":3013376,"thread_start_us":332946,"threads_started":1,"wal-append.queue_time_us":580203}
I20260709 12:10:55.859057 12601 rpcz_store.cc:276] 0709 12:10:54.756691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.756803 (+   112us) service_pool.cc:224] Handling call
0709 12:10:54.757120 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:54.835880 (+ 78760us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:54.835913 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:54.849347 (+ 13434us) log.cc:844] Serialized 26 byte log entry
0709 12:10:54.966969 (+117622us) spinlock_profiling.cc:243] Waited 2.85 ms on lock 0x7b480024c258. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:10:55.431719 (+464750us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:55.431786 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:55.431894 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:55.858224 (+426330us) raft_consensus.cc:1685] finished
0709 12:10:55.858247 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:55.858563 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91529,"raft.queue_time_us":93898,"raft.run_cpu_time_us":2320,"raft.run_wall_time_us":111410,"spinlock_wait_cycles":6550784,"thread_start_us":559600,"threads_started":1,"wal-append.queue_time_us":825787}
I20260709 12:10:55.715209 15649 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 12:10:55.858107 12676 rpcz_store.cc:276] 0709 12:10:54.541357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.541531 (+   174us) service_pool.cc:224] Handling call
0709 12:10:54.541913 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:54.803180 (+261267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:54.803211 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:54.803749 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 12:10:55.135715 (+331966us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:10:55.135774 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:55.135870 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:55.857449 (+721579us) raft_consensus.cc:1685] finished
0709 12:10:55.857471 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:55.857619 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":260727,"raft.queue_time_us":260908,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":74,"thread_start_us":331388,"threads_started":1,"wal-append.queue_time_us":668975}
W20260709 12:10:55.867090 12643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.019s	sys 0.010s
W20260709 12:10:55.907385 12643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.019s	sys 0.010s
I20260709 12:10:55.861339 12675 rpcz_store.cc:276] 0709 12:10:54.734862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.737570 (+  2708us) service_pool.cc:224] Handling call
0709 12:10:54.759509 (+ 21939us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:54.827267 (+ 67758us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:54.827307 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:54.827771 (+   464us) log.cc:844] Serialized 26 byte log entry
0709 12:10:55.499385 (+671614us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:55.499434 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:55.499517 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:55.860671 (+361154us) raft_consensus.cc:1685] finished
0709 12:10:55.860694 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:55.860856 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49514,"raft.queue_time_us":68509,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":14886,"thread_start_us":617397,"threads_started":1,"wal-append.queue_time_us":946794}
W20260709 12:10:55.897508 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:55.812538 15357 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:55.947695 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 766679us
W20260709 12:10:55.952977 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:10:55.916009 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 417062us
I20260709 12:10:55.805244 15601 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:56.013708 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 64875.3us
I20260709 12:10:55.963178 15613 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:55.873488 15742 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:56.020361 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1999) took 1268 ms. Trace:
I20260709 12:10:56.026933 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:56.028673 12643 leader_election.cc:400] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5.
I20260709 12:10:56.028972 12643 leader_election.cc:403] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:10:56.026023 12598 rpcz_store.cc:276] 0709 12:10:54.751898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.752715 (+   817us) service_pool.cc:224] Handling call
0709 12:10:54.753062 (+   347us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:54.828302 (+ 75240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:54.828350 (+    48us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:54.849646 (+ 21296us) log.cc:844] Serialized 52 byte log entry
0709 12:10:55.378470 (+528824us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:55.726443 (+347973us) log.cc:844] Serialized 14 byte log entry
0709 12:10:55.747387 (+ 20944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:55.747507 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:56.020118 (+272611us) raft_consensus.cc:1685] finished
0709 12:10:56.020140 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:56.020292 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":135105,"raft.queue_time_us":87271,"raft.run_cpu_time_us":348,"raft.run_wall_time_us":42771,"thread_start_us":757408,"threads_started":2,"wal-append.queue_time_us":697907}
W20260709 12:10:56.015579 15735 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:10:56.043517 13332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 485) took 1300 ms. Trace:
I20260709 12:10:56.050464 13332 rpcz_store.cc:276] 0709 12:10:54.743154 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.760547 (+ 17393us) service_pool.cc:224] Handling call
0709 12:10:54.760814 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:54.761007 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:10:55.643121 (+882114us) log.cc:844] Serialized 14 byte log entry
0709 12:10:56.042981 (+399860us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:56.043059 (+    78us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:10:56.043134 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:56.043229 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:56.043417 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20724,"prepare.queue_time_us":1027970,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"thread_start_us":1207281,"threads_started":2}
I20260709 12:10:56.066676 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1735) took 1054 ms. Trace:
W20260709 12:10:56.033093 15756 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.252s	user 0.001s	sys 0.001s
I20260709 12:10:56.076431 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1733) took 1271 ms. Trace:
I20260709 12:10:56.079312 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:56.069804 12526 rpcz_store.cc:276] 0709 12:10:55.011852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:55.013699 (+  1847us) service_pool.cc:224] Handling call
0709 12:10:55.014268 (+   569us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:55.014775 (+   507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:55.014809 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:55.015397 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 12:10:55.514969 (+499572us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:56.063845 (+548876us) log.cc:844] Serialized 14 byte log entry
0709 12:10:56.064993 (+  1148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:56.065498 (+   505us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:56.065996 (+   498us) raft_consensus.cc:1685] finished
0709 12:10:56.066012 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:56.066613 (+   601us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19941,"prepare.queue_time_us":548310,"raft.queue_time_us":974,"raft.run_cpu_time_us":61,"raft.run_wall_time_us":62,"thread_start_us":969196,"threads_started":2,"wal-append.queue_time_us":709220}
W20260709 12:10:56.074851 13302 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.621s	user 0.004s	sys 0.020s
I20260709 12:10:56.076954 12674 rpcz_store.cc:276] 0709 12:10:54.804584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.806318 (+  1734us) service_pool.cc:224] Handling call
0709 12:10:54.806758 (+   440us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:54.837390 (+ 30632us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:54.837440 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:54.838210 (+   770us) log.cc:844] Serialized 52 byte log entry
0709 12:10:55.510504 (+672294us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:10:55.510549 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:55.510646 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:56.076160 (+565514us) raft_consensus.cc:1685] finished
0709 12:10:56.076184 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:56.076357 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30134,"raft.queue_time_us":35380,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":9272,"thread_start_us":627867,"threads_started":1,"wal-append.queue_time_us":826003}
W20260709 12:10:56.084326 13302 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.653s	user 0.005s	sys 0.021s
I20260709 12:10:56.031342 15773 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:56.085671 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 669494us
I20260709 12:10:56.060365 15761 raft_consensus.cc:2764] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260709 12:10:56.093204 12525 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.558s	user 0.014s	sys 0.013s
I20260709 12:10:55.981268 15733 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:56.062824 15743 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 4. Current term is 5.
W20260709 12:10:56.101570 12525 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.566s	user 0.015s	sys 0.014s
W20260709 12:10:56.096969 12604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.025s	sys 0.048s
W20260709 12:10:56.092051 15731 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.801s	user 0.005s	sys 0.000s
W20260709 12:10:56.105746 12604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.026s	sys 0.049s
I20260709 12:10:56.104542 12569 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:56.100518 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:56.093513 15594 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:56.115536 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1738) took 1102 ms. Trace:
I20260709 12:10:56.116029 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:56.117547 12530 rpcz_store.cc:276] 0709 12:10:55.012788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:55.012927 (+   139us) service_pool.cc:224] Handling call
0709 12:10:55.013302 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:55.013693 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:55.013724 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:55.014246 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 12:10:55.471055 (+456809us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:55.735925 (+264870us) log.cc:844] Serialized 14 byte log entry
0709 12:10:56.024348 (+288423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:56.024546 (+   198us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:56.115213 (+ 90667us) raft_consensus.cc:1685] finished
0709 12:10:56.115244 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:56.115456 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2128,"prepare.queue_time_us":544173,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":94248,"raft.run_cpu_time_us":209,"raft.run_wall_time_us":211,"thread_start_us":965818,"threads_started":3,"wal-append.queue_time_us":738253}
I20260709 12:10:56.131178 12643 leader_election.cc:400] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8.
I20260709 12:10:56.131603 12643 leader_election.cc:403] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:10:56.135848 13334 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: []
I20260709 12:10:56.141949 15613 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:10:56.114431 15633 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:56.152827 15193 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:56.157269 15742 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:56.158177 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1735) took 1352 ms. Trace:
I20260709 12:10:56.158620 12671 rpcz_store.cc:276] 0709 12:10:54.805278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.807784 (+  2506us) service_pool.cc:224] Handling call
0709 12:10:54.808088 (+   304us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:55.330524 (+522436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:55.330555 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:55.366762 (+ 36207us) log.cc:844] Serialized 52 byte log entry
0709 12:10:55.710779 (+344017us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:55.710855 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:55.713013 (+  2158us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:56.157806 (+444793us) raft_consensus.cc:1685] finished
0709 12:10:56.157839 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:56.158061 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29919,"raft.queue_time_us":55996,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":3077,"thread_start_us":781620,"threads_started":2,"wal-append.queue_time_us":709732}
W20260709 12:10:56.164881 15766 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:10:56.166103 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1734) took 1361 ms. Trace:
I20260709 12:10:56.167397 15633 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:56.166548 12672 rpcz_store.cc:276] 0709 12:10:54.804937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.807220 (+  2283us) service_pool.cc:224] Handling call
0709 12:10:54.807495 (+   275us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:55.319319 (+511824us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:55.319353 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:55.369940 (+ 50587us) log.cc:844] Serialized 26 byte log entry
0709 12:10:55.693195 (+323255us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:55.693307 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:55.693473 (+   166us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:56.165861 (+472388us) raft_consensus.cc:1685] finished
0709 12:10:56.165885 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:56.166043 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34770,"raft.queue_time_us":124439,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":6682,"thread_start_us":742694,"threads_started":2,"wal-append.queue_time_us":624072}
I20260709 12:10:56.168010 15633 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:56.168588 15763 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:10:56.170301 15633 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:56.152663 15759 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:10:56.128834 15743 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:10:56.184525 13304 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.229s (SENT) [suppressed 21 similar messages]
I20260709 12:10:56.189232 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:56.190510 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 59483.7us
I20260709 12:10:56.192252 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2042) took 1382 ms. Trace:
I20260709 12:10:56.192679 12680 rpcz_store.cc:276] 0709 12:10:54.810075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.815082 (+  5007us) service_pool.cc:224] Handling call
0709 12:10:54.815452 (+   370us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:55.318736 (+503284us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:55.318777 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:55.321120 (+  2343us) log.cc:844] Serialized 52 byte log entry
0709 12:10:55.730180 (+409060us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:55.730308 (+   128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:55.730438 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:56.167906 (+437468us) spinlock_profiling.cc:243] Waited 3.83 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:56.174362 (+  6456us) raft_consensus.cc:1685] finished
0709 12:10:56.174382 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:56.192182 (+ 17800us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4046,"raft.queue_time_us":124164,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":118,"spinlock_wait_cycles":8802176,"thread_start_us":881920,"threads_started":2,"wal-append.queue_time_us":741925}
I20260709 12:10:56.093627 15773 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:56.198009 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:56.202060 12603 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:10:56.199962 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:56.133733 15761 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8.
I20260709 12:10:56.206605 12643 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:56.211364 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:56.210551 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:56.215238 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:56.217389 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 1970) took 2265 ms. Trace:
I20260709 12:10:56.217593 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:56.190570 15726 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:56.198299 15744 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:56.217959 12605 rpcz_store.cc:276] 0709 12:10:53.951594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:53.955221 (+  3627us) service_pool.cc:224] Handling call
0709 12:10:53.955631 (+   410us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:53.956032 (+   401us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:53.956063 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:54.615403 (+659340us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b2c00000e98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:10:54.615943 (+   540us) log.cc:844] Serialized 52 byte log entry
0709 12:10:54.827137 (+211194us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:10:54.827182 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:54.827266 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:55.846015 (+1018749us) spinlock_profiling.cc:243] Waited 12.7 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:10:56.217165 (+371150us) raft_consensus.cc:1685] finished
0709 12:10:56.217191 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:56.217327 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":330,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":85,"spinlock_wait_cycles":61453312,"thread_start_us":210729,"threads_started":1,"wal-append.queue_time_us":507268}
I20260709 12:10:56.223989 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:56.222855 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:56.225342 12603 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 10.
I20260709 12:10:56.214178 15743 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:56.231671 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:56.256526 13338 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:56.213608 15742 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:10:56.257638 15742 consensus_queue.cc:1243] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:10:56.258558 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:56.207907 15594 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:10:56.232704 15774 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:10:56.233698 13332 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:10:56.247859 15693 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:56.250408 15744 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:10:56.251552 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:56.252200 15727 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:56.254293 15773 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:56.232781 15745 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:56.255203 15199 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:56.280462 13338 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:56.265844 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:56.228008 15726 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:56.213824 15573 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.057s
I20260709 12:10:56.227263 15601 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:56.303985 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:56.269508 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:56.272625 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:56.271757 12672 raft_consensus.cc:1217] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Deduplicated request from leader. Original: 2.2->[3.3-5.4]   Dedup: 3.3->[5.4-5.4]
I20260709 12:10:56.310398 12672 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 12:10:56.223812 15638 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:56.272029 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:56.277953 12523 raft_consensus.cc:1217] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Deduplicated request from leader. Original: 2.2->[3.3-5.4]   Dedup: 3.3->[5.4-5.4]
I20260709 12:10:56.315496 12523 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: [3.3-5.4]
I20260709 12:10:56.270087 15594 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:56.279014 15717 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:56.265733 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:56.267091 15725 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.818s	user 0.008s	sys 0.006s
W20260709 12:10:56.272037 15324 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.771s	user 0.004s	sys 0.011s
I20260709 12:10:56.278761 15723 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:56.317137 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 86779us
I20260709 12:10:56.273316 12643 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:56.323046 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:56.361726 12672 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:10:56.364302 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:56.326426 15594 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:56.863445 15594 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:56.380041 15587 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:56.831984 13335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 281) took 1501 ms. Trace:
I20260709 12:10:56.499717 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 257) took 1542 ms. Trace:
W20260709 12:10:56.758517 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:56.369030 15324 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:56.853682 15199 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:10:56.443401 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:56.833521 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:56.858884 12680 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
W20260709 12:10:56.802858 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 529413us
I20260709 12:10:56.864161 15573 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:56.389784 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1742) took 1375 ms. Trace:
I20260709 12:10:56.331472 12494 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:56.319978 15699 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:56.360265 15638 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:56.880136 15324 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:56.356599 15725 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:56.881405 15638 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:56.881677 15573 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:56.318173 15727 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:56.884325 15725 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:56.884559 15573 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:56.319803 15732 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
I20260709 12:10:56.330499 15777 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:56.886772 15777 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:56.335186 15717 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:56.532694 15736 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.053s	user 0.004s	sys 0.001s
I20260709 12:10:56.878074 15199 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:56.893234 15199 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:56.894977 15638 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:56.896143 15638 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:56.527249 15726 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:56.879441 15733 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:56.897518 15638 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:56.365650 15723 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:56.925559 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:56.931710 15298 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:56.931501 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:56.932236 15638 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:56.932508 15298 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:56.936226 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 604742us
W20260709 12:10:56.937084 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 104092us
I20260709 12:10:56.944705 15511 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:56.925166 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:56.947335 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:56.947252 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:56.929883 12673 rpcz_store.cc:276] 0709 12:10:54.957598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:54.961329 (+  3731us) service_pool.cc:224] Handling call
0709 12:10:54.961695 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:54.992610 (+ 30915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:56.499165 (+1506555us) spinlock_profiling.cc:243] Waited 25.7 ms on lock 0x7b54000924d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:56.499612 (+   447us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59210624}
I20260709 12:10:56.950672 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:56.950946 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 627871us
I20260709 12:10:56.915779 12528 rpcz_store.cc:276] 0709 12:10:55.013913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:55.019037 (+  5124us) service_pool.cc:224] Handling call
0709 12:10:55.019405 (+   368us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:10:55.029125 (+  9720us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:55.029154 (+    29us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:10:55.029793 (+   639us) log.cc:844] Serialized 78 byte log entry
0709 12:10:55.457942 (+428149us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:55.735838 (+277896us) log.cc:844] Serialized 14 byte log entry
0709 12:10:56.021343 (+285505us) log.cc:844] Serialized 14 byte log entry
0709 12:10:56.027619 (+  6276us) log.cc:844] Serialized 14 byte log entry
0709 12:10:56.027951 (+   332us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:56.028064 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:56.389553 (+361489us) raft_consensus.cc:1685] finished
0709 12:10:56.389574 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:56.389717 (+   143us) inbound_call.cc:177] Queueing success response
0709 12:10:56.430754 (+ 41037us) spinlock_profiling.cc:243] Waited 8.85 ms on lock 0x7b180025abc0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485933557 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
Metrics: {"mutex_wait_us":8103,"prepare.queue_time_us":714193,"prepare.run_cpu_time_us":170,"prepare.run_wall_time_us":1629,"raft.queue_time_us":60355,"raft.run_cpu_time_us":456,"raft.run_wall_time_us":53898,"spinlock_wait_cycles":81087744,"thread_start_us":961912,"threads_started":3,"wal-append.queue_time_us":931513}
I20260709 12:10:56.952589 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:56.940348 15782 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:56.993811 15777 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:56.865711 13335 rpcz_store.cc:276] 0709 12:10:55.330025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:55.366642 (+ 36617us) service_pool.cc:224] Handling call
0709 12:10:55.409452 (+ 42810us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:55.922154 (+512702us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:55.922208 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:55.958407 (+ 36199us) log.cc:844] Serialized 26 byte log entry
0709 12:10:56.103486 (+145079us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:10:56.103524 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:56.560465 (+456941us) spinlock_profiling.cc:243] Waited 88.6 ms on lock 0x7b54004e84d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:56.560529 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:56.560600 (+    71us) raft_consensus.cc:1685] finished
0709 12:10:56.560621 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:56.831516 (+270895us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":172641,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"spinlock_wait_cycles":203860864,"thread_start_us":568119,"threads_started":2,"wal-append.queue_time_us":203822}
I20260709 12:10:57.002633 15638 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:10:57.002880 15752 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:57.003254 15638 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:57.003782 15638 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:57.003630 15752 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:56.992704 15782 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:10:57.008173 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 97559.9us
I20260709 12:10:57.008411 15782 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:57.008841 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:57.009457 12643 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:57.001024 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.013849 15724 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:56.953969 15298 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:57.021737 15573 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:57.022202 15324 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:57.022786 15573 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:57.024075 15594 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:57.024194 15324 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:57.024658 15594 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:57.026196 15752 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:57.024780 15324 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:57.028029 15725 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:57.028314 15723 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:57.029026 15723 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:57.028805 15725 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:57.029748 15723 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.031196 15732 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:57.031977 15732 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.034390 15717 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:57.035658 15734 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.577s	user 0.001s	sys 0.002s
I20260709 12:10:57.035917 15723 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:57.026463 15699 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:57.030007 15324 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.040525 15727 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:57.048911 13302 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:57.041762 15724 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.038179 15699 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.070336 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:57.071367 12601 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:57.071813 12494 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:57.073925 15618 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:57.074967 15618 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:57.076498 15618 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:57.078022 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:57.078938 12599 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:57.068936 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.068378 15732 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:57.080250 15727 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:57.052125 15594 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.094458 15511 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:57.106168 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.122418 15511 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.123410 15603 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:10:57.124130 15603 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:57.102586 15324 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:57.131381 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.099296 15661 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:57.140398 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:57.131388 15765 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:10:57.125188 15511 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:57.160455 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.161792 15622 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:57.161936 15573 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:10:57.162833 15622 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:57.164415 15806 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:57.165244 15806 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.168112 15806 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:57.179132 15720 consensus_queue.cc:260] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.181381 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.185648 15442 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:57.186185 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:57.187131 12604 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:10:57.190220 15727 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.192932 15727 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:57.193542 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 184328us
I20260709 12:10:57.194494 15661 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.197391 15661 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:57.199388 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:57.200546 12597 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:57.204663 15725 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:57.207398 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:57.160620 13335 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
W20260709 12:10:57.217815 15749 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.069s	user 0.000s	sys 0.003s
I20260709 12:10:57.130975 15732 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:57.236196 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:57.158393 15765 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:57.239518 12601 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:10:57.173230 15622 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:57.241031 15594 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:57.243772 15765 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:10:57.260486 15724 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:57.260800 15587 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:57.261822 15622 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:57.266206 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:57.249568 15718 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:57.283886 15720 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:57.261853 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:57.282852 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:57.295176 13332 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 8: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:57.297860 15622 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.301203 15622 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:57.297869 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.266453 15732 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:57.328542 15638 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.332594 12529 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:57.332700 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.334245 15638 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:57.340224 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:57.344225 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:57.357275 12496 connection.cc:256] RPC call timeout handler was delayed by -1.1210155640001176s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1555s for a response
I20260709 12:10:57.358968 15777 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:57.359702 15777 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:57.357991 12496 connection.cc:256] RPC call timeout handler was delayed by -1.1152160139999978s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16110000000000002s for a response
I20260709 12:10:57.373848 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.306311 15718 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.388741 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.362089 15777 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:57.405189 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:57.406199 12601 raft_consensus.cc:2410] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:10:57.406754 12522 raft_consensus.cc:2410] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:10:57.407738 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.362346 15638 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:57.329443 15587 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.411739 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:57.397897 15612 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:57.414438 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 205421us
I20260709 12:10:57.421924 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:57.433580 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:10:57.425741 15733 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.505s	user 0.007s	sys 0.007s
W20260709 12:10:57.437582 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:57.442862 12531 raft_consensus.cc:2393] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 8: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:10:57.448356 15808 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:10:57.450759 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:57.454963 15811 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:10:57.447009 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.461706 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:57.463218 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1736) took 2407 ms. Trace:
I20260709 12:10:57.469589 13332 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:57.473129 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.478595 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:57.443776 15638 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.478766 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.486829 12598 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:10:57.490924 13335 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 8: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:57.494915 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:57.496337 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:57.497334 12674 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 8: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:10:57.503870 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 60426.9us
I20260709 12:10:57.444940 15612 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.519796 15612 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:57.523816 15522 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:57.524471 15522 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:57.535485 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.541898 15613 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:57.542833 15613 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.545462 15613 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:57.546908 15732 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:10:57.443776 15777 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.112s
I20260709 12:10:57.551605 15603 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:57.556852 15782 consensus_meta.cc:287] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.548s	user 0.012s	sys 0.000s
I20260709 12:10:57.558369 15718 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:57.560678 15522 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:57.564549 15804 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.558s	user 0.001s	sys 0.002s
I20260709 12:10:57.571542 12671 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:10:57.572119 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.573130 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.574733 15718 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:57.579603 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:57.581277 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:57.582641 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:57.604421 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 92888.6us
I20260709 12:10:57.541709 15324 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:57.557437 15782 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:57.607847 15718 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.609165 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:57.610437 13332 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:10:57.635403 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.611037 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:57.637487 13338 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:57.637627 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.638789 15587 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:57.615573 15612 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:10:57.616940 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:57.617650 12606 raft_consensus.cc:2410] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260709 12:10:57.617501 12601 raft_consensus.cc:2376] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7. Re-sending same reply.
I20260709 12:10:57.618238 12530 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:10:57.618408 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:57.633173 12597 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:10:57.669003 12674 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:57.466817 12525 rpcz_store.cc:276] 0709 12:10:55.012178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:55.015921 (+  3743us) service_pool.cc:224] Handling call
0709 12:10:55.016318 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:55.016672 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:55.016705 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:55.017239 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 12:10:55.508052 (+490813us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:55.803816 (+295764us) spinlock_profiling.cc:243] Waited 6.45 ms on lock 0x7b4800002958. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:10:57.176237 (+1372421us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b180031c9c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf
0709 12:10:57.176497 (+   260us) log.cc:844] Serialized 14 byte log entry
0709 12:10:57.397693 (+221196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:57.410642 (+ 12949us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:57.410690 (+    48us) raft_consensus.cc:1685] finished
0709 12:10:57.410702 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:57.419268 (+  8566us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":549816,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":19987,"raft.run_cpu_time_us":1758,"raft.run_wall_time_us":781805,"spinlock_wait_cycles":45843456,"thread_start_us":1219374,"threads_started":3,"wal-append.queue_time_us":956493,"wal-append.run_cpu_time_us":2203,"wal-append.run_wall_time_us":1072121}
I20260709 12:10:57.619563 15638 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:57.613760 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.616993 15720 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:10:57.620126 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:57.621368 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:57.623674 12427 catalog_manager.cc:5697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 5 to 7, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 7 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:57.628558 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:57.630441 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:57.674418 12605 raft_consensus.cc:2410] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:10:57.636492 12524 raft_consensus.cc:2393] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:57.614815 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.642591 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.645498 15819 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:57.651974 15718 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:57.653331 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.657433 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:57.687068 12599 raft_consensus.cc:2410] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:10:57.688465 12675 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:10:57.695883 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.696240 12680 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:10:57.702803 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:57.676394 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 200330us
I20260709 12:10:57.679219 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:57.681835 12529 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
W20260709 12:10:57.701771 12425 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58992 (request call id 170) took 25772 ms (client timeout 15000 ms). Trace:
I20260709 12:10:57.705071 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:57.709213 12602 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:10:57.710273 15819 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.710520 15820 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:10:57.712829 15819 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:57.712901 12522 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:10:57.713743 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:57.683314 15725 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:57.715235 15725 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.715517 12427 catalog_manager.cc:5697] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 4 to 6, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 6 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:10:57.704911 12425 rpcz_store.cc:269] 0709 12:10:31.894699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:31.895101 (+   402us) service_pool.cc:224] Handling call
0709 12:10:57.667433 (+25772332us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:49.330143 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:49.477751 (+147608us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:49.477816 (+    65us) write_op.cc:435] Acquired schema lock
0709 12:10:49.477839 (+    23us) tablet.cc:662] Decoding operations
0709 12:10:49.668780 (+190941us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:49.668860 (+    80us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:49.668891 (+    31us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:10:49.669943 (+  1052us) tablet.cc:701] Row locks acquired
0709 12:10:49.669997 (+    54us) write_op.cc:260] PREPARE: finished
0709 12:10:49.670225 (+   228us) write_op.cc:270] Start()
0709 12:10:49.670379 (+   154us) write_op.cc:276] Timestamp: P: 1783599049670143 usec, L: 0
0709 12:10:49.670401 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:10:49.671226 (+   825us) log.cc:844] Serialized 4057 byte log entry
0709 12:10:50.083612 (+412386us) op_driver.cc:464] REPLICATION: finished
0709 12:10:50.181182 (+ 97570us) write_op.cc:301] APPLY: starting
0709 12:10:50.187620 (+  6438us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:50.238254 (+ 50634us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:50.238417 (+   163us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:50.291426 (+ 53009us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:50.291974 (+   548us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=10
0709 12:10:50.292038 (+    64us) write_op.cc:312] APPLY: finished
0709 12:10:50.292576 (+   538us) log.cc:844] Serialized 76 byte log entry
0709 12:10:50.292880 (+   304us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:50.293083 (+   203us) write_op.cc:454] Released schema lock
0709 12:10:50.293177 (+    94us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:50.308572 (+ 15395us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:50.315747 (+  7175us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":917817,"reported_tablets":44,"child_traces":[["op",{"apply.queue_time_us":93300,"apply.run_cpu_time_us":5638,"apply.run_wall_time_us":112619,"num_ops":10,"prepare.queue_time_us":246997,"prepare.run_cpu_time_us":21763,"prepare.run_wall_time_us":468622,"raft.queue_time_us":114360,"raft.run_cpu_time_us":15029,"raft.run_wall_time_us":101817,"replication_time_us":413002,"thread_start_us":324271,"threads_started":4,"wal-append.queue_time_us":125914,"wal-append.run_cpu_time_us":8939,"wal-append.run_wall_time_us":1509360}]]}
I20260709 12:10:57.722862 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:57.723354 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:57.723639 13334 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:10:57.724195 12603 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:57.724893 13305 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:10:57.725908 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 55051.2us
I20260709 12:10:57.726296 15324 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.726935 15522 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:57.722002 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.699132 15612 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:57.727324 15522 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:57.717854 12427 catalog_manager.cc:5697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 3 to 5, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 5 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:10:57.764314 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:57.762734 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 145943us
W20260709 12:10:57.766286 12643 connection.cc:256] RPC call timeout handler was delayed by -1.4246105150000403s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.31150000000000005s for a response
I20260709 12:10:57.768077 12603 pending_rounds.cc:85] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 2
I20260709 12:10:57.770443 13332 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 7: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:57.733062 15820 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:57.774708 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 294941us
I20260709 12:10:57.771121 12604 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:57.768522 12603 pending_rounds.cc:107] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260709 12:10:57.780622 12427 catalog_manager.cc:5697] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 4 to 6, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 6 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:10:57.763366 15612 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:57.769548 15522 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:57.783845 12597 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:10:57.779385 12603 raft_consensus.cc:2889] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:57.750174 15819 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:57.785606 12599 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:57.776041 15638 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:57.793037 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:57.796277 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.798924 15818 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:57.799114 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.786919 15612 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:57.796181 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:57.800169 12678 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:10:57.794584 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.805194 12679 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 7. Current term is 8. Ops: [5.4-7.5]
I20260709 12:10:57.799782 15818 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.821086 12427 catalog_manager.cc:5697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 3 to 5, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 5 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:10:57.830653 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.781281 15820 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:57.843700 12671 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:10:57.842609 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:57.863588 12672 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:57.863730 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.864120 15703 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:57.864838 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:57.864755 15703 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:57.865134 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.863687 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.867479 12677 raft_consensus.cc:2410] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:10:57.863842 12676 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:10:57.876696 12525 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:57.857822 12427 catalog_manager.cc:5697] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 3 to 4, leader changed from <none> to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 4 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } }
I20260709 12:10:57.874166 12673 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:10:57.807788 15819 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:57.887223 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 91680us
I20260709 12:10:57.821981 15638 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:57.895532 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:57.899827 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:57.902309 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.902806 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:57.907655 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 74436.2us
I20260709 12:10:57.908512 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.919373 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.919597 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.908490 12671 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:10:57.921490 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.923032 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.926151 12673 raft_consensus.cc:2393] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 8: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:10:57.928658 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.928771 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.942026 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:57.906704 15324 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:57.929433 12523 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:10:57.910180 15782 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:57.948004 12675 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 5: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:57.950402 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:57.947681 12427 catalog_manager.cc:5697] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 4 to 5, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 5 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:10:57.949215 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.959144 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.965500 12677 raft_consensus.cc:2410] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:10:57.966049 12674 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:10:57.971212 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.971474 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.947963 15717 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:57.983832 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.984275 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:57.981701 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 63501.1us
I20260709 12:10:57.972342 12427 catalog_manager.cc:5697] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 2 to 5, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 5 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:10:57.987175 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.988801 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.990020 15517 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:57.990243 12674 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:57.992019 12672 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:10:57.994601 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:58.001034 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:58.002185 12675 raft_consensus.cc:2410] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 7 because replica has last-logged OpId of term: 4 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:10:58.004925 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:58.000001 12530 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:58.006927 12494 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:58.000931 12427 catalog_manager.cc:5697] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:58.010481 12671 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:10:58.011667 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:58.013228 12677 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:10:58.018963 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:58.010795 12646 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:10:58.023707 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:58.000730 15517 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:58.026459 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46862.6us
I20260709 12:10:58.028357 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:58.028558 15832 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:10:58.029239 12675 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:10:58.034760 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:58.035938 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:57.995678 15661 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:58.037151 15824 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260709 12:10:58.038938 15826 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.140s	user 0.000s	sys 0.000s
I20260709 12:10:58.039291 15672 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:10:58.040180 15828 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:10:58.043542 12680 raft_consensus.cc:2410] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:10:58.021677 15823 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:58.048110 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:58.053296 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:58.055704 12675 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:10:58.056167 12671 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:10:58.050384 15842 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:58.007349 15717 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.069268 12677 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:10:58.060451 12672 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:10:58.028411 15838 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:58.070173 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:58.043470 12427 catalog_manager.cc:5697] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 1 to 5, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 5 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:10:58.064126 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:58.047349 15830 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.130s	user 0.001s	sys 0.000s
I20260709 12:10:58.076637 12676 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 10.
I20260709 12:10:58.089388 12674 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: []
I20260709 12:10:58.090476 12530 raft_consensus.cc:2410] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:10:58.054759 15672 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:10:58.076267 15825 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 12:10:58.109592 12597 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:10:58.097541 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:58.063127 15823 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:58.101679 12494 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:58.071290 15842 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:58.113770 15842 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:58.114241 15819 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:10:58.116469 15837 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:10:58.118287 15823 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:10:58.119529 15834 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.236s	user 0.002s	sys 0.000s
I20260709 12:10:58.074839 15820 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:58.076016 15703 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:58.124420 15324 raft_consensus.cc:3055] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:10:58.124929 15324 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:10:58.125738 13304 leader_election.cc:336] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.969s (SENT)
I20260709 12:10:58.106169 15517 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:58.142697 15841 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.015s
I20260709 12:10:58.144517 15661 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:10:58.145507 15831 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:10:58.146167 15838 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.148322 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:58.148841 15838 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:58.149031 12675 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:58.149811 15840 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.130s
I20260709 12:10:58.151576 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:10:58.152154 12606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1883) took 3083 ms (client timeout 3115 ms). Trace:
W20260709 12:10:58.152892 12606 rpcz_store.cc:269] 0709 12:10:55.068241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:55.140992 (+ 72751us) service_pool.cc:224] Handling call
0709 12:10:58.151667 (+3010675us) spinlock_profiling.cc:243] Waited 2.47 s on lock 0x7b54002e63d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:58.152029 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5681963904}
I20260709 12:10:58.152446 12525 raft_consensus.cc:2410] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 12:10:58.159338 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:58.166142 15827 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.201s	user 0.000s	sys 0.001s
W20260709 12:10:58.166481 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 70871.2us
I20260709 12:10:58.167057 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:58.169744 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:58.108907 15672 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:58.109138 15587 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:58.203119 12571 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:58.162914 15661 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:10:58.208590 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.203716 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:58.210927 12680 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
W20260709 12:10:58.211303 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 120283us
I20260709 12:10:58.125845 15324 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.217774 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:58.218415 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:58.206818 15820 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:58.229385 12571 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:58.231468 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 105780us
W20260709 12:10:58.232714 13304 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.017s (SENT)
I20260709 12:10:58.223555 12680 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:10:58.236202 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:58.236922 12680 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:58.238622 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:58.180462 15612 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:58.225992 15765 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.945s	user 0.013s	sys 0.001s
I20260709 12:10:58.246995 12571 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:58.247139 12643 leader_election.cc:336] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 3.115s (SENT)
I20260709 12:10:58.245678 13304 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:58.219233 15806 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:58.261993 15806 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:58.262480 15806 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:58.248338 15661 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:58.253196 15839 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:58.258474 12600 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:10:58.259070 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:10:58.259797 15717 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:58.250970 12496 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:58.261229 15845 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:58.206467 15838 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:58.279510 15838 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.284178 15842 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.284209 15638 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:58.248173 15613 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:58.289561 15725 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:58.268586 12600 pending_rounds.cc:85] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 3
I20260709 12:10:58.295863 12600 pending_rounds.cc:107] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 5.4
W20260709 12:10:58.295858 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54894.4us
I20260709 12:10:58.296415 12600 raft_consensus.cc:2889] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:58.297040 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:58.281867 15838 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:58.222530 15587 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:58.307829 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 95480.8us
I20260709 12:10:58.307997 12572 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:58.308568 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:58.311308 12572 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:58.317215 12572 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:58.297938 15839 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:58.321020 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 88376.1us
I20260709 12:10:58.321237 15839 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.322595 13304 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:58.322677 15853 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:58.323351 15853 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:58.323927 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:58.294256 15820 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:58.324709 15603 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:10:58.324939 15624 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:10:58.325224 13304 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:58.323442 15442 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:58.377611 15720 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:58.378296 15720 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:58.378849 15720 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:10:58.388641 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 80148.9us
I20260709 12:10:58.390093 12571 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:58.303658 15661 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:58.396128 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:58.388610 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 166662us
W20260709 12:10:58.329015 15522 consensus_meta.cc:287] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.535s	user 0.007s	sys 0.006s
I20260709 12:10:58.268925 15765 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:58.408102 12571 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:58.409193 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 174654us
I20260709 12:10:58.317544 15517 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:58.421690 15522 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:58.334472 15624 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:58.334723 15732 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:58.394620 15728 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:58.374954 15717 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:58.401506 15699 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:58.402248 15661 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:58.376084 15818 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:58.316140 15845 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.440251 12680 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:10:58.437126 12672 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:10:58.442338 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.449661 15612 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:58.441318 15699 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:58.450834 15699 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:58.426766 15517 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:58.324492 15853 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:58.460286 15587 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:58.437619 15842 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:58.469266 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:58.395115 15823 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.472930 15856 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.005s
I20260709 12:10:58.479619 15765 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:58.471598 12496 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:58.483983 12643 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:58.453305 15661 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:58.492097 15823 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:58.457850 15773 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:58.467823 15517 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:58.495671 12523 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:58.498484 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:10:58.538919 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 143382us
I20260709 12:10:58.543110 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:58.543171 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 103507us
I20260709 12:10:58.545050 15765 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:58.545440 13338 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: []
W20260709 12:10:58.551029 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 82887.5us
I20260709 12:10:58.552130 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:58.552953 15856 consensus_queue.cc:1243] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 8.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
W20260709 12:10:58.555358 15862 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:10:58.561897 15845 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:58.567170 15522 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:58.545845 15613 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:10:58.589143 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 111529us
W20260709 12:10:58.540333 15849 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.240s	user 0.002s	sys 0.000s
W20260709 12:10:58.599498 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 55165.8us
I20260709 12:10:58.582109 15842 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:58.588595 15839 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:10:58.629981 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 77988.2us
W20260709 12:10:58.630784 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 223754us
I20260709 12:10:58.593323 15612 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:58.640571 12572 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:58.642287 12494 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:58.643507 12572 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:10:58.647881 15864 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.195s	user 0.002s	sys 0.000s
W20260709 12:10:58.648978 15861 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:10:58.654615 15855 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:58.661949 15672 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.668936 15672 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:58.630748 15618 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:10:58.674657 12527 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:58.630450 15845 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:10:58.594744 15720 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.632083 15839 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:10:58.683048 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50643.6us
W20260709 12:10:58.681418 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 273393us
I20260709 12:10:58.685036 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:58.664690 15727 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:58.690584 15727 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:58.691169 15727 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:58.699831 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:58.701431 12571 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:58.698840 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 100276us
I20260709 12:10:58.688225 12572 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:58.702310 15720 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:58.702430 15732 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:58.701574 15633 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:58.689507 12643 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:58.716830 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:58.712864 12571 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:58.681154 15613 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:58.729971 15733 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:58.730036 15613 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.730629 15733 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:58.731047 15733 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:58.732504 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:58.734933 12571 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:58.725180 12601 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:58.725008 15839 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.725216 15824 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:10:58.726330 15725 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:10:58.727035 15865 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.237s	user 0.001s	sys 0.003s
I20260709 12:10:58.727850 12605 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:58.744982 15726 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:10:58.745573 15726 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:58.728607 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 44294.4us
I20260709 12:10:58.745997 15726 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:58.746766 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:58.747176 15855 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:10:58.729334 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:58.747851 15855 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.748719 15819 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:58.749577 15819 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:58.750514 15874 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:10:58.751324 15874 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:58.752408 15874 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.756650 12571 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:58.758018 12599 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260709 12:10:58.759321 15875 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:58.710366 15199 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:58.732566 15613 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:58.698588 15511 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:58.761961 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:58.763769 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:58.668464 15806 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.746840 12494 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:10:58.714563 15732 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:58.781550 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:58.781818 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.769667 15720 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:58.786779 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:58.787462 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:58.788370 13334 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:10:58.784711 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:58.788806 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:58.780514 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:58.790740 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 44001.8us
W20260709 12:10:58.793537 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:58.796073 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:58.779027 15820 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:58.827389 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 118692us
I20260709 12:10:58.830817 12496 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:58.788856 15732 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:58.780619 15199 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:10:58.778617 15875 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:10:58.842957 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:58.844436 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:58.786844 15720 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:58.798915 15819 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:10:58.832206 15840 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:58.851665 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:58.781062 15511 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.798578 15613 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:10:58.853108 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 65716.4us
I20260709 12:10:58.857043 15461 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:10:58.848979 15839 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:58.866878 15511 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:58.867008 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:58.888824 15511 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:58.889634 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:58.891024 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:58.891359 15855 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:58.895274 15806 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:58.838318 15820 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:58.899147 13338 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:10:58.900211 15820 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:58.901042 15727 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:58.903842 12529 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:10:58.904220 12529 pending_rounds.cc:85] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 4
I20260709 12:10:58.904430 12529 pending_rounds.cc:107] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:10:58.904639 12529 raft_consensus.cc:2889] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:10:58.906349 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:58.867307 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:58.878762 15718 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:58.911774 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:58.905632 15461 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:58.915881 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:58.799353 15633 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.058s
I20260709 12:10:58.852363 15720 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:58.889367 15511 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.914635 15461 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:10:58.956008 12680 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:58.957254 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:58.957509 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:58.917018 15661 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.924211 13338 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 8: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:10:58.924705 15820 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:58.916755 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:58.911765 15517 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.870766 15199 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.931982 15718 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:58.932723 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:58.933997 15699 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:10:58.941943 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:58.986632 12672 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:10:58.944331 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:58.987324 12671 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:58.945308 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:10:58.946740 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 116032us
I20260709 12:10:58.943113 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 1886) took 3865 ms. Trace:
I20260709 12:10:58.940882 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:58.988224 12673 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:10:58.989418 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:58.952852 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:58.954612 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:58.957875 12680 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:58.988008 12600 rpcz_store.cc:276] 0709 12:10:55.069218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:55.146713 (+ 77495us) service_pool.cc:224] Handling call
0709 12:10:58.213486 (+3066773us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:58.296955 (+ 83469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:58.933611 (+636656us) spinlock_profiling.cc:243] Waited 3.01 s on lock 0x7b54002e63d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:58.934035 (+   424us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6930524544}
I20260709 12:10:58.922407 15838 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:10:58.960058 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:58.964700 12679 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 6: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:58.965315 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:58.913452 15727 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:58.968461 15726 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.970825 15511 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:58.974264 15517 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:58.994244 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:58.994544 12677 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
W20260709 12:10:58.994902 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 62271.1us
I20260709 12:10:58.996044 12643 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:58.997437 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:58.997592 15699 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:10:58.974601 15885 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:58.852180 15875 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:58.977946 15633 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:10:58.905575 15733 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.989651 12571 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:58.995162 12674 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:58.998536 12606 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:58.959028 15717 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:58.999881 12673 raft_consensus.cc:2410] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:10:59.001060 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:59.005169 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:59.007066 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:59.007727 12680 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:10:58.939405 15613 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:59.010761 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:59.011178 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:59.011922 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:58.998631 15819 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:59.021214 12602 raft_consensus.cc:2368] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7.
I20260709 12:10:59.041209 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1909) took 1108 ms. Trace:
W20260709 12:10:59.043339 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 45095.3us
I20260709 12:10:59.043906 12677 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:59.041658 12597 rpcz_store.cc:276] 0709 12:10:57.910283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:57.919403 (+  9120us) service_pool.cc:224] Handling call
0709 12:10:59.014925 (+1095522us) spinlock_profiling.cc:243] Waited 158 ms on lock 0x7b54002706d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:10:59.015092 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":364255488}
W20260709 12:10:59.044682 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 47346.6us
I20260709 12:10:59.046445 15822 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:10:59.046929 15822 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:59.047358 15822 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:59.046466 15727 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:59.020529 15717 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:58.998437 15718 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:59.062849 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:59.063947 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:59.068429 12671 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:10:59.068471 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1816) took 1013 ms. Trace:
I20260709 12:10:59.045750 12571 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:59.075807 12674 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:10:59.077939 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:59.072021 12527 rpcz_store.cc:276] 0709 12:10:58.055580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:58.055704 (+   124us) service_pool.cc:224] Handling call
0709 12:10:58.056057 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:58.680386 (+624329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:59.068221 (+387835us) spinlock_profiling.cc:243] Waited 618 ms on lock 0x7b54003c0d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:59.068408 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1422213888}
I20260709 12:10:59.080407 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:59.075910 12571 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:58.986877 15820 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:59.084769 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:59.085197 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:59.083537 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:59.086167 12672 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:10:59.086268 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:59.087488 13332 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:10:59.088222 12599 raft_consensus.cc:2410] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:10:59.013996 15517 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:59.087671 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:59.094218 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:59.093114 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:10:59.094470 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50340.3us
I20260709 12:10:59.096220 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:59.098278 15873 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:59.099013 12569 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:59.099380 15875 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:59.100051 12676 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:59.100137 15883 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:10:59.100591 15883 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:10:59.081761 15633 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:59.062866 15718 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:59.101001 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:10:59.102532 15633 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:59.093312 15594 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:59.103077 15633 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:10:59.103263 15594 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:59.104128 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:59.104951 12674 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 7: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:10:59.106323 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 248) took 1119 ms. Trace:
I20260709 12:10:59.100947 15883 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:10:59.112386 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:59.112639 12672 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:10:59.116329 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:59.089480 15838 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:10:59.091840 15603 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:59.101332 15845 raft_consensus.cc:2764] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260709 12:10:59.124544 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:59.106722 12598 rpcz_store.cc:276] 0709 12:10:57.987413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:57.988086 (+   673us) service_pool.cc:224] Handling call
0709 12:10:58.030576 (+ 42490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:58.200271 (+169695us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:10:58.239497 (+ 39226us) log.cc:844] Serialized 14 byte log entry
0709 12:10:58.680485 (+440988us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:58.680555 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:10:58.680625 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:59.106059 (+425434us) spinlock_profiling.cc:243] Waited 128 ms on lock 0x7b5400115dd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:59.106104 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:59.106238 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":162360,"prepare.queue_time_us":26543,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":2696,"raft.queue_time_us":178838,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":31657,"spinlock_wait_cycles":294083456,"thread_start_us":252281,"threads_started":2,"wal-append.queue_time_us":314976}
I20260709 12:10:59.093003 15517 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:59.128312 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:59.113603 15875 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:10:59.098315 15522 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:59.117537 15873 raft_consensus.cc:697] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:59.101514 15874 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:10:59.146466 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 100753us
I20260709 12:10:59.170061 13302 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:59.016731 15885 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:59.168625 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:59.187369 12643 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:59.190742 12672 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:59.204876 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:59.121748 15732 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:59.133669 15720 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:59.258500 12676 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:10:59.257413 12597 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:10:59.180323 15843 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:59.259407 12674 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
W20260709 12:10:59.262697 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 108657us
I20260709 12:10:59.296422 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:59.302989 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 132974us
I20260709 12:10:59.267549 12672 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:10:59.268474 15845 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:10:59.288375 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 101113us
I20260709 12:10:59.279250 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:59.260834 15624 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:10:59.304500 15843 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:59.315032 15461 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:59.321671 12603 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:59.356405 12531 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:10:59.194567 15634 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 8.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.101s
I20260709 12:10:59.307571 15661 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:59.332320 12643 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:59.319185 15845 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:59.358286 15845 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:59.358884 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:10:59.333423 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:10:59.361496 12599 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:59.362648 15613 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:59.305092 15594 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:59.363678 15594 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:10:59.275904 15517 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:59.341113 15883 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:59.351759 15885 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:59.352494 15873 consensus_queue.cc:237] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:59.398787 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:59.313818 15633 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:59.467777 12526 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
W20260709 12:10:59.469144 13304 leader_election.cc:336] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.740s (SENT)
I20260709 12:10:59.452139 15725 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:10:59.471555 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 175230us
I20260709 12:10:59.447150 15876 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:10:59.476552 12572 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:10:59.492100 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:59.491973 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:59.494252 12572 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:10:59.396256 15819 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260709 12:10:59.499639 15819 raft_consensus.cc:3055] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:10:59.500099 15819 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:59.500587 15843 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:59.501134 15819 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:59.489369 12599 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:10:59.510576 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:59.454859 15517 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:59.529789 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:10:59.489758 15699 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:59.534780 12598 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
I20260709 12:10:59.538672 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1814) took 1553 ms. Trace:
I20260709 12:10:59.538725 15511 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:10:59.540313 15511 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:59.539166 12523 rpcz_store.cc:276] 0709 12:10:57.985679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:57.986395 (+   716us) service_pool.cc:224] Handling call
0709 12:10:58.002232 (+ 15837us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:58.540699 (+538467us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:59.538380 (+997681us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b54000eef54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:10:59.538572 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1127918848}
I20260709 12:10:59.533522 15822 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:59.547561 15891 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:10:59.517028 15883 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:59.574667 15883 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:10:59.575668 15883 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:59.576726 15883 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:59.578433 15900 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.014s
I20260709 12:10:59.516790 15896 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:59.541190 15885 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:10:59.564213 15818 consensus_queue.cc:1059] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260709 12:10:59.568490 15822 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:59.570309 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:10:59.556258 15603 raft_consensus.cc:697] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:59.611142 12598 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
W20260709 12:10:59.595654 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 235085us
I20260709 12:10:59.553344 15634 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.062s
I20260709 12:10:59.553642 15777 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.060s
W20260709 12:10:59.648231 12677 consensus_meta.cc:287] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.572s	user 0.010s	sys 0.002s
I20260709 12:10:59.649201 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:10:59.608896 15894 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:10:59.652841 12677 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:10:59.658987 12675 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:10:59.659837 15843 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:10:59.661279 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:59.662189 12677 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:10:59.595531 15875 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:59.681820 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:59.683723 12643 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:10:59.618821 15633 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:59.685413 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:59.653038 15822 raft_consensus.cc:3055] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:10:59.695182 12677 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: []
I20260709 12:10:59.706601 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:10:59.712316 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:10:59.713169 13332 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:10:59.596390 15896 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:10:59.703148 15874 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:59.701251 15822 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:10:59.712756 15901 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:10:59.735353 13304 leader_election.cc:336] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.834s (SENT)
I20260709 12:10:59.774888 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:59.735989 12677 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:10:59.739523 13335 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
I20260709 12:10:59.746364 13332 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:10:59.754499 15875 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:59.723412 15818 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:10:59.729283 15603 consensus_queue.cc:237] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:59.706524 15843 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.064s
I20260709 12:10:59.764691 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:59.780887 12571 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:10:59.771870 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:10:59.794209 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58938.1us
I20260709 12:10:59.788864 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:59.805929 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1913) took 1031 ms. Trace:
I20260709 12:10:59.807349 12600 rpcz_store.cc:276] 0709 12:10:58.774707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:59.771483 (+996776us) service_pool.cc:224] Handling call
0709 12:10:59.805860 (+ 34377us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:10:59.808334 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2038) took 1087 ms. Trace:
I20260709 12:10:59.808617 12677 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:10:59.803217 15624 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:59.720293 15845 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:10:59.814968 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:59.811498 12569 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:10:59.813784 12679 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:10:59.780977 15461 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:10:59.807307 15909 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:59.761089 15876 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:59.775441 15819 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:10:59.776163 15511 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:59.808805 12604 rpcz_store.cc:276] 0709 12:10:58.721437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:59.475361 (+753924us) service_pool.cc:224] Handling call
0709 12:10:59.503452 (+ 28091us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:59.503714 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:10:59.504080 (+   366us) log.cc:844] Serialized 14 byte log entry
0709 12:10:59.782985 (+278905us) log.cc:844] Serialized 14 byte log entry
0709 12:10:59.792568 (+  9583us) log.cc:844] Serialized 14 byte log entry
0709 12:10:59.807973 (+ 15405us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:59.808023 (+    50us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:59.808075 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:59.808150 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:59.808279 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23920,"prepare.queue_time_us":2620,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":26,"raft.queue_time_us":16775,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":12785,"spinlock_wait_cycles":30286592,"thread_start_us":272050,"threads_started":1}
I20260709 12:10:59.782685 15718 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:10:59.774663 15901 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:59.813490 15905 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:10:59.767179 15874 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:59.886741 12677 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:10:59.878583 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:59.891311 15726 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:59.891870 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:59.893314 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:10:59.864182 15909 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:10:59.861582 15912 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:10:59.896986 15905 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:10:59.914176 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:59.915933 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:10:59.866621 15822 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:59.907274 15726 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:10:59.933939 12674 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:10:59.908082 15912 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:10:59.955950 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 148043us
I20260709 12:10:59.934854 15587 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:10:59.946520 15845 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:10:59.944976 15726 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:10:59.971248 15726 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:59.971493 15720 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:10:59.973217 15717 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:10:59.974954 15915 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
W20260709 12:10:59.980226 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 106168us
W20260709 12:10:59.987211 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:10:59.989711 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:10:59.990948 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 307325us
I20260709 12:10:59.996846 15912 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:00.002437 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:00.003579 15924 consensus_queue.cc:1059] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:10:59.955675 15917 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.055s
I20260709 12:11:00.025276 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:00.014959 15199 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:11:00.055835 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 53512.5us
I20260709 12:11:00.023170 15845 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:10:59.992146 15777 consensus_queue.cc:1059] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:11:00.004786 15924 consensus_queue.cc:1059] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.030s
I20260709 12:11:00.035440 15921 raft_consensus.cc:3055] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:11:00.062592 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:00.041927 15199 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:00.055189 15773 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:10:59.968235 15909 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:00.073750 15916 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:11:00.074635 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1911) took 1301 ms. Trace:
I20260709 12:11:00.088418 12597 rpcz_store.cc:276] 0709 12:10:58.773284 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:59.062544 (+289260us) service_pool.cc:224] Handling call
0709 12:11:00.074420 (+1011876us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b5400375d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:00.074568 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":393428864}
I20260709 12:11:00.066375 15901 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:00.087242 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:00.068388 15921 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:11:00.080668 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 122725us
I20260709 12:11:00.095633 12643 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:00.083869 15603 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:00.121788 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:00.087327 15920 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:00.066713 15594 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260709 12:11:00.024449 15905 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:00.092227 15916 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:11:00.130976 15922 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 12:11:00.131603 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:00.067528 15918 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.063s
I20260709 12:11:00.095520 15845 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:00.143107 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1825) took 1121 ms. Trace:
I20260709 12:11:00.127930 15603 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:00.087164 15773 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:00.127483 15838 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:00.171828 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:00.195967 13335 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:00.170847 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:00.148330 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:00.169087 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:00.153674 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:00.167695 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:00.182967 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:00.154141 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:00.195510 12531 rpcz_store.cc:276] 0709 12:10:59.022391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:59.023014 (+   623us) service_pool.cc:224] Handling call
0709 12:10:59.023480 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:59.356973 (+333493us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:00.142861 (+785888us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b540027f6d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:00.143043 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":765019520}
I20260709 12:11:00.208387 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:00.210912 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:00.211715 12598 raft_consensus.cc:2393] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:00.212277 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:00.213361 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:00.213955 12523 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:00.168519 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:00.213521 12572 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:00.125717 15920 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:00.219921 12529 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:00.146401 15733 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:00.223258 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:00.144362 15925 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:11:00.201193 15916 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:00.138370 15843 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:00.236068 12531 raft_consensus.cc:2393] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 7: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:00.223572 15934 raft_consensus.cc:3055] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:00.238808 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:00.251673 13332 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:11:00.226526 15918 raft_consensus.cc:3055] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:11:00.253006 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:00.248633 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 227787us
I20260709 12:11:00.249281 15938 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:00.204012 15838 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:00.235787 15920 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:00.240953 15884 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:00.252959 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:00.266449 12604 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:00.253949 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:00.271723 12598 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:00.272809 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:00.238372 15925 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:00.274868 15925 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:00.275962 15925 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:11:00.273954 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:00.278673 12598 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:00.282984 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:00.283663 12604 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:00.251435 15934 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:00.183338 15199 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.3, Last appended by leader: 3, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:00.254786 15918 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:00.266908 15838 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:00.325692 15918 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:00.327381 15918 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:00.154935 15921 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:00.331377 15921 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:11:00.331836 15773 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:00.267989 15938 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:00.336628 15938 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:00.340649 15929 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:00.354276 12604 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:00.262324 15845 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.059s
I20260709 12:11:00.361615 15898 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:00.362324 15898 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:00.361009 12604 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:00.361766 15845 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:00.364545 15845 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:00.365221 15898 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:00.365574 15845 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:00.367013 15845 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:00.358995 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2034) took 1638 ms. Trace:
I20260709 12:11:00.369580 12605 rpcz_store.cc:276] 0709 12:10:58.720138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:58.745399 (+ 25261us) service_pool.cc:224] Handling call
0709 12:10:58.745821 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:58.866393 (+120572us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:58.866425 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:58.866471 (+    46us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:10:58.866506 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:00.356785 (+1490279us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54003110d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:00.356858 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:00.358895 (+  2037us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":276737920}
W20260709 12:11:00.385309 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:00.386453 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 116744us
I20260709 12:11:00.389585 12523 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:00.396275 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2070) took 1280 ms. Trace:
I20260709 12:11:00.396759 12675 rpcz_store.cc:276] 0709 12:10:59.109202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:59.109371 (+   169us) service_pool.cc:224] Handling call
0709 12:10:59.658731 (+549360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:59.659514 (+   783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:00.389694 (+730180us) spinlock_profiling.cc:243] Waited 535 ms on lock 0x7b540016eed0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:00.389858 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1229565312}
I20260709 12:11:00.397454 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:00.398089 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:00.398694 12675 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:00.386030 15819 consensus_meta.cc:287] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.518s	user 0.012s	sys 0.000s
I20260709 12:11:00.399837 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1755) took 2584 ms. Trace:
I20260709 12:11:00.398105 12605 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:00.407116 12676 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:00.406442 15905 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:00.413349 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:00.416198 12643 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:00.403064 12678 rpcz_store.cc:276] 0709 12:10:57.806263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:57.811624 (+  5361us) service_pool.cc:224] Handling call
0709 12:10:57.884770 (+ 73146us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:58.042090 (+157320us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:10:58.092277 (+ 50187us) log.cc:844] Serialized 14 byte log entry
0709 12:10:58.215689 (+123412us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:58.255743 (+ 40054us) spinlock_profiling.cc:243] Waited 2.63 ms on lock 0x7b4800060858. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:10:58.307145 (+ 51402us) log.cc:844] Serialized 52 byte log entry
0709 12:10:58.307321 (+   176us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:10:58.581089 (+273768us) log.cc:844] Serialized 14 byte log entry
0709 12:10:58.593878 (+ 12789us) log.cc:844] Serialized 14 byte log entry
0709 12:10:58.594109 (+   231us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:58.594307 (+   198us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:10:58.594348 (+    41us) raft_consensus.cc:1685] finished
0709 12:10:58.594360 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:58.667697 (+ 73337us) spinlock_profiling.cc:243] Waited 3.6 ms on lock 0x7b1800076060. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:10:59.382573 (+714876us) spinlock_profiling.cc:243] Waited 30.3 ms on lock 0x7b1800076060. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4
0709 12:10:59.758403 (+375830us) spinlock_profiling.cc:243] Waited 3.6 ms on lock 0x7b1800076060. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:11:00.390195 (+631792us) spinlock_profiling.cc:243] Waited 41.7 ms on lock 0x7b540036d150. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:00.390305 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38341,"prepare.queue_time_us":294814,"prepare.run_cpu_time_us":150,"prepare.run_wall_time_us":159,"raft.queue_time_us":86447,"raft.run_cpu_time_us":3014,"raft.run_wall_time_us":44000,"spinlock_wait_cycles":188075648,"thread_start_us":512803,"threads_started":3,"wal-append.queue_time_us":237396,"wal-append.run_cpu_time_us":4395,"wal-append.run_wall_time_us":1285023}
I20260709 12:11:00.416339 15905 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:00.399663 15934 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:00.420648 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:00.445051 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:00.431756 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:00.448725 12598 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:00.448262 12643 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:00.451543 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:00.454800 12675 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
I20260709 12:11:00.454226 15594 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:00.431654 15934 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:00.438479 15905 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:00.429064 15907 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:11:00.434508 15822 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:00.492220 15885 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:00.493047 15885 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:00.497476 13304 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:00.500404 15898 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:00.502453 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2029) took 1784 ms. Trace:
I20260709 12:11:00.502884 12606 rpcz_store.cc:276] 0709 12:10:58.718291 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:58.730209 (+ 11918us) service_pool.cc:224] Handling call
0709 12:10:58.730558 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:59.008777 (+278219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:00.502201 (+1493424us) spinlock_profiling.cc:243] Waited 268 ms on lock 0x7b5400079c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:00.502389 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":615581568}
I20260709 12:11:00.503835 15820 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:00.508062 15324 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:00.518153 15885 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:00.451129 15819 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:00.460194 15845 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:00.455930 15938 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:00.458300 15909 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:00.527609 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2032) took 1808 ms. Trace:
I20260709 12:11:00.528163 12601 rpcz_store.cc:276] 0709 12:10:58.719438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:58.771273 (+ 51835us) service_pool.cc:224] Handling call
0709 12:10:59.015313 (+244040us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:59.038749 (+ 23436us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:10:59.127853 (+ 89104us) log.cc:844] Serialized 14 byte log entry
0709 12:10:59.411408 (+283555us) log.cc:844] Serialized 14 byte log entry
0709 12:10:59.470175 (+ 58767us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:10:59.470229 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:10:59.470289 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:59.470382 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:10:59.760838 (+290456us) spinlock_profiling.cc:243] Waited 5.74 ms on lock 0x7b480024c258. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:11:00.508078 (+747240us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b18003b0980. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485933522 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:11:00.527387 (+ 19309us) spinlock_profiling.cc:243] Waited 231 ms on lock 0x7b54002706d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:00.527527 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":146175,"prepare.queue_time_us":86835,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":122,"raft.queue_time_us":208328,"raft.run_cpu_time_us":3397,"raft.run_wall_time_us":298529,"spinlock_wait_cycles":585029632,"thread_start_us":271967,"threads_started":2,"wal-append.queue_time_us":218411,"wal-append.run_cpu_time_us":2320,"wal-append.run_wall_time_us":1157722}
I20260709 12:11:00.470063 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:00.470825 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:00.533001 12531 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 8: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:11:00.466595 15904 log.cc:927] Time spent T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.704s	user 0.001s	sys 0.003s
W20260709 12:11:00.470580 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 59473.3us
I20260709 12:11:00.538900 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2071) took 1429 ms. Trace:
I20260709 12:11:00.539742 15587 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:00.540437 15587 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:00.541431 15732 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:00.550709 12678 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:00.551226 15587 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:00.551389 15920 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:00.552495 12678 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:00.553663 15587 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:00.554976 12677 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:00.503852 15916 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:00.478348 15935 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:00.477377 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:00.481195 15838 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:00.485507 15923 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:00.486006 12602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1912) took 1712 ms (client timeout 1611 ms). Trace:
I20260709 12:11:00.487396 15699 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:00.578855 15699 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:00.579018 15903 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.817s	user 0.000s	sys 0.008s
I20260709 12:11:00.579510 12601 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:11:00.524184 15777 raft_consensus.cc:3055] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:00.527354 15909 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:00.592185 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2072) took 1480 ms. Trace:
I20260709 12:11:00.592610 12572 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:00.436975 15603 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:00.559556 12672 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260709 12:11:00.481518 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2036) took 1760 ms. Trace:
W20260709 12:11:00.458956 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 51522.4us
W20260709 12:11:00.561347 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 110046us
I20260709 12:11:00.539422 12673 rpcz_store.cc:276] 0709 12:10:59.109833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:59.109985 (+   152us) service_pool.cc:224] Handling call
0709 12:10:59.110525 (+   540us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:59.119354 (+  8829us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:59.119381 (+    27us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:59.150066 (+ 30685us) log.cc:844] Serialized 52 byte log entry
0709 12:10:59.508423 (+358357us) raft_consensus.cc:1645] Marking committed up to 1
0709 12:10:59.508487 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:59.508585 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:00.441377 (+932792us) spinlock_profiling.cc:243] Waited 39.3 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:00.538468 (+ 97091us) raft_consensus.cc:1685] finished
0709 12:11:00.538494 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:00.538815 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11333,"raft.run_cpu_time_us":221,"raft.run_wall_time_us":44752,"spinlock_wait_cycles":90305280,"thread_start_us":199980,"threads_started":1,"wal-append.queue_time_us":546197}
I20260709 12:11:00.564656 15896 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:00.574550 12602 rpcz_store.cc:269] 0709 12:10:58.773848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:58.782780 (+  8932us) service_pool.cc:224] Handling call
0709 12:11:00.485720 (+1702940us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b540023fd54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:00.485912 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":515853568}
I20260709 12:11:00.550746 12675 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
W20260709 12:11:00.590021 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:00.595963 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:00.595407 12603 rpcz_store.cc:276] 0709 12:10:58.720819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:58.749829 (+ 29010us) service_pool.cc:224] Handling call
0709 12:10:58.750268 (+   439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:59.355914 (+605646us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:00.481109 (+1125195us) spinlock_profiling.cc:243] Waited 571 ms on lock 0x7b5400376254. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:00.481418 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1313590912}
I20260709 12:11:00.602274 12572 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:00.602676 12673 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: [7.6-8.7]
I20260709 12:11:00.603747 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:00.604552 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:00.603886 12677 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:00.592867 12680 rpcz_store.cc:276] 0709 12:10:59.111974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:59.112186 (+   212us) service_pool.cc:224] Handling call
0709 12:10:59.112681 (+   495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:10:59.168078 (+ 55397us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:10:59.219135 (+ 51057us) log.cc:844] Serialized 14 byte log entry
0709 12:10:59.580314 (+361179us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:10:59.619114 (+ 38800us) log.cc:844] Serialized 26 byte log entry
0709 12:10:59.619265 (+   151us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:10:59.619306 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:10:59.619407 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:00.572574 (+953167us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:00.582019 (+  9445us) raft_consensus.cc:1685] finished
0709 12:11:00.591738 (+  9719us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:00.592105 (+   367us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34701,"prepare.queue_time_us":63491,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":9298,"raft.queue_time_us":42449,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":23418,"spinlock_wait_cycles":48735488,"thread_start_us":250128,"threads_started":1,"wal-append.queue_time_us":404234}
W20260709 12:11:00.606925 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:00.607079 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:00.609090 12531 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: []
I20260709 12:11:00.615029 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:00.616971 12643 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:00.618268 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:00.618435 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:00.611405 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:00.620577 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:00.621042 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:00.611901 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:00.588562 15594 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:00.623744 15905 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:00.624213 15912 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:11:00.624498 15949 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:00.625236 12643 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:00.625388 15907 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:00.644094 15907 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:00.614475 15511 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:11:00.627326 15845 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:00.614743 15897 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.013s
I20260709 12:11:00.610414 15913 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:11:00.628737 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:00.589480 15777 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:00.589641 15820 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:00.588922 15923 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:00.622788 12531 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:11:00.610097 15822 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:00.612825 15603 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:00.633368 15905 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:00.616385 15587 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:11:00.641588 15920 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:00.643244 15838 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:00.643630 15940 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.219s	user 0.002s	sys 0.000s
I20260709 12:11:00.625767 15819 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:00.595520 15942 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.040s
W20260709 12:11:00.673816 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:00.640235 15949 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:00.634400 15618 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:00.613296 15931 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
W20260709 12:11:00.754683 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 137707us
I20260709 12:11:00.699146 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:00.623153 15921 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
W20260709 12:11:00.708655 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 83507.8us
I20260709 12:11:00.697491 12572 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:00.765372 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 68163us
I20260709 12:11:00.685237 15907 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:00.713850 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 102532us
I20260709 12:11:00.685752 15594 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:11:00.768514 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 67395.9us
I20260709 12:11:00.774906 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:00.749841 15822 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:00.776168 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 91570.1us
I20260709 12:11:00.706094 15820 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:00.752619 15890 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:00.791110 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:00.788882 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:00.787457 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:00.793186 12601 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:11:00.751261 15949 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:00.766932 15603 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:11:00.794955 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:00.795790 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:00.803277 12604 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:11:00.790408 13304 leader_election.cc:336] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.071s (SENT)
I20260709 12:11:00.804086 13304 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:00.798913 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:00.794828 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:00.770364 15845 raft_consensus.cc:697] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:00.806258 15845 consensus_queue.cc:237] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:00.810213 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:00.770747 15819 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:00.820292 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:00.826680 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:00.827610 12678 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:11:00.832974 12643 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:00.824070 15925 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:11:00.802639 15603 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:00.800199 15594 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:00.812804 15838 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:00.851450 12494 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:00.854648 15612 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:00.855490 12494 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:00.855363 15612 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:00.856992 15918 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:00.847535 15957 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:00.858330 15936 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:11:00.859040 15773 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:00.857776 15918 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:11:00.859362 12571 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:00.860718 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:00.861146 15929 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:00.861613 15929 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:00.862424 15897 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260709 12:11:00.862807 15931 consensus_queue.cc:1059] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:11:00.863086 15897 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:11:00.863586 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:00.863965 15942 raft_consensus.cc:3055] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:11:00.864367 15942 raft_consensus.cc:740] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:00.857954 15612 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:00.864801 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:00.865301 15942 consensus_queue.cc:260] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:00.869086 15951 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:00.870621 15934 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:00.871021 15934 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:00.871958 15934 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:00.847381 15618 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:00.875739 15820 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:00.876160 15918 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:00.795277 15822 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:00.878484 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:00.883318 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:00.865424 12673 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:11:00.883987 12678 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:00.880659 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:00.885372 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:00.830051 15920 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:00.834960 15725 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:00.778007 15777 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:00.851837 15603 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:00.915805 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:00.920473 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 100313us
W20260709 12:11:00.922263 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 89456us
I20260709 12:11:00.924686 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2066) took 1825 ms. Trace:
I20260709 12:11:00.800361 15890 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:00.936059 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:00.932672 12678 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:00.937690 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:00.938380 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:00.939436 12571 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:00.825598 15958 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.096s
I20260709 12:11:00.934062 12671 rpcz_store.cc:276] 0709 12:10:59.099020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:59.102208 (+  3188us) service_pool.cc:224] Handling call
0709 12:10:59.102539 (+   331us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:10:59.106093 (+  3554us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:10:59.106154 (+    61us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:10:59.106821 (+   667us) log.cc:844] Serialized 52 byte log entry
0709 12:11:00.687866 (+1581045us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b18000c98e0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:11:00.919083 (+231217us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:11:00.919137 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:00.919237 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:00.924336 (+  5099us) raft_consensus.cc:1685] finished
0709 12:11:00.924356 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:00.924610 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34879,"raft.queue_time_us":3443,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":2317,"spinlock_wait_cycles":4807424,"thread_start_us":230946,"threads_started":1,"wal-append.queue_time_us":1787052}
I20260709 12:11:00.939033 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:00.942655 15612 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:00.911228 15909 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:00.887203 15920 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:00.910982 15725 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:00.816802 15905 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:00.927135 15842 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:00.952211 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:00.953378 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:00.929657 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 159952us
I20260709 12:11:00.955072 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:00.961891 12675 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:11:00.962679 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:00.964356 12678 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:00.936586 15916 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:00.965751 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:00.994021 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:01.053170 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:00.983844 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:11:00.950748 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 73586.1us
I20260709 12:11:00.936756 15901 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:00.946774 15777 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:00.950377 15612 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:01.061543 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 77816.7us
I20260709 12:11:00.958802 15873 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:01.072376 12602 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:01.072923 12598 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
I20260709 12:11:00.968461 15909 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:01.073722 15909 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:01.074157 15909 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:01.076617 12598 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:00.956728 15845 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:01.080247 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:00.960057 15929 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:01.082710 12602 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:01.083516 15929 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:00.989703 15594 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.090276 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:01.092231 15976 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:01.092515 15974 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:01.092970 15976 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.093716 15974 raft_consensus.cc:3055] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:01.095057 12602 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: [4.2-6.3]
I20260709 12:11:01.095472 15970 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:01.098413 15970 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.100289 15974 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:01.103204 15974 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:01.097151 15918 consensus_queue.cc:1059] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:01.101612 15973 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:11:01.100841 15970 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:00.961298 15920 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:00.956089 15955 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:01.120981 15955 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.123950 15955 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:01.114329 13338 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
I20260709 12:11:01.124714 13338 pending_rounds.cc:85] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 6
I20260709 12:11:01.125025 13338 pending_rounds.cc:107] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260709 12:11:01.125180 15934 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:01.123816 15845 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:01.126623 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:01.127856 13334 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 11: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:11:01.111106 15324 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.602s	user 0.013s	sys 0.000s
I20260709 12:11:01.128398 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:01.129405 13331 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:01.129840 13339 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:11:01.134569 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:01.135792 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1812) took 1314 ms. Trace:
I20260709 12:11:01.136072 12674 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 6. (term mismatch)
I20260709 12:11:01.136457 12674 pending_rounds.cc:85] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 5
I20260709 12:11:01.136229 12676 rpcz_store.cc:276] 0709 12:10:59.821456 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:59.915588 (+ 94132us) service_pool.cc:224] Handling call
0709 12:11:01.135603 (+1220015us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b54003d40d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:01.135702 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1127593216}
I20260709 12:11:01.136885 12674 pending_rounds.cc:107] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:11:01.137195 12674 raft_consensus.cc:2889] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:01.068802 15612 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:01.137894 15843 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:01.138624 15843 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.141057 15823 raft_consensus.cc:3055] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:01.141625 15823 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:00.981567 15819 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.142930 13331 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:01.137764 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:01.150851 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:01.152467 15199 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:01.152735 12676 raft_consensus.cc:2376] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9. Re-sending same reply.
I20260709 12:11:01.155743 12601 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
I20260709 12:11:01.156641 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:01.158006 15935 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:01.158810 15935 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:01.160004 15935 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.172930 15822 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:01.068497 15901 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:01.174913 15901 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:01.176249 15725 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:01.182816 15699 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.603s	user 0.005s	sys 0.008s
I20260709 12:11:01.183511 15699 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.119830 15964 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:00.968354 15517 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:01.206569 12529 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:11:01.206974 15517 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.209127 15909 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:01.139088 15876 raft_consensus.cc:3055] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:11:01.220496 15876 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:01.221714 15876 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.158085 15958 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:01.223413 15958 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 9 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:01.224649 15958 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:01.118044 15969 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.011s
I20260709 12:11:01.226727 15969 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:01.227214 15969 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:01.228637 15938 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:01.228252 15969 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.229225 15938 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.229763 15969 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:01.231529 15938 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:01.233564 15964 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.236778 15964 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:01.239403 15961 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:01.132562 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:01.240182 15961 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:01.240908 13330 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:01.243496 15961 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:01.247347 15618 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:01.245003 15909 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:01.162058 15935 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:01.254316 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:01.119910 15890 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:01.198486 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:01.125269 13338 raft_consensus.cc:2889] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:01.112608 15898 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:01.215092 15920 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.300753 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:01.301367 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 342359us
I20260709 12:11:01.301560 12676 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:11:01.289669 12673 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:01.217238 15725 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:01.218534 12643 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:01.219305 15612 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:01.109186 15974 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:01.293440 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:01.297528 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 201428us
I20260709 12:11:01.299304 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:01.216291 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1811) took 1401 ms. Trace:
W20260709 12:11:01.306746 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 120650us
I20260709 12:11:01.306805 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:01.308893 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:01.302013 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:01.142815 15823 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:01.415927 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 107652us
I20260709 12:11:01.438064 15594 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:01.439309 13330 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:01.439500 15594 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.316730 15618 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:01.440915 15618 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:01.395617 15822 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:01.442389 15952 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:01.442862 15822 raft_consensus.cc:697] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:11:01.443666 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:01.328670 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:01.319154 12606 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:01.444710 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:01.445802 12526 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 8: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:01.298202 15517 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:01.316242 15883 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.560s	user 0.011s	sys 0.004s
I20260709 12:11:01.447400 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:01.446671 12674 rpcz_store.cc:276] 0709 12:10:59.815277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:59.962386 (+147109us) service_pool.cc:224] Handling call
0709 12:10:59.986716 (+ 24330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:01.137725 (+1151009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:01.215992 (+ 78267us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400013454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:01.216217 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2642349952}
I20260709 12:11:01.453379 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:01.297058 15898 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:01.463379 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:01.465546 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:01.458108 12569 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:01.441994 15594 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:01.443701 15618 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:01.450023 15930 raft_consensus.cc:3055] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:01.491086 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:01.492209 12674 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:01.470026 12676 raft_consensus.cc:2410] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:11:01.472854 15977 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
W20260709 12:11:01.478752 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:01.482707 13330 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 7. Current term is 8. Ops: []
W20260709 12:11:01.486981 15949 consensus_meta.cc:287] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.668s	user 0.014s	sys 0.000s
I20260709 12:11:01.490152 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:11:01.457160 15883 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.701s	user 0.018s	sys 0.055s
I20260709 12:11:01.305378 15838 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:11:01.466666 15964 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:01.508422 15977 raft_consensus.cc:3055] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:01.508682 13330 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:01.508975 15977 raft_consensus.cc:740] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:11:01.509260 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 51268.6us
I20260709 12:11:01.509989 15977 consensus_queue.cc:260] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.511008 15912 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:01.511289 15977 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:01.466233 15898 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:01.512219 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:01.491608 15930 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:01.497167 15819 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:01.499244 15594 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:11:01.528698 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:01.508569 15964 raft_consensus.cc:697] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:01.502472 15699 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:01.531883 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:01.535887 12603 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:11:01.539894 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:01.540740 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:01.541110 12673 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
W20260709 12:11:01.552403 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:01.571066 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:01.557044 12673 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:01.504809 15949 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.491175 15822 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:01.579083 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:01.530565 15594 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.534600 15938 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:01.586422 12674 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:11:01.520860 15905 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:11:01.595490 12676 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260709 12:11:01.601964 15954 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:01.602662 15954 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.603271 15938 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:01.608373 15907 consensus_meta.cc:287] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.830s	user 0.012s	sys 0.001s
I20260709 12:11:01.609151 15907 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.612453 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:01.613094 15612 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:01.616670 12494 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:11:01.617153 15603 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.675s	user 0.011s	sys 0.000s
I20260709 12:11:01.618010 13330 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:01.618541 15931 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:01.619426 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:01.620083 15931 raft_consensus.cc:3055] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:01.619076 15941 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:01.549903 15324 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:01.532687 15587 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:01.621745 12675 raft_consensus.cc:2393] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 9: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:01.621682 13338 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:01.536406 15517 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:01.530853 15819 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.523139 15873 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:11:01.624418 12571 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:01.628501 12672 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:11:01.629242 15324 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:01.626041 15981 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 12:11:01.640760 15324 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:01.643065 15819 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:01.618572 15920 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:01.650370 15942 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:11:01.650367 15901 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.643916 15517 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:01.676086 15901 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:01.651798 12524 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 8 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 12:11:01.632856 15587 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:01.678006 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1816) took 1041 ms. Trace:
I20260709 12:11:01.647050 15982 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 12:11:01.654872 15949 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:01.677565 15819 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:01.637900 15883 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:11:01.677805 15941 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:01.624408 15931 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:01.594815 15964 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:01.681264 15587 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:01.679836 15517 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:01.682297 15982 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:01.691164 15982 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:01.692027 15982 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:01.693217 15982 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:01.682931 15819 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:01.685774 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:01.678419 12524 rpcz_store.cc:276] 0709 12:11:00.637105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:00.637283 (+   178us) service_pool.cc:224] Handling call
0709 12:11:00.676303 (+ 39020us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:01.676868 (+1000565us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:01.677756 (+   888us) spinlock_profiling.cc:243] Waited 975 ms on lock 0x7b540042a4d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:01.677948 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2242643200}
I20260709 12:11:01.702395 12678 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:01.704193 15594 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:01.704360 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:01.705736 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:01.617895 15603 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.686647 12602 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:01.625833 15934 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.053s
I20260709 12:11:01.688690 15912 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:01.760224 12427 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47524 (request call id 95) took 10889 ms. Trace:
I20260709 12:11:01.748517 12672 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 9: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:01.707016 15843 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:01.770201 15958 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:01.772593 12427 rpcz_store.cc:276] 0709 12:10:50.870923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:50.871922 (+   999us) service_pool.cc:224] Handling call
0709 12:11:01.760103 (+10888181us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:10:58.378150 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:10:58.379214 (+  1064us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:10:58.379293 (+    79us) write_op.cc:435] Acquired schema lock
0709 12:10:58.379332 (+    39us) tablet.cc:662] Decoding operations
0709 12:10:58.381316 (+  1984us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:10:58.381470 (+   154us) write_op.cc:641] Partition lock acquired for write op
0709 12:10:58.381517 (+    47us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:10:58.383910 (+  2393us) tablet.cc:701] Row locks acquired
0709 12:10:58.383973 (+    63us) write_op.cc:260] PREPARE: finished
0709 12:10:58.384225 (+   252us) write_op.cc:270] Start()
0709 12:10:58.384443 (+   218us) write_op.cc:276] Timestamp: P: 1783599058384134 usec, L: 0
0709 12:10:58.384475 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 12:10:58.385756 (+  1281us) log.cc:844] Serialized 5578 byte log entry
0709 12:10:58.939222 (+553466us) op_driver.cc:464] REPLICATION: finished
0709 12:10:59.282494 (+343272us) write_op.cc:301] APPLY: starting
0709 12:10:59.314448 (+ 31954us) tablet.cc:1367] starting BulkCheckPresence
0709 12:10:59.314830 (+   382us) tablet.cc:1370] finished BulkCheckPresence
0709 12:10:59.314969 (+   139us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:10:59.493664 (+178695us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:10:59.494174 (+   510us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=14
0709 12:10:59.494232 (+    58us) write_op.cc:312] APPLY: finished
0709 12:10:59.494731 (+   499us) log.cc:844] Serialized 100 byte log entry
0709 12:10:59.494961 (+   230us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:59.495127 (+   166us) write_op.cc:454] Released schema lock
0709 12:10:59.495217 (+    90us) write_op.cc:341] FINISH: Updating metrics
0709 12:10:59.495766 (+   549us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:10:59.495799 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":5728264,"reported_tablets":22,"child_traces":[["op",{"apply.queue_time_us":236099,"apply.run_cpu_time_us":6585,"apply.run_wall_time_us":263320,"num_ops":14,"prepare.queue_time_us":121567,"prepare.run_cpu_time_us":17662,"prepare.run_wall_time_us":61274,"raft.queue_time_us":194750,"raft.run_cpu_time_us":84887,"raft.run_wall_time_us":249192,"replication_time_us":493793,"thread_start_us":352403,"threads_started":4,"wal-append.queue_time_us":66031,"wal-append.run_cpu_time_us":43290,"wal-append.run_wall_time_us":2090239}]]}
I20260709 12:11:01.775211 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2090) took 2163 ms. Trace:
I20260709 12:11:01.786273 12679 rpcz_store.cc:276] 0709 12:10:59.611792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:59.740932 (+129140us) service_pool.cc:224] Handling call
0709 12:10:59.741272 (+   340us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:10:59.817728 (+ 76456us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:01.774899 (+1957171us) spinlock_profiling.cc:243] Waited 71 ms on lock 0x7b540015dbd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:01.775136 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":163382528}
I20260709 12:11:01.787683 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:01.788151 15823 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:01.789121 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:01.789968 12679 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:01.793669 15913 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:01.684136 15941 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.799098 12599 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:01.795886 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1818) took 1105 ms. Trace:
I20260709 12:11:01.794565 12603 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:01.588455 15930 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.811619 15930 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:01.813992 12674 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 8 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
W20260709 12:11:01.734108 15947 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 1.095s	user 0.008s	sys 0.000s
I20260709 12:11:01.761791 15926 raft_consensus.cc:3055] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:01.799188 12678 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:01.785317 15958 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:01.799772 12527 rpcz_store.cc:276] 0709 12:11:00.690108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:00.690379 (+   271us) service_pool.cc:224] Handling call
0709 12:11:00.690900 (+   521us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:00.703173 (+ 12273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:00.703225 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:00.749421 (+ 46196us) log.cc:844] Serialized 52 byte log entry
0709 12:11:00.830965 (+ 81544us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:00.877804 (+ 46839us) log.cc:844] Serialized 14 byte log entry
0709 12:11:01.071070 (+193266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:01.071168 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:01.615030 (+543862us) spinlock_profiling.cc:243] Waited 9.82 ms on lock 0x7b18002d48a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485933522 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:11:01.783313 (+168283us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b18002d48a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:11:01.795657 (+ 12344us) raft_consensus.cc:1685] finished
0709 12:11:01.795682 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:01.795818 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2024,"prepare.queue_time_us":50887,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":65122,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":21743,"spinlock_wait_cycles":28720384,"thread_start_us":235013,"threads_started":2,"wal-append.queue_time_us":172392}
I20260709 12:11:01.818885 15926 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:01.820273 15941 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:01.687860 15931 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:01.822907 15958 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:01.688091 15923 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:01.774907 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:01.833979 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:01.834640 12672 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:01.835379 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:01.799405 15909 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:01.780000 15948 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.159s	user 0.008s	sys 0.000s
I20260709 12:11:01.802892 15913 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:01.805946 12605 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:01.807109 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:01.836098 15909 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:01.778460 15603 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:01.802932 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:01.810065 15843 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:01.839188 15843 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:01.843189 15909 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:01.844352 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1821) took 1142 ms. Trace:
I20260709 12:11:01.844763 12675 rpcz_store.cc:276] 0709 12:11:00.701532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:00.965377 (+263845us) service_pool.cc:224] Handling call
0709 12:11:01.829870 (+864493us) spinlock_profiling.cc:243] Waited 653 ms on lock 0x7b5400010c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:01.830125 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1502418688}
I20260709 12:11:01.822454 15728 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:01.846143 12675 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
W20260709 12:11:01.815183 15902 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.788s	user 0.011s	sys 0.008s
I20260709 12:11:01.830857 15907 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:01.835800 12672 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:11:01.825307 12678 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:01.841730 15843 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:01.876495 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 77332.7us
I20260709 12:11:01.877952 13302 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:01.851105 15902 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.012s	sys 0.009s
I20260709 12:11:01.820485 15926 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.915709 12673 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:01.846436 15728 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:01.964313 15822 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:01.892637 15923 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:01.916730 15974 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
I20260709 12:11:02.004619 15898 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:02.007874 15913 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:01.893343 12672 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:11:01.889564 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:01.980469 12675 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
W20260709 12:11:02.011482 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 133660us
I20260709 12:11:05.142692 12671 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:03.902171 15993 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:05.144300 15994 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:11:04.393602 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 2.50104e+06us
I20260709 12:11:04.697579 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2110) took 3893 ms. Trace:
I20260709 12:11:05.145670 12676 rpcz_store.cc:276] 0709 12:11:00.804143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:01.542866 (+738723us) service_pool.cc:224] Handling call
0709 12:11:01.595211 (+ 52345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:01.596021 (+   810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:04.697249 (+3101228us) spinlock_profiling.cc:243] Waited 51.8 ms on lock 0x7b54003d4fd0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:04.697492 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":119115520}
I20260709 12:11:05.144305 15993 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.147709 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2122) took 3627 ms. Trace:
I20260709 12:11:05.148067 12676 rpcz_store.cc:276] 0709 12:11:01.520708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.146845 (+3626137us) service_pool.cc:224] Handling call
0709 12:11:05.147170 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.147349 (+   179us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:05.147387 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:05.147431 (+    44us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:11:05.147471 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.147541 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:05.147651 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:05.148669 15993 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:01.925746 12571 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.117s (SENT) [suppressed 11 similar messages]
I20260709 12:11:04.781057 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2113) took 3976 ms. Trace:
W20260709 12:11:05.149957 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.149832 12673 rpcz_store.cc:276] 0709 12:11:00.804801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:01.572597 (+767796us) service_pool.cc:224] Handling call
0709 12:11:01.572909 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:01.999102 (+426193us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:04.780720 (+2781618us) spinlock_profiling.cc:243] Waited 343 ms on lock 0x7b54002c4f54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:04.780976 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":787828096}
I20260709 12:11:05.151717 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2124) took 3548 ms. Trace:
I20260709 12:11:05.152024 12673 rpcz_store.cc:276] 0709 12:11:01.603407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.150909 (+3547502us) service_pool.cc:224] Handling call
0709 12:11:05.151216 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.151398 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:05.151423 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:05.151466 (+    43us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:05.151514 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.151563 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:05.151662 (+    99us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:05.153160 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2125) took 3529 ms. Trace:
I20260709 12:11:05.153370 12673 rpcz_store.cc:276] 0709 12:11:01.623299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.152609 (+3529310us) service_pool.cc:224] Handling call
0709 12:11:05.152821 (+   212us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.152949 (+   128us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:05.152966 (+    17us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:05.152995 (+    29us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:05.153018 (+    23us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.153051 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:05.153120 (+    69us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:11:04.784806 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:05.154825 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.155382 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1834) took 3522 ms. Trace:
W20260709 12:11:04.814576 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:04.819851 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:04.947921 15995 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:04.950449 15996 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:04.953475 15997 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:04.957329 12643 connection.cc:256] RPC call timeout handler was delayed by -2.521943869000097s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1654s for a response
I20260709 12:11:05.156522 15912 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:04.967427 12572 connection.cc:256] RPC call timeout handler was delayed by -1.8157278529999985s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.191s for a response
W20260709 12:11:05.157701 12572 connection.cc:256] RPC call timeout handler was delayed by -1.8123264870000639s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17410000000000003s for a response
W20260709 12:11:04.971342 12606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1933) took 4195 ms (client timeout 1654 ms). Trace:
I20260709 12:11:04.979415 12600 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
W20260709 12:11:05.158020 12572 connection.cc:256] RPC call timeout handler was delayed by -1.723027853000076s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2013s for a response
I20260709 12:11:04.984581 15999 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:05.158500 16007 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:05.158731 12572 connection.cc:256] RPC call timeout handler was delayed by -1.6818264870000803s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1886s for a response
I20260709 12:11:04.988809 15998 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.159137 16007 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:05.159427 15997 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.159549 15998 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.161751 16007 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:04.994138 16001 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:04.997677 12569 connection.cc:256] RPC call timeout handler was delayed by -1.6825862319999487s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1874s for a response
W20260709 12:11:05.159082 12572 connection.cc:256] RPC call timeout handler was delayed by -1.46222648700018s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.213s for a response
W20260709 12:11:05.162679 12569 connection.cc:256] RPC call timeout handler was delayed by -1.3893547919999492s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.213s for a response
I20260709 12:11:05.162488 16001 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:05.163311 12572 connection.cc:256] RPC call timeout handler was delayed by -1.442426487000148s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21520000000000003s for a response
I20260709 12:11:05.002559 16003 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:05.165071 16001 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.162426 15997 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:05.003058 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2120) took 3607 ms. Trace:
I20260709 12:11:05.166224 12678 rpcz_store.cc:276] 0709 12:11:01.396094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:01.867060 (+470966us) service_pool.cc:224] Handling call
0709 12:11:01.867582 (+   522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:01.867863 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:11:01.887362 (+ 19499us) log.cc:844] Serialized 14 byte log entry
0709 12:11:04.992496 (+3105134us) spinlock_profiling.cc:243] Waited 28 ms on lock 0x7b18002f5420. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a 00007f1488a2aa2a
0709 12:11:05.002387 (+  9891us) log.cc:844] Serialized 14 byte log entry
0709 12:11:05.002614 (+   227us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:05.002671 (+    57us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:05.002742 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.002836 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:05.002983 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":22529,"prepare.run_cpu_time_us":179,"prepare.run_wall_time_us":41149,"raft.queue_time_us":9210,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":262,"spinlock_wait_cycles":65971968,"thread_start_us":8534,"threads_started":3,"wal-append.queue_time_us":3107337}
I20260709 12:11:05.168533 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1835) took 3534 ms. Trace:
I20260709 12:11:05.012776 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1829) took 3710 ms. Trace:
W20260709 12:11:05.022586 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.030529 16004 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:11:05.041179 12603 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.170s	user 0.009s	sys 0.013s
I20260709 12:11:05.050925 12680 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
W20260709 12:11:05.053684 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:04.856150 15895 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:04.962965 15718 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:05.321273 15718 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:05.322989 16023 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:03.887216 15819 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:11:03.011479 15822 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:02.927747 15884 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:04.783550 15890 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.356395 15884 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.357003 15890 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:04.966977 15913 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.359170 15884 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.359061 15913 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.360419 12605 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260709 12:11:05.361300 15913 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:02.812922 15930 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.001s	user 0.010s	sys 0.001s
I20260709 12:11:02.953886 15931 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:05.031008 15938 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:03.074298 15942 consensus_meta.cc:287] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.401s	user 0.011s	sys 0.000s
W20260709 12:11:01.978473 15947 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:11:03.672174 15954 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:03.762630 15970 raft_consensus.cc:3055] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:11:05.371951 15970 raft_consensus.cc:740] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:05.372997 15970 consensus_queue.cc:260] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:03.768438 15976 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:03.069365 15977 consensus_meta.cc:287] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.558s	user 0.009s	sys 0.002s
W20260709 12:11:03.077718 15982 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.384s	user 0.009s	sys 0.003s
W20260709 12:11:03.072770 15986 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 1.109s	user 0.000s	sys 0.001s
W20260709 12:11:05.098754 12602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1934) took 3903 ms (client timeout 1863 ms). Trace:
W20260709 12:11:05.378422 12602 rpcz_store.cc:269] 0709 12:11:01.195198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:01.301778 (+106580us) service_pool.cc:224] Handling call
0709 12:11:05.098319 (+3796541us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b54001946d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:05.098641 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":543936768}
I20260709 12:11:05.379598 12602 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
mode: GRACEFUL
new_leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
 from {username='slave'} at 127.0.0.1:32832
I20260709 12:11:05.380256 12602 raft_consensus.cc:606] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Received request to transfer leadership to TS a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:05.380559 12602 raft_consensus.cc:614] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:05.117520 15876 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:05.119973 15951 raft_consensus.cc:3055] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:05.386397 15951 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:05.386521 13331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 500) took 3896 ms. Trace:
I20260709 12:11:05.387324 15951 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:05.386890 13331 rpcz_store.cc:276] 0709 12:11:01.489832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:01.489951 (+   119us) service_pool.cc:224] Handling call
0709 12:11:01.490341 (+   390us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:01.499212 (+  8871us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:11:01.581152 (+ 81940us) log.cc:844] Serialized 14 byte log entry
0709 12:11:01.678409 (+ 97257us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:01.685978 (+  7569us) log.cc:844] Serialized 52 byte log entry
0709 12:11:01.686169 (+   191us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:01.686209 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:01.686281 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:05.386234 (+3699953us) raft_consensus.cc:1685] finished
0709 12:11:05.386258 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:05.386432 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":142796,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":632,"raft.run_cpu_time_us":404,"raft.run_wall_time_us":8799,"thread_start_us":115583,"threads_started":2,"wal-append.queue_time_us":225714}
I20260709 12:11:05.388582 15951 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:11:05.123232 12646 connection.cc:256] RPC call timeout handler was delayed by -3.1046740179999688s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18680000000000002s for a response
W20260709 12:11:05.389083 12646 connection.cc:256] RPC call timeout handler was delayed by -3.0812654480000674s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1654s for a response
I20260709 12:11:01.929976 12679 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:11:03.207044 15898 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.128051 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:05.134806 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:05.400683 12646 connection.cc:256] RPC call timeout handler was delayed by -2.4689368639999429s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22330000000000003s for a response
W20260709 12:11:05.401113 12646 connection.cc:256] RPC call timeout handler was delayed by -2.0491170009997859s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
W20260709 12:11:05.401408 12646 connection.cc:256] RPC call timeout handler was delayed by -2.0365977439998915s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
I20260709 12:11:05.401643 12599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
mode: GRACEFUL
new_leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
 from {username='slave'} at 127.0.0.1:32832
I20260709 12:11:05.140275 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2118) took 3849 ms. Trace:
I20260709 12:11:05.402516 12675 rpcz_store.cc:276] 0709 12:11:01.290698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:01.846943 (+556245us) service_pool.cc:224] Handling call
0709 12:11:01.980194 (+133251us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.139922 (+3159728us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.140206 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:05.143496 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2121) took 3655 ms. Trace:
I20260709 12:11:05.403467 12671 rpcz_store.cc:276] 0709 12:11:01.487893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.142088 (+3654195us) service_pool.cc:224] Handling call
0709 12:11:05.142485 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.143233 (+   748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.143436 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:11:05.401684 12646 connection.cc:256] RPC call timeout handler was delayed by -1.9690170009998837s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18630000000000002s for a response
I20260709 12:11:05.405057 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1840) took 3766 ms. Trace:
W20260709 12:11:05.405207 12646 connection.cc:256] RPC call timeout handler was delayed by -1.8909972569999809s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1753s for a response
W20260709 12:11:05.405548 12646 connection.cc:256] RPC call timeout handler was delayed by -1.8818536519997906s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1775s for a response
W20260709 12:11:05.405882 12646 connection.cc:256] RPC call timeout handler was delayed by -1.7485170009997546s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21080000000000002s for a response
I20260709 12:11:05.405432 12671 rpcz_store.cc:276] 0709 12:11:01.638731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.404250 (+3765519us) service_pool.cc:224] Handling call
0709 12:11:05.404511 (+   261us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.404704 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:05.404731 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:05.404779 (+    48us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:05.404823 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.404879 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:05.404999 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:11:05.406332 12646 connection.cc:256] RPC call timeout handler was delayed by -1.6886846009999772s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19230000000000003s for a response
W20260709 12:11:05.406656 12646 connection.cc:256] RPC call timeout handler was delayed by -1.5756326020000415s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2046s for a response
W20260709 12:11:05.407048 12646 connection.cc:256] RPC call timeout handler was delayed by -1.2022170009997808s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2715s for a response
I20260709 12:11:05.407233 12671 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:11:05.407353 12646 connection.cc:256] RPC call timeout handler was delayed by -1.1950170009997692s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2723s for a response
I20260709 12:11:05.407974 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1841) took 3760 ms. Trace:
I20260709 12:11:05.408339 12671 rpcz_store.cc:276] 0709 12:11:01.647531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.406579 (+3759048us) service_pool.cc:224] Handling call
0709 12:11:05.406856 (+   277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.407719 (+   863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.407910 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:05.156517 15995 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:04.991791 16000 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.161870 15993 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.168766 12678 rpcz_store.cc:276] 0709 12:11:01.634282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.167892 (+3533610us) service_pool.cc:224] Handling call
0709 12:11:05.168165 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.168297 (+   132us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:05.168313 (+    16us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:05.168341 (+    28us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:11:05.168364 (+    23us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.168397 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:05.168489 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:05.170277 12597 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 5: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:11:05.162232 12606 rpcz_store.cc:269] 0709 12:11:00.775446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:00.775720 (+   274us) service_pool.cc:224] Handling call
0709 12:11:04.971019 (+4195299us) spinlock_profiling.cc:243] Waited 500 ms on lock 0x7b54002706d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:04.971215 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1151095040}
W20260709 12:11:05.171586 12643 connection.cc:256] RPC call timeout handler was delayed by -1.9748627839999244s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18630000000000002s for a response
I20260709 12:11:05.155830 12673 rpcz_store.cc:276] 0709 12:11:01.633263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.154054 (+3520791us) service_pool.cc:224] Handling call
0709 12:11:05.154779 (+   725us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.154964 (+   185us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:05.154989 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:05.155119 (+   130us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:05.155152 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.155204 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:05.155332 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:05.169449 12672 rpcz_store.cc:276] 0709 12:11:01.302684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:01.851550 (+548866us) service_pool.cc:224] Handling call
0709 12:11:01.883971 (+ 32421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.012279 (+3128308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.012637 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:05.165747 16001 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:05.177886 12680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1849) took 3303 ms (client timeout 1695 ms). Trace:
W20260709 12:11:05.177387 12603 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.307s	user 0.010s	sys 0.014s
I20260709 12:11:05.183422 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2123) took 3587 ms. Trace:
I20260709 12:11:05.165239 16003 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:05.157891 16006 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.175462 16007 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.419102 16006 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:05.420310 16003 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:05.419420 16007 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.174393 15777 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:05.421176 15777 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.421578 16006 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:05.412202 15995 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:05.191869 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 3.26592e+06us
I20260709 12:11:05.422631 16007 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:05.425735 15884 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.193197 16009 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.426266 15884 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.426491 16009 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:05.428562 15884 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.428632 16009 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:05.429229 15884 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.429716 15884 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.430518 15913 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.431062 15913 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.432417 15884 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.193246 16014 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:05.434706 16014 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:05.196297 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52710.6us
I20260709 12:11:05.157456 15912 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.159067 15999 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:05.200083 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 100158us
I20260709 12:11:05.210191 16017 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:05.444216 15999 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:05.199488 16010 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:05.445165 16010 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:05.446220 16010 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:05.447351 16010 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:05.213447 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:05.448565 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.412511 16000 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:05.449797 12494 connection.cc:256] RPC call timeout handler was delayed by -1.8056304609997369s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19510000000000002s for a response
I20260709 12:11:05.213610 16015 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:05.450831 16000 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.451288 15982 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:05.416465 16001 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.452191 15982 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.452687 12599 raft_consensus.cc:606] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Received request to transfer leadership to TS a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:05.453097 12599 raft_consensus.cc:614] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:05.454298 15977 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.454789 15977 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.456905 15977 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.457437 15977 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:05.458086 12597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:32832
I20260709 12:11:05.458866 12597 raft_consensus.cc:606] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:05.185395 16004 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.209249 15997 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:05.459350 12597 raft_consensus.cc:614] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:05.460311 16024 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.454020 16001 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.461124 16024 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:05.462028 12602 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
I20260709 12:11:05.200565 16012 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.463339 16012 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.223295 15998 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.224421 16013 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:05.437740 16014 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:05.277282 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1836) took 3640 ms. Trace:
I20260709 12:11:05.284168 15896 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:05.466506 12677 rpcz_store.cc:276] 0709 12:11:01.637466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.276071 (+3638605us) service_pool.cc:224] Handling call
0709 12:11:05.276545 (+   474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.276797 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:05.276838 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:05.276899 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:05.276947 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.277032 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:05.277194 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:11:05.265709 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 71663.1us
I20260709 12:11:05.468202 12677 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:11:05.469005 12677 raft_consensus.cc:606] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Received request to transfer leadership to TS a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:05.469488 12677 raft_consensus.cc:614] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:11:05.294866 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:05.470649 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:05.295436 13304 connection.cc:256] RPC call timeout handler was delayed by -2.0713960119999228s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1706s for a response
I20260709 12:11:05.298648 15895 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:05.299078 16019 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:05.295679 15925 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:05.297354 15994 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:05.299786 16021 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:05.324015 15918 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:03.071839 15728 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.329102 16018 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:05.410085 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1842) took 3736 ms. Trace:
I20260709 12:11:05.474346 15994 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:05.474663 12671 rpcz_store.cc:276] 0709 12:11:01.673710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.409095 (+3735385us) service_pool.cc:224] Handling call
0709 12:11:05.409379 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.409708 (+   329us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:05.409738 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:05.409789 (+    51us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:05.409834 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.409907 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:05.410027 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:05.474737 16018 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.327925 15718 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:05.329954 15926 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:05.331593 15929 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.477192 16021 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:05.478329 15929 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.331677 15824 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:11:05.332185 12496 connection.cc:256] RPC call timeout handler was delayed by -2.3816544650001106s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18430000000000002s for a response
I20260709 12:11:05.479861 15918 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:05.480772 12496 connection.cc:256] RPC call timeout handler was delayed by -2.2844544649999534s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19510000000000002s for a response
I20260709 12:11:05.480655 15824 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:05.328789 16023 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:05.337778 12429 catalog_manager.cc:5697] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 2 to 5, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 5 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:05.459924 16025 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:05.483001 16025 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.482972 12429 catalog_manager.cc:5697] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 2 to 7, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 7 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
W20260709 12:11:05.352952 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.485374 12677 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:11:05.417790 12676 rpcz_store.cc:276] 0709 12:11:01.596012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.154546 (+3558534us) service_pool.cc:224] Handling call
0709 12:11:05.182665 (+ 28119us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.182952 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:05.182993 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:05.183057 (+    64us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:05.183111 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.183185 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:05.183342 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:05.355114 15822 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:05.363350 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:05.486718 12676 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
 from {username='slave'} at 127.0.0.1:39924
I20260709 12:11:05.375934 15634 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.487461 12676 raft_consensus.cc:606] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Received request to transfer leadership to TS a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:05.487972 12676 raft_consensus.cc:614] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:05.487687 15634 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:05.444271 13305 connection.cc:256] RPC call timeout handler was delayed by -2.0842399930002102s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1706s for a response
W20260709 12:11:05.489310 13305 connection.cc:256] RPC call timeout handler was delayed by -1.9474399930002164s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18580000000000002s for a response
W20260709 12:11:05.489727 13305 connection.cc:256] RPC call timeout handler was delayed by -1.8561156770001617s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1555s for a response
I20260709 12:11:05.121721 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:05.459437 15930 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:05.390282 12679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1848) took 3612 ms (client timeout 2947 ms). Trace:
I20260709 12:11:05.394876 13332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38482 (request call id 372) took 5009 ms. Trace:
I20260709 12:11:05.374006 15970 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:11:05.482003 15925 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.397858 15976 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:05.398587 16022 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:05.137871 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 273) took 3341 ms. Trace:
I20260709 12:11:05.444034 16017 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.156584 15996 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:05.490093 13305 connection.cc:256] RPC call timeout handler was delayed by -1.372339993000196s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2497s for a response
I20260709 12:11:05.511720 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1837) took 3873 ms. Trace:
I20260709 12:11:05.458281 15931 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:05.414655 12643 connection.cc:256] RPC call timeout handler was delayed by -1.7395692579998467s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1774s for a response
I20260709 12:11:05.512671 15931 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:05.513482 12643 connection.cc:256] RPC call timeout handler was delayed by -1.5984835300000668s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1736s for a response
W20260709 12:11:05.513952 12643 connection.cc:256] RPC call timeout handler was delayed by -1.448827109999911s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21080000000000002s for a response
W20260709 12:11:05.514397 12643 connection.cc:256] RPC call timeout handler was delayed by -1.1458681439999054s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2723s for a response
I20260709 12:11:05.227360 16011 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:05.518213 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:11:05.518599 16011 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:05.490906 12679 rpcz_store.cc:269] 0709 12:11:01.778280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:01.788867 (+ 10587us) service_pool.cc:224] Handling call
0709 12:11:05.390201 (+3601334us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:05.491214 16020 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.522921 16020 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.502542 13332 rpcz_store.cc:276] 0709 12:11:00.382046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:00.382826 (+   780us) service_pool.cc:224] Handling call
0709 12:11:00.383309 (+   483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:00.383755 (+   446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:00.383789 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:00.384357 (+   568us) log.cc:844] Serialized 26 byte log entry
0709 12:11:00.573140 (+188783us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:00.611950 (+ 38810us) log.cc:844] Serialized 14 byte log entry
0709 12:11:00.612289 (+   339us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:00.612367 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:01.675038 (+1062671us) spinlock_profiling.cc:243] Waited 15.2 ms on lock 0x7b1800302f80. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db
0709 12:11:01.676331 (+  1293us) spinlock_profiling.cc:243] Waited 2.22 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:05.370962 (+3694631us) raft_consensus.cc:1685] finished
0709 12:11:05.370986 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:05.391598 (+ 20612us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":113111,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":50,"raft.queue_time_us":875,"raft.run_cpu_time_us":284,"raft.run_wall_time_us":8298,"spinlock_wait_cycles":40056576,"thread_start_us":219573,"threads_started":2,"wal-append.queue_time_us":253587}
I20260709 12:11:05.458715 15942 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.459626 15997 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:05.412762 15993 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.253201 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2058) took 4159 ms. Trace:
I20260709 12:11:05.254454 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 283) took 4302 ms. Trace:
I20260709 12:11:05.532086 12674 rpcz_store.cc:276] 0709 12:11:00.952119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:01.815254 (+863135us) service_pool.cc:224] Handling call
0709 12:11:05.102382 (+3287128us) spinlock_profiling.cc:243] Waited 1.19 ms on lock 0x7b58001f2c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:11:05.176112 (+ 73730us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.176305 (+   193us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:05.176330 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:05.176371 (+    41us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:05.176409 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.176463 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:05.254248 (+ 77785us) spinlock_profiling.cc:243] Waited 73.5 ms on lock 0x7b54000924d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:05.254380 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":171672704}
W20260709 12:11:05.257995 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.534705 12678 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:11:05.536020 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1845) took 3828 ms. Trace:
I20260709 12:11:05.536329 12678 rpcz_store.cc:276] 0709 12:11:01.707767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.413067 (+3705300us) service_pool.cc:224] Handling call
0709 12:11:05.413363 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.535138 (+121775us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.535769 (+   631us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b540036cc54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:05.535962 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271909632}
I20260709 12:11:05.537461 12600 rpcz_store.cc:276] 0709 12:11:01.093910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:01.094048 (+   138us) service_pool.cc:224] Handling call
0709 12:11:04.979160 (+3885112us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.158575 (+179415us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.252904 (+ 94329us) spinlock_profiling.cc:243] Waited 3.88 s on lock 0x7b54002706d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:05.253116 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8934165888}
W20260709 12:11:05.473439 13304 connection.cc:256] RPC call timeout handler was delayed by -1.8103306299999531s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2497s for a response
W20260709 12:11:05.539111 13304 connection.cc:256] RPC call timeout handler was delayed by -1.5478879370000413s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21400000000000002s for a response
I20260709 12:11:05.490739 15930 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.476159 16019 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.512295 12675 rpcz_store.cc:276] 0709 12:11:01.637833 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.403627 (+3765794us) service_pool.cc:224] Handling call
0709 12:11:05.403931 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.510495 (+106564us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:05.510542 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:05.510601 (+    59us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:05.510661 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.511402 (+   741us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54000f00d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:05.511465 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:05.511637 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":244452224}
W20260709 12:11:05.481220 12496 connection.cc:256] RPC call timeout handler was delayed by -1.8829126529999485s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15990000000000001s for a response
I20260709 12:11:05.363081 15890 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:05.417187 12680 rpcz_store.cc:269] 0709 12:11:01.871633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:01.889160 (+ 17527us) service_pool.cc:224] Handling call
0709 12:11:05.175327 (+3286167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:05.507345 15976 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:05.465853 16013 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.507695 12598 rpcz_store.cc:276] 0709 12:11:01.796917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:01.797183 (+   266us) service_pool.cc:224] Handling call
0709 12:11:01.797500 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:01.837695 (+ 40195us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:01.837730 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:01.837935 (+   205us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:01.837974 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.137525 (+3299551us) spinlock_profiling.cc:243] Waited 40 ms on lock 0x7b54002f1054. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:05.137592 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:05.137790 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91951872}
I20260709 12:11:05.415767 12673 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:11:05.458639 15777 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.442044 15912 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.554435 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1846) took 3846 ms. Trace:
I20260709 12:11:05.554569 15777 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.554888 12673 rpcz_store.cc:276] 0709 12:11:01.708033 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.415006 (+3706973us) service_pool.cc:224] Handling call
0709 12:11:05.415378 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.554107 (+138729us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.554376 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:05.555344 15777 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.417021 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1847) took 3708 ms. Trace:
I20260709 12:11:05.557628 12672 rpcz_store.cc:276] 0709 12:11:01.708270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.416084 (+3707814us) service_pool.cc:224] Handling call
0709 12:11:05.416414 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:05.416644 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:05.416679 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:05.416737 (+    58us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:05.416773 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.416827 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:05.416964 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:05.507535 16022 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:05.563715 12494 connection.cc:256] RPC call timeout handler was delayed by -1.7165304609998202s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.205s for a response
I20260709 12:11:05.541221 15942 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.569268 15942 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.571272 12604 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:05.586429 15982 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.586941 16023 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:05.587188 16011 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:05.587572 16023 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.587762 16011 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.588114 16023 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.588280 16011 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.586345 15913 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.589527 16018 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.590531 15929 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.591288 16024 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:05.591498 15997 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:05.592378 15931 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.592537 12429 catalog_manager.cc:5697] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:05.594277 15913 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:05.594205 12429 catalog_manager.cc:5697] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 5 to 6, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 6 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:05.594889 15913 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.596613 15993 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:05.609086 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:05.597163 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:11:05.619616 12571 connection.cc:256] RPC call timeout handler was delayed by -2.8535163609999472s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20800000000000002s for a response
I20260709 12:11:05.595065 15976 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:05.602270 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 91131.5us
I20260709 12:11:05.621207 15994 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.621558 16014 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:11:05.622529 15895 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:05.622916 12496 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:05.623126 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:05.602977 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 62554.8us
I20260709 12:11:05.623924 15777 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.607195 15822 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:05.624487 15777 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:05.624893 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:05.588339 16017 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:05.620561 15930 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.620781 15913 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:05.627162 12571 connection.cc:256] RPC call timeout handler was delayed by -2.5898163609999756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23730000000000004s for a response
W20260709 12:11:05.640727 12571 connection.cc:256] RPC call timeout handler was delayed by -2.4018567669997992s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.191s for a response
I20260709 12:11:05.640470 16034 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:11:05.641110 12571 connection.cc:256] RPC call timeout handler was delayed by -2.3513163610000447s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2638s for a response
W20260709 12:11:05.641484 12571 connection.cc:256] RPC call timeout handler was delayed by -2.3091567669998767s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2013s for a response
I20260709 12:11:05.641767 16034 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:05.633044 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:05.633478 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:05.634110 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:05.634258 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:05.649652 12531 raft_consensus.cc:2410] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:11:05.634291 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:05.650723 12530 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:11:05.652184 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:05.625491 15777 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:11:05.628672 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:05.642470 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:05.643139 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:05.643188 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:05.641788 12571 connection.cc:256] RPC call timeout handler was delayed by -2.1051178079999318s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1874s for a response
I20260709 12:11:05.625173 15822 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.457882 15977 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:05.596081 12429 catalog_manager.cc:5697] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 4 to 5, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 5 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:05.645581 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:05.586941 15996 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.771212 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:05.690690 12673 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:11:05.764050 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:05.775993 12597 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:11:05.767937 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:05.756511 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 316715us
I20260709 12:11:05.703279 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:05.739670 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:05.731698 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 167121us
I20260709 12:11:05.695447 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:05.656091 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:05.783666 12571 connection.cc:256] RPC call timeout handler was delayed by -2.0481582989998515s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1886s for a response
I20260709 12:11:05.776500 12676 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
W20260709 12:11:05.757043 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:05.784148 12571 connection.cc:256] RPC call timeout handler was delayed by -2.0265299170000617s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1695s for a response
W20260709 12:11:05.784698 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.696158 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:05.784703 12571 connection.cc:256] RPC call timeout handler was delayed by -1.9445163610000691s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.309s for a response
I20260709 12:11:05.758765 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:05.786490 12602 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:05.740269 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:05.724859 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:05.783881 12680 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
W20260709 12:11:05.785657 12571 connection.cc:256] RPC call timeout handler was delayed by -1.854917807999982s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21520000000000003s for a response
W20260709 12:11:05.789041 12571 connection.cc:256] RPC call timeout handler was delayed by -1.8285582989999511s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.213s for a response
I20260709 12:11:05.692703 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:05.741557 12523 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260709 12:11:05.789464 12571 connection.cc:256] RPC call timeout handler was delayed by -1.712316360999921s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3348s for a response
W20260709 12:11:05.790238 12571 connection.cc:256] RPC call timeout handler was delayed by -1.4028567669997756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.30200000000000005s for a response
I20260709 12:11:05.662914 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:05.790963 12571 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:05.787840 12600 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:11:05.785511 12530 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:11:05.689808 12604 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:05.656556 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:05.793883 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 169073us
I20260709 12:11:05.570003 15942 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:05.788295 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 134609us
W20260709 12:11:05.794435 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.795461 12531 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:11:05.791839 12678 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:05.796720 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:05.797593 12530 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: []
I20260709 12:11:05.798645 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:05.799264 12523 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:05.797935 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:05.800570 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:05.673550 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:05.795531 15876 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:05.803413 15913 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:05.803377 15876 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.803966 15913 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:05.804382 15913 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:05.804608 15942 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:05.805389 12524 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:11:05.806593 15634 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.807301 16012 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.807900 15824 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:05.759750 15918 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.808663 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:05.801749 12525 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:11:05.809288 16025 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.809505 16012 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:11:05.810626 12530 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:11:05.810789 16025 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:05.812662 16025 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:05.816658 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:05.816933 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:05.631764 16014 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:05.814193 16025 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:05.696575 16024 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:05.817848 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:05.817160 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:05.645912 16034 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:05.821403 12525 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:11:05.644788 16023 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.712489 15925 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.857828 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:05.848603 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:05.870477 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 73759.8us
I20260709 12:11:05.759678 15994 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:05.858446 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:05.856773 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:05.794431 12675 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
I20260709 12:11:05.829397 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:05.869540 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:05.824297 16014 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:06.010540 12671 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:11:05.853665 12526 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:05.858860 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:05.856492 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2064) took 4058 ms. Trace:
I20260709 12:11:05.871218 12531 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:05.824849 12673 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:05.868965 12530 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:05.870257 16024 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:05.854887 15918 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:11:05.900264 15925 consensus_queue.cc:1059] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:05.899544 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:05.832628 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:05.874065 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 71649.8us
I20260709 12:11:05.906769 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:05.908118 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:05.896880 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:05.904395 16004 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:05.905397 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1940) took 1089 ms. Trace:
I20260709 12:11:05.893378 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:05.899266 16013 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.898713 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:05.859505 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:05.982200 15876 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.862764 16007 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:05.984898 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 132711us
I20260709 12:11:05.800717 15996 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:05.872157 12527 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:11:05.988754 12523 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
W20260709 12:11:05.993438 15926 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.516s	user 0.015s	sys 0.001s
I20260709 12:11:05.889446 16037 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:05.994932 15994 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:06.003171 16020 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:06.004818 16021 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.997418 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:06.006959 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:05.872546 12672 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
W20260709 12:11:06.009845 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:05.874677 12597 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 6: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:05.862404 16022 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:05.997381 15822 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:06.320228 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:06.593277 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:06.636302 12675 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:06.636900 12676 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 6: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:06.637535 12571 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:06.264449 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:06.358853 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:06.282266 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.610559 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:06.289892 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:06.269313 13336 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:06.446199 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:06.633726 12605 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:06.210409 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:06.195734 12429 catalog_manager.cc:5697] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:06.288327 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:06.218734 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.238193 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:06.264153 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:06.595911 12522 consensus_meta.cc:287] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.948s	user 0.026s	sys 0.056s
I20260709 12:11:06.638388 12679 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:06.634608 12674 raft_consensus.cc:2410] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:11:06.609686 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:06.639043 12678 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 4.
I20260709 12:11:05.890051 15942 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:06.650619 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:06.649791 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:06.659451 12531 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:06.657574 12429 catalog_manager.cc:5697] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 1 to 4, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 4 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:05.910859 16019 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:06.639552 12601 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:11:06.639823 12524 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:06.639624 12604 rpcz_store.cc:276] 0709 12:11:04.816264 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.127723 (+311459us) service_pool.cc:224] Handling call
0709 12:11:05.905155 (+777432us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b5400375d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:05.905317 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":392230784}
I20260709 12:11:06.640520 12600 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:06.640956 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1832) took 1008 ms. Trace:
I20260709 12:11:06.597179 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:06.641021 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:06.643616 13334 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:06.644388 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:06.675058 12606 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 7: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:06.646189 12522 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:06.676209 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1833) took 1043 ms. Trace:
I20260709 12:11:06.676666 12522 rpcz_store.cc:276] 0709 12:11:05.632890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.633011 (+   121us) service_pool.cc:224] Handling call
0709 12:11:06.676128 (+1043117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:06.638597 13335 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 4.
W20260709 12:11:06.643944 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:06.651016 12599 raft_consensus.cc:2393] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:11:06.651161 12602 consensus_meta.cc:287] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.863s	user 0.008s	sys 0.004s
I20260709 12:11:06.679365 12602 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
W20260709 12:11:06.497649 15970 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.001s	user 0.010s	sys 0.005s
I20260709 12:11:06.680096 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1945) took 1035 ms. Trace:
I20260709 12:11:06.680631 12602 rpcz_store.cc:276] 0709 12:11:05.645172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.645326 (+   154us) service_pool.cc:224] Handling call
0709 12:11:06.680018 (+1034692us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:06.639801 12528 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:06.672194 12526 rpcz_store.cc:276] 0709 12:11:05.632545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.632708 (+   163us) service_pool.cc:224] Handling call
0709 12:11:06.640835 (+1008127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:06.596167 16025 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:06.645123 12525 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:06.657536 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:06.683223 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.683939 12597 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:06.684149 12531 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
W20260709 12:11:06.331049 16010 consensus_meta.cc:287] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.883s	user 0.012s	sys 0.000s
I20260709 12:11:06.650797 16037 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 8 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:06.690832 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.691046 13331 raft_consensus.cc:2410] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:11:06.692140 12522 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:06.690987 16037 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:06.693142 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.680357 15876 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:11:06.694602 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.696523 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.697430 16021 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:06.697683 12525 raft_consensus.cc:2410] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:11:06.701314 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:06.685793 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.702419 12528 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:11:06.703148 12496 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:06.087922 16014 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:06.658416 15822 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:06.693737 12522 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:06.687697 12429 catalog_manager.cc:5697] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 1 to 6. New cstate: current_term: 6 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:06.708928 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.709666 12530 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:11:06.709947 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.710662 12522 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:11:06.706221 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:06.682631 15824 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:11:06.090065 16006 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.600s
I20260709 12:11:06.637658 12603 rpcz_store.cc:276] 0709 12:11:01.798106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:01.806158 (+  8052us) service_pool.cc:224] Handling call
0709 12:11:01.806548 (+   390us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:01.816821 (+ 10273us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:11:01.838348 (+ 21527us) log.cc:844] Serialized 14 byte log entry
0709 12:11:05.626910 (+3788562us) log.cc:844] Serialized 14 byte log entry
0709 12:11:05.778170 (+151260us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:05.782141 (+  3971us) log.cc:844] Serialized 52 byte log entry
0709 12:11:05.782302 (+   161us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:05.791996 (+  9694us) log.cc:844] Serialized 14 byte log entry
0709 12:11:05.792533 (+   537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:05.792618 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:05.856249 (+ 63631us) raft_consensus.cc:1685] finished
0709 12:11:05.856268 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:05.856423 (+   155us) inbound_call.cc:177] Queueing success response
0709 12:11:06.001077 (+144654us) spinlock_profiling.cc:243] Waited 8.45 ms on lock 0x7b18003b0860. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485933522 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
Metrics: {"mutex_wait_us":18243,"prepare.queue_time_us":59934,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":2528,"raft.queue_time_us":11602,"raft.run_cpu_time_us":234,"raft.run_wall_time_us":3915,"spinlock_wait_cycles":19431680,"thread_start_us":3925583,"threads_started":4,"wal-append.queue_time_us":1721798,"wal-append.run_cpu_time_us":1733,"wal-append.run_wall_time_us":1038554}
I20260709 12:11:06.658093 16004 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:06.698202 16021 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:06.699204 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:06.643698 16007 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:06.699822 15918 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:06.700606 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:06.634366 16011 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:06.097530 13360 debug-util.cc:398] Leaking SignalData structure 0x7b08002e0ba0 after lost signal to thread 11321
I20260709 12:11:06.320196 16034 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:06.703820 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.711287 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.689147 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:06.741019 12523 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:06.717617 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.688581 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:06.485827 15977 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:06.721872 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:06.742297 12525 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:06.722448 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.722728 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:06.743590 12522 raft_consensus.cc:2410] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 8 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260709 12:11:06.704653 16022 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.024s
I20260709 12:11:06.725677 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:06.729142 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:06.730286 12429 catalog_manager.cc:5697] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 5 to 6, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 6 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:06.750227 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.734534 12527 raft_consensus.cc:2376] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7. Re-sending same reply.
W20260709 12:11:06.739916 13360 debug-util.cc:398] Leaking SignalData structure 0x7b08002e01a0 after lost signal to thread 12463
I20260709 12:11:06.735474 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.753221 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:06.748065 12429 catalog_manager.cc:5697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 3 to 5. New cstate: current_term: 5 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:11:06.753579 13360 debug-util.cc:398] Leaking SignalData structure 0x7b08003c03e0 after lost signal to thread 12555
I20260709 12:11:06.720582 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:06.703991 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:06.723729 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:06.755473 13339 raft_consensus.cc:2410] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:11:06.742571 12672 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:06.718799 12496 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:06.745317 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:06.756505 13360 debug-util.cc:398] Leaking SignalData structure 0x7b08003cd5a0 after lost signal to thread 12630
W20260709 12:11:06.756985 13360 debug-util.cc:398] Leaking SignalData structure 0x7b08001f2c00 after lost signal to thread 12631
I20260709 12:11:06.721719 15970 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:06.755704 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.761632 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:06.762929 13330 raft_consensus.cc:2410] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:11:06.740823 12528 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:06.759105 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:06.764671 12525 raft_consensus.cc:2410] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:11:06.766398 12429 catalog_manager.cc:5697] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 2 to 4, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 4 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:06.770511 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:06.765178 12496 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:06.778262 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.781333 12496 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:06.721354 15822 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:06.785948 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.779960 12527 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
W20260709 12:11:06.760746 13360 debug-util.cc:398] Leaking SignalData structure 0x7b08003a9680 after lost signal to thread 12704
W20260709 12:11:06.793955 13360 debug-util.cc:398] Leaking SignalData structure 0x7b0800338180 after lost signal to thread 12705
I20260709 12:11:06.793845 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:06.765228 15977 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:06.761569 15970 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:06.797492 13334 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:11:06.795348 13335 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:06.802816 12525 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260709 12:11:06.764941 15925 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:06.740662 16004 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:06.804116 12429 catalog_manager.cc:5697] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 3 to 4, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 4 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:06.789587 15918 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:11:06.810406 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:06.812088 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:06.812594 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:06.813495 12671 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:11:06.803951 15925 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:06.814623 15925 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:06.817483 16007 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:06.818310 16007 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:06.819748 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:06.820595 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:06.801875 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:06.804298 13332 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.058s	sys 0.101s
I20260709 12:11:06.731992 16024 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:06.821604 12671 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
W20260709 12:11:06.799345 13360 debug-util.cc:398] Leaking SignalData structure 0x7b08003d1a20 after lost signal to thread 13363
I20260709 12:11:06.821892 12672 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
W20260709 12:11:06.825176 13360 debug-util.cc:398] Leaking SignalData structure 0x7b080032b300 after lost signal to thread 13364
I20260709 12:11:06.705391 16019 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:06.808121 12429 catalog_manager.cc:5697] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:11:06.812745 12673 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:06.804112 15970 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:06.820787 12675 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:11:06.819185 16006 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:11:06.805279 12527 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 7 index: 3. (term mismatch)
I20260709 12:11:06.788331 16020 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:06.825496 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:06.826215 13360 debug-util.cc:398] Leaking SignalData structure 0x7b08003549a0 after lost signal to thread 15984
I20260709 12:11:06.799544 15977 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:06.834872 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:06.841681 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:06.822225 13332 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.620s	user 0.058s	sys 0.101s
I20260709 12:11:06.844002 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:06.847052 16020 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:06.862721 16020 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:06.847352 12676 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:11:06.866959 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2141) took 1021 ms. Trace:
I20260709 12:11:06.867278 12676 rpcz_store.cc:276] 0709 12:11:05.845522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:06.825217 (+979695us) service_pool.cc:224] Handling call
0709 12:11:06.866890 (+ 41673us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:06.855512 16006 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:06.868793 16006 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:06.858573 12527 pending_rounds.cc:85] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 2
I20260709 12:11:06.870544 12527 pending_rounds.cc:107] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 6.3
I20260709 12:11:06.870805 12527 raft_consensus.cc:2889] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:06.872098 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.873447 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.846768 12429 catalog_manager.cc:5697] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 3 to 4, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 4 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:06.876302 12429 catalog_manager.cc:5697] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 3 to 4, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 4 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:06.836295 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:06.810881 15918 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:06.840855 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:06.883232 15918 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:06.883761 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.884042 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.886468 12673 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:11:06.898650 12429 catalog_manager.cc:5697] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 1 to 6, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 6 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:06.907788 12429 catalog_manager.cc:5697] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 1 to 6, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 6 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:06.910560 15918 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:06.916905 12646 leader_election.cc:336] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.508s (SENT)
I20260709 12:11:06.924012 15918 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:06.909593 12429 catalog_manager.cc:5697] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 1 to 4, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 4 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:06.932049 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2143) took 1037 ms. Trace:
I20260709 12:11:06.932448 12671 rpcz_store.cc:276] 0709 12:11:05.894605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:06.834446 (+939841us) service_pool.cc:224] Handling call
0709 12:11:06.931970 (+ 97524us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:06.859972 12529 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:06.934861 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 1853) took 1076 ms. Trace:
I20260709 12:11:06.935199 12529 rpcz_store.cc:276] 0709 12:11:05.858477 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:06.824500 (+966023us) service_pool.cc:224] Handling call
0709 12:11:06.934802 (+110302us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:11:06.942961 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:06.951987 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 311012us
I20260709 12:11:06.953195 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:06.954695 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:06.955569 12646 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.547s (SENT) [suppressed 37 similar messages]
I20260709 12:11:06.955574 12525 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:06.931728 15977 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:06.959133 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.827395 16007 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:06.962308 12429 catalog_manager.cc:5697] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 1 to 3, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 3 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
W20260709 12:11:06.960584 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 1.06372e+06us
I20260709 12:11:06.962607 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.963505 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:06.965756 12494 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:06.966574 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 285811us
I20260709 12:11:06.949024 16024 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:06.972577 12605 raft_consensus.cc:2410] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:11:07.002152 12496 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:07.005488 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1951) took 1209 ms. Trace:
I20260709 12:11:06.996059 12569 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:07.006067 12605 rpcz_store.cc:276] 0709 12:11:05.795751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.796121 (+   370us) service_pool.cc:224] Handling call
0709 12:11:05.874215 (+ 78094us) spinlock_profiling.cc:243] Waited 24.4 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:11:07.005396 (+1131181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":56217856}
I20260709 12:11:06.948238 16019 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.019973 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:07.028288 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 324967us
W20260709 12:11:07.030369 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 75878.6us
I20260709 12:11:07.051906 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:07.059141 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.059556 12672 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 6
W20260709 12:11:07.047003 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 91438.5us
W20260709 12:11:07.069592 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 424900us
I20260709 12:11:07.023552 16042 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:07.104805 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:07.109066 13335 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:07.037532 16021 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:07.075527 15913 consensus_meta.cc:287] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.271s	user 0.011s	sys 0.005s
I20260709 12:11:07.112061 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:07.115485 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:07.114820 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:07.115674 12646 leader_election.cc:336] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.654s (SENT)
I20260709 12:11:07.130208 13337 raft_consensus.cc:2393] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 8: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:07.130759 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.135277 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1850) took 1277 ms. Trace:
I20260709 12:11:07.135722 12522 rpcz_store.cc:276] 0709 12:11:05.857570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:06.757257 (+899687us) service_pool.cc:224] Handling call
0709 12:11:06.757635 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:06.757891 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:11:06.958639 (+200748us) log.cc:844] Serialized 14 byte log entry
0709 12:11:07.134901 (+176262us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:07.134977 (+    76us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:07.135032 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.135099 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:07.135220 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":282092,"prepare.run_cpu_time_us":8941,"prepare.run_wall_time_us":12033,"raft.queue_time_us":96145,"spinlock_wait_cycles":8498432,"thread_start_us":371246,"threads_started":3}
I20260709 12:11:07.023277 16041 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:07.107796 15822 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:07.143599 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 112149us
I20260709 12:11:07.112736 16049 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:07.124094 16046 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:11:07.027680 16020 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.153801 12672 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:11:07.141533 16011 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:07.133827 16004 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:07.157900 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 128261us
I20260709 12:11:07.131928 15913 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.163739 15913 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:07.164331 15913 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:11:07.165606 15913 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:11:07.167071 16019 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:07.150151 16049 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:07.171110 16049 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:07.171972 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:07.155972 16007 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:07.168174 15996 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:07.169795 15926 raft_consensus.cc:3055] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:07.168967 16020 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:07.153039 15822 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:07.151343 16051 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:07.157298 16041 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.150923 16055 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:07.165200 16006 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.122355 16042 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:07.181706 15926 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:07.185577 15822 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.198045 15970 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:07.199937 15822 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:07.200263 15970 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:07.184366 16051 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:07.201385 16051 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:07.187095 16041 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:07.194852 16034 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:07.195363 16013 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:11:07.192094 16048 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:11:07.210153 16013 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:07.213889 16013 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.215219 16013 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:07.219726 16011 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:07.220583 16011 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:07.220818 16041 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:07.205222 16049 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.189118 16055 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.224994 16055 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:07.225346 16042 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:07.227465 16049 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:07.261673 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 151049us
I20260709 12:11:07.269932 12531 raft_consensus.cc:2410] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:11:07.269802 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1859) took 1409 ms. Trace:
I20260709 12:11:07.272478 12526 rpcz_store.cc:276] 0709 12:11:05.860309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:06.762295 (+901986us) service_pool.cc:224] Handling call
0709 12:11:06.762600 (+   305us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:06.762977 (+   377us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:06.763010 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:06.774052 (+ 11042us) log.cc:844] Serialized 26 byte log entry
0709 12:11:07.024994 (+250942us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:07.025033 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.026506 (+  1473us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:07.268884 (+242378us) raft_consensus.cc:1685] finished
0709 12:11:07.268992 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:07.269522 (+   530us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":93,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":12707,"thread_start_us":224617,"threads_started":1,"wal-append.queue_time_us":324161}
I20260709 12:11:07.276392 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:07.276772 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:07.283068 12524 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 7: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:11:07.287317 12646 leader_election.cc:336] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.736s (SENT)
W20260709 12:11:07.288492 12646 leader_election.cc:336] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.753s (SENT)
W20260709 12:11:07.289376 12646 leader_election.cc:336] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.774s (SENT)
W20260709 12:11:07.290308 12646 leader_election.cc:336] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.775s (SENT)
I20260709 12:11:07.290845 12646 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:07.272020 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.297706 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:07.298091 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:07.298907 12678 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 4.
I20260709 12:11:07.300887 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2149) took 1318 ms. Trace:
I20260709 12:11:07.298091 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:07.297719 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:07.303443 12677 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:11:07.303676 12679 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:11:07.301522 12675 rpcz_store.cc:276] 0709 12:11:05.982812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:06.838623 (+855811us) service_pool.cc:224] Handling call
0709 12:11:06.838940 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:06.839292 (+   352us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:06.839319 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:06.839824 (+   505us) log.cc:844] Serialized 26 byte log entry
0709 12:11:07.086980 (+247156us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:07.087044 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.087149 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:07.300248 (+213099us) raft_consensus.cc:1685] finished
0709 12:11:07.300295 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:07.300656 (+   361us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":201399,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":64737,"spinlock_wait_cycles":1554816,"thread_start_us":143600,"threads_started":1,"wal-append.queue_time_us":335086}
I20260709 12:11:07.301741 12680 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:11:07.311439 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:07.313354 12646 leader_election.cc:336] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.863s (SENT)
I20260709 12:11:07.306358 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.316959 13338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 316) took 1514 ms. Trace:
I20260709 12:11:07.317097 13332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 314) took 1539 ms. Trace:
I20260709 12:11:07.325335 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:07.319079 15994 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.209352 15925 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.323390 15996 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:07.326604 16006 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:07.296020 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.352839 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.332568 13332 rpcz_store.cc:276] 0709 12:11:05.777484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.777606 (+   122us) service_pool.cc:224] Handling call
0709 12:11:05.800281 (+ 22675us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:05.819346 (+ 19065us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:11:06.201663 (+382317us) spinlock_profiling.cc:243] Waited 7.43 ms on lock 0x7b18004714c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf 00007f1488a3a038
0709 12:11:06.933206 (+731543us) log.cc:844] Serialized 14 byte log entry
0709 12:11:07.116214 (+183008us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:07.128449 (+ 12235us) log.cc:844] Serialized 26 byte log entry
0709 12:11:07.128614 (+   165us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:07.128657 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.128726 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:07.316149 (+187423us) raft_consensus.cc:1685] finished
0709 12:11:07.316378 (+   229us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:07.316801 (+   423us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19061,"prepare.queue_time_us":1121604,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":2035,"raft.queue_time_us":19555,"raft.run_cpu_time_us":267,"raft.run_wall_time_us":46001,"spinlock_wait_cycles":17086208,"thread_start_us":811035,"threads_started":2,"wal-append.queue_time_us":237424}
W20260709 12:11:07.353575 12646 leader_election.cc:336] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.868s (SENT)
W20260709 12:11:07.355235 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:07.331949 13338 rpcz_store.cc:276] 0709 12:11:05.802613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:05.802905 (+   292us) service_pool.cc:224] Handling call
0709 12:11:05.803231 (+   326us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:06.095366 (+292135us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:06.097657 (+  2291us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:06.295321 (+197664us) log.cc:844] Serialized 52 byte log entry
0709 12:11:06.955337 (+660016us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:07.074877 (+119540us) log.cc:844] Serialized 14 byte log entry
0709 12:11:07.111031 (+ 36154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.111103 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:07.316115 (+205012us) raft_consensus.cc:1685] finished
0709 12:11:07.316177 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:07.316649 (+   472us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17330,"prepare.queue_time_us":38026,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":2332,"raft.queue_time_us":50622,"raft.run_cpu_time_us":344,"raft.run_wall_time_us":11901,"thread_start_us":546955,"threads_started":3,"wal-append.queue_time_us":389897}
I20260709 12:11:07.360278 16049 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:11:07.362962 16042 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:07.365617 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.366464 16055 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.367233 15925 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:07.367422 16055 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.368254 15913 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.039s
I20260709 12:11:07.371404 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:07.372836 13336 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:07.374951 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:07.375398 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:07.376303 12646 leader_election.cc:336] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.923s (SENT)
I20260709 12:11:07.347729 15994 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:07.379876 15994 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:07.362967 12527 raft_consensus.cc:2368] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6.
I20260709 12:11:07.388249 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.388602 13333 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:11:07.389163 12527 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:07.391742 16042 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:07.392905 16042 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.399220 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:11:07.347729 16014 consensus_meta.cc:287] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.620s	user 0.004s	sys 0.009s
I20260709 12:11:07.401669 16042 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:07.410754 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.401684 16014 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.412458 15926 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.415055 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:07.403540 12529 raft_consensus.cc:2376] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8. Re-sending same reply.
I20260709 12:11:07.424638 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:07.406622 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:07.424654 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.407111 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:07.427711 12674 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:07.427917 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.407560 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:07.408381 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:07.431710 13333 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:11:07.416419 12528 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 12:11:07.429742 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.434651 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.435725 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1861) took 1574 ms. Trace:
I20260709 12:11:07.436364 12528 rpcz_store.cc:276] 0709 12:11:05.860887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:06.763740 (+902853us) service_pool.cc:224] Handling call
0709 12:11:06.764155 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:07.432523 (+668368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:07.435014 (+  2491us) spinlock_profiling.cc:243] Waited 652 ms on lock 0x7b540042a4d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:07.435459 (+   445us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1498966912}
I20260709 12:11:07.437453 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.417508 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1848) took 1413 ms. Trace:
I20260709 12:11:07.417582 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.438861 16055 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:07.439527 12526 raft_consensus.cc:2393] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 8: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:07.439913 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.440255 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:07.440675 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.442031 16010 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:11:07.442659 16010 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:07.443149 16010 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:07.445421 15994 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:07.405871 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:07.446410 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 45780.8us
I20260709 12:11:07.367820 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.448091 12522 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:07.448637 16070 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:11:07.402712 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:07.420096 12597 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:07.449391 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:07.448122 16021 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:11:07.421351 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 1849) took 1564 ms. Trace:
I20260709 12:11:07.451112 15926 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:07.450898 12524 rpcz_store.cc:276] 0709 12:11:05.857270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:06.754804 (+897534us) service_pool.cc:224] Handling call
0709 12:11:07.421005 (+666201us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b540038f4d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:07.421262 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1131550592}
I20260709 12:11:07.453383 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.454281 12524 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:11:07.455368 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.456346 12524 raft_consensus.cc:2410] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 8 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:11:07.456218 16011 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.457011 16014 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:07.457357 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:07.422941 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:07.409085 15876 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:07.459012 16011 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:07.459086 15876 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:07.438638 12531 rpcz_store.cc:276] 0709 12:11:06.004479 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:06.705826 (+701347us) service_pool.cc:224] Handling call
0709 12:11:07.416792 (+710966us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b540038ead4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:07.417384 (+   592us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1279971712}
I20260709 12:11:07.462805 16051 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:07.463120 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.463860 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.464602 12531 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:07.418460 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:07.443912 12529 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:07.466063 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.466914 12603 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:11:07.467113 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.467468 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.449152 16070 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:07.419750 12605 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:07.420838 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:07.469026 16070 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:07.468174 12529 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:07.470572 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.471441 12529 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 7: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:07.468349 12597 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:07.472644 12680 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:11:07.448913 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:07.473708 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:07.468122 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.474967 16062 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:07.475538 16062 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:07.475975 16062 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:07.476099 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.477020 12602 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 6: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:07.478631 12522 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:07.478969 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.444088 12526 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:11:07.480626 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.481587 12526 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260709 12:11:07.481882 12523 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:07.483510 12526 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:11:07.474484 12572 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:07.461660 15876 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:07.486289 15994 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:11:07.486527 13305 leader_election.cc:336] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.555s (SENT)
I20260709 12:11:07.474475 12529 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 7: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:07.479614 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.466915 12530 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:07.443804 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.506786 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.511185 12524 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260709 12:11:07.495229 15994 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:07.522388 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 47763.6us
W20260709 12:11:07.516570 12646 leader_election.cc:336] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 2.046s (SENT)
I20260709 12:11:07.532253 12646 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:07.532639 16066 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:07.533211 16066 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:07.533634 16066 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:07.534382 16055 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:07.534413 12528 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:07.535408 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.536275 12530 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 7: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:07.536352 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.537177 12528 raft_consensus.cc:2410] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:11:07.537113 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.540482 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.541502 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.542550 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.543541 12524 raft_consensus.cc:2410] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 9 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260709 12:11:07.542582 12523 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
W20260709 12:11:07.529532 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:07.437173 15977 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.547886 12525 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:07.549052 15977 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:07.551793 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 440024us
I20260709 12:11:07.553843 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.555353 15876 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:07.558380 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:07.559614 16055 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:11:07.560061 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 52255.2us
W20260709 12:11:07.562599 12496 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.012s	sys 0.024s
W20260709 12:11:07.563097 12496 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.012s	sys 0.024s
W20260709 12:11:07.563562 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 561494us
I20260709 12:11:07.563906 16051 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:07.567842 16066 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.568980 12494 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:07.569321 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:07.571828 12671 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:07.571880 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.526556 12531 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 6: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:07.572574 12526 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:11:07.523244 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.577425 12674 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:11:07.578449 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:07.579576 12680 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 7: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:07.581096 16066 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:07.582800 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.584010 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.584661 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.585609 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.586511 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:07.577479 16024 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:07.587275 16024 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:07.588071 16024 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:07.588999 12523 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:07.589536 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.590294 12600 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:07.581705 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.594357 12597 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:11:07.595758 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.596640 12603 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:11:07.586483 12606 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 8: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:07.587982 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.598584 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:07.599272 16070 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:07.600394 12646 leader_election.cc:336] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 2.108s (SENT)
I20260709 12:11:07.600816 12602 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 6: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:07.602586 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:07.599556 12680 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:07.587534 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.574424 12529 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:11:07.608064 12496 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:07.608532 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.587337 12675 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:07.610731 12524 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
W20260709 12:11:07.625485 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 63401us
I20260709 12:11:07.627414 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:07.629245 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54007.3us
I20260709 12:11:07.623201 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:07.639573 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:07.642459 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:07.643198 12674 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:07.644001 12571 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:07.647619 12643 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:07.649255 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:07.649961 12598 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:07.650523 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.498747 16010 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:07.654307 12606 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:11:07.655519 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:07.658442 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50563.4us
I20260709 12:11:07.660809 12496 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:07.661279 12598 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
W20260709 12:11:07.661989 16072 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.129s	user 0.000s	sys 0.003s
I20260709 12:11:07.662763 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.663589 12604 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:11:07.651160 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:07.661929 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.653398 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:07.667263 12603 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 6: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:07.628930 12671 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:07.668293 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:07.668874 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 177215us
I20260709 12:11:07.669222 12601 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: [7.4-8.5]
I20260709 12:11:07.669318 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:07.688584 12605 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:11:07.649502 16062 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:07.672016 12646 leader_election.cc:336] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 2.233s (SENT)
I20260709 12:11:07.675076 12600 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
W20260709 12:11:07.675889 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:07.677700 15977 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:07.687346 12680 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:07.696223 16070 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:07.651311 16055 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:11:07.697359 16062 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:07.694049 15977 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:07.698951 15977 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:07.699492 16066 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260709 12:11:07.693168 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49311.9us
W20260709 12:11:07.701308 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 108079us
I20260709 12:11:07.697405 12496 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:07.702443 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:07.651629 16042 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:07.666627 16010 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:07.707456 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:07.686074 16051 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:11:07.711701 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 140044us
I20260709 12:11:07.713822 12527 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:07.716694 12679 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:07.719816 13305 leader_election.cc:336] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.706s (SENT)
I20260709 12:11:07.720594 13305 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:07.722896 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:07.725100 16071 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:07.726254 16068 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:07.726770 16068 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:07.727180 16068 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:07.708207 16042 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:07.729753 16042 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:07.716091 16051 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:07.730760 16051 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:07.731802 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:07.733263 16013 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:07.733343 12496 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:07.733857 16013 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:07.734398 16013 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:07.737758 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:07.738448 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:07.742556 15926 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:11:07.743083 15926 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:07.743428 15926 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:07.737849 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:07.746716 13337 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:07.747319 16011 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:07.747776 16011 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:07.748183 16011 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:07.652678 16006 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:11:07.725701 16071 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:07.758576 16071 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:07.761138 12494 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:07.762207 16014 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:07.762394 12675 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:07.759219 12496 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:07.764480 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:07.790047 12494 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:07.792543 16064 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:07.793030 16064 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:07.793423 16064 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:07.794545 16037 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:07.794637 15925 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:11:07.794878 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:07.795406 16021 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:07.796312 12494 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:07.796613 16037 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:07.798030 15925 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:11:07.799032 12572 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:07.799674 16037 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:07.800752 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 81003.2us
I20260709 12:11:07.764254 16014 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.025s
I20260709 12:11:07.803849 15925 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:07.804240 15925 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:07.805383 15925 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.802654 13304 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:07.810097 13337 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:07.813491 12674 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
W20260709 12:11:07.813607 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 141662us
I20260709 12:11:07.816488 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:07.837121 16010 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:07.837211 15996 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:07.837939 15996 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:07.837811 16010 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:07.838616 15996 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:07.838958 16010 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:07.841569 16051 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.843015 12602 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:07.844295 16051 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:07.848846 16024 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:07.850644 12675 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:07.853235 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:07.851946 12602 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:07.855381 12572 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:07.853818 12569 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:07.856395 16070 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:07.808405 15925 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:07.859908 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 100889us
W20260709 12:11:07.865562 13305 leader_election.cc:336] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.797s (SENT)
I20260709 12:11:07.861080 16070 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:07.873570 16070 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:07.876585 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:07.878031 15824 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:07.879159 16046 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:07.879685 16046 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:07.880442 16046 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:07.882362 16041 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:07.882963 16041 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:07.883587 16041 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:07.884370 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 164752us
I20260709 12:11:07.885584 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:07.886065 16064 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:07.886677 12572 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:07.888499 16006 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:07.900513 12496 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:07.901171 15977 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.894642 16051 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:07.902346 15970 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:11:07.902614 16051 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:07.916684 16051 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:07.918296 12679 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:07.896663 16066 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:07.903366 15970 raft_consensus.cc:697] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:07.920574 16066 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:07.921233 15970 consensus_queue.cc:237] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.922192 16066 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:07.923489 16024 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:07.905615 16065 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:07.906924 15977 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:07.906852 12571 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:07.908102 15913 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:07.927409 15913 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:07.852025 16042 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:07.929788 16042 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:07.930228 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:07.930464 15994 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:07.931782 13330 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:07.934098 15994 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:07.834172 15926 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.948385 15926 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:07.951681 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 135282us
I20260709 12:11:07.961462 16014 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.961373 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.961802 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.962258 12646 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:07.961401 16011 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.960642 15977 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:07.961026 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.962788 12528 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:07.962193 16014 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:07.966475 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 64409.2us
I20260709 12:11:07.968709 15977 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.970134 16011 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:07.970302 12527 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:07.971038 15977 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:07.963555 16042 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:07.971671 16014 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:07.964955 16006 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.007s
I20260709 12:11:07.967504 16064 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:07.963380 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:07.954275 13305 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:07.977320 12524 raft_consensus.cc:2410] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:11:07.983162 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:07.985430 12524 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:07.991178 16010 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:07.987113 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:07.929703 16013 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:07.981045 16055 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.019s
W20260709 12:11:08.009627 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:08.014842 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:11:08.014901 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 180221us
W20260709 12:11:08.016404 13305 leader_election.cc:336] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.858s (SENT)
I20260709 12:11:08.016839 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:07.981916 15913 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:11:07.993084 15994 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:08.027424 12646 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:08.034255 16070 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:08.038151 12525 raft_consensus.cc:2368] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10.
I20260709 12:11:08.038048 15977 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:08.046293 15942 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:08.051402 16013 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:08.051793 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:07.995764 16071 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:08.041759 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54753.2us
I20260709 12:11:08.055182 12646 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:11:08.057487 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 170963us
W20260709 12:11:08.063458 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 48650.4us
I20260709 12:11:08.067239 16065 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:08.072873 16065 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:08.077394 16006 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:08.077648 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 61288us
I20260709 12:11:08.078258 16006 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:08.080651 16006 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:08.057396 15977 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:08.082684 15977 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:08.083701 16071 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:08.086476 12494 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:08.088831 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:08.091732 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:08.092046 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:08.061635 15942 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:08.059449 16037 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:08.099143 12496 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:08.039757 16041 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:08.102895 16070 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:08.103785 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 197068us
I20260709 12:11:08.104727 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:08.099535 15942 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:08.107218 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:08.107945 16011 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:08.071085 16049 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:08.114451 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:08.087718 16042 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260709 12:11:08.121652 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 66628.3us
W20260709 12:11:08.122779 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:08.125325 16037 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:08.125711 13305 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:08.127947 16041 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:08.052911 16068 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:08.131217 16010 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:08.129230 16046 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:08.139838 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 69041.9us
I20260709 12:11:08.152837 12679 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 9. Current term is 10. Ops: [7.6-9.7]
I20260709 12:11:08.141515 16049 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:08.150964 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 43840.2us
I20260709 12:11:08.096115 15996 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:08.155076 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:08.153384 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:08.140589 16011 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:08.168610 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:08.148984 15925 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:08.169317 12671 raft_consensus.cc:2376] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8. Re-sending same reply.
I20260709 12:11:08.171672 12674 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260709 12:11:08.170571 12496 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:08.152652 16064 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:08.121677 16051 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:08.155611 16049 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:08.162470 15876 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:08.174225 16041 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:08.188405 16068 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:08.188705 16041 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:08.174378 16062 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:08.189564 16062 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:08.190037 16062 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:08.189735 16041 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:08.187469 16084 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.005s
I20260709 12:11:08.179239 15926 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:08.154568 16037 consensus_queue.cc:1243] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:11:08.181804 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:08.182951 15913 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:11:08.184135 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 58530.5us
I20260709 12:11:08.173950 12494 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:08.176964 12496 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:08.178546 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:08.185213 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:08.165966 16010 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:08.179920 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:08.199798 12677 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:08.198046 12672 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:08.201506 12679 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:11:08.173455 16011 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:08.181960 15996 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:08.192642 15876 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:08.217351 15876 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:08.204150 12496 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:08.204794 16013 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:08.206450 12646 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:08.187934 16046 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:08.210593 15926 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:08.214031 16085 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:11:08.214551 15970 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:08.211894 16021 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:11:08.204123 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:08.234463 13337 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:08.193629 15925 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:08.226506 12646 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:08.227607 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:08.226641 16088 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:08.236789 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:08.244705 12678 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:08.236825 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:08.238025 15994 raft_consensus.cc:3055] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:08.238461 16013 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:08.243300 12646 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:08.243768 15970 raft_consensus.cc:697] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:08.212584 16037 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:08.244287 16088 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:08.250084 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:08.258289 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:11:08.266050 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 98125.7us
I20260709 12:11:08.258984 13337 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
W20260709 12:11:08.260280 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 171716us
I20260709 12:11:08.260830 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:08.268102 12671 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 9: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:08.261694 16042 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:11:08.261960 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:08.262141 16055 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:08.262526 16013 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:08.257051 16014 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:08.263523 12674 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:08.267391 16088 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:08.270911 16006 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:08.270923 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:08.276404 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:08.277944 12672 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
I20260709 12:11:08.282730 12496 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:08.296571 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:08.282932 12673 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:08.289031 12677 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:11:08.296554 12678 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:08.290489 12680 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:08.300060 12676 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:11:08.303328 15876 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:08.298767 16014 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:08.304288 15925 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:08.298506 16006 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:08.306008 16006 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:08.291946 12646 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:08.232679 16084 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 2, Time since last communication: 0.035s
I20260709 12:11:08.250341 16085 raft_consensus.cc:697] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:08.249198 15994 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:08.239427 15926 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.3, Last appended by leader: 3, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:08.289194 16042 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:08.316833 15994 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:08.317667 16042 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:08.318610 15994 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:08.322233 16068 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:08.322739 16068 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:08.322872 12674 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:08.323149 16068 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:08.313624 16013 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:08.315043 12427 catalog_manager.cc:5697] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 5 to 8. New cstate: current_term: 8 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:11:08.326786 12427 catalog_manager.cc:5697] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:08.328416 12427 catalog_manager.cc:5697] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 4 to 7, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 7 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:08.330706 16042 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:08.332492 12679 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 6: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:08.334400 16042 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:08.334115 15970 consensus_queue.cc:237] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:08.341784 16049 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:08.342274 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49093.4us
I20260709 12:11:08.315222 16085 consensus_queue.cc:237] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:08.293430 16037 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:08.393090 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 165541us
I20260709 12:11:08.346787 16051 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:08.346885 15977 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:08.282296 16011 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:08.350661 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 65550.3us
I20260709 12:11:08.351801 12676 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:08.354209 12678 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:08.361490 12496 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:08.358980 12571 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:11:08.367336 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:08.370728 16014 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:08.374146 12427 catalog_manager.cc:5697] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 4 to 6, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 6 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:11:08.380678 12572 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:08.354058 16086 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.159s	user 0.000s	sys 0.001s
W20260709 12:11:08.400949 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:08.396430 12643 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:11:08.395390 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 125030us
I20260709 12:11:08.405038 12680 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
W20260709 12:11:08.405712 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 144372us
W20260709 12:11:08.414353 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52992.4us
I20260709 12:11:08.411136 16042 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:11:08.435565 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 124501us
I20260709 12:11:08.442445 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:08.419341 12496 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:08.441054 12646 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:08.449646 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:08.417732 16014 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:08.405457 16062 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:08.486371 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:08.436286 15913 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:08.458722 12496 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:08.500315 16087 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
I20260709 12:11:08.498412 12427 catalog_manager.cc:5697] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 5 to 9, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 9 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:11:08.501397 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 61161.4us
I20260709 12:11:08.444988 16042 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:08.454156 16024 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:08.514560 16042 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:08.514823 16024 raft_consensus.cc:697] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:11:08.511384 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 55757us
I20260709 12:11:08.457495 16014 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:08.459551 15876 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:08.517393 16014 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:08.517503 15876 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:08.518384 15876 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:08.517558 12496 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:08.526717 16013 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:08.526909 12496 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:08.527782 15926 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:08.528266 15926 raft_consensus.cc:697] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:08.529227 15926 consensus_queue.cc:237] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:08.494602 16051 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:11:08.493428 16037 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:08.496418 15913 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:08.470283 16006 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:08.516386 16024 consensus_queue.cc:237] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:08.562793 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:08.537490 15925 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:08.565836 12427 catalog_manager.cc:5697] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 6 to 7, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 7 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:11:08.568511 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 283043us
I20260709 12:11:08.536649 16013 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:11:08.580904 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 70237.6us
W20260709 12:11:08.586855 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:08.554556 16051 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:08.625487 16051 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:08.626823 12572 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:08.627700 15996 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:08.628130 15996 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:08.628506 15996 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:08.629441 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:08.590092 15970 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:08.590416 16013 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:11:08.634649 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 238295us
I20260709 12:11:08.591535 16006 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:08.603616 16062 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:08.568091 15913 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:08.604956 16014 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:08.606079 16099 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:08.608875 16064 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:08.612260 16049 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:08.611428 12427 catalog_manager.cc:5697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 5 to 6, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 6 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:08.615087 12525 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:08.616236 15977 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:08.617588 16011 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:08.570842 16042 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:08.618072 12429 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58980 (request call id 182) took 28598 ms (client timeout 15000 ms). Trace:
I20260709 12:11:08.621064 15942 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:08.623665 16068 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:08.618008 15925 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:08.635670 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:11:08.679754 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 50380us
W20260709 12:11:08.637702 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 220766us
W20260709 12:11:08.664275 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 113925us
W20260709 12:11:08.664687 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:08.666445 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:08.684170 12427 catalog_manager.cc:5697] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 6 to 8, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 8 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:11:08.670483 12429 rpcz_store.cc:269] 0709 12:10:40.019455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:40.019774 (+   319us) service_pool.cc:224] Handling call
0709 12:10:41.950893 (+1931119us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b54000104d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148898a090 00007f1488a28f3a 00007f148e7dee9c 00007f148e8dd04e 00007f148cd9e2a4 00007f148cd9e232 00007f148cd9e161 00007f148cd9e0dc 00007f148cd9d392 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:08.617796 (+26666903us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:11:07.340414 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:11:07.341560 (+  1146us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:11:07.341623 (+    63us) write_op.cc:435] Acquired schema lock
0709 12:11:07.341662 (+    39us) tablet.cc:662] Decoding operations
0709 12:11:07.343734 (+  2072us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:11:07.343862 (+   128us) write_op.cc:641] Partition lock acquired for write op
0709 12:11:07.343897 (+    35us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:11:07.345823 (+  1926us) tablet.cc:701] Row locks acquired
0709 12:11:07.345878 (+    55us) write_op.cc:260] PREPARE: finished
0709 12:11:07.346098 (+   220us) write_op.cc:270] Start()
0709 12:11:07.346338 (+   240us) write_op.cc:276] Timestamp: P: 1783599067346013 usec, L: 0
0709 12:11:07.346398 (+    60us) op_driver.cc:348] REPLICATION: starting
0709 12:11:07.378978 (+ 32580us) log.cc:844] Serialized 7511 byte log entry
0709 12:11:07.689604 (+310626us) op_driver.cc:464] REPLICATION: finished
0709 12:11:07.814789 (+125185us) write_op.cc:301] APPLY: starting
0709 12:11:07.814897 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 12:11:07.815348 (+   451us) tablet.cc:1370] finished BulkCheckPresence
0709 12:11:07.815508 (+   160us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:11:07.944567 (+129059us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:11:07.945439 (+   872us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=19
0709 12:11:07.945511 (+    72us) write_op.cc:312] APPLY: finished
0709 12:11:07.993319 (+ 47808us) log.cc:844] Serialized 131 byte log entry
0709 12:11:08.013328 (+ 20009us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:08.038645 (+ 25317us) write_op.cc:454] Released schema lock
0709 12:11:08.057267 (+ 18622us) write_op.cc:341] FINISH: Updating metrics
0709 12:11:08.088961 (+ 31694us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:08.089001 (+    40us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4407707,"reported_tablets":45,"spinlock_wait_cycles":47988992,"child_traces":[["op",{"apply.queue_time_us":120950,"apply.run_cpu_time_us":9178,"apply.run_wall_time_us":274126,"num_ops":19,"prepare.queue_time_us":78214,"prepare.run_cpu_time_us":17372,"prepare.run_wall_time_us":162645,"raft.queue_time_us":25245,"raft.run_cpu_time_us":17837,"raft.run_wall_time_us":32728,"replication_time_us":342952,"thread_start_us":252778,"threads_started":4,"wal-append.queue_time_us":123648}]]}
W20260709 12:11:08.692047 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:08.619961 16088 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:08.668501 16085 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:08.697479 12671 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:08.697774 12427 catalog_manager.cc:5697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 5 to 7, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 7 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:08.589535 15876 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:08.695180 12524 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:08.670590 16041 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:08.699735 12427 catalog_manager.cc:5697] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 reported cstate change: config changed from index 2 to 5, e56d487365394ac5ac5f1002926a5645 (127.11.13.132) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:11:08.687284 16099 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:08.710906 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:11:08.713855 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 78372.7us
I20260709 12:11:08.701301 15926 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:08.722743 16100 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:08.723357 16100 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:08.723773 16100 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:08.726497 12572 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:08.742084 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:08.737649 12525 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:08.693650 16065 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.024s
I20260709 12:11:08.742489 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:08.813596 12679 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:08.839560 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 111647us
I20260709 12:11:08.740309 12671 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:08.814630 12675 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:11:08.788565 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:08.742955 12680 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:11:08.717063 12427 catalog_manager.cc:5697] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 5 to 6, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 6 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:11:08.844226 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 142586us
I20260709 12:11:08.734520 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:08.841620 12678 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:08.738798 16070 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 8.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.108s
W20260709 12:11:08.816398 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52998us
I20260709 12:11:08.731956 16071 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:08.845013 12427 catalog_manager.cc:5697] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 4 to 9. New cstate: current_term: 9 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:11:08.849098 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 142086us
I20260709 12:11:08.707355 15994 consensus_queue.cc:1059] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.113s
W20260709 12:11:08.738425 16093 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:11:08.729987 16088 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:08.888705 12571 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:08.863329 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:08.889516 12646 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:08.845211 16021 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:11:08.868108 16014 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:08.872910 15970 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:08.843405 15977 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:08.875981 16064 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:08.877600 12496 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:08.878453 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:08.902319 12671 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:08.879500 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:08.732329 15876 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:08.735368 16068 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:08.905151 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:08.863087 16010 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:11:08.894893 16109 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:08.731029 16099 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:08.901225 12679 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:08.813892 15926 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:08.911885 15876 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:08.923563 15876 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:08.924067 12679 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 3. Preceding OpId from leader: term: 10 index: 4. (index mismatch)
I20260709 12:11:08.712034 16041 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:08.737586 15942 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:08.912113 16071 raft_consensus.cc:2764] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:11:08.927016 16106 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:08.908071 12524 raft_consensus.cc:2376] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6. Re-sending same reply.
I20260709 12:11:08.927908 16041 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:08.927711 16106 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:08.894923 16006 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:11:08.928517 16014 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:08.928702 16006 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:08.929160 16024 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:08.926079 15876 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:08.929160 16006 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:11:08.739220 15925 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:08.899616 16037 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:11:08.932312 12678 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:08.945834 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:08.925974 16088 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:08.905192 15977 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:08.951442 12671 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:08.966017 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:08.953877 12496 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:08.935377 12646 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:11:08.962833 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 83403.5us
I20260709 12:11:08.901947 16055 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:11:08.966920 12524 raft_consensus.cc:2393] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:08.930073 16106 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:08.953140 16011 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:08.983217 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:08.966239 15977 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:08.983382 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 67139.3us
I20260709 12:11:08.983043 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:08.962986 16014 raft_consensus.cc:697] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:08.954152 15996 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:08.989241 12646 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:08.988976 13333 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:08.990481 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.001545 12602 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:08.990773 16014 consensus_queue.cc:237] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:08.991295 16101 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:08.978951 16042 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:08.994166 16102 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:08.996860 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:08.997500 12675 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
W20260709 12:11:09.013219 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 116082us
I20260709 12:11:08.992911 16051 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:09.012758 16021 consensus_queue.cc:1243] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:11:09.017369 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:08.987591 15942 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.031s
I20260709 12:11:09.018986 16049 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
I20260709 12:11:09.024524 12601 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:09.026619 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.027724 12598 raft_consensus.cc:2368] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6.
I20260709 12:11:09.028126 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.028734 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.029400 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.030072 12602 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:09.018918 12646 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:09.019415 12496 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:09.035917 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 156615us
I20260709 12:11:09.037261 12598 raft_consensus.cc:2393] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 7: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:09.025990 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.013866 16106 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:09.034207 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:09.043848 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.044193 15926 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:09.044552 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.045148 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.045861 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:09.046677 12606 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:09.047480 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.045408 12604 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:09.048357 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:09.049006 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:08.988154 16011 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:09.043071 12569 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:09.018201 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:09.064338 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.061021 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.017786 16101 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:09.078749 12605 raft_consensus.cc:2368] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6.
I20260709 12:11:09.067253 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.067514 12646 leader_election.cc:400] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10.
I20260709 12:11:09.079603 12646 leader_election.cc:403] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:09.073320 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:09.034044 16064 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:09.075445 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.076195 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.034996 16049 raft_consensus.cc:2764] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:11:09.041908 16041 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:09.077698 12675 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:11:09.089493 12597 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
W20260709 12:11:09.090703 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 55453.3us
I20260709 12:11:09.081847 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:09.041702 16106 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:09.083689 15970 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:11:09.105607 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 88492.7us
W20260709 12:11:09.107414 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 173318us
I20260709 12:11:09.145951 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:09.146921 12676 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:11:09.038439 16037 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
I20260709 12:11:09.136029 12599 raft_consensus.cc:2376] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7. Re-sending same reply.
W20260709 12:11:09.166849 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 115084us
I20260709 12:11:09.143379 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:09.089356 16070 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:09.150885 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:09.151489 16068 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:09.157274 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:09.308585 12524 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:09.158879 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:09.159790 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:09.310743 12527 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:09.164294 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:09.173070 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:09.312669 12680 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:09.040051 16010 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:09.314250 16010 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:11:09.246304 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:09.248656 16101 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:09.316418 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:09.316625 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:09.317180 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:09.317613 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:09.317757 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.318416 12602 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 9: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:09.319649 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:09.318593 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:09.320619 12605 raft_consensus.cc:2393] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:09.243026 16119 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:09.320945 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:09.320627 12602 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 8: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:11:09.207712 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 126138us
I20260709 12:11:09.148711 16106 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:09.172407 16024 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:09.323024 12571 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:09.322971 16024 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:09.167973 15926 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:09.324451 12571 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:09.227510 16120 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:09.173810 16041 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:09.284704 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:09.325924 13331 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 9: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:09.151111 16064 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:09.197997 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:09.142494 15970 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:09.326974 12598 raft_consensus.cc:2368] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7.
I20260709 12:11:09.327248 15970 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:09.326685 16041 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:09.318365 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.328751 12604 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:09.329582 16085 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:11:09.224524 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:09.330040 16085 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:09.330487 16085 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:09.334714 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:09.335897 12604 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 7: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:09.336954 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:09.338074 12602 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:11:09.338711 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:09.339550 12602 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:09.326884 16064 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:09.343930 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:09.283581 12526 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: [7.3-7.3]
I20260709 12:11:09.346460 12598 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:11:09.179267 16070 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:09.347646 16070 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:09.339946 12602 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:09.173251 16088 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10.
I20260709 12:11:09.318506 12603 raft_consensus.cc:2393] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:09.321373 16119 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 8 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:09.277890 16011 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:09.350901 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.351532 12598 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:09.351408 16119 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:09.353186 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:09.135589 16006 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:09.354545 12601 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:09.356928 16006 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:09.357990 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.359664 12598 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 7: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:09.349471 12602 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:09.325369 16120 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:09.361208 16120 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:09.363996 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.365815 12604 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:09.361171 12602 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:09.376394 12526 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:11:09.379420 12526 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:11:09.383670 12526 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:11:09.385538 15970 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:09.391826 12526 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:09.404024 12597 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 7: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:09.405973 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.409137 15994 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:09.435247 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:09.420815 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 314120us
I20260709 12:11:09.423336 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:09.423794 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:09.424382 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:09.425014 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:09.425640 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:09.429668 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:09.433320 12602 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:11:09.438606 16114 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:09.438894 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:09.438894 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:09.492467 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.282450 16084 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:09.495257 12680 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 8: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:09.497550 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:09.492522 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 349173us
W20260709 12:11:09.498597 12643 leader_election.cc:336] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.654s (SENT)
I20260709 12:11:09.506366 12527 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:09.513152 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:09.514634 13304 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:09.515906 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:09.496073 12599 raft_consensus.cc:2368] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8.
I20260709 12:11:09.522531 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.522712 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:09.508708 12675 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:11:09.523773 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:09.430946 15913 consensus_meta.cc:287] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.761s	user 0.014s	sys 0.000s
I20260709 12:11:09.527904 12529 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:09.534381 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:09.521118 13335 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:11:09.536227 12597 raft_consensus.cc:2393] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:09.422696 16109 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:09.545594 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:09.555517 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:09.567974 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:09.569703 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 127153us
I20260709 12:11:09.413164 16068 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:09.577517 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:09.572033 12680 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:11:09.572397 12672 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:09.573019 12526 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:09.505563 15994 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:09.588436 12674 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:11:09.574260 12523 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:09.429632 16120 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:09.594422 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:09.594383 16104 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.539s	user 0.000s	sys 0.030s
I20260709 12:11:09.595384 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:09.613890 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:09.554746 16102 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:09.621588 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:09.619736 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.623236 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:09.623101 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 202862us
I20260709 12:11:09.547147 16114 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:09.649041 16068 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:09.621620 15996 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:09.654222 16051 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:09.654686 16100 consensus_meta.cc:287] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.930s	user 0.012s	sys 0.002s
I20260709 12:11:09.576323 16099 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:09.662984 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:09.642655 12646 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:09.667191 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:09.544826 15913 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:09.672554 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.627544 12524 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:11:09.666234 16116 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:09.663758 16119 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:11:09.677930 12643 leader_election.cc:336] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.736s (SENT)
I20260709 12:11:09.679034 12676 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:11:09.681730 12602 raft_consensus.cc:2376] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9. Re-sending same reply.
I20260709 12:11:09.684607 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.943044 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:09.695369 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:09.709606 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:09.681041 16104 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.003s	sys 0.030s
W20260709 12:11:09.685166 16133 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.065s	user 0.000s	sys 0.001s
I20260709 12:11:09.797248 16102 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:09.960629 16102 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:09.961987 12600 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 8: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:09.963277 15913 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:09.949589 16070 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.601s	user 0.022s	sys 0.000s
I20260709 12:11:09.971617 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:09.858359 15977 consensus_meta.cc:287] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.851s	user 0.003s	sys 0.010s
I20260709 12:11:09.825569 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:09.982844 12673 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 9: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:09.851418 16120 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:09.792913 16128 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:09.756469 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:09.807459 15970 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:09.794658 16121 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.120s	user 0.000s	sys 0.002s
I20260709 12:11:09.784024 16100 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:09.705084 15994 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:09.926873 12680 raft_consensus.cc:2393] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:09.792112 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.837643 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:09.929556 12526 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
W20260709 12:11:09.834168 16106 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.512s	user 0.005s	sys 0.005s
I20260709 12:11:09.994652 12603 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:09.842082 12496 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:09.794517 16126 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:09.996206 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:09.833091 16105 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:09.812844 16085 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:09.844064 16114 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:09.935020 12527 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:09.937026 12522 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:11:09.998670 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:09.999236 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:09.997906 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:09.937906 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 249421us
I20260709 12:11:10.000777 12528 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:09.898870 12523 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:09.947300 12675 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 7: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:09.999401 12680 raft_consensus.cc:2368] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9.
I20260709 12:11:09.690505 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:10.002033 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:10.002183 12601 raft_consensus.cc:2410] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:11:09.946836 12598 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:11:10.001029 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 159049us
I20260709 12:11:09.952749 12678 raft_consensus.cc:1217] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Deduplicated request from leader. Original: 7.5->[9.6-9.6]   Dedup: 9.6->[]
I20260709 12:11:09.953372 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:09.978341 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:10.013181 12678 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:11:09.997412 16126 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:10.015997 12677 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 6: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:11:10.014680 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 341260us
W20260709 12:11:09.980559 12494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.679s	user 0.009s	sys 0.019s
I20260709 12:11:09.804461 16068 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:09.982457 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:09.786535 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:09.792547 16134 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.098s	user 0.000s	sys 0.001s
W20260709 12:11:09.985946 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 360410us
I20260709 12:11:09.964607 16116 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 9 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:10.027992 16116 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:10.030355 16099 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:11:09.987612 16110 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:10.031652 16110 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:10.032689 16110 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:10.033087 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:09.999570 16129 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:09.986527 15977 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:10.034909 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:10.016033 16104 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:10.035634 16085 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:10.035914 16104 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:10.036613 16104 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:09.952237 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:09.819213 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:09.997217 16089 raft_consensus.cc:3055] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:09.997462 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:10.039963 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:10.040872 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:10.041075 12675 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:10.041603 12522 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 9: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:09.997964 12673 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 8: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:11:10.016959 12494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.716s	user 0.009s	sys 0.019s
I20260709 12:11:10.017845 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:10.043251 12523 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 7: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:10.017823 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:10.043721 12680 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:10.015671 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:10.025604 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:10.028642 16126 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:10.034605 16129 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:10.036489 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:10.025873 12604 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:10.037828 12676 raft_consensus.cc:2393] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:09.947559 12606 raft_consensus.cc:2368] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10.
I20260709 12:11:09.999821 16120 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:09.950274 12597 raft_consensus.cc:2410] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
I20260709 12:11:10.038166 12524 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 8: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:09.999259 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:09.986367 16070 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:10.065335 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:10.016525 16105 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:10.070503 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:10.066490 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:10.066658 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:10.073408 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:10.076462 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:10.042230 12528 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:10.067682 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:10.070246 12673 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 5. Current term is 6. Ops: [5.4-5.4]
W20260709 12:11:10.076939 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 73817.6us
I20260709 12:11:10.077486 12522 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:10.036551 15977 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:10.039671 16089 raft_consensus.cc:740] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:10.079907 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:10.081524 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:10.081974 12678 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:11:10.094089 12673 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:10.077466 16105 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:10.021708 16055 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:11:10.085428 12523 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:10.107681 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:10.084187 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:09.989373 16128 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:10.011901 16106 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:10.083657 12528 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:10.077438 12597 raft_consensus.cc:2368] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8.
I20260709 12:11:10.073375 15994 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:10.084690 12643 leader_election.cc:336] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.774s (SENT)
I20260709 12:11:10.163690 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:10.095186 12603 raft_consensus.cc:2368] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 8. Current term is 9.
I20260709 12:11:10.164479 12523 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:11:10.165059 12601 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:10.095373 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:10.080001 16110 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:10.105036 12602 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:11:10.084153 16140 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:10.085407 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 55565.2us
I20260709 12:11:10.157763 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:10.215355 12597 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:10.214457 12602 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:10.216169 12606 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:10.215804 12599 raft_consensus.cc:2376] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12. Re-sending same reply.
I20260709 12:11:10.038663 16068 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:10.157855 13302 leader_election.cc:336] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.555s (SENT)
I20260709 12:11:10.219623 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:10.162479 12494 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:10.178411 12680 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:10.217931 12646 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:10.213389 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 110115us
I20260709 12:11:10.185928 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:10.220785 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 58423.7us
I20260709 12:11:10.077243 16120 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:10.214224 12601 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260709 12:11:10.110577 16105 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:10.159348 12522 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:11:10.092928 16099 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.113s
W20260709 12:11:10.176981 16147 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
W20260709 12:11:10.227047 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 142383us
I20260709 12:11:10.185792 16153 raft_consensus.cc:3055] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:11:10.235896 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 78131us
I20260709 12:11:10.166738 15977 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
W20260709 12:11:10.242295 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 155237us
W20260709 12:11:10.247972 12643 leader_election.cc:336] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.863s (SENT)
I20260709 12:11:10.248489 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:10.250809 13302 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.697s (SENT)
I20260709 12:11:10.251313 13302 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:10.247697 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:10.256960 16120 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:10.261998 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:10.262784 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2196) took 1178 ms. Trace:
I20260709 12:11:10.263293 16084 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:10.263751 12671 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: [9.4-9.4]
I20260709 12:11:10.253098 12673 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:11:10.232470 16149 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:10.265705 16105 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:10.268416 16010 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:10.269488 16070 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:10.269680 16010 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:10.271850 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:10.273344 12677 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:10.274901 12680 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:11:10.275041 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:10.276475 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:10.277141 12675 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:10.277189 12677 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
W20260709 12:11:10.277477 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:10.280196 12677 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:11:10.281435 16117 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:11:10.281977 12677 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:11:10.283694 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:10.263309 12679 rpcz_store.cc:276] 0709 12:11:09.084495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:09.106482 (+ 21987us) service_pool.cc:224] Handling call
0709 12:11:09.106823 (+   341us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:09.404627 (+297804us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:09.404661 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:09.428337 (+ 23676us) log.cc:844] Serialized 26 byte log entry
0709 12:11:09.983884 (+555547us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:11:09.983938 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:09.984034 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:10.262330 (+278296us) raft_consensus.cc:1685] finished
0709 12:11:10.262361 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:10.262663 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3960,"raft.queue_time_us":14148,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":119094,"thread_start_us":416120,"threads_started":1,"wal-append.queue_time_us":577396}
I20260709 12:11:10.288664 12679 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260709 12:11:10.289588 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:10.234311 16153 raft_consensus.cc:740] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:11:10.287742 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 69957.8us
I20260709 12:11:10.298061 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:10.299037 12673 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:10.299278 12646 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:10.244210 16116 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:10.309930 12598 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 8. Current term is 9. Ops: []
W20260709 12:11:10.315897 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 65122.1us
I20260709 12:11:10.315104 12679 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:10.318745 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:10.322949 16164 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:10.326479 12529 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:10.253046 16152 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:10.327808 16152 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:10.328763 16152 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:11:10.329273 16152 raft_consensus.cc:697] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:11:10.333843 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 85898.8us
I20260709 12:11:10.338399 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:11:10.333457 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 111960us
I20260709 12:11:10.340652 12494 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:10.358580 12603 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:10.367475 12603 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:10.224159 16140 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:10.369724 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:10.369939 13337 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 9 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:10.374255 13332 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:10.375483 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:10.375927 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:10.376667 12602 raft_consensus.cc:2393] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 6: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:10.379109 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:10.383489 12678 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:10.377743 12494 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:10.387645 13335 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: []
I20260709 12:11:10.339540 16100 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:10.405457 16149 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:10.390321 16101 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:11:10.378283 12603 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:10.390734 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:10.416339 12599 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 8: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:10.391541 16158 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:11:10.422425 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 123282us
I20260709 12:11:10.423235 12646 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:10.374063 12606 raft_consensus.cc:3055] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:10.423962 12606 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:10.402793 16153 consensus_queue.cc:260] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:10.404703 16128 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:10.420325 12603 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:10.392957 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:10.423015 12494 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:10.434429 12522 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:10.435760 12523 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 8. Current term is 9. Ops: []
W20260709 12:11:10.441358 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:10.518846 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 95910.4us
I20260709 12:11:10.406963 16068 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:10.406046 16106 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:10.528661 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 151098us
I20260709 12:11:10.522439 13337 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:11:10.515769 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:10.531782 12599 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:10.524184 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:10.408618 16140 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:10.255293 16089 consensus_queue.cc:260] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:10.535413 16120 consensus_queue.cc:1059] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:11:10.547525 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:10.415767 16119 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:10.419939 15977 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:10.556070 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:10.555660 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:10.572044 12679 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
W20260709 12:11:10.574558 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 54337.5us
I20260709 12:11:10.574820 12524 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:11:10.576720 12673 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:10.559836 12599 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:10.516849 16100 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:10.581770 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:10.582531 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:10.582733 12522 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
I20260709 12:11:10.520726 16158 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:10.529000 12606 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:10.526481 16128 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:10.525208 16129 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:10.526671 16116 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.040s
I20260709 12:11:10.551601 16101 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:11:10.606560 16119 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:10.571512 16051 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:10.578917 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:10.578917 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:10.581233 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:10.613164 12678 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:10.600157 16070 consensus_queue.cc:1059] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.014s
I20260709 12:11:10.614573 16084 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:10.615152 16084 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:10.561486 16110 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:10.616459 16110 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:10.617592 16084 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:10.584376 16140 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:11:10.585098 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:10.586171 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:10.619513 16130 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.997s	user 0.001s	sys 0.003s
I20260709 12:11:10.619830 12524 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:10.622345 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:10.623020 12671 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:11:10.588099 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:10.623837 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:10.624936 12643 leader_election.cc:400] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6.
I20260709 12:11:10.624964 16165 raft_consensus.cc:2764] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:11:10.619534 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:10.626502 12679 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:10.592052 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:10.631150 12571 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:10.632419 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:10.633615 16114 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:10.634406 16114 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:11:10.634778 16114 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:10.635162 16114 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:10.635321 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:10.637253 12676 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: [6.4-7.5]
I20260709 12:11:10.639652 16116 raft_consensus.cc:3055] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:10.640154 16116 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:10.641191 16116 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:10.642666 16140 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:10.643119 16140 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:10.644132 12601 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:10.645579 12571 leader_election.cc:400] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9.
I20260709 12:11:10.406251 16149 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:10.646320 12525 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 4.
I20260709 12:11:10.435350 16164 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:10.600320 15996 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260709 12:11:10.664849 16149 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:10.602334 16157 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:10.602561 12603 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:11:10.439235 16014 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.143s
I20260709 12:11:10.665745 12603 pending_rounds.cc:85] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 4
I20260709 12:11:10.665838 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:10.666512 12603 pending_rounds.cc:107] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:11:10.667029 12603 raft_consensus.cc:2889] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:10.581172 16011 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.088s
I20260709 12:11:10.669339 16175 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:10.578610 15977 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:10.670199 16175 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:10.672151 15977 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:10.567049 16161 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.146s
I20260709 12:11:10.671921 16164 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:10.673516 16161 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:10.651477 16051 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:10.674281 16161 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:10.676040 16051 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:10.675800 16161 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:10.645664 16062 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:11:10.677081 12678 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:10.678094 12599 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:10.678597 15913 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:10.679855 15996 consensus_queue.cc:1243] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 9.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:11:10.680426 15996 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:10.680827 15996 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:10.680593 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:10.575344 16153 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:10.681878 15996 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:10.683422 15996 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:10.684330 16158 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:10.610626 16100 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:10.625319 12643 leader_election.cc:403] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:10.685793 16100 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:10.687362 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 62544.8us
I20260709 12:11:10.688297 16140 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:10.689658 16165 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6.
I20260709 12:11:10.690097 12524 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:10.668825 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:10.597959 12606 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:10.598905 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:10.624568 12671 raft_consensus.cc:2393] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:10.692703 12522 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:10.689899 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:10.646265 12571 leader_election.cc:403] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
W20260709 12:11:10.655637 12600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1977) took 1618 ms (client timeout 2059 ms). Trace:
I20260709 12:11:10.656755 16110 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:10.657500 16129 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:10.662376 16168 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:10.417572 16152 consensus_queue.cc:237] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:10.691566 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:10.681679 12599 raft_consensus.cc:2376] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12. Re-sending same reply.
I20260709 12:11:10.650235 12679 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:10.694579 12676 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: [6.4-7.5]
I20260709 12:11:10.690325 16106 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:11:10.693536 12600 rpcz_store.cc:269] 0709 12:11:09.025923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:09.030072 (+  4149us) service_pool.cc:224] Handling call
0709 12:11:10.641942 (+1611870us) spinlock_profiling.cc:243] Waited 895 ms on lock 0x7b54002f1054. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:10.642142 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2058873088}
I20260709 12:11:10.698486 12603 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:11:10.700647 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 55220.1us
W20260709 12:11:10.701900 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 152424us
W20260709 12:11:10.704365 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:10.705772 12678 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:11:10.708655 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:10.677708 16161 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:10.689699 16140 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:10.712497 12603 raft_consensus.cc:3055] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:10.712761 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 307) took 1055 ms. Trace:
I20260709 12:11:10.712984 12603 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:10.711701 16165 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:10.720934 12678 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:11:10.713212 12529 rpcz_store.cc:276] 0709 12:11:09.657473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:10.056741 (+399268us) service_pool.cc:224] Handling call
0709 12:11:10.326247 (+269506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:10.327039 (+   792us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:10.712546 (+385507us) spinlock_profiling.cc:243] Waited 247 ms on lock 0x7b5400015750. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:10.712698 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":568246528}
I20260709 12:11:10.717617 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:10.722412 12603 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:10.722266 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:10.724409 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:10.724824 13335 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:11:10.701987 15994 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:10.705026 16070 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9.
I20260709 12:11:10.710244 16116 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:10.727139 12679 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:10.726646 12599 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:11:10.740406 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:10.740710 12522 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:10.741070 12603 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:10.746742 12600 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:11:10.763026 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:10.741796 16110 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:10.766826 12603 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:11:10.767001 16125 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.969s	user 0.001s	sys 0.001s
I20260709 12:11:10.724047 16014 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:11:10.744283 16104 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.707s	user 0.011s	sys 0.000s
I20260709 12:11:10.726006 16176 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:10.776181 15913 raft_consensus.cc:3055] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:10.706492 16129 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:10.787268 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:10.778463 12643 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:10.802681 12606 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260709 12:11:10.803253 12606 pending_rounds.cc:85] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 4
I20260709 12:11:10.803588 12606 pending_rounds.cc:107] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:11:10.803936 12606 raft_consensus.cc:2889] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:10.789669 12571 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:10.817435 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:10.742312 15994 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:10.824988 16165 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:10.826217 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:10.834263 12679 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 8: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:10.806242 15913 raft_consensus.cc:740] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:10.787899 16110 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.042s
I20260709 12:11:10.840200 12679 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:11:10.744894 16149 consensus_queue.cc:1059] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
W20260709 12:11:10.829854 16135 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.113s	user 0.001s	sys 0.013s
I20260709 12:11:10.814726 16055 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:10.788149 16051 consensus_queue.cc:1059] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:11:10.801299 16084 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:11:10.852767 16011 raft_consensus.cc:3055] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:11:10.854362 13339 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:11:10.859498 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1877) took 1468 ms. Trace:
I20260709 12:11:10.802242 16104 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:10.822580 16175 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:10.874802 16011 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:10.882674 16011 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:10.889559 13337 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: []
I20260709 12:11:10.890275 16024 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:10.891136 16024 consensus_queue.cc:1243] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:11:10.806929 16176 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:10.876308 12530 rpcz_store.cc:276] 0709 12:11:09.391143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:09.394695 (+  3552us) service_pool.cc:224] Handling call
0709 12:11:09.395400 (+   705us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:09.395960 (+   560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:09.396001 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:09.396660 (+   659us) log.cc:844] Serialized 52 byte log entry
0709 12:11:09.516949 (+120289us) raft_consensus.cc:1645] Marking committed up to 2
0709 12:11:09.517010 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:09.517111 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:10.756256 (+1239145us) spinlock_profiling.cc:243] Waited 50.4 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:10.859137 (+102881us) raft_consensus.cc:1685] finished
0709 12:11:10.859160 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:10.859401 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9846,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":17148,"spinlock_wait_cycles":115942272,"thread_start_us":119732,"threads_started":1,"wal-append.queue_time_us":232550}
I20260709 12:11:10.902761 12572 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:10.886000 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:10.875725 15996 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:10.911566 13337 pending_rounds.cc:85] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 6
I20260709 12:11:10.922679 13337 pending_rounds.cc:107] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:11:10.923067 13337 raft_consensus.cc:2889] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:11:10.927095 16171 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
I20260709 12:11:10.927428 15994 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:10.928499 15996 raft_consensus.cc:697] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:10.929721 15996 consensus_queue.cc:237] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:10.933624 16119 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:10.934448 16119 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:10.936880 16119 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:10.938042 16119 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:10.938305 16163 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:10.938833 16119 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:10.938984 16163 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:10.939591 16104 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:10.941147 16129 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:10.945190 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 66147.5us
I20260709 12:11:10.905733 16110 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:11:10.967222 16110 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:11:10.967501 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:10.968360 16110 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:10.947768 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 45152.6us
I20260709 12:11:10.909049 16051 raft_consensus.cc:3055] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:10.958840 16159 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:10.961907 13333 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:10.962064 16084 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:11:10.962675 12604 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 1985) took 1536 ms (client timeout 1547 ms). Trace:
W20260709 12:11:10.984853 12604 rpcz_store.cc:269] 0709 12:11:09.426321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:09.426641 (+   320us) service_pool.cc:224] Handling call
0709 12:11:10.962507 (+1535866us) spinlock_profiling.cc:243] Waited 531 ms on lock 0x7b54002f1554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:10.962613 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1220168064}
I20260709 12:11:10.962771 15913 consensus_queue.cc:260] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:10.987028 15913 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:10.964021 12643 leader_election.cc:400] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6.
I20260709 12:11:10.991483 12643 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:10.992660 16055 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:11:10.994629 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:10.965054 16114 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:10.998252 16114 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:10.965495 16176 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:10.974397 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 184795us
I20260709 12:11:10.999747 16055 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:11:11.000336 16055 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:11.000869 16055 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
W20260709 12:11:11.001792 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:11.007818 16070 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:11:11.009222 16070 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:11.010293 12643 leader_election.cc:400] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7.
I20260709 12:11:11.011404 16181 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:11.017102 16181 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:11.018688 16181 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:11.021375 16181 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:11.012490 16070 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:11.022730 16070 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:11.024081 16070 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:11.005789 12571 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:11.027040 16114 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:11.027737 16114 raft_consensus.cc:697] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:11.031453 12571 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:11:11.034657 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:11.029109 16114 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:11.025882 16070 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:11.050352 12643 leader_election.cc:403] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260709 12:11:11.051471 16104 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7.
I20260709 12:11:11.053049 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:11.054409 16104 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:11:11.057443 12643 leader_election.cc:400] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8.
I20260709 12:11:11.074457 16182 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:11.102541 16182 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:11.103650 16182 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:11.106581 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:11.107295 12672 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:11.108062 16084 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:11.108600 16084 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:11.102000 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:11.115972 16126 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:11.121764 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:11.110836 16084 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:11.116966 16126 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:11.128616 16126 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:10.984362 16159 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:11.141407 16159 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:11.142797 16159 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:11.143087 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:11.144397 12604 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:11:11.078316 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:11.084645 16105 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:10.982304 16051 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:11.081854 16119 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:11.084662 12572 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:11.146494 16051 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:11.147454 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:10.970091 16110 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:11:11.088826 16167 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
W20260709 12:11:11.094338 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:11.148989 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1915) took 1486 ms. Trace:
I20260709 12:11:11.149405 12527 rpcz_store.cc:276] 0709 12:11:09.662805 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:10.057460 (+394655us) service_pool.cc:224] Handling call
0709 12:11:10.082534 (+ 25074us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:10.082878 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:10.082906 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:10.281034 (+198128us) spinlock_profiling.cc:243] Waited 5.69 ms on lock 0x7b2c00058028. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:11:10.281511 (+   477us) log.cc:844] Serialized 26 byte log entry
0709 12:11:10.423627 (+142116us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:10.527231 (+103604us) log.cc:844] Serialized 14 byte log entry
0709 12:11:10.527513 (+   282us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:10.527633 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:10.969217 (+441584us) spinlock_profiling.cc:243] Waited 18.3 ms on lock 0x7b18002c0740. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:11:11.148726 (+179509us) raft_consensus.cc:1685] finished
0709 12:11:11.148752 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:11.148923 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":127561,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":2973,"raft.queue_time_us":9859,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":9948,"spinlock_wait_cycles":91512832,"thread_start_us":141525,"threads_started":1,"wal-append.queue_time_us":246443}
I20260709 12:11:11.153155 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:11.153817 12680 raft_consensus.cc:2376] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9. Re-sending same reply.
I20260709 12:11:11.098033 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:11.098486 16179 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.004s
I20260709 12:11:11.122783 12601 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:11.135766 16011 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:11:11.136016 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:11.139290 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:11.179332 12598 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
W20260709 12:11:11.183414 13339 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 524) took 1881 ms (client timeout 1719 ms). Trace:
W20260709 12:11:11.183933 13339 rpcz_store.cc:269] 0709 12:11:09.302352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:09.302507 (+   155us) service_pool.cc:224] Handling call
0709 12:11:11.183184 (+1880677us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b54001261d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:11.183298 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2933510912}
I20260709 12:11:11.185683 16014 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:11.147744 16051 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:11.092430 15996 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:11.202890 16181 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:11:11.203029 16150 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.985s	user 0.005s	sys 0.000s
I20260709 12:11:11.203435 16181 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:11.204579 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1912) took 1655 ms. Trace:
I20260709 12:11:11.197500 16089 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:11.185771 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2118) took 1866 ms. Trace:
I20260709 12:11:11.206199 12605 rpcz_store.cc:276] 0709 12:11:09.319355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:09.321157 (+  1802us) service_pool.cc:224] Handling call
0709 12:11:09.321429 (+   272us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:09.591545 (+270116us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:09.591592 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:09.591654 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:09.591710 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:11.185418 (+1593708us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b540007a154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:11.185486 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:11.185660 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":620617600}
W20260709 12:11:11.094623 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:11.191202 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:11.080327 16127 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 1.285s	user 0.003s	sys 0.000s
I20260709 12:11:11.209048 16014 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:11.116066 16131 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.324s	user 0.000s	sys 0.002s
I20260709 12:11:11.163815 12602 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 10 index: 4. (term mismatch)
I20260709 12:11:11.211167 12602 pending_rounds.cc:85] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 3
I20260709 12:11:11.189585 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:11.211604 12602 pending_rounds.cc:107] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 9.4
I20260709 12:11:11.211880 12602 raft_consensus.cc:2889] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:11.164526 12646 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:11.214195 16163 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:11.214300 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 49893.1us
I20260709 12:11:11.215474 12646 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:11.147432 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 62884.4us
I20260709 12:11:11.167778 16109 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:11.218060 16181 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:11.170091 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:11.171283 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:11.173063 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 119835us
I20260709 12:11:11.175124 16084 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:11.204988 12531 rpcz_store.cc:276] 0709 12:11:09.549617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:09.574970 (+ 25353us) service_pool.cc:224] Handling call
0709 12:11:09.622749 (+ 47779us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:09.675024 (+ 52275us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:09.675057 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:09.714694 (+ 39637us) log.cc:844] Serialized 26 byte log entry
0709 12:11:09.981976 (+267282us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:10.632667 (+650691us) spinlock_profiling.cc:243] Waited 2.5 ms on lock 0x7b1800336940. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf
0709 12:11:10.632832 (+   165us) log.cc:844] Serialized 14 byte log entry
0709 12:11:10.668989 (+ 36157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:10.669065 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:11.202315 (+533250us) spinlock_profiling.cc:243] Waited 8.06 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:11.204340 (+  2025us) raft_consensus.cc:1685] finished
0709 12:11:11.204366 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:11.204513 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":128223,"prepare.queue_time_us":55406,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":78,"raft.queue_time_us":71800,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":24298,"spinlock_wait_cycles":24280192,"thread_start_us":253836,"threads_started":2,"wal-append.queue_time_us":311288}
I20260709 12:11:11.218705 16109 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:11.221683 16106 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:11.222535 16106 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:11.222792 16109 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:11.224359 16109 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:11.223781 16106 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:11.168650 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:11.226536 12597 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 9: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:11.230427 16163 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:11.230561 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:11.231547 15942 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:11.231174 16163 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:11.232951 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:11.167150 16155 raft_consensus.cc:3055] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:11.237620 16155 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:11.238219 12525 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:11:11.237037 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:11.238590 16155 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:11.239934 16155 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:11.240598 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:11.241283 12530 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
W20260709 12:11:11.097867 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 64061us
W20260709 12:11:11.284375 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 75742.2us
I20260709 12:11:11.284633 16064 raft_consensus.cc:2764] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:11:11.288496 12675 raft_consensus.cc:2410] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:11:11.289067 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:11.291512 16163 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:11.292726 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 77366.5us
I20260709 12:11:11.293751 12646 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:11.285908 16181 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:11.294778 16152 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:11.295616 16102 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260709 12:11:11.296010 16102 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:11.296407 16102 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:11:11.298305 16152 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:11.303648 12679 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:11:11.304512 16152 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:11.305154 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:11.306468 12494 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:11.306572 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 249322us
I20260709 12:11:11.307322 16064 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:11.307742 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:11.307876 16064 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:11.308084 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:11.308486 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:11.308605 16128 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:11.309636 16169 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:11.301054 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:11.309644 15942 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:11.310539 12673 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:11.308923 16064 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:11.310678 16128 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:11.309026 12679 raft_consensus.cc:2410] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 10 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:11:11.310762 15942 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:11.299999 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:11.312101 15942 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:11.312511 12675 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
W20260709 12:11:11.309952 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 91681.8us
I20260709 12:11:11.314996 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:11.316002 16084 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:11:11.284914 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 186840us
I20260709 12:11:11.302171 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:11.317459 12572 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:11.317878 12680 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:11.318506 12496 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:11.286602 16014 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:11.322638 12427 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47524 (request call id 96) took 4623 ms. Trace:
W20260709 12:11:11.301869 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:11.355497 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:11.356271 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:11.337193 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:11.357398 12600 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:11.356796 12677 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:11.338217 12672 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:11.331075 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:11.335529 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 46558.4us
W20260709 12:11:11.360996 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:11.342319 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 48658.3us
I20260709 12:11:11.364856 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:11.365765 12601 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:11:11.366726 12605 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:11:11.367588 12572 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:11.368086 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 59679.5us
I20260709 12:11:11.350597 12531 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:11.374284 12598 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:11:11.375732 12646 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:11.387917 16153 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:11.388810 16153 raft_consensus.cc:697] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:11.390753 16153 consensus_queue.cc:237] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:11.404657 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46008.8us
I20260709 12:11:11.405755 12496 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:11.406038 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:11.405435 13305 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:11.333536 16158 raft_consensus.cc:2764] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260709 12:11:11.345126 16084 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:11.419198 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:11.419307 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:11.420101 16119 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:11.439122 12598 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:11:11.373888 16174 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:11:11.421684 16182 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.006s
I20260709 12:11:11.337015 12427 rpcz_store.cc:276] 0709 12:11:06.698734 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:06.699006 (+   272us) service_pool.cc:224] Handling call
0709 12:11:06.855219 (+156213us) spinlock_profiling.cc:243] Waited 3.13 ms on lock 0x7b5400280138. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e7dee43 00007f148e8dd04e 00007f148cd9e2a4 00007f148cd9e232 00007f148cd9e161 00007f148cd9e0dc 00007f148cd9d392 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:11.322509 (+4467290us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:11:09.627675 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:11:09.837187 (+209512us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:11:09.837247 (+    60us) write_op.cc:435] Acquired schema lock
0709 12:11:09.837280 (+    33us) tablet.cc:662] Decoding operations
0709 12:11:09.838903 (+  1623us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:11:09.838990 (+    87us) write_op.cc:641] Partition lock acquired for write op
0709 12:11:09.839035 (+    45us) tablet.cc:685] Acquiring locks for 12 operations
0709 12:11:09.841160 (+  2125us) tablet.cc:701] Row locks acquired
0709 12:11:09.841191 (+    31us) write_op.cc:260] PREPARE: finished
0709 12:11:09.841510 (+   319us) write_op.cc:270] Start()
0709 12:11:09.841669 (+   159us) write_op.cc:276] Timestamp: P: 1783599069841428 usec, L: 0
0709 12:11:09.841690 (+    21us) op_driver.cc:348] REPLICATION: starting
0709 12:11:09.990620 (+148930us) log.cc:844] Serialized 4812 byte log entry
0709 12:11:10.568509 (+577889us) op_driver.cc:464] REPLICATION: finished
0709 12:11:10.812599 (+244090us) write_op.cc:301] APPLY: starting
0709 12:11:10.818217 (+  5618us) tablet.cc:1367] starting BulkCheckPresence
0709 12:11:10.905470 (+ 87253us) tablet.cc:1370] finished BulkCheckPresence
0709 12:11:10.945199 (+ 39729us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:11:10.949578 (+  4379us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:11:10.949892 (+   314us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0709 12:11:10.949949 (+    57us) write_op.cc:312] APPLY: finished
0709 12:11:10.950322 (+   373us) log.cc:844] Serialized 88 byte log entry
0709 12:11:10.950519 (+   197us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:10.950678 (+   159us) write_op.cc:454] Released schema lock
0709 12:11:10.950733 (+    55us) write_op.cc:341] FINISH: Updating metrics
0709 12:11:10.951213 (+   480us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:10.951246 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":118019,"reported_tablets":20,"spinlock_wait_cycles":7241856,"child_traces":[["op",{"apply.queue_time_us":206843,"apply.run_cpu_time_us":4268,"apply.run_wall_time_us":147864,"num_ops":12,"prepare.queue_time_us":550752,"prepare.run_cpu_time_us":32557,"prepare.run_wall_time_us":573508,"raft.queue_time_us":173917,"raft.run_cpu_time_us":65677,"raft.run_wall_time_us":134009,"replication_time_us":726587,"thread_start_us":590870,"threads_started":4,"wal-append.queue_time_us":243639}]]}
I20260709 12:11:11.347110 16041 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.092s
I20260709 12:11:11.404192 15913 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:11.333882 16105 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.104s
I20260709 12:11:11.411527 16197 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:11.424854 16183 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:11:11.443300 12643 leader_election.cc:336] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.547s (SENT)
I20260709 12:11:11.449131 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:11.335189 16163 consensus_queue.cc:1059] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.104s
I20260709 12:11:11.414713 16176 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:11.334302 16037 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:11.334947 16068 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.103s
I20260709 12:11:11.480794 16013 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.016s
I20260709 12:11:11.439954 13339 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:11:11.352411 15925 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.088s
W20260709 12:11:11.471545 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 65992.3us
I20260709 12:11:11.495330 16145 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:11:11.495683 12496 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:11.472436 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46018.4us
I20260709 12:11:11.330041 15942 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:11.499164 15942 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:11.496984 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:11.450513 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:11.423511 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:11.496533 16158 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:11.440277 16119 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:11.490926 12425 catalog_manager.cc:5697] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 5 to 8, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 8 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:11.491246 16055 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:11.484942 16037 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:11.496819 16206 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:11:11.480283 12598 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 5. Current term is 6. Ops: []
I20260709 12:11:11.447858 16106 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:11.503160 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 59915.1us
W20260709 12:11:11.506364 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 137967us
I20260709 12:11:11.476783 16126 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:11.500859 12600 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:11.481865 16176 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:11.496274 16179 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.025s
I20260709 12:11:11.505826 16158 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:11.516032 16119 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:11.534411 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 166880us
I20260709 12:11:11.480441 15913 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:11.538606 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:11.514052 16014 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.033s
I20260709 12:11:11.524083 16037 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:11.573421 12572 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:11.532073 16126 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:11.513731 15977 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.025s
W20260709 12:11:11.555539 16070 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.511s	user 0.011s	sys 0.000s
I20260709 12:11:11.514299 16064 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:11.562350 12425 catalog_manager.cc:5697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 5 to 6, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 6 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:11.535754 16114 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:11.573719 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:11.547704 16176 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:11.596004 15913 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:11.596941 12603 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:11:11.598057 12527 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:11:11.538228 12571 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:11.588478 12597 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
W20260709 12:11:11.599026 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 67306.7us
I20260709 12:11:11.589121 12425 catalog_manager.cc:5697] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 6 to 7, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 7 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:11.530551 16106 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:11.593708 16114 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:11.577960 16126 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:11.596815 16191 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:11:11.609827 16114 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:11.598423 16208 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:11:11.555022 16158 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:11.601078 16209 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:11.600879 16102 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:11.600224 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:11.604627 16120 raft_consensus.cc:3055] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:11.604187 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:11:11.624394 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:11.591562 16109 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:11.629254 12571 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:11.625078 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 118023us
I20260709 12:11:11.625463 12425 catalog_manager.cc:5697] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 2 to 6, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 6 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:11:11.630255 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:11.591220 16064 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:11.488793 16197 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:11.628770 16120 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:11.638105 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:11.623775 16169 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:11.631244 16051 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:11.628504 16209 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:11.645210 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:11.587069 12531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:11.654503 12680 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:11:11.655126 12531 raft_consensus.cc:606] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:11.655797 12531 raft_consensus.cc:614] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:11:11.647065 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 120912us
I20260709 12:11:11.647374 12677 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: []
I20260709 12:11:11.644203 12572 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:11.642920 12425 catalog_manager.cc:5697] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 6 to 7, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 7 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:11.644748 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:11.658980 12675 raft_consensus.cc:2410] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 11 because replica has last-logged OpId of term: 10 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 3.
I20260709 12:11:11.638314 16064 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:11.648154 16051 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:11.638037 16110 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:11:11.665789 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 165040us
I20260709 12:11:11.642269 15913 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:11.666416 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:11.669054 12530 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:11:11.667459 12425 catalog_manager.cc:5697] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 1 to 6, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 6 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:11.670279 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:11.670900 12425 catalog_manager.cc:5697] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:11.671602 16084 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:11.672104 16084 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:11.672523 16084 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:11.676115 12525 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:11.506280 16157 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.081s
I20260709 12:11:11.669287 16062 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:11.684476 16062 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:11:11.685402 12572 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:11.641819 16163 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:11:11.687844 12496 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:11.650789 16191 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:11.695014 16163 consensus_queue.cc:1243] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 8.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:11:11.695310 16196 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:11:11.698817 12601 pending_rounds.cc:85] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 6
I20260709 12:11:11.699213 12601 pending_rounds.cc:107] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 12:11:11.699589 12601 raft_consensus.cc:2889] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:11.643055 16089 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:11:11.660262 16209 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:11.709889 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:11.717289 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:11.707276 12680 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:11:11.672793 12425 catalog_manager.cc:5697] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 4 to 5, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 5 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:11.720801 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:11.727874 16055 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:11.728718 16055 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:11:11.669235 16110 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:11.731352 16110 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:11.668848 16120 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:11.735072 16153 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:11.735365 16120 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:11.735749 16153 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:11.738492 16102 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:11.669054 16051 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:11.688882 16011 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 12:11:11.757903 12531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:11.758958 12531 raft_consensus.cc:606] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:11.759526 12531 raft_consensus.cc:614] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:11.653338 16169 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:11.761577 16169 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:11.762216 16212 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.163s	user 0.001s	sys 0.001s
I20260709 12:11:11.676690 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:11.763134 12526 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:11:11.685482 16062 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:11.701756 16197 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:11.765818 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 80502.5us
I20260709 12:11:11.776995 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:11.778294 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:11.778658 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:11.781329 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:11.786324 12425 catalog_manager.cc:5697] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 8 to 10, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 10 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:11:11.792975 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:11.790163 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 102389us
I20260709 12:11:11.803092 12425 catalog_manager.cc:5697] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 1 to 7. New cstate: current_term: 7 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:11.813810 12531 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:11:11.814976 12530 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:11.815476 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:11.764611 16163 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:11.834283 12673 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:11:11.766206 16106 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:11.831385 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 54516.8us
I20260709 12:11:11.844674 12526 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:11:11.841465 15942 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:11.859411 12671 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:11:11.859731 16037 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:11.860579 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:11.861063 12600 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:11.785914 15996 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:11:11.776147 16102 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:11.872349 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:11.874059 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:11.875211 12523 raft_consensus.cc:2368] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8.
I20260709 12:11:11.861173 12671 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:11.786695 16153 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:11.884683 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:11.890187 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:11.892323 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48349.1us
I20260709 12:11:11.892524 16126 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:11.896904 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:11.898458 12598 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:11.900465 12425 catalog_manager.cc:5697] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:11.764250 16214 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.079s
I20260709 12:11:11.901460 12522 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:11.903594 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:11.905294 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:11.906083 12522 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:11.907189 12522 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 4. Current term is 5. Ops: []
I20260709 12:11:11.908854 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:11.909515 12531 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:11.909585 12523 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:11.915418 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:11.832834 16221 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:11.920222 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:11.921221 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:11.921945 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:11.832204 16128 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:11.913306 12425 catalog_manager.cc:5697] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 1 to 6, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 6 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } }
I20260709 12:11:11.926847 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:11.939031 12530 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:11.867614 16114 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:11.944299 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:11.952359 12524 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:11.931515 12604 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:11.958482 12571 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:11.894729 16181 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:11.895098 15996 raft_consensus.cc:697] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:11.965134 16152 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:11.951067 15942 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:11.983655 12425 catalog_manager.cc:5697] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 6 to 8, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 8 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:11.967993 15913 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:11:11.969010 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:11.997186 12603 raft_consensus.cc:2410] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 8 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:11:11.955502 16223 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:11:11.996897 15996 consensus_queue.cc:237] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:11.998224 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:11.999929 12603 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: [7.6-7.6]
I20260709 12:11:11.970679 12679 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:11:11.980630 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:11.956754 12643 connection.cc:256] RPC call timeout handler was delayed by -1.6328562009998677s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1753s for a response
I20260709 12:11:11.987954 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:11.899924 16102 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:12.009027 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:11.956369 16106 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:11.943773 16148 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:11:11.865278 16163 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:12.003506 12643 connection.cc:256] RPC call timeout handler was delayed by -1.3583562009998786s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20579999999999998s for a response
I20260709 12:11:12.022986 12671 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
W20260709 12:11:11.993759 16207 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.406s	user 0.002s	sys 0.000s
I20260709 12:11:11.923807 15970 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.061s
W20260709 12:11:12.007781 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 508600us
I20260709 12:11:12.044579 12494 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:11.978813 16197 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:11:12.010946 12527 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:11:12.017300 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 123513us
I20260709 12:11:12.049806 12527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:12.050556 12527 raft_consensus.cc:606] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:12.050875 12572 leader_election.cc:400] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8.
I20260709 12:11:12.051004 12527 raft_consensus.cc:614] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:12.051558 12572 leader_election.cc:403] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:11:11.955695 16128 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:12.037806 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:12.061746 12572 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:12.064785 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:12.011649 16037 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:11.985884 16181 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.078483 12528 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:11:12.058990 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:12.014304 16220 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:12.055869 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:11.951571 16221 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:12.072702 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:12.072772 12496 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:12.063409 16191 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:11.995965 16152 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:12.007863 16084 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.056470 15977 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:11:12.028614 12531 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 10.
I20260709 12:11:12.061968 16225 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:12.068137 16070 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:12.019692 16106 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.073900 16145 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:12.059943 16158 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.128024 12530 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:12.130609 12530 raft_consensus.cc:606] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:12.130997 12530 raft_consensus.cc:614] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:12.115671 12425 catalog_manager.cc:5697] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 1 to 5, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 5 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:12.086364 16109 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260709 12:11:12.098420 16105 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:12.151196 13339 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
W20260709 12:11:12.150245 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 63680us
I20260709 12:11:12.076071 16051 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:12.152627 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:12.162982 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 56693.9us
I20260709 12:11:12.165125 13305 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:12.165663 12572 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:12.166066 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:12.129575 16013 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:12.169008 12527 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:11:12.171181 12531 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260709 12:11:12.153044 16155 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.913s	user 0.016s	sys 0.001s
I20260709 12:11:12.172686 12425 catalog_manager.cc:5697] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 2 to 5, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 5 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:12.196774 16221 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:12.196208 12425 catalog_manager.cc:5697] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 3 to 6, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 6 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:12.177755 16070 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:12.198138 16109 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:11:12.197903 12425 catalog_manager.cc:5697] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 1 to 7. New cstate: current_term: 7 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:12.179381 12531 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10. Ops: [9.9-9.9]
I20260709 12:11:12.180027 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:12.201007 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:12.201588 12673 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:12.202507 16124 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:12.202879 16106 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:12.203075 16124 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.203471 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:12.205673 12531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:12.206285 12531 raft_consensus.cc:606] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:12.206171 16197 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:12.206627 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:12.087724 16229 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.093s
I20260709 12:11:12.207823 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:12.208422 12679 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 10
I20260709 12:11:12.209606 16120 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:11:12.182982 16222 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:11:12.183579 16231 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:12.190284 16037 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:12.193709 16102 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:12.146977 16105 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:12.194628 16126 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:12.127651 16110 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.194986 16128 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:12.178210 12646 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:12.229147 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 51070.2us
I20260709 12:11:12.147627 16183 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:12.098762 16220 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:12.213220 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:12.240648 12526 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:11:12.214627 12425 catalog_manager.cc:5697] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 2 to 6. New cstate: current_term: 6 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
W20260709 12:11:12.239490 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:12.175725 16013 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:12.226006 12673 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:11:12.103113 16211 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.119s
I20260709 12:11:12.250941 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:12.264989 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:12.125162 16225 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.256645 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:12.107362 15942 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.144490 16145 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.261266 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:12.221094 16037 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:12.272650 12425 catalog_manager.cc:5697] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:12.253153 12679 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:12.266028 12673 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:11:12.235461 16062 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:12.276757 12671 raft_consensus.cc:2410] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
W20260709 12:11:12.248713 16119 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.675s	user 0.010s	sys 0.003s
I20260709 12:11:12.225560 16120 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:12.279008 12425 catalog_manager.cc:5697] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 4 to 5, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 5 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:12.242651 16168 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:11:12.297655 16168 raft_consensus.cc:697] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:12.288489 12673 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:12.298692 16168 consensus_queue.cc:237] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.288251 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:12.255632 16013 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:12.306768 16220 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:12.306802 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:12.308246 16241 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:12.309085 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:12.311144 16152 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:12.308831 16241 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:12.312598 16181 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:12.313191 16110 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:12.314199 16241 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:12.248293 16106 consensus_queue.cc:1059] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260709 12:11:12.315699 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:12.264439 16164 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260709 12:11:12.253355 16183 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:12.461971 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 115078us
I20260709 12:11:12.295516 12425 catalog_manager.cc:5697] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 3 to 4, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 4 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:12.345762 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:12.338495 13333 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:11:12.464308 13333 pending_rounds.cc:85] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 8
W20260709 12:11:12.318861 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:12.463933 12425 catalog_manager.cc:5697] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 2 to 6, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 6 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:12.355008 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:12.316009 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:12.424280 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 368523us
I20260709 12:11:12.465759 13333 pending_rounds.cc:107] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:11:12.316563 12523 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:12.466441 13333 raft_consensus.cc:2889] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:12.466626 12494 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:12.317772 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:12.316419 13335 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:12.468518 12526 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:11:12.465438 13304 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.538s (SENT) [suppressed 38 similar messages]
I20260709 12:11:12.349720 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:11:12.463351 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:12.275826 16037 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:12.490815 12496 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:12.475014 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:12.488430 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:12.474239 12672 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:11:12.400601 16226 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.383s	user 0.001s	sys 0.001s
W20260709 12:11:12.329058 16176 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.733s	user 0.009s	sys 0.000s
W20260709 12:11:12.317005 16224 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 12:11:12.518417 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:12.518780 12673 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
I20260709 12:11:12.333432 16051 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:12.349938 16174 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:12.325665 16013 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.520156 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:12.521042 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:12.579025 12672 raft_consensus.cc:2376] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7. Re-sending same reply.
I20260709 12:11:12.469033 16211 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:12.513409 16221 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:12.515933 16101 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:12.294386 16062 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.524415 16183 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:12.468693 16114 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:12.319183 16105 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:12.517148 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 49912.3us
I20260709 12:11:12.347096 16198 raft_consensus.cc:3055] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:11:12.275858 16209 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.533898 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:12.569558 12677 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:12.341557 16158 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:12.469610 16163 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:12.349198 16201 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.833s	user 0.000s	sys 0.003s
W20260709 12:11:12.317900 16064 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.642s	user 0.010s	sys 0.003s
I20260709 12:11:12.577095 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:12.323798 16145 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:12.587729 16211 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.580302 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:12.655185 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:12.655929 12672 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
W20260709 12:11:12.656122 16191 consensus_meta.cc:287] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.554s	user 0.010s	sys 0.001s
I20260709 12:11:12.656560 16191 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8.
I20260709 12:11:12.270051 16223 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.243s
I20260709 12:11:12.343178 16124 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:12.316557 16106 raft_consensus.cc:3055] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:11:12.632562 16198 raft_consensus.cc:740] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:12.530377 16155 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:12.668073 16155 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.668411 16198 consensus_queue.cc:260] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.670543 12599 raft_consensus.cc:2368] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8.
I20260709 12:11:12.671130 16155 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:12.586443 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 66381.3us
I20260709 12:11:12.671729 16209 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:12.589347 12678 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:11:12.536047 16240 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:12.575057 16241 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:12.633818 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:12.633940 12680 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:12.675473 16241 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.634114 16225 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:12.543264 16176 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:12.641736 12676 raft_consensus.cc:2393] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 8: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:12.679168 16176 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:12.679924 16240 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:12.680898 16240 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.681170 16064 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.682493 12530 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 6. Current term is 7. Ops: [6.4-6.4]
I20260709 12:11:12.683395 16064 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:12.600874 16179 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:12.348282 16120 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:12.586563 16101 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:12.650079 12673 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:11:12.634768 16114 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.667309 16106 raft_consensus.cc:740] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:12.705040 12598 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:11:12.633159 16084 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:12.684615 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:12.295790 16119 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:12.709436 12598 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:12.623679 16230 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260709 12:11:12.689702 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 224270us
I20260709 12:11:12.670027 16198 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:11:12.677884 16241 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:12.700866 16223 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Stepping down as leader of term 9
W20260709 12:11:12.587428 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 56006.8us
I20260709 12:11:12.704814 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:12.582612 16174 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.685653 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:12.712214 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:12.720202 12643 leader_election.cc:336] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.753s (SENT)
I20260709 12:11:12.721271 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:12.763690 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:12.723188 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:12.693189 16101 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:12.764703 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:12.760746 12605 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:12.746975 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:12.748222 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:12.754827 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 225033us
I20260709 12:11:12.751958 12604 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
W20260709 12:11:12.725451 13304 leader_election.cc:336] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.642s (SENT)
I20260709 12:11:12.769191 13304 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:12.770145 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 44705.3us
I20260709 12:11:12.763870 12597 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
W20260709 12:11:12.724291 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:12.766178 13337 raft_consensus.cc:2368] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 7. Current term is 8.
I20260709 12:11:12.589850 16221 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:12.773753 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:12.776763 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:12.775708 16163 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:12.787078 16163 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:12.713268 16230 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:12.794039 12529 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 8: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:11:12.795650 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 75484.3us
I20260709 12:11:12.800360 12677 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:12.767251 16223 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:12.776302 16084 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:12.695307 16179 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:12.799926 16102 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:12.780524 16064 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:11:12.799644 16230 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:12.807312 16145 consensus_queue.cc:1059] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.035s
W20260709 12:11:12.854197 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 80526.5us
I20260709 12:11:12.883056 13304 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:12.816104 16084 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.891547 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:12.898535 13304 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:12.894769 16163 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:12.810717 16106 consensus_queue.cc:260] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.886334 16155 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:12.892892 16241 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:12.886878 16168 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:12.988837 16264 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:12.989619 16264 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:12.989082 16168 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:12.990128 16264 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:12.992120 16250 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.265s	user 0.002s	sys 0.000s
W20260709 12:11:12.929558 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:12.997335 13337 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 533) took 1547 ms (client timeout 1719 ms). Trace:
I20260709 12:11:12.993893 13305 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:11:13.000447 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 203950us
I20260709 12:11:12.947118 16106 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:11:12.998044 13337 rpcz_store.cc:269] 0709 12:11:11.450020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:11.450203 (+   183us) service_pool.cc:224] Handling call
0709 12:11:12.997078 (+1546875us) spinlock_profiling.cc:243] Waited 1.27 s on lock 0x7b54001261d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:12.997253 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2910007040}
I20260709 12:11:12.937831 16183 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:12.962678 16243 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
W20260709 12:11:12.967525 16249 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
W20260709 12:11:12.975160 16248 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.164s	user 0.000s	sys 0.000s
W20260709 12:11:12.977584 16255 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
W20260709 12:11:12.981338 16253 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:11:12.901955 16062 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:13.003886 12643 leader_election.cc:336] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.058s (SENT)
W20260709 12:11:13.080335 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 76520us
W20260709 12:11:13.081390 12643 leader_election.cc:336] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.131s (SENT)
I20260709 12:11:13.081894 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:12.892294 16037 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:13.034370 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:12.889734 16223 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:13.083261 12529 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:13.023252 16209 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:13.086941 12643 leader_election.cc:336] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.230s (SENT)
I20260709 12:11:13.087452 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:13.088778 12643 connection.cc:256] RPC call timeout handler was delayed by -1.024354465999977s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3178s for a response
I20260709 12:11:13.025940 16191 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:13.028518 16119 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:13.094000 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:13.102449 12522 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
I20260709 12:11:13.075690 16174 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:13.039013 16163 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:11:13.098263 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 280315us
W20260709 12:11:13.094012 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 195915us
I20260709 12:11:13.023469 16155 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:13.035183 16051 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:13.035308 16211 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:13.092934 16225 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:13.081178 16114 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:13.111090 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 117275us
I20260709 12:11:13.024560 16241 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:13.138063 12525 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260709 12:11:13.138906 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:13.093171 16152 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:13.025745 16197 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:13.035972 16183 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:13.153594 16256 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:11:13.147641 16179 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:13.164619 16037 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:13.169420 16159 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:13.172406 16155 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:13.099076 16148 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:13.151506 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:13.183777 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:13.105996 16120 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:13.090672 16209 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:13.197161 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 43985.9us
I20260709 12:11:13.191603 16148 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:13.195381 16037 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:13.191759 16163 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:13.203999 16114 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:13.207329 12671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 310) took 1160 ms (client timeout 1538 ms). Trace:
I20260709 12:11:13.189339 16119 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:13.197077 16120 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:13.221542 16124 raft_consensus.cc:3055] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:11:13.148767 16051 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:13.298432 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:13.299315 12677 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:13.234503 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:13.234597 12569 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:13.233866 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:13.299980 13334 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:13.235359 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:13.300771 13337 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:11:13.236585 16211 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:11:13.237522 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2240) took 1398 ms. Trace:
W20260709 12:11:13.238253 16273 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 12:11:13.240048 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:13.240234 16244 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.531s	user 0.002s	sys 0.000s
I20260709 12:11:13.280117 16120 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:13.242106 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1907) took 1323 ms. Trace:
I20260709 12:11:13.242336 16152 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:13.242786 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:13.243707 16274 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:13.244427 13305 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:11:13.246565 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 62919us
I20260709 12:11:13.294795 16084 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:13.295559 16124 raft_consensus.cc:740] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:13.298395 16231 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:13.277932 16114 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:13.252774 12529 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:11:13.260490 12602 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:11:13.217146 16013 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:13.266675 12671 rpcz_store.cc:269] 0709 12:11:12.047339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:12.054418 (+  7079us) service_pool.cc:224] Handling call
0709 12:11:13.207047 (+1152629us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b540016f8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:13.207226 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":429564672}
I20260709 12:11:13.280340 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:13.284607 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:13.346382 12673 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:13.347358 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:13.287180 12676 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260709 12:11:13.295742 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:13.233037 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:13.236032 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:13.265772 16221 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:13.107963 16191 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:13.189617 16159 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:13.308458 12679 rpcz_store.cc:276] 0709 12:11:11.839189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:12.292911 (+453722us) service_pool.cc:224] Handling call
0709 12:11:12.422964 (+130053us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:12.423519 (+   555us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:12.423560 (+    41us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:12.473890 (+ 50330us) log.cc:844] Serialized 52 byte log entry
0709 12:11:12.755641 (+281751us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:12.804617 (+ 48976us) log.cc:844] Serialized 14 byte log entry
0709 12:11:12.805939 (+  1322us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:12.806227 (+   288us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:13.237256 (+431029us) raft_consensus.cc:1685] finished
0709 12:11:13.237288 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:13.237458 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":85217,"prepare.queue_time_us":48656,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"raft.queue_time_us":45896,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":53933,"spinlock_wait_cycles":5376,"thread_start_us":235405,"threads_started":1,"wal-append.queue_time_us":297537}
I20260709 12:11:13.277456 16119 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:13.310851 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:13.314728 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:13.195942 16261 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:11:13.318807 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:13.365589 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:13.366304 12679 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:13.367245 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:13.367930 12671 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:11:13.368839 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:13.370572 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:13.371917 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:13.372594 12671 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:13.375013 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:13.380127 16124 consensus_queue.cc:260] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:13.381767 16124 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:13.266671 15942 raft_consensus.cc:2764] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:11:13.327941 12525 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260709 12:11:13.332734 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1913) took 1070 ms. Trace:
I20260709 12:11:13.331485 13337 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 5.
I20260709 12:11:13.323336 12523 rpcz_store.cc:276] 0709 12:11:11.919208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:11.924186 (+  4978us) service_pool.cc:224] Handling call
0709 12:11:12.316363 (+392177us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:12.471321 (+154958us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:13.241900 (+770579us) spinlock_profiling.cc:243] Waited 392 ms on lock 0x7b54003cbed0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:13.242041 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":901103360}
W20260709 12:11:13.336277 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52075.2us
I20260709 12:11:13.316196 16225 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:13.351960 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:13.395845 12522 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:13.352739 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:13.396528 12606 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:13.279461 16163 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:13.355501 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:13.357034 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:13.357879 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 62202.1us
I20260709 12:11:13.411643 12496 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:13.349442 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:13.320227 16164 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:13.359761 13335 raft_consensus.cc:2368] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7.
I20260709 12:11:13.382263 16268 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.081s
W20260709 12:11:13.383172 16251 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.222s	user 0.002s	sys 0.000s
W20260709 12:11:13.385991 12600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2001) took 1501 ms (client timeout 1585 ms). Trace:
I20260709 12:11:13.356340 16084 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:13.386646 12598 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:11:13.387830 12677 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:13.430429 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:13.389763 12527 rpcz_store.cc:276] 0709 12:11:12.261885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:12.262854 (+   969us) service_pool.cc:224] Handling call
0709 12:11:12.263145 (+   291us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:12.285083 (+ 21938us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:12.285131 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:12.285792 (+   661us) log.cc:844] Serialized 26 byte log entry
0709 12:11:12.774500 (+488708us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:12.778086 (+  3586us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:12.778240 (+   154us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:13.332386 (+554146us) raft_consensus.cc:1685] finished
0709 12:11:13.332422 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:13.332643 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21444,"raft.queue_time_us":22669,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":20164,"thread_start_us":300269,"threads_started":1,"wal-append.queue_time_us":520715}
W20260709 12:11:13.393481 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 135339us
I20260709 12:11:13.431272 12525 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:13.437668 16280 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:13.438364 16280 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:13.438825 16280 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:13.397187 12679 raft_consensus.cc:2410] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:11:13.365943 16119 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:13.413421 12523 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:11:13.415642 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:13.442829 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:13.443981 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:13.445087 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:13.350531 16264 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:13.353603 16231 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:13.342551 16152 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:13.455993 16231 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:13.456429 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:13.456811 16152 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:13.375433 16179 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:13.457703 16179 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:13.458253 16179 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:13.393146 16182 consensus_queue.cc:1059] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.065s
I20260709 12:11:13.458472 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1914) took 1196 ms. Trace:
I20260709 12:11:13.458613 16260 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:13.459164 16182 raft_consensus.cc:3055] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Stepping down as leader of term 6
W20260709 12:11:13.434330 16101 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.644s	user 0.012s	sys 0.002s
I20260709 12:11:13.459909 16182 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:13.460322 16216 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:13.460188 16101 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:13.460901 16182 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:13.425392 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:11:13.423542 12600 rpcz_store.cc:269] 0709 12:11:11.884290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:11.884448 (+   158us) service_pool.cc:224] Handling call
0709 12:11:13.385763 (+1501315us) spinlock_profiling.cc:243] Waited 413 ms on lock 0x7b540007ba54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:13.385929 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":948758784}
I20260709 12:11:13.462522 12673 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:13.463068 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:13.463281 16183 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:13.443135 12606 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:13.464025 12530 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:11:13.464644 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:13.465193 12606 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:13.430423 16084 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:13.466696 16084 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:13.459015 12531 rpcz_store.cc:276] 0709 12:11:12.262200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:12.347182 (+ 84982us) service_pool.cc:224] Handling call
0709 12:11:12.347550 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:12.347961 (+   411us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:12.347993 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:12.348505 (+   512us) log.cc:844] Serialized 26 byte log entry
0709 12:11:12.704044 (+355539us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:12.725010 (+ 20966us) log.cc:844] Serialized 14 byte log entry
0709 12:11:12.725382 (+   372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:12.725456 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:13.214381 (+488925us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b1800332440. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:11:13.458163 (+243782us) raft_consensus.cc:1685] finished
0709 12:11:13.458268 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:13.458405 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31459,"prepare.queue_time_us":20649,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":591,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":203,"spinlock_wait_cycles":48427776,"thread_start_us":242416,"threads_started":1,"wal-append.queue_time_us":565966}
I20260709 12:11:13.468535 12531 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260709 12:11:13.196587 16197 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:13.472090 16197 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:13.472126 16062 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:13.457799 12494 leader_election.cc:400] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 7. Current term is 8.
I20260709 12:11:13.473209 12494 leader_election.cc:403] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260709 12:11:13.475788 16013 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 7. Current term is 8.
I20260709 12:11:13.419257 16225 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:13.476953 12673 raft_consensus.cc:2410] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:11:13.459437 16260 consensus_queue.cc:1243] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:11:13.478691 16225 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:13.412542 12676 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:13.369921 16221 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:11:13.480314 12675 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:13.475993 16159 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:13.445852 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:13.450753 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:13.483314 12677 raft_consensus.cc:2410] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 11 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:11:13.421483 13339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 360) took 1654 ms. Trace:
I20260709 12:11:13.485587 13339 rpcz_store.cc:276] 0709 12:11:11.766750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:11.766894 (+   144us) service_pool.cc:224] Handling call
0709 12:11:11.797688 (+ 30794us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:12.151609 (+353921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:13.421105 (+1269496us) spinlock_profiling.cc:243] Waited 350 ms on lock 0x7b540063ecd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:13.421397 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":805467520}
I20260709 12:11:13.490489 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:13.491418 12602 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 9: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:13.434331 12671 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:13.353435 16274 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:13.503511 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 269023us
I20260709 12:11:13.494005 12525 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:11:13.506191 13336 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:11:13.511160 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:13.490054 16182 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:11:13.513935 16279 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:11:13.432724 16164 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:13.515805 16258 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.606s	user 0.002s	sys 0.002s
I20260709 12:11:13.523437 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:13.483484 16145 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:13.482915 16174 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:11:13.530364 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 199608us
I20260709 12:11:13.545924 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:13.514416 16120 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:13.548569 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:13.591795 12531 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:13.553028 12606 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:13.556900 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:13.558172 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:13.593451 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:13.594228 12602 raft_consensus.cc:2410] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 9 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
W20260709 12:11:13.565785 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:13.594553 12606 raft_consensus.cc:2393] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 8: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:13.570623 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:13.596973 12571 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:13.530409 16126 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:13.547313 16220 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:13.603935 12673 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:11:13.605615 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:13.611131 12605 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:13.614781 16191 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:13.619230 16279 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:11:13.548025 16280 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:13.622954 16280 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:13.623162 16264 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:13.623852 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:13.624099 16264 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:13.506996 16179 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:13.624624 16264 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:13.627501 16120 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:13.630493 16287 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:13.631116 16287 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:13.633152 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 318525us
I20260709 12:11:13.633646 16287 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:13.635149 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:13.635751 16276 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260709 12:11:13.636335 16106 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.611s	user 0.007s	sys 0.004s
I20260709 12:11:13.636581 16276 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:13.637253 12676 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 4. (term mismatch)
I20260709 12:11:13.637635 12676 pending_rounds.cc:85] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 3
I20260709 12:11:13.637876 12676 pending_rounds.cc:107] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 5.4
W20260709 12:11:13.636346 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 53719.5us
I20260709 12:11:13.624773 12602 raft_consensus.cc:2410] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:11:13.638218 12676 raft_consensus.cc:2889] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:13.639209 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:13.639816 12525 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 7: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:13.606714 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:13.640599 12606 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:13.641906 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:13.641955 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:13.645931 12606 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:13.582634 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 337929us
I20260709 12:11:13.657260 12571 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:13.596354 16211 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:13.582401 16209 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:13.582037 16197 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:13.591166 16183 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:13.703488 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:13.702284 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 92339.3us
I20260709 12:11:13.639209 16174 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:13.666702 16120 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:13.590101 16145 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:13.673276 16287 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:13.750463 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:13.608081 16126 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:13.653152 16220 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:13.676445 16084 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:13.701361 16211 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:13.733554 16062 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.150s
I20260709 12:11:13.986982 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:13.707280 16197 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:13.953145 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:13.811614 13305 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:13.840849 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:14.009205 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:14.009781 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 198358us
I20260709 12:11:13.985378 16275 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:14.007516 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 372445us
I20260709 12:11:14.010162 13334 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 10: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:11:14.008258 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 351085us
I20260709 12:11:14.012279 16308 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:13.995312 16230 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.146s	user 0.014s	sys 0.004s
I20260709 12:11:14.012794 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:14.001068 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:14.007476 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:14.013536 16230 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:14.015300 12680 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:14.015230 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:14.017022 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:14.017624 16230 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:14.017874 12531 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:14.018464 16312 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:14.018908 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:14.013065 16308 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:14.019294 16312 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:14.011600 16275 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:14.023352 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:14.024377 12531 raft_consensus.cc:2410] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 9 because replica has last-logged OpId of term: 7 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
I20260709 12:11:14.021793 16308 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:14.024703 16275 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:14.025421 16308 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:14.026161 16308 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:14.026752 16308 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:14.030948 16211 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:14.033335 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:14.035168 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:14.036702 12601 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:11:14.038872 12606 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
W20260709 12:11:14.044741 13302 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.004s	sys 0.018s
I20260709 12:11:13.741181 16051 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:11:13.751274 16037 raft_consensus.cc:697] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:13.746611 16114 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:13.738981 16286 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:13.757723 16120 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:14.007826 16163 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:13.733537 16183 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:14.072788 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:14.199911 13335 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
W20260709 12:11:14.216033 13302 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.687s	user 0.012s	sys 0.020s
W20260709 12:11:14.139019 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 126844us
I20260709 12:11:14.160887 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:14.220645 12531 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 8: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:14.214102 12571 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:14.141373 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:14.152536 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:14.219084 12643 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:14.035636 16264 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:14.035368 16287 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:11:14.103514 16282 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:11:14.242106 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:14.244768 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:14.239502 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:14.245608 16256 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:11:14.099956 16293 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:14.132475 16298 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:14.250547 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:14.293381 13334 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:14.240583 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:14.096896 16168 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:14.132282 16266 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
I20260709 12:11:14.052743 16312 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:14.119990 16230 raft_consensus.cc:2764] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:11:14.006524 16283 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:11:14.292100 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50089.1us
I20260709 12:11:14.327102 12643 leader_election.cc:400] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8.
I20260709 12:11:14.131664 16314 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:14.138182 16084 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:14.041870 16209 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.223s
I20260709 12:11:14.249903 16275 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:14.272884 16191 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:11:14.356941 12604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.009s	sys 0.005s
W20260709 12:11:14.363387 12604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.010s	sys 0.005s
W20260709 12:11:14.367586 16289 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.601s	user 0.004s	sys 0.000s
I20260709 12:11:14.306509 16114 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:14.368039 16275 consensus_queue.cc:1059] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:14.368381 16114 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:14.370078 12597 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:14.370967 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2185) took 1021 ms. Trace:
I20260709 12:11:14.285918 16286 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:14.292259 16106 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:11:14.371376 12597 rpcz_store.cc:276] 0709 12:11:13.349981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:14.318846 (+968865us) service_pool.cc:224] Handling call
0709 12:11:14.369831 (+ 50985us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:14.370640 (+   809us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:14.370897 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:14.378219 16114 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:14.391646 16278 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.013s	user 0.000s	sys 0.003s
W20260709 12:11:14.395152 16294 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:11:14.397238 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2248) took 1933 ms. Trace:
I20260709 12:11:14.397768 12678 rpcz_store.cc:276] 0709 12:11:12.464087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:12.673593 (+209506us) service_pool.cc:224] Handling call
0709 12:11:12.673985 (+   392us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:12.709360 (+ 35375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:12.709391 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:13.171838 (+462447us) spinlock_profiling.cc:243] Waited 3.49 ms on lock 0x7b2c00037868. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:11:13.172820 (+   982us) log.cc:844] Serialized 52 byte log entry
0709 12:11:13.312863 (+140043us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:13.312901 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:13.312996 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:14.390824 (+1077828us) spinlock_profiling.cc:243] Waited 25.1 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:14.396939 (+  6115us) raft_consensus.cc:1685] finished
0709 12:11:14.396971 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:14.397164 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34929,"raft.queue_time_us":39097,"spinlock_wait_cycles":79215360,"thread_start_us":139541,"threads_started":1,"wal-append.queue_time_us":190719}
I20260709 12:11:14.399997 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1944) took 1772 ms. Trace:
I20260709 12:11:14.400712 12530 rpcz_store.cc:276] 0709 12:11:12.628011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:12.775094 (+147083us) service_pool.cc:224] Handling call
0709 12:11:12.775552 (+   458us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:13.464422 (+688870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:14.399579 (+935157us) spinlock_profiling.cc:243] Waited 688 ms on lock 0x7b54003cb4d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:14.399907 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1582790016}
I20260709 12:11:14.324076 16168 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:14.435738 12678 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:11:14.441576 16300 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.529s	user 0.001s	sys 0.000s
I20260709 12:11:14.104782 16174 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:14.330495 16283 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:14.331308 16164 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:14.448539 12678 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:14.453084 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2245) took 2387 ms. Trace:
I20260709 12:11:14.458428 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1919) took 1174 ms. Trace:
I20260709 12:11:14.494105 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1918) took 1210 ms. Trace:
I20260709 12:11:14.494527 12675 rpcz_store.cc:276] 0709 12:11:13.283581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.298104 (+ 14523us) service_pool.cc:224] Handling call
0709 12:11:14.458176 (+1160072us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b540012d754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:14.458339 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":264597760}
I20260709 12:11:14.468559 16179 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:14.476686 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1952) took 1147 ms. Trace:
I20260709 12:11:14.528573 12523 rpcz_store.cc:276] 0709 12:11:13.304619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.390830 (+ 86211us) service_pool.cc:224] Handling call
0709 12:11:13.391225 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:13.442366 (+ 51141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:14.451937 (+1009571us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b54003f8c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:14.452150 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49973504}
I20260709 12:11:14.484826 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:14.362836 16293 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 8 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:14.493709 12674 rpcz_store.cc:276] 0709 12:11:12.065563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:12.334957 (+269394us) service_pool.cc:224] Handling call
0709 12:11:13.207193 (+872236us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:13.207453 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:11:13.207792 (+   339us) log.cc:844] Serialized 14 byte log entry
0709 12:11:13.300618 (+ 92826us) log.cc:844] Serialized 14 byte log entry
0709 12:11:13.308812 (+  8194us) log.cc:844] Serialized 14 byte log entry
0709 12:11:13.525986 (+217174us) spinlock_profiling.cc:243] Waited 9.44 ms on lock 0x7b1800319b40. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:11:13.526194 (+   208us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:13.526242 (+    48us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:13.526292 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:13.526365 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:14.128625 (+602260us) spinlock_profiling.cc:243] Waited 23.6 ms on lock 0x7b1800319b40. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:11:14.452783 (+324158us) spinlock_profiling.cc:243] Waited 872 ms on lock 0x7b540016f8d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:14.452956 (+   173us) inbound_call.cc:177] Queueing success response
0709 12:11:14.463379 (+ 10423us) spinlock_profiling.cc:243] Waited 6.14 ms on lock 0x7b1800319b40. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
Metrics: {"mutex_wait_us":343918,"prepare.queue_time_us":11249,"prepare.run_cpu_time_us":169,"prepare.run_wall_time_us":177,"raft.queue_time_us":231684,"raft.run_cpu_time_us":381,"raft.run_wall_time_us":8709,"spinlock_wait_cycles":2096780800,"thread_start_us":92259,"threads_started":1,"wal-append.queue_time_us":156957}
I20260709 12:11:14.494829 12672 rpcz_store.cc:276] 0709 12:11:13.283197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.287944 (+  4747us) service_pool.cc:224] Handling call
0709 12:11:13.288385 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:13.288914 (+   529us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:13.288958 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:13.318172 (+ 29214us) log.cc:844] Serialized 26 byte log entry
0709 12:11:13.553652 (+235480us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:13.553703 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:13.553783 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:14.493717 (+939934us) raft_consensus.cc:1685] finished
0709 12:11:14.493756 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:14.493973 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":214436,"raft.queue_time_us":7118,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":7045,"thread_start_us":194409,"threads_started":1,"wal-append.queue_time_us":393038}
W20260709 12:11:14.550765 16291 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 12:11:14.552791 16183 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:14.492900 16283 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:14.404428 16230 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:14.558506 16230 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:14.560076 16013 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:14.560938 16013 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:14.562860 16197 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:14.372925 16102 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.134s
I20260709 12:11:14.385068 16314 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:14.404702 16037 consensus_queue.cc:237] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:14.377730 16191 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:14.577665 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1943) took 1950 ms. Trace:
I20260709 12:11:14.569319 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1923) took 1284 ms. Trace:
I20260709 12:11:14.649384 12524 rpcz_store.cc:276] 0709 12:11:12.627560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:12.627763 (+   203us) service_pool.cc:224] Handling call
0709 12:11:12.628275 (+   512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:12.799125 (+170850us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:12.799176 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:12.800916 (+  1740us) log.cc:844] Serialized 26 byte log entry
0709 12:11:13.098909 (+297993us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:13.098950 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:13.099097 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:14.577350 (+1478253us) raft_consensus.cc:1685] finished
0709 12:11:14.577381 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:14.577575 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4235,"raft.queue_time_us":47411,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":67,"thread_start_us":456486,"threads_started":2,"wal-append.queue_time_us":452594}
W20260709 12:11:14.475059 16309 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:11:14.711311 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:14.716528 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:14.717448 12528 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:14.718822 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:14.719761 12530 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:14.650818 12673 rpcz_store.cc:276] 0709 12:11:13.284745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.477706 (+192961us) service_pool.cc:224] Handling call
0709 12:11:13.478048 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:13.610821 (+132773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:14.568958 (+958137us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b5400091fd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:14.569219 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":288417152}
I20260709 12:11:14.721127 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:14.709863 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:14.731045 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:14.731994 12531 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:14.734592 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 249841us
I20260709 12:11:14.746392 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:14.580058 16119 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.092s	user 0.016s	sys 0.000s
I20260709 12:11:14.496071 16214 raft_consensus.cc:3055] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:14.763744 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:14.546154 16223 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:14.538671 16260 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:14.564483 16126 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:14.569764 16120 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:14.541646 16145 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:14.589219 16241 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:14.771934 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:14.806684 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:14.753429 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:14.811157 12524 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:14.582006 16274 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:14.725488 16230 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:14.497047 16168 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:14.773069 16257 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:14.773510 16214 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:14.813442 12646 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:14.770509 12523 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:11:14.758824 16179 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:14.782022 16119 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:14.831059 16168 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:14.831627 16277 consensus_queue.cc:1059] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260709 12:11:14.832299 12522 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:11:14.816459 16145 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:14.836088 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1927) took 1550 ms. Trace:
I20260709 12:11:14.843703 16013 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:14.860230 16152 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.763s	user 0.010s	sys 0.091s
I20260709 12:11:14.794358 16070 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:14.801995 16062 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:14.863457 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 56174.4us
I20260709 12:11:14.808671 16105 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:14.779309 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:14.841945 16182 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.293s	user 0.009s	sys 0.001s
W20260709 12:11:14.870184 16198 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.129s	user 0.037s	sys 0.041s
I20260709 12:11:14.845644 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1917) took 1562 ms. Trace:
I20260709 12:11:14.590027 16293 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:14.872205 12676 rpcz_store.cc:276] 0709 12:11:13.286091 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.479699 (+193608us) service_pool.cc:224] Handling call
0709 12:11:13.480009 (+   310us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:13.639826 (+159817us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:14.835808 (+1195982us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54002c5954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:14.836018 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":360670592}
I20260709 12:11:14.821025 16106 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:11:14.827114 16148 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:14.868142 16179 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:14.801666 16260 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:14.925705 12600 raft_consensus.cc:2410] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
W20260709 12:11:14.926292 12525 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 1924) took 1342 ms (client timeout 1773 ms). Trace:
W20260709 12:11:14.938632 12525 rpcz_store.cc:269] 0709 12:11:13.542443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.545561 (+  3118us) service_pool.cc:224] Handling call
0709 12:11:14.884851 (+1339290us) spinlock_profiling.cc:243] Waited 57 ms on lock 0x7b54003c1c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:14.885060 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131201280}
W20260709 12:11:14.920702 13336 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 399) took 1687 ms (client timeout 1588 ms). Trace:
I20260709 12:11:14.940078 16179 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:14.940912 13336 rpcz_store.cc:269] 0709 12:11:13.233205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.233585 (+   380us) service_pool.cc:224] Handling call
0709 12:11:14.920308 (+1686723us) spinlock_profiling.cc:243] Waited 205 ms on lock 0x7b540055ee54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:14.920584 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":472461568}
W20260709 12:11:14.942947 12571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.721s	user 0.024s	sys 0.063s
I20260709 12:11:14.920392 13333 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 10: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:14.922079 16070 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:14.874717 16102 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:14.923233 12680 rpcz_store.cc:276] 0709 12:11:13.282824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.287826 (+  5002us) service_pool.cc:224] Handling call
0709 12:11:13.306873 (+ 19047us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:14.015859 (+708986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:14.845213 (+829354us) spinlock_profiling.cc:243] Waited 708 ms on lock 0x7b54003bdb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:14.845567 (+   354us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1628647680}
W20260709 12:11:14.923013 16198 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.184s	user 0.037s	sys 0.042s
I20260709 12:11:14.846122 16230 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
W20260709 12:11:14.944990 12571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.723s	user 0.024s	sys 0.063s
W20260709 12:11:14.978418 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 764389us
I20260709 12:11:14.979347 12571 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:14.938478 16308 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.911s	user 0.012s	sys 0.004s
I20260709 12:11:14.862880 16257 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:14.991593 12600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 2182) took 1642 ms (client timeout 1683 ms). Trace:
W20260709 12:11:14.995481 12600 rpcz_store.cc:269] 0709 12:11:13.349448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.462790 (+113342us) service_pool.cc:224] Handling call
0709 12:11:14.991256 (+1528466us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54003a11d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:14.991487 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3361435520}
I20260709 12:11:14.992249 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2020) took 2264 ms. Trace:
I20260709 12:11:14.993474 12571 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:14.996961 12598 rpcz_store.cc:276] 0709 12:11:12.714248 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:12.716365 (+  2117us) service_pool.cc:224] Handling call
0709 12:11:13.386445 (+670080us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:13.430550 (+ 44105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:14.978558 (+1548008us) spinlock_profiling.cc:243] Waited 670 ms on lock 0x7b540007ba50. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:14.978774 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1540315520}
I20260709 12:11:14.999583 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:14.999611 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:14.972371 16102 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:15.001888 12598 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 8: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:15.035030 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:15.050086 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 236041us
I20260709 12:11:15.054486 13305 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:11:15.050715 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 271479us
I20260709 12:11:15.063648 12494 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:14.880779 16152 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.010s	sys 0.092s
I20260709 12:11:15.069638 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:14.999388 16198 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:15.074047 16267 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:15.029175 16062 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:15.078884 16280 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:15.079299 13336 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:15.079514 16280 raft_consensus.cc:697] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:15.008455 16109 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:15.080655 16280 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:15.069039 16120 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:15.091938 16120 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:15.024319 16102 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:15.098915 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:15.105224 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:15.086366 16316 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:11:15.110332 12646 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:15.111812 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:15.113217 12600 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:14.993185 16308 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:14.880559 16145 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:15.085660 16241 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:15.094987 16191 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:15.087929 16274 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:11:15.128993 12671 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.993s	user 0.066s	sys 0.073s
I20260709 12:11:15.095175 16306 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election won for term 7
W20260709 12:11:15.136987 12671 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.001s	user 0.066s	sys 0.073s
I20260709 12:11:15.116739 16267 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:15.028987 16105 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:15.122885 12494 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:15.094579 16314 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:15.152248 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2263) took 1623 ms. Trace:
I20260709 12:11:15.138850 12528 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 10.
W20260709 12:11:15.156090 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 46101.5us
I20260709 12:11:15.130074 16062 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:15.169395 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2261) took 1853 ms. Trace:
I20260709 12:11:15.174775 16220 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:15.174944 13333 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 397) took 1942 ms. Trace:
I20260709 12:11:15.175390 13333 rpcz_store.cc:276] 0709 12:11:13.232581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.232731 (+   150us) service_pool.cc:224] Handling call
0709 12:11:15.174728 (+1941997us) spinlock_profiling.cc:243] Waited 1.53 s on lock 0x7b54001b49d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:15.174879 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3510926720}
I20260709 12:11:15.176779 13338 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: [7.8-9.9]
I20260709 12:11:15.177011 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:15.165454 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 64496.3us
W20260709 12:11:15.181109 12679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.408s	user 0.041s	sys 0.081s
W20260709 12:11:15.181988 12679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.409s	user 0.041s	sys 0.081s
I20260709 12:11:15.169778 12677 rpcz_store.cc:276] 0709 12:11:13.316577 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:14.022553 (+705976us) service_pool.cc:224] Handling call
0709 12:11:14.022929 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:14.023484 (+   555us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:11:14.031717 (+  8233us) log.cc:844] Serialized 14 byte log entry
0709 12:11:14.239025 (+207308us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:14.239592 (+   567us) log.cc:844] Serialized 26 byte log entry
0709 12:11:14.239770 (+   178us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:14.239811 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:14.239894 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:15.169173 (+929279us) raft_consensus.cc:1685] finished
0709 12:11:15.169196 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:15.169325 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":79730,"prepare.queue_time_us":9160,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":87376,"raft.run_cpu_time_us":289,"raft.run_wall_time_us":20412,"thread_start_us":134716,"threads_started":2,"wal-append.queue_time_us":780636}
I20260709 12:11:15.182972 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2255) took 1869 ms. Trace:
I20260709 12:11:15.181030 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:15.151749 16306 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:15.202083 16260 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:15.202836 16145 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:15.203097 16214 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:15.199790 16128 raft_consensus.cc:2764] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 12:11:15.205616 16321 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:11:15.206004 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:15.163718 16119 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:15.154930 16230 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:15.166653 16267 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:15.166566 16281 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:15.146412 16241 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:15.148243 16274 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:15.209820 12569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.987s	user 0.010s	sys 0.006s
W20260709 12:11:15.225482 12569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.061s	user 0.011s	sys 0.006s
I20260709 12:11:15.196390 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:15.206250 16120 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:15.230633 16101 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:15.227949 16182 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:15.232666 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:15.253204 12598 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 7: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:15.253854 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:15.254253 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:15.217751 16230 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:15.225765 16281 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:15.244462 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2169) took 1904 ms. Trace:
I20260709 12:11:15.200228 16284 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.032s
W20260709 12:11:15.258874 12572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.007s	user 0.030s	sys 0.031s
W20260709 12:11:15.268002 12572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.023s	user 0.030s	sys 0.031s
W20260709 12:11:15.268460 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 1.11595e+06us
W20260709 12:11:15.269172 12603 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.984s	user 0.014s	sys 0.046s
W20260709 12:11:15.271749 12603 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.986s	user 0.014s	sys 0.046s
I20260709 12:11:15.248572 12600 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:11:15.218875 16267 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:11:15.218469 12606 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.970s	user 0.034s	sys 0.119s
W20260709 12:11:15.278801 12606 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.031s	user 0.034s	sys 0.119s
I20260709 12:11:15.259825 12604 rpcz_store.cc:276] 0709 12:11:13.339925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.343201 (+  3276us) service_pool.cc:224] Handling call
0709 12:11:13.358696 (+ 15495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:13.583334 (+224638us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:13.583375 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:13.598798 (+ 15423us) log.cc:844] Serialized 26 byte log entry
0709 12:11:14.364622 (+765824us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:14.364681 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:14.364785 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:15.204963 (+840178us) spinlock_profiling.cc:243] Waited 3.08 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:15.243417 (+ 38454us) raft_consensus.cc:1685] finished
0709 12:11:15.243485 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:15.244369 (+   884us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":22562,"raft.run_cpu_time_us":54,"raft.run_wall_time_us":55,"spinlock_wait_cycles":7089536,"thread_start_us":883066,"threads_started":2,"wal-append.queue_time_us":1326906}
I20260709 12:11:15.230633 16306 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:15.285164 16128 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:15.289527 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:15.290584 12418 maintenance_manager.cc:419] P d5d8248387cf49cfbd3416fcacedae86: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033390
I20260709 12:11:15.251971 16062 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:15.261724 16281 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:15.259209 16070 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:15.261924 16257 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:15.318888 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:15.260520 16230 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:15.333094 12604 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:15.284011 16267 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:15.323107 12602 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.002s	user 0.005s	sys 0.013s
I20260709 12:11:15.259038 16182 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:15.339926 12602 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.021s	user 0.005s	sys 0.013s
I20260709 12:11:15.296060 12494 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:15.258644 16014 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.060s
I20260709 12:11:15.343881 16241 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:15.299440 16128 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:15.339316 16105 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:15.339450 16308 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:15.340762 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:15.345909 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49992.3us
I20260709 12:11:15.348017 13305 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:15.349596 16260 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:11:15.325048 16324 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:15.299644 16101 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260709 12:11:15.333109 16191 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:15.417730 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:15.417656 12644 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.082s	user 0.000s	sys 0.021s
I20260709 12:11:15.330965 16197 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:15.333132 16062 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:15.158285 12671 rpcz_store.cc:276] 0709 12:11:13.529229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:14.120594 (+591365us) service_pool.cc:224] Handling call
0709 12:11:14.134738 (+ 14144us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:14.134939 (+   201us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:11:14.135178 (+   239us) log.cc:844] Serialized 14 byte log entry
0709 12:11:15.151750 (+1016572us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:15.151830 (+    80us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:15.151898 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:15.152000 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:15.152159 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8978,"prepare.queue_time_us":1394,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":9166,"thread_start_us":1006732,"threads_started":1}
W20260709 12:11:15.335522 12531 consensus_meta.cc:287] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.603s	user 0.013s	sys 0.000s
I20260709 12:11:15.392292 13330 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
I20260709 12:11:15.392273 16119 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:11:15.406049 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:15.378700 16120 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:11:15.395853 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 107723us
W20260709 12:11:15.368088 12675 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.841s	user 0.002s	sys 0.024s
W20260709 12:11:15.492058 12675 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.965s	user 0.003s	sys 0.024s
I20260709 12:11:15.492970 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2268) took 1910 ms. Trace:
I20260709 12:11:15.493374 12675 rpcz_store.cc:276] 0709 12:11:13.582953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:14.496395 (+913442us) service_pool.cc:224] Handling call
0709 12:11:14.526312 (+ 29917us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:14.526616 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:11:14.527068 (+   452us) log.cc:844] Serialized 14 byte log entry
0709 12:11:15.492534 (+965466us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:15.492602 (+    68us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:15.492661 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:15.492743 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:15.492898 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44438,"prepare.queue_time_us":4708,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":83,"thread_start_us":965048,"threads_started":1}
I20260709 12:11:15.405845 12604 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
I20260709 12:11:15.501902 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:15.510217 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:15.512832 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:15.462438 12599 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.153s	user 0.011s	sys 0.013s
I20260709 12:11:15.458056 13336 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:11:15.472600 16191 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:15.475538 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:15.466101 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:15.482149 12531 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
I20260709 12:11:15.520854 12600 raft_consensus.cc:2376] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9. Re-sending same reply.
I20260709 12:11:15.521732 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:15.482110 13302 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.080s	user 0.016s	sys 0.027s
I20260709 12:11:15.486296 13338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 541) took 1675 ms. Trace:
I20260709 12:11:15.522507 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:15.522846 13338 rpcz_store.cc:276] 0709 12:11:13.811108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.811283 (+   175us) service_pool.cc:224] Handling call
0709 12:11:15.176466 (+1365183us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:15.177525 (+  1059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:15.485970 (+308445us) spinlock_profiling.cc:243] Waited 1.36 s on lock 0x7b54001b49d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:15.486197 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3138671872}
I20260709 12:11:15.393748 16105 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:11:15.472554 12644 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.137s	user 0.002s	sys 0.022s
I20260709 12:11:15.521168 13333 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:15.490216 16062 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:15.524014 13302 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.122s	user 0.017s	sys 0.027s
I20260709 12:11:15.523732 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:15.532524 12604 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
W20260709 12:11:15.533370 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 1.13194e+06us
W20260709 12:11:15.517050 12599 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.221s	user 0.011s	sys 0.013s
W20260709 12:11:15.525486 12526 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.245s	user 0.008s	sys 0.016s
W20260709 12:11:15.539629 12526 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.259s	user 0.008s	sys 0.017s
I20260709 12:11:15.549955 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:15.514185 16283 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.959s	user 0.010s	sys 0.006s
I20260709 12:11:15.511864 16119 raft_consensus.cc:697] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:15.617026 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:15.619278 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:15.421335 16324 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:15.523708 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:15.463182 16240 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.094s
I20260709 12:11:15.619876 12528 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 7: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:15.512614 16070 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:15.623660 12600 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:15.488583 16197 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:15.629802 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:15.636842 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:15.183588 12679 rpcz_store.cc:276] 0709 12:11:13.314023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.492796 (+178773us) service_pool.cc:224] Handling call
0709 12:11:13.493157 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:13.493393 (+   236us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:11:13.772611 (+279218us) log.cc:844] Serialized 14 byte log entry
0709 12:11:15.182598 (+1409987us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:15.182659 (+    61us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:15.182711 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:15.182783 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:15.182913 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":537389,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":152,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":6720,"thread_start_us":1677200,"threads_started":2}
I20260709 12:11:15.504535 16013 consensus_queue.cc:1059] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.112s
I20260709 12:11:15.619339 16158 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:15.629755 12494 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:15.630085 16274 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:15.526254 12601 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.230s	user 0.012s	sys 0.012s
W20260709 12:11:15.727213 12601 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.431s	user 0.013s	sys 0.012s
I20260709 12:11:15.632797 12676 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:11:15.529903 16289 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.157s	user 0.003s	sys 0.018s
I20260709 12:11:15.645924 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:15.505510 16120 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:15.514942 16314 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:15.741396 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2179) took 2394 ms. Trace:
I20260709 12:11:15.654606 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:15.618659 16268 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:15.574730 16164 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:15.617242 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:15.671351 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2180) took 2323 ms. Trace:
I20260709 12:11:15.751267 12604 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:15.616523 16283 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:15.672186 16197 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:15.654052 16119 consensus_queue.cc:237] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:15.657588 16267 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:15.791338 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:15.637508 12600 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
I20260709 12:11:15.796203 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:15.797032 12600 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:11:15.660446 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:15.628605 16152 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:15.729533 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:15.735124 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 105498us
I20260709 12:11:15.734957 12677 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:15.650061 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:15.740381 16211 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:15.644718 16070 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:15.740756 16114 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.345s	user 0.006s	sys 0.010s
I20260709 12:11:15.527814 16128 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:15.734722 16289 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.366s	user 0.003s	sys 0.019s
I20260709 12:11:15.751111 12606 rpcz_store.cc:276] 0709 12:11:13.347960 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:14.246495 (+898535us) service_pool.cc:224] Handling call
0709 12:11:14.246965 (+   470us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:14.247221 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:14.247248 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:14.247831 (+   583us) log.cc:844] Serialized 26 byte log entry
0709 12:11:15.279436 (+1031605us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:15.279504 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:15.279620 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:15.671074 (+391454us) raft_consensus.cc:1685] finished
0709 12:11:15.671100 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:15.671268 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":1031138,"threads_started":1,"wal-append.queue_time_us":1379484}
W20260709 12:11:15.756151 12678 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.160s	user 0.007s	sys 0.045s
I20260709 12:11:15.769479 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:15.680269 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:15.789811 12643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.461s	user 0.014s	sys 0.013s
I20260709 12:11:15.742971 12601 rpcz_store.cc:276] 0709 12:11:13.347455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:14.056281 (+708826us) service_pool.cc:224] Handling call
0709 12:11:14.056697 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:14.056971 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:11:14.295963 (+238992us) log.cc:844] Serialized 14 byte log entry
0709 12:11:15.738990 (+1443027us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:15.739075 (+    85us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:15.739152 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:15.739266 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:15.741299 (+  2033us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":255068,"prepare.queue_time_us":303539,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":19603,"raft.queue_time_us":12554,"raft.run_cpu_time_us":177,"raft.run_wall_time_us":179,"thread_start_us":1431523,"threads_started":1}
W20260709 12:11:15.837067 12678 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.241s	user 0.009s	sys 0.047s
W20260709 12:11:15.745582 15970 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.269s	user 0.014s	sys 0.054s
I20260709 12:11:15.801231 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:15.575157 16105 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:15.765127 16174 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.793s	user 0.013s	sys 0.032s
I20260709 12:11:15.826771 13336 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
W20260709 12:11:15.829510 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 169129us
I20260709 12:11:15.861294 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:15.862666 12674 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.089s	user 0.017s	sys 0.043s
W20260709 12:11:15.863214 12674 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.089s	user 0.017s	sys 0.043s
I20260709 12:11:15.864277 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2269) took 2274 ms. Trace:
W20260709 12:11:15.838313 16037 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.979s	user 0.008s	sys 0.045s
W20260709 12:11:15.850636 15970 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.379s	user 0.018s	sys 0.054s
I20260709 12:11:15.694552 16158 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:15.837428 12604 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
W20260709 12:11:15.840164 12643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.511s	user 0.014s	sys 0.013s
I20260709 12:11:15.825799 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:15.849422 12494 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:15.841663 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2191) took 2365 ms. Trace:
I20260709 12:11:15.885120 12601 rpcz_store.cc:276] 0709 12:11:13.476508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:15.840508 (+2364000us) service_pool.cc:224] Handling call
0709 12:11:15.840957 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:15.841199 (+   242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:15.841241 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:15.841318 (+    77us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:15.841369 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:15.841443 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:15.841600 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:15.789251 16182 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:15.886019 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:15.765273 16164 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:15.832828 16274 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:11:15.862377 16174 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.925s	user 0.013s	sys 0.036s
I20260709 12:11:15.908788 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:15.913532 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:15.888463 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 1.55945e+06us
I20260709 12:11:15.951022 12643 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:15.952646 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:15.887431 12675 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 4. Current term is 5. Ops: []
I20260709 12:11:15.750185 16314 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:15.765508 16268 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:15.877307 16037 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.018s	user 0.009s	sys 0.045s
W20260709 12:11:15.937361 12605 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.870s	user 0.025s	sys 0.034s
W20260709 12:11:15.832660 12527 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.029s	user 0.011s	sys 0.015s
I20260709 12:11:15.941702 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2193) took 2405 ms. Trace:
W20260709 12:11:15.960891 12605 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.894s	user 0.026s	sys 0.034s
I20260709 12:11:15.961063 12604 rpcz_store.cc:276] 0709 12:11:13.536115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:15.936809 (+2400694us) service_pool.cc:224] Handling call
0709 12:11:15.940926 (+  4117us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:15.941195 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:15.941246 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:15.941317 (+    71us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:15.941367 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:15.941456 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:15.941620 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:11:15.927603 16221 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.851s	user 0.017s	sys 0.031s
I20260709 12:11:15.944836 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:15.878808 16284 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader election won for term 9
W20260709 12:11:15.954651 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:15.956938 16070 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:15.833891 16114 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:15.962878 12527 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.160s	user 0.011s	sys 0.015s
W20260709 12:11:15.942226 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 92931.6us
I20260709 12:11:16.016629 12677 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:11:15.855427 16211 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:15.961725 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2188) took 2594 ms. Trace:
I20260709 12:11:16.029906 12604 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:16.082077 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:16.082579 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2194) took 2545 ms. Trace:
I20260709 12:11:16.082921 12604 rpcz_store.cc:276] 0709 12:11:13.537216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:15.962773 (+2425557us) service_pool.cc:224] Handling call
0709 12:11:16.022320 (+ 59547us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:16.082305 (+ 59985us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:16.082523 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:15.855672 16152 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:11:16.030418 12645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.744s	user 0.020s	sys 0.043s
I20260709 12:11:16.039906 13337 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 5: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:11:16.095148 12645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.809s	user 0.021s	sys 0.043s
I20260709 12:11:16.041384 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:16.006419 16221 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.930s	user 0.017s	sys 0.031s
W20260709 12:11:15.959712 16169 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.957s	user 0.010s	sys 0.020s
I20260709 12:11:16.030413 16223 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:16.022543 16220 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:11:16.028156 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 75601.8us
I20260709 12:11:16.063540 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:16.115908 12531 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:11:16.079918 12494 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:16.081668 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:16.034313 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:16.081790 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:16.094352 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1956) took 1385 ms. Trace:
I20260709 12:11:16.040522 16174 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:16.081650 12605 rpcz_store.cc:276] 0709 12:11:13.367661 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:14.725396 (+1357735us) service_pool.cc:224] Handling call
0709 12:11:14.725842 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:14.726138 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:11:15.066752 (+340614us) log.cc:844] Serialized 14 byte log entry
0709 12:11:15.961391 (+894639us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:15.961449 (+    58us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:15.961495 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:15.961562 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:15.961673 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":326312,"prepare.queue_time_us":347576,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":746,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":6259,"thread_start_us":894222,"threads_started":1}
W20260709 12:11:16.098060 13304 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.043s	user 0.024s	sys 0.045s
I20260709 12:11:16.098309 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:16.104322 12598 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:16.117694 12494 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:16.088012 16324 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:16.103783 16169 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.109s	user 0.011s	sys 0.021s
W20260709 12:11:16.100342 12672 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.306s	user 0.030s	sys 0.061s
W20260709 12:11:16.124902 13304 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.070s	user 0.024s	sys 0.045s
W20260709 12:11:16.124855 16148 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.967s	user 0.011s	sys 0.027s
I20260709 12:11:16.132826 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2195) took 1339 ms. Trace:
I20260709 12:11:16.114650 16284 raft_consensus.cc:697] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 9 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:16.132225 16120 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:16.142879 12605 rpcz_store.cc:276] 0709 12:11:14.794007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:16.131902 (+1337895us) service_pool.cc:224] Handling call
0709 12:11:16.132293 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:16.132474 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:16.132499 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:16.132545 (+    46us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:16.132594 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:16.132651 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:16.132767 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:16.144111 16164 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:16.143606 16284 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:16.144606 16164 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:16.145185 16164 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:16.116191 16293 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.950s	user 0.011s	sys 0.029s
I20260709 12:11:16.100320 16013 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:16.103420 16223 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:16.147605 16223 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:16.136969 12672 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.342s	user 0.030s	sys 0.061s
I20260709 12:11:16.139122 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:16.137964 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2196) took 1341 ms. Trace:
I20260709 12:11:16.096690 16191 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:16.146420 16268 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:16.129024 16174 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:11:16.146636 16328 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.199s	user 0.000s	sys 0.000s
I20260709 12:11:16.135164 16101 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:16.123420 12527 rpcz_store.cc:276] 0709 12:11:14.708793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:14.739562 (+ 30769us) service_pool.cc:224] Handling call
0709 12:11:14.740050 (+   488us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:14.740558 (+   508us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:11:14.803009 (+ 62451us) log.cc:844] Serialized 14 byte log entry
0709 12:11:16.093792 (+1290783us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:16.093871 (+    79us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:16.093950 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:16.094053 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:16.094252 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9538,"prepare.queue_time_us":49760,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":56,"raft.queue_time_us":16916,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":16036,"thread_start_us":1276177,"threads_started":1}
I20260709 12:11:16.144606 12605 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:11:16.174320 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2198) took 1377 ms. Trace:
I20260709 12:11:16.149906 12604 rpcz_store.cc:276] 0709 12:11:14.796501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:16.134631 (+1338130us) service_pool.cc:224] Handling call
0709 12:11:16.137299 (+  2668us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:16.137505 (+   206us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:16.137545 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:16.137608 (+    63us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:16.137660 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:16.137735 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:16.137875 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:11:16.149966 16148 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.991s	user 0.011s	sys 0.027s
W20260709 12:11:16.139894 13304 connection.cc:256] RPC call timeout handler was delayed by -1.3211911030000465s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15380000000000002s for a response
W20260709 12:11:16.176973 13304 connection.cc:256] RPC call timeout handler was delayed by -1.2635911029999534s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1602s for a response
W20260709 12:11:16.177558 13304 connection.cc:256] RPC call timeout handler was delayed by -1.1825911030000498s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16920000000000002s for a response
W20260709 12:11:16.172062 12523 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.944s	user 0.014s	sys 0.033s
I20260709 12:11:16.174961 12605 rpcz_store.cc:276] 0709 12:11:14.797170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:16.143970 (+1346800us) service_pool.cc:224] Handling call
0709 12:11:16.144279 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:16.173896 (+ 29617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:16.174228 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:11:16.149013 12597 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.749s	user 0.026s	sys 0.036s
W20260709 12:11:16.181705 12597 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.782s	user 0.026s	sys 0.036s
W20260709 12:11:16.182639 12523 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.955s	user 0.014s	sys 0.033s
I20260709 12:11:16.173283 16101 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:16.148720 16333 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:16.201198 12604 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 10 index: 5. (index mismatch)
I20260709 12:11:16.187301 16101 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:16.171085 16283 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:16.175195 16214 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:16.169581 16174 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:16.228793 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2187) took 2867 ms. Trace:
I20260709 12:11:16.199958 15977 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260709 12:11:16.147603 16293 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.981s	user 0.012s	sys 0.029s
W20260709 12:11:16.208123 16319 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.882s	user 0.004s	sys 0.012s
W20260709 12:11:16.256788 13332 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 401) took 3015 ms (client timeout 2072 ms). Trace:
W20260709 12:11:16.257647 13332 rpcz_store.cc:269] 0709 12:11:13.233831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.263464 (+ 29633us) service_pool.cc:224] Handling call
0709 12:11:16.248744 (+2985280us) spinlock_profiling.cc:243] Waited 2.68 s on lock 0x7b540047b8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:16.249001 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6159134080}
I20260709 12:11:16.229449 12597 rpcz_store.cc:276] 0709 12:11:13.361914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:14.398677 (+1036763us) service_pool.cc:224] Handling call
0709 12:11:14.399063 (+   386us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:14.399439 (+   376us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:11:14.399908 (+   469us) log.cc:844] Serialized 14 byte log entry
0709 12:11:16.208731 (+1808823us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:16.226515 (+ 17784us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:16.228442 (+  1927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:16.228540 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:16.228715 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":470255,"prepare.queue_time_us":141816,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":44105,"thread_start_us":1781924,"threads_started":1}
I20260709 12:11:16.206343 16334 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:16.292598 13304 leader_election.cc:336] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.538s (SENT)
I20260709 12:11:16.293262 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:16.261380 16280 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.176s	user 0.045s	sys 0.062s
W20260709 12:11:16.296257 16280 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.211s	user 0.045s	sys 0.062s
I20260709 12:11:16.298264 13335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38472 (request call id 371) took 2078 ms. Trace:
I20260709 12:11:16.298609 16152 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:11:16.299206 16126 raft_consensus.cc:3055] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:16.298904 13335 rpcz_store.cc:276] 0709 12:11:14.222584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:14.244472 (+ 21888us) service_pool.cc:224] Handling call
0709 12:11:16.298047 (+2053575us) spinlock_profiling.cc:243] Waited 1.99 s on lock 0x7b54001261d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:16.298199 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4581898112}
W20260709 12:11:16.302863 13304 leader_election.cc:336] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.602s (SENT)
I20260709 12:11:16.304294 16240 raft_consensus.cc:3055] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:11:16.304744 16240 raft_consensus.cc:740] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:16.298146 16041 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:16.307747 16041 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:16.307828 16220 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:16.308444 16041 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:16.250221 16318 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.930s	user 0.002s	sys 0.013s
W20260709 12:11:16.310321 12598 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 307) took 1308 ms (client timeout 1660 ms). Trace:
W20260709 12:11:16.310853 12598 rpcz_store.cc:269] 0709 12:11:15.001551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:15.253948 (+252397us) service_pool.cc:224] Handling call
0709 12:11:16.310013 (+1056065us) spinlock_profiling.cc:243] Waited 844 ms on lock 0x7b5400231c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:16.310235 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1940970752}
I20260709 12:11:16.257524 16158 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:16.248219 16264 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.171s	user 0.014s	sys 0.020s
I20260709 12:11:16.299618 16152 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:16.210790 16333 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:16.291265 16105 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:16.299799 16126 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:11:16.371482 16340 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 12:11:16.372054 16126 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:16.363075 16114 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:16.333253 16334 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:16.382849 16164 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:16.383780 16334 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:16.366829 16338 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.267s	user 0.003s	sys 0.000s
W20260709 12:11:16.367178 16332 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.337s	user 0.002s	sys 0.000s
W20260709 12:11:16.368638 16317 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.027s	user 0.003s	sys 0.008s
I20260709 12:11:16.390230 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2167) took 3051 ms. Trace:
I20260709 12:11:16.390743 12603 rpcz_store.cc:276] 0709 12:11:13.339279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.342265 (+  2986us) service_pool.cc:224] Handling call
0709 12:11:14.133510 (+791245us) spinlock_profiling.cc:243] Waited 2.23 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fbeee 000000000036eb39 00007f148d96f1e0 00007f148da380c4 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:11:14.133676 (+   166us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:14.134094 (+   418us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:11:14.256900 (+122806us) log.cc:844] Serialized 14 byte log entry
0709 12:11:15.280748 (+1023848us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:15.281856 (+  1108us) log.cc:844] Serialized 26 byte log entry
0709 12:11:15.282041 (+   185us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:15.282089 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:15.306626 (+ 24537us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:16.356678 (+1050052us) raft_consensus.cc:1685] finished
0709 12:11:16.389951 (+ 33273us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:16.390156 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":102432,"prepare.queue_time_us":189783,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":32366,"raft.run_cpu_time_us":322,"raft.run_wall_time_us":39333,"spinlock_wait_cycles":6200192,"thread_start_us":989281,"threads_started":1,"wal-append.queue_time_us":1550948}
I20260709 12:11:16.392886 16211 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:16.395941 16013 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:16.396513 16013 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:16.401337 16267 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:16.370563 16145 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:16.403066 16145 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:15.864732 12674 rpcz_store.cc:276] 0709 12:11:13.589656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:14.581941 (+992285us) service_pool.cc:224] Handling call
0709 12:11:14.582509 (+   568us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:14.582853 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:11:14.773461 (+190608us) log.cc:844] Serialized 14 byte log entry
0709 12:11:15.863896 (+1090435us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:15.863961 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:15.864016 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:15.864080 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:15.864222 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":144577,"prepare.queue_time_us":212967,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":66,"raft.queue_time_us":212,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":9804,"thread_start_us":1089984,"threads_started":1}
I20260709 12:11:16.317152 16293 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:16.361939 16264 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.285s	user 0.014s	sys 0.021s
I20260709 12:11:16.410230 16114 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:16.410949 16114 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:16.370762 16329 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
W20260709 12:11:16.421399 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:16.417380 12569 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:16.439369 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:16.442377 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:16.447060 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:16.448181 12531 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:16.413793 16114 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:16.455029 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:16.458601 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:16.459005 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:16.460237 12531 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:16.493166 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:16.490527 16283 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:16.501058 16283 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:16.501186 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:16.501506 16283 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:16.501853 12525 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:16.502540 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:16.503495 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:16.504820 16198 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:16.505272 16198 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:16.505640 16198 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:16.507207 16333 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:16.508140 15977 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:16.508685 15977 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:11:16.509618 12496 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.034s	user 0.030s	sys 0.074s
I20260709 12:11:16.357085 16314 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:16.305694 16240 consensus_queue.cc:260] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:16.512259 12496 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.037s	user 0.030s	sys 0.075s
I20260709 12:11:16.430722 16152 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:16.513988 12572 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:16.490759 16317 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.102s	user 0.000s	sys 0.006s
W20260709 12:11:16.495432 16341 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
W20260709 12:11:16.497224 16332 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.110s	user 0.000s	sys 0.006s
W20260709 12:11:16.498837 16344 log.cc:927] Time spent T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:11:16.497888 16308 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:16.539367 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:16.540684 12677 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 5.
W20260709 12:11:16.541514 12673 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.904s	user 0.009s	sys 0.019s
I20260709 12:11:16.498982 16114 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:16.541910 12673 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.905s	user 0.009s	sys 0.019s
W20260709 12:11:16.568231 13330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.941s	user 0.016s	sys 0.042s
W20260709 12:11:16.572929 13330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.946s	user 0.017s	sys 0.042s
I20260709 12:11:16.574064 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1931) took 2981 ms. Trace:
I20260709 12:11:16.574522 12526 rpcz_store.cc:276] 0709 12:11:13.558972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:14.030951 (+471979us) service_pool.cc:224] Handling call
0709 12:11:14.031463 (+   512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:14.122063 (+ 90600us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:14.122131 (+    68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:14.141516 (+ 19385us) log.cc:844] Serialized 26 byte log entry
0709 12:11:15.617654 (+1476138us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:11:15.617712 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:15.617801 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:16.539797 (+921996us) raft_consensus.cc:1685] finished
0709 12:11:16.539818 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:16.539981 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":90034,"raft.queue_time_us":95193,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":70463,"thread_start_us":1259747,"threads_started":1,"wal-append.queue_time_us":1907803}
I20260709 12:11:16.572816 12525 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:16.405534 16145 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:16.585407 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:16.578032 16182 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:16.589618 12677 raft_consensus.cc:2393] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 8: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:11:16.567760 16281 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.227s	user 0.011s	sys 0.003s
I20260709 12:11:16.543007 16314 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:16.611775 16014 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:11:16.605898 16182 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:16.578943 16114 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:16.575914 16308 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:16.636375 16346 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:11:16.617494 16041 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:16.628926 16102 consensus_meta.cc:287] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.508s	user 0.009s	sys 0.007s
I20260709 12:11:16.588090 16267 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:16.584919 16260 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:16.704579 16126 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:16.705363 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2171) took 3364 ms. Trace:
W20260709 12:11:16.623178 16306 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.104s	user 0.018s	sys 0.015s
W20260709 12:11:16.698074 12522 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.220s	user 0.033s	sys 0.054s
I20260709 12:11:16.705678 15977 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:16.621487 16105 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.120s
I20260709 12:11:16.625955 16182 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:16.742987 12522 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.265s	user 0.033s	sys 0.054s
I20260709 12:11:16.739223 12602 rpcz_store.cc:276] 0709 12:11:13.340541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:13.675304 (+334763us) service_pool.cc:224] Handling call
0709 12:11:13.747332 (+ 72028us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:14.043083 (+295751us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:11:14.138650 (+ 95567us) log.cc:844] Serialized 14 byte log entry
0709 12:11:15.340787 (+1202137us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:15.345043 (+  4256us) log.cc:844] Serialized 26 byte log entry
0709 12:11:15.345197 (+   154us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:15.345230 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:15.345305 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:16.705110 (+1359805us) raft_consensus.cc:1685] finished
0709 12:11:16.705133 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:16.705285 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46133,"prepare.queue_time_us":155982,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":52,"raft.queue_time_us":268274,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":13695,"thread_start_us":1306253,"threads_started":2,"wal-append.queue_time_us":1601538}
W20260709 12:11:16.731040 12497 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.994s	user 0.012s	sys 0.030s
W20260709 12:11:16.750797 12524 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.962s	user 0.016s	sys 0.026s
W20260709 12:11:16.755267 12497 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.018s	user 0.012s	sys 0.030s
W20260709 12:11:16.755492 12524 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.967s	user 0.017s	sys 0.026s
W20260709 12:11:16.746412 16306 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.229s	user 0.019s	sys 0.016s
I20260709 12:11:16.619700 16281 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:16.750735 16257 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.008s	user 0.011s	sys 0.031s
W20260709 12:11:16.770328 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:16.757571 12678 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.886s	user 0.009s	sys 0.017s
I20260709 12:11:16.628494 16314 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:16.634089 16223 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:16.870905 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:16.871387 12678 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.029s	user 0.011s	sys 0.026s
I20260709 12:11:16.873198 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:16.733070 16102 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:16.928015 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:16.928516 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2267) took 3369 ms. Trace:
W20260709 12:11:16.813262 16257 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.071s	user 0.011s	sys 0.032s
I20260709 12:11:16.932004 12528 raft_consensus.cc:2376] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9. Re-sending same reply.
I20260709 12:11:16.779003 16152 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:16.932036 12678 rpcz_store.cc:276] 0709 12:11:13.559376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:14.539538 (+980162us) service_pool.cc:224] Handling call
0709 12:11:14.539997 (+   459us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:14.540302 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:11:14.596094 (+ 55792us) log.cc:844] Serialized 14 byte log entry
0709 12:11:16.927698 (+2331604us) log.cc:844] Serialized 14 byte log entry
0709 12:11:16.927957 (+   259us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:16.928035 (+    78us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:16.928112 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:16.928207 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:16.928413 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2406,"prepare.queue_time_us":722616,"prepare.run_cpu_time_us":124,"prepare.run_wall_time_us":17591,"raft.queue_time_us":8388,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":16284,"thread_start_us":2282407,"threads_started":2,"wal-append.queue_time_us":1694931}
I20260709 12:11:16.750083 16260 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:16.938855 12643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.006s	sys 0.035s
W20260709 12:11:16.939271 12643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.806s	user 0.007s	sys 0.035s
W20260709 12:11:16.940940 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 824988us
W20260709 12:11:16.942499 13302 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.003s	user 0.005s	sys 0.017s
W20260709 12:11:16.943545 13302 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.004s	user 0.005s	sys 0.017s
I20260709 12:11:16.946331 12569 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:11:16.933447 16230 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.565s	user 0.010s	sys 0.013s
I20260709 12:11:16.951309 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:16.977437 16337 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:11:16.954151 16211 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:11:16.983296 16041 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:16.935400 12600 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.062s	user 0.007s	sys 0.020s
W20260709 12:11:16.935632 12606 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.080s	user 0.013s	sys 0.046s
W20260709 12:11:16.989722 12606 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.135s	user 0.014s	sys 0.046s
I20260709 12:11:16.746678 16267 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:16.989722 12600 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.155s	user 0.008s	sys 0.022s
I20260709 12:11:17.005487 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2189) took 3630 ms. Trace:
W20260709 12:11:16.995262 12680 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.964s	user 0.001s	sys 0.041s
W20260709 12:11:17.011428 12680 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.988s	user 0.002s	sys 0.042s
W20260709 12:11:17.006920 12645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.910s	user 0.016s	sys 0.039s
W20260709 12:11:17.012756 12645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.916s	user 0.017s	sys 0.039s
I20260709 12:11:16.949766 16230 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:17.017333 12679 raft_consensus.cc:2368] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9.
I20260709 12:11:16.941911 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:17.023089 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 81315.9us
I20260709 12:11:17.023178 16333 raft_consensus.cc:2764] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:11:17.024048 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:17.024889 16333 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:17.031955 16230 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:17.032946 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:17.037285 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:16.976082 16214 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.746s	user 0.008s	sys 0.004s
I20260709 12:11:17.040299 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:17.052505 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:17.052560 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:17.047706 12678 raft_consensus.cc:2410] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 11 because replica has last-logged OpId of term: 10 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 4.
I20260709 12:11:17.053889 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:16.996140 16322 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.517s	user 0.018s	sys 0.010s
I20260709 12:11:17.054087 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:17.058449 12671 raft_consensus.cc:2410] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 8 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:11:17.058835 12677 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:17.059870 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:17.055177 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:17.060822 12678 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:17.061800 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:17.062851 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:17.065114 12598 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.704s	user 0.007s	sys 0.008s
W20260709 12:11:17.055346 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:17.047503 12673 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
W20260709 12:11:17.065526 12598 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.007s	sys 0.008s
I20260709 12:11:17.066977 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:17.066947 12677 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:17.071494 12673 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:17.075343 13302 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:17.072504 12672 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.915s	user 0.003s	sys 0.040s
W20260709 12:11:17.079175 12672 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.930s	user 0.003s	sys 0.040s
I20260709 12:11:17.081056 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:17.082820 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:17.083004 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2271) took 3474 ms. Trace:
I20260709 12:11:17.083810 12673 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 7: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:17.085887 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:17.086759 12674 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:17.083683 12672 rpcz_store.cc:276] 0709 12:11:13.608458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:14.545219 (+936761us) service_pool.cc:224] Handling call
0709 12:11:14.545553 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:14.545831 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:11:16.148524 (+1602693us) log.cc:844] Serialized 14 byte log entry
0709 12:11:17.082035 (+933511us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:17.082111 (+    76us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:17.082678 (+   567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:17.082779 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:17.082933 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":227984,"prepare.queue_time_us":263194,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"thread_start_us":2286479,"threads_started":2}
I20260709 12:11:17.025343 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:17.094283 13333 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:17.094681 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:17.095955 12676 raft_consensus.cc:2410] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:11:17.062321 16334 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:17.103470 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 78217.9us
I20260709 12:11:17.104931 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:17.104297 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:17.119275 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:17.092859 16347 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:11:17.106243 16179 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.844s	user 0.020s	sys 0.018s
I20260709 12:11:17.131158 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:17.135525 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:17.097972 12677 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:17.137657 12643 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:17.118630 15977 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:17.115475 16333 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260709 12:11:17.107676 16169 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.965s	user 0.022s	sys 0.032s
W20260709 12:11:17.147951 12604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.831s	user 0.010s	sys 0.014s
I20260709 12:11:17.150031 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:17.146783 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:17.156306 13330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 373) took 1607 ms. Trace:
I20260709 12:11:17.157469 12673 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
W20260709 12:11:17.127200 16179 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.869s	user 0.021s	sys 0.018s
I20260709 12:11:17.157490 12674 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
W20260709 12:11:17.163597 16350 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 12:11:17.159951 16114 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:17.159157 12604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.843s	user 0.010s	sys 0.014s
W20260709 12:11:17.124152 16284 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.976s	user 0.021s	sys 0.026s
I20260709 12:11:17.126507 16347 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:17.162264 12672 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 5: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:17.111408 16334 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:17.148015 16230 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:17.157657 13330 rpcz_store.cc:276] 0709 12:11:15.548969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:15.549194 (+   225us) service_pool.cc:224] Handling call
0709 12:11:15.549743 (+   549us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:15.575780 (+ 26037us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:15.575830 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:15.616542 (+ 40712us) log.cc:844] Serialized 26 byte log entry
0709 12:11:16.574108 (+957566us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:16.574194 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:16.574295 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:17.156093 (+581798us) raft_consensus.cc:1685] finished
0709 12:11:17.156113 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:17.156244 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25432,"raft.queue_time_us":26148,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":20349,"thread_start_us":946458,"threads_started":1,"wal-append.queue_time_us":1449670}
W20260709 12:11:17.171535 16284 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.023s	user 0.021s	sys 0.026s
I20260709 12:11:17.151043 16308 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:17.176049 13335 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10. Ops: []
W20260709 12:11:17.162669 16101 consensus_meta.cc:287] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.953s	user 0.007s	sys 0.007s
I20260709 12:11:17.146371 16306 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 12:11:17.159099 16152 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:17.157027 16169 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.014s	user 0.022s	sys 0.032s
W20260709 12:11:17.179726 16128 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.876s	user 0.004s	sys 0.016s
W20260709 12:11:17.163205 16148 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.933s	user 0.010s	sys 0.009s
W20260709 12:11:17.181916 12597 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.797s	user 0.014s	sys 0.003s
I20260709 12:11:17.175511 16223 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:17.188658 12679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2279) took 1544 ms (client timeout 1556 ms). Trace:
W20260709 12:11:17.186331 16241 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.800s	user 0.005s	sys 0.013s
W20260709 12:11:17.185205 16317 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.646s	user 0.001s	sys 0.004s
W20260709 12:11:17.217260 12572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.639s	user 0.005s	sys 0.016s
I20260709 12:11:17.176203 16347 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:17.181334 16333 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:17.210886 16354 log.cc:927] Time spent T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.273s	user 0.001s	sys 0.000s
W20260709 12:11:17.189960 16358 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
W20260709 12:11:17.334353 16357 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
W20260709 12:11:17.499619 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:17.297093 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:11:17.330852 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:17.319159 16101 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:17.248200 16264 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.840s	user 0.008s	sys 0.019s
W20260709 12:11:17.350804 12603 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.927s	user 0.003s	sys 0.032s
W20260709 12:11:17.660085 12603 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.266s	user 0.004s	sys 0.032s
I20260709 12:11:17.191068 16102 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:17.351853 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:17.669600 12674 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:17.335039 16267 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:17.369408 12675 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.759s	user 0.034s	sys 0.018s
W20260709 12:11:17.188144 16339 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.874s	user 0.003s	sys 0.022s
I20260709 12:11:17.213104 16314 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:17.279239 16260 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:17.541371 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:17.515033 16355 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.468s	user 0.000s	sys 0.001s
I20260709 12:11:17.506531 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:17.616793 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:17.652570 12597 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.267s	user 0.018s	sys 0.009s
I20260709 12:11:17.770884 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:17.653131 12572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.107s	user 0.010s	sys 0.021s
W20260709 12:11:17.652647 16241 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.267s	user 0.009s	sys 0.021s
I20260709 12:11:17.771746 12676 raft_consensus.cc:2393] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 11: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:11:17.347855 12571 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.776s	user 0.022s	sys 0.030s
W20260709 12:11:17.653738 12679 rpcz_store.cc:269] 0709 12:11:15.644070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:15.679925 (+ 35855us) service_pool.cc:224] Handling call
0709 12:11:17.185253 (+1505328us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54000131d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:17.188575 (+  3322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2712419584}
W20260709 12:11:17.420866 16366 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.087s	user 0.006s	sys 0.000s
W20260709 12:11:17.452672 16360 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.343s	user 0.001s	sys 0.006s
I20260709 12:11:17.506367 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:17.779789 12677 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:17.346920 16179 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:17.672134 12675 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.083s	user 0.034s	sys 0.019s
I20260709 12:11:17.672777 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:17.467856 16364 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
W20260709 12:11:17.468219 12601 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.968s	user 0.014s	sys 0.033s
W20260709 12:11:17.789330 12601 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.293s	user 0.028s	sys 0.043s
W20260709 12:11:17.505985 15970 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.777s	user 0.021s	sys 0.042s
W20260709 12:11:17.372315 16128 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.069s	user 0.005s	sys 0.016s
W20260709 12:11:17.651430 16148 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.424s	user 0.015s	sys 0.014s
W20260709 12:11:17.427731 16221 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.286s	user 0.026s	sys 0.019s
W20260709 12:11:17.435577 16119 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.020s	user 0.014s	sys 0.030s
W20260709 12:11:17.762524 13304 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.583s (SENT) [suppressed 15 similar messages]
I20260709 12:11:17.643013 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:17.771076 12673 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:17.773351 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:17.774763 12643 outbound_call.cc:321] RPC callback for RPC call kudu.master.MasterService.TSHeartbeat -> {remote=127.11.13.190:46629, user_credentials={real_user=slave}} blocked reactor thread for 95997.3us
W20260709 12:11:17.669693 16264 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.262s	user 0.008s	sys 0.019s
W20260709 12:11:17.675788 16339 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.381s	user 0.004s	sys 0.023s
W20260709 12:11:17.795652 15970 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.067s	user 0.022s	sys 0.061s
W20260709 12:11:17.811053 16221 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.678s	user 0.042s	sys 0.023s
W20260709 12:11:17.788852 16105 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.023s	sys 0.044s
I20260709 12:11:17.669333 16333 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:17.814288 16357 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
W20260709 12:11:17.818921 16119 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.396s	user 0.021s	sys 0.050s
I20260709 12:11:17.864715 12677 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:18.021142 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1965) took 2911 ms. Trace:
W20260709 12:11:17.500754 12494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.320s	user 0.040s	sys 0.072s
W20260709 12:11:17.892978 12678 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.743s	user 0.009s	sys 0.037s
W20260709 12:11:18.044466 12494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.864s	user 0.048s	sys 0.084s
W20260709 12:11:17.865720 12680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.853s	user 0.015s	sys 0.068s
W20260709 12:11:18.045131 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 1.86503e+06us
W20260709 12:11:18.045567 12680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.033s	user 0.015s	sys 0.068s
W20260709 12:11:18.022069 12527 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.083s	user 0.037s	sys 0.044s
W20260709 12:11:18.046499 12527 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.110s	user 0.038s	sys 0.044s
I20260709 12:11:18.047966 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 339) took 1174 ms. Trace:
W20260709 12:11:17.373529 12496 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.760s	user 0.026s	sys 0.027s
W20260709 12:11:18.049247 12496 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.435s	user 0.026s	sys 0.027s
I20260709 12:11:18.048373 12527 rpcz_store.cc:276] 0709 12:11:16.873429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:16.874108 (+   679us) service_pool.cc:224] Handling call
0709 12:11:16.874748 (+   640us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:18.047280 (+1172532us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:18.047371 (+    91us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:18.047437 (+    66us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:18.047504 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:18.047653 (+   149us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:18.047879 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65398,"raft.queue_time_us":174858,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":33698,"thread_start_us":1109924,"threads_started":1}
W20260709 12:11:18.049685 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 1.43603e+06us
I20260709 12:11:18.027534 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:18.034971 12571 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.463s	user 0.022s	sys 0.031s
I20260709 12:11:17.678761 12529 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:18.050838 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 1.51773e+06us
I20260709 12:11:18.050976 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1969) took 1097 ms. Trace:
W20260709 12:11:17.771883 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 1.25791e+06us
I20260709 12:11:18.051584 12526 rpcz_store.cc:276] 0709 12:11:16.953560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:18.026454 (+1072894us) service_pool.cc:224] Handling call
0709 12:11:18.050885 (+ 24431us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:11:17.836701 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:18.053403 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:11:18.053813 12643 connection.cc:256] RPC call timeout handler was delayed by -3.05085064900004s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
W20260709 12:11:18.054426 12643 connection.cc:256] RPC call timeout handler was delayed by -3.0490506490000371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
W20260709 12:11:18.054934 12643 connection.cc:256] RPC call timeout handler was delayed by -3.0319506490000094s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15880000000000002s for a response
W20260709 12:11:18.055752 12643 connection.cc:256] RPC call timeout handler was delayed by -2.6674506490001022s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19930000000000003s for a response
W20260709 12:11:18.056339 12643 connection.cc:256] RPC call timeout handler was delayed by -2.5963506489999872s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20720000000000002s for a response
W20260709 12:11:18.034770 12598 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.919s	user 0.019s	sys 0.018s
W20260709 12:11:18.052031 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 393267us
W20260709 12:11:18.058513 12598 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.943s	user 0.019s	sys 0.018s
W20260709 12:11:18.059450 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 296833us
I20260709 12:11:18.043990 12523 rpcz_store.cc:276] 0709 12:11:15.109137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:15.117543 (+  8406us) service_pool.cc:224] Handling call
0709 12:11:15.118099 (+   556us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:15.171900 (+ 53801us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:15.171929 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:15.172464 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 12:11:16.183178 (+1010714us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:11:16.183238 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:16.530895 (+347657us) spinlock_profiling.cc:243] Waited 53.4 ms on lock 0x7b54003f8754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:16.530962 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:18.020073 (+1489111us) raft_consensus.cc:1685] finished
0709 12:11:18.020147 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:18.020770 (+   623us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":777,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":21900,"spinlock_wait_cycles":122804096,"thread_start_us":955862,"threads_started":1,"wal-append.queue_time_us":1553190}
I20260709 12:11:18.036196 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2175) took 4694 ms. Trace:
I20260709 12:11:18.061846 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:18.062374 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:18.025591 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 334) took 2595 ms. Trace:
I20260709 12:11:18.064301 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:18.064745 12677 raft_consensus.cc:2410] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:11:18.062446 12599 rpcz_store.cc:276] 0709 12:11:13.341993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:14.031976 (+689983us) service_pool.cc:224] Handling call
0709 12:11:14.032292 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:14.032655 (+   363us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:11:14.032980 (+   325us) log.cc:844] Serialized 14 byte log entry
0709 12:11:15.538592 (+1505612us) log.cc:844] Serialized 14 byte log entry
0709 12:11:15.575568 (+ 36976us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:15.625745 (+ 50177us) log.cc:844] Serialized 26 byte log entry
0709 12:11:15.625964 (+   219us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:15.626012 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:15.626109 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:16.261271 (+635162us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:17.176966 (+915695us) spinlock_profiling.cc:243] Waited 9.54 ms on lock 0x7b240035fdc8. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14857b99c4 00007f14857b7acb 00007f1485972dc9 00007f1485972a55 00007f148597209d 00007f14885a410e 00007f14885bb88c 00007f14885babb6 00007f14885ba7d8 00007f14885c4221
0709 12:11:18.035446 (+858480us) raft_consensus.cc:1685] finished
0709 12:11:18.035495 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:18.035859 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":275839,"prepare.queue_time_us":367294,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":105,"raft.queue_time_us":280504,"raft.run_cpu_time_us":372,"raft.run_wall_time_us":24925,"spinlock_wait_cycles":52000512,"thread_start_us":1352356,"threads_started":2,"wal-append.queue_time_us":1261570}
I20260709 12:11:18.065757 12679 raft_consensus.cc:2410] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:11:18.066571 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:18.067761 12676 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 7
I20260709 12:11:18.067911 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2286) took 2214 ms. Trace:
W20260709 12:11:18.045143 12678 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.895s	user 0.009s	sys 0.037s
I20260709 12:11:18.063347 12523 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:11:18.037976 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2190) took 4562 ms. Trace:
I20260709 12:11:18.071828 12606 rpcz_store.cc:276] 0709 12:11:13.475283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:15.837499 (+2362216us) service_pool.cc:224] Handling call
0709 12:11:15.837896 (+   397us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:15.843503 (+  5607us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:15.843536 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:15.854862 (+ 11326us) log.cc:844] Serialized 26 byte log entry
0709 12:11:17.004159 (+1149297us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:17.004215 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:17.004308 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:18.037259 (+1032951us) raft_consensus.cc:1685] finished
0709 12:11:18.037316 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:18.037735 (+   419us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5118,"raft.queue_time_us":5815,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":9133,"thread_start_us":1135038,"threads_started":1,"wal-append.queue_time_us":1329223}
I20260709 12:11:18.074697 12529 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
W20260709 12:11:17.816679 13337 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.018s	sys 0.027s
W20260709 12:11:18.087293 13337 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.993s	user 0.018s	sys 0.027s
I20260709 12:11:18.063376 12522 rpcz_store.cc:276] 0709 12:11:15.429696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:15.459098 (+ 29402us) service_pool.cc:224] Handling call
0709 12:11:15.473224 (+ 14126us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:15.474852 (+  1628us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:11:15.477896 (+  3044us) log.cc:844] Serialized 14 byte log entry
0709 12:11:16.754407 (+1276511us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:16.766767 (+ 12360us) log.cc:844] Serialized 26 byte log entry
0709 12:11:16.766948 (+   181us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:16.766997 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:16.767087 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:18.024880 (+1257793us) raft_consensus.cc:1685] finished
0709 12:11:18.024940 (+    60us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:18.025345 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12892,"prepare.queue_time_us":2876,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"raft.queue_time_us":184545,"raft.run_cpu_time_us":235,"raft.run_wall_time_us":7289,"thread_start_us":1275875,"threads_started":1,"wal-append.queue_time_us":1584583}
I20260709 12:11:18.088548 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:18.089740 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:18.053987 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:18.096122 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1970) took 1142 ms. Trace:
I20260709 12:11:18.096724 12526 rpcz_store.cc:276] 0709 12:11:16.953885 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:18.053457 (+1099572us) service_pool.cc:224] Handling call
0709 12:11:18.096009 (+ 42552us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:18.076990 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:18.089805 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1972) took 1135 ms. Trace:
I20260709 12:11:18.098402 12529 rpcz_store.cc:276] 0709 12:11:16.954466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:18.088037 (+1133571us) service_pool.cc:224] Handling call
0709 12:11:18.089730 (+  1693us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:18.099298 12679 raft_consensus.cc:2410] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:11:18.100118 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:18.100636 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:18.101228 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1971) took 1147 ms. Trace:
I20260709 12:11:18.101708 12529 rpcz_store.cc:276] 0709 12:11:16.954178 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:18.099567 (+1145389us) service_pool.cc:224] Handling call
0709 12:11:18.101156 (+  1589us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:18.103469 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:18.068637 12680 rpcz_store.cc:276] 0709 12:11:15.853650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:15.887560 (+ 33910us) service_pool.cc:224] Handling call
0709 12:11:15.946955 (+ 59395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:15.947199 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:11:17.011993 (+1064794us) log.cc:844] Serialized 14 byte log entry
0709 12:11:18.067027 (+1055034us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:18.067165 (+   138us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:18.067326 (+   161us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:18.067483 (+   157us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:18.067750 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":561838,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"thread_start_us":2022161,"threads_started":2,"wal-append.queue_time_us":959645}
I20260709 12:11:18.079907 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:18.105471 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:17.852883 16105 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.842s	user 0.024s	sys 0.046s
I20260709 12:11:17.803642 16179 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:17.850672 16126 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.103s	user 0.014s	sys 0.000s
W20260709 12:11:17.915786 16350 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
W20260709 12:11:17.858832 16363 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.668s	user 0.001s	sys 0.001s
W20260709 12:11:17.982215 16371 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.206s	user 0.000s	sys 0.002s
W20260709 12:11:17.966490 16374 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
W20260709 12:11:17.967526 16375 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.112s	user 0.000s	sys 0.000s
I20260709 12:11:18.086002 12676 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:11:18.106619 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:18.290912 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 1.08358e+06us
I20260709 12:11:18.081384 12523 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:18.291302 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:18.291724 12523 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1976) took 1259 ms (client timeout 1640 ms). Trace:
W20260709 12:11:18.292188 12523 rpcz_store.cc:269] 0709 12:11:17.032789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:18.061872 (+1029083us) service_pool.cc:224] Handling call
0709 12:11:18.291652 (+229780us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:11:18.131908 12530 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 330) took 3414 ms (client timeout 2159 ms). Trace:
I20260709 12:11:18.292868 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:18.293685 12523 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 8. Current term is 9. Ops: []
W20260709 12:11:18.166082 16378 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.263s	user 0.000s	sys 0.001s
I20260709 12:11:18.294462 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1974) took 1262 ms. Trace:
I20260709 12:11:18.174397 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:18.295545 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1977) took 1178 ms. Trace:
I20260709 12:11:18.295778 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:18.296108 12526 rpcz_store.cc:276] 0709 12:11:17.117339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:18.108120 (+990781us) service_pool.cc:224] Handling call
0709 12:11:18.295457 (+187337us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:11:18.106029 16013 consensus_meta.cc:287] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.709s	user 0.008s	sys 0.016s
I20260709 12:11:18.293849 12676 raft_consensus.cc:2410] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:11:18.112501 16101 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:18.242055 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:18.307389 12680 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 7: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:18.308553 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:18.309510 12680 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:18.242300 16126 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260709 12:11:18.310323 16283 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.745s	user 0.008s	sys 0.011s
W20260709 12:11:18.310585 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 221297us
W20260709 12:11:18.251367 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 202776us
I20260709 12:11:18.310912 16283 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:18.313097 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:18.252983 16284 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.007s	sys 0.015s
W20260709 12:11:18.314417 16284 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.007s	sys 0.016s
I20260709 12:11:17.010386 12600 rpcz_store.cc:276] 0709 12:11:13.375473 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:15.831296 (+2455823us) service_pool.cc:224] Handling call
0709 12:11:15.834237 (+  2941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:15.834553 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:11:15.834866 (+   313us) log.cc:844] Serialized 14 byte log entry
0709 12:11:17.005046 (+1170180us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:17.005108 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:17.005175 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:17.005255 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:17.005421 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":285608,"prepare.run_cpu_time_us":123,"prepare.run_wall_time_us":5873,"raft.queue_time_us":1783,"thread_start_us":1169711,"threads_started":1}
W20260709 12:11:18.259186 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 208409us
I20260709 12:11:18.280138 12522 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:18.318454 16174 consensus_meta.cc:287] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 2.067s	user 0.007s	sys 0.022s
W20260709 12:11:18.283953 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:18.289592 16179 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:18.291141 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1973) took 1336 ms. Trace:
W20260709 12:11:18.293279 12530 rpcz_store.cc:269] 0709 12:11:14.717444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:14.779869 (+ 62425us) service_pool.cc:224] Handling call
0709 12:11:18.131172 (+3351303us) spinlock_profiling.cc:243] Waited 2.38 s on lock 0x7b540027fbd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:18.131575 (+   403us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5468816640}
I20260709 12:11:18.165132 16145 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:18.106366 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1968) took 1183 ms. Trace:
I20260709 12:11:18.335320 12527 rpcz_store.cc:276] 0709 12:11:16.923050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:18.079334 (+1156284us) service_pool.cc:224] Handling call
0709 12:11:18.106253 (+ 26919us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:18.162681 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:18.298585 16013 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:18.122737 16128 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:18.138849 16119 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:18.301221 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1975) took 1268 ms. Trace:
I20260709 12:11:18.347945 12526 rpcz_store.cc:276] 0709 12:11:17.032506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:18.297095 (+1264589us) service_pool.cc:224] Handling call
0709 12:11:18.297838 (+   743us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:18.300671 (+  2833us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:18.300718 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:18.300810 (+    92us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:18.300861 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:18.300950 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:18.301120 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:18.301791 12523 rpcz_store.cc:276] 0709 12:11:17.032182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:18.292942 (+1260760us) service_pool.cc:224] Handling call
0709 12:11:18.293393 (+   451us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:18.294144 (+   751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:18.294360 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:18.323463 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2292) took 1800 ms. Trace:
W20260709 12:11:18.302997 13304 leader_election.cc:336] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.692s (SENT)
I20260709 12:11:18.305579 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1951) took 1213 ms. Trace:
I20260709 12:11:18.318957 16174 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:18.322075 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1966) took 2957 ms. Trace:
I20260709 12:11:18.333421 12529 rpcz_store.cc:276] 0709 12:11:16.954744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:18.102624 (+1147880us) service_pool.cc:224] Handling call
0709 12:11:18.291051 (+188427us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:18.302481 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 316) took 2765 ms. Trace:
I20260709 12:11:18.291215 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:18.357872 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:11:18.357357 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 252122us
I20260709 12:11:18.350385 12672 rpcz_store.cc:276] 0709 12:11:17.092261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:17.135072 (+ 42811us) service_pool.cc:224] Handling call
0709 12:11:18.305069 (+1169997us) spinlock_profiling.cc:243] Waited 25.8 ms on lock 0x7b54002c4554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:18.305343 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59408768}
I20260709 12:11:18.333212 16283 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:18.253320 16267 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.103s
I20260709 12:11:18.357190 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:18.322072 16281 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:18.377200 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 317) took 2318 ms. Trace:
I20260709 12:11:18.381479 16128 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:18.379165 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:18.355062 12524 rpcz_store.cc:276] 0709 12:11:15.364642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:15.364886 (+   244us) service_pool.cc:224] Handling call
0709 12:11:15.365337 (+   451us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:15.365778 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:15.365827 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:15.378020 (+ 12193us) log.cc:844] Serialized 26 byte log entry
0709 12:11:15.788613 (+410593us) spinlock_profiling.cc:243] Waited 1.89 ms on lock 0x7b180043e100. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:11:16.756268 (+967655us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:16.756348 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:16.756472 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:18.321264 (+1564792us) raft_consensus.cc:1685] finished
0709 12:11:18.321302 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:18.321855 (+   553us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69778,"raft.queue_time_us":3478,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":20650,"spinlock_wait_cycles":4355456,"thread_start_us":967311,"threads_started":1,"wal-append.queue_time_us":1678041}
I20260709 12:11:18.391033 16410 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:18.391960 16410 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:18.400104 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2210) took 2553 ms. Trace:
I20260709 12:11:18.400419 12522 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:18.403187 16401 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:18.403991 16401 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:18.406301 16376 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:11:18.413069 12676 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:18.418018 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46944.8us
I20260709 12:11:18.349545 12675 rpcz_store.cc:276] 0709 12:11:16.522601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:16.522806 (+   205us) service_pool.cc:224] Handling call
0709 12:11:16.567995 (+ 45189us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:16.568225 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:11:16.589145 (+ 20920us) log.cc:844] Serialized 14 byte log entry
0709 12:11:18.286862 (+1697717us) log.cc:844] Serialized 14 byte log entry
0709 12:11:18.322665 (+ 35803us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:18.322760 (+    95us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:18.322864 (+   104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:18.323038 (+   174us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:18.323259 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13357,"prepare.queue_time_us":524719,"prepare.run_cpu_time_us":175,"prepare.run_wall_time_us":2555,"raft.queue_time_us":30861,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":9301,"spinlock_wait_cycles":607142656,"thread_start_us":1677813,"threads_started":2,"wal-append.queue_time_us":1390472}
I20260709 12:11:18.400699 12601 rpcz_store.cc:276] 0709 12:11:15.846786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:16.385634 (+538848us) service_pool.cc:224] Handling call
0709 12:11:16.385994 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:16.415686 (+ 29692us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:11:16.495965 (+ 80279us) log.cc:844] Serialized 14 byte log entry
0709 12:11:18.347001 (+1851036us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:18.353738 (+  6737us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:18.399488 (+ 45750us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:18.399646 (+   158us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:18.399879 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":470613,"prepare.queue_time_us":113152,"prepare.run_cpu_time_us":224,"prepare.run_wall_time_us":8735,"spinlock_wait_cycles":673368704,"thread_start_us":1325975,"threads_started":2,"wal-append.queue_time_us":1476825}
W20260709 12:11:18.425683 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 109242us
I20260709 12:11:18.418635 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:18.377785 12603 rpcz_store.cc:276] 0709 12:11:16.059115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:16.392698 (+333583us) service_pool.cc:224] Handling call
0709 12:11:16.393745 (+  1047us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:16.393976 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:11:16.394283 (+   307us) log.cc:844] Serialized 14 byte log entry
0709 12:11:18.376351 (+1982068us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:18.376518 (+   167us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:11:18.376648 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:18.376789 (+   141us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:18.377000 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":661633,"prepare.queue_time_us":104618,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"thread_start_us":1295949,"threads_started":2,"wal-append.queue_time_us":1499775}
I20260709 12:11:18.424136 16119 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 13 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:11:18.437290 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 154251us
I20260709 12:11:18.438709 16119 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:18.441383 16198 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.935s	user 0.013s	sys 0.008s
I20260709 12:11:18.441926 12646 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:18.445217 16223 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:18.446025 16223 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:18.447371 16373 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:18.448473 16373 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:18.453204 12528 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:18.449265 12679 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:11:18.455541 12679 pending_rounds.cc:85] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 8
I20260709 12:11:18.453286 16324 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:18.458989 16324 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:18.455754 12679 pending_rounds.cc:107] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 9.9
W20260709 12:11:18.461335 12671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2273) took 2839 ms (client timeout 1654 ms). Trace:
I20260709 12:11:18.447412 16223 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:18.461942 12671 rpcz_store.cc:269] 0709 12:11:15.621584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:15.621777 (+   193us) service_pool.cc:224] Handling call
0709 12:11:18.460896 (+2839119us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b54003d4ad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:18.461114 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3006489984}
I20260709 12:11:18.461301 12679 raft_consensus.cc:2889] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:18.451082 13337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 381) took 1377 ms. Trace:
I20260709 12:11:18.356906 12604 rpcz_store.cc:276] 0709 12:11:15.536497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:16.227381 (+690884us) service_pool.cc:224] Handling call
0709 12:11:16.251310 (+ 23929us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:16.251582 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:11:16.316247 (+ 64665us) log.cc:844] Serialized 14 byte log entry
0709 12:11:17.167074 (+850827us) log.cc:844] Serialized 14 byte log entry
0709 12:11:18.140660 (+973586us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b18003bd280. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:11:18.223514 (+ 82854us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:18.243669 (+ 20155us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:18.269642 (+ 25973us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:18.281039 (+ 11397us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:18.302258 (+ 21219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7509,"prepare.queue_time_us":529844,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":97,"raft.queue_time_us":85,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":1920,"spinlock_wait_cycles":26477312,"thread_start_us":850144,"threads_started":1,"wal-append.queue_time_us":1469116}
I20260709 12:11:18.482347 12680 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
W20260709 12:11:18.453084 16182 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.658s	user 0.011s	sys 0.012s
I20260709 12:11:18.504877 12524 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:18.506021 12529 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:11:18.450500 16013 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:18.513019 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:18.515290 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:18.448837 16373 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:18.451222 16281 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:18.483610 16128 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:18.468428 13337 rpcz_store.cc:276] 0709 12:11:17.073683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:17.073799 (+   116us) service_pool.cc:224] Handling call
0709 12:11:17.074072 (+   273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:17.074291 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:11:18.088172 (+1013881us) log.cc:844] Serialized 14 byte log entry
0709 12:11:18.367639 (+279467us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:18.386575 (+ 18936us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:18.450594 (+ 64019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:18.450728 (+   134us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:18.450954 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3706,"prepare.queue_time_us":882956,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":1257,"raft.queue_time_us":31552,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":36353,"thread_start_us":1190879,"threads_started":2}
I20260709 12:11:18.541321 12672 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:18.543076 12675 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260709 12:11:18.585669 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:18.586578 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:18.615001 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 107667us
W20260709 12:11:18.617770 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 86415.3us
I20260709 12:11:18.622633 12680 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:11:18.622978 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:18.623950 12530 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:11:18.625697 16324 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:18.626101 12522 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 1978) took 1474 ms (client timeout 1726 ms). Trace:
W20260709 12:11:18.626614 12522 rpcz_store.cc:269] 0709 12:11:17.151622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:18.089342 (+937720us) service_pool.cc:224] Handling call
0709 12:11:18.625911 (+536569us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54003ca5d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:18.626039 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":340170624}
I20260709 12:11:18.617799 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:18.629124 16373 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:18.478333 16198 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:18.632030 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:18.540656 16401 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:18.635908 16152 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:18.636818 16152 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:18.639230 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:18.638657 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:18.638736 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:18.644577 12672 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 7: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:18.645565 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:18.648129 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 595216us
I20260709 12:11:18.650555 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:11:18.651209 16211 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.673s	user 0.015s	sys 0.007s
I20260709 12:11:18.651520 12679 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 10 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:11:18.651760 16211 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260709 12:11:18.524276 16182 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:18.653262 16373 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:18.654981 16182 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:18.656358 13304 leader_election.cc:336] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.164s (SENT)
I20260709 12:11:18.656896 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:18.657219 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:18.658233 16401 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:18.658741 16220 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:18.628433 16174 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:18.662276 12530 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:18.663242 12522 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: []
I20260709 12:11:18.622000 16334 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:18.694761 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:18.659296 16220 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:18.695722 12673 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
W20260709 12:11:18.703034 13302 leader_election.cc:336] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.660s (SENT)
W20260709 12:11:18.704648 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 48121.5us
I20260709 12:11:18.723237 16152 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:18.732990 12604 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:18.700606 16220 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:18.744711 16387 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.764s	user 0.004s	sys 0.004s
W20260709 12:11:18.748560 16398 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.758s	user 0.000s	sys 0.011s
I20260709 12:11:18.750476 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1945) took 1707 ms. Trace:
I20260709 12:11:18.754618 16105 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:18.750937 12678 rpcz_store.cc:276] 0709 12:11:17.043188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:17.067965 (+ 24777us) service_pool.cc:224] Handling call
0709 12:11:17.068246 (+   281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:17.072964 (+  4718us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:17.072994 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:17.090869 (+ 17875us) log.cc:844] Serialized 26 byte log entry
0709 12:11:18.071164 (+980295us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:18.071246 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:18.071371 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:18.747596 (+676225us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:18.750146 (+  2550us) raft_consensus.cc:1685] finished
0709 12:11:18.750164 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:18.750390 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18660,"raft.queue_time_us":4839,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":7020,"spinlock_wait_cycles":596661504,"thread_start_us":920941,"threads_started":1,"wal-append.queue_time_us":867051}
W20260709 12:11:18.815387 16379 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.980s	user 0.000s	sys 0.009s
W20260709 12:11:18.820050 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:18.820715 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:18.858104 16347 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.179s	user 0.014s	sys 0.007s
I20260709 12:11:18.858716 16347 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:18.841269 16152 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:18.862650 13330 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:18.862704 16152 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:18.865479 16152 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:18.913462 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:18.919670 13330 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:18.920503 13332 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:18.920975 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:18.911504 16220 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:18.921877 12604 raft_consensus.cc:2368] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9.
W20260709 12:11:18.919697 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 213596us
I20260709 12:11:18.923465 12676 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
W20260709 12:11:18.927831 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 282678us
W20260709 12:11:18.925005 12677 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.015s	sys 0.021s
W20260709 12:11:18.928515 12677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.016s	sys 0.021s
W20260709 12:11:18.929487 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 68080.6us
I20260709 12:11:18.926033 16198 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:18.956457 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:18.945581 16152 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:18.962642 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:18.965695 12643 leader_election.cc:400] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7.
I20260709 12:11:18.966086 12643 leader_election.cc:403] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:18.968919 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:18.969913 12600 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 7: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:18.952183 16347 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:18.975274 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:18.976464 12602 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:18.976656 13302 leader_election.cc:336] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.841s (SENT)
I20260709 12:11:18.975679 16220 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:18.984079 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:18.986801 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:18.988350 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:18.986150 16223 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7.
I20260709 12:11:19.021103 12606 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: []
W20260709 12:11:18.991572 16380 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.014s	sys 0.024s
I20260709 12:11:19.032286 16347 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:19.033380 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.034462 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:19.036124 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:19.036248 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:18.987965 16334 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:11:19.042068 12599 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:19.044275 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:19.044464 12494 connection.cc:256] RPC call timeout handler was delayed by -1.7325221470000542s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1562s for a response
W20260709 12:11:19.044929 12494 connection.cc:256] RPC call timeout handler was delayed by -1.6902221469999859s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16090000000000002s for a response
W20260709 12:11:19.035521 16380 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.015s	sys 0.024s
I20260709 12:11:19.047803 12530 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:19.023429 16152 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:19.051964 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:19.050485 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 108027us
I20260709 12:11:19.054620 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:19.052459 12527 raft_consensus.cc:2376] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9. Re-sending same reply.
I20260709 12:11:19.043793 16119 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:19.062307 16369 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 3, Time since last communication: 0.010s
I20260709 12:11:19.053344 16220 raft_consensus.cc:3055] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:19.063190 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:19.063970 12605 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:19.066231 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:19.119424 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 94554.7us
I20260709 12:11:19.066780 16119 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:19.066370 16220 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:19.071177 12530 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:11:19.176615 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.180070 12602 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:19.177964 12643 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:19.184849 12604 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:19.184862 12606 raft_consensus.cc:2410] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 9 because replica has last-logged OpId of term: 7 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 2.
I20260709 12:11:19.125754 16333 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:19.183473 12494 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:19.181561 16119 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:19.190629 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 109838us
I20260709 12:11:19.198645 16240 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:11:19.203279 16102 raft_consensus.cc:3055] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Stepping down as leader of term 5
I20260709 12:11:19.203802 16102 raft_consensus.cc:740] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:19.217499 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.217880 12646 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:19.205083 16102 consensus_queue.cc:260] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:19.184495 16169 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
W20260709 12:11:19.230072 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 178092us
I20260709 12:11:19.226547 16333 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:19.233953 16223 raft_consensus.cc:3055] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:19.233285 16145 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:11:19.248121 16390 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.263s	user 0.006s	sys 0.010s
I20260709 12:11:19.249522 16114 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:19.250485 16369 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:19.254060 12494 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:19.255213 16145 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:11:19.264704 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 197170us
I20260709 12:11:19.253216 16102 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:19.265471 16334 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:19.265618 16145 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:19.267227 16314 raft_consensus.cc:3055] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:19.267799 16314 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:19.268283 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 1967) took 2394 ms. Trace:
I20260709 12:11:19.269340 16152 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:19.268936 16314 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:19.269909 16114 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:19.270442 16410 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:19.271591 16432 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 12:11:19.257839 16308 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:11:19.273181 16385 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 1.298s	user 0.006s	sys 0.013s
I20260709 12:11:19.252737 16223 raft_consensus.cc:740] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:19.268694 12528 rpcz_store.cc:276] 0709 12:11:16.874088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:16.952533 (+ 78445us) service_pool.cc:224] Handling call
0709 12:11:17.003915 (+ 51382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:18.460406 (+1456491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:19.267931 (+807525us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b54003cbed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:19.268206 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3179487104}
I20260709 12:11:19.261204 16169 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:11:19.298691 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52539.4us
I20260709 12:11:19.298051 16169 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:19.300551 16169 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:19.300702 16314 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:19.275954 16333 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:19.306967 16333 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:19.309751 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 856399us
W20260709 12:11:19.311802 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 66356.3us
I20260709 12:11:19.324033 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:11:19.325309 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 674571us
I20260709 12:11:19.326198 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:19.327924 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:19.328718 12601 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:19.331070 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2208) took 3701 ms. Trace:
I20260709 12:11:19.332155 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:19.341480 12427 catalog_manager.cc:5697] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:11:19.343168 12427 catalog_manager.cc:5697] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 6 to 8, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 8 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:11:19.267218 16334 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:19.344681 12427 catalog_manager.cc:5697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:19.342060 16114 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:19.320361 16369 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:19.348430 16369 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:19.349457 16281 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:11:19.349972 16220 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:19.350713 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:19.345481 16126 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:19.317610 16410 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:19.351418 12526 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:19.336997 16182 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:19.362015 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:19.370669 16435 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.179s	user 0.000s	sys 0.001s
I20260709 12:11:19.394302 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:19.341457 16333 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:19.371469 16436 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:11:19.374948 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:19.375747 12427 catalog_manager.cc:5697] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 4 to 8, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 8 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:11:19.377331 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:19.379210 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 63600.6us
I20260709 12:11:19.351203 16220 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:19.370497 16223 consensus_queue.cc:260] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:19.357089 16369 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:19.380286 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:19.409715 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 69888.9us
I20260709 12:11:19.355098 16410 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:19.393826 12605 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:11:19.331501 12597 rpcz_store.cc:276] 0709 12:11:15.617572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:16.318811 (+701239us) service_pool.cc:224] Handling call
0709 12:11:16.353327 (+ 34516us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:16.353653 (+   326us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:11:18.035662 (+1682009us) log.cc:844] Serialized 14 byte log entry
0709 12:11:19.275331 (+1239669us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:19.275473 (+   142us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:19.296121 (+ 20648us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:19.314492 (+ 18371us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:19.318995 (+  4503us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1001475,"prepare.queue_time_us":35779,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":994138,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":122,"thread_start_us":1621448,"threads_started":2,"wal-append.queue_time_us":335463}
I20260709 12:11:19.397800 12601 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:19.363479 16126 consensus_queue.cc:1243] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:11:19.428244 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:19.427403 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:19.431465 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:19.432796 12526 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:19.435212 16223 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:11:19.436507 16394 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.455s	user 0.000s	sys 0.014s
I20260709 12:11:19.437093 12643 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:11:19.438377 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 77889.3us
I20260709 12:11:19.439639 12496 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:19.440363 16334 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:19.443060 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:19.443780 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:19.424042 16410 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:19.457268 12523 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:19.443614 16416 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:19.471000 12523 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:19.475000 12646 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:19.452899 16333 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:19.482260 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 345) took 1015 ms. Trace:
I20260709 12:11:19.482770 12529 rpcz_store.cc:276] 0709 12:11:18.462644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:18.641924 (+179280us) service_pool.cc:224] Handling call
0709 12:11:18.663055 (+ 21131us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:18.687647 (+ 24592us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:18.687693 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:18.702468 (+ 14775us) log.cc:844] Serialized 26 byte log entry
0709 12:11:18.943111 (+240643us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:18.943146 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:18.943205 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:19.477526 (+534321us) raft_consensus.cc:1685] finished
0709 12:11:19.477546 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:19.477687 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":195702,"threads_started":1,"wal-append.queue_time_us":324548}
I20260709 12:11:19.481232 16169 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:19.477053 16367 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:19.487779 16367 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:19.489965 12569 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:19.471901 16369 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:19.478977 16182 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:19.494133 16070 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260709 12:11:19.493886 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:19.510627 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.520483 12603 raft_consensus.cc:2410] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:11:19.494426 16369 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:19.522939 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:19.523692 12606 raft_consensus.cc:2368] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8.
I20260709 12:11:19.521163 16416 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:19.525686 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:19.526899 16416 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:19.527184 12600 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:11:19.529033 16182 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:19.490106 16367 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:19.530301 16241 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:19.531071 16241 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:19.532866 16367 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:11:19.533551 16241 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:19.535494 12525 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 5.
I20260709 12:11:19.536547 16240 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:19.536988 16240 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:11:19.537173 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 67882.5us
I20260709 12:11:19.537731 16240 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:19.525417 16145 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:19.538509 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:19.495854 16389 consensus_queue.cc:1059] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
W20260709 12:11:19.539387 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 45634.8us
I20260709 12:11:19.540257 16389 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:19.540858 12571 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:19.542203 16421 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:11:19.542379 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:19.498167 16362 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:19.543278 16362 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:19.543964 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:19.544222 16362 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:19.544806 16421 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:19.529717 16119 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:19.538530 16334 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:19.494515 16198 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.052s
I20260709 12:11:19.546919 16334 raft_consensus.cc:697] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:19.547022 13332 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:19.547910 16334 consensus_queue.cc:237] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.3, Last appended by leader: 3, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:19.549633 16119 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:19.550758 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:19.551525 12603 raft_consensus.cc:2368] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 8. Current term is 9.
I20260709 12:11:19.552431 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.553333 12603 raft_consensus.cc:2410] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 10 because replica has last-logged OpId of term: 9 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260709 12:11:19.546299 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:19.554648 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.541023 16389 raft_consensus.cc:3055] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:19.561271 12605 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:19.563273 12597 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:19.568130 12604 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:11:19.572798 12526 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:19.574834 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.576234 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.576586 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 1960) took 3119 ms. Trace:
I20260709 12:11:19.577443 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.491770 16283 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:19.571671 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:19.587956 12603 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:19.590880 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.544819 16145 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:19.576982 12531 rpcz_store.cc:276] 0709 12:11:16.457524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:16.461047 (+  3523us) service_pool.cc:224] Handling call
0709 12:11:16.515966 (+ 54919us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:18.941802 (+2425836us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:18.944136 (+  2334us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:18.944167 (+    31us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:18.946475 (+  2308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:19.576156 (+629681us) spinlock_profiling.cc:243] Waited 2.26 s on lock 0x7b54003caad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:19.576200 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:19.576423 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1801,"raft.queue_time_us":29992,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":2603,"spinlock_wait_cycles":5190464768}
W20260709 12:11:19.572778 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 133237us
I20260709 12:11:19.593518 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:19.600862 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:19.601828 12606 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 5. Current term is 6. Ops: []
W20260709 12:11:19.597105 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 169824us
W20260709 12:11:19.603731 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:19.605950 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:19.562496 16389 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:19.579357 16126 raft_consensus.cc:2764] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260709 12:11:19.609395 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:19.610713 12605 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:19.610256 16114 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:19.614353 16367 consensus_queue.cc:1059] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:19.621246 16220 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
I20260709 12:11:19.605379 16333 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:19.623741 16220 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:19.624239 16220 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:19.624341 16333 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:19.625192 16220 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:19.623770 16367 raft_consensus.cc:3055] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:19.626546 16367 raft_consensus.cc:740] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:19.627190 16389 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:19.627578 16367 consensus_queue.cc:260] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:19.628629 16419 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:19.628773 16389 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:19.629163 16419 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:19.618521 16102 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:19.615146 13304 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:19.622103 16114 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:19.630942 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.626237 16447 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.007s
I20260709 12:11:19.629227 16367 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:19.630484 16102 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:19.637605 12605 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:11:19.615772 16119 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:11:19.620008 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.579855 16416 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:19.634910 16447 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:19.635444 16438 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:19.636276 16152 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:19.636819 16373 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:19.638651 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:19.643031 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 96884.8us
I20260709 12:11:19.638288 16445 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:19.645742 16152 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:19.635644 16419 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:19.647131 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:19.645926 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.652294 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:19.660673 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.640373 16102 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:19.661630 12604 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:19.662066 16102 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:19.662711 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.662616 16102 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:19.664285 12604 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:11:19.664806 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.641201 16447 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:19.666467 16447 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:19.667963 16447 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:19.663928 16438 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:19.670454 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.674491 16438 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:19.655781 16373 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:19.675740 12525 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 335) took 2803 ms (client timeout 1583 ms). Trace:
I20260709 12:11:19.676867 16333 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:19.691783 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 44783.1us
I20260709 12:11:19.692842 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.677022 16152 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:19.677594 12496 leader_election.cc:400] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9.
I20260709 12:11:19.695834 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:19.696128 12496 leader_election.cc:403] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:19.693564 12600 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
W20260709 12:11:19.683305 12525 rpcz_store.cc:269] 0709 12:11:16.871956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:16.872846 (+   890us) service_pool.cc:224] Handling call
0709 12:11:19.675459 (+2802613us) spinlock_profiling.cc:243] Waited 2.66 s on lock 0x7b54003f6954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:19.675645 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6108711296}
I20260709 12:11:19.694998 13304 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:19.659265 16445 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:19.696063 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.671164 12605 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:19.697046 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:19.702073 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:19.707811 12603 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:19.711054 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:19.712968 12599 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:19.696596 16220 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:19.717062 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:19.720156 12605 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 9
I20260709 12:11:19.695494 16373 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:19.721570 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:19.719856 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:19.722970 16281 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:19.725518 16223 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:19.724921 13337 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:19.713521 16419 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:19.717676 16416 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:11:19.733850 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 56403.6us
I20260709 12:11:19.735850 12601 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 8. Current term is 9. Ops: [8.5-8.5]
I20260709 12:11:19.726430 16223 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:19.742877 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47147.6us
I20260709 12:11:19.742509 16448 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9.
I20260709 12:11:19.764258 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:19.765565 16169 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:19.747948 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 53029.5us
I20260709 12:11:19.766659 13304 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:19.768344 16281 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:19.768765 16281 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:19.769172 16281 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:19.751698 16102 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:19.745395 16369 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:19.758334 12427 catalog_manager.cc:5697] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 9 to 10, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 10 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:11:19.762651 16240 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:19.777990 16182 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:19.779551 16182 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:19.779973 16182 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:19.779567 16114 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:19.763280 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:19.739578 16416 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:19.765966 12571 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:19.772426 12605 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
W20260709 12:11:19.786226 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:19.789122 16452 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:11:19.790876 12601 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:19.793294 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:19.798334 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 56574.3us
I20260709 12:11:19.769876 16410 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:19.781032 16182 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:19.787734 16416 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:19.805431 12605 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
W20260709 12:11:19.815399 16397 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.830s	user 0.008s	sys 0.004s
I20260709 12:11:19.818449 16423 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:19.819056 16423 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:19.819433 16423 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:19.823164 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:19.798913 16198 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:19.827908 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:19.839080 12603 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
W20260709 12:11:19.848057 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:19.852145 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:19.828084 16453 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:11:19.856146 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:19.858572 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:19.859676 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:19.860325 12600 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:19.862808 12606 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:19.866909 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.868113 12604 raft_consensus.cc:2410] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:11:19.869174 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.869990 12604 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:19.872617 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.880553 16410 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:19.882874 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2207) took 4308 ms. Trace:
I20260709 12:11:19.887300 12602 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:32832
I20260709 12:11:19.883443 12598 rpcz_store.cc:276] 0709 12:11:15.575060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:16.335295 (+760235us) service_pool.cc:224] Handling call
0709 12:11:16.335833 (+   538us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:11:16.336443 (+   610us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:11:17.072537 (+736094us) log.cc:844] Serialized 14 byte log entry
0709 12:11:18.059396 (+986859us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:11:18.061108 (+  1712us) log.cc:844] Serialized 78 byte log entry
0709 12:11:18.061381 (+   273us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:18.061968 (+   587us) log.cc:844] Serialized 14 byte log entry
0709 12:11:18.519350 (+457382us) log.cc:844] Serialized 14 byte log entry
0709 12:11:19.316222 (+796872us) log.cc:844] Serialized 14 byte log entry
0709 12:11:19.405396 (+ 89174us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:19.425216 (+ 19820us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:19.793086 (+367870us) spinlock_profiling.cc:243] Waited 283 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:19.882532 (+ 89446us) raft_consensus.cc:1685] finished
0709 12:11:19.882563 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:19.882781 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1203910,"prepare.queue_time_us":162880,"prepare.run_cpu_time_us":368,"prepare.run_wall_time_us":7795,"raft.queue_time_us":2113436,"raft.run_cpu_time_us":723,"raft.run_wall_time_us":21049,"spinlock_wait_cycles":651571072,"thread_start_us":1814102,"threads_started":3,"wal-append.queue_time_us":876051}
I20260709 12:11:19.895143 16240 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:19.896855 16223 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:19.902604 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 136720us
I20260709 12:11:19.903761 12571 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:19.905597 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:19.906528 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:19.905757 12427 catalog_manager.cc:5697] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 6 to 7, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 7 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:19.907905 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.877768 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.914366 12599 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:19.910848 12427 catalog_manager.cc:5697] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 5 to 7, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 7 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:11:19.917244 12602 raft_consensus.cc:606] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:19.917644 12602 raft_consensus.cc:614] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:11:19.934286 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 77722.5us
I20260709 12:11:19.936182 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.931795 12496 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:19.942750 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.942946 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.943632 12603 raft_consensus.cc:2368] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7.
I20260709 12:11:19.944411 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:19.910686 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 100308us
I20260709 12:11:19.909381 16182 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:19.915452 16114 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:19.955045 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:19.959609 12572 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:19.961813 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:19.962852 16421 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:19.962983 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.963781 12604 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:19.965610 12572 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:19.966567 16421 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:19.967092 16421 raft_consensus.cc:697] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:19.968081 16421 consensus_queue.cc:237] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:19.972216 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:19.962263 12601 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:19.975543 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:19.946703 16198 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:19.986567 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:19.985558 12598 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:19.990697 12606 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:11:19.990993 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:19.994320 16070 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:19.988292 16126 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:19.997313 16070 raft_consensus.cc:697] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:19.998718 12494 leader_election.cc:400] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9.
I20260709 12:11:19.999082 12494 leader_election.cc:403] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:11:19.998476 16070 consensus_queue.cc:237] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:19.993340 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:19.952277 16114 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:20.006743 16114 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:20.000164 12525 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:11:20.010438 12643 leader_election.cc:336] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.838s (SENT)
I20260709 12:11:20.010970 12643 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:20.014782 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 109279us
I20260709 12:11:20.019093 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:20.031567 16423 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:20.034226 16334 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260709 12:11:20.034262 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 178844us
I20260709 12:11:20.037961 12571 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:20.041153 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:11:20.045709 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 47178us
W20260709 12:11:20.047005 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:20.050840 12643 leader_election.cc:336] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.858s (SENT)
I20260709 12:11:20.054819 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:20.035336 16126 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9.
I20260709 12:11:20.065539 16427 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:20.070235 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:20.071398 12522 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:20.050097 16198 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:20.083130 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:20.093350 12675 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:20.094753 12677 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:20.095253 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:20.095992 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:20.097441 16281 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:20.098515 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:20.098615 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:20.111019 12522 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:20.099510 12530 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:20.100162 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:20.100687 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:20.102720 12427 catalog_manager.cc:5697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:20.113880 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:20.114516 16460 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:20.114912 16460 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:20.111060 12671 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:11:20.116236 12531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:20.116801 12531 raft_consensus.cc:606] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:20.117101 12531 raft_consensus.cc:614] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:20.115330 16460 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:20.120961 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:20.121419 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:20.121904 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:20.122148 12672 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:11:20.113541 12427 catalog_manager.cc:5697] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 10 to 13, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 13 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:20.125104 12427 catalog_manager.cc:5697] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 7 to 10, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 10 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:20.123334 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:20.127164 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:20.127445 12523 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:20.133744 12672 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:20.134316 12523 raft_consensus.cc:606] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:20.134835 12523 raft_consensus.cc:614] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:20.129750 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:20.130904 12676 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:20.139144 12674 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:11:20.140188 12600 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:20.142755 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:20.146981 12427 catalog_manager.cc:5697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:20.148474 16367 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:20.149343 16367 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:20.151865 16367 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:20.164340 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 109601us
I20260709 12:11:20.164415 12528 raft_consensus.cc:2368] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7.
I20260709 12:11:20.166684 13330 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: []
W20260709 12:11:20.171917 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 121135us
W20260709 12:11:20.172736 12643 leader_election.cc:336] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.863s (SENT)
I20260709 12:11:20.180260 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:20.191576 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:20.142238 16283 raft_consensus.cc:2764] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260709 12:11:20.198853 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:20.207309 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:20.212047 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:20.214350 12525 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
W20260709 12:11:20.215727 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50582us
I20260709 12:11:20.214977 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:20.221055 12494 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:20.228374 12494 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:20.230578 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:11:20.236069 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:20.214332 16334 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:20.237589 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:20.233613 16281 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:20.245721 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:20.221807 16145 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:20.217247 16423 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:20.234079 16126 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:20.253760 12679 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:20.191066 16070 consensus_queue.cc:1059] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:11:20.244997 12522 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
W20260709 12:11:20.243083 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 202006us
I20260709 12:11:20.258005 12524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b825dff40dc4290a25d71534b598330"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:20.258709 12524 raft_consensus.cc:606] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:11:20.258850 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:20.259032 12524 raft_consensus.cc:614] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:20.265355 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:20.272528 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 234647us
I20260709 12:11:20.277912 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:20.279146 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:20.275789 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:20.280105 12606 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:20.266855 16145 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:20.281731 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:20.282557 12605 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:20.279958 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:20.239027 16416 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:20.293416 12600 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:20.263608 16126 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:20.307516 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:20.260598 16462 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:20.264008 16457 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:20.323714 12522 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:20.313597 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:20.314568 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:20.319058 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:20.319937 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:20.326444 12606 raft_consensus.cc:2376] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9. Re-sending same reply.
I20260709 12:11:20.327343 12571 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:20.328985 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:20.260159 16240 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:11:20.336048 12604 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 8: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:20.334676 12494 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:20.343894 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 68324.6us
I20260709 12:11:20.345175 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:20.337729 16462 raft_consensus.cc:697] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:20.337376 16410 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:20.348475 16410 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:20.263844 16334 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:20.353533 12672 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 11: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:20.357479 12531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:20.358528 12531 raft_consensus.cc:606] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:20.359047 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:20.344547 16070 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:20.364820 16410 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:20.401785 16471 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:11:20.403988 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:20.404678 12676 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:20.353760 16389 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.724s	user 0.017s	sys 0.000s
W20260709 12:11:20.385754 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:20.282914 16145 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:20.411809 12601 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
W20260709 12:11:20.397270 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 102918us
I20260709 12:11:20.379570 12646 leader_election.cc:400] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7.
I20260709 12:11:20.438627 12646 leader_election.cc:403] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:20.373018 16419 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:11:20.346964 16114 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:20.462765 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:20.464330 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:20.467139 16421 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:20.467790 16421 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:20.471705 16114 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:11:20.457281 16240 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:20.473100 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:20.473443 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:20.473824 12600 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:20.474184 12602 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:20.383522 16070 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:20.475268 16070 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:20.477205 16152 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 7
W20260709 12:11:20.478385 12643 leader_election.cc:336] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.064s (SENT)
I20260709 12:11:20.478863 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:20.476779 16421 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:20.482498 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 111705us
I20260709 12:11:20.484897 12677 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
W20260709 12:11:20.486251 16480 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.142s	user 0.001s	sys 0.000s
W20260709 12:11:20.487406 16476 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 12:11:20.493160 16152 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7.
W20260709 12:11:20.505290 16477 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
W20260709 12:11:20.531945 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 174017us
W20260709 12:11:20.517979 16182 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.565s	user 0.009s	sys 0.000s
I20260709 12:11:20.348752 16462 consensus_queue.cc:237] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:20.373723 16457 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.155s
I20260709 12:11:20.544636 12571 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:20.455929 16283 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.086s
I20260709 12:11:20.534281 16152 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:11:20.569635 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 224564us
I20260709 12:11:20.576788 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:20.538237 16419 raft_consensus.cc:697] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:20.588969 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:20.630424 13335 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:11:20.631572 12673 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
W20260709 12:11:20.642387 16474 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.358s	user 0.001s	sys 0.003s
I20260709 12:11:20.544137 16223 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.060s
I20260709 12:11:20.592336 16182 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:20.652190 13335 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:20.537642 16126 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:20.660959 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:20.645056 16457 raft_consensus.cc:3055] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:11:20.644345 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 158037us
I20260709 12:11:20.670383 12676 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 10.
W20260709 12:11:20.674171 12671 consensus_meta.cc:287] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.540s	user 0.008s	sys 0.002s
I20260709 12:11:20.674642 12671 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
W20260709 12:11:20.685307 13302 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.583s (SENT)
I20260709 12:11:20.685946 13302 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:20.699411 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 169466us
I20260709 12:11:20.666900 12646 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:20.670467 16460 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.552s	user 0.012s	sys 0.000s
I20260709 12:11:20.655185 16485 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:20.661748 16416 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:20.663686 16457 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:11:20.711810 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 233473us
I20260709 12:11:20.715396 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:20.704445 16119 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:11:20.709273 16485 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:20.713820 16334 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:20.788347 16410 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:20.746979 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:20.725083 16421 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:11:20.709432 16460 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:20.732420 16485 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:20.758174 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:20.810496 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:20.703945 16283 consensus_queue.cc:1059] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.022s
I20260709 12:11:20.762549 16223 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:20.719280 16419 consensus_queue.cc:237] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:20.802848 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 226165us
I20260709 12:11:20.814427 12524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
W20260709 12:11:20.815122 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 222186us
I20260709 12:11:20.824501 12526 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:20.824563 12524 raft_consensus.cc:606] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:20.825736 12524 raft_consensus.cc:614] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:20.826857 12494 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:20.838053 12525 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:20.813464 16416 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:20.853619 12496 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:20.855896 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:20.857908 16070 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:20.851043 16446 raft_consensus.cc:3055] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:20.863790 16495 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:20.864447 16495 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:20.866776 16460 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:20.867064 16496 raft_consensus.cc:993] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: : Instructing follower 193a40da6e364ead8ab561e14c3cc8d1 to start an election
I20260709 12:11:20.867669 16496 raft_consensus.cc:1081] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Signalling peer 193a40da6e364ead8ab561e14c3cc8d1 to start an election
W20260709 12:11:20.868081 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 121289us
I20260709 12:11:20.857157 12531 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:11:20.868738 12605 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:20.877247 16362 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:20.877709 12671 pending_rounds.cc:85] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 4
I20260709 12:11:20.877995 16362 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:20.878222 12671 pending_rounds.cc:107] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260709 12:11:20.878628 16070 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:20.878710 12671 raft_consensus.cc:2889] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:20.879094 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:20.854983 16416 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:20.882732 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:20.887542 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:20.889694 12601 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:20.865483 16495 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:20.870012 16446 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:20.878633 16362 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:20.902596 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:20.904592 13335 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 9: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:20.902943 12602 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:20.907308 12597 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:20.908660 12600 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 4. (term mismatch)
I20260709 12:11:20.909109 12600 pending_rounds.cc:85] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 3
I20260709 12:11:20.909359 12600 pending_rounds.cc:107] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 5.4
I20260709 12:11:20.909825 12600 raft_consensus.cc:2889] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:20.909322 12494 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:20.896519 16347 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:20.917397 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:11:20.919656 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:20.924067 12571 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:20.927217 12572 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:20.894199 16416 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:20.896603 16472 raft_consensus.cc:3055] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:20.942538 16472 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:20.943889 16472 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:20.945396 16472 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:20.947077 16421 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260709 12:11:20.947672 16421 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:20.948143 16421 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 5 FOLLOWER]: Advancing to term 6
W20260709 12:11:20.950304 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:20.959556 13330 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:20.963856 16347 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:20.964730 16491 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 12:11:20.968827 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 301992us
W20260709 12:11:20.970499 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 61279.4us
I20260709 12:11:20.971436 16441 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:20.971597 12494 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:11:20.976284 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:20.976956 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:20.981077 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:11:20.987462 16489 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 12:11:20.989542 16446 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:20.990752 16446 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:20.991492 16421 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:20.972837 16497 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.052s	user 0.001s	sys 0.001s
W20260709 12:11:21.001359 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 182283us
I20260709 12:11:20.978340 16459 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:11:21.006716 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 120307us
I20260709 12:11:20.929682 16492 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
I20260709 12:11:21.017973 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:21.018049 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:21.044889 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:21.053155 12603 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:21.047219 16126 consensus_queue.cc:1059] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260709 12:11:21.060402 16126 raft_consensus.cc:3055] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:21.060913 16126 raft_consensus.cc:740] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:21.052265 16347 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:21.048283 16459 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260709 12:11:21.066531 16126 consensus_queue.cc:260] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:21.068634 16126 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:21.047379 16487 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:11:21.070086 16459 raft_consensus.cc:3055] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:21.070731 16459 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:11:21.001330 16460 consensus_queue.cc:1175] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e56d487365394ac5ac5f1002926a5645" responder_term: 10 status { last_received { term: 10 index: 8 } last_committed_idx: 8 last_received_current_leader { term: 10 index: 8 } }
I20260709 12:11:21.024626 16410 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:21.016489 16223 raft_consensus.cc:697] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:21.083134 12571 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:21.072072 16459 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:21.120323 16182 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:21.121398 16182 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:21.123845 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 86039.4us
I20260709 12:11:21.124799 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:21.126012 16485 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:21.126837 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:21.128809 16182 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:21.129652 16485 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:21.039907 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:21.158871 12603 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:21.047706 16494 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.096s
I20260709 12:11:21.047062 16441 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.092s
I20260709 12:11:21.109846 16283 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:11:21.138808 16502 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:21.176858 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:21.163403 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:21.200824 16421 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:21.202291 16421 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:21.202948 16421 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:21.205225 16421 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:21.207376 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:21.208168 13330 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:21.214497 16145 raft_consensus.cc:3055] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:21.215099 16145 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:21.216166 16145 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:21.217624 16145 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:21.218744 16223 consensus_queue.cc:237] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:21.219460 16416 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:21.139205 16362 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:21.224284 16362 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:21.225215 12675 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:11:21.226624 12676 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:51308
I20260709 12:11:21.226585 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:21.227576 12676 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:21.228024 12676 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:21.229803 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 68602.8us
I20260709 12:11:21.230360 16240 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:21.230993 12680 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
W20260709 12:11:21.233255 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:21.235837 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:21.239928 12672 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:21.240621 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:21.244710 12598 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:21.246644 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:21.247372 13332 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:21.254317 16119 consensus_queue.cc:1059] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:21.256009 16487 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:21.256628 16487 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:21.230940 16240 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:21.244415 12676 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:21.263801 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 224013us
I20260709 12:11:21.193928 16459 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:21.265244 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 352) took 1016 ms. Trace:
I20260709 12:11:21.288442 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 351) took 1039 ms. Trace:
I20260709 12:11:21.289288 12643 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:21.243252 16416 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:21.306401 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 72355.8us
I20260709 12:11:21.307124 12676 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:21.308497 12643 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:21.313486 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:21.285005 12673 rpcz_store.cc:276] 0709 12:11:20.249021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:20.650994 (+401973us) service_pool.cc:224] Handling call
0709 12:11:20.664728 (+ 13734us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:20.664944 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:11:20.675846 (+ 10902us) log.cc:844] Serialized 14 byte log entry
0709 12:11:20.937125 (+261279us) log.cc:844] Serialized 14 byte log entry
0709 12:11:21.264773 (+327648us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:21.264852 (+    79us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:21.264914 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:21.265004 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:21.265155 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15129,"prepare.queue_time_us":26511,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":3408,"raft.queue_time_us":149536,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":122,"thread_start_us":556126,"threads_started":2,"wal-append.queue_time_us":349426}
I20260709 12:11:21.288842 16502 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:11:21.317904 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:21.290107 12679 rpcz_store.cc:276] 0709 12:11:20.248703 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:20.650794 (+402091us) service_pool.cc:224] Handling call
0709 12:11:20.663750 (+ 12956us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:20.664090 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:20.664117 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:20.702286 (+ 38169us) log.cc:844] Serialized 26 byte log entry
0709 12:11:20.958503 (+256217us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:20.973206 (+ 14703us) log.cc:844] Serialized 14 byte log entry
0709 12:11:21.283646 (+310440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:21.283732 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:21.283783 (+    51us) raft_consensus.cc:1685] finished
0709 12:11:21.283802 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:21.288195 (+  4393us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":423,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":58,"raft.queue_time_us":85449,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":20132,"thread_start_us":544866,"threads_started":2,"wal-append.queue_time_us":341068}
I20260709 12:11:21.282089 16347 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.030s
W20260709 12:11:21.340737 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:21.337304 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:21.347536 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:21.309216 16510 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:21.349567 12531 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:21.348249 13337 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:21.321570 16502 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:21.310586 16487 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:21.328851 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:21.375734 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 47007.1us
I20260709 12:11:21.323868 16508 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:21.378358 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:21.427444 12672 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:21.426837 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:21.450093 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: true dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:21.455253 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 222730us
I20260709 12:11:21.377136 16517 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260709 12:11:21.456893 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 332161us
I20260709 12:11:21.458798 12523 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:21.459533 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:21.377483 16502 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:21.449476 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:21.470393 12530 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 8: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:21.457602 16119 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:21.478633 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:21.479154 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:21.480512 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:21.481134 12527 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:21.452209 16508 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:21.461959 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:21.463507 12528 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:11:21.466876 13330 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:11:21.468449 12529 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 8. Current term is 9. Ops: []
W20260709 12:11:21.497601 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 119500us
I20260709 12:11:21.498454 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:21.498624 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:21.499783 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:21.501291 12643 leader_election.cc:400] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 6. Current term is 7.
I20260709 12:11:21.419135 16240 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:21.479916 16494 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:21.490888 16517 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:21.468497 16283 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.041s
I20260709 12:11:21.489969 16515 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:21.523298 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 61468.5us
I20260709 12:11:21.486562 16508 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:21.526821 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 194298us
I20260709 12:11:21.474654 16119 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:21.548225 12643 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:21.549499 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 66563.9us
I20260709 12:11:21.536988 16472 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:21.542425 16515 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:21.559219 16513 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:11:21.560367 16515 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:21.561286 16512 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.208s	user 0.000s	sys 0.001s
W20260709 12:11:21.562407 16511 log.cc:927] Time spent T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:11:21.563668 16119 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:21.564007 16527 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:21.539398 16508 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:21.538849 16334 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:21.568957 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:21.570324 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:21.571033 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:21.574460 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:21.572487 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:21.544911 16446 consensus_meta.cc:287] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.554s	user 0.014s	sys 0.001s
I20260709 12:11:21.583400 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:21.527894 16494 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:21.577284 16517 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:21.612347 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:21.612859 12673 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:21.613076 12671 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:21.614486 12673 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:11:21.616822 12679 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
W20260709 12:11:21.618563 13302 leader_election.cc:336] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.510s (SENT)
I20260709 12:11:21.593065 16515 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:21.593179 16283 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:21.594460 16334 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:21.580729 16472 raft_consensus.cc:697] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:21.635675 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:21.647258 12677 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:21.641683 13332 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:21.602321 16211 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:21.648429 12671 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
W20260709 12:11:21.651486 13302 leader_election.cc:336] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.832s (SENT)
I20260709 12:11:21.652107 13302 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:21.632805 16515 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:21.631078 16182 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:21.646488 16334 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:21.631438 16283 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:21.646642 16510 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:11:21.678237 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 94893.3us
I20260709 12:11:21.620887 16530 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.061s
I20260709 12:11:21.594048 16457 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:21.673270 16515 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:21.659756 16539 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:21.682238 16334 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:21.729123 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 108227us
I20260709 12:11:21.744957 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:21.750541 16546 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:21.751086 16546 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:21.751524 16546 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:21.753803 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:21.727380 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:21.683789 16283 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:21.736600 16540 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:21.732203 16126 consensus_meta.cc:287] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.659s	user 0.007s	sys 0.006s
I20260709 12:11:21.741678 16539 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:21.686235 16514 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:11:21.771663 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 44633us
I20260709 12:11:21.739078 16493 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:21.781450 12643 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:21.670730 16517 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.084s
I20260709 12:11:21.793264 16510 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:21.734638 16223 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
I20260709 12:11:21.810063 12571 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:21.782709 16539 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:21.824759 16531 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:21.825708 16531 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:21.826578 16461 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:21.827077 16461 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:21.829905 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:21.832079 16523 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.265s	user 0.003s	sys 0.000s
I20260709 12:11:21.832574 12571 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:21.828078 16461 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:11:21.841485 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 56225.2us
W20260709 12:11:21.844566 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 63203.4us
I20260709 12:11:21.845397 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:21.849740 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 66096.4us
I20260709 12:11:21.846626 16126 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:21.846868 16527 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:21.847324 16070 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:21.851702 16528 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:21.861883 16517 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:11:21.862950 16542 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 12:11:21.866015 16551 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:21.860728 16540 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:21.865842 16545 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:21.870383 16126 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:21.788489 16472 consensus_queue.cc:237] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:21.871714 16549 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
W20260709 12:11:21.872221 16535 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 12:11:21.809836 16514 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:21.883131 16544 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:21.882800 16515 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:21.884001 16517 raft_consensus.cc:3055] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:21.883972 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:21.887665 16538 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.317s	user 0.001s	sys 0.001s
I20260709 12:11:21.892985 12531 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:21.893184 16281 raft_consensus.cc:3055] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:11:21.894353 16534 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 12:11:21.894942 16494 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:21.922483 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:21.941175 12527 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:11:21.941780 12602 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:21.955529 12572 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:21.956656 16528 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader election won for term 6
I20260709 12:11:21.957293 16528 raft_consensus.cc:697] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:21.958323 16528 consensus_queue.cc:237] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:21.961297 16541 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.392s	user 0.001s	sys 0.000s
I20260709 12:11:21.942798 12531 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:21.963876 16126 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:21.963608 12531 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:21.965117 12531 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:21.945959 12606 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:11:21.904495 16540 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:21.978348 12600 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:11:21.946702 16281 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:21.914966 16070 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:21.984280 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:21.902526 16551 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:21.987468 12529 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 8: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:11:21.987576 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 103673us
I20260709 12:11:21.939604 16517 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:21.955502 16531 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:22.001030 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:21.994845 16558 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:22.006295 16514 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:22.005767 16517 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:22.010970 16281 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:22.019960 16517 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:22.020206 16281 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:22.027896 16119 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:22.028347 16119 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:22.036767 12531 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
W20260709 12:11:22.042230 16538 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:11:22.048060 16550 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
W20260709 12:11:22.049765 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 48812.6us
I20260709 12:11:22.050920 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:22.051808 16152 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:22.052234 12643 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:22.052357 16152 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:22.052923 16152 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:22.015825 16515 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:21.980706 16540 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:22.062943 12643 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:22.025589 16508 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:22.050063 16544 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:11:22.060094 16502 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.574s	user 0.006s	sys 0.007s
W20260709 12:11:22.082473 16548 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.314s	user 0.005s	sys 0.000s
I20260709 12:11:22.029279 16119 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:21.992458 16070 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:22.002038 16551 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:22.097101 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 105179us
I20260709 12:11:22.099709 16562 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:22.091706 16524 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.027s	sys 0.049s
W20260709 12:11:22.108938 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46137.2us
I20260709 12:11:22.128780 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:22.088986 16502 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:22.109656 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:22.129999 12527 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:22.109957 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:22.110327 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:22.110653 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:22.110953 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:22.133359 12680 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:22.111325 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:22.123015 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:22.135516 12673 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:22.124984 12646 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:22.109334 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:22.077210 16539 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:22.092455 16510 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:11:22.135669 16524 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.027s	sys 0.050s
I20260709 12:11:22.145723 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:22.146553 12671 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:22.145273 12527 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
W20260709 12:11:22.151546 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:22.106031 16545 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:22.178776 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.126483 16119 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:22.180732 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.186678 12600 raft_consensus.cc:2376] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9. Re-sending same reply.
I20260709 12:11:22.185086 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:22.189710 12679 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:22.150074 16515 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:22.192279 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:11:22.192613 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:22.189114 16555 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:11:22.203598 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.121568 16421 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.219345 13304 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:22.173974 16558 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:22.115752 16152 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:22.239482 16562 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:22.226385 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:22.245831 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:22.247005 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:22.246922 12676 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 5.
I20260709 12:11:22.239612 12680 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:22.240790 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.240972 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.130087 16544 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.065s
I20260709 12:11:22.146481 16334 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:22.241976 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:22.194543 16459 consensus_meta.cc:287] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.888s	user 0.013s	sys 0.002s
I20260709 12:11:22.256230 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:22.215463 16510 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.260614 12679 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 8: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:22.251117 16545 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:22.263542 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:22.268546 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:22.273658 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.275187 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:22.279343 12676 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:11:22.236011 16421 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:22.281095 12598 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:22.272364 16562 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.284041 16559 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:11:22.285090 16566 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 12:11:22.285952 16560 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.286537 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: true dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:22.287293 16564 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:11:22.287710 16561 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:22.301270 16562 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:22.298187 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:22.312023 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:22.267128 16416 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:11:22.313822 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.260293 16510 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:22.308911 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:22.293895 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.319775 12675 raft_consensus.cc:2393] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 9: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:22.327500 12598 raft_consensus.cc:2376] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9. Re-sending same reply.
I20260709 12:11:22.328835 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:22.307788 16492 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.248351 16558 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:22.315027 16283 consensus_meta.cc:287] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.536s	user 0.005s	sys 0.009s
I20260709 12:11:22.332865 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:22.350836 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.319517 16410 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:22.358273 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.321111 16502 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:22.361866 12605 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:22.333441 16070 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:22.331307 16544 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:22.334848 16561 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:22.370802 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:22.376196 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.329979 16487 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:11:22.381109 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.384624 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:22.334661 16514 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.036s
I20260709 12:11:22.313400 16560 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:22.391726 12496 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:22.393415 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.357095 16283 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:22.399377 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.400585 12605 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 9: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:22.350034 16492 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:22.401729 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:22.414547 12605 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:22.387883 16539 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:22.390194 16530 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:22.386895 16559 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:22.411010 12606 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
I20260709 12:11:22.383806 16410 raft_consensus.cc:697] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:22.422819 12606 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:22.398358 16152 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:22.417083 16569 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.055s	user 0.001s	sys 0.001s
I20260709 12:11:22.409760 16419 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:22.430451 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:22.450837 12529 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 8: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:22.430672 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:22.437639 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:22.429097 12606 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:22.452570 12598 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 9 index: 4. (index mismatch)
I20260709 12:11:22.428913 16558 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:22.383206 16544 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:22.426762 16410 consensus_queue.cc:237] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:22.469262 16544 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:22.469841 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.470618 12604 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:22.474897 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:22.475731 12680 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 5 FOLLOWER]: Advancing to term 6
I20260709 12:11:22.478178 12598 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:22.478945 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:22.479403 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.479790 12677 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 13: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:11:22.432924 16532 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.852s	user 0.002s	sys 0.002s
I20260709 12:11:22.482461 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:22.483247 12672 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:11:22.484301 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:22.485397 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:22.485463 12680 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 6.
I20260709 12:11:22.451959 16502 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:22.486441 16457 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:22.486526 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:22.487914 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:22.488703 12606 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 9: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:22.489573 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.418228 16530 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:22.432579 16567 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 12:11:22.463181 12494 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:22.428519 16492 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:22.534623 12680 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:11:22.480737 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:22.480737 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:22.536635 12677 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
I20260709 12:11:22.433491 16421 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:22.425937 16334 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:22.543605 12571 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:22.514621 12606 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 7: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:22.482656 16283 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:22.489995 16546 consensus_meta.cc:287] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.738s	user 0.011s	sys 0.001s
I20260709 12:11:22.471140 16559 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:22.548290 16490 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:22.461606 16419 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:11:22.508850 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:22.482550 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2257) took 1404 ms. Trace:
I20260709 12:11:22.507895 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:22.495668 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:22.517256 12672 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 8: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:22.492837 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:22.537767 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:22.556756 13330 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:22.474033 12604 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:11:22.536484 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 73469.6us
I20260709 12:11:22.436404 16560 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:22.539716 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.560808 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:22.561866 12598 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 9: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:22.563872 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.549026 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:22.492931 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:22.564981 12680 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:22.495137 12524 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:11:22.539458 16530 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:22.535871 16472 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:22.567088 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:22.569038 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:22.569218 12598 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:22.567718 12677 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:22.569937 12672 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:22.572731 12672 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:22.566090 12606 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:22.515313 12525 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:22.559458 16283 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:22.575438 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 174549us
I20260709 12:11:22.579887 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:22.504048 16537 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.941s	user 0.000s	sys 0.004s
I20260709 12:11:22.560392 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:22.587396 12677 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:22.585457 12496 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:22.577385 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:22.589039 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:22.556567 12603 rpcz_store.cc:276] 0709 12:11:21.077725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:21.187221 (+109496us) service_pool.cc:224] Handling call
0709 12:11:21.187744 (+   523us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:21.188199 (+   455us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:21.188242 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:21.188887 (+   645us) log.cc:844] Serialized 26 byte log entry
0709 12:11:21.469280 (+280393us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:11:21.469322 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:21.469396 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:22.349429 (+880033us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:22.482242 (+132813us) raft_consensus.cc:1685] finished
0709 12:11:22.482275 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:22.482485 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13864,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":116,"spinlock_wait_cycles":27783936,"thread_start_us":178208,"threads_started":1,"wal-append.queue_time_us":301815}
I20260709 12:11:22.554855 16421 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:22.581914 12598 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 9: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:22.583487 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:22.590356 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 261648us
I20260709 12:11:22.490367 16561 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:22.552202 16546 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:22.580929 16490 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:22.602212 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:22.602381 12598 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:22.575374 12672 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 7 index: 4. (index mismatch)
I20260709 12:11:22.727190 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:22.728169 12675 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:11:22.607486 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:22.576722 16472 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:22.594614 16493 consensus_meta.cc:287] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.806s	user 0.013s	sys 0.002s
I20260709 12:11:22.633613 16546 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:22.634457 12601 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
W20260709 12:11:22.635003 16543 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.972s	user 0.003s	sys 0.002s
I20260709 12:11:22.653926 16545 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:22.653987 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:22.666313 16419 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:22.669417 16508 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:22.672111 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:22.767035 12600 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 13: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:22.766937 12604 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 10.
I20260709 12:11:22.768029 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.768195 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:22.768723 12600 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:11:22.769675 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:22.676268 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:22.770726 12530 raft_consensus.cc:2393] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 11: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:22.593819 16494 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:22.597204 16462 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:22.680418 16510 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:22.681200 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:22.683466 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:22.684396 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.774323 16494 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:22.686095 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:22.686331 12674 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:22.689599 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:22.780879 16508 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:11:22.691246 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:22.696506 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 1983) took 1377 ms. Trace:
I20260709 12:11:22.782807 12525 rpcz_store.cc:276] 0709 12:11:21.318778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:21.318928 (+   150us) service_pool.cc:224] Handling call
0709 12:11:22.696170 (+1377242us) spinlock_profiling.cc:243] Waited 893 ms on lock 0x7b5400054e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:22.696370 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2054084352}
I20260709 12:11:22.628216 16211 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:22.596963 16126 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:22.750914 16459 consensus_queue.cc:1059] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260709 12:11:22.784473 16126 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:22.784961 16126 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:22.785204 16459 raft_consensus.cc:3055] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:22.785652 16459 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:11:22.715085 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 153403us
W20260709 12:11:22.786587 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:22.783763 16540 consensus_meta.cc:287] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.693s	user 0.013s	sys 0.001s
I20260709 12:11:22.585732 16487 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.791388 16487 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:22.793519 16551 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.673s	user 0.011s	sys 0.002s
W20260709 12:11:22.747622 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 140304us
I20260709 12:11:22.793880 16487 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:22.795720 12679 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:11:22.795971 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:22.754715 12675 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260709 12:11:22.797950 16515 consensus_meta.cc:287] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.597s	user 0.006s	sys 0.006s
I20260709 12:11:22.798460 16515 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:22.799907 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:22.802965 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:22.804771 12571 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:22.807288 12571 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:22.756794 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 128362us
I20260709 12:11:22.756119 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:22.808629 12598 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:22.767098 16223 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:22.774797 12600 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 8: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:22.748273 16421 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:22.780354 16494 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:22.756011 16493 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:22.581761 16283 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:22.756379 16485 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:11:22.787985 13304 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:22.790041 12572 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:22.763144 12605 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:22.813354 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:22.813258 12597 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:11:22.830449 16575 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:22.831673 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:22.832875 13335 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 6: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:22.832991 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:22.835675 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:22.834883 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:22.838456 13337 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:22.839180 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 50815.2us
W20260709 12:11:22.843343 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 55455.2us
I20260709 12:11:22.844496 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:22.810309 16517 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.844270 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:22.845556 13336 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:22.853017 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:22.766278 16554 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.068s
I20260709 12:11:22.859491 12571 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:22.812901 16546 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:11:22.854406 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 44477.7us
I20260709 12:11:22.823092 16211 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:22.823807 16527 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:22.822855 16416 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:22.820499 16462 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:22.745685 16490 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:22.873555 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 84367.3us
I20260709 12:11:22.793998 16551 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:22.853883 16533 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.858976 16281 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:22.876509 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:22.884706 12605 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:22.863456 16545 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:22.800659 16459 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:22.865386 16517 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:22.810835 16540 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:22.836005 16571 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.051s
W20260709 12:11:22.956804 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:22.982674 12677 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:22.962355 12676 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
W20260709 12:11:23.020602 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:22.900942 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:11:23.029935 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 129077us
I20260709 12:11:22.889168 16546 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:22.898813 16527 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:22.980243 16446 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:23.039549 12671 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:22.905891 16553 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.982316 16539 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:22.905656 16575 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:22.980768 16281 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:23.016539 16416 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:23.017807 16070 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:23.106068 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 246667us
I20260709 12:11:23.028133 16487 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:22.979194 16460 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.065s
I20260709 12:11:23.140993 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:23.037761 16536 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:23.021903 16545 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:22.983240 16581 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.070s
W20260709 12:11:23.131289 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 254847us
I20260709 12:11:22.981709 16419 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.119s
I20260709 12:11:23.183876 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:23.049599 16493 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:23.137072 16459 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:23.123869 16560 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:23.141786 16281 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:23.127590 16576 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:22.980053 16533 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:23.181448 16545 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:23.169243 16536 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:23.141606 16145 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.073s
I20260709 12:11:23.142231 16539 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:23.237478 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 53738.5us
I20260709 12:11:23.245064 12643 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:23.040751 16530 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:23.239342 12679 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:23.137554 16553 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:23.130091 16446 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:23.266405 16554 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:23.276507 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:23.209481 16462 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:23.277700 16493 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:23.278174 16446 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:23.278323 16493 raft_consensus.cc:697] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:11:23.232867 16588 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
I20260709 12:11:23.265023 16540 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:23.231542 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:23.166982 16416 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:23.166363 16070 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:23.181427 16487 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:23.233546 16589 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 12:11:23.283743 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:23.311674 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:23.261044 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:23.289815 13337 raft_consensus.cc:2410] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:11:23.225893 16421 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:23.224395 16515 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:23.292053 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:23.241384 16536 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:23.236088 16562 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:23.252223 16283 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:23.200349 16546 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:23.238458 16576 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:23.279549 16493 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:23.314406 16421 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:23.330229 16416 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:23.330435 16421 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:23.334555 16533 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:23.336423 16576 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:23.321987 12643 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:23.322752 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 288972us
I20260709 12:11:23.220511 16560 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:23.328663 16546 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:23.348507 12600 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260709 12:11:23.308189 16490 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:22.496687 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:23.408094 12602 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: []
W20260709 12:11:23.441525 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 459599us
I20260709 12:11:23.319352 16540 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:23.444492 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 122615us
W20260709 12:11:23.447077 12672 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.013s	sys 0.006s
I20260709 12:11:23.447129 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:23.448475 12597 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
W20260709 12:11:23.450068 12672 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.013s	sys 0.006s
I20260709 12:11:23.354285 16515 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:11:23.455001 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 314085us
I20260709 12:11:23.455976 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:23.355702 16551 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:23.274844 16575 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:23.475069 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:23.514745 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:23.515723 13337 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
W20260709 12:11:23.473984 13302 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.506s (SENT) [suppressed 36 similar messages]
I20260709 12:11:23.469048 16562 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260709 12:11:23.472072 16419 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:23.547629 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:23.475360 16283 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:23.520932 16588 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:11:23.521447 16596 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:11:23.544548 16595 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 12:11:23.519807 16457 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:11:23.539625 16591 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.043s
I20260709 12:11:23.541374 16540 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.044s
I20260709 12:11:23.581542 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1974) took 1457 ms. Trace:
I20260709 12:11:23.582911 16487 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:23.560817 16596 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:23.551842 16419 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:23.584607 12680 rpcz_store.cc:276] 0709 12:11:22.124610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:22.566363 (+441753us) service_pool.cc:224] Handling call
0709 12:11:22.573078 (+  6715us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:22.578966 (+  5888us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:22.578997 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:22.582795 (+  3798us) log.cc:844] Serialized 26 byte log entry
0709 12:11:22.870218 (+287423us) spinlock_profiling.cc:243] Waited 4.71 ms on lock 0x7b180001f2a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:11:23.037082 (+166864us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:23.254908 (+217826us) log.cc:844] Serialized 14 byte log entry
0709 12:11:23.255251 (+   343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:23.255350 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:23.581087 (+325737us) raft_consensus.cc:1685] finished
0709 12:11:23.581234 (+   147us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:23.581457 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":294360,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":236,"raft.run_cpu_time_us":360,"raft.run_wall_time_us":19042,"spinlock_wait_cycles":10835328,"thread_start_us":282975,"threads_started":2,"wal-append.queue_time_us":615433}
I20260709 12:11:23.563915 16553 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:23.552090 16571 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.026s
I20260709 12:11:23.594339 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:23.602972 12678 consensus_meta.cc:287] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.747s	user 0.011s	sys 0.000s
I20260709 12:11:23.608333 12678 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:23.607329 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:23.546873 16594 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 2, Time since last communication: 0.050s
W20260709 12:11:23.651743 16527 consensus_meta.cc:287] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.600s	user 0.009s	sys 0.001s
I20260709 12:11:23.652361 16527 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:23.611064 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2366) took 1024 ms. Trace:
I20260709 12:11:23.595456 16530 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:23.619510 16546 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:23.595854 16539 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:23.591688 16596 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:23.623978 16182 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:23.629597 16126 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.844s	user 0.011s	sys 0.000s
I20260709 12:11:23.634079 16560 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:23.645539 12680 raft_consensus.cc:2410] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 12:11:23.656404 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 179598us
I20260709 12:11:23.656495 12671 rpcz_store.cc:276] 0709 12:11:22.586996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:23.233156 (+646160us) service_pool.cc:224] Handling call
0709 12:11:23.233461 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:23.233684 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:11:23.364128 (+130444us) log.cc:844] Serialized 14 byte log entry
0709 12:11:23.609696 (+245568us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:23.610528 (+   832us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:23.610596 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:23.610793 (+   197us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:23.610981 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":97974,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":88,"raft.queue_time_us":113,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":2890,"thread_start_us":374784,"threads_started":2}
I20260709 12:11:23.842842 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:23.843459 12671 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:23.812602 16547 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:23.817764 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2271) took 1356 ms. Trace:
I20260709 12:11:23.858623 12606 rpcz_store.cc:276] 0709 12:11:22.461546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:22.587082 (+125536us) service_pool.cc:224] Handling call
0709 12:11:22.587479 (+   397us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:22.766870 (+179391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:22.766902 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:22.766949 (+    47us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:22.766988 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:23.817361 (+1050373us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b54001176d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:23.817435 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:23.817651 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":412040576}
W20260709 12:11:23.859683 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:23.669298 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 195333us
I20260709 12:11:23.669520 16182 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:23.813820 16568 raft_consensus.cc:3055] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:23.822636 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:23.823109 12602 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
W20260709 12:11:23.820534 16459 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.595s	user 0.015s	sys 0.016s
I20260709 12:11:23.661602 16493 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:23.824213 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:23.824728 16616 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.157s	user 0.005s	sys 0.000s
I20260709 12:11:23.873647 12671 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:23.668236 16546 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:23.668193 16602 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:23.876549 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:23.876849 16472 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:23.877148 16602 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:23.879683 16602 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:23.819527 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1989) took 1341 ms. Trace:
W20260709 12:11:23.815968 16494 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.969s	user 0.016s	sys 0.011s
I20260709 12:11:23.881528 16494 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:23.884243 16494 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:23.817509 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:23.885099 12680 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:23.889084 12678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1977) took 1763 ms (client timeout 1708 ms). Trace:
I20260709 12:11:23.889365 16493 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:23.678613 16419 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:23.705430 16126 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:23.836728 16596 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:23.820092 16610 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.165s	user 0.007s	sys 0.000s
W20260709 12:11:23.841563 16281 consensus_meta.cc:287] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.603s	user 0.008s	sys 0.012s
I20260709 12:11:23.643483 16421 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:23.863372 12643 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:23.670248 16603 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:11:23.875070 16619 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:23.868649 16568 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:23.890312 16553 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:23.891028 16530 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:23.891847 16531 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:23.925594 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:23.917973 16459 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:23.828788 16536 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:23.898384 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:23.898859 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:23.900272 12680 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
W20260709 12:11:23.889791 12678 rpcz_store.cc:269] 0709 12:11:22.125531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:22.170543 (+ 45012us) service_pool.cc:224] Handling call
0709 12:11:23.888802 (+1718259us) spinlock_profiling.cc:243] Waited 661 ms on lock 0x7b54002c5e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:23.888966 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1520759552}
W20260709 12:11:23.920517 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 57230us
I20260709 12:11:23.874230 16594 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:23.876606 16182 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:23.919325 16545 consensus_meta.cc:287] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.671s	user 0.010s	sys 0.007s
I20260709 12:11:23.876880 16546 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:23.954021 16619 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:23.920085 16283 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:23.958675 13302 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:23.959905 12671 raft_consensus.cc:2410] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 10 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 3.
I20260709 12:11:23.975456 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:23.963467 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 64762.3us
I20260709 12:11:23.961364 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:23.956629 16531 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:23.924281 16457 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:23.967108 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:23.867000 16547 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:23.984315 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:23.976672 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:23.957433 16553 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:23.960094 16565 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.217s	user 0.023s	sys 0.012s
W20260709 12:11:23.997712 16565 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.277s	user 0.024s	sys 0.012s
I20260709 12:11:23.920404 16514 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:23.999905 16514 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:24.000581 16493 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:24.001659 12425 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58992 (request call id 171) took 23036 ms (client timeout 15000 ms). Trace:
I20260709 12:11:24.002394 16514 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:23.968837 16459 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:24.004469 16459 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:23.982506 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:23.922829 16281 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:24.006640 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:24.007203 16536 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:23.962060 16576 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:24.007545 16281 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:24.007936 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:23.989362 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:24.008663 16562 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:24.008287 16281 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:11:24.009148 16562 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:24.009346 16527 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:24.009640 16562 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:24.009882 16527 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:23.990309 16126 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:23.990830 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1988) took 1580 ms. Trace:
I20260709 12:11:23.991947 16531 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:23.992530 16334 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:23.992933 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2368) took 1395 ms. Trace:
I20260709 12:11:24.021991 12678 rpcz_store.cc:276] 0709 12:11:22.597796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:23.991718 (+1393922us) service_pool.cc:224] Handling call
0709 12:11:23.992194 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:23.992445 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:23.992484 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:23.992549 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:23.992606 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:23.992690 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:23.992854 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:24.023434 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:24.024040 16547 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:23.968415 16594 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:24.024926 16553 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:24.027040 16594 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:23.993126 16568 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:23.983275 16457 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:23.983064 16182 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:23.972873 16545 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:23.958751 16530 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:23.958688 16606 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 12:11:23.989960 16591 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:24.010798 12675 rpcz_store.cc:276] 0709 12:11:22.410257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:22.797610 (+387353us) service_pool.cc:224] Handling call
0709 12:11:22.798047 (+   437us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:22.798927 (+   880us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:22.799077 (+   150us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:23.893212 (+1094135us) spinlock_profiling.cc:243] Waited 928 us on lock 0x7b2c00037868. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:11:23.893673 (+   461us) log.cc:844] Serialized 26 byte log entry
0709 12:11:23.973557 (+ 79884us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:23.990158 (+ 16601us) log.cc:844] Serialized 14 byte log entry
0709 12:11:23.990392 (+   234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:23.990470 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:23.990521 (+    51us) raft_consensus.cc:1685] finished
0709 12:11:23.990627 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:23.990769 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17513,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":1891,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":92,"spinlock_wait_cycles":2133376,"thread_start_us":79456,"threads_started":1,"wal-append.queue_time_us":83634}
I20260709 12:11:24.015463 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:24.024233 12678 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 9: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:23.973323 16602 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:24.030105 12569 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:23.984012 16510 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:24.014174 16560 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:23.987421 16559 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:24.013912 16576 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:24.038597 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2367) took 1444 ms. Trace:
W20260709 12:11:24.039915 12676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 1983) took 1768 ms (client timeout 2027 ms). Trace:
I20260709 12:11:24.042063 16539 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:24.043092 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2262) took 2135 ms. Trace:
I20260709 12:11:24.064813 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2351) took 1775 ms. Trace:
W20260709 12:11:24.062933 12676 rpcz_store.cc:269] 0709 12:11:22.271352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:22.280287 (+  8935us) service_pool.cc:224] Handling call
0709 12:11:24.039732 (+1759445us) spinlock_profiling.cc:243] Waited 474 ms on lock 0x7b54000ac8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:24.039851 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1090830592}
W20260709 12:11:24.044756 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 58073.9us
I20260709 12:11:24.065435 12601 rpcz_store.cc:276] 0709 12:11:21.907538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:21.907837 (+   299us) service_pool.cc:224] Handling call
0709 12:11:21.942605 (+ 34768us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:22.750926 (+808321us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:24.042773 (+1291847us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b54001617d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:24.043003 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1590386560}
I20260709 12:11:23.999264 16619 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:24.031432 16493 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:24.046227 16457 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:24.049312 16568 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:24.054463 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2370) took 1337 ms. Trace:
I20260709 12:11:24.050010 16594 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:24.062772 12679 rpcz_store.cc:276] 0709 12:11:22.594039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:23.275624 (+681585us) service_pool.cc:224] Handling call
0709 12:11:23.295065 (+ 19441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:23.295485 (+   420us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:23.295514 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:23.296014 (+   500us) log.cc:844] Serialized 26 byte log entry
0709 12:11:23.883041 (+587027us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b1800104e60. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:11:23.957868 (+ 74827us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:23.957925 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:23.958016 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:24.038211 (+ 80195us) raft_consensus.cc:1685] finished
0709 12:11:24.038239 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:24.038513 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14774,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":2393,"spinlock_wait_cycles":23260416,"thread_start_us":74580,"threads_started":1,"wal-append.queue_time_us":715641}
I20260709 12:11:24.066429 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:24.045995 16618 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:24.065485 12677 rpcz_store.cc:276] 0709 12:11:22.289255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:22.588518 (+299263us) service_pool.cc:224] Handling call
0709 12:11:22.588936 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:23.020136 (+431200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:24.064511 (+1044375us) spinlock_profiling.cc:243] Waited 393 ms on lock 0x7b54002c4a54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:24.064747 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":904863104}
I20260709 12:11:24.062748 16571 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:24.068041 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:24.067638 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:24.071008 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:24.071147 16510 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:24.071713 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:24.072484 13330 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:11:24.072642 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:24.072934 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:24.073185 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:24.073341 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:24.073644 16223 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:24.077857 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:24.086560 13338 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:11:24.062384 16602 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:24.079433 12676 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
I20260709 12:11:24.047545 16182 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:24.060923 16591 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:24.062636 16576 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:24.092166 16571 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:24.092324 13332 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:24.093048 12678 raft_consensus.cc:2393] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 9: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:24.064767 16540 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:24.093472 13337 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:24.076524 12675 rpcz_store.cc:276] 0709 12:11:22.717381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:24.052546 (+1335165us) service_pool.cc:224] Handling call
0709 12:11:24.052862 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:24.053077 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:24.053109 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:24.053156 (+    47us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:24.053185 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:24.053237 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:24.053361 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:24.094151 13335 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:24.093595 13333 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:24.099202 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:24.081622 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:24.138764 12677 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 11 because replica has last-logged OpId of term: 9 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:11:24.139235 12675 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:24.052183 16527 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:24.095458 16493 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:24.158896 16493 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:24.076952 16619 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:24.161150 16493 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:24.161612 16619 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:24.164575 16619 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:24.156236 16182 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:24.082690 16594 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:24.168097 12494 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:11:24.002091 12425 rpcz_store.cc:269] 0709 12:11:00.965639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:00.981269 (+ 15630us) service_pool.cc:224] Handling call
0709 12:11:05.096062 (+4114793us) spinlock_profiling.cc:243] Waited 3.59 ms on lock 0x7b5400280190. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fc0b2 00007f148e875759 00007f148e80da76 00007f148e7ded58 00007f148e8dd04e 00007f148cd9e2a4 00007f148cd9e232 00007f148cd9e161 00007f148cd9e0dc
0709 12:11:24.001401 (+18905339us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:11:13.037117 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:11:13.154714 (+117597us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:11:13.154770 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:11:13.154798 (+    28us) tablet.cc:662] Decoding operations
0709 12:11:13.157516 (+  2718us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:11:13.157611 (+    95us) write_op.cc:641] Partition lock acquired for write op
0709 12:11:13.157657 (+    46us) tablet.cc:685] Acquiring locks for 22 operations
0709 12:11:13.160102 (+  2445us) tablet.cc:701] Row locks acquired
0709 12:11:13.160132 (+    30us) write_op.cc:260] PREPARE: finished
0709 12:11:13.160338 (+   206us) write_op.cc:270] Start()
0709 12:11:13.160525 (+   187us) write_op.cc:276] Timestamp: P: 1783599073160265 usec, L: 0
0709 12:11:13.160557 (+    32us) op_driver.cc:348] REPLICATION: starting
0709 12:11:13.345570 (+185013us) log.cc:844] Serialized 8676 byte log entry
0709 12:11:16.208604 (+2863034us) op_driver.cc:464] REPLICATION: finished
0709 12:11:17.894478 (+1685874us) write_op.cc:301] APPLY: starting
0709 12:11:17.912960 (+ 18482us) tablet.cc:1367] starting BulkCheckPresence
0709 12:11:17.987030 (+ 74070us) tablet.cc:1370] finished BulkCheckPresence
0709 12:11:17.987198 (+   168us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:11:17.989583 (+  2385us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:11:17.990078 (+   495us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=44
0709 12:11:18.249981 (+259903us) write_op.cc:312] APPLY: finished
0709 12:11:18.409291 (+159310us) log.cc:844] Serialized 152 byte log entry
0709 12:11:19.212952 (+803661us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b1800304b40. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a
0709 12:11:19.213262 (+   310us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:19.213770 (+   508us) write_op.cc:454] Released schema lock
0709 12:11:19.213862 (+    92us) write_op.cc:341] FINISH: Updating metrics
0709 12:11:19.270900 (+ 57038us) spinlock_profiling.cc:243] Waited 3.45 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:19.315206 (+ 44306us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:19.317902 (+  2696us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2299,"reported_tablets":51,"spinlock_wait_cycles":8260992,"child_traces":[["op",{"apply.queue_time_us":1611305,"apply.run_cpu_time_us":52097,"apply.run_wall_time_us":1388613,"num_ops":22,"prepare.queue_time_us":283919,"prepare.run_cpu_time_us":14779,"prepare.run_wall_time_us":557269,"raft.queue_time_us":1772548,"raft.run_cpu_time_us":32620,"raft.run_wall_time_us":1684477,"replication_time_us":3016841,"spinlock_wait_cycles":12590592,"thread_start_us":3959237,"threads_started":5,"wal-append.queue_time_us":782967,"wal-append.run_cpu_time_us":31580,"wal-append.run_wall_time_us":4639990}]]}
I20260709 12:11:24.143679 12675 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:24.174929 13333 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
W20260709 12:11:24.169934 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 59334.4us
I20260709 12:11:24.178694 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:24.180335 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:24.184381 12643 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:24.098338 16630 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:24.188917 12676 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:11:24.189666 12678 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:11:24.116380 16571 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:24.196489 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:24.097587 16223 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:24.185158 16536 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:24.228611 16536 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:24.099960 16510 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:24.089040 16618 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:24.177518 16540 raft_consensus.cc:697] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:24.209761 16461 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:24.110288 16495 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.081s
I20260709 12:11:24.180255 16126 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:11:24.191388 16514 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:24.214818 16576 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:24.216032 16530 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:24.215971 16602 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:24.187834 16459 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:24.218571 16223 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:24.219139 16591 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:24.225430 16457 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:24.272044 16457 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:24.188695 16493 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:24.191272 16560 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:24.273164 16560 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:24.273511 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:24.274340 12526 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:24.227805 16619 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:24.228631 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:24.232141 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:24.275808 13334 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
W20260709 12:11:24.231592 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:24.233330 13335 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:24.235381 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:24.283593 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48320.9us
I20260709 12:11:24.283416 16540 consensus_queue.cc:237] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:24.284950 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:23.880348 12672 rpcz_store.cc:276] 0709 12:11:22.478229 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:22.840637 (+362408us) service_pool.cc:224] Handling call
0709 12:11:22.864320 (+ 23683us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:22.864528 (+   208us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:11:23.452239 (+587711us) log.cc:844] Serialized 14 byte log entry
0709 12:11:23.819104 (+366865us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:23.819176 (+    72us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:23.819246 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:23.819316 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:23.819462 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":466563,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":39484,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":8272,"thread_start_us":794616,"threads_started":2,"wal-append.queue_time_us":354752}
I20260709 12:11:24.286329 16629 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:24.286791 16629 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:24.287139 16629 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:24.244109 12530 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:11:24.252969 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:24.290382 16517 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:24.257889 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:24.291869 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:24.256666 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:24.293030 12678 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:11:24.293229 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:24.294443 12528 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:24.294565 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:24.261250 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 194301us
I20260709 12:11:24.297115 16618 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:24.297760 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2100) took 1699 ms. Trace:
I20260709 12:11:24.298641 16594 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:24.298178 12603 rpcz_store.cc:276] 0709 12:11:22.598319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:22.598488 (+   169us) service_pool.cc:224] Handling call
0709 12:11:22.598953 (+   465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:22.611527 (+ 12574us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:22.611570 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:22.716955 (+105385us) log.cc:844] Serialized 26 byte log entry
0709 12:11:22.850556 (+133601us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:22.874490 (+ 23934us) log.cc:844] Serialized 14 byte log entry
0709 12:11:23.894972 (+1020482us) spinlock_profiling.cc:243] Waited 40.8 ms on lock 0x7b1800369520. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485933557 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:11:24.297294 (+402322us) spinlock_profiling.cc:243] Waited 2.45 ms on lock 0x7b1800369520. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287
0709 12:11:24.297446 (+   152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:24.297517 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:24.297559 (+    42us) raft_consensus.cc:1685] finished
0709 12:11:24.297572 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:24.297692 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12007,"prepare.queue_time_us":23609,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":2527,"raft.queue_time_us":37226,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":35675,"spinlock_wait_cycles":103442688,"thread_start_us":232560,"threads_started":2,"wal-append.queue_time_us":273479}
I20260709 12:11:24.244634 16539 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:24.241389 16283 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:24.273784 12675 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:24.227383 16545 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:24.235067 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:24.229064 16536 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:24.254056 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:24.384812 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 205509us
I20260709 12:11:24.300817 12676 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
I20260709 12:11:24.303834 16510 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:24.304638 16576 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:24.308449 16421 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:24.309208 16596 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:24.310227 16459 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:24.194718 16630 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:24.312582 16281 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:24.316033 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2369) took 1703 ms. Trace:
I20260709 12:11:24.425925 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:24.426599 12605 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:24.264189 16514 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:24.273604 16560 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:24.263756 12525 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:24.426204 12674 rpcz_store.cc:276] 0709 12:11:22.612999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:24.033067 (+1420068us) service_pool.cc:224] Handling call
0709 12:11:24.033413 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:24.043517 (+ 10104us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:24.043579 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:24.046339 (+  2760us) log.cc:844] Serialized 26 byte log entry
0709 12:11:24.098275 (+ 51936us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:24.098323 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:24.098403 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:24.315749 (+217346us) raft_consensus.cc:1685] finished
0709 12:11:24.315778 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:24.315955 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9674,"raft.queue_time_us":10155,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":3207,"thread_start_us":39218,"threads_started":1,"wal-append.queue_time_us":198592}
W20260709 12:11:24.388370 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:24.390599 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:24.435271 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:24.391665 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:24.435851 12603 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:24.436424 12600 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:24.437041 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:24.437652 12603 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:24.438504 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:24.439065 12603 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:24.439858 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:24.393097 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:24.401147 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:24.414084 12672 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:24.415704 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:24.447199 12601 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:24.325158 16462 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:24.418206 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:24.417768 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:24.423360 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 132097us
I20260709 12:11:24.316249 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:24.428138 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:24.376204 16461 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:11:24.321797 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:24.435829 12674 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:24.464236 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:24.465243 12678 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:24.400486 16517 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:24.483433 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:24.484186 12602 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:24.384383 16562 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:24.477216 12603 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:11:24.427757 16510 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:24.497980 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:24.499219 12603 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
W20260709 12:11:24.485985 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 68324.7us
I20260709 12:11:24.502089 12572 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:24.505115 12572 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:24.476419 12674 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:24.485463 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:24.507349 16530 consensus_queue.cc:1059] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:24.508950 13335 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260709 12:11:24.412904 16283 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:24.569097 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 62634.9us
I20260709 12:11:24.539592 12605 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:24.587324 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:24.594513 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 89522.7us
I20260709 12:11:24.468338 16527 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:24.601346 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:24.599308 12601 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:24.602207 13330 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:24.603629 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:24.612634 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:24.495479 16485 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:24.508797 16618 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:24.593222 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:24.624300 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:24.477958 16545 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.144s
I20260709 12:11:24.485416 16421 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:24.634078 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:24.495471 16517 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:24.509420 16575 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:24.672412 16517 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:24.564183 16211 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.073s
W20260709 12:11:24.674830 16634 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:11:24.599249 16410 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.076s
I20260709 12:11:24.676316 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:24.677016 12676 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:24.678082 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:24.679248 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:24.681802 16629 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:24.683386 16211 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:24.683971 16211 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:24.684852 16629 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:24.687350 13338 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:24.689155 13334 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:24.689555 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:24.479070 16126 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.136s
I20260709 12:11:24.464313 16462 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:24.641762 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:24.741061 13330 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:24.471715 16461 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:24.591463 16619 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:24.563993 16182 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:24.664645 12600 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:24.471285 16495 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.130s
W20260709 12:11:24.691354 16641 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:11:24.644301 16539 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:24.698338 16421 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:24.697883 16211 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:24.839180 16539 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:24.839913 16539 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:24.840862 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:24.680332 12676 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:24.636170 16485 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:24.708297 16575 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:24.509747 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:24.844101 12603 raft_consensus.cc:2376] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 10. Re-sending same reply.
I20260709 12:11:24.711624 16562 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:24.713685 16459 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:24.715479 16630 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:24.716101 16508 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:24.592099 16510 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:24.637717 16618 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:24.720701 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2028) took 1499 ms. Trace:
I20260709 12:11:24.728864 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:24.730527 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 96555.4us
W20260709 12:11:24.737839 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 48411.6us
I20260709 12:11:24.747224 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:24.871357 12674 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:24.875062 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:24.597095 16591 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:24.791788 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:24.733580 16568 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.653s	user 0.012s	sys 0.000s
I20260709 12:11:24.797384 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:24.806154 13335 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260709 12:11:24.788862 16629 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:11:24.804733 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 345053us
I20260709 12:11:24.894775 13304 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:24.808917 16182 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:24.621556 16527 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:24.813305 12494 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:11:24.904359 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 91610.7us
W20260709 12:11:24.875527 16571 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.669s	user 0.010s	sys 0.002s
I20260709 12:11:24.830220 16410 raft_consensus.cc:3055] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:11:24.832916 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:24.846416 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:24.631734 16561 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:24.865856 12603 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:11:24.793454 16619 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:24.901719 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:24.926760 12676 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:24.927309 12605 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:24.829394 16536 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:24.882632 16211 consensus_queue.cc:1059] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
W20260709 12:11:24.932539 16590 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 2.079s	user 0.005s	sys 0.006s
I20260709 12:11:24.932824 16211 raft_consensus.cc:3055] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:24.933228 16211 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:24.903374 16182 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:24.934053 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2354) took 2642 ms. Trace:
I20260709 12:11:24.914417 16419 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:24.935804 16419 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:24.874436 16508 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:24.938547 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:24.914911 16459 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:11:24.939128 16626 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:11:24.939241 12676 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:24.939371 16459 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:24.940127 16459 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:24.941217 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 343) took 1079 ms. Trace:
I20260709 12:11:24.943854 16508 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:24.923874 12671 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 8: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:24.947798 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:24.948580 12671 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:24.952569 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:24.953254 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:24.953567 12529 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 11: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:24.956261 16283 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:24.905270 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:24.913362 16410 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:24.874083 16575 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:24.897152 16629 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:24.902774 16562 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:11:24.971468 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 66328.2us
I20260709 12:11:24.901962 16630 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:25.013360 12494 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:24.949972 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:25.014663 12671 raft_consensus.cc:2376] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8. Re-sending same reply.
W20260709 12:11:25.014942 12602 consensus_meta.cc:287] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.530s	user 0.012s	sys 0.000s
I20260709 12:11:24.934585 12673 rpcz_store.cc:276] 0709 12:11:22.292087 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:22.708656 (+416569us) service_pool.cc:224] Handling call
0709 12:11:22.747887 (+ 39231us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:22.757773 (+  9886us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:22.757826 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:22.758839 (+  1013us) log.cc:844] Serialized 26 byte log entry
0709 12:11:22.854244 (+ 95405us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:11:22.854294 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:22.854396 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:24.931912 (+2077516us) spinlock_profiling.cc:243] Waited 3.1 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:24.933727 (+  1815us) raft_consensus.cc:1685] finished
0709 12:11:24.933750 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:24.933973 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10080,"raft.queue_time_us":13057,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":128,"spinlock_wait_cycles":7121152,"thread_start_us":39894,"threads_started":1,"wal-append.queue_time_us":66447}
W20260709 12:11:25.007962 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 133057us
I20260709 12:11:25.016669 12602 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:25.018110 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:24.941772 12606 rpcz_store.cc:276] 0709 12:11:23.861615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:23.861790 (+   175us) service_pool.cc:224] Handling call
0709 12:11:23.862234 (+   444us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:23.862463 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:23.862488 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:23.869525 (+  7037us) log.cc:844] Serialized 26 byte log entry
0709 12:11:24.045878 (+176353us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:24.052601 (+  6723us) log.cc:844] Serialized 14 byte log entry
0709 12:11:24.058333 (+  5732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:24.058825 (+   492us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:24.221266 (+162441us) spinlock_profiling.cc:243] Waited 8.93 ms on lock 0x7b1800489640. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485933522 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:11:24.713914 (+492648us) spinlock_profiling.cc:243] Waited 6.63 ms on lock 0x7b1800489640. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:11:24.940912 (+226998us) raft_consensus.cc:1685] finished
0709 12:11:24.940945 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:24.941122 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":7761,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":2321,"raft.queue_time_us":5990,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":3107,"spinlock_wait_cycles":35767424,"thread_start_us":122186,"threads_started":1,"wal-append.queue_time_us":151199}
W20260709 12:11:25.009272 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 114631us
I20260709 12:11:25.010253 13335 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:25.019201 13335 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:25.006644 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:25.019428 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:25.005620 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:25.005620 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:24.868150 12522 rpcz_store.cc:276] 0709 12:11:23.221095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:23.221277 (+   182us) service_pool.cc:224] Handling call
0709 12:11:23.226028 (+  4751us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:23.323916 (+ 97888us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:11:24.319898 (+995982us) spinlock_profiling.cc:243] Waited 5.13 ms on lock 0x7b18003f8aa0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf
0709 12:11:24.320046 (+   148us) log.cc:844] Serialized 14 byte log entry
0709 12:11:24.487474 (+167428us) log.cc:844] Serialized 14 byte log entry
0709 12:11:24.487681 (+   207us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:24.487746 (+    65us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:24.487813 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:24.715928 (+228115us) spinlock_profiling.cc:243] Waited 10 ms on lock 0x7b18003f8aa0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419
0709 12:11:24.718649 (+  2721us) spinlock_profiling.cc:243] Waited 97.6 ms on lock 0x7b54003ca5d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:24.718725 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:24.720524 (+  1799us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11890,"prepare.queue_time_us":610238,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":75,"raft.queue_time_us":14345,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":86389,"spinlock_wait_cycles":259197312,"thread_start_us":358356,"threads_started":3,"wal-append.queue_time_us":277189}
W20260709 12:11:25.018680 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:25.026391 13305 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:25.020852 12675 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:11:24.925724 16571 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:25.048868 13339 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:25.052210 12572 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:25.056488 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:25.053700 12675 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:25.053799 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:25.047335 13335 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:25.057765 12673 raft_consensus.cc:2393] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 11: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:24.927664 16561 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:25.006837 16145 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:25.012778 16629 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:25.025676 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:25.074756 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49182.7us
I20260709 12:11:25.023257 16562 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:24.902206 16568 consensus_queue.cc:1059] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.122s
I20260709 12:11:25.059190 12674 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:25.059551 12675 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:11:25.045404 16517 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:25.075685 12643 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:25.056444 13333 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:11:25.028702 16126 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:25.084887 16281 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:25.056187 16639 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:25.089946 12675 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: [7.4-7.4]
I20260709 12:11:25.097494 12674 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:11:25.060365 12676 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:25.051432 16536 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:25.023697 16630 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:25.101815 16419 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:25.099486 12671 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:25.102792 16421 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:25.105137 16575 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:25.072431 16561 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:25.106087 16070 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:25.106117 16536 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:25.107020 16536 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:25.108372 16510 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:25.101847 16630 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:25.100418 16334 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:25.085963 16410 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:25.110689 16334 raft_consensus.cc:697] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:25.111384 16410 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:25.111687 16334 consensus_queue.cc:237] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:25.056878 16640 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:25.058959 16211 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:25.114274 16461 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:25.114658 16640 raft_consensus.cc:697] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 9 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:25.115442 16211 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:25.116076 16640 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:25.122429 16527 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:25.092865 16562 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:25.023535 16283 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260709 12:11:25.093513 16540 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:25.107544 16619 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:25.131150 16540 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:25.131649 16540 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:25.028936 16223 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 12:11:25.074577 16514 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:25.137609 16591 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:25.137827 16517 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:25.145975 16126 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:25.199555 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:25.197889 13302 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:25.200624 13333 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 11: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:25.173494 16421 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:25.182019 16527 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:25.208246 12674 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:25.212137 12674 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:11:25.149986 16594 consensus_queue.cc:1059] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.053s
I20260709 12:11:25.138875 16281 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:25.221139 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:25.217937 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:25.209812 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:25.223569 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:25.181640 16459 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:25.228124 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:25.229768 12671 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:25.232625 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:25.208642 16618 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:25.219973 16560 consensus_meta.cc:287] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.745s	user 0.011s	sys 0.003s
I20260709 12:11:25.217353 16647 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:25.246840 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:25.248055 12528 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 9: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:11:25.244482 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 149185us
I20260709 12:11:25.209374 16653 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:11:25.254313 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:25.205796 16539 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:25.255695 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:25.256582 12529 raft_consensus.cc:2368] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 9. Current term is 10.
I20260709 12:11:25.227393 16568 consensus_queue.cc:1243] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:11:25.250360 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:25.259042 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:25.202428 16591 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:25.227442 16656 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:25.251899 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:25.264631 12522 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:25.249850 16654 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:25.269395 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:25.216706 16527 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:25.263396 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:25.269531 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:25.271440 13333 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:25.274971 12671 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:11:25.284154 12678 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: [9.5-9.5]
I20260709 12:11:25.284533 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:25.285398 12528 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:25.250051 16560 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:25.283490 12606 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:11:25.273763 12572 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:25.297092 13302 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:25.309314 12569 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:25.309444 12572 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:25.310801 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:25.314575 12496 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.004s	sys 0.003s
I20260709 12:11:25.250917 12643 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:25.319751 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 69928.2us
I20260709 12:11:25.254989 16545 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:11:25.281782 16664 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:25.254101 16421 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:25.259624 16657 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:11:25.328684 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:25.351344 12606 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:25.344786 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:25.352473 13333 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:25.347106 12496 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.004s	sys 0.003s
I20260709 12:11:25.322016 16665 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:25.355553 16665 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:25.355976 16665 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:25.310276 16145 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:25.351227 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:25.362380 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:25.363377 12671 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:25.354444 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 617949us
I20260709 12:11:25.365881 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:25.367686 12675 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:25.370286 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:25.374065 13331 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:25.323277 16571 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:25.325345 16211 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:25.376278 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:25.389815 12605 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:25.321735 16472 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:25.316826 16606 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:25.350521 16619 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:25.383630 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:25.359764 16662 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:11:25.339186 16223 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election won for term 7
I20260709 12:11:25.282447 16517 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:25.268415 16656 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:25.263612 16126 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:25.388196 16657 consensus_queue.cc:1243] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:11:25.350558 16664 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:25.372648 16630 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:25.384707 16182 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:25.288679 16654 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:25.369889 16334 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:25.360817 16421 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:25.421559 12599 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 8: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:11:25.370471 16651 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 12:11:25.442024 16281 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:25.389220 16211 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:25.457052 12531 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:25.373255 16561 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:25.435675 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 43806.4us
I20260709 12:11:25.391762 16472 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:25.440191 16576 consensus_meta.cc:287] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.023s	user 0.018s	sys 0.002s
I20260709 12:11:25.470492 12604 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:25.413081 16540 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:25.401479 13333 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
W20260709 12:11:25.420063 16660 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.113s	user 0.000s	sys 0.001s
I20260709 12:11:25.400065 16606 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:25.443490 12671 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 10.
I20260709 12:11:25.456512 16334 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:25.461766 12598 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:11:25.461966 16211 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:25.463279 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2119) took 1067 ms. Trace:
I20260709 12:11:25.665814 12601 rpcz_store.cc:276] 0709 12:11:24.395525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:24.603120 (+207595us) service_pool.cc:224] Handling call
0709 12:11:24.659529 (+ 56409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:25.189413 (+529884us) raft_consensus.cc:1534] Early marking committed up to index 3
0709 12:11:25.266279 (+ 76866us) log.cc:844] Serialized 14 byte log entry
0709 12:11:25.393879 (+127600us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:25.446786 (+ 52907us) log.cc:844] Serialized 26 byte log entry
0709 12:11:25.446965 (+   179us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:25.447208 (+   243us) log.cc:844] Serialized 14 byte log entry
0709 12:11:25.447400 (+   192us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:25.447474 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:25.463022 (+ 15548us) raft_consensus.cc:1685] finished
0709 12:11:25.463047 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:25.463207 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21238,"prepare.queue_time_us":89712,"prepare.run_cpu_time_us":133,"prepare.run_wall_time_us":1875,"raft.queue_time_us":94049,"raft.run_cpu_time_us":395,"raft.run_wall_time_us":18272,"thread_start_us":575103,"threads_started":3,"wal-append.queue_time_us":178135}
I20260709 12:11:25.463872 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:25.667872 12522 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:25.464538 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:25.392299 16461 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.061s
W20260709 12:11:25.482847 12496 connection.cc:256] RPC call timeout handler was delayed by -1.0302431840000281s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18230000000000002s for a response
I20260709 12:11:25.474946 16472 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:25.425746 16619 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:25.414952 16508 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:25.655390 16539 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:25.656983 16661 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.315s	user 0.001s	sys 0.000s
I20260709 12:11:25.657481 16421 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:25.451732 16533 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:11:25.674578 16668 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:25.650816 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:25.414958 16145 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:25.717602 12673 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
W20260709 12:11:25.711933 16629 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.617s	user 0.000s	sys 0.010s
I20260709 12:11:25.724268 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:25.725332 12524 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:25.727540 16664 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:25.730067 16619 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:25.734669 16629 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:25.735960 16546 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:25.737177 16629 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:25.737751 16654 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:25.738886 16618 raft_consensus.cc:3055] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:25.739315 16618 raft_consensus.cc:740] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:25.740168 16618 consensus_queue.cc:260] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:25.689383 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 320362us
I20260709 12:11:25.742475 16618 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:25.743258 16540 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:25.743392 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:25.663364 16606 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:25.663753 16459 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:25.675750 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:25.749301 16182 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:25.752123 16670 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:25.752795 16670 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:25.663115 16560 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:25.685364 16562 consensus_meta.cc:287] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.559s	user 0.003s	sys 0.009s
I20260709 12:11:25.756134 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:25.756932 12671 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:11:25.735255 16656 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:25.758358 12522 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:25.761212 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:25.689216 16223 raft_consensus.cc:697] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:25.759300 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:25.759930 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:25.699144 16591 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:25.766094 12572 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:25.767956 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:25.769590 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:25.772432 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:25.781165 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:25.775866 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:25.773917 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:25.782872 12599 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:25.776155 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:25.784641 12605 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:25.776559 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:25.727578 16630 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:25.797782 12597 raft_consensus.cc:2410] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:11:25.756063 16527 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:11:25.802661 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:25.803335 12606 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:25.719986 16539 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:11:25.782166 16591 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:25.780405 16421 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:25.773676 16681 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:25.825590 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:25.828125 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 78767.9us
I20260709 12:11:25.780190 16560 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:25.774180 16334 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:25.851199 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:25.861660 12675 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:25.717602 16668 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:25.810998 16680 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:25.858186 12417 maintenance_manager.cc:643] P d5d8248387cf49cfbd3416fcacedae86: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 10.514s	user 0.200s	sys 0.087s Metrics: {"bytes_written":30694,"cfile_init":1,"delete_count":0,"dirs.queue_time_us":457821,"dirs.run_cpu_time_us":476,"dirs.run_wall_time_us":1953,"drs_written":1,"lbm_read_time_us":79126,"lbm_reads_1-10_ms":1,"lbm_reads_10-100_ms":2,"lbm_reads_lt_1ms":1,"lbm_write_time_us":322199,"lbm_writes_10-100_ms":10,"lbm_writes_lt_1ms":23,"peak_mem_usage":0,"reinsert_count":0,"rows_written":63,"thread_start_us":319547,"threads_started":2,"update_count":33,"wal-append.queue_time_us":92819}
I20260709 12:11:25.799216 16576 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:25.763377 16562 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:25.869663 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:25.872558 12675 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:25.871438 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:25.828801 16591 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:25.861142 16560 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:25.862038 16410 consensus_meta.cc:287] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.750s	user 0.012s	sys 0.005s
I20260709 12:11:25.878579 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:25.878837 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2006) took 1648 ms. Trace:
I20260709 12:11:25.891027 12523 rpcz_store.cc:276] 0709 12:11:24.230475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:24.233696 (+  3221us) service_pool.cc:224] Handling call
0709 12:11:24.234101 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:24.462972 (+228871us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:24.463025 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:24.463092 (+    67us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:24.463167 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:25.878531 (+1415364us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b540038f9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:25.878592 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:25.878757 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":525497600}
I20260709 12:11:25.888098 12643 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:25.884829 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:25.875224 16561 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:25.850518 16640 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:25.856837 16421 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:25.871925 16680 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:25.903636 16680 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:25.904194 16561 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:25.904008 16421 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:25.904632 16561 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:25.905089 16561 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:25.883401 12599 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:25.892513 16334 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:25.898905 12571 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:25.887120 16591 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:25.860623 16681 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:25.912233 16673 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:25.913130 16681 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:11:25.883669 16576 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:25.913724 16546 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:25.914366 12671 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:25.914453 16576 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:25.915058 16334 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:25.915761 16334 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:25.902027 16508 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:25.913043 16673 consensus_queue.cc:1059] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:11:25.889981 16560 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:25.902948 16670 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:25.928187 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:25.948644 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:25.949393 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:25.949638 12523 raft_consensus.cc:2410] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260709 12:11:25.951977 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:25.906632 16182 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:25.959385 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:25.960772 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:25.961032 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:25.865741 16223 consensus_queue.cc:237] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:25.961606 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:25.965020 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:26.032696 12418 maintenance_manager.cc:419] P d5d8248387cf49cfbd3416fcacedae86: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 7543 bytes on disk
I20260709 12:11:26.001969 12523 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 12: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:26.035341 12522 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:26.036329 12606 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:25.955149 16562 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:25.909389 16640 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:26.048023 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:26.053064 12527 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:11:26.056032 12524 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:26.107898 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:26.043944 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 132194us
I20260709 12:11:26.091611 16671 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:11:26.041770 16650 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
W20260709 12:11:26.078558 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:26.083369 16668 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:26.092249 16665 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.736s	user 0.011s	sys 0.000s
I20260709 12:11:26.053472 12417 maintenance_manager.cc:643] P d5d8248387cf49cfbd3416fcacedae86: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.003s	user 0.001s	sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":503,"lbm_reads_lt_1ms":4}
I20260709 12:11:26.110996 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:26.129904 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:26.129586 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:26.129578 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:26.133613 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:26.149627 16688 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:26.114184 16562 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:26.154366 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:26.152848 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:26.115115 16671 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:11:26.156777 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 268796us
I20260709 12:11:26.161176 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:26.161913 12671 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:11:26.163568 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:26.165637 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:26.136639 16593 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:26.168507 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:26.169163 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:26.187980 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:26.189041 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:26.157488 16668 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:26.192055 12524 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 8: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:26.133034 16665 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:26.164002 16606 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:26.198918 16472 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:26.199913 16665 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:26.200028 16606 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:26.201073 12603 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 12:11:26.201342 16472 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:26.202126 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:26.202374 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:26.208406 12571 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:26.196906 16591 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:26.131433 16680 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:26.169440 16618 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:26.185263 16182 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:26.155426 16211 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:26.215292 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:26.206485 16571 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:26.222903 12606 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 10. Ops: []
I20260709 12:11:26.207093 16597 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:26.161382 16562 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:26.158403 16688 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:26.225972 12671 raft_consensus.cc:2368] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 8. Current term is 9.
I20260709 12:11:26.219076 16593 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:26.215868 16618 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:26.226554 16597 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:26.230545 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:26.230638 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:26.236506 12571 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:26.233600 16618 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:26.239840 16462 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:26.241634 16421 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:26.236244 16571 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260709 12:11:26.246251 12530 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:11:26.244256 16562 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:26.250474 16680 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:26.256606 12571 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:26.228075 16472 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:26.227239 16182 raft_consensus.cc:697] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:26.263128 13339 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
I20260709 12:11:26.261636 12524 raft_consensus.cc:3055] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:26.266520 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:26.269064 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:26.269938 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:26.270653 12530 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:26.263813 12524 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:26.268693 12571 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:26.320463 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:26.285367 13332 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
W20260709 12:11:26.326608 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 57981.4us
I20260709 12:11:26.340536 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:26.340745 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:26.285404 16593 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:26.341347 16527 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:26.321669 12524 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:26.342407 16665 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:26.342694 12524 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:26.342856 16334 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:26.344641 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:26.344914 16591 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:11:26.345306 16334 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:26.229818 16671 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:26.257350 16559 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:26.256335 16668 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:26.283903 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:26.279031 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:26.346002 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:26.286403 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:26.358150 13333 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:26.358156 12606 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:26.359136 12530 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:26.273607 16687 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:11:26.269512 16472 raft_consensus.cc:697] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:26.360132 16619 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:26.360618 16510 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:26.363750 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:26.373461 12527 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:11:26.281208 16211 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:26.342927 16665 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:26.296474 13335 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:26.352897 12571 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:26.381418 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:26.381421 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:26.346216 16640 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:26.269028 16571 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:26.357954 16591 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:26.382442 12606 raft_consensus.cc:2410] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 8 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:11:26.383009 13332 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:26.397892 12524 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:11:26.398949 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2022) took 1169 ms. Trace:
I20260709 12:11:26.399335 12531 rpcz_store.cc:276] 0709 12:11:25.229389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:25.229506 (+   117us) service_pool.cc:224] Handling call
0709 12:11:25.252262 (+ 22756us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:25.457519 (+205257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:26.398698 (+941179us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b5400054e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:26.398885 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":470133504}
I20260709 12:11:26.401108 13333 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:26.275134 16688 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:26.382921 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:26.384449 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:26.380656 16510 raft_consensus.cc:697] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:26.358896 16559 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:26.286767 16223 consensus_queue.cc:1059] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.098s
I20260709 12:11:26.388402 16606 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:26.388111 16560 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:26.390820 16561 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:26.391892 13335 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:11:26.407249 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:26.424126 12606 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:26.425292 13335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 409) took 1416 ms. Trace:
I20260709 12:11:26.357790 16597 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:26.408782 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:26.394693 16668 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:26.395171 16553 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:26.397918 16591 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:26.376801 16619 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:26.414569 12646 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:26.414769 16510 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:26.415550 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:26.425648 13335 rpcz_store.cc:276] 0709 12:11:25.009160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:25.009734 (+   574us) service_pool.cc:224] Handling call
0709 12:11:25.010098 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:26.295879 (+1285781us) spinlock_profiling.cc:243] Waited 32.9 ms on lock 0x7b5800538dd0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f148897c0aa 00007f14889fe447 00007f1488a16fe7 00007f1488a187de 00007f1488a22b49 00007f1488a23416 00007f1488a1fd8f 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4
0709 12:11:26.424956 (+129077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:26.425215 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76296320}
I20260709 12:11:26.382820 16665 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:26.417326 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:26.441625 12606 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:26.444970 12680 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:26.445331 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:26.446202 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:26.446476 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:11:26.428990 16635 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:11:26.438980 12646 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:26.418370 16688 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:26.450809 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:26.452334 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2374) took 2117 ms. Trace:
I20260709 12:11:26.454511 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:26.455140 12678 rpcz_store.cc:276] 0709 12:11:24.334759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:25.024949 (+690190us) service_pool.cc:224] Handling call
0709 12:11:25.067803 (+ 42854us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:25.284604 (+216801us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:26.452026 (+1167422us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54003d40d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:26.452262 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496998144}
I20260709 12:11:26.455742 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:26.421715 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 114727us
I20260709 12:11:26.457580 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:26.457607 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:26.458281 12675 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:26.454488 12673 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:26.458400 13305 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:26.461014 12646 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:26.462699 12524 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:26.439961 16591 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:26.473196 12524 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:26.474409 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:26.475164 12524 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:26.397485 16182 consensus_queue.cc:237] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:26.473495 16597 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:26.461994 16593 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:26.482743 12524 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:26.453320 16211 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:26.452342 16536 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:26.426846 16606 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:26.462446 16527 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:26.462733 16334 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:26.485739 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:26.475315 16517 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:26.472429 16540 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:26.440832 16553 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:26.507695 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:26.528287 13335 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:11:26.511907 12571 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:26.522271 12524 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:11:26.459614 16688 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:26.492018 16211 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:26.531214 13332 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260709 12:11:26.535946 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:26.548087 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2128) took 1205 ms. Trace:
I20260709 12:11:26.548588 12604 rpcz_store.cc:276] 0709 12:11:25.342819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:25.343344 (+   525us) service_pool.cc:224] Handling call
0709 12:11:26.547847 (+1204503us) spinlock_profiling.cc:243] Waited 99.3 ms on lock 0x7b5400376254. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:26.547980 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":228351872}
I20260709 12:11:26.535589 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:25.029726 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:26.553973 13331 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 8: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:26.470691 16472 consensus_queue.cc:237] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:26.559206 12427 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47524 (request call id 97) took 7352 ms. Trace:
I20260709 12:11:26.564525 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:26.492511 16593 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:26.567878 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:26.570399 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 112078us
I20260709 12:11:26.531787 16688 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:26.537101 16211 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:26.539526 16640 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:26.525869 16334 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:11:26.520267 16527 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:26.559904 12427 rpcz_store.cc:276] 0709 12:11:19.201110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:19.202000 (+   890us) service_pool.cc:224] Handling call
0709 12:11:26.553114 (+7351114us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:11:20.726646 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:11:20.744059 (+ 17413us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:11:20.744137 (+    78us) write_op.cc:435] Acquired schema lock
0709 12:11:20.744186 (+    49us) tablet.cc:662] Decoding operations
0709 12:11:20.745787 (+  1601us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:11:20.745890 (+   103us) write_op.cc:641] Partition lock acquired for write op
0709 12:11:20.745939 (+    49us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:11:20.752454 (+  6515us) tablet.cc:701] Row locks acquired
0709 12:11:20.752521 (+    67us) write_op.cc:260] PREPARE: finished
0709 12:11:20.756150 (+  3629us) write_op.cc:270] Start()
0709 12:11:20.756387 (+   237us) write_op.cc:276] Timestamp: P: 1783599080752666 usec, L: 0
0709 12:11:20.756425 (+    38us) op_driver.cc:348] REPLICATION: starting
0709 12:11:20.757497 (+  1072us) log.cc:844] Serialized 4367 byte log entry
0709 12:11:21.565369 (+807872us) op_driver.cc:464] REPLICATION: finished
0709 12:11:21.959877 (+394508us) spinlock_profiling.cc:243] Waited 16.2 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14859160ac 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f148c456ef0
0709 12:11:22.255007 (+295130us) write_op.cc:301] APPLY: starting
0709 12:11:22.264359 (+  9352us) tablet.cc:1367] starting BulkCheckPresence
0709 12:11:22.302447 (+ 38088us) tablet.cc:1370] finished BulkCheckPresence
0709 12:11:22.302490 (+    43us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:11:22.632755 (+330265us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:11:22.633224 (+   469us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=22
0709 12:11:22.633305 (+    81us) write_op.cc:312] APPLY: finished
0709 12:11:22.633772 (+   467us) log.cc:844] Serialized 83 byte log entry
0709 12:11:23.892969 (+1259197us) spinlock_profiling.cc:243] Waited 4.39 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14859160ac 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b
0709 12:11:23.998279 (+105310us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:23.998458 (+   179us) write_op.cc:454] Released schema lock
0709 12:11:23.998518 (+    60us) write_op.cc:341] FINISH: Updating metrics
0709 12:11:23.999000 (+   482us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:23.999033 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":657705,"reported_tablets":13,"child_traces":[["op",{"apply.queue_time_us":650846,"apply.run_cpu_time_us":46848,"apply.run_wall_time_us":1758171,"num_ops":11,"prepare.queue_time_us":435351,"prepare.run_cpu_time_us":23298,"prepare.run_wall_time_us":212864,"raft.queue_time_us":488690,"raft.run_cpu_time_us":77301,"raft.run_wall_time_us":611310,"replication_time_us":808768,"spinlock_wait_cycles":47390208,"thread_start_us":2723329,"threads_started":5,"wal-append.queue_time_us":1614030,"wal-append.run_cpu_time_us":25644,"wal-append.run_wall_time_us":2317025}]]}
I20260709 12:11:26.606945 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:26.611660 12604 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:26.615484 16649 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 9 current term: 9 is higher.
I20260709 12:11:26.619706 12604 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 10.
I20260709 12:11:26.596788 16640 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:26.621690 16640 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:26.622200 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:26.623018 12604 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:26.623638 16606 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:26.624949 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:26.621124 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:26.626628 16561 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:26.631323 12604 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:26.635867 12604 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:11:26.578219 16517 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 8.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:11:26.641445 16655 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:26.644191 16655 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:26.644688 16655 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:26.651901 16575 raft_consensus.cc:3055] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:26.652313 16619 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:26.652576 16575 raft_consensus.cc:740] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:26.653851 16546 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:26.532895 16421 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:11:26.655318 16699 raft_consensus.cc:3055] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:26.672276 12675 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:26.675375 12530 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:26.679746 12571 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:26.677057 16699 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:26.682653 16699 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:26.684162 16699 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:26.685722 16561 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:26.686156 16561 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:26.686429 16559 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:26.687199 16593 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:26.687001 16561 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:26.688825 16553 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:26.554196 16618 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:26.692021 16618 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:26.697086 12556 heartbeater.cc:646] Failed to heartbeat to 127.11.13.190:46629 (1 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.11.13.190:46629 timed out after 15.000s (SENT)
I20260709 12:11:26.697798 16562 raft_consensus.cc:3055] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Stepping down as leader of term 7
W20260709 12:11:26.713588 12529 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 379) took 1707 ms (client timeout 1511 ms). Trace:
W20260709 12:11:26.714244 12529 rpcz_store.cc:269] 0709 12:11:25.005944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:25.006276 (+   332us) service_pool.cc:224] Handling call
0709 12:11:26.713197 (+1706921us) spinlock_profiling.cc:243] Waited 236 ms on lock 0x7b540038e5d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:26.713456 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":543702272}
I20260709 12:11:26.716411 12523 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:26.700711 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:26.730044 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:26.737032 12675 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:26.687063 16559 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:26.693444 16688 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:26.735001 16633 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 2.238s	user 0.035s	sys 0.065s
I20260709 12:11:26.746634 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:26.750702 12675 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:26.765219 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:26.765996 12675 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 12: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:26.726656 16421 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:26.654712 16546 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:26.708461 16562 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:26.767041 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:26.770591 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:26.771539 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 283655us
I20260709 12:11:26.774784 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:26.773873 12646 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:26.767277 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 189003us
I20260709 12:11:26.730969 16540 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:26.735432 16571 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:26.769088 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:26.783124 16704 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:26.783763 16704 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:26.785437 16593 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:26.787940 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:26.654091 16575 consensus_queue.cc:260] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:26.790743 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:26.792704 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:26.805832 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:26.800134 12569 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:26.776868 16701 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:26.827584 12680 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:26.875696 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:26.876677 12680 raft_consensus.cc:2376] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8. Re-sending same reply.
W20260709 12:11:26.828089 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 54304.8us
I20260709 12:11:26.830729 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:26.878763 12529 raft_consensus.cc:2368] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 9. Current term is 10.
W20260709 12:11:26.857193 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 57214us
I20260709 12:11:26.880574 12646 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:26.877777 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:26.882045 12680 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:26.786064 16704 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:26.885588 12606 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:26.888949 12646 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:26.779644 16421 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:26.890348 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:26.807369 16211 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:26.891238 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:26.771662 16665 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:26.890051 13305 leader_election.cc:400] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 9. Current term is 10.
I20260709 12:11:26.807331 16618 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:26.897500 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2000) took 3060 ms. Trace:
I20260709 12:11:26.827018 16540 raft_consensus.cc:697] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:26.894610 12569 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:26.826893 16703 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:26.903031 12680 raft_consensus.cc:2376] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11. Re-sending same reply.
I20260709 12:11:26.884891 16571 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:11:26.908778 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:26.927618 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 430346us
W20260709 12:11:26.934619 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:26.935091 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:26.935144 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:26.896623 16510 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:26.791045 16640 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:26.791226 16593 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:26.894722 16547 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:26.900671 16575 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:26.900660 16223 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:26.931087 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:26.947649 12604 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:26.948647 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:26.951793 12494 connection.cc:256] RPC call timeout handler was delayed by -1.8701712509998742s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1787s for a response
I20260709 12:11:26.953586 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:26.901839 16702 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:26.962634 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:26.961689 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:26.963469 13331 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:26.924688 12677 rpcz_store.cc:276] 0709 12:11:23.837411 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:24.139360 (+301949us) service_pool.cc:224] Handling call
0709 12:11:24.139732 (+   372us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:24.160131 (+ 20399us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:11:24.160453 (+   322us) log.cc:844] Serialized 14 byte log entry
0709 12:11:24.339949 (+179496us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:24.375604 (+ 35655us) log.cc:844] Serialized 52 byte log entry
0709 12:11:24.375811 (+   207us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:24.376073 (+   262us) log.cc:844] Serialized 14 byte log entry
0709 12:11:24.807057 (+430984us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b180045c280. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f
0709 12:11:24.808500 (+  1443us) spinlock_profiling.cc:243] Waited 14.5 ms on lock 0x7b4800060858. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:11:26.765746 (+1957246us) spinlock_profiling.cc:243] Waited 30.1 ms on lock 0x7b180045c280. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf 00007f1488a3a038
0709 12:11:26.766034 (+   288us) log.cc:844] Serialized 14 byte log entry
0709 12:11:26.897031 (+130997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:26.897176 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:26.897225 (+    49us) raft_consensus.cc:1685] finished
0709 12:11:26.897237 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:26.897383 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2363849,"prepare.queue_time_us":2401666,"prepare.run_cpu_time_us":117,"prepare.run_wall_time_us":123,"raft.queue_time_us":43546,"raft.run_cpu_time_us":4295,"raft.run_wall_time_us":439846,"spinlock_wait_cycles":141618176,"thread_start_us":303719,"threads_started":2,"wal-append.queue_time_us":273349,"wal-append.run_cpu_time_us":3654,"wal-append.run_wall_time_us":1235157}
I20260709 12:11:27.012888 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 1999) took 3183 ms. Trace:
W20260709 12:11:27.013890 12494 connection.cc:256] RPC call timeout handler was delayed by -1.6656869519999873s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17880000000000001s for a response
I20260709 12:11:27.014863 13337 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 9: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:27.013922 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:11:27.018791 12494 connection.cc:256] RPC call timeout handler was delayed by -1.5119285069999933s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15800000000000003s for a response
I20260709 12:11:27.019083 12673 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:27.019207 12494 connection.cc:256] RPC call timeout handler was delayed by -1.470042365999916s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16060000000000002s for a response
W20260709 12:11:27.019542 12494 connection.cc:256] RPC call timeout handler was delayed by -1.4639957419999519s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1759s for a response
W20260709 12:11:27.019821 12494 connection.cc:256] RPC call timeout handler was delayed by -1.4354903620001096s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.168s for a response
W20260709 12:11:27.020177 12494 connection.cc:256] RPC call timeout handler was delayed by -1.389727417000131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2183s for a response
W20260709 12:11:27.020537 12494 connection.cc:256] RPC call timeout handler was delayed by -1.1401631450000878s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1985s for a response
W20260709 12:11:27.020965 12494 connection.cc:256] RPC call timeout handler was delayed by -1.014228507000098s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21330000000000002s for a response
I20260709 12:11:26.886471 16562 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:26.906991 16596 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.079s
I20260709 12:11:27.036949 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:26.899803 16559 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:11:27.023469 16695 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 12:11:27.024443 12679 rpcz_store.cc:276] 0709 12:11:23.829733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:24.138772 (+309039us) service_pool.cc:224] Handling call
0709 12:11:24.183142 (+ 44370us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:24.192158 (+  9016us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:11:24.205359 (+ 13201us) log.cc:844] Serialized 14 byte log entry
0709 12:11:24.223669 (+ 18310us) spinlock_profiling.cc:243] Waited 3.42 ms on lock 0x7b4800060858. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:11:24.489017 (+265348us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:24.489534 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 12:11:24.489673 (+   139us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:26.942486 (+2452813us) log.cc:844] Serialized 14 byte log entry
0709 12:11:27.012108 (+ 69622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.012218 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:27.012523 (+   305us) raft_consensus.cc:1685] finished
0709 12:11:27.012571 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:27.012796 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2280664,"prepare.queue_time_us":2314510,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":98,"raft.queue_time_us":9975,"raft.run_cpu_time_us":1259,"raft.run_wall_time_us":138446,"spinlock_wait_cycles":7868672,"thread_start_us":464901,"threads_started":3,"wal-append.queue_time_us":670847,"wal-append.run_cpu_time_us":2972,"wal-append.run_wall_time_us":1047124}
W20260709 12:11:27.041931 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 147274us
W20260709 12:11:27.031683 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:27.039439 13335 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:11:27.042899 12646 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:26.946727 16686 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.079s
I20260709 12:11:26.936060 16618 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.044116 16702 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:11:27.054513 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:27.055249 12529 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:27.033799 16701 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:11:27.044998 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 82827.6us
I20260709 12:11:27.045130 16510 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:11:27.064215 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:27.069492 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:27.036513 16649 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:27.078438 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:26.974030 16547 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:27.023187 16597 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 10.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:11:27.077426 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:27.024402 16410 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.042s
I20260709 12:11:27.047554 16619 raft_consensus.cc:2764] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:11:27.090937 12529 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:27.091584 12527 raft_consensus.cc:3055] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:27.097438 12527 raft_consensus.cc:740] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:27.097623 12679 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260709 12:11:27.049057 16562 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:27.088807 12530 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 12: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:27.098327 12527 consensus_queue.cc:260] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.067129 16697 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:27.056241 16718 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:27.034082 16421 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:27.120460 16640 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:27.120640 16410 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:27.107234 16706 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:27.123005 16706 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.124691 16547 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:27.150818 16618 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:27.125509 16706 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:27.126515 12679 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:11:27.123065 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:27.155565 12524 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:27.115839 16619 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.162957 12679 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:27.158479 16619 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.169493 12679 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:27.164848 12524 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
W20260709 12:11:27.127460 16711 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.085s	user 0.000s	sys 0.001s
W20260709 12:11:27.128049 16707 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.189s	user 0.000s	sys 0.001s
I20260709 12:11:27.127831 16720 consensus_queue.cc:1059] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:11:27.133497 16708 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.171s	user 0.000s	sys 0.000s
I20260709 12:11:27.135720 12673 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
W20260709 12:11:27.174212 16655 consensus_meta.cc:287] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.509s	user 0.017s	sys 0.000s
I20260709 12:11:27.056447 16540 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:27.145749 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:27.147166 16712 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
W20260709 12:11:27.147982 16698 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.256s	user 0.000s	sys 0.002s
I20260709 12:11:27.125398 12672 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:27.103009 16571 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:27.164285 16596 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.194339 12679 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:27.195604 16619 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:27.202481 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:27.108863 16652 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.098s
I20260709 12:11:27.158515 16701 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.115686 16697 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.158619 16559 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:27.213148 13331 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:27.093304 16649 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.222223 12673 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: []
W20260709 12:11:27.230474 16699 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.546s	user 0.012s	sys 0.000s
I20260709 12:11:27.164180 16671 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:27.225512 12528 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:11:27.227851 12678 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 12. Ops: []
W20260709 12:11:27.235252 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 168316us
I20260709 12:11:27.237519 12679 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:11:27.214633 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 190543us
I20260709 12:11:27.243428 12671 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 12:11:27.243881 12671 pending_rounds.cc:85] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 7
I20260709 12:11:27.244122 12671 pending_rounds.cc:107] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:11:27.244396 12671 raft_consensus.cc:2889] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:27.246891 16729 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:27.248113 16697 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:27.249495 16729 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 10.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:27.254048 16719 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 8.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:27.255467 16726 raft_consensus.cc:3055] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:27.256012 16726 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:27.257144 16726 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.258483 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2129) took 1915 ms. Trace:
I20260709 12:11:27.258927 12603 rpcz_store.cc:276] 0709 12:11:25.343150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:25.345926 (+  2776us) service_pool.cc:224] Handling call
0709 12:11:25.346314 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:26.201599 (+855285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.258103 (+1056504us) spinlock_profiling.cc:243] Waited 854 ms on lock 0x7b54001171d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:27.258403 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1965207808}
I20260709 12:11:27.261430 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:27.262719 16571 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:27.266491 16485 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:27.269543 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2030) took 2011 ms. Trace:
I20260709 12:11:27.258644 16726 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:27.270062 12526 rpcz_store.cc:276] 0709 12:11:25.258618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:25.258740 (+   122us) service_pool.cc:224] Handling call
0709 12:11:27.269300 (+2010560us) spinlock_profiling.cc:243] Waited 786 ms on lock 0x7b540027fbd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:27.269467 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1807932672}
I20260709 12:11:27.274398 16649 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:27.277226 12601 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:11:27.278218 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2283) took 1145 ms. Trace:
I20260709 12:11:27.278541 12601 rpcz_store.cc:276] 0709 12:11:26.132665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.132808 (+   143us) service_pool.cc:224] Handling call
0709 12:11:27.276642 (+1143834us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:11:27.276921 (+   279us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:27.277805 (+   884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.278075 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2716928}
W20260709 12:11:27.282747 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:27.284178 12496 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:27.285766 12496 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:27.228247 16655 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.288993 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:27.289709 16485 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:27.290146 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:27.236092 16596 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.293892 13332 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: [7.9-7.9]
W20260709 12:11:27.294503 16676 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.529s	user 0.010s	sys 0.012s
I20260709 12:11:27.296236 16655 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:27.293881 16591 consensus_meta.cc:287] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.808s	user 0.007s	sys 0.007s
I20260709 12:11:27.302795 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:27.288240 16680 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:27.291386 16649 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:27.319195 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:27.322759 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:27.326788 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:27.316641 16485 raft_consensus.cc:697] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:27.328563 12601 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 9: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:27.331612 13339 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:27.332064 16723 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.332599 16723 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.333245 16485 consensus_queue.cc:237] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.335132 16723 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:27.319710 16591 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:27.339221 12603 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 14: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:27.340415 16591 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:27.341866 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 354) took 2078 ms. Trace:
I20260709 12:11:27.343782 16596 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:27.342351 12602 rpcz_store.cc:276] 0709 12:11:25.263359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:25.263515 (+   156us) service_pool.cc:224] Handling call
0709 12:11:25.279987 (+ 16472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:25.280437 (+   450us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:25.280465 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:25.280984 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 12:11:25.453895 (+172911us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:11:25.767447 (+313552us) log.cc:844] Serialized 14 byte log entry
0709 12:11:25.799970 (+ 32523us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:25.800048 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:27.250030 (+1449982us) spinlock_profiling.cc:243] Waited 6.89 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:27.341535 (+ 91505us) raft_consensus.cc:1685] finished
0709 12:11:27.341648 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:27.341795 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30791,"prepare.queue_time_us":20385,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":32164,"raft.run_cpu_time_us":265,"raft.run_wall_time_us":11391,"spinlock_wait_cycles":15856512,"thread_start_us":476932,"threads_started":2,"wal-append.queue_time_us":429894}
I20260709 12:11:27.322638 16680 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:27.345232 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:27.345988 12602 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 11: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:27.346693 16680 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:27.347738 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:27.348412 16546 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:27.350039 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:27.351789 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:27.352715 12602 raft_consensus.cc:2410] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:11:27.353389 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:27.354321 12494 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:27.321530 16649 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:27.361753 12494 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:27.324700 16536 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:27.356181 12601 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:27.367733 13333 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: []
W20260709 12:11:27.369596 16514 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:11:27.370987 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:27.371853 13337 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:27.374542 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:27.390106 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2291) took 1239 ms. Trace:
I20260709 12:11:27.363685 16649 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:27.414728 12601 rpcz_store.cc:276] 0709 12:11:26.150999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.383073 (+1232074us) service_pool.cc:224] Handling call
0709 12:11:27.389330 (+  6257us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.389596 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:27.389635 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:27.389703 (+    68us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:27.389777 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.389867 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:27.390025 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.387621 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:27.393693 16728 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:27.384994 16462 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:27.419276 16728 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:27.419834 16728 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:27.406349 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2122) took 3009 ms. Trace:
I20260709 12:11:27.424155 12598 rpcz_store.cc:276] 0709 12:11:24.396888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:24.659881 (+262993us) service_pool.cc:224] Handling call
0709 12:11:24.729945 (+ 70064us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:25.663440 (+933495us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.405980 (+1742540us) spinlock_profiling.cc:243] Waited 731 ms on lock 0x7b54002715d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:27.406275 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1682313472}
I20260709 12:11:27.425321 16462 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:27.425740 12598 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
W20260709 12:11:27.426276 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:27.426537 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2293) took 1259 ms. Trace:
I20260709 12:11:27.426901 12598 rpcz_store.cc:276] 0709 12:11:26.167442 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.425217 (+1257775us) service_pool.cc:224] Handling call
0709 12:11:27.425558 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.426249 (+   691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.426477 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.427452 16462 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:27.407727 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2289) took 1257 ms. Trace:
I20260709 12:11:27.428011 16462 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:27.428526 12604 rpcz_store.cc:276] 0709 12:11:26.150515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.676971 (+526456us) service_pool.cc:224] Handling call
0709 12:11:26.677329 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:26.959773 (+282444us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.407440 (+447667us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b5400271ad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:27.407652 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":621221248}
I20260709 12:11:27.429370 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:27.430192 12604 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:27.430287 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 2286) took 1291 ms. Trace:
I20260709 12:11:27.430958 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2295) took 1239 ms. Trace:
I20260709 12:11:27.430729 12600 rpcz_store.cc:276] 0709 12:11:26.138676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.138800 (+   124us) service_pool.cc:224] Handling call
0709 12:11:27.429181 (+1290381us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:11:27.430224 (+  1043us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":42247936}
I20260709 12:11:27.432893 12600 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:27.433866 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2296) took 1240 ms. Trace:
I20260709 12:11:27.434347 12600 rpcz_store.cc:276] 0709 12:11:26.193902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.432342 (+1238440us) service_pool.cc:224] Handling call
0709 12:11:27.432685 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.433407 (+   722us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.433779 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.435791 12600 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:27.436592 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2297) took 1222 ms. Trace:
I20260709 12:11:27.438295 12600 rpcz_store.cc:276] 0709 12:11:26.213903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.435203 (+1221300us) service_pool.cc:224] Handling call
0709 12:11:27.435596 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.436270 (+   674us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.436523 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.437252 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 2305) took 1174 ms. Trace:
I20260709 12:11:27.446419 12606 rpcz_store.cc:276] 0709 12:11:26.263175 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.485453 (+222278us) service_pool.cc:224] Handling call
0709 12:11:27.437007 (+951554us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b54002e6dd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:27.437158 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":658507008}
I20260709 12:11:27.443176 12603 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:27.450263 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2299) took 1230 ms. Trace:
I20260709 12:11:27.450721 12603 rpcz_store.cc:276] 0709 12:11:26.219933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.442448 (+1222515us) service_pool.cc:224] Handling call
0709 12:11:27.442883 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.449931 (+  7048us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.450190 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.451141 12600 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:27.451918 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2301) took 1220 ms. Trace:
I20260709 12:11:27.452327 12600 rpcz_store.cc:276] 0709 12:11:26.232016 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.450605 (+1218589us) service_pool.cc:224] Handling call
0709 12:11:27.450955 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.451652 (+   697us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.451861 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.431427 12604 rpcz_store.cc:276] 0709 12:11:26.191736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.429648 (+1237912us) service_pool.cc:224] Handling call
0709 12:11:27.429959 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.430721 (+   762us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.430905 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.458910 12602 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 7. Current term is 9. Ops: []
I20260709 12:11:27.460457 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2290) took 1309 ms. Trace:
W20260709 12:11:27.462303 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 94111.1us
I20260709 12:11:27.468003 12494 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:27.466676 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 356) took 1005 ms. Trace:
I20260709 12:11:27.467015 12600 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:27.445672 16688 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:27.366427 16671 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:11:27.430186 16725 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:11:27.486451 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:27.473533 12496 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:27.487594 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2309) took 1209 ms. Trace:
I20260709 12:11:27.488080 12600 rpcz_store.cc:276] 0709 12:11:26.277784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.466557 (+1188773us) service_pool.cc:224] Handling call
0709 12:11:27.466847 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.470927 (+  4080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.487529 (+ 16602us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.490185 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 359) took 1023 ms. Trace:
I20260709 12:11:27.490635 12600 rpcz_store.cc:276] 0709 12:11:26.467094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.489347 (+1022253us) service_pool.cc:224] Handling call
0709 12:11:27.489663 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.489847 (+   184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:27.489873 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:27.489916 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:27.489948 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.490001 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:27.490107 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.471614 13337 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:11:27.493111 12569 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:27.470417 12604 rpcz_store.cc:276] 0709 12:11:26.461279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.465712 (+1004433us) service_pool.cc:224] Handling call
0709 12:11:27.466023 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.466312 (+   289us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:27.466353 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:27.466400 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:27.466439 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.466501 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:27.466617 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.472133 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:27.506151 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:27.500129 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2310) took 1222 ms. Trace:
I20260709 12:11:27.506946 12604 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:11:27.507269 12601 rpcz_store.cc:276] 0709 12:11:26.278058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.465627 (+1187569us) service_pool.cc:224] Handling call
0709 12:11:27.499441 (+ 33814us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.499684 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:27.499721 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:27.499781 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:27.499840 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.499924 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:27.500053 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.470319 12602 rpcz_store.cc:276] 0709 12:11:26.150764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.366672 (+1215908us) service_pool.cc:224] Handling call
0709 12:11:27.458595 (+ 91923us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.459838 (+  1243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.460309 (+   471us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:27.470357 16703 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.503440 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2033) took 1053 ms. Trace:
I20260709 12:11:27.512843 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:27.469228 16726 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:27.513109 12674 rpcz_store.cc:276] 0709 12:11:26.445251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.448356 (+  3105us) service_pool.cc:224] Handling call
0709 12:11:26.448717 (+   361us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:26.449624 (+   907us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:26.449656 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:26.450208 (+   552us) log.cc:844] Serialized 26 byte log entry
0709 12:11:27.378314 (+928106us) spinlock_profiling.cc:243] Waited 3.01 ms on lock 0x7b18003379c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:11:27.496501 (+118187us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:27.496543 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.496627 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:27.498178 (+  1551us) raft_consensus.cc:1685] finished
0709 12:11:27.498199 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:27.498744 (+   545us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":455,"raft.queue_time_us":605,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"spinlock_wait_cycles":6911872,"thread_start_us":117943,"threads_started":1,"wal-append.queue_time_us":1046343}
W20260709 12:11:27.515167 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 59385.1us
I20260709 12:11:27.526427 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2308) took 1249 ms. Trace:
I20260709 12:11:27.499218 16719 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:27.445952 16728 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:27.528936 12603 rpcz_store.cc:276] 0709 12:11:26.277486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.465032 (+1187546us) service_pool.cc:224] Handling call
0709 12:11:27.465297 (+   265us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.465487 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 4
0709 12:11:27.495530 (+ 30043us) log.cc:844] Serialized 14 byte log entry
0709 12:11:27.522280 (+ 26750us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:27.522381 (+   101us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:27.522475 (+    94us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.522590 (+   115us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:27.525661 (+  3071us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":30716,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":90,"raft.queue_time_us":231,"thread_start_us":26006,"threads_started":1}
I20260709 12:11:27.503436 16546 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:27.542372 16719 raft_consensus.cc:697] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:27.542920 16732 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.079s
I20260709 12:11:27.544111 16719 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:27.514103 16527 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:27.550474 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 82559.2us
I20260709 12:11:27.527055 16536 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:27.495056 16514 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.040s
I20260709 12:11:27.526167 16726 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:27.558357 12525 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:27.532703 16697 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.566594 12494 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:27.548645 16696 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.010s	sys 0.015s
I20260709 12:11:27.546346 16546 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:27.500355 16462 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.527429 16703 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.565263 16726 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:27.595408 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 107769us
W20260709 12:11:27.581670 16696 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.604s	user 0.010s	sys 0.015s
I20260709 12:11:27.597702 13337 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:27.600381 12494 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:27.602074 12599 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:27.604103 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:27.605202 16744 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260709 12:11:27.606419 16546 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:27.564774 16671 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260709 12:11:27.609320 16671 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:27.610599 16223 consensus_queue.cc:1059] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:27.611814 16223 raft_consensus.cc:3055] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:27.612175 16223 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:27.612725 16703 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:27.613546 16703 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:27.614449 16540 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:27.614910 16680 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.615803 16680 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:27.597627 16514 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:27.594936 16667 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.154s
I20260709 12:11:27.619817 16667 consensus_queue.cc:1243] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 10.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:11:27.620925 16514 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:11:27.618487 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:27.593299 16536 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:27.622378 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:27.634442 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:27.633164 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:27.655836 12672 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:27.567303 16697 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:27.634977 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:27.631311 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:27.634951 12679 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:11:27.657111 12673 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:27.625854 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 386) took 1214 ms. Trace:
I20260709 12:11:27.628628 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:27.659041 12523 rpcz_store.cc:276] 0709 12:11:26.411407 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.411836 (+   429us) service_pool.cc:224] Handling call
0709 12:11:26.716022 (+304186us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:26.717583 (+  1561us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.625527 (+907944us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b540038e5d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:27.625672 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":698500992}
I20260709 12:11:27.629896 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:27.635334 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:27.660856 12680 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 11: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:11:27.653326 16739 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.155s	user 0.002s	sys 0.000s
W20260709 12:11:27.662746 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:27.655988 12678 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:27.666990 16736 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:11:27.615089 16540 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.669713 16540 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:27.613315 16223 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:27.670400 12673 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:11:27.641673 16649 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.624017 16514 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:27.672678 16514 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:27.673017 16649 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:27.673221 12677 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:27.654490 16536 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:27.641963 16462 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:27.667269 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:27.675753 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:27.676787 12523 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:27.674943 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2284) took 1537 ms. Trace:
I20260709 12:11:27.678664 12597 rpcz_store.cc:276] 0709 12:11:26.137122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.137264 (+   142us) service_pool.cc:224] Handling call
0709 12:11:27.268442 (+1131178us) spinlock_profiling.cc:243] Waited 19.8 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:11:27.268677 (+   235us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:27.291015 (+ 22338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:27.291057 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:27.309159 (+ 18102us) log.cc:844] Serialized 52 byte log entry
0709 12:11:27.463030 (+153871us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:27.463798 (+   768us) log.cc:844] Serialized 14 byte log entry
0709 12:11:27.469744 (+  5946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.469872 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:27.674463 (+204591us) raft_consensus.cc:1685] finished
0709 12:11:27.674630 (+   167us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:27.674854 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":34787,"prepare.queue_time_us":209,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":28656,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":22613,"spinlock_wait_cycles":45558016,"thread_start_us":148549,"threads_started":1,"wal-append.queue_time_us":236954}
I20260709 12:11:27.683111 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:27.578104 16732 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.076s
I20260709 12:11:27.681504 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:27.686771 13337 raft_consensus.cc:2410] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 10 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:11:27.636708 16419 raft_consensus.cc:3055] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:27.692780 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:27.640132 16728 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:27.621348 16680 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:27.708511 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2300) took 1481 ms. Trace:
I20260709 12:11:27.716229 12606 rpcz_store.cc:276] 0709 12:11:26.226896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.447583 (+1220687us) service_pool.cc:224] Handling call
0709 12:11:27.447998 (+   415us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:27.448437 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:27.448487 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:27.449132 (+   645us) log.cc:844] Serialized 52 byte log entry
0709 12:11:27.592281 (+143149us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:27.604787 (+ 12506us) log.cc:844] Serialized 14 byte log entry
0709 12:11:27.627955 (+ 23168us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.628868 (+   913us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:27.708181 (+ 79313us) raft_consensus.cc:1685] finished
0709 12:11:27.708212 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:27.708414 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":8956,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":16815,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":17512,"spinlock_wait_cycles":222464,"thread_start_us":142582,"threads_started":1,"wal-append.queue_time_us":192335}
I20260709 12:11:27.673017 16697 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:27.720974 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:27.721889 12672 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:27.728149 16653 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:27.698614 16419 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:27.729707 16653 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.730269 16419 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.731607 16419 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:27.733078 16653 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:27.709216 16536 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.737880 16536 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:27.694413 16593 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:27.742713 16211 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:27.665616 16703 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.750088 12605 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: [8.4-9.5]
I20260709 12:11:27.750932 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2285) took 1613 ms. Trace:
I20260709 12:11:27.755270 12678 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:27.751247 12605 rpcz_store.cc:276] 0709 12:11:26.137821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.137946 (+   125us) service_pool.cc:224] Handling call
0709 12:11:27.748869 (+1610923us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:11:27.749815 (+   946us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:27.750651 (+   836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.750873 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":43746176}
W20260709 12:11:27.756331 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 150049us
W20260709 12:11:27.759583 12496 leader_election.cc:336] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.703s (SENT)
I20260709 12:11:27.759958 12672 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 10 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:11:27.760187 12496 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:11:27.762549 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 158523us
I20260709 12:11:27.764801 16649 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:27.772673 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:27.761587 16728 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:11:27.779645 16553 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:27.785068 16553 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:27.769426 16732 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:11:27.778379 16753 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.013s
I20260709 12:11:27.789165 16665 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:27.790489 16703 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:27.791352 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:27.792125 12528 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:27.783851 16514 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.802057 12528 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:11:27.801373 12600 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:27.805614 12600 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:27.806655 16553 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:27.810926 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:27.810895 13305 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:27.818738 12600 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:27.811388 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2411) took 1046 ms. Trace:
I20260709 12:11:27.812256 12646 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:27.822006 12675 rpcz_store.cc:276] 0709 12:11:26.765146 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.768825 (+  3679us) service_pool.cc:224] Handling call
0709 12:11:27.811150 (+1042325us) spinlock_profiling.cc:243] Waited 527 ms on lock 0x7b54003d45d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:27.811316 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1211939072}
I20260709 12:11:27.823649 16606 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:27.824332 16606 raft_consensus.cc:697] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 9 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:27.813671 16726 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:27.814571 16752 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.215s	user 0.004s	sys 0.000s
I20260709 12:11:27.832036 12494 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:27.821725 12600 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:27.823740 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:27.834313 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:27.834615 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:27.835813 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:27.795660 16514 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:27.823776 16635 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.004s
I20260709 12:11:27.827539 16649 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:11:27.827994 16749 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:27.838040 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:27.846256 12530 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:27.843854 12646 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:27.838888 16334 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:27.847378 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:27.847470 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:27.858916 12677 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:27.825851 16703 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:27.846674 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:27.849988 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:27.863198 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:27.863577 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:27.865008 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:27.865556 12523 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:11:27.867944 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:27.868841 12673 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
I20260709 12:11:27.869594 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:27.871343 12528 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:27.871922 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:27.871382 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:27.872674 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:27.845394 16652 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 12:11:27.846067 16753 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:11:27.845942 16749 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:27.852262 16560 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:27.854069 16665 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.889846 16559 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.890259 12677 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:27.890434 16559 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:27.891415 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:27.892392 12677 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:27.892788 16559 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:27.893935 12602 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:27.889488 16749 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.895115 16485 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:27.895383 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:27.873718 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:27.875428 12604 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:27.895800 16749 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:27.861526 12530 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:27.878868 16740 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:27.879026 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:27.899876 12673 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:11:27.880354 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:27.901530 13339 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 12: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:27.880676 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:27.880602 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:27.902686 12605 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:27.903823 12605 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:27.903926 12606 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:11:27.883590 12676 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 9: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:27.884526 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:27.879302 16606 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:27.881922 12646 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:27.888397 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:27.889474 16665 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:27.845038 16472 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:27.895589 16485 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:27.896207 12602 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:27.897704 12597 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260709 12:11:27.861943 16334 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.905297 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 363) took 1003 ms. Trace:
I20260709 12:11:27.907783 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:27.875957 12496 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:27.908977 12523 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:27.911376 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:27.938699 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 104482us
I20260709 12:11:27.939455 12672 raft_consensus.cc:2410] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:11:27.940671 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:27.941265 12679 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 14: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:27.946326 12605 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:11:27.946383 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:27.947063 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:27.848064 16211 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:27.949421 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:27.952432 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 70614.6us
W20260709 12:11:28.003968 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 128078us
I20260709 12:11:28.005066 12494 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:28.005980 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:28.022681 12496 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:27.947590 16070 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:11:27.884065 16576 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:28.059424 12496 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:27.886931 16514 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:28.060987 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:28.061355 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:28.068538 12603 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260709 12:11:28.061584 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 56629.8us
I20260709 12:11:28.069721 12494 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:28.069113 12606 raft_consensus.cc:2376] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10. Re-sending same reply.
I20260709 12:11:28.068710 12604 raft_consensus.cc:2410] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 7.
I20260709 12:11:28.062479 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:28.063381 12677 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:28.072065 12597 raft_consensus.cc:2410] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 9 because replica has last-logged OpId of term: 7 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 3.
I20260709 12:11:28.066913 12530 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:28.073166 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:27.943030 16740 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:28.082083 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:28.082856 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:28.084748 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:28.058199 16729 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:28.085843 12604 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:27.945019 16462 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.156s
I20260709 12:11:28.133195 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:28.133744 13304 leader_election.cc:400] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 8. Current term is 9.
I20260709 12:11:28.134249 13304 leader_election.cc:403] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Cancelling election due to peer responding with higher term
I20260709 12:11:28.144117 12496 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:28.145349 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:28.145634 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:28.150578 12602 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:28.073956 16753 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:28.160028 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:28.080518 16740 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:28.146004 12597 raft_consensus.cc:2376] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11. Re-sending same reply.
I20260709 12:11:28.164851 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:28.004140 16472 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:28.058964 16211 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:28.059181 16334 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:28.173750 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:28.121052 16540 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:11:28.177325 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 231052us
I20260709 12:11:28.178853 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:28.184868 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:28.180969 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:28.127882 16760 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:28.128352 16559 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:28.130719 16728 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:28.139300 16514 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:28.135875 16703 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:28.191741 12678 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:28.192225 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:28.213393 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:28.195712 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:28.076982 16576 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:28.200572 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:28.207329 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 137684us
I20260709 12:11:28.146997 16697 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.056s
W20260709 12:11:28.238866 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 109214us
I20260709 12:11:28.207664 16759 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:28.203044 16593 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260709 12:11:28.244704 16561 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:28.247435 16759 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:28.254983 13333 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 579) took 1282 ms. Trace:
I20260709 12:11:28.255573 13333 rpcz_store.cc:276] 0709 12:11:26.972664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.973001 (+   337us) service_pool.cc:224] Handling call
0709 12:11:27.031112 (+ 58111us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.368190 (+337078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:28.254680 (+886490us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b54001261d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:28.254914 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":773636352}
I20260709 12:11:28.194295 16540 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:28.247495 12527 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:28.205272 16653 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:28.266464 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:28.183035 16715 consensus_queue.cc:1059] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:11:28.307327 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:28.312772 12601 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:28.235579 16728 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:28.330217 16649 consensus_queue.cc:1243] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 10.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:11:28.286180 16540 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:28.335996 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:28.211483 16559 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:28.164871 16485 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:28.220269 16762 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader election lost for term 8. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 8. Current term is 9.
I20260709 12:11:28.344660 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:28.345621 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:28.360162 12679 raft_consensus.cc:2410] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 5.
I20260709 12:11:27.999271 12600 rpcz_store.cc:276] 0709 12:11:26.902257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.491764 (+589507us) service_pool.cc:224] Handling call
0709 12:11:27.492093 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.801217 (+309124us) spinlock_profiling.cc:243] Waited 135 ms on lock 0x7b540002c220. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488997dc0 00007f1488a221c0 00007f1488a23231 00007f1488a1fd8f 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c
0709 12:11:27.901400 (+100183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:27.901434 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:27.901475 (+    41us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:27.901524 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.901970 (+   446us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:27.905237 (+  3267us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":254,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":11847,"spinlock_wait_cycles":309559552}
I20260709 12:11:28.261685 16766 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:28.346562 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:28.365036 12676 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:28.325301 12527 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260709 12:11:28.302464 16571 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:28.363443 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:28.365813 12527 pending_rounds.cc:85] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 4
W20260709 12:11:28.357473 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 144217us
I20260709 12:11:28.391996 12527 pending_rounds.cc:107] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260709 12:11:28.392334 12527 raft_consensus.cc:2889] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:28.392561 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:28.393272 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2067) took 1330 ms. Trace:
I20260709 12:11:28.315418 16726 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:28.366099 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 124186us
I20260709 12:11:28.398948 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:28.400559 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:28.393814 12527 rpcz_store.cc:276] 0709 12:11:27.063343 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.063489 (+   146us) service_pool.cc:224] Handling call
0709 12:11:27.091492 (+ 28003us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:28.246773 (+1155281us) spinlock_profiling.cc:243] Waited 113 ms on lock 0x7b58006479d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f148897c0aa 00007f14889fe447 00007f1488a16fe7 00007f1488a187de 00007f1488a22b49 00007f1488a23416 00007f1488a1fd8f 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4
0709 12:11:28.392906 (+146133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:28.393198 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":259431680}
I20260709 12:11:28.404969 13304 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:28.415733 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 157928us
I20260709 12:11:28.417114 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:28.419656 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:28.354849 16576 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:28.355641 16334 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:28.337881 16765 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:28.436718 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:28.436878 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:28.438594 12679 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 12: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:28.439917 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:28.442322 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:28.447238 12672 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:28.451128 12522 raft_consensus.cc:2410] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 10 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 12:11:28.457952 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:28.356652 16732 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 12:11:28.403980 16766 raft_consensus.cc:697] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:28.437813 16665 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:28.452786 16334 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260709 12:11:28.474761 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 472514us
I20260709 12:11:28.409360 16762 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 12:11:28.404109 16571 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:28.457053 16653 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:28.482049 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:28.472311 16764 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:28.412752 16703 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:28.497834 16769 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:28.482405 16334 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:28.471627 16774 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.034s
I20260709 12:11:28.446017 16740 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:28.462445 16728 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:28.494881 16472 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:28.499150 16211 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:28.500922 16769 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:28.541301 16785 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:28.498960 16762 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:28.524211 16546 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.914s	user 0.011s	sys 0.018s
W20260709 12:11:28.539979 16770 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:11:28.486498 16653 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:28.545951 16769 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:28.545471 16719 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.913s	user 0.015s	sys 0.026s
I20260709 12:11:28.541038 16514 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:28.543006 16715 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:28.560230 16719 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.928s	user 0.016s	sys 0.026s
I20260709 12:11:28.555275 16775 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:28.569340 16760 raft_consensus.cc:3055] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:28.541831 16766 consensus_queue.cc:237] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:28.574774 16775 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:28.586145 16749 consensus_meta.cc:287] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.689s	user 0.007s	sys 0.010s
I20260709 12:11:28.586813 16775 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:28.577493 16760 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:28.546769 16785 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:28.587918 16760 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:28.588768 16785 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:28.629869 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 235327us
I20260709 12:11:28.636086 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:28.642189 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:28.663686 12605 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:11:28.643641 12600 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:11:28.667953 12606 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: [8.9-9.10]
I20260709 12:11:28.668375 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:28.670143 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:28.671108 12600 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:28.661965 16798 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:28.665759 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:28.674774 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:28.639603 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:28.544625 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 139768us
I20260709 12:11:28.512102 16334 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:28.706586 16653 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:28.706370 16785 consensus_queue.cc:1059] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:28.694386 16765 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:28.695295 16571 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:28.716517 16740 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:28.693955 16798 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:28.732234 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 309758us
I20260709 12:11:28.734901 16796 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:28.735577 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:28.736179 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 393) took 1355 ms. Trace:
I20260709 12:11:28.736848 12525 rpcz_store.cc:276] 0709 12:11:27.380463 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.380695 (+   232us) service_pool.cc:224] Handling call
0709 12:11:27.391210 (+ 10515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.559051 (+167841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:28.735446 (+1176395us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b5400015754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:28.735839 (+   393us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":383574272}
I20260709 12:11:28.741073 16765 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:28.741403 12531 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:28.568893 16715 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:28.741921 12531 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:28.742220 16765 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:28.743615 12531 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:28.745105 16740 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:28.747514 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2282) took 2615 ms. Trace:
I20260709 12:11:28.747947 12599 rpcz_store.cc:276] 0709 12:11:26.132372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.133043 (+   671us) service_pool.cc:224] Handling call
0709 12:11:27.253145 (+1120102us) spinlock_profiling.cc:243] Waited 24.3 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:11:27.253462 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:27.603482 (+350020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:28.746937 (+1143455us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b54002f2954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:28.747301 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":856596608}
I20260709 12:11:28.745858 12531 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:28.558713 16546 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.945s	user 0.012s	sys 0.018s
I20260709 12:11:28.747232 16765 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:28.693877 16559 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:28.721953 16798 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:28.737725 16796 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:11:28.767120 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:28.768095 12677 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:28.771139 16789 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:28.769614 13304 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:28.781891 16559 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:28.782691 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:28.783480 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:28.734946 16794 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:28.786589 16789 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:28.788895 16789 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:28.745114 16715 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:28.745977 16740 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:28.800849 12672 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:28.822433 16740 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.4, Last appended by leader: 4, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:28.825443 16576 raft_consensus.cc:3055] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:28.826799 16715 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:28.827481 12496 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:28.829119 12600 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:28.830094 16749 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:28.810865 16794 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:28.834405 16630 raft_consensus.cc:3055] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:28.835075 16630 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:28.834604 16794 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:28.836640 16630 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:28.837320 12600 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:28.837541 16794 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:28.843133 13337 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 10 index: 9. (index mismatch)
I20260709 12:11:28.845258 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:28.846453 13333 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:28.846789 12600 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260709 12:11:28.850749 16796 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:28.854763 16749 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:28.856711 16663 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:28.857718 16663 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:28.858333 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 89155.5us
I20260709 12:11:28.859643 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:28.819800 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:28.862797 13304 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:28.864042 16715 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:28.866848 16553 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:11:28.876101 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 94890.5us
I20260709 12:11:28.857903 16749 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:28.826076 16576 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:28.874025 16715 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:28.894184 12528 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:28.897826 16576 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:28.900027 16576 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:28.906184 16663 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:28.906898 16802 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:28.907970 16802 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:28.909204 16801 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:28.908310 12572 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:28.927631 16713 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.915s	user 0.003s	sys 0.009s
I20260709 12:11:28.927018 16769 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:28.927389 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 44811.4us
I20260709 12:11:28.930075 16801 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:28.936272 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2432) took 1061 ms. Trace:
I20260709 12:11:28.934271 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:28.931178 12599 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
W20260709 12:11:28.956928 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 48937.6us
I20260709 12:11:28.894229 12569 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:28.886621 16796 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:28.936650 16801 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:28.933524 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:28.933832 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:28.966351 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 72485.3us
I20260709 12:11:28.930419 16794 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.036s
I20260709 12:11:28.931459 16546 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:28.927433 16658 raft_consensus.cc:3055] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:28.964186 16769 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:28.979393 16658 raft_consensus.cc:740] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:28.934721 16649 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:11:28.980293 16796 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:28.992458 16283 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:11:29.020687 16744 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.414s	user 0.003s	sys 0.003s
W20260709 12:11:29.021294 16744 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.415s	user 0.004s	sys 0.003s
I20260709 12:11:29.010095 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:28.937028 12678 rpcz_store.cc:276] 0709 12:11:27.875024 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:28.212757 (+337733us) service_pool.cc:224] Handling call
0709 12:11:28.267642 (+ 54885us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:28.299938 (+ 32296us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:11:28.355980 (+ 56042us) log.cc:844] Serialized 14 byte log entry
0709 12:11:28.739457 (+383477us) log.cc:844] Serialized 14 byte log entry
0709 12:11:28.741052 (+  1595us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:28.741998 (+   946us) log.cc:844] Serialized 26 byte log entry
0709 12:11:28.742193 (+   195us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:28.742226 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:28.742392 (+   166us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:28.845950 (+103558us) spinlock_profiling.cc:243] Waited 3 ms on lock 0x7b4800060858. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:11:28.935640 (+ 89690us) raft_consensus.cc:1685] finished
0709 12:11:28.935674 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:28.935938 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":112704,"prepare.queue_time_us":69730,"prepare.run_cpu_time_us":106,"prepare.run_wall_time_us":110,"raft.queue_time_us":283171,"raft.run_cpu_time_us":3748,"raft.run_wall_time_us":411734,"spinlock_wait_cycles":6904704,"thread_start_us":336268,"threads_started":1,"wal-append.queue_time_us":447709}
I20260709 12:11:29.027647 12572 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:29.032002 12496 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:29.035420 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 396045us
I20260709 12:11:29.022810 16283 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:29.042069 16561 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.796s	user 0.014s	sys 0.018s
I20260709 12:11:29.043264 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:29.044595 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:29.045603 16801 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:29.053980 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:29.054826 12523 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:29.056631 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:29.046602 16283 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:29.060504 12531 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 9. (term mismatch)
I20260709 12:11:29.064361 12531 pending_rounds.cc:85] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 8
W20260709 12:11:29.067458 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:29.062436 16653 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 12:11:29.070250 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2426) took 1200 ms. Trace:
I20260709 12:11:29.070734 12674 rpcz_store.cc:276] 0709 12:11:27.864500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.869530 (+  5030us) service_pool.cc:224] Handling call
0709 12:11:27.869971 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:27.951111 (+ 81140us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:27.951978 (+   867us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:28.785570 (+833592us) spinlock_profiling.cc:243] Waited 4 ms on lock 0x7b2c00037868. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:11:28.786333 (+   763us) log.cc:844] Serialized 26 byte log entry
0709 12:11:28.997944 (+211611us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:29.047695 (+ 49751us) log.cc:844] Serialized 14 byte log entry
0709 12:11:29.052565 (+  4870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:29.060296 (+  7731us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:29.060348 (+    52us) raft_consensus.cc:1685] finished
0709 12:11:29.064998 (+  4650us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:29.065111 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1880,"prepare.queue_time_us":20835,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":4838,"raft.queue_time_us":14837,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":9086,"spinlock_wait_cycles":9188736,"thread_start_us":291651,"threads_started":2,"wal-append.queue_time_us":228023}
I20260709 12:11:29.074020 12522 raft_consensus.cc:2410] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 8 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260709 12:11:29.075330 12523 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:29.075582 16760 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:29.066797 16472 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:29.081506 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2142) took 1207 ms. Trace:
I20260709 12:11:29.080838 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:29.081923 12601 rpcz_store.cc:276] 0709 12:11:27.874119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.874283 (+   164us) service_pool.cc:224] Handling call
0709 12:11:29.081102 (+1206819us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54002f1054. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:29.081305 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":494307200}
I20260709 12:11:29.089139 16802 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:29.089556 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:29.085970 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:29.092054 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:29.062450 16767 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 12:11:29.058575 16796 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.101253 16334 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.102860 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:29.103178 12679 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:29.105101 12643 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:29.106201 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:29.094081 16789 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:29.107075 16789 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:29.090596 16658 consensus_queue.cc:260] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.108002 16658 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:29.111514 16747 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.513s	user 0.003s	sys 0.000s
W20260709 12:11:29.111822 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 104669us
W20260709 12:11:29.112192 16763 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.752s	user 0.001s	sys 0.002s
W20260709 12:11:29.113922 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50482.4us
I20260709 12:11:29.114650 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2425) took 1250 ms. Trace:
I20260709 12:11:29.114961 12571 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:29.115782 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2070) took 1637 ms. Trace:
I20260709 12:11:29.116403 16211 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:29.116568 16472 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:29.117138 16211 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:29.117234 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:29.117700 16211 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:29.118854 16472 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:29.123373 16419 consensus_meta.cc:287] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.391s	user 0.014s	sys 0.015s
W20260709 12:11:29.123893 16540 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.726s	user 0.014s	sys 0.014s
I20260709 12:11:29.124810 12530 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:29.115208 12680 rpcz_store.cc:276] 0709 12:11:27.864116 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.867967 (+  3851us) service_pool.cc:224] Handling call
0709 12:11:27.868403 (+   436us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:27.869178 (+   775us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:27.869232 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:27.879977 (+ 10745us) log.cc:844] Serialized 52 byte log entry
0709 12:11:28.137072 (+257095us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:28.362391 (+225319us) log.cc:844] Serialized 14 byte log entry
0709 12:11:28.362780 (+   389us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:28.362910 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:29.111238 (+748328us) spinlock_profiling.cc:243] Waited 41 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:29.113961 (+  2723us) raft_consensus.cc:1685] finished
0709 12:11:29.114107 (+   146us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:29.114428 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30945,"prepare.queue_time_us":101594,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":76480,"raft.run_cpu_time_us":273,"raft.run_wall_time_us":15871,"spinlock_wait_cycles":94349568,"thread_start_us":422253,"threads_started":2,"wal-append.queue_time_us":375804}
W20260709 12:11:29.111505 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:29.126049 16796 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:29.126612 16744 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:29.124576 16767 consensus_queue.cc:1243] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 10.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:11:29.094014 12529 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:29.067606 12531 pending_rounds.cc:107] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260709 12:11:29.139734 12531 raft_consensus.cc:2889] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:29.129936 16744 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:29.148799 16744 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:29.116227 12524 rpcz_store.cc:276] 0709 12:11:27.477997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.478241 (+   244us) service_pool.cc:224] Handling call
0709 12:11:27.478709 (+   468us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:27.479145 (+   436us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:27.479185 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:27.479781 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 12:11:27.533190 (+ 53409us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:27.533253 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.533365 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:29.110399 (+1577034us) spinlock_profiling.cc:243] Waited 41.1 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:29.115243 (+  4844us) raft_consensus.cc:1685] finished
0709 12:11:29.115267 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:29.115568 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7223,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":12881,"spinlock_wait_cycles":94638848,"thread_start_us":16709,"threads_started":1,"wal-append.queue_time_us":61146}
W20260709 12:11:29.153548 16729 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.011s	user 0.015s	sys 0.011s
I20260709 12:11:29.113684 16334 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:29.154167 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:29.154593 16796 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:11:29.155125 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50218.7us
I20260709 12:11:29.128782 16540 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.158238 16777 consensus_queue.cc:1059] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:29.161617 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:29.162894 16789 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.163607 16635 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:29.165031 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:11:29.173750 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 60440.2us
I20260709 12:11:29.128217 16785 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.241168 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:29.245520 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 260389us
W20260709 12:11:29.246984 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 115634us
I20260709 12:11:29.190052 16729 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:29.257462 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:29.253218 12675 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
W20260709 12:11:29.255303 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 68397.8us
I20260709 12:11:29.264565 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:29.264631 13337 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 8. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:11:29.189703 16789 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:29.268096 16517 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.253278 16410 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Stepping down as leader of term 10
W20260709 12:11:29.270284 16652 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
W20260709 12:11:29.284034 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:29.288511 16751 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.686s	user 0.001s	sys 0.002s
I20260709 12:11:29.290727 16765 raft_consensus.cc:3055] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:29.300812 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2420) took 1438 ms. Trace:
I20260709 12:11:29.312366 12675 rpcz_store.cc:276] 0709 12:11:27.862296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.862918 (+   622us) service_pool.cc:224] Handling call
0709 12:11:29.300393 (+1437475us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54000933d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:29.300663 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2719074048}
I20260709 12:11:29.266784 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:11:29.293632 16768 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.766s	user 0.004s	sys 0.007s
W20260709 12:11:29.312129 16763 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:11:29.297200 16472 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:29.309680 16410 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:29.298177 16223 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:29.310149 16765 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:29.312028 16517 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.340961 16485 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:29.341182 16789 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:11:29.342023 16680 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:29.341643 16283 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:29.362862 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:29.349215 16796 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.366950 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:29.369488 16410 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.4, Last appended by leader: 4, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.372934 16211 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.385977 12571 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:29.393175 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 45336.4us
I20260709 12:11:29.323899 16785 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:29.400174 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 82229us
W20260709 12:11:29.405416 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 208410us
I20260709 12:11:29.382650 16680 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:29.411542 16410 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:29.411936 16283 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:29.413106 16664 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:29.414271 12529 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:29.414908 16781 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:29.415555 16781 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:29.415860 12526 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:29.416028 16759 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:29.416432 12526 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:29.416674 16781 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:29.417404 12526 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.420289 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:29.421482 16766 consensus_queue.cc:1059] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:29.425745 13337 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:29.423686 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2050) took 1291 ms. Trace:
I20260709 12:11:29.424682 16779 raft_consensus.cc:3055] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:29.424503 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:29.422276 16759 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:29.385243 16472 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:29.447094 12528 rpcz_store.cc:276] 0709 12:11:28.132713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:28.132898 (+   185us) service_pool.cc:224] Handling call
0709 12:11:29.423354 (+1290456us) spinlock_profiling.cc:243] Waited 494 ms on lock 0x7b54003cafd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:29.423584 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1135372160}
I20260709 12:11:29.451807 16789 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:29.452689 16789 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:29.455029 16781 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:29.458841 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 141505us
I20260709 12:11:29.461474 16070 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:29.408537 16801 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:29.375754 16765 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.462427 16705 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:29.463403 16765 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:29.466087 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:29.467108 12522 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:29.468685 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:29.470613 12529 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:29.471645 16766 raft_consensus.cc:3055] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:29.472121 16766 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:29.470507 16794 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:11:29.474408 16794 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:29.474987 16794 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:29.462503 16801 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:29.475972 16801 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:29.476670 16794 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:29.474400 16560 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.274s	user 0.011s	sys 0.009s
I20260709 12:11:29.478332 12571 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:29.479002 16560 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.359s	user 0.011s	sys 0.010s
I20260709 12:11:29.480074 16070 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:29.480775 16070 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:29.482697 16070 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.484798 12678 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:11:29.485438 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:29.469871 12646 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:29.486569 12679 raft_consensus.cc:2410] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
W20260709 12:11:29.490653 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 230818us
I20260709 12:11:29.475036 12643 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:29.427284 16576 consensus_meta.cc:287] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.508s	user 0.014s	sys 0.002s
I20260709 12:11:29.502750 12646 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:29.409356 16680 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:29.449713 16779 raft_consensus.cc:740] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:29.527386 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:29.399056 16796 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.473420 16766 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.533054 12672 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:11:29.537446 12674 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:29.519306 16410 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.554489 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:29.493479 16664 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 12:11:29.559392 12679 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:11:29.538100 16540 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:29.567947 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48339.4us
I20260709 12:11:29.523149 16705 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:11:29.529778 16729 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:29.582231 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 107457us
W20260709 12:11:29.583115 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:29.561023 16794 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:29.589390 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 100112us
I20260709 12:11:29.578822 16517 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:29.568581 16472 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.606395 16783 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:29.606945 16742 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:29.607107 16783 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:29.608031 16728 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:11:29.593156 16419 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.595449 16766 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:29.611949 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 44220.9us
W20260709 12:11:29.600577 16809 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.334s	user 0.002s	sys 0.000s
I20260709 12:11:29.614668 12674 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:11:29.581199 16410 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.598549 16421 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:11:29.621620 16796 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:29.602880 16789 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:29.626760 12494 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:29.632273 12527 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:29.626863 12679 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:29.635382 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:29.636098 12529 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 10.
I20260709 12:11:29.636315 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:29.637495 12529 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 9. Current term is 10. Ops: [9.10-9.10]
I20260709 12:11:29.638006 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:29.643288 12572 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:29.644047 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:29.644733 16653 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:29.645795 12528 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:29.688009 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:29.688488 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:29.689569 12604 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:29.689643 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2053) took 1420 ms. Trace:
I20260709 12:11:29.608744 16754 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:29.646370 16772 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:29.646673 12572 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:29.631404 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:29.630882 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:29.638146 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:29.710911 12530 rpcz_store.cc:276] 0709 12:11:28.277134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:28.277311 (+   177us) service_pool.cc:224] Handling call
0709 12:11:28.326944 (+ 49633us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:29.125372 (+798428us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:29.689397 (+564025us) spinlock_profiling.cc:243] Waited 797 ms on lock 0x7b54003c1754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:29.689587 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1833138048}
W20260709 12:11:29.719983 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 81952.5us
W20260709 12:11:29.720839 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:29.638489 16462 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.722551 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:29.723327 12603 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:29.723009 16462 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.724921 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:29.639606 12522 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260709 12:11:29.634336 16211 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:29.730423 12599 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:29.626893 16824 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.739261 13335 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:11:29.739871 16824 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:29.742442 16797 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:29.650408 16703 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:29.651928 16783 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:29.743247 12605 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:11:29.631166 16410 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:29.654316 16728 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:29.654819 16767 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.744092 16824 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:29.655331 16649 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.658447 16719 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:29.596684 16729 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.664095 16485 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.664538 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:29.752535 16772 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.752856 16485 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:29.753543 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 89098.5us
I20260709 12:11:29.754794 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:29.755262 16485 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:29.664992 16726 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:29.756115 16785 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:29.756856 13339 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:11:29.756431 16726 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.759859 16726 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:29.665274 16472 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:29.605926 16759 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.667902 12646 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:29.765919 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 98095.4us
I20260709 12:11:29.767524 16824 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:29.767956 16824 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:29.768203 16462 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:29.768316 16824 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:29.626606 16760 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.550s	user 0.013s	sys 0.002s
I20260709 12:11:29.673959 12671 raft_consensus.cc:2410] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:11:29.684881 16778 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.685688 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2164) took 1487 ms. Trace:
I20260709 12:11:29.603747 16517 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:29.692281 13337 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:29.692407 16630 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:29.693580 16769 raft_consensus.cc:3055] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:29.695030 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:29.695706 16766 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:29.697041 13302 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:29.697992 16680 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:29.699237 16746 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 2.097s	user 0.008s	sys 0.015s
I20260709 12:11:29.699872 12526 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:29.707126 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:29.708482 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:29.615911 16806 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:29.724258 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:29.737062 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:29.738394 12599 raft_consensus.cc:3055] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:11:29.739090 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 92535.9us
I20260709 12:11:29.622045 16421 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:29.744565 16767 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.749146 16649 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.730659 16576 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:29.614087 16779 consensus_queue.cc:260] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:29.750972 12600 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:29.616647 16419 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:29.751974 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 121220us
I20260709 12:11:29.715524 16754 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:29.788077 16517 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:29.798615 16410 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:29.785172 12531 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:29.775966 12597 rpcz_store.cc:276] 0709 12:11:28.198281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:28.198458 (+   177us) service_pool.cc:224] Handling call
0709 12:11:28.199207 (+   749us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:28.209105 (+  9898us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:28.209172 (+    67us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:28.209241 (+    69us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:28.209299 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:29.685211 (+1475912us) spinlock_profiling.cc:243] Waited 9.57 ms on lock 0x7b5400310bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:29.685290 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:29.685481 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":22004480}
I20260709 12:11:29.792714 12528 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:29.862833 12675 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:29.802389 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:29.863065 12601 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:29.786765 16334 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.800691 12599 raft_consensus.cc:740] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:11:29.849651 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 94969.3us
I20260709 12:11:29.755364 16772 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:29.803191 16485 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:29.752152 16729 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:29.768604 16703 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260709 12:11:29.794561 16769 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:11:29.863498 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 166554us
I20260709 12:11:29.842438 16765 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:29.802304 16421 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:29.802881 16806 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:29.856917 16576 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:29.789814 16559 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.071s
W20260709 12:11:29.889659 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:29.865268 16517 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:29.902828 16421 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.904577 16421 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:29.908209 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:29.910723 16744 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.761s	user 0.010s	sys 0.004s
I20260709 12:11:29.911199 16744 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.912354 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:29.859997 16715 consensus_queue.cc:1059] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:11:29.871029 16334 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:29.914007 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:29.914542 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:29.915308 12674 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:29.915939 12678 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:11:29.916893 16767 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:29.917166 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:29.861917 16766 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.917582 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:29.857053 16778 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:29.919425 16766 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:29.920204 16778 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:29.911448 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:29.888150 16806 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:29.906322 12599 consensus_queue.cc:260] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:29.928246 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:29.929199 12673 raft_consensus.cc:2410] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 9 because replica has last-logged OpId of term: 7 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:11:29.890395 16576 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:29.928475 12599 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:29.935290 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 346331us
I20260709 12:11:29.937500 12601 raft_consensus.cc:2410] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 10 because replica has last-logged OpId of term: 8 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
W20260709 12:11:29.943193 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 74198.6us
I20260709 12:11:29.949952 12526 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:11:29.926090 13304 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.879s (SENT) [suppressed 22 similar messages]
I20260709 12:11:29.925865 16765 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:29.931640 16778 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:29.961135 12572 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:29.960841 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2062) took 1520 ms. Trace:
I20260709 12:11:29.958912 12673 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:29.959816 12494 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:29.961984 12526 rpcz_store.cc:276] 0709 12:11:28.440430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:28.442224 (+  1794us) service_pool.cc:224] Handling call
0709 12:11:28.442548 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:29.415579 (+973031us) spinlock_profiling.cc:243] Waited 696 ms on lock 0x7b54002adb20. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488997dc0 00007f1488a221c0 00007f1488a23231 00007f1488a1fd8f 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c
0709 12:11:29.699305 (+283726us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b58008435d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f148897c0aa 00007f14889fe447 00007f1488a16fe7 00007f1488a187de 00007f1488a22b49 00007f1488a23416 00007f1488a1fd8f 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4
0709 12:11:29.951076 (+251771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:29.960761 (+  9685us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1629245056}
W20260709 12:11:29.955854 16740 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.059s	user 0.021s	sys 0.006s
I20260709 12:11:29.946998 16472 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:29.973300 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:29.969111 12494 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:29.948452 16766 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.024s
I20260709 12:11:29.952322 16410 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260709 12:11:30.003592 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:29.973325 16652 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:11:30.004281 12600 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:30.005398 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:29.980576 16769 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:30.006182 12602 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:30.007031 16769 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:30.009033 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:30.010283 16671 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 10 current term: 10 is higher.
I20260709 12:11:29.996838 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:30.002671 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 355) took 3531 ms. Trace:
W20260709 12:11:29.983058 16740 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.083s	user 0.021s	sys 0.006s
I20260709 12:11:29.973311 16783 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:29.963933 16772 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:11:30.029049 12680 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:30.041419 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:30.042680 12597 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:29.980908 16070 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:11:30.048952 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 87912.1us
I20260709 12:11:30.044080 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:30.054086 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 164247us
I20260709 12:11:30.000828 16766 consensus_queue.cc:1243] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 12:11:30.063943 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 155844us
I20260709 12:11:30.057037 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:30.058576 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:30.059250 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:29.972018 16778 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:30.056629 12603 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:30.015864 12598 rpcz_store.cc:276] 0709 12:11:26.460894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.464605 (+1003711us) service_pool.cc:224] Handling call
0709 12:11:27.464992 (+   387us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:27.465378 (+   386us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:27.465406 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:27.472318 (+  6912us) log.cc:844] Serialized 26 byte log entry
0709 12:11:27.539597 (+ 67279us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:27.539668 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:27.539769 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:27.797551 (+257782us) spinlock_profiling.cc:243] Waited 13.9 ms on lock 0x7b480024c258. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:11:29.698258 (+1900707us) spinlock_profiling.cc:243] Waited 39.7 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:29.966990 (+268732us) raft_consensus.cc:1685] finished
0709 12:11:29.967025 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:29.992022 (+ 24997us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3605,"raft.run_cpu_time_us":2766,"raft.run_wall_time_us":325500,"spinlock_wait_cycles":123255552,"thread_start_us":23578,"threads_started":1,"wal-append.queue_time_us":64798}
I20260709 12:11:30.063565 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:30.085788 12600 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
W20260709 12:11:30.086817 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 160772us
W20260709 12:11:30.092615 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:30.087628 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:30.043314 16783 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:30.083587 12494 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:29.996850 16472 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:30.024822 16767 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 12:11:30.081254 16774 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:30.098529 16781 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.637s	user 0.012s	sys 0.008s
I20260709 12:11:30.114586 12605 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:30.110422 12572 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:30.114950 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 342377us
I20260709 12:11:30.111296 16783 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:30.088074 16744 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:30.122052 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:30.124264 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:30.139539 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:30.150501 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:30.151415 16753 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:30.151829 16536 consensus_queue.cc:1059] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.155s
W20260709 12:11:30.152447 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:30.152782 16732 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:30.156018 16472 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:30.159108 16720 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260709 12:11:30.160394 16419 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:30.102046 16680 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:30.162865 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:30.106174 12494 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:30.164805 12523 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:11:30.170995 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:30.121445 16839 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:30.202991 12673 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:30.212234 12680 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 8: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:30.213192 12677 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 9: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:11:30.213013 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 124561us
W20260709 12:11:30.224938 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 123662us
I20260709 12:11:30.214912 12676 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 6.
I20260709 12:11:30.209964 16732 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:30.211676 16797 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:11:30.238022 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 132145us
I20260709 12:11:30.239477 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:30.223377 16536 consensus_queue.cc:1243] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 7.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:11:30.235101 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:30.126111 16765 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:30.234858 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:30.240860 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:30.246640 12676 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 11: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:30.246793 12680 raft_consensus.cc:2410] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 9 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:11:30.245829 12673 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:30.252139 12496 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:30.248150 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:30.255246 12496 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:30.256974 12572 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:30.261723 12494 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:30.264479 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:30.265753 12674 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:30.241139 16774 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:30.237574 16732 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:30.233496 16846 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:30.264051 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:30.271111 16729 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:30.291615 12677 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
W20260709 12:11:30.271996 16841 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.099s	user 0.001s	sys 0.001s
W20260709 12:11:30.273036 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 158289us
I20260709 12:11:30.229219 16839 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:30.269181 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:30.281142 16561 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:30.244195 16419 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:30.283728 12676 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:30.234270 16824 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:30.264846 16536 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:30.269198 16720 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:30.252274 16772 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:11:30.276404 16837 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:30.283455 16846 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:30.303460 16652 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:30.285863 16680 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:30.311821 16839 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:30.312275 16561 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:30.342028 16561 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:30.313277 12672 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:30.299015 16797 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:30.314425 16824 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:11:30.303508 16419 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:30.345463 16419 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:30.319265 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 70049.2us
I20260709 12:11:30.303174 12496 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:30.304726 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:30.322436 16778 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:30.322803 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:30.318259 16536 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:11:30.351629 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48545.8us
I20260709 12:11:30.330716 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:30.304104 16843 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:11:30.335034 16652 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:30.353941 12674 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:30.336148 12646 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:30.353631 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:30.355003 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:30.321147 16801 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.840s	user 0.017s	sys 0.000s
I20260709 12:11:30.318197 16720 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:30.332777 16837 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:30.338042 16846 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:30.387892 16597 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:30.401657 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:30.340055 16680 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:30.396221 16845 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:30.353871 16536 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:30.391208 16649 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:30.393784 16774 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:30.403801 12673 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:30.365223 16652 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:30.383410 16720 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:11:30.369138 16842 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
I20260709 12:11:30.408113 16754 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:30.368887 16839 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:30.356423 16728 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:30.335404 16850 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.080s
I20260709 12:11:30.383118 16837 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:30.402923 16658 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:30.468791 16658 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:30.472307 16726 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:30.455955 13331 pending_rounds.cc:85] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 7
I20260709 12:11:30.474577 16726 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:30.475034 16726 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:11:30.476320 16630 consensus_meta.cc:287] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.635s	user 0.009s	sys 0.004s
I20260709 12:11:30.475931 16726 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:30.474613 13331 pending_rounds.cc:107] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 8.8
I20260709 12:11:30.365849 16744 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
I20260709 12:11:30.458353 16856 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:30.447230 16597 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:30.458645 16754 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:30.459131 16778 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:30.452554 16855 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:30.502236 16856 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:30.502806 16855 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:30.505096 16806 consensus_meta.cc:287] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.554s	user 0.011s	sys 0.000s
I20260709 12:11:30.463378 16680 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:30.450629 16671 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:30.505713 16856 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:30.479041 16797 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:11:30.482352 16853 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:30.482498 16658 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:30.456576 16839 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:30.485597 16729 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:30.477519 16726 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:30.369405 16801 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:30.458424 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 74502us
I20260709 12:11:30.478545 12675 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: [8.6-8.6]
W20260709 12:11:30.465891 16421 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.561s	user 0.010s	sys 0.003s
W20260709 12:11:30.479058 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 148518us
I20260709 12:11:30.476342 12569 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:30.544881 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 68627.2us
I20260709 12:11:30.488953 13331 raft_consensus.cc:2889] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:30.460157 16850 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:30.505998 16671 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:30.438661 12671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 384) took 2537 ms (client timeout 1879 ms). Trace:
I20260709 12:11:30.506879 16806 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:30.508805 16652 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:30.482254 16728 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:30.510169 16778 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:11:30.529016 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:30.438406 16845 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:30.469653 16837 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:30.450307 16576 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:30.549412 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:30.535537 16517 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.597s	user 0.012s	sys 0.002s
W20260709 12:11:30.548863 12671 rpcz_store.cc:269] 0709 12:11:27.867528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.867651 (+   123us) service_pool.cc:224] Handling call
0709 12:11:28.782202 (+914551us) spinlock_profiling.cc:243] Waited 4.42 ms on lock 0x7b58001f2c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:11:30.382479 (+1600277us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b54000c26d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:30.405118 (+ 22639us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2056681088}
I20260709 12:11:30.530714 16839 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:30.561395 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:30.630860 13333 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:30.565716 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:30.579806 16801 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:30.583628 16652 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:30.636389 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 396) took 1002 ms. Trace:
I20260709 12:11:30.636924 12523 rpcz_store.cc:276] 0709 12:11:29.633824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:29.638621 (+  4797us) service_pool.cc:224] Handling call
0709 12:11:29.639024 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:30.211762 (+572738us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:30.636060 (+424298us) spinlock_profiling.cc:243] Waited 497 ms on lock 0x7b54003cb9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:30.636303 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1144130304}
I20260709 12:11:30.584350 16766 raft_consensus.cc:3055] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:30.638172 16766 raft_consensus.cc:740] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:30.639015 16671 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:30.530715 16597 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:30.587051 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:30.587749 16211 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:11:30.537906 16630 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:30.593504 16680 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:30.593190 16732 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:30.595026 12673 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:30.596388 16806 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:30.558967 16850 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:30.553776 16472 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:30.587628 12601 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:30.599480 12571 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:30.643211 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:30.563047 16728 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:30.650782 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 64008.7us
I20260709 12:11:30.654454 12606 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:30.531229 16853 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:30.639626 16766 consensus_queue.cc:260] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 8.4, Last appended by leader: 4, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:30.664350 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:30.676270 12671 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:30.652741 16211 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:30.689987 12572 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:30.681602 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:30.691418 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:30.693424 16472 raft_consensus.cc:697] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:30.698100 16472 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:30.641139 16517 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:30.698876 16211 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:30.700001 12678 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:30.700338 16211 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:30.700867 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:30.701550 16517 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:30.701826 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:30.702615 12527 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:30.702659 16779 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:30.706501 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 107177us
I20260709 12:11:30.707324 12571 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:30.701153 16778 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:30.708670 16778 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:30.710445 16070 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:30.715729 16850 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:30.716073 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:30.716784 12526 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:30.716886 12671 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:30.718827 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:30.646667 16783 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:30.721575 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:30.722652 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:30.723539 12529 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:30.724910 12643 connection.cc:256] RPC call timeout handler was delayed by -2.2426947689998542s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2211s for a response
I20260709 12:11:30.726105 12525 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:30.726500 12528 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:11:30.735796 12571 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:30.723067 12526 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:30.736275 12529 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:30.737543 12646 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:30.742663 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:30.740003 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 51948.4us
I20260709 12:11:30.746191 12527 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:30.755986 16778 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:30.756381 12529 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:30.756008 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:30.757040 16846 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:30.751266 16837 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:30.758378 13337 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:30.763939 16846 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:30.766520 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:30.767916 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:30.768723 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:30.694747 16766 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:30.696905 16576 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:30.774942 12496 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:30.769652 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:30.776656 12529 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 10 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:11:30.741757 16801 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:30.661010 16680 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:30.660609 16630 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:30.795006 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:30.746433 16774 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:11:30.785051 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 74964.3us
I20260709 12:11:30.792289 13335 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 9 index: 8. (index mismatch)
I20260709 12:11:30.788082 16767 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:30.797345 12526 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:30.799610 13333 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:30.787587 12646 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:30.800777 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:30.787926 16837 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:30.772065 16797 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:30.815795 12646 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:30.803695 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:30.776542 16824 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:30.698984 16536 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:30.784229 16789 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.036s
I20260709 12:11:30.820364 16767 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:30.820891 16781 raft_consensus.cc:3055] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:30.812354 16837 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:30.801564 16774 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:30.751049 16070 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.054s
I20260709 12:11:30.829211 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:30.833951 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:30.835089 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:30.820364 16517 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:30.848675 12522 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:30.838752 16421 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:30.795713 16561 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.047s
I20260709 12:11:30.792228 16801 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:30.798345 16845 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:30.866942 16744 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:30.834445 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:30.851079 12531 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2095) took 1386 ms (client timeout 1637 ms). Trace:
I20260709 12:11:30.848438 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:30.876175 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:30.841859 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:30.919179 12531 rpcz_store.cc:269] 0709 12:11:29.464541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:29.494068 (+ 29527us) service_pool.cc:224] Handling call
0709 12:11:30.850822 (+1356754us) spinlock_profiling.cc:243] Waited 96.7 ms on lock 0x7b54003cc8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:30.850987 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":222315264}
I20260709 12:11:30.880939 12673 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:11:30.873621 13335 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:11:30.833922 16765 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:30.885214 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:30.881735 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:30.844612 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:30.843304 16781 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:30.896876 12569 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:30.879786 16774 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:30.880645 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:30.932152 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:30.932271 16597 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:30.892050 16801 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:30.935444 16858 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.341s	user 0.002s	sys 0.004s
I20260709 12:11:30.938517 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:30.946133 12603 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:30.933677 16334 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:30.923801 12529 raft_consensus.cc:2368] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12.
I20260709 12:11:30.957458 12524 raft_consensus.cc:2410] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 9 because replica has last-logged OpId of term: 8 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 12:11:30.957953 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 113459us
W20260709 12:11:30.958891 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 62113.2us
I20260709 12:11:30.834738 16824 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:30.964776 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:31.008332 12675 raft_consensus.cc:2410] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:11:31.010015 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:31.011962 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:31.012868 12675 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:30.948827 16671 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:31.026625 12604 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:11:31.028029 12526 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:30.969398 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:31.029477 12529 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:11:30.967370 16421 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:31.036509 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:30.999809 16732 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:31.102926 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:30.942888 16744 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:30.958614 16860 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:31.155417 12527 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:31.006790 16783 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:30.998915 16778 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:31.032081 16561 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:31.159067 16334 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:31.034721 16671 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.159530 16789 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:31.174212 16789 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.035678 16421 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:31.161995 12529 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: []
W20260709 12:11:31.172655 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 340128us
I20260709 12:11:30.958253 16728 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.021198 16839 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.177420 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.180958 12525 raft_consensus.cc:2410] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 11 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:11:31.171705 16561 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:11:31.173702 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:31.140709 16070 consensus_queue.cc:1059] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:11:31.180471 16421 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:30.831776 16797 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:31.170390 16860 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.200242 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.183429 12646 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:31.204638 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.210309 13338 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:31.210975 12675 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:31.160976 16781 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.179780 16334 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:31.206988 16720 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.755s	user 0.005s	sys 0.009s
I20260709 12:11:31.182917 16517 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:31.211004 16536 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:31.146329 16837 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.218565 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:31.216569 16767 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:31.278983 12675 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:31.291141 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:31.212009 16680 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:31.279582 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:31.280740 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 276760us
I20260709 12:11:31.289443 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:31.243386 16211 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.015s	sys 0.019s
I20260709 12:11:31.227353 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:31.301088 12597 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:31.299767 13330 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:31.303835 12675 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:31.302556 16211 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.016s	sys 0.019s
I20260709 12:11:31.278028 16874 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:31.227360 16720 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.338654 12675 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 10.
I20260709 12:11:31.340914 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.311954 16680 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:31.376170 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:31.378852 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:31.326347 16630 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:31.231691 16561 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.343292 16789 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:31.388892 12526 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:31.338387 16765 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:31.400485 12675 raft_consensus.cc:2410] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 8.
I20260709 12:11:31.390460 12602 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:11:31.357215 16850 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:31.397743 16671 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:31.391124 16744 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:31.426458 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.386214 16680 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:31.428596 16680 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.431872 16765 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:31.420970 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:31.357681 16472 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
W20260709 12:11:31.418408 16868 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.272s	user 0.000s	sys 0.001s
I20260709 12:11:31.456648 12675 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:11:31.462847 12604 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:31.459851 12603 raft_consensus.cc:2410] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:11:31.426496 16839 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:31.471745 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:31.477471 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:31.478223 12675 raft_consensus.cc:2376] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 10. Re-sending same reply.
I20260709 12:11:31.437481 16728 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:31.480651 16871 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
I20260709 12:11:31.383105 16874 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.495316 12604 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260709 12:11:31.498237 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2074) took 1372 ms. Trace:
I20260709 12:11:31.492898 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:31.498905 12674 rpcz_store.cc:276] 0709 12:11:30.125994 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:30.359237 (+233243us) service_pool.cc:224] Handling call
0709 12:11:30.359706 (+   469us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:30.429422 (+ 69716us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:30.429477 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:30.446489 (+ 17012us) log.cc:844] Serialized 26 byte log entry
0709 12:11:30.805291 (+358802us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:30.805356 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:30.805465 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:31.497842 (+692377us) raft_consensus.cc:1685] finished
0709 12:11:31.497878 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:31.498112 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39464,"raft.queue_time_us":50733,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":115,"thread_start_us":318739,"threads_started":1,"wal-append.queue_time_us":475007}
I20260709 12:11:31.500483 12675 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 12 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 8.
I20260709 12:11:31.491504 16860 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:31.494187 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:31.504367 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:31.505223 12527 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:31.473541 16887 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:31.448773 16850 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:31.512187 12675 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: [8.4-8.4]
I20260709 12:11:31.492750 16472 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:31.509773 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:31.533622 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:31.534749 12527 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:31.420194 16517 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.501978 16845 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:31.536970 12646 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:31.526893 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2470) took 1039 ms. Trace:
I20260709 12:11:31.539052 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:31.547643 12675 rpcz_store.cc:276] 0709 12:11:30.487900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:31.507500 (+1019600us) service_pool.cc:224] Handling call
0709 12:11:31.511852 (+  4352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:31.526007 (+ 14155us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:31.526808 (+   801us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:31.549536 12675 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:31.550246 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2473) took 1061 ms. Trace:
I20260709 12:11:31.550609 12675 rpcz_store.cc:276] 0709 12:11:30.488853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:31.548796 (+1059943us) service_pool.cc:224] Handling call
0709 12:11:31.549141 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:31.549941 (+   800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:31.550182 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:31.559623 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2471) took 1071 ms. Trace:
I20260709 12:11:31.560109 12674 rpcz_store.cc:276] 0709 12:11:30.488213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:31.533493 (+1045280us) service_pool.cc:224] Handling call
0709 12:11:31.559223 (+ 25730us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:31.559372 (+   149us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:31.559390 (+    18us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:31.559439 (+    49us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:31.559461 (+    22us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:31.559494 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:31.559573 (+    79us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:31.518350 16334 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:31.566267 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.570785 12603 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
W20260709 12:11:31.542985 16869 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.331s	user 0.001s	sys 0.001s
I20260709 12:11:31.532975 16472 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:31.547426 16797 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:31.591837 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:31.588001 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:31.593079 12527 raft_consensus.cc:2376] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8. Re-sending same reply.
I20260709 12:11:31.555498 16766 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:31.594723 12674 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 9 index: 9. (term mismatch)
I20260709 12:11:31.595254 12674 pending_rounds.cc:85] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 8
I20260709 12:11:31.594883 16766 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.595511 12674 pending_rounds.cc:107] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 8.9
I20260709 12:11:31.595786 12674 raft_consensus.cc:2889] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:31.514794 16630 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:11:31.567185 16875 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:31.600334 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.601064 12603 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
W20260709 12:11:31.602921 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 308148us
I20260709 12:11:31.604867 12680 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260709 12:11:31.604252 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 97411.6us
I20260709 12:11:31.598902 16875 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:31.608289 16875 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:31.608937 16884 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.146s	user 0.000s	sys 0.000s
W20260709 12:11:31.606094 16754 consensus_meta.cc:287] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.107s	user 0.014s	sys 0.001s
I20260709 12:11:31.614569 16597 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:31.615090 16421 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:31.615640 16874 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:31.556171 16845 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:31.616528 12606 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:11:31.617086 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:31.618206 16845 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:31.617859 16766 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:31.546965 16887 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:31.619278 16883 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:11:31.614064 16472 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:31.620313 16472 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:31.620895 16887 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:31.618291 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:31.623313 12605 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:31.627264 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:31.628647 16774 consensus_meta.cc:287] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.608s	user 0.008s	sys 0.002s
I20260709 12:11:31.614148 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:31.631605 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:11:31.618291 16888 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:11:31.615259 16597 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.610654 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2080) took 1458 ms. Trace:
I20260709 12:11:31.628664 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:31.634022 12597 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:31.635725 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:31.639329 16885 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:31.597066 16894 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:31.640918 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:31.643278 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:31.645129 16839 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:31.645781 16421 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:31.646359 16789 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:31.646809 16789 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:31.647262 16789 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:31.629405 16774 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.645944 16839 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.005s
I20260709 12:11:31.652104 12643 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:11:31.657537 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 53561.4us
I20260709 12:11:31.659538 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.660104 12571 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:31.633240 12677 rpcz_store.cc:276] 0709 12:11:30.152070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:30.842355 (+690285us) service_pool.cc:224] Handling call
0709 12:11:30.842934 (+   579us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:30.870835 (+ 27901us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:30.870887 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:30.877607 (+  6720us) log.cc:844] Serialized 26 byte log entry
0709 12:11:31.200228 (+322621us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:31.200293 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:31.200391 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:31.610352 (+409961us) raft_consensus.cc:1685] finished
0709 12:11:31.610380 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:31.610574 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8200,"raft.queue_time_us":57607,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":21375,"thread_start_us":241473,"threads_started":1,"wal-append.queue_time_us":460936}
I20260709 12:11:31.662626 12643 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:31.663623 16874 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:31.664192 16887 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:31.664801 16887 raft_consensus.cc:697] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:11:31.664891 16419 consensus_meta.cc:287] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.319s	user 0.017s	sys 0.000s
I20260709 12:11:31.665405 16419 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.665885 16887 consensus_queue.cc:237] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.668450 16419 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:31.669471 12605 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:31.671759 16894 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:31.671589 16875 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.672694 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:31.674505 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2467) took 1420 ms. Trace:
I20260709 12:11:31.674947 12673 rpcz_store.cc:276] 0709 12:11:30.256485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:30.926472 (+669987us) service_pool.cc:224] Handling call
0709 12:11:30.926962 (+   490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:30.927467 (+   505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:30.927522 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:30.928241 (+   719us) log.cc:844] Serialized 26 byte log entry
0709 12:11:31.303103 (+374862us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:31.303236 (+   133us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:31.303438 (+   202us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:31.674207 (+370769us) raft_consensus.cc:1685] finished
0709 12:11:31.674233 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:31.674412 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":92348,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":7340,"thread_start_us":270493,"threads_started":1,"wal-append.queue_time_us":553750}
I20260709 12:11:31.676198 16778 raft_consensus.cc:3055] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:31.676700 16778 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:31.640053 16885 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:31.677738 16885 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:31.673308 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:31.679864 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:31.680302 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:31.678113 16801 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.658s	user 0.010s	sys 0.000s
I20260709 12:11:31.633663 16720 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:31.693114 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:31.699087 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:31.677560 16778 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.710088 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.711426 16895 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.014s
W20260709 12:11:31.719030 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:31.718879 16778 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:31.721231 12597 pending_rounds.cc:85] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 5
I20260709 12:11:31.721591 12597 pending_rounds.cc:107] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260709 12:11:31.721379 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.721879 12597 raft_consensus.cc:2889] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:31.722389 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.723080 12603 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:31.726259 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:31.726946 12598 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:31.702385 16874 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:31.728322 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:31.685657 16801 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.733771 12604 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:11:31.736035 13331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 591) took 1287 ms. Trace:
I20260709 12:11:31.742539 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.707065 16797 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:11:31.745395 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.746153 12600 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:31.736882 13331 rpcz_store.cc:276] 0709 12:11:30.444001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:30.444149 (+   148us) service_pool.cc:224] Handling call
0709 12:11:30.447427 (+  3278us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:11:30.564100 (+116673us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:30.564164 (+    64us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:11:30.565253 (+  1089us) log.cc:844] Serialized 78 byte log entry
0709 12:11:30.946604 (+381351us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:31.234376 (+287772us) log.cc:844] Serialized 14 byte log entry
0709 12:11:31.278662 (+ 44286us) log.cc:844] Serialized 14 byte log entry
0709 12:11:31.279010 (+   348us) log.cc:844] Serialized 14 byte log entry
0709 12:11:31.279169 (+   159us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:31.279260 (+    91us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:31.484817 (+205557us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b1800247000. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:11:31.731093 (+246276us) raft_consensus.cc:1685] finished
0709 12:11:31.731150 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:31.731439 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4955,"prepare.queue_time_us":691567,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":82,"raft.queue_time_us":53045,"raft.run_cpu_time_us":405,"raft.run_wall_time_us":10635,"spinlock_wait_cycles":61267200,"thread_start_us":641802,"threads_started":2,"wal-append.queue_time_us":610400}
I20260709 12:11:31.748790 12604 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:31.703235 16517 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.033s
W20260709 12:11:31.758939 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:31.759994 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:31.760576 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:31.761492 12604 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:31.755061 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 51419.8us
I20260709 12:11:31.757032 12598 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:31.765935 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:31.766645 12673 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:31.766776 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:31.768532 12677 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:11:31.768610 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:31.769882 12598 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:31.770157 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:31.771157 12603 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 12: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:31.773643 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:31.748090 16875 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:31.741914 16680 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:31.774780 12675 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:11:31.768460 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:31.774863 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.776113 12603 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:31.782557 12598 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:31.784482 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.785331 12598 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:31.745414 16774 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:31.787818 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:31.790860 12674 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 11: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:31.795074 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:31.795955 12600 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:31.792214 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.789014 16895 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:31.798821 16895 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:31.799391 16897 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.097s	user 0.000s	sys 0.001s
I20260709 12:11:31.801052 16895 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:31.794770 16887 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:31.801770 16895 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:31.802469 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.802245 16887 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.786525 16680 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.813027 12598 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:31.810701 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.814347 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.814951 12602 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:31.818675 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.825140 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:31.827941 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.828460 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2109) took 1163 ms. Trace:
I20260709 12:11:31.828799 12522 rpcz_store.cc:276] 0709 12:11:30.665006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:30.665133 (+   127us) service_pool.cc:224] Handling call
0709 12:11:31.828287 (+1163154us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54003f7d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:31.828397 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":300667776}
I20260709 12:11:31.832402 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2080) took 1066 ms. Trace:
I20260709 12:11:31.833074 12526 rpcz_store.cc:276] 0709 12:11:30.765981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:31.041070 (+275089us) service_pool.cc:224] Handling call
0709 12:11:31.159125 (+118055us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:31.392467 (+233342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:31.832001 (+439534us) spinlock_profiling.cc:243] Waited 230 ms on lock 0x7b540038f9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:31.832303 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":527945984}
I20260709 12:11:31.838511 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:31.840057 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.809618 16895 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:31.819258 16850 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:31.854007 12600 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:11:31.852125 13304 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:31.862967 16760 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:31.863296 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:31.864581 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.865140 16801 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:31.865795 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:31.817900 16774 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:31.849414 12604 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:31.877947 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:31.879760 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:31.882053 16760 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:31.884268 12643 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:31.885239 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2087) took 1015 ms. Trace:
I20260709 12:11:31.887542 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2106) took 1231 ms. Trace:
W20260709 12:11:31.885813 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 339996us
I20260709 12:11:31.849727 16895 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:31.885241 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2105) took 1231 ms. Trace:
I20260709 12:11:31.888036 12530 rpcz_store.cc:276] 0709 12:11:30.657644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:30.657795 (+   151us) service_pool.cc:224] Handling call
0709 12:11:30.663296 (+  5501us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:30.663506 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:11:30.692208 (+ 28702us) log.cc:844] Serialized 14 byte log entry
0709 12:11:31.465192 (+772984us) spinlock_profiling.cc:243] Waited 10.3 ms on lock 0x7b1800171220. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a 00007f1488a2aa2a
0709 12:11:31.887121 (+421929us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:31.887180 (+    59us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:31.887239 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:31.887312 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:31.887469 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":324348,"prepare.queue_time_us":26621,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":93,"raft.queue_time_us":290393,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"spinlock_wait_cycles":23603584,"thread_start_us":132226,"threads_started":1,"wal-append.queue_time_us":955722}
I20260709 12:11:31.890031 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2084) took 1105 ms. Trace:
I20260709 12:11:31.890470 12525 rpcz_store.cc:276] 0709 12:11:30.784769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:30.930882 (+146113us) service_pool.cc:224] Handling call
0709 12:11:31.889797 (+958915us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b54003cb9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:31.889966 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":339711232}
I20260709 12:11:31.891640 12494 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:31.891898 16720 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:31.892196 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2064) took 1979 ms. Trace:
I20260709 12:11:31.892467 16720 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.886870 16760 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:31.892611 12672 rpcz_store.cc:276] 0709 12:11:29.912484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:29.912599 (+   115us) service_pool.cc:224] Handling call
0709 12:11:29.912929 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:30.343259 (+430330us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:31.891951 (+1548692us) spinlock_profiling.cc:243] Waited 400 ms on lock 0x7b540016eed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:31.892135 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":920300288}
I20260709 12:11:31.896351 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:31.889153 12523 rpcz_store.cc:276] 0709 12:11:30.654280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:30.662245 (+  7965us) service_pool.cc:224] Handling call
0709 12:11:30.683373 (+ 21128us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:30.683669 (+   296us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:11:31.188351 (+504682us) log.cc:844] Serialized 14 byte log entry
0709 12:11:31.884899 (+696548us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:31.884961 (+    62us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:31.885009 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:31.885073 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:31.885185 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":350545,"prepare.queue_time_us":57787,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":15819,"raft.queue_time_us":344759,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":50,"thread_start_us":805865,"threads_started":2,"wal-append.queue_time_us":458172}
I20260709 12:11:31.892879 12494 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:31.890700 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:31.900236 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48271.6us
I20260709 12:11:31.854697 16797 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:31.901925 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:31.875538 16906 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:11:31.887812 12529 rpcz_store.cc:276] 0709 12:11:30.869670 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:31.205117 (+335447us) service_pool.cc:224] Handling call
0709 12:11:31.214515 (+  9398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:31.218818 (+  4303us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:31.218855 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:31.219428 (+   573us) log.cc:844] Serialized 26 byte log entry
0709 12:11:31.504009 (+284581us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:31.531848 (+ 27839us) log.cc:844] Serialized 14 byte log entry
0709 12:11:31.884559 (+352711us) spinlock_profiling.cc:243] Waited 9.32 ms on lock 0x7b180023f260. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:11:31.884777 (+   218us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:31.884872 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:31.884922 (+    50us) raft_consensus.cc:1685] finished
0709 12:11:31.885023 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:31.885173 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14735,"prepare.queue_time_us":15480,"prepare.run_cpu_time_us":3272,"prepare.run_wall_time_us":9475,"raft.queue_time_us":357174,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":10197,"spinlock_wait_cycles":21430144,"thread_start_us":177204,"threads_started":1,"wal-append.queue_time_us":327123}
I20260709 12:11:31.905773 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:31.892802 16837 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:31.908691 12530 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:31.909320 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:31.909379 12494 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:31.910918 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:31.911598 12603 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:31.912247 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.912786 12523 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: [9.6-9.6]
I20260709 12:11:31.896025 16875 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:31.938611 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:31.938855 13304 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:31.902911 16850 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:31.941039 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:31.941105 12494 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:31.906172 16671 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:31.910694 16719 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:31.944851 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:31.958702 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:31.945344 12525 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 11: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:31.948549 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:31.950433 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:31.907078 16797 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:31.953711 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:31.897233 16774 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:31.963974 12603 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:31.969309 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:31.959787 12494 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:31.946543 16914 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.096s	user 0.003s	sys 0.000s
I20260709 12:11:31.970388 16900 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:31.970589 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.971025 16900 raft_consensus.cc:697] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:31.971267 12605 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
I20260709 12:11:31.971393 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:31.972301 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.971978 16900 consensus_queue.cc:237] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:31.972535 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:31.973194 12597 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:31.974318 16846 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:31.976294 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:31.967442 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:31.940335 16837 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:31.983729 16837 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.984371 16419 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:31.985427 16419 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:31.946326 16728 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:31.933050 16652 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.051s
I20260709 12:11:31.966602 16789 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:31.989975 16774 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:31.990957 12673 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:11:31.943765 16760 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:31.956357 16671 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.994628 16671 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:31.994917 16846 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:31.957986 16894 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.038s
I20260709 12:11:31.998062 12600 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:11:31.998845 16419 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.999578 16561 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:31.990979 16789 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:31.979131 16719 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:31.980458 16797 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:31.951903 16845 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:31.963740 16754 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:31.963615 16744 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:11:31.998174 16875 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:31.977069 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:31.962971 16759 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:31.979254 16850 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:32.005196 16719 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:32.021498 16719 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:31.962209 16885 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:32.030802 12680 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 9: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:32.031899 16885 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:32.023212 16759 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:32.033010 16759 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:32.041599 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:32.043772 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:32.007215 16845 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:32.028908 16853 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:32.045107 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 68100.1us
I20260709 12:11:32.046808 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:32.051494 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 99999us
I20260709 12:11:32.052641 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:32.053182 12672 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:32.047406 16845 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:32.062873 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2069) took 2148 ms. Trace:
I20260709 12:11:32.063483 12678 rpcz_store.cc:276] 0709 12:11:29.914046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:29.916722 (+  2676us) service_pool.cc:224] Handling call
0709 12:11:29.917045 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:30.700467 (+783422us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:32.062457 (+1361990us) spinlock_profiling.cc:243] Waited 783 ms on lock 0x7b540036cc54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:32.062739 (+   282us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1800141568}
I20260709 12:11:32.052268 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:32.065160 16870 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.770s	user 0.004s	sys 0.000s
I20260709 12:11:32.047945 16797 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:32.065691 16211 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:32.066311 16211 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:32.066720 16211 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:32.070037 16855 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:32.070698 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:32.071369 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:32.072068 12672 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:32.073658 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:32.071588 12678 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 11: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:31.978211 12604 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 10.
I20260709 12:11:32.077457 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:32.073416 16893 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:32.080235 16908 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:11:32.080538 16893 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:32.081032 16421 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:32.080533 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:32.081224 16908 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:32.079439 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:32.085898 16766 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:32.086464 12496 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:32.084589 16908 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:32.087828 16908 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:32.089351 16908 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:32.089947 16536 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:32.081832 16472 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.009s
I20260709 12:11:32.091218 16744 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:11:32.090624 16536 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:32.092803 13339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 436) took 1545 ms. Trace:
I20260709 12:11:32.097065 12597 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:32.098556 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:32.100204 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 123984us
I20260709 12:11:32.100835 12677 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 9: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:32.037266 16719 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:32.096922 13339 rpcz_store.cc:276] 0709 12:11:30.547269 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:30.547484 (+   215us) service_pool.cc:224] Handling call
0709 12:11:30.562108 (+ 14624us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:30.831129 (+269021us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:30.831164 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:30.831214 (+    50us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:30.831248 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:32.092541 (+1261293us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b54006f8454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:32.092594 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:32.092721 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":618017536}
I20260709 12:11:32.075560 16885 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:32.109790 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:32.095999 16875 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election won for term 8
W20260709 12:11:32.113473 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 133638us
I20260709 12:11:32.111651 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:32.119498 12496 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:32.126380 12680 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:32.107013 16885 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:32.130944 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:32.131721 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:32.134620 12496 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:32.140121 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:32.140100 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:32.142799 13339 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:32.143349 16894 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:32.134354 16837 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:32.144181 16894 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:32.144728 16894 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:32.113867 16789 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:32.146375 16879 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:32.136819 16850 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:32.148048 16850 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:32.148469 16850 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:32.134805 16900 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.018s
I20260709 12:11:32.131618 16855 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:32.151132 16855 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:32.136405 16472 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260709 12:11:32.152779 16472 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:32.153162 16472 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:32.153491 16855 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:32.153640 16887 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:32.150292 16774 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:32.146595 16789 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:11:32.118220 16726 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.183s	user 0.010s	sys 0.006s
I20260709 12:11:32.155988 12496 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:32.167925 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:32.169597 13302 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:32.172523 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:32.173161 12672 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:32.174398 12496 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:32.188452 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:32.158967 16419 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:11:32.169627 16726 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.234s	user 0.010s	sys 0.007s
I20260709 12:11:32.154856 16728 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.043s
I20260709 12:11:32.228072 16853 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:32.228806 16853 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:32.183904 16900 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:32.229439 16719 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:32.229795 16900 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 11 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:32.230803 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2346) took 2244 ms. Trace:
I20260709 12:11:32.231105 13337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 592) took 1739 ms. Trace:
I20260709 12:11:32.231297 12601 rpcz_store.cc:276] 0709 12:11:29.986620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:30.001307 (+ 14687us) service_pool.cc:224] Handling call
0709 12:11:30.001636 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:30.666987 (+665351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:32.230535 (+1563548us) spinlock_profiling.cc:243] Waited 586 ms on lock 0x7b540022ecd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:32.230734 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1347203328}
I20260709 12:11:32.233577 12571 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:32.229261 16853 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:32.231642 13337 rpcz_store.cc:276] 0709 12:11:30.494017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:30.496896 (+  2879us) service_pool.cc:224] Handling call
0709 12:11:30.497332 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:30.758977 (+261645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:32.230771 (+1471794us) spinlock_profiling.cc:243] Waited 261 ms on lock 0x7b5400469954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:32.230948 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":599684352}
I20260709 12:11:32.237445 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:32.239041 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:32.178911 16839 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:32.135349 16885 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:32.154013 16472 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:32.174360 16806 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:32.253029 13304 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:32.222066 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52543.2us
I20260709 12:11:32.183434 16789 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:32.256973 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:32.257440 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:32.169998 16774 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:32.129438 16875 raft_consensus.cc:697] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:32.253217 16760 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:32.267750 16760 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:32.268221 16760 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:32.268004 16875 consensus_queue.cc:237] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:32.269167 16774 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:32.258021 16855 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:32.223856 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:32.227051 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:32.276746 13339 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:32.187847 16536 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:32.240078 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:32.235548 16837 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:32.230904 16929 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:32.244933 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:32.244472 16900 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:32.245101 16874 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:32.246280 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:32.239163 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:32.230806 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48098us
W20260709 12:11:32.251861 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 140178us
I20260709 12:11:32.259101 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:32.259871 12678 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:11:32.265177 16759 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:32.253450 16211 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:32.253810 16895 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:32.257267 16806 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:32.303978 12530 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:11:32.248163 16893 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:32.309333 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 70245.8us
I20260709 12:11:32.316018 12525 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:32.303491 16929 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:32.333307 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:32.311548 12496 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:32.318503 16911 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:32.323274 16824 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:32.324360 12671 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:32.318501 16879 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:32.342767 16929 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:32.352596 16925 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:11:32.333935 12530 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:32.329186 16895 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:32.340840 12529 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:32.350423 13339 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:32.318842 16070 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.025s
I20260709 12:11:32.302071 16845 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:32.254931 16839 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:32.318650 16874 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:32.360327 16837 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:32.361603 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 44022.1us
W20260709 12:11:32.361898 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50335.3us
W20260709 12:11:32.360570 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 45573.2us
I20260709 12:11:32.456454 12494 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:32.357820 16421 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 12:11:32.375258 13305 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:32.342293 16908 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260709 12:11:32.402515 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:32.462957 16754 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260709 12:11:32.463181 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 88119.4us
W20260709 12:11:32.464803 16924 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.162s	user 0.000s	sys 0.001s
I20260709 12:11:32.465492 13305 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:32.466696 16923 log.cc:927] Time spent T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
W20260709 12:11:32.468304 16927 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
W20260709 12:11:32.453066 16921 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
I20260709 12:11:32.410367 16211 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:32.418453 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:32.457656 12494 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:11:32.488881 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 70578us
I20260709 12:11:32.327617 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:32.455683 12496 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:32.457952 12672 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
W20260709 12:11:32.459781 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 129364us
I20260709 12:11:32.376840 16806 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:32.405390 16760 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:32.417191 16895 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:32.474037 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:32.474519 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:32.505666 12671 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:32.492725 13305 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:32.494426 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:32.497763 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:32.460821 16913 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:11:32.508050 16719 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:32.478564 16759 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:32.490154 16908 consensus_queue.cc:1243] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:11:32.474051 16597 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:32.499197 16779 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:32.503506 16930 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:32.536957 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:32.511824 16561 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:32.509383 16919 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:32.548522 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:32.434791 16879 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:32.507838 16824 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:32.521936 16806 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:32.542490 16900 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:32.474058 16774 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:32.559268 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 65262.8us
I20260709 12:11:32.560185 12572 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:32.556607 12671 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:32.520601 16719 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:32.499470 16911 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:11:32.563580 12572 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:32.545208 16867 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.591s	user 0.009s	sys 0.014s
I20260709 12:11:32.567211 12603 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:32.567590 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:32.504210 16853 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:32.508886 16929 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:32.553825 16070 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:32.652320 12680 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:11:32.657733 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 94239us
I20260709 12:11:32.660864 12572 leader_election.cc:400] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12.
I20260709 12:11:32.530383 16894 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:32.661267 12572 leader_election.cc:403] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:32.563649 16561 raft_consensus.cc:697] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:32.569702 16908 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:11:32.663185 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 95670.9us
I20260709 12:11:32.569136 16930 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:11:32.666348 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 173752us
I20260709 12:11:32.560137 16211 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.099s
I20260709 12:11:32.572424 16935 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:32.649426 16875 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:32.568873 16824 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:32.685413 12524 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2078) took 1920 ms (client timeout 2124 ms). Trace:
W20260709 12:11:32.688015 12524 rpcz_store.cc:269] 0709 12:11:30.765445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:30.812744 (+ 47299us) service_pool.cc:224] Handling call
0709 12:11:32.681750 (+1869006us) spinlock_profiling.cc:243] Waited 82.7 ms on lock 0x7b54003c0854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:32.685337 (+  3587us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":190302592}
I20260709 12:11:32.659482 16070 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:32.753299 16930 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:32.678478 16911 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12.
I20260709 12:11:32.758877 16561 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:32.686829 16926 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:32.760296 16760 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:32.761264 16936 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 12:11:32.767716 16928 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.415s	user 0.006s	sys 0.021s
I20260709 12:11:32.770532 16536 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:32.645548 16850 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:32.804711 16536 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:32.809652 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 319827us
I20260709 12:11:32.810985 16850 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:32.812093 16759 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:32.812871 16895 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:32.813438 16759 raft_consensus.cc:3055] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:32.816431 16926 raft_consensus.cc:697] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:32.818149 16941 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:32.818863 16941 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:32.825186 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:32.667464 16334 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:32.668946 16421 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:11:32.781555 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:32.791313 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:32.798146 16934 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 12:11:32.795673 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:32.796711 12672 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:32.793831 16719 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:32.817662 16926 consensus_queue.cc:237] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:32.757519 16944 consensus_queue.cc:1059] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.065s
I20260709 12:11:32.690733 16875 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:32.813936 16759 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:11:32.791481 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:32.866742 13331 pending_rounds.cc:85] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 9
W20260709 12:11:32.846467 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:32.870193 13331 pending_rounds.cc:107] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 9.10
I20260709 12:11:32.870630 13331 raft_consensus.cc:2889] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:32.846570 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:32.693476 16908 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.114s
I20260709 12:11:32.852443 16334 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:32.840443 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:32.843118 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:32.867256 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 95538.9us
I20260709 12:11:32.850807 12605 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:32.873294 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:32.876509 12603 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:11:32.876888 12604 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 9: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:32.880676 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:32.824043 16839 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:32.881560 12604 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:32.884526 12525 consensus_meta.cc:287] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.551s	user 0.008s	sys 0.004s
I20260709 12:11:32.944655 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:33.062449 12525 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
W20260709 12:11:33.044528 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:33.045969 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:32.858896 16937 log.cc:927] Time spent T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
I20260709 12:11:32.866343 16671 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:32.919442 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:32.878430 16944 consensus_queue.cc:1243] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 8.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 12:11:33.053407 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 374505us
I20260709 12:11:33.070081 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:33.070343 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:33.059119 12605 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:11:33.063354 12598 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:33.047468 12677 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:11:32.907877 16941 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:33.079114 13334 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 9: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:33.080199 13330 raft_consensus.cc:2368] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11.
W20260709 12:11:33.068459 12530 consensus_meta.cc:287] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.679s	user 0.013s	sys 0.001s
I20260709 12:11:33.087147 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:33.088415 12530 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:33.091557 13338 raft_consensus.cc:2368] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12.
I20260709 12:11:32.978207 16895 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:32.851646 16845 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:33.047437 16421 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.047191 16211 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.082443 16908 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:33.092454 16789 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:33.136096 12598 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:11:33.136695 13338 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260709 12:11:33.130893 16908 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.131091 16421 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.143056 16943 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.143826 16943 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:33.146318 16940 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.318s	user 0.000s	sys 0.004s
I20260709 12:11:33.149326 16951 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:33.149720 16894 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:33.121775 16853 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:33.156666 12602 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:33.128150 16774 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:33.129464 16879 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:33.155145 16874 consensus_meta.cc:287] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.695s	user 0.012s	sys 0.000s
W20260709 12:11:33.151098 16947 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:11:33.176332 12603 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:11:33.177235 12604 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:33.172770 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:33.081524 16671 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:33.152851 16948 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:11:33.151412 16875 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:33.141868 16908 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:33.171010 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:33.152463 16334 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:33.151396 16824 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:33.164417 16911 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:33.188719 16911 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:33.182957 16885 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.897s	user 0.012s	sys 0.003s
I20260709 12:11:33.191766 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2079) took 3040 ms. Trace:
I20260709 12:11:33.179306 16874 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:33.193298 12676 rpcz_store.cc:276] 0709 12:11:30.151776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:30.767268 (+615492us) service_pool.cc:224] Handling call
0709 12:11:30.767650 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:31.150321 (+382671us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:31.150374 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:32.655642 (+1505268us) spinlock_profiling.cc:243] Waited 7.42 ms on lock 0x7b2c00037868. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:11:32.656098 (+   456us) log.cc:844] Serialized 26 byte log entry
0709 12:11:32.793475 (+137377us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:32.793516 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:32.793588 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:33.191352 (+397764us) raft_consensus.cc:1685] finished
0709 12:11:33.191384 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:33.191683 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30134,"raft.queue_time_us":45313,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":13658,"spinlock_wait_cycles":17057152,"thread_start_us":488491,"threads_started":2,"wal-append.queue_time_us":197380}
I20260709 12:11:33.192483 16885 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.196846 16070 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:33.132002 16211 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.198427 12678 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:33.186451 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:33.189730 16671 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:33.158527 16789 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:33.207826 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 135095us
I20260709 12:11:33.200942 16421 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:33.217585 16806 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.653s	user 0.004s	sys 0.009s
I20260709 12:11:33.221560 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:33.240626 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:33.234223 16837 consensus_meta.cc:287] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.777s	user 0.007s	sys 0.007s
I20260709 12:11:33.234637 16908 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:33.245867 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:33.252409 16962 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.254703 12569 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:33.255635 16070 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:33.256253 12569 leader_election.cc:400] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11.
I20260709 12:11:33.231447 16421 consensus_queue.cc:1059] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.017s
I20260709 12:11:33.239953 16597 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:33.239717 16671 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:33.243397 16754 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.288774 16754 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.268110 12569 leader_election.cc:403] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:11:33.290542 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 44785.9us
I20260709 12:11:33.292058 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:33.246615 16885 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:33.245749 16837 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:33.243569 16955 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:33.273624 16726 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:33.296430 12569 leader_election.cc:400] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12.
I20260709 12:11:33.296927 12569 leader_election.cc:403] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:33.276968 12604 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:11:33.277340 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:33.279531 12525 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:33.280892 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:33.298627 12530 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 11: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:33.281383 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:33.300158 12524 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 12: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:33.282374 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:33.283531 12523 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:33.285167 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:33.266882 16744 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:33.286674 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:33.253376 16962 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.287750 16671 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.258839 16960 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:33.246299 16958 consensus_queue.cc:1059] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
I20260709 12:11:33.306958 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:33.307209 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:33.308868 16652 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.310940 16778 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:11:33.311410 16774 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:33.259742 16959 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:33.303417 16908 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11.
I20260709 12:11:33.303498 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:33.350360 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:33.351423 12604 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:33.353039 12526 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:33.346976 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:33.356557 12601 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:33.298897 16824 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:33.346630 12522 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 8 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:11:33.359238 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:33.360821 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:33.368112 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:33.369575 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:33.295186 16597 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.302362 16964 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:11:33.376060 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:33.300839 16726 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:33.385419 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:33.348850 16879 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:11:33.389149 12601 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:33.390416 16719 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.391031 16719 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.391616 16597 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:33.391620 16726 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:33.393304 16967 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:11:33.393491 16719 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:33.313045 16744 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:33.399359 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:33.400494 16952 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:11:33.410000 16744 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:33.409861 12601 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 10. Current term is 11. Ops: [8.4-10.5]
I20260709 12:11:33.301015 16955 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:33.402449 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:33.416008 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:33.418887 12529 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:33.416779 12601 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:33.422286 12601 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: [11.8-11.8]
I20260709 12:11:33.424140 12601 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:11:33.345624 16652 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:33.420243 12598 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:33.370779 16958 raft_consensus.cc:3055] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:33.435778 12598 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:33.374498 16824 raft_consensus.cc:697] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:33.354594 16926 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 12:11:33.445876 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:33.446671 12529 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:33.382591 16211 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:33.444080 12598 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:33.448882 12598 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:33.452983 16962 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:33.452692 16824 consensus_queue.cc:237] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.443125 16789 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:33.456838 16671 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:33.453323 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:33.449792 16754 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:33.470680 16922 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.171s	user 0.000s	sys 0.003s
I20260709 12:11:33.443780 16958 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:33.473682 16958 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.475251 16958 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:33.478197 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:33.478984 12529 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:33.483227 16334 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.483441 16561 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:33.483870 16334 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.484114 16561 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.486260 16879 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.486824 16893 raft_consensus.cc:3055] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:33.487277 16893 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:33.486824 16879 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.490546 16960 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:33.491225 16960 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.497056 16874 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:33.497936 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:33.499423 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:33.500406 16334 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:33.502043 16561 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:33.502112 16778 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12.
I20260709 12:11:33.504547 16879 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:33.504730 16781 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:33.516965 12572 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:33.514910 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:33.501920 16955 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:33.518159 12646 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:33.507002 16334 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:33.509339 12426 catalog_manager.cc:5697] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 5 to 7, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 7 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:33.523543 12598 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 11 index: 6. (term mismatch)
I20260709 12:11:33.537846 12598 pending_rounds.cc:85] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 5
I20260709 12:11:33.538295 12598 pending_rounds.cc:107] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 10.6
I20260709 12:11:33.538569 12598 raft_consensus.cc:2889] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:33.524569 12572 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:33.520061 16801 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260709 12:11:33.526036 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:33.526312 16935 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:33.507181 16472 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:33.554019 12494 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:33.557003 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 244551us
I20260709 12:11:33.556385 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:33.535594 16879 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:33.545758 16960 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:33.541214 16778 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:33.562016 12572 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:33.546973 16955 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:33.564749 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 96118.9us
I20260709 12:11:33.546902 16561 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:33.566869 12426 catalog_manager.cc:5697] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:33.561359 12494 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:33.569311 12572 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:33.546552 16874 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.022s
I20260709 12:11:33.560200 16978 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.560408 16860 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:33.564355 16760 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:33.581096 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:33.581622 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:33.544574 16893 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.560585 16885 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260709 12:11:33.563478 16774 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
I20260709 12:11:33.575968 16671 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:33.594954 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:33.565896 16935 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.018s
I20260709 12:11:33.572685 16977 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:33.596019 13334 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 11: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:33.603780 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:33.606467 12679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2066) took 3693 ms (client timeout 1835 ms). Trace:
I20260709 12:11:33.586767 16778 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:33.588568 16962 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260709 12:11:33.606869 12679 rpcz_store.cc:269] 0709 12:11:29.913147 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:29.913348 (+   201us) service_pool.cc:224] Handling call
0709 12:11:32.168941 (+2255593us) spinlock_profiling.cc:243] Waited 83.8 ms on lock 0x7b5400288d20. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488997dc0 00007f1488a23e18 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:33.603917 (+1434976us) spinlock_profiling.cc:243] Waited 866 ms on lock 0x7b540008eb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:33.606401 (+  2484us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2184186112}
I20260709 12:11:33.594780 16671 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:33.592212 16561 raft_consensus.cc:697] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:33.602838 16935 consensus_queue.cc:1243] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 3
I20260709 12:11:33.609463 16671 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:33.602241 16977 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:33.576649 16955 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.610647 16652 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:33.603700 16874 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260709 12:11:33.626623 16977 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:33.609649 16778 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:33.628229 12530 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:33.628674 16778 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:33.629158 12530 raft_consensus.cc:606] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:33.629601 12530 raft_consensus.cc:614] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:11:33.631235 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 55949.8us
I20260709 12:11:33.610148 16962 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:33.642292 16962 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:33.642513 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:33.621114 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:33.632632 16797 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:11:33.634057 16932 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.269s	user 0.004s	sys 0.000s
I20260709 12:11:33.651988 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 400) took 1414 ms. Trace:
I20260709 12:11:33.652467 12675 rpcz_store.cc:276] 0709 12:11:32.237989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:32.241118 (+  3129us) service_pool.cc:224] Handling call
0709 12:11:32.246828 (+  5710us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:32.247244 (+   416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:32.247274 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:32.247801 (+   527us) log.cc:844] Serialized 26 byte log entry
0709 12:11:32.319306 (+ 71505us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:32.331723 (+ 12417us) log.cc:844] Serialized 14 byte log entry
0709 12:11:32.331939 (+   216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:32.332010 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:33.633372 (+1301362us) spinlock_profiling.cc:243] Waited 9.3 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:33.651661 (+ 18289us) raft_consensus.cc:1685] finished
0709 12:11:33.651767 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:33.651916 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16681,"prepare.queue_time_us":12658,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":50,"raft.queue_time_us":8594,"raft.run_cpu_time_us":262,"raft.run_wall_time_us":9202,"spinlock_wait_cycles":21383424,"thread_start_us":70058,"threads_started":1,"wal-append.queue_time_us":112335}
I20260709 12:11:33.655026 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:33.655418 16720 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:33.646495 16797 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:33.657047 16797 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:33.659750 16938 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.898s	user 0.003s	sys 0.000s
I20260709 12:11:33.660449 16797 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:33.638195 16789 raft_consensus.cc:3055] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:33.662489 16789 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:33.663587 16789 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.664328 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:33.664944 16789 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:33.665309 16778 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.638484 12426 catalog_manager.cc:5697] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 2 to 6, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 6 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:33.666558 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:33.667573 13334 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 12: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:33.667156 12426 catalog_manager.cc:5697] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 3 to 6, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 6 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:33.649421 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:33.669219 12598 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 12: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:33.669505 16806 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:33.658548 16943 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:33.659091 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:33.671010 12600 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 6.
I20260709 12:11:33.585806 16978 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.669251 13338 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:11:33.672698 16887 raft_consensus.cc:3055] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:33.673098 16887 raft_consensus.cc:740] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:33.673708 16978 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:33.633287 16561 consensus_queue.cc:237] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.674057 16887 consensus_queue.cc:260] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.675722 16887 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:33.677593 16930 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.678416 12523 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: [11.6-11.6]
I20260709 12:11:33.678238 16930 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.680053 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:33.636445 16955 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:33.683315 12522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:33.683616 12529 pending_rounds.cc:85] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 3
I20260709 12:11:33.684151 12529 pending_rounds.cc:107] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 8.4
I20260709 12:11:33.684083 12522 raft_consensus.cc:606] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:33.684935 12522 raft_consensus.cc:614] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:33.685395 12528 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:33.686627 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:33.687413 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:33.688200 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:33.689040 12603 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:33.689122 12522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:33.690090 12522 raft_consensus.cc:606] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:33.692226 16910 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:33.692659 16910 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:33.692941 16955 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.693194 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:33.693440 16955 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.683691 16919 raft_consensus.cc:3055] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:33.695266 12531 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 10.
I20260709 12:11:33.695384 16919 raft_consensus.cc:740] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:11:33.671756 16907 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 11 current term: 11 is higher.
I20260709 12:11:33.696228 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:33.697012 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:33.698307 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:33.701491 12598 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:11:33.698566 12529 raft_consensus.cc:2889] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:33.706730 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:33.706236 12646 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:33.708349 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:33.709403 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:33.710004 12531 raft_consensus.cc:2376] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11. Re-sending same reply.
I20260709 12:11:33.702853 16955 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:33.713034 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:33.714267 16879 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:33.714836 16879 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:33.715255 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:33.715833 12523 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:33.717123 16879 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:33.717681 16879 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:33.718809 16919 consensus_queue.cc:260] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:33.717247 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2133) took 1347 ms. Trace:
I20260709 12:11:33.713479 16680 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:33.721038 12526 rpcz_store.cc:276] 0709 12:11:32.369783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:32.370013 (+   230us) service_pool.cc:224] Handling call
0709 12:11:33.717055 (+1347042us) spinlock_profiling.cc:243] Waited 843 ms on lock 0x7b5400054454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:33.717162 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1939960704}
I20260709 12:11:33.723253 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:33.724645 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:33.725771 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:33.726274 12680 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:33.694564 16894 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:33.738042 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:33.739046 12526 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:33.728789 13302 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:33.725271 12676 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:33.729151 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:33.731176 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:33.741726 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:33.742576 12678 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:33.704455 16720 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.733319 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:33.734999 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:33.750814 12522 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 12:11:33.751435 16680 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:11:33.720278 16919 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:33.752116 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:33.752847 12522 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:33.693482 16910 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.754667 16910 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:33.737397 16894 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:33.757575 16894 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.758646 16774 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:33.745412 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:33.762818 16846 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:11:33.764513 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:33.765848 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:33.732462 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:33.740006 12646 leader_election.cc:336] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.731s (SENT)
I20260709 12:11:33.720080 16517 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:33.769030 16517 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.770315 16720 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:33.737273 16837 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:33.748199 12674 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:33.771376 16837 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:33.721849 16879 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:33.776721 12525 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:11:33.748559 16806 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:33.779114 12676 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:33.753782 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:33.783872 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 48947.3us
I20260709 12:11:33.784667 12643 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:33.787431 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:33.787446 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:33.745747 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:33.790488 12426 catalog_manager.cc:5697] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 2 to 7, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 7 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:33.788862 12678 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
W20260709 12:11:33.811250 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 103537us
I20260709 12:11:33.811939 12678 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:33.757448 16671 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.823642 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:33.837648 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:33.842609 12426 catalog_manager.cc:5697] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 5 to 7, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 7 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:11:33.843897 12598 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 9 index: 10. (index mismatch)
I20260709 12:11:33.839229 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:33.846060 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:33.815637 12680 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:33.826606 12604 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260709 12:11:33.776319 16779 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:11:33.834429 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:33.847333 12531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:33.850214 12531 raft_consensus.cc:606] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:33.851032 12601 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:11:33.850666 12531 raft_consensus.cc:614] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:33.851611 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:33.848536 12426 catalog_manager.cc:5697] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:33.818840 16943 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:33.820816 16885 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:33.847925 12672 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:33.863649 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:33.863904 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:33.868117 12522 raft_consensus.cc:2393] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 7: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:33.841542 12523 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:33.865315 12601 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 10. Current term is 11. Ops: [7.8-10.9]
I20260709 12:11:33.851140 16779 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:33.866385 16939 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.869562 16837 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:33.870159 16806 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.869642 16939 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.870695 16806 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.872826 16806 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:33.873294 16720 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:33.875644 16671 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:33.869863 16779 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:33.878924 16977 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:33.879576 16930 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:33.887686 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:33.823187 16744 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:33.890851 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:33.893800 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:33.894002 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:33.891489 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:33.889793 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:33.895399 12604 raft_consensus.cc:2410] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 13 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:11:33.897765 12598 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:11:33.899252 12526 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:11:33.848421 16517 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:33.900266 13330 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 12: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:33.895999 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:33.901862 12531 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:33.894002 12643 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:33.908686 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:33.907851 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:33.867395 16885 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:33.916911 12674 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:11:33.867127 16943 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:33.910524 12522 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:33.974876 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:33.971014 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 55267.6us
I20260709 12:11:33.975615 12523 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:33.907265 16855 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:33.900656 16874 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:33.913525 16211 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:33.967064 16885 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:33.915258 16908 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:33.915540 16778 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:33.966940 16671 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.967399 16943 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:33.971524 16561 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:33.992111 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:34.013027 12527 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:11:33.987049 16958 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:33.979775 16824 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.021s	sys 0.032s
I20260709 12:11:33.904248 16744 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.006796 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:33.985174 12526 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:33.979027 16960 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:33.992519 16879 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:33.974392 16517 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:11:33.986166 16846 raft_consensus.cc:697] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 11 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:33.993731 16211 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:34.001565 12523 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:34.012672 16561 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:34.025625 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2107) took 1263 ms. Trace:
I20260709 12:11:34.024749 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:34.039750 16744 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:34.026814 16960 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.044162 16960 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:34.016525 16962 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.047561 16561 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:34.048561 16962 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:34.048775 16759 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.049458 12602 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:34.049310 13305 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:34.050575 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:34.051175 12526 raft_consensus.cc:2376] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10. Re-sending same reply.
I20260709 12:11:34.051659 16919 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:34.052134 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:34.053272 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:34.015887 16671 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.057183 16671 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:34.049899 16759 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:34.058578 16846 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:34.059862 16894 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:34.060407 16929 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:33.993613 16874 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.062803 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:34.033295 12671 rpcz_store.cc:276] 0709 12:11:32.762582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:32.762791 (+   209us) service_pool.cc:224] Handling call
0709 12:11:32.763207 (+   416us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:33.071119 (+307912us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:33.071169 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:33.107459 (+ 36290us) log.cc:844] Serialized 52 byte log entry
0709 12:11:33.267825 (+160366us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:11:33.267876 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:34.021143 (+753267us) spinlock_profiling.cc:243] Waited 182 ms on lock 0x7b540015e0d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:34.021241 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:34.021298 (+    57us) raft_consensus.cc:1685] finished
0709 12:11:34.021319 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:34.025548 (+  4229us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107764,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":17652,"spinlock_wait_cycles":417743232,"thread_start_us":112955,"threads_started":1,"wal-append.queue_time_us":223929}
I20260709 12:11:34.070240 12598 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:34.087941 12646 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:34.089892 12646 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:34.027595 16980 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:34.099291 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:34.032104 16211 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
W20260709 12:11:34.032141 16824 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s	user 0.022s	sys 0.032s
I20260709 12:11:34.098292 12646 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:34.099575 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:34.085351 12572 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:34.107370 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:34.109323 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:34.025151 16958 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:34.119765 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:34.121331 12598 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:34.121827 12646 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:34.128008 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:34.128952 12526 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:34.129068 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:34.130086 12598 raft_consensus.cc:2410] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:11:34.099284 16806 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:34.126811 12494 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:34.103596 16960 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:34.137471 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:34.139657 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:34.141137 12598 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 11: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:34.104640 16774 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:34.142882 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:34.112082 16962 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:34.149433 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:34.148123 12599 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:32832
I20260709 12:11:34.152107 12599 raft_consensus.cc:606] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:34.152621 12599 raft_consensus.cc:614] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:34.148489 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:34.155359 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:34.156610 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:34.157625 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:34.123576 16910 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:34.131245 16930 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:34.099349 16919 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
I20260709 12:11:34.148164 12494 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:34.167996 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:34.120154 16744 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:34.127869 16879 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:34.158066 16875 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:34.105055 16980 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.195566 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:34.196023 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:34.209239 12605 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 11: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:34.135422 16720 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.031s
I20260709 12:11:34.146472 16977 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:34.142676 16652 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:11:34.193276 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 45604.5us
W20260709 12:11:34.166983 16789 consensus_meta.cc:287] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.501s	user 0.013s	sys 0.001s
I20260709 12:11:34.196470 16783 raft_consensus.cc:2764] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
I20260709 12:11:34.218413 12523 raft_consensus.cc:2393] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 7: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:34.233654 12600 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:34.235877 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:34.211750 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 235567us
I20260709 12:11:34.221976 16910 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:34.246438 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:34.192294 16960 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:11:34.252947 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:34.169385 16962 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.252779 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:34.258039 12599 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 12: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:11:34.239853 16968 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.788s	user 0.004s	sys 0.000s
I20260709 12:11:34.255219 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:34.265288 12605 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:11:34.267233 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:34.269510 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:34.212951 16930 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.270613 12571 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:34.274164 12674 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:34.277611 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:34.274860 12678 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260709 12:11:34.245321 16652 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:34.283849 12599 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:34.280493 12598 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
W20260709 12:11:34.280836 12528 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2122) took 2946 ms (client timeout 1791 ms). Trace:
W20260709 12:11:34.285075 12528 rpcz_store.cc:269] 0709 12:11:31.319834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:31.493794 (+173960us) service_pool.cc:224] Handling call
0709 12:11:34.266399 (+2772605us) spinlock_profiling.cc:243] Waited 2.18 s on lock 0x7b540042a9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:34.266497 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5015319296}
I20260709 12:11:34.278867 12674 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
W20260709 12:11:34.263726 16887 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.587s	user 0.014s	sys 0.001s
I20260709 12:11:34.288301 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:34.280179 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:34.285964 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:34.289492 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:34.290072 12523 raft_consensus.cc:2376] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11. Re-sending same reply.
I20260709 12:11:34.290014 12528 raft_consensus.cc:2376] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12. Re-sending same reply.
I20260709 12:11:34.291209 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:34.291679 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:34.292747 12528 raft_consensus.cc:2368] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12.
I20260709 12:11:34.294792 16929 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:11:34.295609 16992 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:11:34.296128 16929 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:34.296579 16929 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:34.298792 16999 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:34.298622 16910 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.299422 16999 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:34.299746 12672 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:34.300369 16999 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 9.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.304067 16958 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:34.278759 16879 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:34.306777 16879 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:34.309118 16879 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:34.297585 16929 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.311529 16929 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:34.312431 17007 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:11:34.312865 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:34.279816 16070 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:34.317647 16070 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:34.318059 16070 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:34.322228 16990 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.164s	user 0.000s	sys 0.001s
W20260709 12:11:34.323486 16995 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:11:34.324270 16962 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:34.325007 16874 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:34.258482 16895 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 12:11:34.327858 16895 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:34.328274 16895 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:34.329203 16895 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:34.282229 16978 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.048s
I20260709 12:11:34.324174 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:34.293718 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:34.335253 12528 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:34.336795 16652 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.340615 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2384) took 1469 ms. Trace:
I20260709 12:11:34.343050 16855 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:11:34.347190 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 193801us
I20260709 12:11:34.341128 12606 rpcz_store.cc:276] 0709 12:11:32.870696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:32.870843 (+   147us) service_pool.cc:224] Handling call
0709 12:11:33.047654 (+176811us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:33.216168 (+168514us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:33.216222 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:33.219108 (+  2886us) log.cc:844] Serialized 52 byte log entry
0709 12:11:33.333404 (+114296us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:33.333466 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:34.340126 (+1006660us) spinlock_profiling.cc:243] Waited 160 ms on lock 0x7b54002f1a54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:34.340178 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:34.340226 (+    48us) raft_consensus.cc:1685] finished
0709 12:11:34.340239 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:34.340534 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":118,"raft.queue_time_us":173,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":5557,"spinlock_wait_cycles":367277824,"thread_start_us":56867,"threads_started":1,"wal-append.queue_time_us":173057}
I20260709 12:11:34.352428 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:34.353742 12605 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
W20260709 12:11:34.296609 16987 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.151s	user 0.000s	sys 0.001s
W20260709 12:11:34.295447 16993 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
W20260709 12:11:34.373598 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 61791.6us
I20260709 12:11:34.357499 12676 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:11:34.376837 12646 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:34.347208 16334 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:34.378079 12646 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:34.378527 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:34.389082 12523 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:34.377072 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:34.355032 16981 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:34.392835 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:34.396468 12599 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 12: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:11:34.397234 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:34.399099 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:34.391845 12528 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:11:34.401557 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:34.402395 12528 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:34.403821 12646 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:34.405306 12646 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:34.408906 12523 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:34.435602 12572 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:34.437490 12572 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:34.389858 16824 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:34.350879 16885 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.446581 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:34.355870 16789 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:11:34.455578 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:34.456399 12599 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:34.388343 16806 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260709 12:11:34.424573 16998 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:34.389099 16879 consensus_queue.cc:1059] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260709 12:11:34.388504 16875 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:11:34.445897 16801 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:34.389981 16561 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.392401 16334 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.470449 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:34.446372 16887 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:34.441694 17005 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:34.473090 12673 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:34.475227 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:34.485203 12528 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:34.424700 16981 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.492228 12674 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:34.461716 17006 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:34.470757 16824 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:34.490041 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:34.491835 17005 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:34.480948 16887 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:34.495635 16875 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:34.499341 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:34.502308 12572 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:34.496831 16943 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:34.506860 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 222165us
I20260709 12:11:34.482304 16801 raft_consensus.cc:697] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:34.511493 13302 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:34.491849 16806 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:34.512634 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:34.520136 12528 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:34.513046 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:34.521078 12605 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
W20260709 12:11:34.516309 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 69816.1us
I20260709 12:11:34.523444 12646 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:34.525753 12604 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:11:34.529215 12528 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:34.530288 12528 raft_consensus.cc:606] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:34.506470 16824 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:34.510027 16887 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:34.536630 12572 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:34.512372 16894 raft_consensus.cc:3055] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:34.523999 16910 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:34.543201 12677 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:34.543486 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:34.544552 12678 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:34.544711 16334 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:34.545229 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
W20260709 12:11:34.546633 17001 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 12:11:34.548520 16943 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:34.549536 17005 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.550552 16801 consensus_queue.cc:237] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.586229 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:34.586349 16875 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:34.552456 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:34.553222 13330 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:11:34.587872 13330 pending_rounds.cc:85] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 8
I20260709 12:11:34.588344 13330 pending_rounds.cc:107] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:11:34.588599 13330 raft_consensus.cc:2889] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:34.561151 16783 raft_consensus.cc:993] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: : Instructing follower 193a40da6e364ead8ab561e14c3cc8d1 to start an election
I20260709 12:11:34.563990 16935 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:34.473008 16998 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.536213 12599 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
W20260709 12:11:34.573717 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 105252us
I20260709 12:11:34.593715 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:34.537820 16919 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:34.549381 16070 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.541869 16824 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:34.562569 16885 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:34.594794 12528 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:34.596567 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:34.599892 12604 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:34.568388 16894 raft_consensus.cc:740] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:34.573048 16910 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:34.602003 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:34.607831 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:34.621990 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:34.623713 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:34.624876 16846 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:34.625260 12604 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:34.625504 16846 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:34.627744 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:34.629535 12602 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:11:34.640244 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:34.660084 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:34.662634 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:34.663713 12528 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
W20260709 12:11:34.700055 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:34.708694 12603 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:34.709406 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:34.710915 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:34.714390 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 115667us
W20260709 12:11:34.741093 16988 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.713s	user 0.002s	sys 0.000s
I20260709 12:11:34.618264 16783 raft_consensus.cc:1081] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Signalling peer 193a40da6e364ead8ab561e14c3cc8d1 to start an election
I20260709 12:11:34.816203 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 412) took 1125 ms. Trace:
I20260709 12:11:34.820266 12525 rpcz_store.cc:276] 0709 12:11:33.691006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:33.698033 (+  7027us) service_pool.cc:224] Handling call
0709 12:11:34.815920 (+1117887us) spinlock_profiling.cc:243] Waited 75 ms on lock 0x7b5400054954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:34.816106 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172565248}
I20260709 12:11:34.615669 16958 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:34.629287 16910 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:34.840839 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:34.601307 16935 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.865696 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:34.861266 12523 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 12:11:34.592042 17010 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.228s
I20260709 12:11:34.864130 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:34.871850 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:34.872725 12671 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:34.873805 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:34.877151 12604 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:34.887064 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:34.906806 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:34.908437 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:34.907967 12678 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:11:34.912665 12528 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 7. Current term is 8. Ops: []
I20260709 12:11:34.916574 12680 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:11:34.862908 16908 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:34.923688 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:34.863915 16930 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:34.926434 12680 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:34.928733 12528 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:34.825776 16885 consensus_queue.cc:1059] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.091s
I20260709 12:11:34.935050 12524 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:11:34.935031 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:34.955709 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:34.864058 16958 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:34.936041 16981 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:34.953593 16561 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:34.962889 16919 raft_consensus.cc:697] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:34.966193 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:34.982493 16885 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:34.986276 16885 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.986843 16911 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:34.987449 16919 consensus_queue.cc:237] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:34.988788 16885 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:34.863559 16846 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:34.907917 16894 consensus_queue.cc:260] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:34.984193 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:34.997318 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:35.009665 12646 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:35.011109 12646 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:35.026744 12646 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:34.963562 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 245628us
I20260709 12:11:35.028761 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:35.028949 12646 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:35.031339 12646 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:34.999344 17021 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:35.033138 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:35.035851 12531 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:35.031042 16885 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:35.043066 16885 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:35.043807 16885 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:35.044287 16885 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:35.025846 16680 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:35.045260 16885 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:35.052006 16839 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:35.052688 16839 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:35.032487 16998 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:35.054414 16998 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:11:35.055049 16998 consensus_queue.cc:1243] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:11:35.030602 16561 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:35.032835 16070 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:35.097162 13331 pending_rounds.cc:85] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 7
I20260709 12:11:35.112416 13331 pending_rounds.cc:107] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260709 12:11:35.053586 16839 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:35.069339 16211 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.951s	user 0.005s	sys 0.008s
W20260709 12:11:35.099506 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:35.112816 13331 raft_consensus.cc:2889] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:35.097436 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:35.123631 12526 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:35.123473 12531 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
W20260709 12:11:35.129395 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:35.128501 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 99815.5us
I20260709 12:11:35.136795 16895 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:35.138522 16935 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:35.129483 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 99390.9us
I20260709 12:11:35.148769 12646 leader_election.cc:400] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12.
I20260709 12:11:35.149395 12646 leader_election.cc:403] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:35.110683 16855 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:35.153527 12646 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:35.066660 17021 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:35.140776 17027 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:35.156409 13305 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:35.179203 12678 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:35.181592 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:35.130692 16211 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:35.192992 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:35.193657 12603 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:35.209265 12603 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:11:35.189229 16994 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:35.164043 16855 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:35.216573 12601 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:35.187917 17039 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:35.217286 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:35.224977 12678 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:35.222246 12572 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:35.202682 16801 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:35.184718 17042 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12.
I20260709 12:11:35.206025 16070 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:35.207427 16935 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:35.220670 17031 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:35.232807 12643 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:35.211548 17005 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:35.211740 17041 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:35.212805 16907 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:35.184958 17027 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:35.230679 17028 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:11:35.270329 13304 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.589s (SENT) [suppressed 16 similar messages]
I20260709 12:11:35.269769 12678 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:35.231462 17039 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:35.342934 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 110223us
I20260709 12:11:35.239320 16801 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:35.265558 17031 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:35.346312 16939 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:35.347021 16939 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:35.285714 17041 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:35.347543 16939 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:35.347944 17031 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:35.349190 17041 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:35.350134 17030 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:11:35.259346 16070 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:35.264266 16935 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:35.260016 16994 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.093s
I20260709 12:11:35.354789 17027 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:35.352552 17032 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:11:35.352305 17057 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:35.357487 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2234) took 1665 ms. Trace:
I20260709 12:11:35.293609 16907 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:35.361480 12673 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:11:35.364488 12597 rpcz_store.cc:276] 0709 12:11:33.691649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:33.691772 (+   123us) service_pool.cc:224] Handling call
0709 12:11:33.692112 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:33.700836 (+  8724us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:33.700865 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:33.703806 (+  2941us) log.cc:844] Serialized 26 byte log entry
0709 12:11:33.979026 (+275220us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:33.979079 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:35.357079 (+1378000us) spinlock_profiling.cc:243] Waited 8.31 ms on lock 0x7b5400079c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:35.357138 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:35.357192 (+    54us) raft_consensus.cc:1685] finished
0709 12:11:35.357203 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:35.357422 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82397,"raft.queue_time_us":658,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":4651,"spinlock_wait_cycles":19104000,"thread_start_us":274759,"threads_started":1,"wal-append.queue_time_us":342499,"wal-append.run_cpu_time_us":1714,"wal-append.run_wall_time_us":1003757}
I20260709 12:11:35.358549 16998 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:35.352473 16070 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:35.389941 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 119638us
I20260709 12:11:35.365411 17021 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:35.365767 17043 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:35.411311 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 236836us
I20260709 12:11:35.374719 17057 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:35.307646 17028 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:35.386714 16998 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:35.355465 17049 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.047s
I20260709 12:11:35.406030 16954 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:11:35.423903 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 218071us
I20260709 12:11:35.397796 16939 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:11:35.416424 16935 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:35.436415 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 193994us
I20260709 12:11:35.418392 16855 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:35.422075 17021 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:35.448014 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:35.421190 16211 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:35.426545 16998 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:35.398494 17052 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.060s
I20260709 12:11:35.454212 12571 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:35.447098 16887 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.869s	user 0.015s	sys 0.000s
I20260709 12:11:35.467880 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:35.469854 12678 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:11:35.417256 17043 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:35.470602 17047 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:11:35.452126 16839 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:35.437904 16954 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:35.468359 17039 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:35.459095 17021 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:35.477008 16887 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:35.469903 17056 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:35.485087 17057 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:35.536939 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 55715.2us
I20260709 12:11:35.483728 16726 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:35.541383 12603 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:11:35.484508 16935 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:35.541924 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:35.485214 16980 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:35.522215 16680 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260709 12:11:35.548450 17050 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
W20260709 12:11:35.591384 17038 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 12:11:35.591594 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:35.595033 16070 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:35.610554 16070 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:35.609961 12524 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:35.614940 16887 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:35.616149 16824 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.957s	user 0.011s	sys 0.004s
I20260709 12:11:35.616604 16824 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:35.617957 12680 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 13: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:35.618507 16954 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:35.545421 17027 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 12:11:35.619125 16954 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:35.619669 16954 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:35.625034 16801 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:35.627178 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2399) took 1494 ms. Trace:
I20260709 12:11:35.627846 12601 rpcz_store.cc:276] 0709 12:11:34.133216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:34.141887 (+  8671us) service_pool.cc:224] Handling call
0709 12:11:34.142648 (+   761us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:35.225094 (+1082446us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:35.626809 (+401715us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540007a154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:35.627084 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2463268096}
I20260709 12:11:35.536839 16837 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:35.602644 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 55258.1us
I20260709 12:11:35.609769 12679 raft_consensus.cc:2410] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 12:11:35.632129 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 217866us
I20260709 12:11:35.634227 12646 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:35.621800 16954 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:35.487010 16839 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:35.657608 12646 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:35.582749 16726 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:35.668380 12646 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:35.669799 12646 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:35.669498 12524 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:11:35.693868 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:35.694090 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:35.695015 12601 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:11:35.631662 17051 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:35.640587 16887 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:35.637748 16907 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:35.542979 17056 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:35.668545 17064 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:35.672894 17041 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:35.672708 16680 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:35.699019 12603 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:11:35.637823 16837 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:35.730926 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:35.731770 12522 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2151) took 1765 ms (client timeout 2004 ms). Trace:
I20260709 12:11:35.698089 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:35.734614 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 59310.8us
W20260709 12:11:35.732251 12522 rpcz_store.cc:269] 0709 12:11:33.966102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:35.730619 (+1764517us) service_pool.cc:224] Handling call
0709 12:11:35.731702 (+  1083us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:35.718653 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:35.736644 12679 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:35.746274 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 54339.7us
I20260709 12:11:35.733050 17028 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:35.753657 17073 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:11:35.755683 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 75934us
I20260709 12:11:35.756691 13339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 453) took 1197 ms. Trace:
I20260709 12:11:35.757143 13339 rpcz_store.cc:276] 0709 12:11:34.559677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:34.562139 (+  2462us) service_pool.cc:224] Handling call
0709 12:11:34.562666 (+   527us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:34.905139 (+342473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:34.905192 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:34.915656 (+ 10464us) log.cc:844] Serialized 26 byte log entry
0709 12:11:35.117953 (+202297us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:35.198684 (+ 80731us) log.cc:844] Serialized 14 byte log entry
0709 12:11:35.214580 (+ 15896us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:35.214779 (+   199us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:35.756500 (+541721us) raft_consensus.cc:1685] finished
0709 12:11:35.756517 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:35.756638 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":110184,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":80,"raft.queue_time_us":522688,"raft.run_cpu_time_us":320,"raft.run_wall_time_us":13368,"thread_start_us":626878,"threads_started":4,"wal-append.queue_time_us":387923}
I20260709 12:11:35.709964 17043 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:35.720345 17041 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:35.703895 16954 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:11:35.719743 16680 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:35.752524 16561 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.640s	user 0.011s	sys 0.005s
I20260709 12:11:35.750536 16824 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:35.745829 17059 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:35.755317 16978 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:35.733363 16726 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:35.778836 17066 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:11:35.779163 17078 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:35.714243 16887 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:35.715575 17072 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:11:35.703537 17074 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260709 12:11:35.775590 17041 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:35.776510 16680 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:35.714545 17051 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:35.784144 16978 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:35.796727 16726 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:35.806941 17071 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.105s	user 0.001s	sys 0.000s
I20260709 12:11:35.788182 16561 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:35.803323 16978 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:35.803390 17069 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:35.821398 16908 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:35.824337 17056 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:35.790920 16824 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260709 12:11:35.719745 17064 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:35.832175 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 134175us
I20260709 12:11:35.831868 16908 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:35.833791 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:35.791646 17078 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:35.841375 17083 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:35.843873 17076 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:35.844600 17076 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:35.835307 16879 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.347s	user 0.000s	sys 0.022s
W20260709 12:11:35.864347 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 116779us
I20260709 12:11:35.836388 16839 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:35.788654 17059 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:35.861658 16879 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.377s	user 0.000s	sys 0.023s
W20260709 12:11:35.870847 17070 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
W20260709 12:11:35.885552 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:35.885617 16954 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 7.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:35.887243 16726 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Stepping down as leader of term 7
I20260709 12:11:35.887658 16726 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:35.888427 16726 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:35.890450 16954 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:11:35.902835 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:35.908815 12528 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:35.909369 12528 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:35.910282 12528 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:35.911545 12528 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:35.928881 17082 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:35.929900 17083 raft_consensus.cc:3055] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:35.929085 17079 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:35.930449 17083 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:35.945811 17076 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:35.930132 17019 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.138s
W20260709 12:11:35.939116 16910 consensus_meta.cc:287] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.052s	user 0.014s	sys 0.003s
I20260709 12:11:35.953223 12679 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:35.938567 16908 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:35.958163 16978 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:35.959494 16895 consensus_meta.cc:287] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.822s	user 0.014s	sys 0.003s
W20260709 12:11:35.955250 17086 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.078s	user 0.001s	sys 0.000s
I20260709 12:11:35.938630 16726 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:35.965013 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:35.963462 16887 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:35.967309 17003 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 1.654s	user 0.000s	sys 0.015s
I20260709 12:11:35.964802 16958 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:35.950976 16837 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:35.951331 16824 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:36.025379 16824 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:35.979775 17051 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:35.981148 17041 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:35.983422 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:35.989248 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:35.993441 17064 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:35.995733 16908 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:35.997689 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2108) took 2320 ms. Trace:
I20260709 12:11:35.964103 17076 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.055s
I20260709 12:11:36.039590 12529 rpcz_store.cc:276] 0709 12:11:33.677246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:33.682827 (+  5581us) service_pool.cc:224] Handling call
0709 12:11:33.683348 (+   521us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:33.702702 (+ 19354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:33.702735 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:33.703869 (+  1134us) log.cc:844] Serialized 52 byte log entry
0709 12:11:35.268601 (+1564732us) spinlock_profiling.cc:243] Waited 11.9 ms on lock 0x7b180033c280. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:11:35.585588 (+316987us) raft_consensus.cc:1645] Marking committed up to 3
0709 12:11:35.585657 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:35.586035 (+   378us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:35.997239 (+411204us) raft_consensus.cc:1685] finished
0709 12:11:35.997267 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:35.997583 (+   316us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10336,"raft.queue_time_us":783,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":1672,"spinlock_wait_cycles":27372928,"thread_start_us":316652,"threads_started":1,"wal-append.queue_time_us":1988756}
I20260709 12:11:36.016736 17083 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:35.999981 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:36.049024 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2243) took 1453 ms. Trace:
I20260709 12:11:36.049759 17041 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:36.050974 12676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 404) took 2239 ms (client timeout 1589 ms). Trace:
W20260709 12:11:36.051582 12676 rpcz_store.cc:269] 0709 12:11:33.811714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:33.812014 (+   300us) service_pool.cc:224] Handling call
0709 12:11:36.050700 (+2238686us) spinlock_profiling.cc:243] Waited 315 ms on lock 0x7b54000c3ad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:36.050871 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":724730112}
I20260709 12:11:36.052510 16824 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:36.000555 17059 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:35.928972 17015 consensus_queue.cc:1059] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.127s
I20260709 12:11:36.053247 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:36.024528 17078 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:35.965109 16910 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:36.081007 16910 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:36.015478 17056 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:36.082604 17056 raft_consensus.cc:697] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:36.084021 17056 consensus_queue.cc:237] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:36.028134 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2517) took 1680 ms. Trace:
I20260709 12:11:36.086338 12674 rpcz_store.cc:276] 0709 12:11:34.347531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:34.347661 (+   130us) service_pool.cc:224] Handling call
0709 12:11:34.347961 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:34.492706 (+144745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:36.027837 (+1535131us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b54000ab9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:36.028062 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":331301120}
I20260709 12:11:36.088235 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:35.999573 17021 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:36.031477 12528 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:11:36.033144 12526 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:36.034020 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:36.092126 12679 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:11:36.093354 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:35.954567 17079 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:36.094426 12528 pending_rounds.cc:85] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 6
I20260709 12:11:36.095247 12528 pending_rounds.cc:107] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 11.7
W20260709 12:11:36.012061 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47121.9us
I20260709 12:11:36.095750 12528 raft_consensus.cc:2889] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:36.089833 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:36.054378 16998 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.585s	user 0.006s	sys 0.008s
I20260709 12:11:36.056383 12672 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:11:36.049593 12602 rpcz_store.cc:276] 0709 12:11:34.595873 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:34.596049 (+   176us) service_pool.cc:224] Handling call
0709 12:11:36.048799 (+1452750us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b540022ecd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:36.048941 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29458816}
I20260709 12:11:36.097101 12679 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:36.098070 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:36.098925 12602 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:36.099390 17079 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:36.057195 16978 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:36.059197 12676 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:36.059670 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:36.100384 16998 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:36.101069 12603 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:36.101284 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:36.102190 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:36.102571 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:36.102854 12603 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:36.103487 16680 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:36.099968 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:36.104784 12602 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:36.103858 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:36.105818 12676 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:36.108774 12526 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:11:36.110682 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:36.110615 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:36.110988 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:36.112278 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:36.112464 12674 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:36.112481 12679 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:36.113075 12676 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:36.112088 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:36.113512 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:36.114276 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:36.114238 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:36.115237 12676 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:36.115483 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:36.114544 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:36.118027 12528 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:36.118366 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:36.119242 12525 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:36.119328 17017 raft_consensus.cc:3055] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:36.119834 17017 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:36.119911 17059 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:36.120397 17059 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:36.120815 17059 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:36.121448 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:36.121919 12679 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:36.123895 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:36.124554 12529 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:36.126106 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:36.126475 16998 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:36.121820 16908 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:36.127252 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:36.127892 16908 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:36.128332 16908 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:36.128715 16879 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:36.129062 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:36.129325 16879 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:36.131909 16879 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:36.132027 16977 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:11:36.132510 16879 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:36.134197 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:36.133013 17041 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:36.142745 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:36.139825 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:36.145781 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:36.149659 12676 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:36.152796 12674 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:36.117460 16726 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:36.144693 16879 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:36.136999 16680 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:36.144460 16977 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:36.166955 12572 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:36.169267 12572 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:36.171355 16958 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:36.179352 12679 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:11:36.146942 17041 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:36.167555 16977 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:36.166035 12674 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:51308
I20260709 12:11:36.186327 12674 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:11:36.187093 12674 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:36.164233 16726 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:36.192714 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2519) took 1766 ms. Trace:
I20260709 12:11:36.193336 12677 rpcz_store.cc:276] 0709 12:11:34.426572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:34.426796 (+   224us) service_pool.cc:224] Handling call
0709 12:11:34.447974 (+ 21178us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:34.573346 (+125372us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:36.192282 (+1618936us) spinlock_profiling.cc:243] Waited 94 ms on lock 0x7b54000c35d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:36.192593 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":216244608}
I20260709 12:11:36.184487 17064 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:36.200863 16893 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:36.201407 17041 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:36.190294 16977 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:36.203226 17006 raft_consensus.cc:3055] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Stepping down as leader of term 8
W20260709 12:11:36.214973 12598 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 2403) took 2071 ms (client timeout 1739 ms). Trace:
W20260709 12:11:36.215706 12598 rpcz_store.cc:269] 0709 12:11:34.143972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:34.147519 (+  3547us) service_pool.cc:224] Handling call
0709 12:11:36.214707 (+2067188us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b54003a11d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:36.214857 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":299164416}
I20260709 12:11:36.217324 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:36.182143 16680 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:36.221485 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:36.222491 16997 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:36.232343 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:36.239182 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 493075us
I20260709 12:11:36.218550 17006 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:36.250867 12674 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:36.263073 17014 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.768s	user 0.005s	sys 0.004s
I20260709 12:11:36.242300 17074 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:36.289490 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:36.278172 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:36.246335 16997 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:36.200924 17064 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:36.263337 16977 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:36.249534 16879 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:36.241247 16680 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:36.312500 17074 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:36.312022 17052 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.009s	sys 0.025s
I20260709 12:11:36.309648 17073 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:36.357937 16726 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:36.369688 17052 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.607s	user 0.009s	sys 0.025s
I20260709 12:11:36.313277 16997 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:36.353948 17025 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:36.402653 12496 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.026s	sys 0.013s
W20260709 12:11:36.404393 12496 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.026s	sys 0.013s
I20260709 12:11:36.379155 16536 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:36.411947 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 114858us
I20260709 12:11:36.411340 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:36.358572 17006 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:36.413306 12597 raft_consensus.cc:2368] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10.
I20260709 12:11:36.430573 12674 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:36.414093 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 524788us
I20260709 12:11:36.437884 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:36.438663 12496 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:36.438727 13331 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:36.441556 12496 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:36.443305 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:36.447546 12496 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:36.448789 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:36.445199 16879 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:36.460117 12597 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:36.456033 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:36.379691 17059 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:36.405226 16977 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:36.460433 16879 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:36.472172 16726 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:36.494443 16726 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:36.445668 16789 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:36.493701 12597 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 6. (index mismatch)
I20260709 12:11:36.473287 12602 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:11:36.507005 12602 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:36.508821 12602 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:36.472625 17006 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:36.403136 17025 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:36.449241 16536 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:36.513892 16789 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:36.454988 16839 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:36.449009 16671 raft_consensus.cc:2764] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
W20260709 12:11:36.547770 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 406343us
W20260709 12:11:36.597226 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:36.597226 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:36.611001 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2530) took 1629 ms. Trace:
I20260709 12:11:36.611768 16789 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:36.611544 12680 rpcz_store.cc:276] 0709 12:11:34.981290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:34.983781 (+  2491us) service_pool.cc:224] Handling call
0709 12:11:36.610758 (+1626977us) spinlock_profiling.cc:243] Waited 623 ms on lock 0x7b540008eb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:36.610926 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1433244928}
I20260709 12:11:36.616330 12571 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:36.647572 16958 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:36.648105 16958 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:36.648442 16958 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:36.649166 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:36.650327 16998 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:36.650836 16998 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:36.651211 16998 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:36.652058 16879 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:36.613526 16977 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:36.652854 16977 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:36.653216 16977 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:36.600171 16726 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:36.658197 16719 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:36.658792 16719 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:36.650375 17025 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:36.672043 16719 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:36.672960 17090 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.267s	user 0.007s	sys 0.001s
I20260709 12:11:36.674911 17064 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:36.661441 16911 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:36.676396 16998 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:36.676515 16911 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:36.595942 16839 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:36.684032 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:36.667039 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:36.738952 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 72041.6us
I20260709 12:11:36.740321 12571 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:36.753722 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:36.786597 16998 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:36.679606 16908 consensus_meta.cc:287] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.551s	user 0.014s	sys 0.001s
W20260709 12:11:36.788961 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48810.3us
I20260709 12:11:36.794561 13302 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:36.790093 17074 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:36.797217 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2227) took 3139 ms. Trace:
I20260709 12:11:36.800495 12600 rpcz_store.cc:276] 0709 12:11:33.684768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:33.684969 (+   201us) service_pool.cc:224] Handling call
0709 12:11:33.685490 (+   521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:34.243740 (+558250us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:36.796809 (+2553069us) spinlock_profiling.cc:243] Waited 548 ms on lock 0x7b540023fd54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:36.797105 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1259420416}
I20260709 12:11:36.808765 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:36.814947 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:36.815902 12602 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:11:36.792079 16893 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:36.828846 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:36.832314 12602 raft_consensus.cc:2410] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 10 because replica has last-logged OpId of term: 9 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260709 12:11:36.796576 17027 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:36.829224 16797 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:36.838582 12602 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:36.811563 16680 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:36.828773 17025 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:36.850785 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:36.806327 16908 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:36.854461 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2536) took 1333 ms. Trace:
W20260709 12:11:36.869647 12603 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.008s	sys 0.020s
I20260709 12:11:36.835309 17092 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:36.876073 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:36.873492 12603 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.526s	user 0.009s	sys 0.020s
I20260709 12:11:36.838984 16893 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:36.881803 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2251) took 1021 ms. Trace:
I20260709 12:11:36.835911 17076 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:11:36.882251 12603 rpcz_store.cc:276] 0709 12:11:35.860717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:36.103610 (+242893us) service_pool.cc:224] Handling call
0709 12:11:36.103925 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:36.104157 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:11:36.347552 (+243395us) log.cc:844] Serialized 14 byte log entry
0709 12:11:36.881426 (+533874us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:36.881480 (+    54us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:36.881531 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:36.881596 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:36.881743 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4131,"prepare.queue_time_us":497189,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":5516,"thread_start_us":772294,"threads_started":2}
I20260709 12:11:36.900601 16070 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:36.901315 16958 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:36.846606 16797 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:36.903062 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 409) took 2457 ms. Trace:
I20260709 12:11:36.903765 16958 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:36.903484 12672 rpcz_store.cc:276] 0709 12:11:34.445458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:34.445581 (+   123us) service_pool.cc:224] Handling call
0709 12:11:36.055968 (+1610387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:36.096984 (+ 41016us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:36.902826 (+805842us) spinlock_profiling.cc:243] Waited 1.6 s on lock 0x7b54000c3ad0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:36.902988 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3690803968}
I20260709 12:11:36.881026 16839 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:36.906564 12597 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:36.882598 17092 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:36.911168 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:36.908584 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:36.905102 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:36.912778 12672 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:36.874215 17091 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:36.914489 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:36.912750 13331 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:11:36.903766 16797 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:36.921433 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:36.927574 12603 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:11:36.875164 17079 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:11:36.931653 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:36.933456 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:36.935678 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:36.898036 17059 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:36.872880 17025 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:36.943909 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:36.940389 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:36.920166 17091 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:36.948585 12602 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:36.948437 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:36.954948 12603 raft_consensus.cc:2393] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 8: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:36.937892 17010 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:36.917959 17092 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:36.964044 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:36.958968 17010 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:36.951890 17091 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:36.975759 12602 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:36.983006 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:36.994693 12603 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:36.995752 12602 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: []
W20260709 12:11:36.999969 12494 connection.cc:256] RPC call timeout handler was delayed by -2.2126263170000584s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1574s for a response
W20260709 12:11:36.986907 16997 consensus_meta.cc:287] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.589s	user 0.012s	sys 0.000s
W20260709 12:11:37.000841 12494 connection.cc:256] RPC call timeout handler was delayed by -1.9381263170000693s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1879s for a response
W20260709 12:11:37.002213 12494 connection.cc:256] RPC call timeout handler was delayed by -1.9304435720000583s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15810000000000002s for a response
W20260709 12:11:37.003415 12494 connection.cc:256] RPC call timeout handler was delayed by -1.9111263170000257s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19090000000000001s for a response
I20260709 12:11:37.001698 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:37.003870 12494 connection.cc:256] RPC call timeout handler was delayed by -1.7936435720000645s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1733s for a response
W20260709 12:11:37.004405 12494 connection.cc:256] RPC call timeout handler was delayed by -1.7882435720000558s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17390000000000003s for a response
I20260709 12:11:37.003736 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:36.999718 17018 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.011s	sys 0.028s
I20260709 12:11:36.966176 16893 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:37.002874 16999 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.684s	user 0.026s	sys 0.033s
I20260709 12:11:36.999297 16680 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.008020 12603 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:37.002914 16726 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.004240 16720 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:11:37.022327 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 320958us
W20260709 12:11:37.023500 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:37.024085 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:37.023834 12494 connection.cc:256] RPC call timeout handler was delayed by -1.7367309390001537s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1658s for a response
I20260709 12:11:37.015806 12602 raft_consensus.cc:2410] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 11 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 12:11:37.027010 12494 connection.cc:256] RPC call timeout handler was delayed by -1.6721435720000954s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18680000000000002s for a response
W20260709 12:11:37.027748 12494 connection.cc:256] RPC call timeout handler was delayed by -1.5810309390001294s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1831s for a response
W20260709 12:11:37.028302 12494 connection.cc:256] RPC call timeout handler was delayed by -1.5243309390000377s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1894s for a response
W20260709 12:11:37.028805 12494 connection.cc:256] RPC call timeout handler was delayed by -1.4854775720000362s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17290000000000003s for a response
W20260709 12:11:37.029345 12494 connection.cc:256] RPC call timeout handler was delayed by -1.4426722579999023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15940000000000001s for a response
W20260709 12:11:37.029850 12494 connection.cc:256] RPC call timeout handler was delayed by -1.4165435720001369s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21520000000000003s for a response
W20260709 12:11:37.015784 16999 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.700s	user 0.026s	sys 0.033s
I20260709 12:11:37.030867 12603 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:11:37.034686 12603 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:11:37.010645 16797 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:37.020088 16996 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 2.688s	user 0.004s	sys 0.012s
W20260709 12:11:37.030435 12494 connection.cc:256] RPC call timeout handler was delayed by -1.3823435720000816s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.219s for a response
I20260709 12:11:37.016410 17010 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:37.041283 12494 connection.cc:256] RPC call timeout handler was delayed by -1.0759775720000562s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21840000000000004s for a response
I20260709 12:11:37.006193 16997 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.043746 12606 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:37.049005 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:37.034021 16720 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.041262 16908 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:37.056763 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:37.042339 16797 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.032841 16680 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.031970 16726 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.071281 12672 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:11:37.075229 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 402) took 3286 ms. Trace:
I20260709 12:11:37.083307 12494 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:37.080344 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:37.054899 16719 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:37.059500 12598 raft_consensus.cc:2368] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 10. Current term is 11.
I20260709 12:11:37.084115 13330 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:37.083561 12675 rpcz_store.cc:276] 0709 12:11:33.789053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:33.789206 (+   153us) service_pool.cc:224] Handling call
0709 12:11:33.834704 (+ 45498us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:33.835279 (+   575us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:33.835325 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:33.866787 (+ 31462us) log.cc:844] Serialized 26 byte log entry
0709 12:11:34.084153 (+217366us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:34.222619 (+138466us) log.cc:844] Serialized 14 byte log entry
0709 12:11:34.223589 (+   970us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:34.223681 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:34.321396 (+ 97715us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b1800425200. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:11:36.950349 (+2628953us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:37.074978 (+124629us) raft_consensus.cc:1685] finished
0709 12:11:37.075002 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:37.075156 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3494,"prepare.queue_time_us":167613,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":46,"raft.queue_time_us":841,"raft.run_cpu_time_us":290,"raft.run_wall_time_us":25567,"spinlock_wait_cycles":47261440,"thread_start_us":280165,"threads_started":2,"wal-append.queue_time_us":349954}
I20260709 12:11:37.088069 16720 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:37.088750 16671 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:37.086328 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:37.090034 16680 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:37.090204 16671 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:37.091454 12494 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:37.092325 16671 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:37.092612 16797 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:37.093428 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:37.093673 12494 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:37.094178 12675 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:37.058996 17025 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:37.095317 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:37.088177 16726 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:37.097431 16671 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:11:37.099939 17100 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 12:11:37.100597 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:37.106180 12603 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:11:37.107618 12602 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:37.109088 17088 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:11:37.109627 16997 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:37.111435 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:37.112614 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:37.110931 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:37.114629 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:37.114156 16879 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:37.119939 16879 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.122323 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:37.122496 12677 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:11:37.122668 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:37.123795 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:37.125478 12494 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:37.130774 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:37.131511 12600 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:37.131947 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:37.132618 12603 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:37.134352 12494 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:37.134534 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:37.136111 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:37.139264 16719 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:37.138434 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:37.140920 16719 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:37.140793 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:37.142562 16719 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:11:37.143548 16671 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:37.144567 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:37.145008 16719 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260709 12:11:37.146306 17101 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.137s	user 0.001s	sys 0.000s
I20260709 12:11:37.146929 12494 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:37.096047 17074 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:37.150454 12675 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:37.151687 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:37.152355 12675 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:11:37.153208 16879 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:37.153550 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:37.152366 13330 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:11:37.130847 17018 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:37.156621 17018 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:37.157058 17018 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:37.158236 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:37.158850 12603 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:37.159731 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:37.160410 12672 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 13: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:11:37.162369 17104 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.111s	user 0.001s	sys 0.000s
W20260709 12:11:37.164934 16977 consensus_meta.cc:287] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.511s	user 0.012s	sys 0.000s
I20260709 12:11:37.166462 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:37.167948 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2133) took 1475 ms. Trace:
I20260709 12:11:37.168452 12679 rpcz_store.cc:276] 0709 12:11:35.692886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:36.182484 (+489598us) service_pool.cc:224] Handling call
0709 12:11:36.182933 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:36.220691 (+ 37758us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:36.220736 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:36.220784 (+    48us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:36.220852 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:37.167641 (+946789us) spinlock_profiling.cc:243] Waited 23.2 ms on lock 0x7b54000131d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:37.167703 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:37.167875 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14132,"raft.queue_time_us":14644,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":9495,"spinlock_wait_cycles":53312256}
W20260709 12:11:37.170835 17106 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:11:37.170810 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:37.174971 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:37.175390 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:37.176962 12600 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:11:37.177218 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:37.178975 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:37.179834 12522 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:37.183123 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:37.183646 12530 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:37.184190 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:37.185241 12531 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:37.186033 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:37.186920 12522 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:37.182914 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:37.188591 12526 raft_consensus.cc:2410] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 13 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:11:37.171379 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2534) took 2166 ms. Trace:
I20260709 12:11:37.189579 12678 rpcz_store.cc:276] 0709 12:11:35.004835 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:35.295800 (+290965us) service_pool.cc:224] Handling call
0709 12:11:35.314597 (+ 18797us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:35.470421 (+155824us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:37.171014 (+1700593us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b54000f00d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:37.171286 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":356236672}
I20260709 12:11:37.190603 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:37.191537 12679 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 10.
I20260709 12:11:37.196120 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:37.199941 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:37.201932 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:37.202834 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:37.204675 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:37.204870 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:37.206771 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:37.207415 12530 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:37.156773 16719 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:37.208950 12572 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:37.214563 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:37.217275 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:37.220258 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:37.219287 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:37.211645 16962 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:37.223379 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:37.166404 17083 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:37.223990 12524 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:37.224566 12529 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 11: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:37.225427 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:37.225618 12572 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:37.237859 12494 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:37.238310 12530 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:37.239876 12530 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:11:37.239959 12572 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:37.222296 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:37.242405 12675 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15.
I20260709 12:11:37.240594 12524 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:37.208169 16797 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:37.211969 16908 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:37.252647 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:37.246765 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:37.249929 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:37.253600 12678 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 10.
I20260709 12:11:37.258601 16720 raft_consensus.cc:2764] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:11:37.260681 12494 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:37.262789 16720 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:37.264396 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:37.265275 12531 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:37.268719 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:37.269236 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 54760.5us
I20260709 12:11:37.237605 16680 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:37.236100 16999 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:37.210574 16977 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.278110 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:37.278558 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:37.279695 12531 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:37.279196 12529 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:37.274032 16680 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:37.271615 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:37.252048 16719 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:37.245693 17059 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:37.251322 16671 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:37.249219 16893 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:37.290506 16893 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:37.291005 16893 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:37.294775 17013 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 2.797s	user 0.000s	sys 0.012s
I20260709 12:11:37.295922 17077 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:37.296912 17077 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:37.276438 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:37.297998 12531 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:37.277403 12675 raft_consensus.cc:2393] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 13: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:37.284265 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:37.243713 16962 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:37.289675 16671 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:37.275988 16999 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:37.303233 12572 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:37.284490 16680 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:37.286506 16789 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:37.316447 12572 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:37.318620 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:37.318691 16797 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:37.320806 16962 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:37.313701 16789 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:37.322674 16789 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:37.313311 16879 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:37.327862 12643 leader_election.cc:400] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10.
I20260709 12:11:37.328378 12643 leader_election.cc:403] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Cancelling election due to peer responding with higher term
I20260709 12:11:37.306227 16999 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:37.333879 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:37.337759 12572 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:37.337934 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:37.338948 12572 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:37.297888 17077 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:37.340065 17077 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:37.313560 17059 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:37.341876 16797 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:11:37.343273 17067 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.752s	user 0.003s	sys 0.000s
I20260709 12:11:37.343667 16774 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:11:37.332216 16720 raft_consensus.cc:697] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:37.350303 16962 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:37.351312 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 399) took 2160 ms. Trace:
I20260709 12:11:37.359587 12604 rpcz_store.cc:276] 0709 12:11:35.191104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:35.194218 (+  3114us) service_pool.cc:224] Handling call
0709 12:11:35.204309 (+ 10091us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:35.307710 (+103401us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:11:35.307773 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:35.308438 (+   665us) log.cc:844] Serialized 26 byte log entry
0709 12:11:37.155195 (+1846757us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b1800311ce0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:11:37.219215 (+ 64020us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:37.219258 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:37.219329 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:37.350939 (+131610us) raft_consensus.cc:1685] finished
0709 12:11:37.350961 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:37.351226 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17427,"raft.queue_time_us":89794,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":33262,"spinlock_wait_cycles":27584768,"thread_start_us":144448,"threads_started":2,"wal-append.queue_time_us":1961666}
I20260709 12:11:37.336455 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:37.365549 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:11:37.374303 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 106222us
I20260709 12:11:37.310316 17092 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:37.378577 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:37.335830 17114 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10.
I20260709 12:11:37.382762 12572 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:37.383723 17122 consensus_queue.cc:1059] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:37.384123 17114 consensus_queue.cc:1059] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:11:37.384817 17074 raft_consensus.cc:3055] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:37.385224 17074 raft_consensus.cc:740] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:37.386049 17074 consensus_queue.cc:260] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.387389 12572 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:37.368462 16797 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:11:37.383255 17006 consensus_meta.cc:287] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.820s	user 0.015s	sys 0.001s
I20260709 12:11:37.377053 17059 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:37.379211 16874 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:37.359915 16908 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
W20260709 12:11:37.388475 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:37.396598 13331 raft_consensus.cc:2368] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13.
I20260709 12:11:37.350569 16720 consensus_queue.cc:237] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:37.399236 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:37.383181 16726 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:37.387164 16962 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:37.393435 16958 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:37.394411 16997 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:37.411732 16726 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:37.413691 16797 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.421988 16874 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:37.433002 12599 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:11:37.399974 17074 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:37.381722 16879 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.029s
I20260709 12:11:37.422107 16958 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:37.437372 12603 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: [12.8-12.8]
I20260709 12:11:37.466046 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:37.481752 12604 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:37.425351 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:37.478874 16789 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.456030 16726 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:37.505146 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 61236.5us
I20260709 12:11:37.507658 12599 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:37.509927 12604 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:37.411453 17120 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.092s
I20260709 12:11:37.479939 16958 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:37.476771 16977 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:37.453877 17118 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.092s
W20260709 12:11:37.523579 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:37.504276 17125 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:37.512804 16997 raft_consensus.cc:697] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:37.547526 16789 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:37.539163 16977 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.548728 16977 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.550984 16719 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:37.551292 16977 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:37.551597 16719 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:37.552580 16719 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.7, Last appended by leader: 7, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.553325 16977 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.553694 16719 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:37.530454 17125 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.534211 17018 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:37.526098 16874 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.579010 17092 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:37.554025 16977 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:37.574734 16680 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.607128 12672 raft_consensus.cc:2368] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 11. Current term is 12.
I20260709 12:11:37.613922 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2136) took 1906 ms. Trace:
I20260709 12:11:37.614437 12677 rpcz_store.cc:276] 0709 12:11:35.707058 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:36.223420 (+516362us) service_pool.cc:224] Handling call
0709 12:11:36.223797 (+   377us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:37.122987 (+899190us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:37.610588 (+487601us) spinlock_profiling.cc:243] Waited 898 ms on lock 0x7b5400010c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:37.613849 (+  3261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2065978624}
W20260709 12:11:37.619604 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:37.593099 17130 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:37.652729 17058 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:37.653450 17058 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.655730 17058 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:37.658870 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 429) took 2211 ms. Trace:
W20260709 12:11:37.664758 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:36.872221 12676 rpcz_store.cc:276] 0709 12:11:35.521532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:36.150809 (+629277us) service_pool.cc:224] Handling call
0709 12:11:36.151171 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:36.151443 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:11:36.421436 (+269993us) log.cc:844] Serialized 14 byte log entry
0709 12:11:36.853923 (+432487us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:36.854016 (+    93us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:36.854084 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:36.854212 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:36.854377 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":17505,"thread_start_us":688668,"threads_started":2}
I20260709 12:11:37.676424 16958 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.677475 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2532) took 2673 ms. Trace:
I20260709 12:11:37.678231 17127 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:37.678857 17127 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.679538 16958 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:37.680791 12569 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:37.682554 16908 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:11:37.677974 17091 consensus_meta.cc:287] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.688s	user 0.008s	sys 0.005s
I20260709 12:11:37.683451 17018 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:37.683619 17091 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:37.684103 16908 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:37.646749 17130 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.686704 16977 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:37.685631 16908 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:37.662431 17135 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.687405 16759 raft_consensus.cc:3055] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:37.687894 16759 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:37.688308 16680 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:37.687671 17135 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.688910 16759 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:37.691119 16893 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:37.693018 17124 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
I20260709 12:11:37.607134 16997 consensus_queue.cc:237] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.660161 17094 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:37.696872 17033 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 2.463s	user 0.004s	sys 0.001s
I20260709 12:11:37.698453 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2175) took 3156 ms. Trace:
I20260709 12:11:37.699596 17130 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:37.698843 12523 rpcz_store.cc:276] 0709 12:11:34.541775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:34.871284 (+329509us) service_pool.cc:224] Handling call
0709 12:11:34.871725 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:34.872206 (+   481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:34.872244 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:34.926887 (+ 54643us) log.cc:844] Serialized 26 byte log entry
0709 12:11:35.156743 (+229856us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:35.156812 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:35.156915 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:37.695992 (+2539077us) spinlock_profiling.cc:243] Waited 4.69 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:37.698092 (+  2100us) raft_consensus.cc:1685] finished
0709 12:11:37.698150 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:37.698380 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":117,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":25265,"spinlock_wait_cycles":10775936,"thread_start_us":228696,"threads_started":1,"wal-append.queue_time_us":262014}
I20260709 12:11:37.699909 16846 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:37.700855 17135 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:37.677974 12673 rpcz_store.cc:276] 0709 12:11:35.004227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:35.252534 (+248307us) service_pool.cc:224] Handling call
0709 12:11:35.284288 (+ 31754us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:35.367322 (+ 83034us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:37.677200 (+2309878us) spinlock_profiling.cc:243] Waited 74.7 ms on lock 0x7b54003d40d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:37.677401 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":171855232}
I20260709 12:11:37.702943 17125 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:37.705329 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:37.660591 12527 rpcz_store.cc:276] 0709 12:11:35.447545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:35.448666 (+  1121us) service_pool.cc:224] Handling call
0709 12:11:35.465670 (+ 17004us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:35.466079 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:35.466108 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:35.466859 (+   751us) log.cc:844] Serialized 26 byte log entry
0709 12:11:37.613285 (+2146426us) spinlock_profiling.cc:243] Waited 2.5 ms on lock 0x7b18004904e0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:11:37.657384 (+ 44099us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:37.657411 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:37.657498 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:37.658113 (+   615us) raft_consensus.cc:1685] finished
0709 12:11:37.658164 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:37.658781 (+   617us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":241,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":1595,"spinlock_wait_cycles":5742976,"thread_start_us":43937,"threads_started":1,"wal-append.queue_time_us":2189462}
I20260709 12:11:37.714927 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:37.720480 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:37.689935 16759 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:37.727836 12599 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:37.729031 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:37.730216 17056 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:37.731005 17056 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:37.732165 17056 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.734715 17091 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:37.736188 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:37.740061 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:37.741837 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:37.742707 12530 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:37.760638 12530 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:37.776528 17021 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.777695 17021 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:37.779724 17127 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:37.780632 13338 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:37.785733 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2150) took 1986 ms. Trace:
I20260709 12:11:37.786280 12680 rpcz_store.cc:276] 0709 12:11:35.799705 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:36.642678 (+842973us) service_pool.cc:224] Handling call
0709 12:11:36.654270 (+ 11592us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:36.815688 (+161418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:36.815743 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:36.827980 (+ 12237us) log.cc:844] Serialized 52 byte log entry
0709 12:11:37.043345 (+215365us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:37.073013 (+ 29668us) log.cc:844] Serialized 14 byte log entry
0709 12:11:37.074791 (+  1778us) log.cc:844] Serialized 14 byte log entry
0709 12:11:37.075839 (+  1048us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:37.075947 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:37.782925 (+706978us) spinlock_profiling.cc:243] Waited 879 us on lock 0x7b180049dce0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:11:37.785441 (+  2516us) raft_consensus.cc:1685] finished
0709 12:11:37.785472 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:37.785655 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":166985,"prepare.queue_time_us":29648,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":90,"raft.queue_time_us":162727,"raft.run_cpu_time_us":425,"raft.run_wall_time_us":9477,"spinlock_wait_cycles":2022144,"thread_start_us":191384,"threads_started":1,"wal-append.queue_time_us":243893}
I20260709 12:11:37.764482 16999 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.790617 16999 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:37.773573 16726 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.787571 17021 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:37.796555 16958 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:37.797128 16958 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.801713 12599 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:11:37.812259 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:37.815056 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 160099us
I20260709 12:11:37.767771 16893 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:37.813720 17029 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:37.819109 17018 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:37.820245 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:37.820776 17134 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.821153 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:37.821792 16874 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:37.822530 12676 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: [10.7-12.8]
I20260709 12:11:37.854007 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:37.852577 17018 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.855584 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:37.856810 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:37.822695 16824 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.857587 17018 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:37.857738 16824 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:37.812465 16894 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:37.860980 12678 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:37.823946 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:37.861793 12675 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 7.
I20260709 12:11:37.820029 17041 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:37.827661 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:37.865810 12679 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 10: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:11:37.830272 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 450005us
I20260709 12:11:37.867746 16824 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:37.868268 16943 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:37.869614 16958 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:37.860641 17029 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 11 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:37.874948 17094 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.875734 17094 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:37.876106 17029 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:37.843475 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2415) took 3491 ms. Trace:
I20260709 12:11:37.816264 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:37.878700 12606 rpcz_store.cc:276] 0709 12:11:34.351645 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:34.352043 (+   398us) service_pool.cc:224] Handling call
0709 12:11:34.374570 (+ 22527us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:37.049842 (+2675272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:37.843208 (+793366us) spinlock_profiling.cc:243] Waited 2.67 s on lock 0x7b5400230d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:37.843404 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6138178944}
I20260709 12:11:37.823560 12677 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: [8.7-8.7]
I20260709 12:11:37.861546 12676 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
W20260709 12:11:37.881067 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:37.883332 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:37.884759 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:37.884652 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:37.886056 12530 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
I20260709 12:11:37.886739 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:37.892468 12527 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:11:37.887816 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:37.894225 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:37.896431 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:37.897996 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:37.898819 12525 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:37.899163 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:37.899820 12522 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:11:37.889477 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:37.903920 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:37.905484 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:37.889940 17041 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.906319 17042 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.003s
I20260709 12:11:37.902671 16760 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:37.872615 17018 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:11:37.867069 16874 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:37.918530 12526 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
W20260709 12:11:37.907045 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 82762.8us
I20260709 12:11:37.932668 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:37.941365 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:37.941982 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:37.942345 12531 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:37.942793 12524 raft_consensus.cc:2368] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10.
I20260709 12:11:37.888942 16720 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:11:37.966612 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 61150us
I20260709 12:11:37.906383 17126 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:11:37.982484 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 236262us
W20260709 12:11:38.005797 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:38.006841 12496 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:37.988071 12525 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
W20260709 12:11:37.966387 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 61439.4us
I20260709 12:11:38.002954 12643 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:37.908012 17114 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:38.023406 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.023799 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.024227 12529 raft_consensus.cc:2368] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 10. Current term is 11.
W20260709 12:11:38.020733 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 116762us
I20260709 12:11:37.906961 16943 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:37.910905 17041 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:38.028963 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.029765 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:38.030350 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.031024 12531 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:38.031934 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:38.033036 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.033622 12531 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
W20260709 12:11:38.035154 17144 log.cc:927] Time spent T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 12:11:38.036428 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:38.036966 12606 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 6. (term mismatch)
I20260709 12:11:38.037099 12679 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:38.037376 12606 pending_rounds.cc:85] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 5
I20260709 12:11:38.037417 17041 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:38.037809 12606 pending_rounds.cc:107] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 9.6
I20260709 12:11:38.038210 12606 raft_consensus.cc:2889] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:38.038657 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:37.863313 17134 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:37.929466 17094 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:37.910699 16760 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:38.039885 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:38.040438 12606 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:38.024688 16759 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:38.023569 17114 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:38.041258 17114 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:38.041818 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:38.042620 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:38.043407 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.044050 12531 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:38.045289 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:38.047341 16960 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:38.046285 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.048303 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:38.049088 12601 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:38.038175 17041 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:38.025221 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:38.051076 12525 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:38.051162 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:38.051489 12525 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:38.052963 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:38.054029 12601 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:38.055050 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:38.055658 12679 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:11:38.056006 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:38.057088 12601 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:38.040539 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:38.001896 16726 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:38.052963 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.019322 16824 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:38.058799 16211 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:38.004314 17020 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:11:38.066881 12529 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:11:38.020195 16720 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:38.074949 16926 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:38.091684 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:38.053176 12525 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:38.061883 16874 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:38.081321 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.079510 12572 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:38.082669 12531 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 11: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:38.025063 17018 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:38.042546 17062 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:38.106494 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:38.083848 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:38.108649 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:38.109151 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:38.089407 12601 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:38.111397 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:38.112982 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:38.113775 12601 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:38.092506 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 50768.9us
I20260709 12:11:38.092768 12525 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:38.026605 17056 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:11:38.098845 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:11:38.213122 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 114336us
I20260709 12:11:38.214177 12496 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:38.096768 12679 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:38.105326 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.094818 12606 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:38.119243 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.048700 17127 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:11:38.083516 16824 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:38.126917 16893 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:38.143512 17020 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:38.112185 16954 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:11:38.109736 17074 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.028s
I20260709 12:11:38.134326 16720 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:38.129629 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:38.143862 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:38.237005 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 93206.3us
I20260709 12:11:38.237980 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:38.121778 17137 consensus_queue.cc:1059] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.117s
I20260709 12:11:38.047916 16960 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:38.106851 16926 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:38.125604 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.092773 16719 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:11:38.257434 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.146955 17017 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:38.096027 16211 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:38.117051 17018 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:38.208101 13305 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:38.272291 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 64309.3us
I20260709 12:11:38.271270 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.273707 13305 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:38.283185 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:38.215830 12525 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:11:38.216471 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:38.286177 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:11:38.287204 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:38.254621 16875 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:38.224076 12601 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:38.226796 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.231487 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.117879 17062 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:38.094308 16680 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:38.236864 17127 consensus_queue.cc:1243] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:11:38.224779 16726 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:11:38.252728 12598 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2262) took 2101 ms (client timeout 1845 ms). Trace:
W20260709 12:11:38.327172 12598 rpcz_store.cc:269] 0709 12:11:36.151408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:36.231971 (+ 80563us) service_pool.cc:224] Handling call
0709 12:11:38.252406 (+2020435us) spinlock_profiling.cc:243] Waited 826 ms on lock 0x7b5400270bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:38.252622 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1899427712}
W20260709 12:11:38.251551 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:38.230150 16824 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:38.239539 17074 consensus_queue.cc:1243] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 10.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:11:38.342911 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:38.178437 17114 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:38.239374 16893 raft_consensus.cc:697] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:38.290783 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:38.255694 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:38.256639 16894 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:38.223239 12671 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:38.315623 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:38.269985 17153 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.096s	user 0.001s	sys 0.000s
I20260709 12:11:38.320739 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 415) took 1168 ms. Trace:
I20260709 12:11:38.336051 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.304790 17077 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:38.350391 12675 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 7: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:38.352365 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:38.353989 12525 raft_consensus.cc:2368] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11.
I20260709 12:11:38.367640 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.312849 17021 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:38.357916 12531 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:38.319594 17017 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:38.361943 12672 rpcz_store.cc:276] 0709 12:11:37.152817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:37.161494 (+  8677us) service_pool.cc:224] Handling call
0709 12:11:38.320278 (+1158784us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b54002c5e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:38.320640 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1011437440}
I20260709 12:11:38.362367 12524 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:38.369688 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.383551 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:38.308691 16875 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:38.339612 17042 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:38.341625 17127 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:38.410331 12673 raft_consensus.cc:2368] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11.
I20260709 12:11:38.357553 17058 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:38.364488 17077 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:38.415025 13305 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:38.410075 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.416170 17007 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:38.449415 17042 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:38.462177 12525 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
I20260709 12:11:38.467465 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.473438 12524 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 6.
I20260709 12:11:38.412186 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:38.475271 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:38.475881 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 220268us
I20260709 12:11:38.391091 16894 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:11:38.532523 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 117789us
I20260709 12:11:38.533533 12496 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:38.534040 13305 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:38.483902 12529 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:38.485623 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:38.534459 12531 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:11:38.479416 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:38.536983 13331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38472 (request call id 458) took 1556 ms. Trace:
I20260709 12:11:38.537426 13331 rpcz_store.cc:276] 0709 12:11:36.920930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:36.921152 (+   222us) service_pool.cc:224] Handling call
0709 12:11:38.472449 (+1551297us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b5400469954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:38.476707 (+  4258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1085633792}
I20260709 12:11:38.438642 16875 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:38.418753 17021 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 13 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:11:38.486752 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:38.540593 12526 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:38.439484 17127 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:11:38.570067 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 227273us
I20260709 12:11:38.570688 12526 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:38.471450 17077 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:38.477806 12600 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:38.553371 12529 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: [11.7-12.8]
I20260709 12:11:38.551990 12531 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:38.577486 12531 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:38.578408 12531 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:38.579252 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:38.579576 12531 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:38.536533 12678 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:38.469302 16719 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:38.471767 17007 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:38.594758 17007 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:38.551469 16954 raft_consensus.cc:2764] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260709 12:11:38.591413 17114 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:38.599444 12600 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 12. Ops: [8.4-11.6]
I20260709 12:11:38.600519 12531 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:11:38.582996 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 97506.6us
I20260709 12:11:38.551774 17002 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:38.536953 16911 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:38.568478 16893 consensus_queue.cc:237] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:38.435814 17018 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:38.589679 16894 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:38.590328 17159 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
I20260709 12:11:38.590713 16211 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:38.588613 17126 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:38.591058 17021 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:38.462324 17058 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:38.603796 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:38.570305 17051 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:38.628158 12678 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:38.573551 17127 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:38.609766 12524 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 8 index: 7. Preceding OpId from leader: term: 9 index: 7. (term mismatch)
I20260709 12:11:38.592873 17042 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:38.606251 16919 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:38.620599 12569 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:38.612957 17156 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.342s	user 0.000s	sys 0.002s
I20260709 12:11:38.605293 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:38.635071 17114 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:38.650365 16926 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:38.658259 17154 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.370s	user 0.000s	sys 0.001s
I20260709 12:11:38.632413 16211 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:38.628958 17051 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:38.663651 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:38.839473 12676 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:38.665421 17164 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
W20260709 12:11:38.666517 17166 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:11:38.690733 17062 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:38.665503 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:38.615446 17059 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.194s	user 0.014s	sys 0.038s
I20260709 12:11:38.773520 12602 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 12. Ops: [10.8-11.9]
I20260709 12:11:38.811707 17171 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:38.823804 17126 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:38.787276 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:38.812644 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 192157us
I20260709 12:11:38.809880 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:38.804391 17123 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.728s	user 0.009s	sys 0.012s
I20260709 12:11:38.812446 16855 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:38.856747 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 69576.5us
I20260709 12:11:38.857144 12569 leader_election.cc:400] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13.
I20260709 12:11:38.853222 12602 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:38.775520 12524 pending_rounds.cc:85] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 6
I20260709 12:11:38.859505 12524 pending_rounds.cc:107] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 8.7
I20260709 12:11:38.859871 12524 raft_consensus.cc:2889] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:38.843643 17051 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:38.855257 12526 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:38.854318 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2550) took 1028 ms. Trace:
I20260709 12:11:38.863592 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:38.865236 12675 rpcz_store.cc:276] 0709 12:11:37.825796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:38.665248 (+839452us) service_pool.cc:224] Handling call
0709 12:11:38.854252 (+189004us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:38.865454 12678 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:38.866415 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:38.836618 16211 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:38.871214 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:38.872937 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:38.875250 12678 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:11:38.847291 16919 raft_consensus.cc:2764] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
W20260709 12:11:38.850855 17059 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.436s	user 0.015s	sys 0.038s
I20260709 12:11:38.872112 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2558) took 1044 ms. Trace:
I20260709 12:11:38.917549 12675 rpcz_store.cc:276] 0709 12:11:37.827989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:38.870948 (+1042959us) service_pool.cc:224] Handling call
0709 12:11:38.872051 (+  1103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:38.835732 17114 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:38.864566 16926 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:11:38.861014 17123 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.785s	user 0.010s	sys 0.012s
I20260709 12:11:38.919337 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:38.905535 12676 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:38.890550 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:38.856057 17126 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:38.926645 12675 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:38.927006 12678 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:38.870692 16211 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:38.862679 16855 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:38.932272 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:38.926499 17126 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:38.938344 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2556) took 1110 ms. Trace:
I20260709 12:11:38.926412 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2161) took 1036 ms. Trace:
I20260709 12:11:38.939000 12675 rpcz_store.cc:276] 0709 12:11:37.827484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:38.928210 (+1100726us) service_pool.cc:224] Handling call
0709 12:11:38.938258 (+ 10048us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:38.939688 12676 rpcz_store.cc:276] 0709 12:11:37.889828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:38.663390 (+773562us) service_pool.cc:224] Handling call
0709 12:11:38.926325 (+262935us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.071380 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2279) took 1056 ms. Trace:
I20260709 12:11:39.073608 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:39.076263 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:39.077373 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:38.884289 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:38.021646 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:11:39.057992 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 180148us
I20260709 12:11:39.067538 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2278) took 1052 ms. Trace:
I20260709 12:11:39.068490 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2289) took 1041 ms. Trace:
I20260709 12:11:39.070012 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2286) took 1049 ms. Trace:
I20260709 12:11:39.073328 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:39.056519 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:39.072253 12605 rpcz_store.cc:276] 0709 12:11:38.015122 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:38.015265 (+   143us) service_pool.cc:224] Handling call
0709 12:11:38.053195 (+ 37930us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:38.093349 (+ 40154us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:11:38.093381 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:38.115699 (+ 22318us) log.cc:844] Serialized 26 byte log entry
0709 12:11:38.296850 (+181151us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:38.296925 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:38.297068 (+   143us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:39.070525 (+773457us) raft_consensus.cc:1685] finished
0709 12:11:39.070591 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.071116 (+   525us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57443,"raft.queue_time_us":40838,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":15325,"thread_start_us":121203,"threads_started":1,"wal-append.queue_time_us":427732}
I20260709 12:11:39.082791 12597 rpcz_store.cc:276] 0709 12:11:38.027323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:38.359137 (+331814us) service_pool.cc:224] Handling call
0709 12:11:38.408044 (+ 48907us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:38.408519 (+   475us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:38.408565 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:38.409197 (+   632us) log.cc:844] Serialized 26 byte log entry
0709 12:11:38.812297 (+403100us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:38.812462 (+   165us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:38.812571 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:39.067878 (+255307us) raft_consensus.cc:1685] finished
0709 12:11:39.067928 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.068260 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2420,"raft.queue_time_us":615,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":23108,"thread_start_us":379907,"threads_started":1,"wal-append.queue_time_us":504660}
I20260709 12:11:39.084887 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:39.098104 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2127) took 2977 ms. Trace:
I20260709 12:11:39.098788 12675 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:39.099186 12528 rpcz_store.cc:276] 0709 12:11:36.120718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:36.122827 (+  2109us) service_pool.cc:224] Handling call
0709 12:11:36.130435 (+  7608us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:37.674473 (+1544038us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:11:37.674506 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:37.674552 (+    46us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:37.674598 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.097180 (+1422582us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b5400210cd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:39.097402 (+   222us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.097804 (+   402us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12274,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":126,"spinlock_wait_cycles":3549853568}
I20260709 12:11:38.959761 13332 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 13 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 13. Ops: []
W20260709 12:11:39.108863 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 503724us
I20260709 12:11:39.110430 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:39.111205 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2285) took 1093 ms. Trace:
I20260709 12:11:39.111626 12677 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 11: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:39.111677 12604 rpcz_store.cc:276] 0709 12:11:38.017844 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:38.228896 (+211052us) service_pool.cc:224] Handling call
0709 12:11:38.254300 (+ 25404us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:38.280191 (+ 25891us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:38.280229 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:38.280283 (+    54us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:38.280316 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.110612 (+830296us) spinlock_profiling.cc:243] Waited 25.6 ms on lock 0x7b5400310bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:39.110673 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.111004 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58841472}
I20260709 12:11:39.112936 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:39.113544 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:39.082945 12601 rpcz_store.cc:276] 0709 12:11:38.020441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:38.310654 (+290213us) service_pool.cc:224] Handling call
0709 12:11:38.311113 (+   459us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:38.457274 (+146161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:38.457313 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:38.458253 (+   940us) log.cc:844] Serialized 52 byte log entry
0709 12:11:38.847662 (+389409us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:38.847754 (+    92us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:38.847854 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:39.069402 (+221548us) raft_consensus.cc:1685] finished
0709 12:11:39.069452 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.069780 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":182,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":108,"thread_start_us":388725,"threads_started":1,"wal-append.queue_time_us":499911}
I20260709 12:11:39.115688 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:39.116454 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:39.117399 12606 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:39.118870 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:39.120147 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:39.126178 13304 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:39.127871 12671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2126) took 4262 ms (client timeout 1855 ms). Trace:
W20260709 12:11:39.128471 12671 rpcz_store.cc:269] 0709 12:11:34.865469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:34.873515 (+  8046us) service_pool.cc:224] Handling call
0709 12:11:39.127332 (+4253817us) spinlock_profiling.cc:243] Waited 3.22 s on lock 0x7b5400013e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:39.127591 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7402576256}
I20260709 12:11:39.121481 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:39.130795 12597 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
W20260709 12:11:39.133598 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:39.101598 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:38.946393 16726 raft_consensus.cc:3055] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:38.964359 16760 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:38.944840 16855 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:38.935477 16926 raft_consensus.cc:697] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:11:39.210572 17002 consensus_meta.cc:287] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.601s	user 0.014s	sys 0.000s
I20260709 12:11:39.211333 17002 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:39.203734 17029 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:39.216605 17029 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:39.217283 17029 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:38.948561 17017 consensus_meta.cc:287] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.529s	user 0.010s	sys 0.000s
W20260709 12:11:39.149076 16962 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.675s	user 0.010s	sys 0.017s
W20260709 12:11:39.221299 16962 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.749s	user 0.010s	sys 0.018s
I20260709 12:11:39.221712 12680 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:39.221882 16962 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:38.980526 17007 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:39.082692 12599 rpcz_store.cc:276] 0709 12:11:38.014816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:38.014942 (+   126us) service_pool.cc:224] Handling call
0709 12:11:38.036122 (+ 21180us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:38.082467 (+ 46345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:38.082517 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:38.083232 (+   715us) log.cc:844] Serialized 52 byte log entry
0709 12:11:38.323694 (+240462us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:38.323760 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:38.323864 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:39.066828 (+742964us) raft_consensus.cc:1685] finished
0709 12:11:39.066873 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.067253 (+   380us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":40243,"raft.queue_time_us":40406,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":12742,"thread_start_us":154995,"threads_started":1,"wal-append.queue_time_us":454060}
I20260709 12:11:39.223768 16962 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:38.917472 17059 raft_consensus.cc:697] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:11:39.226037 12673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2177) took 1314 ms (client timeout 1686 ms). Trace:
W20260709 12:11:39.226651 12673 rpcz_store.cc:269] 0709 12:11:37.911104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:38.048155 (+137051us) service_pool.cc:224] Handling call
0709 12:11:39.225498 (+1177343us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b54000933d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:39.225796 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":730668928}
I20260709 12:11:39.227294 17007 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:39.228304 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:38.958316 17114 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:38.966068 17077 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:39.085558 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:38.996833 17126 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:39.231401 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:39.143893 16726 raft_consensus.cc:740] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:39.135630 12677 raft_consensus.cc:2410] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 4.
W20260709 12:11:39.008275 17157 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.403s	user 0.002s	sys 0.000s
I20260709 12:11:39.234045 16726 consensus_queue.cc:260] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.235736 16726 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
W20260709 12:11:39.008821 17161 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.402s	user 0.003s	sys 0.000s
W20260709 12:11:38.954807 17163 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:11:39.237483 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2554) took 1410 ms. Trace:
I20260709 12:11:38.861424 17171 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:39.239711 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 437) took 1346 ms. Trace:
I20260709 12:11:39.241155 17171 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:39.134104 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2549) took 1308 ms. Trace:
I20260709 12:11:39.242574 12672 rpcz_store.cc:276] 0709 12:11:37.825548 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.132590 (+1307042us) service_pool.cc:224] Handling call
0709 12:11:39.133184 (+   594us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.133612 (+   428us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:39.133641 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:39.133695 (+    54us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:39.133731 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.133796 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.133905 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.243661 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:39.078112 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:39.137288 12601 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:11:39.088127 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:39.134948 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2553) took 1308 ms. Trace:
I20260709 12:11:39.234669 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2160) took 1345 ms. Trace:
W20260709 12:11:39.165971 16759 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 12:11:39.089476 12496 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:39.090837 12494 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:39.201013 17039 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:39.221256 16824 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.860s	user 0.011s	sys 0.004s
I20260709 12:11:39.201869 12674 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260709 12:11:39.203645 16671 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:39.205724 16720 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:39.206598 17064 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:39.227830 17059 consensus_queue.cc:237] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.137936 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:39.201241 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 75234.2us
I20260709 12:11:39.249929 12677 rpcz_store.cc:276] 0709 12:11:37.889492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.112501 (+1223009us) service_pool.cc:224] Handling call
0709 12:11:39.234569 (+122068us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.249387 12675 rpcz_store.cc:276] 0709 12:11:37.826643 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.100790 (+1274147us) service_pool.cc:224] Handling call
0709 12:11:39.134853 (+ 34063us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.238008 12673 rpcz_store.cc:276] 0709 12:11:37.826921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.235888 (+1408967us) service_pool.cc:224] Handling call
0709 12:11:39.236472 (+   584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.237009 (+   537us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:39.237043 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:39.237138 (+    95us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:39.237197 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.237289 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.237410 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.259723 12605 raft_consensus.cc:2410] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 12 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:11:39.248499 17077 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:39.209357 16926 consensus_queue.cc:237] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.261379 16720 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:11:39.285276 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 195991us
I20260709 12:11:39.293807 17062 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:39.294531 17062 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.240258 12527 rpcz_store.cc:276] 0709 12:11:37.893380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:37.893565 (+   185us) service_pool.cc:224] Handling call
0709 12:11:37.928082 (+ 34517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:37.928467 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:37.928502 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:37.931390 (+  2888us) log.cc:844] Serialized 26 byte log entry
0709 12:11:38.259403 (+328013us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:38.259474 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:38.259566 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:39.007711 (+748145us) spinlock_profiling.cc:243] Waited 7.36 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:39.238752 (+231041us) raft_consensus.cc:1685] finished
0709 12:11:39.238796 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.239213 (+   417us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1320,"raft.run_cpu_time_us":158,"raft.run_wall_time_us":12919,"spinlock_wait_cycles":16936832,"thread_start_us":230734,"threads_started":1,"wal-append.queue_time_us":446193}
I20260709 12:11:39.298298 17039 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:39.299350 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 208952us
I20260709 12:11:39.300928 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:39.301488 17039 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:39.304792 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 443) took 1103 ms. Trace:
I20260709 12:11:39.306548 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2287) took 1279 ms. Trace:
I20260709 12:11:39.307443 12598 rpcz_store.cc:276] 0709 12:11:38.026489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:38.328550 (+302061us) service_pool.cc:224] Handling call
0709 12:11:38.329227 (+   677us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:38.355476 (+ 26249us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:38.355511 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:38.467488 (+111977us) log.cc:844] Serialized 52 byte log entry
0709 12:11:38.831907 (+364419us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:38.944641 (+112734us) spinlock_profiling.cc:243] Waited 7.66 ms on lock 0x7b480024c258. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:11:39.305101 (+360460us) log.cc:844] Serialized 14 byte log entry
0709 12:11:39.305636 (+   535us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.305778 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:39.305852 (+    74us) raft_consensus.cc:1685] finished
0709 12:11:39.305997 (+   145us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.306266 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25683,"raft.queue_time_us":29593,"raft.run_cpu_time_us":2609,"raft.run_wall_time_us":585678,"spinlock_wait_cycles":17615104,"thread_start_us":705485,"threads_started":2,"wal-append.queue_time_us":392575}
I20260709 12:11:39.310322 17168 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:39.311342 17168 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.313135 16960 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:39.331940 12671 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: []
W20260709 12:11:39.322767 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 79477.4us
I20260709 12:11:39.328044 12496 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:39.334594 17134 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:39.335109 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:39.298368 16720 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.337558 12572 leader_election.cc:400] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11.
I20260709 12:11:39.337844 16720 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:39.322907 16671 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.323801 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:39.342563 17122 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:39.324968 17041 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:39.344009 16954 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:39.344300 17039 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:39.345042 16954 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:39.343120 17122 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:39.349381 17122 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:39.350548 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2559) took 1522 ms. Trace:
I20260709 12:11:39.350977 12677 rpcz_store.cc:276] 0709 12:11:37.828246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.282493 (+1454247us) service_pool.cc:224] Handling call
0709 12:11:39.349674 (+ 67181us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.350001 (+   327us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:39.350035 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:39.350111 (+    76us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:39.350217 (+   106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.350323 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.350457 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.352468 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:39.353505 12677 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 11: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:11:39.354708 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 217290us
I20260709 12:11:39.356112 12678 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:11:39.357002 12672 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:39.358539 16919 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:39.359037 16919 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:39.359450 16919 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:39.359495 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2555) took 1532 ms. Trace:
I20260709 12:11:39.359972 12672 rpcz_store.cc:276] 0709 12:11:37.827221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.244166 (+1416945us) service_pool.cc:224] Handling call
0709 12:11:39.356796 (+112630us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.357387 (+   591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.359287 (+  1900us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54000c2bd0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:39.359438 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":257634688}
I20260709 12:11:39.363557 13331 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:11:39.366305 12676 raft_consensus.cc:2368] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 11. Current term is 12.
I20260709 12:11:39.366827 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 424) took 1021 ms. Trace:
I20260709 12:11:39.367309 12676 rpcz_store.cc:276] 0709 12:11:38.345598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.077651 (+732053us) service_pool.cc:224] Handling call
0709 12:11:39.366757 (+289106us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.369190 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2170) took 1472 ms. Trace:
W20260709 12:11:39.370424 16875 consensus_meta.cc:287] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.718s	user 0.004s	sys 0.013s
I20260709 12:11:39.371101 16875 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.366772 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2171) took 1469 ms. Trace:
I20260709 12:11:39.337968 12572 leader_election.cc:403] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:11:39.369544 12676 rpcz_store.cc:276] 0709 12:11:37.897311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.368037 (+1470726us) service_pool.cc:224] Handling call
0709 12:11:39.368514 (+   477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.368808 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:39.368836 (+    28us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:39.368926 (+    90us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:39.368963 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.369020 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.369138 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.362043 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2168) took 1470 ms. Trace:
I20260709 12:11:39.377902 12672 rpcz_store.cc:276] 0709 12:11:37.892261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.360912 (+1468651us) service_pool.cc:224] Handling call
0709 12:11:39.361329 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.361664 (+   335us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:39.361693 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:39.361779 (+    86us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:39.361822 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.361879 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.361987 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.305316 12531 rpcz_store.cc:276] 0709 12:11:38.201802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:38.601334 (+399532us) service_pool.cc:224] Handling call
0709 12:11:38.601704 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:38.601953 (+   249us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:11:38.613711 (+ 11758us) log.cc:844] Serialized 14 byte log entry
0709 12:11:39.090767 (+477056us) spinlock_profiling.cc:243] Waited 3.12 ms on lock 0x7b18004908a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a 00007f1488a2aa2a
0709 12:11:39.303892 (+213125us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:39.304017 (+   125us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:39.304127 (+   110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.304281 (+   154us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.304532 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9639,"prepare.queue_time_us":11378,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":4361,"spinlock_wait_cycles":7186816,"thread_start_us":160452,"threads_started":1,"wal-append.queue_time_us":655034}
I20260709 12:11:39.375145 12678 rpcz_store.cc:276] 0709 12:11:37.897814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:38.890305 (+992491us) service_pool.cc:224] Handling call
0709 12:11:39.366662 (+476357us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.378829 16671 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:39.385564 12571 leader_election.cc:400] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11.
I20260709 12:11:39.264724 16824 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:39.314308 17158 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.708s	user 0.005s	sys 0.000s
I20260709 12:11:39.388860 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2551) took 1557 ms. Trace:
I20260709 12:11:39.393898 12671 rpcz_store.cc:276] 0709 12:11:37.826067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.166751 (+1340684us) service_pool.cc:224] Handling call
0709 12:11:39.261783 (+ 95032us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.380370 (+118587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.383271 (+  2901us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.280030 16759 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:39.396100 12606 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:39.345974 16954 consensus_queue.cc:1243] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260709 12:11:39.393280 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 69724.5us
I20260709 12:11:39.374073 16875 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:39.382915 17168 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:39.384338 16960 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:39.413699 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:39.413980 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:39.411386 16671 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:39.415537 12679 raft_consensus.cc:2368] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9.
I20260709 12:11:39.415835 16671 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:11:39.417482 12679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2174) took 1514 ms (client timeout 1793 ms). Trace:
I20260709 12:11:39.418387 16671 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:39.419775 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:39.420598 12527 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
W20260709 12:11:39.421515 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 205190us
W20260709 12:11:39.417927 12679 rpcz_store.cc:269] 0709 12:11:37.903138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:38.285818 (+382680us) service_pool.cc:224] Handling call
0709 12:11:39.417251 (+1131433us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54001e2154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:39.417418 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2581917952}
I20260709 12:11:39.429826 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:39.429291 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:39.435786 16719 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:39.436051 17137 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:39.436305 16719 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:39.436607 17137 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:39.436872 16719 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:39.437086 17137 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:39.438280 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:39.436877 16824 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:39.439672 17126 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:39.440647 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: true dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:39.440831 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:39.441613 12527 raft_consensus.cc:2376] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13. Re-sending same reply.
I20260709 12:11:39.442325 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:39.443562 17123 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:39.444437 17123 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:39.446787 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:39.424104 16759 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.447606 12522 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:39.450654 13332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 612) took 2323 ms. Trace:
I20260709 12:11:39.451120 13332 rpcz_store.cc:276] 0709 12:11:37.126890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:37.127120 (+   230us) service_pool.cc:224] Handling call
0709 12:11:38.472742 (+1345622us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.106279 (+633537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.109560 (+  3281us) spinlock_profiling.cc:243] Waited 487 ms on lock 0x7b5400469954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:39.450278 (+340718us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400469950. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:39.450464 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4205891200}
I20260709 12:11:39.456192 16960 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:39.456854 17127 consensus_meta.cc:287] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.636s	user 0.004s	sys 0.014s
I20260709 12:11:39.461171 12525 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:39.461391 12673 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:11:39.462262 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2180) took 1550 ms. Trace:
I20260709 12:11:39.462610 12673 rpcz_store.cc:276] 0709 12:11:37.911887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.460227 (+1548340us) service_pool.cc:224] Handling call
0709 12:11:39.460970 (+   743us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.461916 (+   946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.462184 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.427594 17134 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:39.465694 17134 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:11:39.477512 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46233.8us
I20260709 12:11:39.486752 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 423) took 1141 ms. Trace:
I20260709 12:11:39.486474 16671 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 6.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:39.463846 16759 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:39.487324 16760 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:39.487841 16671 raft_consensus.cc:3055] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:39.488286 16671 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:39.488489 16760 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:39.489009 16760 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:39.465952 16960 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:39.498767 13331 raft_consensus.cc:2368] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12.
I20260709 12:11:39.457620 17127 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.490682 12673 rpcz_store.cc:276] 0709 12:11:38.345211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.474609 (+1129398us) service_pool.cc:224] Handling call
0709 12:11:39.485927 (+ 11318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.486362 (+   435us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:39.486387 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:39.486451 (+    64us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:39.486489 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.486543 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.486654 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.507745 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2163) took 1617 ms. Trace:
I20260709 12:11:39.512775 12675 rpcz_store.cc:276] 0709 12:11:37.890372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.281414 (+1391042us) service_pool.cc:224] Handling call
0709 12:11:39.407936 (+126522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.479926 (+ 71990us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:39.479973 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:39.507430 (+ 27457us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:39.507483 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.507554 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.507677 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.515349 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2173) took 1616 ms. Trace:
I20260709 12:11:39.516652 12676 rpcz_store.cc:276] 0709 12:11:37.898483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.392088 (+1493605us) service_pool.cc:224] Handling call
0709 12:11:39.424859 (+ 32771us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.508610 (+ 83751us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:39.508644 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:39.514960 (+  6316us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:39.515006 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.515177 (+   171us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.515296 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.485409 16943 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.022s
I20260709 12:11:39.445631 17123 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:39.522670 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:39.522467 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:39.523523 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 48130us
I20260709 12:11:39.522286 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:39.485226 17074 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:39.521984 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2190) took 1540 ms. Trace:
I20260709 12:11:39.526186 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:39.522220 16943 consensus_queue.cc:1243] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:11:39.507079 16760 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.527966 12530 rpcz_store.cc:276] 0709 12:11:37.939605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:37.939726 (+   121us) service_pool.cc:224] Handling call
0709 12:11:37.940093 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:37.940508 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:11:38.027243 (+ 86735us) log.cc:844] Serialized 14 byte log entry
0709 12:11:39.085796 (+1058553us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b1800088c00. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a 00007f1488a2aa2a
0709 12:11:39.402832 (+317036us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:39.431573 (+ 28741us) log.cc:844] Serialized 26 byte log entry
0709 12:11:39.448843 (+ 17270us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:39.448887 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.449003 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:39.462536 (+ 13533us) raft_consensus.cc:1685] finished
0709 12:11:39.474895 (+ 12359us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.479624 (+  4729us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":109545,"prepare.queue_time_us":90350,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":75964,"raft.run_cpu_time_us":384,"raft.run_wall_time_us":90008,"spinlock_wait_cycles":36119296,"thread_start_us":179861,"threads_started":1,"wal-append.queue_time_us":1378502}
I20260709 12:11:39.529985 16760 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:39.530570 12523 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:39.526469 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2167) took 1635 ms. Trace:
I20260709 12:11:39.489522 16671 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.531812 12677 rpcz_store.cc:276] 0709 12:11:37.891568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.384616 (+1493048us) service_pool.cc:224] Handling call
0709 12:11:39.425654 (+ 41038us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.520181 (+ 94527us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:39.520222 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:39.526192 (+  5970us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:39.526233 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.526295 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.526408 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.515148 17041 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:39.539489 12671 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:11:39.551846 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 224035us
I20260709 12:11:39.477456 17134 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 10.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.548344 17058 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:39.553134 12525 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:39.564932 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2179) took 1653 ms. Trace:
I20260709 12:11:39.566095 12671 rpcz_store.cc:276] 0709 12:11:37.911628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.404748 (+1493120us) service_pool.cc:224] Handling call
0709 12:11:39.472320 (+ 67572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.562761 (+ 90441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.564867 (+  2106us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:11:39.563709 12680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2542) took 1744 ms (client timeout 1690 ms). Trace:
I20260709 12:11:39.566570 12522 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:11:39.583504 12676 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:39.584302 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2184) took 1015 ms. Trace:
I20260709 12:11:39.586766 12676 rpcz_store.cc:276] 0709 12:11:38.568922 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.517746 (+948824us) service_pool.cc:224] Handling call
0709 12:11:39.555203 (+ 37457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.584019 (+ 28816us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.584241 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.589619 16977 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:11:39.566995 17074 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:39.591351 17051 consensus_meta.cc:287] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.700s	user 0.004s	sys 0.011s
I20260709 12:11:39.591492 16893 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:39.591924 17051 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:39.592247 16893 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:39.592752 17074 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:39.595930 17165 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.989s	user 0.004s	sys 0.000s
I20260709 12:11:39.580778 17041 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:39.599980 17039 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260709 12:11:39.601418 16211 consensus_meta.cc:287] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.659s	user 0.013s	sys 0.005s
W20260709 12:11:39.608453 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:39.609083 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:39.611042 16671 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:39.615960 16960 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:39.616808 16960 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:39.575470 12680 rpcz_store.cc:269] 0709 12:11:37.819517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:37.819846 (+   329us) service_pool.cc:224] Handling call
0709 12:11:39.550467 (+1730621us) spinlock_profiling.cc:243] Waited 1.37 s on lock 0x7b5400012cd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:39.563588 (+ 13121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3151319296}
I20260709 12:11:39.623489 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2186) took 1054 ms. Trace:
I20260709 12:11:39.623960 12671 rpcz_store.cc:276] 0709 12:11:38.569496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.576846 (+1007350us) service_pool.cc:224] Handling call
0709 12:11:39.622822 (+ 45976us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.623059 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:39.623086 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:39.623145 (+    59us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:39.623222 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.623278 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.623406 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.625275 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:39.627573 13305 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:39.629252 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 54756.1us
I20260709 12:11:39.629765 17077 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260709 12:11:39.630277 17077 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:39.630622 17077 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:39.631758 17029 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:39.632519 17137 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.634878 12496 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:39.636198 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48453.3us
I20260709 12:11:39.637055 12496 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:39.637266 16806 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:39.637703 16806 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:39.601986 16211 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.638839 17074 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:39.640357 16211 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:39.642736 12600 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:39.642927 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 425) took 1297 ms. Trace:
I20260709 12:11:39.643112 16806 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.643388 12679 rpcz_store.cc:276] 0709 12:11:38.345912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.470432 (+1124520us) service_pool.cc:224] Handling call
0709 12:11:39.538489 (+ 68057us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.618860 (+ 80371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:39.618892 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:39.642436 (+ 23544us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:39.642515 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.642727 (+   212us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.642856 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.644289 16806 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:39.644805 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2442) took 1536 ms. Trace:
I20260709 12:11:39.645283 12600 rpcz_store.cc:276] 0709 12:11:38.107956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:38.614974 (+507018us) service_pool.cc:224] Handling call
0709 12:11:38.630944 (+ 15970us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.643221 (+1012277us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.644536 (+  1315us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b5400375d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:39.644751 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2326058880}
I20260709 12:11:39.638324 17197 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:39.646991 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:39.647560 12646 connection.cc:256] RPC call timeout handler was delayed by -1.1117327380000006s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1625s for a response
I20260709 12:11:39.648257 17127 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:39.644730 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:39.652530 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 84233.1us
I20260709 12:11:39.650014 12676 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:11:39.656785 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2187) took 1082 ms. Trace:
I20260709 12:11:39.663724 12676 rpcz_store.cc:276] 0709 12:11:38.574283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.618748 (+1044465us) service_pool.cc:224] Handling call
0709 12:11:39.649611 (+ 30863us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.656224 (+  6613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.656709 (+   485us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.665197 12569 leader_election.cc:400] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12.
I20260709 12:11:39.665634 12569 leader_election.cc:403] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:11:39.662925 17029 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:39.668548 17094 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:39.669005 17094 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:39.669986 17094 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.671025 16893 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:11:39.671393 17094 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:39.676638 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:39.677488 13332 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 12: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:39.677347 16919 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:39.679818 13331 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:39.649928 12646 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:39.683820 12525 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
W20260709 12:11:39.683651 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 123035us
I20260709 12:11:39.689193 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:39.651400 16875 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:39.690320 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 353039us
W20260709 12:11:39.693876 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:39.565496 17168 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.117s
I20260709 12:11:39.696288 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:39.695005 16893 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11.
I20260709 12:11:39.707858 16960 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:39.709432 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 59654.4us
I20260709 12:11:39.710688 12646 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:39.712618 16719 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.685449 17051 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:39.686971 16926 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:39.713523 16760 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:39.717592 12524 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 12: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:11:39.714429 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 77514us
I20260709 12:11:39.717909 16943 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:39.723673 16943 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:39.723518 16875 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.724283 16943 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:39.726289 16797 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:39.729377 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2185) took 1160 ms. Trace:
I20260709 12:11:39.733201 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 423) took 1471 ms. Trace:
I20260709 12:11:39.733673 12602 rpcz_store.cc:276] 0709 12:11:38.244581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:38.859310 (+614729us) service_pool.cc:224] Handling call
0709 12:11:38.862654 (+  3344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:38.863073 (+   419us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:11:39.376186 (+513113us) log.cc:844] Serialized 14 byte log entry
0709 12:11:39.578985 (+202799us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:39.621108 (+ 42123us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:39.684640 (+ 63532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.688037 (+  3397us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.716010 (+ 27973us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8372,"prepare.queue_time_us":460016,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":2670,"raft.run_cpu_time_us":346,"raft.run_wall_time_us":38448,"spinlock_wait_cycles":66193152,"thread_start_us":549341,"threads_started":2,"wal-append.queue_time_us":231610}
I20260709 12:11:39.736385 12679 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:11:39.729868 12677 rpcz_store.cc:276] 0709 12:11:38.569209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.565651 (+996442us) service_pool.cc:224] Handling call
0709 12:11:39.681444 (+115793us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.728977 (+ 47533us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:39.729013 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:39.729081 (+    68us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:39.729128 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.729194 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.729315 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.740814 12679 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 8. Current term is 9. Ops: []
W20260709 12:11:39.746851 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 64613.2us
I20260709 12:11:39.684737 17122 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.761138 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:39.748085 17137 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:39.772176 12679 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: [10.7-12.8]
I20260709 12:11:39.768759 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:39.787796 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:39.788069 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2182) took 1220 ms. Trace:
I20260709 12:11:39.788574 12673 rpcz_store.cc:276] 0709 12:11:38.568206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.516188 (+947982us) service_pool.cc:224] Handling call
0709 12:11:39.559294 (+ 43106us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:39.588372 (+ 29078us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:39.588413 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:39.589045 (+   632us) log.cc:844] Serialized 26 byte log entry
0709 12:11:39.688465 (+ 99420us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:39.688523 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.688618 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:39.787760 (+ 99142us) raft_consensus.cc:1685] finished
0709 12:11:39.787787 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:39.787971 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":94764,"threads_started":1,"wal-append.queue_time_us":150141}
I20260709 12:11:39.789671 17197 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:39.790586 17197 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:11:39.793267 12646 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:39.794865 16760 raft_consensus.cc:2764] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:11:39.795751 12646 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:39.796775 17123 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:39.799160 17123 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:39.721853 17127 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:39.802657 17127 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:39.805365 12496 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:39.806605 17197 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:39.812072 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:39.812904 12677 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:39.757164 16875 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:39.790011 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:39.814111 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:39.824889 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:39.829267 12679 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:39.815626 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2145) took 4050 ms. Trace:
I20260709 12:11:39.847780 12674 rpcz_store.cc:276] 0709 12:11:35.765467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:36.526155 (+760688us) service_pool.cc:224] Handling call
0709 12:11:39.147010 (+2620855us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.254246 (+107236us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.815341 (+561095us) spinlock_profiling.cc:243] Waited 2.62 s on lock 0x7b5400013e50. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:39.815528 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6026120576}
I20260709 12:11:39.854815 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:39.858381 12530 raft_consensus.cc:2410] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 13 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:11:39.858445 12680 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:11:39.855914 16211 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:11:39.861068 16720 consensus_meta.cc:287] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.523s	user 0.016s	sys 0.000s
W20260709 12:11:39.863605 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 58618.2us
I20260709 12:11:39.864578 12646 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:11:39.866475 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 125714us
I20260709 12:11:39.866591 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:39.867376 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:39.868275 16893 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:39.868772 16893 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:39.868908 12572 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:39.869228 16893 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:39.869894 17126 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:39.817406 16719 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:39.870071 12572 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:39.870551 12528 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:11:39.884867 12528 pending_rounds.cc:85] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 7
I20260709 12:11:39.861608 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:39.887907 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:39.861966 12680 pending_rounds.cc:85] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 4
I20260709 12:11:39.885129 12528 pending_rounds.cc:107] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:11:39.892804 12528 raft_consensus.cc:2889] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:39.892993 12646 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:39.892377 12680 pending_rounds.cc:107] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:11:39.857924 17127 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:39.894869 12680 raft_consensus.cc:2889] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:39.870585 16875 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:11:39.896289 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:39.897748 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:39.870491 17126 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:39.921051 16211 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:11:39.899477 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 109528us
I20260709 12:11:39.889161 16720 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:39.926632 16211 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:11:39.927675 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:39.931757 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:11:39.937170 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 71076.3us
I20260709 12:11:39.897672 17077 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:39.939482 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:39.941310 17077 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:39.942034 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:39.942721 12530 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:39.943451 17123 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:39.937001 16720 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.947335 12522 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:11:39.900568 17162 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 1.296s	user 0.000s	sys 0.005s
I20260709 12:11:39.948326 16720 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:39.899770 16671 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:39.956405 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:39.951817 13304 leader_election.cc:400] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 11. Current term is 12.
I20260709 12:11:39.940874 16211 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:39.960407 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2567) took 1082 ms. Trace:
I20260709 12:11:39.963835 12680 rpcz_store.cc:276] 0709 12:11:38.877591 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.748196 (+870605us) service_pool.cc:224] Handling call
0709 12:11:39.848923 (+100727us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.960135 (+111212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:39.960346 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:39.929526 16919 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:39.971204 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:39.953892 16760 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:39.975524 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:39.977221 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 107266us
I20260709 12:11:39.991801 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:39.994055 16935 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:11:39.996135 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 56763.3us
I20260709 12:11:40.004941 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:39.971740 16671 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:40.005780 12530 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:40.014966 12646 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:40.020051 16893 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.022867 16893 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:40.024138 17074 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:11:40.030349 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 108570us
I20260709 12:11:40.032446 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:40.036118 16760 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:40.036600 16760 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:40.037010 16760 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:40.038211 13338 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:40.039525 17126 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.040800 16875 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:11:40.041590 16875 consensus_queue.cc:1243] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 13.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:11:40.005321 17127 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
I20260709 12:11:40.044420 17127 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:40.044905 17127 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:40.047088 12496 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:40.054854 12676 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 12. Ops: [9.5-11.6]
I20260709 12:11:40.056088 17094 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:40.056620 17094 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:40.057112 17094 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:39.994891 16935 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.060350 16935 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:40.061544 16671 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.062315 16919 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:11:40.063664 16919 consensus_queue.cc:1243] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 12.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:11:40.064296 17077 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:40.064730 17077 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:40.016688 17122 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:40.065688 17122 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:11:40.065853 17077 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:40.067243 17077 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:40.068915 17062 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:40.075796 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:40.007596 16926 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12.
W20260709 12:11:40.080937 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 63390.4us
W20260709 12:11:40.082020 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:40.011829 16720 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 10.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 12:11:40.086513 12646 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:40.087098 12527 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:11:40.084589 13305 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:40.093575 13305 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:40.096364 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:40.096717 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 806574us
I20260709 12:11:40.097641 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:40.102171 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2566) took 1224 ms. Trace:
I20260709 12:11:40.102824 12676 rpcz_store.cc:276] 0709 12:11:38.877334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.716927 (+839593us) service_pool.cc:224] Handling call
0709 12:11:39.753216 (+ 36289us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:40.055410 (+302194us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:40.101785 (+ 46375us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b54003d40d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:40.102050 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":603244416}
I20260709 12:11:40.104732 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:40.111719 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:40.117071 12602 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:40.098193 16919 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:40.119024 16919 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:40.119937 13305 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:40.121330 12646 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:40.122324 12646 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:40.122479 16875 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:40.123114 16875 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:40.123272 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2173) took 1793 ms. Trace:
I20260709 12:11:40.123658 16875 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:40.123736 12523 rpcz_store.cc:276] 0709 12:11:38.330098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:38.633441 (+303343us) service_pool.cc:224] Handling call
0709 12:11:38.633726 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:39.531093 (+897367us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:40.111241 (+580148us) spinlock_profiling.cc:243] Waited 896 ms on lock 0x7b540042a4d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:40.123213 (+ 11972us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2061584256}
I20260709 12:11:40.125792 17123 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:40.126279 17123 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:40.126668 17123 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:40.127511 12530 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:40.127585 16926 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:40.128382 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:40.090333 16720 consensus_queue.cc:1243] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 12.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:11:40.130281 16962 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:40.130869 16962 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:40.131901 16962 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.045933 17127 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.126295 17021 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:40.139863 16719 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:40.133549 17041 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:40.141695 17127 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:40.140175 17021 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:40.142510 16726 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:40.142987 16726 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:40.143074 17021 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:40.144023 16726 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.145187 16726 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:40.148078 12603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 417) took 2428 ms (client timeout 2734 ms). Trace:
W20260709 12:11:40.148545 12603 rpcz_store.cc:269] 0709 12:11:37.719917 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:37.720059 (+   142us) service_pool.cc:224] Handling call
0709 12:11:40.147694 (+2427635us) spinlock_profiling.cc:243] Waited 1.86 s on lock 0x7b54002e59d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:40.147878 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4288620928}
I20260709 12:11:40.150615 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:40.151248 12600 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:40.151640 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2183) took 1583 ms. Trace:
I20260709 12:11:40.152127 12675 rpcz_store.cc:276] 0709 12:11:38.568559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.522519 (+953960us) service_pool.cc:224] Handling call
0709 12:11:39.556163 (+ 33644us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:39.628462 (+ 72299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:39.628493 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:39.629018 (+   525us) log.cc:844] Serialized 26 byte log entry
0709 12:11:40.150567 (+521549us) spinlock_profiling.cc:243] Waited 54 ms on lock 0x7b540039b5a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148898a090 00007f14889976b8 00007f1488a2092b 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c
0709 12:11:40.150633 (+    66us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:40.150964 (+   331us) log.cc:844] Serialized 14 byte log entry
0709 12:11:40.151164 (+   200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:40.151240 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:40.151302 (+    62us) raft_consensus.cc:1685] finished
0709 12:11:40.151408 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:40.151557 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12869,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"spinlock_wait_cycles":124235520,"thread_start_us":67625,"threads_started":1,"wal-append.queue_time_us":62213}
I20260709 12:11:40.153182 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:40.153926 12606 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:40.155000 16960 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:40.155764 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:40.161585 12671 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:40.088456 17074 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:40.094502 17062 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:40.167881 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:40.169134 16720 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:40.169749 16720 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:40.170943 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:40.172369 16720 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:40.173483 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:40.173659 13331 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:40.174242 12602 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:40.176146 13332 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:40.177616 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:40.179343 16719 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:40.179621 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:40.179932 16719 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:40.183194 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 83946.3us
I20260709 12:11:40.218506 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:40.182402 16719 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:40.184465 17168 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:40.222698 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:40.224925 12522 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 13
W20260709 12:11:40.225855 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:40.237317 12530 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 10.
I20260709 12:11:40.244067 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:40.243675 12427 catalog_manager.cc:5697] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:11:40.246657 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:40.247793 13339 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 7: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:40.243923 17168 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:40.250746 17168 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:40.251044 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:40.253789 17094 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.245491 12427 catalog_manager.cc:5697] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:40.258056 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2188) took 1679 ms. Trace:
I20260709 12:11:40.258598 12671 rpcz_store.cc:276] 0709 12:11:38.578239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:39.664864 (+1086625us) service_pool.cc:224] Handling call
0709 12:11:39.699385 (+ 34521us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:40.162121 (+462736us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:40.257783 (+ 95662us) spinlock_profiling.cc:243] Waited 462 ms on lock 0x7b54003be054. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:40.257987 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1061792640}
W20260709 12:11:40.270671 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 56312us
I20260709 12:11:40.215269 16720 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:40.288453 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:40.114639 16943 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:40.293098 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 99198.1us
I20260709 12:11:40.297962 13331 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 13 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13. Ops: []
W20260709 12:11:40.300556 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:40.274623 17168 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:40.302109 17168 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.302922 17056 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:40.303694 17056 consensus_queue.cc:1243] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:11:40.304458 17168 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:40.304358 17056 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:40.305284 17197 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:40.306012 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:40.307852 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:40.323175 13305 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:40.326411 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:40.329174 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:40.239106 17074 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.333772 12606 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 12. Ops: [10.8-11.9]
I20260709 12:11:40.330348 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:40.331084 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:40.332480 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:40.258598 16875 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.338579 12528 raft_consensus.cc:2410] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:11:40.313925 17094 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:40.354665 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:40.373509 16211 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:40.373997 16211 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:40.321959 16926 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:40.319840 16719 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:40.291808 16893 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.045s
I20260709 12:11:40.377064 12527 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:40.377694 12524 raft_consensus.cc:2410] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
W20260709 12:11:40.378878 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 43911us
I20260709 12:11:40.379642 12531 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:40.381270 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:40.289892 16720 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:40.387050 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:40.387383 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:40.387984 17172 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
W20260709 12:11:40.428938 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:40.395380 16943 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:40.382422 16211 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:40.430730 16211 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:40.405194 16935 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:11:40.422842 13302 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.734s (SENT) [suppressed 16 similar messages]
I20260709 12:11:40.406090 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:40.434588 12531 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:40.405948 16926 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.406844 16893 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:40.409063 16875 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:40.410414 12529 raft_consensus.cc:2368] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 10. Current term is 11.
I20260709 12:11:40.412696 17094 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:40.375599 17051 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:40.445091 16935 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.445605 17051 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:40.390275 16719 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.401106 12427 catalog_manager.cc:5697] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:40.448578 17051 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:40.448859 16760 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.449441 16926 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:40.450237 12602 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:40.451344 17051 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:40.452260 16720 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:40.452620 17074 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:40.439793 12523 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:40.453474 16719 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:40.454288 12524 pending_rounds.cc:85] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 6
I20260709 12:11:40.454666 12524 pending_rounds.cc:107] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:11:40.454912 12524 raft_consensus.cc:2889] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:11:40.463479 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 76594.2us
I20260709 12:11:40.464852 16719 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:11:40.465521 12494 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:40.461984 12527 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
W20260709 12:11:40.444240 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 315965us
I20260709 12:11:40.378805 17197 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.437850 12528 raft_consensus.cc:2368] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10.
I20260709 12:11:40.477138 12427 catalog_manager.cc:5697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:40.483459 12527 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:40.436959 16726 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:40.508538 12529 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:40.509546 12529 raft_consensus.cc:606] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:40.509899 12529 raft_consensus.cc:614] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:40.513267 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:40.517081 12522 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
I20260709 12:11:40.528560 16211 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:40.538703 12427 catalog_manager.cc:5697] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 9 to 10, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 10 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:40.466384 17074 raft_consensus.cc:3055] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:40.477669 17168 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:40.547572 17123 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.466233 16719 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 12:11:40.550557 16719 raft_consensus.cc:3055] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:40.551009 16760 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:40.496429 16893 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.552500 12571 leader_election.cc:400] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9.
I20260709 12:11:40.447727 16935 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:40.495976 17094 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.557228 16962 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:40.559533 12527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:40.560344 12527 raft_consensus.cc:606] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:40.560832 12527 raft_consensus.cc:614] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:40.564949 12527 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 12 index: 9. (term mismatch)
I20260709 12:11:40.541265 12427 catalog_manager.cc:5697] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 8 to 9, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 9 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:40.566650 16935 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:11:40.551090 16719 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:40.565424 12527 pending_rounds.cc:85] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 8
I20260709 12:11:40.573431 12527 pending_rounds.cc:107] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 11.9
I20260709 12:11:40.573817 12527 raft_consensus.cc:2889] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:40.518415 17197 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:40.547685 17074 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:40.554020 16893 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:40.576020 12523 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:11:40.594873 12527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:40.595876 12527 raft_consensus.cc:606] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:40.596356 12527 raft_consensus.cc:614] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:40.601554 16211 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.008s
W20260709 12:11:40.612278 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:40.585813 17094 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:40.619349 12494 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:40.636570 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:11:40.644752 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 342623us
I20260709 12:11:40.616103 12525 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:11:40.657789 12494 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:40.648329 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 325228us
I20260709 12:11:40.664446 13305 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:40.675199 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:40.615783 16720 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:40.680397 12494 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:40.635226 17123 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:40.685973 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:40.686376 12671 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:40.701972 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:40.666805 17197 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:11:40.704485 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47493.8us
I20260709 12:11:40.705245 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:40.709223 12676 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 10: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:40.681720 17122 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:40.718688 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:40.714741 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:40.719821 13338 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:40.697374 17077 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader election won for term 10
W20260709 12:11:40.721889 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 410131us
I20260709 12:11:40.722669 16720 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:40.737105 17214 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 12:11:40.740106 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:40.742421 16671 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:40.743202 16671 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:40.743716 16671 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.708585 16893 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:40.753260 16893 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:40.714280 17021 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:40.707983 17197 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:40.755084 17197 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:40.759217 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:40.729317 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:40.766908 12646 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:40.763831 12671 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:40.765571 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:40.770068 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 54590us
I20260709 12:11:40.773952 16926 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:40.775859 16893 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:40.778615 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:40.779313 12602 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:40.682237 17074 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:40.782897 16211 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:40.783632 16211 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:40.784054 16806 raft_consensus.cc:3055] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:40.784767 16806 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:40.786566 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:40.786705 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:40.787266 12674 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:40.787907 12600 raft_consensus.cc:2410] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:11:40.788451 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:40.789063 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:40.789839 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:40.790721 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:40.792390 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:40.794230 16962 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:40.794839 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:40.795284 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:40.797067 12675 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:11:40.794862 16962 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:40.800994 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:40.802026 16720 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:40.802224 12601 raft_consensus.cc:2410] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:11:40.804785 16962 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:40.806660 16211 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:40.755146 17212 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:11:40.801939 16719 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:40.814906 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 44045.1us
I20260709 12:11:40.820178 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:40.826439 13339 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 7: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:40.745870 16671 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:40.770291 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 45398.5us
I20260709 12:11:40.835561 12525 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:40.836516 12525 raft_consensus.cc:606] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:40.836941 12525 raft_consensus.cc:614] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:11:40.839068 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 418067us
I20260709 12:11:40.818718 12496 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:40.842098 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 44030.6us
W20260709 12:11:40.810046 17127 consensus_meta.cc:287] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.647s	user 0.011s	sys 0.001s
W20260709 12:11:40.844074 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:40.844185 12496 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:40.847194 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:40.799212 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:40.811926 17204 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:11:40.853340 12427 catalog_manager.cc:5697] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 9 to 10, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 10 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:40.817548 16719 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:40.785818 16806 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:40.860886 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:40.841849 17134 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:40.847950 17094 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:40.862577 12672 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:40.865206 12427 catalog_manager.cc:5697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 9 to 10, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 10 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:11:40.871505 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 51458.6us
I20260709 12:11:40.872586 12572 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:40.873931 12496 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:40.849860 16726 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:40.854583 16893 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:11:40.882486 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 153336us
I20260709 12:11:40.881093 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:40.885185 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:40.890110 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:40.864759 17127 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:40.898854 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 218648us
I20260709 12:11:40.902308 13339 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 10. (term mismatch)
I20260709 12:11:40.905629 13339 pending_rounds.cc:85] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 9
I20260709 12:11:40.905894 13339 pending_rounds.cc:107] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:11:40.906138 13339 raft_consensus.cc:2889] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:40.905995 12494 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:40.906765 12530 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:40.908701 12525 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:40.909271 12525 raft_consensus.cc:606] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:40.909622 12525 raft_consensus.cc:614] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:40.915360 17056 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:40.925609 12646 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:40.926425 17041 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:40.927151 17062 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:40.927428 12646 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:40.927738 17062 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:40.928251 17062 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:40.878048 17227 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:40.929219 17227 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:40.931375 17227 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:40.878580 16671 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:40.886003 16893 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:40.934319 16671 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:40.934860 16671 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:40.935580 16720 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:40.936113 16720 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.934664 16893 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:40.938338 16720 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:40.938949 17232 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:11:40.939605 17232 consensus_queue.cc:1243] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:11:40.940764 17062 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.877684 17122 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:40.881875 16726 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:40.975750 12646 leader_election.cc:400] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 10. Current term is 11.
I20260709 12:11:40.976155 12646 leader_election.cc:403] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:11:40.974566 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 102112us
I20260709 12:11:40.977531 12572 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:11:40.978528 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:40.978528 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 72627.4us
I20260709 12:11:40.979837 12494 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:40.943503 17127 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:40.907581 17168 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.119s
I20260709 12:11:40.988010 16720 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:41.027757 16720 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:41.027920 17234 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:41.028270 16720 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:41.028680 17234 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:41.029517 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:41.030531 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:41.030803 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:41.024987 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:41.031817 12600 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:41.032266 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:41.032940 12675 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:41.033099 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:41.021867 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:41.034871 12671 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:11:41.035418 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:11:41.035938 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:41.037693 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:41.039383 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:41.041373 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:41.023869 12606 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:41.042917 17230 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:41.018967 12603 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 12:11:41.044245 12603 pending_rounds.cc:85] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 10
I20260709 12:11:41.044507 12603 pending_rounds.cc:107] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:11:41.044770 12603 raft_consensus.cc:2889] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:41.045677 12672 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: [9.7-9.7]
W20260709 12:11:41.045800 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 98973.6us
I20260709 12:11:41.046012 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:41.046583 12646 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:41.046931 13331 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 11: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:41.047401 12676 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 9: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:41.047852 12646 leader_election.cc:400] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10.
I20260709 12:11:41.025727 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:41.048519 17058 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:41.048581 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:41.049082 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:41.049144 17226 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:41.049544 17234 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:41.048623 17126 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:11:41.023314 12426 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59006 (request call id 175) took 43820 ms (client timeout 15000 ms). Trace:
I20260709 12:11:41.050889 12674 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:41.050899 12601 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:11:41.051306 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:41.052085 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:41.052973 16875 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:41.053153 13332 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:40.988507 12529 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
W20260709 12:11:41.013087 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:40.960026 17074 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:40.988677 17229 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:41.018050 16962 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:41.016510 17062 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:41.054785 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:41.055392 12427 catalog_manager.cc:5697] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 4 to 6, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 6 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:11:41.047768 17230 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 10. Current term is 11.
I20260709 12:11:41.021814 16797 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:41.067090 12600 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
I20260709 12:11:41.008563 12494 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:41.079342 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:41.097733 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:41.098417 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:40.986864 16726 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:41.107088 16962 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:41.114353 17229 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:41.103200 16875 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:41.116755 17229 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:41.116957 16875 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:41.088696 17021 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 13 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:41.121349 17021 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:41.027230 17222 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:41.122452 17222 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:41.122852 17222 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:41.103694 17235 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.023s
I20260709 12:11:41.129303 13302 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:41.130313 17231 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260709 12:11:41.130472 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:41.130892 17241 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:41.131958 17241 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:41.132424 17241 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:41.133193 17234 raft_consensus.cc:3055] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:41.133821 17234 raft_consensus.cc:740] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:41.134985 17234 consensus_queue.cc:260] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:41.136322 17234 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:41.138988 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:41.140664 12598 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:11:41.141716 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:40.959460 17122 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:11:41.142493 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:41.142680 17122 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:41.145089 17197 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:41.147327 17197 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:41.028005 17232 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.077s
I20260709 12:11:41.126593 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:41.149394 17094 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:41.149868 17094 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:41.150254 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:41.151494 16797 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:41.151973 16797 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:41.106033 17058 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:11:41.126605 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 78869us
W20260709 12:11:41.154368 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 68140.9us
W20260709 12:11:41.155316 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 146864us
I20260709 12:11:41.156481 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:41.154011 12646 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:41.156683 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:41.157356 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:41.151463 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:41.154023 12604 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 11: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:41.160082 12605 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:11:41.162766 12601 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:11:41.164091 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:41.165730 12598 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:41.167960 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:41.168699 12606 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 7.
I20260709 12:11:41.169459 17041 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:41.153855 17058 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:41.172479 17062 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:41.173610 17168 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:41.174535 17168 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:41.175390 16962 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:41.169979 12606 raft_consensus.cc:1217] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Deduplicated request from leader. Original: 6.4->[9.5-11.6]   Dedup: 9.5->[11.6-11.6]
I20260709 12:11:41.177008 12606 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: [9.5-11.6]
I20260709 12:11:41.165005 12646 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:41.178557 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:41.158223 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:41.190665 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:40.976609 16806 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
W20260709 12:11:41.221128 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 57115.4us
I20260709 12:11:41.222074 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:41.193154 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:41.194765 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:41.069252 16935 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.107s
I20260709 12:11:41.223789 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:11:41.242566 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 77646.6us
I20260709 12:11:41.247752 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:41.248699 12530 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:41.250864 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:41.254274 12679 pending_rounds.cc:85] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 5
I20260709 12:11:41.254745 12679 pending_rounds.cc:107] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:11:41.255070 12679 raft_consensus.cc:2889] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:11:41.299953 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 90524.4us
I20260709 12:11:41.130110 17235 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.112s
I20260709 12:11:41.175323 17062 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.125s
I20260709 12:11:41.175925 17197 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:41.321323 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:41.216589 16962 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:11:41.308557 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:41.338640 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:41.375751 12527 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:41.204283 16893 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:41.262280 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:41.321659 12522 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 6. (term mismatch)
I20260709 12:11:41.381407 12522 pending_rounds.cc:85] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 5
I20260709 12:11:41.381819 12522 pending_rounds.cc:107] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:11:41.382190 12522 raft_consensus.cc:2889] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:41.306008 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:41.386312 12523 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:11:41.283999 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 354309us
I20260709 12:11:41.227025 17077 raft_consensus.cc:697] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:41.262470 16875 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:41.370765 12599 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:11:41.398330 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:41.398552 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:41.318753 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:41.400602 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:41.227993 17126 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.173s
I20260709 12:11:41.378882 17197 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:41.380820 12529 raft_consensus.cc:2393] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 12: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:41.371417 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:41.405092 12531 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:41.241523 17168 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:41.317963 17229 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 7.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.073s
W20260709 12:11:41.309553 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 82130.8us
W20260709 12:11:41.377666 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 323671us
I20260709 12:11:41.407027 12494 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:41.338416 17240 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:41.403416 12603 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:11:41.404131 12602 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 11: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:41.404264 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:41.413245 12600 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
I20260709 12:11:41.381462 17127 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:41.426745 12528 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:11:41.380683 16962 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:41.387833 12523 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:41.306407 17232 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:41.434201 12527 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:41.377916 16935 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.035s
I20260709 12:11:41.401345 17056 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:41.410393 17168 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:41.409957 17197 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:41.495143 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:41.435468 16962 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:41.443500 17127 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:11:41.509402 17134 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.630s	user 0.012s	sys 0.000s
I20260709 12:11:41.485850 17168 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:41.585839 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:41.586728 12597 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
W20260709 12:11:41.572762 16719 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.704s	user 0.012s	sys 0.000s
I20260709 12:11:41.608979 12530 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:41.514780 17127 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:41.487928 17240 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:41.493289 17056 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:41.620802 16671 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.685s	user 0.019s	sys 0.000s
I20260709 12:11:41.607254 17134 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:11:41.643069 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 253354us
W20260709 12:11:41.077991 12426 rpcz_store.cc:269] 0709 12:10:57.202931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:10:57.204001 (+  1070us) service_pool.cc:224] Handling call
0709 12:11:41.023049 (+43819048us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:11:34.551930 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:11:34.552675 (+   745us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:11:34.552736 (+    61us) write_op.cc:435] Acquired schema lock
0709 12:11:34.552770 (+    34us) tablet.cc:662] Decoding operations
0709 12:11:34.553723 (+   953us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:11:34.553825 (+   102us) write_op.cc:641] Partition lock acquired for write op
0709 12:11:34.553856 (+    31us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:11:34.879231 (+325375us) tablet.cc:701] Row locks acquired
0709 12:11:34.921473 (+ 42242us) write_op.cc:260] PREPARE: finished
0709 12:11:34.963166 (+ 41693us) write_op.cc:270] Start()
0709 12:11:34.982695 (+ 19529us) write_op.cc:276] Timestamp: P: 1783599094949705 usec, L: 0
0709 12:11:34.992466 (+  9771us) op_driver.cc:348] REPLICATION: starting
0709 12:11:35.158724 (+166258us) log.cc:844] Serialized 2870 byte log entry
0709 12:11:35.738739 (+580015us) op_driver.cc:464] REPLICATION: finished
0709 12:11:35.803840 (+ 65101us) write_op.cc:301] APPLY: starting
0709 12:11:35.803948 (+   108us) tablet.cc:1367] starting BulkCheckPresence
0709 12:11:36.057857 (+253909us) tablet.cc:1370] finished BulkCheckPresence
0709 12:11:36.057988 (+   131us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:11:36.811362 (+753374us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:11:36.885621 (+ 74259us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=14,mrs_lookups=0
0709 12:11:36.891270 (+  5649us) write_op.cc:312] APPLY: finished
0709 12:11:36.914137 (+ 22867us) log.cc:844] Serialized 73 byte log entry
0709 12:11:37.039941 (+125804us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:37.050050 (+ 10109us) write_op.cc:454] Released schema lock
0709 12:11:37.059168 (+  9118us) write_op.cc:341] FINISH: Updating metrics
0709 12:11:37.114282 (+ 55114us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:37.125240 (+ 10958us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":101560,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":63636,"apply.run_cpu_time_us":81782,"apply.run_wall_time_us":1298648,"cfile_cache_hit":26,"cfile_cache_hit_bytes":27131,"cfile_cache_miss":4,"cfile_cache_miss_bytes":6214,"cfile_init":2,"lbm_read_time_us":21671,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":11,"num_ops":7,"prepare.queue_time_us":271178,"prepare.run_cpu_time_us":25067,"prepare.run_wall_time_us":921510,"raft.queue_time_us":114223,"raft.run_cpu_time_us":21019,"raft.run_wall_time_us":42495,"replication_time_us":746039,"thread_start_us":589615,"threads_started":5,"wal-append.queue_time_us":454393,"wal-append.run_cpu_time_us":34526,"wal-append.run_wall_time_us":2365942}]]}
I20260709 12:11:41.628085 17127 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:41.664426 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:41.672178 12646 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:41.684082 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 228677us
W20260709 12:11:41.684953 17236 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:11:41.685797 16919 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:41.686761 16919 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:41.687220 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2240) took 1370 ms. Trace:
I20260709 12:11:41.687378 16919 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:41.687620 12524 rpcz_store.cc:276] 0709 12:11:40.316467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:40.391271 (+ 74804us) service_pool.cc:224] Handling call
0709 12:11:40.391691 (+   420us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:40.455981 (+ 64290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:40.456015 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:40.456656 (+   641us) log.cc:844] Serialized 52 byte log entry
0709 12:11:40.907450 (+450794us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:40.907492 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:41.024434 (+116942us) spinlock_profiling.cc:243] Waited 62.2 ms on lock 0x7b5400052b54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:41.024482 (+    48us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:41.686846 (+662364us) raft_consensus.cc:1685] finished
0709 12:11:41.686869 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:41.687156 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1534,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"spinlock_wait_cycles":143164800,"thread_start_us":450373,"threads_started":1,"wal-append.queue_time_us":794126}
W20260709 12:11:41.689476 17247 log.cc:927] Time spent T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.170s	user 0.000s	sys 0.001s
I20260709 12:11:41.690290 16919 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:41.690855 17264 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:41.691776 17264 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:41.692307 17264 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:41.694520 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2583) took 1074 ms. Trace:
I20260709 12:11:41.694563 17266 consensus_queue.cc:1059] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:41.695098 12673 rpcz_store.cc:276] 0709 12:11:40.619834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:40.619979 (+   145us) service_pool.cc:224] Handling call
0709 12:11:40.643664 (+ 23685us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:40.677738 (+ 34074us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:40.677812 (+    74us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:40.710242 (+ 32430us) log.cc:844] Serialized 26 byte log entry
0709 12:11:41.226651 (+516409us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:41.226696 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:41.226777 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:41.694193 (+467416us) raft_consensus.cc:1685] finished
0709 12:11:41.694221 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:41.694417 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15929,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":11852,"thread_start_us":383955,"threads_started":1,"wal-append.queue_time_us":646399}
I20260709 12:11:41.701275 17266 raft_consensus.cc:3055] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Stepping down as leader of term 6
I20260709 12:11:41.701944 17266 raft_consensus.cc:740] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:41.703325 17264 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:41.674075 16875 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:41.708375 17240 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:41.703460 17266 consensus_queue.cc:260] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:41.710383 16875 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:41.712646 17266 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 6 FOLLOWER]: Advancing to term 7
I20260709 12:11:41.714592 16893 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:41.643419 17227 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:41.719830 17227 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:41.724570 16720 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.696s	user 0.007s	sys 0.003s
I20260709 12:11:41.725198 16720 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:41.728212 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:41.783985 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:41.657199 17241 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:41.786209 12676 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:41.664654 16671 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:41.929015 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:41.965929 16671 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:41.627552 17077 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:41.797624 17134 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:41.993570 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:41.976121 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:41.795924 17126 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:41.996039 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 323917us
W20260709 12:11:42.022107 12522 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.578s	user 0.081s	sys 0.152s
I20260709 12:11:41.806443 17051 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:42.024535 12522 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.581s	user 0.083s	sys 0.152s
W20260709 12:11:42.030689 12494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.010s	sys 0.010s
W20260709 12:11:42.040800 12494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.633s	user 0.010s	sys 0.010s
W20260709 12:11:42.041339 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 634355us
I20260709 12:11:42.042924 12494 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:42.083108 12646 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:42.066910 12494 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:42.050274 17229 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.005s	sys 0.014s
I20260709 12:11:42.086078 17241 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:11:42.061120 16726 consensus_meta.cc:287] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.950s	user 0.010s	sys 0.001s
W20260709 12:11:42.125346 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 318188us
I20260709 12:11:42.125829 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:42.134193 12497 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.654s	user 0.014s	sys 0.021s
W20260709 12:11:42.135826 12497 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.015s	sys 0.021s
I20260709 12:11:42.077574 16719 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:42.091625 16943 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:11:42.116058 17229 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.005s	sys 0.017s
I20260709 12:11:42.030040 17126 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:42.178026 12645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.013s	sys 0.002s
W20260709 12:11:42.178570 12645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.871s	user 0.013s	sys 0.003s
W20260709 12:11:42.170159 17041 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.000s	user 0.012s	sys 0.001s
I20260709 12:11:42.161981 16943 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:42.198796 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:42.183462 17256 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.606s	user 0.002s	sys 0.000s
I20260709 12:11:42.161536 16671 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:42.042084 17051 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:42.190091 17222 consensus_meta.cc:287] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.067s	user 0.015s	sys 0.000s
W20260709 12:11:42.201274 16797 consensus_meta.cc:287] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.049s	user 0.008s	sys 0.005s
I20260709 12:11:42.144424 16726 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:42.256323 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 127175us
I20260709 12:11:42.265070 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:42.271029 17094 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.121s	user 0.013s	sys 0.001s
I20260709 12:11:42.271102 13331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38482 (request call id 482) took 1217 ms. Trace:
I20260709 12:11:42.226866 17041 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260709 12:11:42.041501 17134 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:42.227016 16943 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:42.290102 12522 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:42.252637 17264 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:42.192370 16760 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:11:42.296945 12531 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.891s	user 0.003s	sys 0.008s
I20260709 12:11:42.215071 17240 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:42.255923 17287 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:11:42.294425 17251 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.612s	user 0.002s	sys 0.000s
I20260709 12:11:42.251371 17074 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:42.275880 13331 rpcz_store.cc:276] 0709 12:11:41.053934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.054416 (+   482us) service_pool.cc:224] Handling call
0709 12:11:41.054795 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:41.056864 (+  2069us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:41.056892 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:41.057325 (+   433us) log.cc:844] Serialized 26 byte log entry
0709 12:11:41.361058 (+303733us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:41.361163 (+   105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:41.361261 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:42.270733 (+909472us) raft_consensus.cc:1685] finished
0709 12:11:42.270772 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:42.270991 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1551,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":80,"spinlock_wait_cycles":1792,"thread_start_us":303226,"threads_started":1,"wal-append.queue_time_us":377758}
I20260709 12:11:42.307649 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:42.308368 12531 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:42.266021 16719 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:42.298018 12524 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:42.320340 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2258) took 1057 ms. Trace:
W20260709 12:11:42.321631 17273 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:11:42.323681 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:42.253566 16671 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:42.329523 12531 rpcz_store.cc:276] 0709 12:11:41.263329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.340044 (+ 76715us) service_pool.cc:224] Handling call
0709 12:11:42.320268 (+980224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:42.332661 12673 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:42.257972 16797 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:42.298548 17231 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:42.255743 17222 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:42.308283 16720 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:42.326211 17258 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.498s	user 0.000s	sys 0.001s
I20260709 12:11:42.285069 17094 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:42.408792 12602 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:42.370424 17051 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:42.326211 17283 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election won for term 8
I20260709 12:11:42.413268 12677 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 11: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:11:42.416558 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 107609us
I20260709 12:11:42.307132 17264 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:42.317212 16760 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:42.314841 17263 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.372s
I20260709 12:11:42.434945 12523 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:42.439399 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:42.411264 17122 consensus_meta.cc:287] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.268s	user 0.013s	sys 0.002s
I20260709 12:11:42.319299 17240 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:42.449775 12597 consensus_meta.cc:287] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.862s	user 0.005s	sys 0.005s
I20260709 12:11:42.450176 12597 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:42.379957 17074 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:42.453480 17260 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.410s	user 0.000s	sys 0.002s
W20260709 12:11:42.454734 17265 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.310s	user 0.000s	sys 0.002s
I20260709 12:11:42.447208 17122 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:42.455588 16962 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.883s	user 0.009s	sys 0.001s
I20260709 12:11:42.456000 16962 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:42.384150 16719 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:42.443058 17263 consensus_queue.cc:1243] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:11:42.370532 17287 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:42.474496 16719 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:42.395735 17231 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:42.476922 17231 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:42.478453 12677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2196) took 1452 ms (client timeout 1681 ms). Trace:
I20260709 12:11:42.479051 16760 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:42.478922 12677 rpcz_store.cc:269] 0709 12:11:41.026497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.048322 (+ 21825us) service_pool.cc:224] Handling call
0709 12:11:42.478244 (+1429922us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b5400012cd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:42.478388 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3089376512}
I20260709 12:11:42.442495 17283 raft_consensus.cc:697] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:42.488613 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 468) took 1348 ms. Trace:
I20260709 12:11:42.491621 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2210) took 1273 ms. Trace:
W20260709 12:11:42.493659 17259 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
I20260709 12:11:42.493696 12679 rpcz_store.cc:276] 0709 12:11:41.218444 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.246252 (+ 27808us) service_pool.cc:224] Handling call
0709 12:11:41.246678 (+   426us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:41.318533 (+ 71855us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:41.318562 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:41.354524 (+ 35962us) log.cc:844] Serialized 52 byte log entry
0709 12:11:41.636603 (+282079us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:41.636649 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:41.636744 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:42.491383 (+854639us) raft_consensus.cc:1685] finished
0709 12:11:42.491405 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:42.491562 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":190601,"raft.run_cpu_time_us":7949,"raft.run_wall_time_us":62272,"thread_start_us":232457,"threads_started":1,"wal-append.queue_time_us":599251}
I20260709 12:11:42.481062 17134 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:42.501362 17126 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:42.546247 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:42.473968 16962 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:42.489261 12526 rpcz_store.cc:276] 0709 12:11:41.139749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.144346 (+  4597us) service_pool.cc:224] Handling call
0709 12:11:41.144678 (+   332us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:41.145060 (+   382us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:41.145088 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:41.145695 (+   607us) log.cc:844] Serialized 52 byte log entry
0709 12:11:41.424950 (+279255us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:11:41.807962 (+383012us) log.cc:844] Serialized 14 byte log entry
0709 12:11:41.808248 (+   286us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:41.808331 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:42.488290 (+679959us) raft_consensus.cc:1685] finished
0709 12:11:42.488321 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:42.488529 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19061,"prepare.queue_time_us":879647,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":37399,"raft.queue_time_us":6041,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":115487,"thread_start_us":589340,"threads_started":2,"wal-append.queue_time_us":532246}
I20260709 12:11:42.643043 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:42.631925 12643 leader_election.cc:336] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.845s (SENT)
I20260709 12:11:42.644066 12526 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:11:42.646740 17094 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:42.647969 16806 consensus_meta.cc:287] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.407s	user 0.008s	sys 0.005s
I20260709 12:11:42.648288 12526 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 7 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:11:42.648509 17298 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:42.498317 17282 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.008s
I20260709 12:11:42.638736 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 462) took 1662 ms. Trace:
I20260709 12:11:42.652824 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:42.639742 17127 consensus_meta.cc:287] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.957s	user 0.007s	sys 0.006s
I20260709 12:11:42.639438 16943 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:42.640528 16726 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:42.640776 17197 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.146s	user 0.007s	sys 0.007s
I20260709 12:11:42.643769 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:42.633291 17222 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:42.652489 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2204) took 1623 ms. Trace:
I20260709 12:11:42.682928 12675 rpcz_store.cc:276] 0709 12:11:41.028955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.042652 (+ 13697us) service_pool.cc:224] Handling call
0709 12:11:41.042991 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:41.043418 (+   427us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:41.043447 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:41.043935 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 12:11:41.412392 (+368457us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:41.412435 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:41.412939 (+   504us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:42.494044 (+1081105us) raft_consensus.cc:1685] finished
0709 12:11:42.651954 (+157910us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:42.652359 (+   405us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1046,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":137,"thread_start_us":368030,"threads_started":1,"wal-append.queue_time_us":535953}
I20260709 12:11:42.654879 17240 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:42.650456 17298 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:42.649053 17230 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.008s
I20260709 12:11:42.655491 17283 consensus_queue.cc:237] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:42.661516 12525 rpcz_store.cc:276] 0709 12:11:40.976075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:40.976277 (+   202us) service_pool.cc:224] Handling call
0709 12:11:40.976680 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:41.066849 (+ 90169us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:41.066923 (+    74us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:41.067556 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 12:11:41.404351 (+336795us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:41.404391 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:41.404457 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:42.638468 (+1234011us) raft_consensus.cc:1685] finished
0709 12:11:42.638499 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:42.638658 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18271,"raft.queue_time_us":142661,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":83,"thread_start_us":281844,"threads_started":1,"wal-append.queue_time_us":453018}
I20260709 12:11:42.672504 17127 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:42.648932 16797 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:42.673535 17234 consensus_meta.cc:287] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.537s	user 0.024s	sys 0.002s
I20260709 12:11:42.673676 17227 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:42.693953 17127 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:42.643615 17062 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:42.695974 17062 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:42.699870 16943 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:42.700171 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 443) took 1020 ms. Trace:
I20260709 12:11:42.701013 17062 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:42.702355 16935 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:42.702826 16935 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:42.684665 17197 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:42.654618 17231 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:42.706878 17231 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:42.694656 12675 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:42.709293 12675 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:42.710597 12675 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:42.714696 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2208) took 1684 ms. Trace:
I20260709 12:11:42.700711 12598 rpcz_store.cc:276] 0709 12:11:41.679497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.679613 (+   116us) service_pool.cc:224] Handling call
0709 12:11:42.699434 (+1019821us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:11:42.699652 (+   218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:42.699842 (+   190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:42.699867 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:42.699914 (+    47us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:42.699952 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:42.700008 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:42.700112 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":48151424}
I20260709 12:11:42.715183 12676 rpcz_store.cc:276] 0709 12:11:41.030096 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.047984 (+ 17888us) service_pool.cc:224] Handling call
0709 12:11:41.048224 (+   240us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:41.807031 (+758807us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:42.714468 (+907437us) spinlock_profiling.cc:243] Waited 678 ms on lock 0x7b54000131d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:42.714636 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1560425856}
I20260709 12:11:42.701401 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2260) took 1354 ms. Trace:
I20260709 12:11:42.714768 12526 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:42.717964 12528 rpcz_store.cc:276] 0709 12:11:41.347296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.427489 (+ 80193us) service_pool.cc:224] Handling call
0709 12:11:41.472244 (+ 44755us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:41.472625 (+   381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:41.472687 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:41.473291 (+   604us) log.cc:844] Serialized 26 byte log entry
0709 12:11:41.798931 (+325640us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:41.798985 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:41.799061 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:42.701148 (+902087us) raft_consensus.cc:1685] finished
0709 12:11:42.701172 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:42.701336 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":218965,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":93,"thread_start_us":220817,"threads_started":1,"wal-append.queue_time_us":1181955}
W20260709 12:11:42.676739 17268 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.181s	user 0.000s	sys 0.011s
I20260709 12:11:42.725808 12677 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:11:42.703934 16935 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:42.727384 17230 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:42.727640 16935 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:42.727982 17230 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:11:42.729549 17242 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.317s	user 0.004s	sys 0.008s
I20260709 12:11:42.731351 12646 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:42.732751 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:42.733376 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:42.733412 13305 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:42.734310 17094 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:42.735332 17234 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:42.735034 17094 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:42.736109 17234 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:42.736912 17056 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:42.724834 16719 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:42.715296 17222 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:42.740418 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:42.742235 17252 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:42.738451 17134 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:42.742839 16943 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:11:42.764935 12530 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2257) took 1495 ms (client timeout 1854 ms). Trace:
I20260709 12:11:42.743319 12524 pending_rounds.cc:85] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 4
W20260709 12:11:42.765525 12530 rpcz_store.cc:269] 0709 12:11:41.250689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.262986 (+ 12297us) service_pool.cc:224] Handling call
0709 12:11:42.746123 (+1483137us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b5400015c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:42.746296 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":480654080}
I20260709 12:11:42.744215 17172 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:42.782256 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:42.782234 12525 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
W20260709 12:11:42.744342 17257 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.067s	user 0.000s	sys 0.002s
I20260709 12:11:42.746675 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 437) took 1620 ms. Trace:
I20260709 12:11:42.729197 17230 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:42.766078 12524 pending_rounds.cc:107] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 9.5
I20260709 12:11:42.743460 17300 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:42.784691 12524 raft_consensus.cc:2889] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:42.783895 12601 rpcz_store.cc:276] 0709 12:11:41.126555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.133134 (+  6579us) service_pool.cc:224] Handling call
0709 12:11:41.160746 (+ 27612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:41.163175 (+  2429us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:42.746455 (+1583280us) spinlock_profiling.cc:243] Waited 27.2 ms on lock 0x7b540007b550. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:42.746606 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62612224}
I20260709 12:11:42.791577 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:42.793660 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:42.794893 12676 raft_consensus.cc:2410] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 12 because replica has last-logged OpId of term: 9 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
W20260709 12:11:42.769625 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:42.745579 17254 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.039s	user 0.004s	sys 0.000s
I20260709 12:11:42.795573 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:42.801105 12523 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:42.801173 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:42.802142 12525 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:42.738095 17094 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:42.815461 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:42.807678 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:42.816354 12530 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
W20260709 12:11:42.816435 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:42.816479 12675 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:42.820076 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:42.822717 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:42.821905 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:42.834347 12601 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 12 index: 10. (index mismatch)
I20260709 12:11:42.823513 12676 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:42.782714 17172 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:42.835760 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:42.843849 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:42.839495 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:42.838678 12496 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:42.845010 12677 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:42.850032 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:42.802731 16671 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:42.800472 17062 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:42.839354 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:42.860970 12597 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:11:42.767234 16719 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:42.876399 12525 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:11:42.852802 17172 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:42.851706 17230 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:42.800246 17222 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:42.860586 17284 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:42.870822 17240 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:42.914373 12528 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:42.821928 17134 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:42.916395 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 181465us
I20260709 12:11:42.918030 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:42.921991 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:42.952739 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2201) took 1924 ms. Trace:
I20260709 12:11:42.954053 12671 rpcz_store.cc:276] 0709 12:11:41.028103 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.037056 (+  8953us) service_pool.cc:224] Handling call
0709 12:11:41.037369 (+   313us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:41.037740 (+   371us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:41.037764 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:41.058026 (+ 20262us) log.cc:844] Serialized 26 byte log entry
0709 12:11:41.371626 (+313600us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:41.371689 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:41.371782 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:42.743689 (+1371907us) spinlock_profiling.cc:243] Waited 25.6 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:42.892708 (+149019us) raft_consensus.cc:1685] finished
0709 12:11:42.892740 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:42.952542 (+ 59802us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":586,"raft.run_cpu_time_us":214,"raft.run_wall_time_us":214,"spinlock_wait_cycles":58812800,"thread_start_us":312244,"threads_started":1,"wal-append.queue_time_us":453977}
I20260709 12:11:42.889411 17234 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:42.988392 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2310) took 1847 ms. Trace:
I20260709 12:11:42.988852 12604 rpcz_store.cc:276] 0709 12:11:41.141095 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.167921 (+ 26826us) service_pool.cc:224] Handling call
0709 12:11:41.168254 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:41.168528 (+   274us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:11:41.168877 (+   349us) log.cc:844] Serialized 14 byte log entry
0709 12:11:41.397957 (+229080us) log.cc:844] Serialized 14 byte log entry
0709 12:11:42.987894 (+1589937us) spinlock_profiling.cc:243] Waited 9.9 ms on lock 0x7b1800414ca0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287
0709 12:11:42.988008 (+   114us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:42.988062 (+    54us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:42.988115 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:42.988192 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:42.988327 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":55122,"prepare.queue_time_us":32203,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":4043,"raft.queue_time_us":328674,"raft.run_cpu_time_us":383,"raft.run_wall_time_us":33372,"spinlock_wait_cycles":50189056,"thread_start_us":450921,"threads_started":2,"wal-append.queue_time_us":414825}
I20260709 12:11:42.992583 17263 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:42.993098 17263 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:42.993507 17263 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:42.994483 17297 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:42.995096 17297 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:42.996732 17074 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:42.997400 17297 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:42.829648 17252 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:42.889556 17264 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:42.971037 17267 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:42.971869 12523 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:11:42.920746 16671 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:42.972504 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2205) took 1943 ms. Trace:
W20260709 12:11:42.974998 17294 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260709 12:11:43.015455 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:42.976246 17222 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:42.976610 16719 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:42.977605 17056 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:42.985489 12528 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:43.027897 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:42.961063 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 123549us
I20260709 12:11:43.036298 12526 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 9: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:43.003816 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:42.950253 17168 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.322s	user 0.019s	sys 0.003s
I20260709 12:11:42.874460 17062 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:43.010527 17197 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:42.913887 17287 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:43.012724 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:43.017584 12527 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 11: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:43.020138 17285 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:42.951107 17300 consensus_queue.cc:1059] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.070s
I20260709 12:11:43.022243 17252 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:43.024498 17134 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:43.017859 12680 rpcz_store.cc:276] 0709 12:11:41.029240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.040888 (+ 11648us) service_pool.cc:224] Handling call
0709 12:11:41.051386 (+ 10498us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:41.052151 (+   765us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:41.052180 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:41.052773 (+   593us) log.cc:844] Serialized 52 byte log entry
0709 12:11:41.380551 (+327778us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:41.380602 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:41.380683 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:42.744858 (+1364175us) spinlock_profiling.cc:243] Waited 2.37 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:42.949754 (+204896us) raft_consensus.cc:1685] finished
0709 12:11:42.949789 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:42.972395 (+ 22606us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"spinlock_wait_cycles":5461888,"thread_start_us":327222,"threads_started":1,"wal-append.queue_time_us":596727}
I20260709 12:11:43.064620 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:43.065773 12527 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 12: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:42.961570 17240 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:43.038237 12671 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
W20260709 12:11:43.057193 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 242090us
I20260709 12:11:42.950836 17284 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.044939 16962 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:43.025758 17267 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.025987 16671 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:43.118600 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 296775us
I20260709 12:11:43.119087 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:43.120110 12671 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:43.120061 12646 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:43.122213 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:43.123487 12680 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:43.128916 12680 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:43.045660 17168 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.091054 17240 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:43.061111 16943 raft_consensus.cc:697] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:43.133526 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:43.140245 12527 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:43.131584 12646 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:43.144188 12646 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:43.146067 12496 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:43.059487 17056 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.101536 17122 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:43.149783 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:43.152771 12680 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:43.159333 12529 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 13. Current term is 14. Ops: [11.8-13.9]
I20260709 12:11:43.157509 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:43.126152 17315 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:11:43.151412 17307 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.091s	user 0.001s	sys 0.000s
I20260709 12:11:43.061599 17285 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.161334 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:43.164913 12496 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:43.172050 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 127627us
W20260709 12:11:43.176669 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:43.171330 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 218711us
I20260709 12:11:43.149648 17231 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:43.151630 17197 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:11:43.185068 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:43.186717 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:43.188108 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:43.189348 12527 raft_consensus.cc:2393] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 11: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:43.113868 16962 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:43.194489 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:43.195813 12531 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
W20260709 12:11:43.202212 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 57916.1us
I20260709 12:11:43.202721 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:43.203109 12646 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:43.203495 12680 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:43.204599 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:43.204756 12526 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:43.193871 16935 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.015s
W20260709 12:11:43.206930 17303 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.236s	user 0.001s	sys 0.001s
I20260709 12:11:43.208772 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:43.209266 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:43.213527 12680 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 11: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:43.187608 17122 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:43.186463 17197 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:43.226452 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 85489.3us
I20260709 12:11:43.174156 17315 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.211558 17134 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:43.214514 17252 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:43.214337 16719 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:43.216874 17122 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:43.218978 17197 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:43.244464 12531 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: [9.7-9.7]
W20260709 12:11:43.246898 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 60964.3us
I20260709 12:11:43.215543 16943 consensus_queue.cc:237] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.252018 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:11:43.230808 17302 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.262s	user 0.000s	sys 0.001s
I20260709 12:11:43.219625 17062 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:43.260774 12675 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:11:43.236756 17234 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:43.253659 17122 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:43.257493 16760 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:11:43.316166 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52160.9us
I20260709 12:11:43.321113 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:43.243470 17252 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:11:43.325335 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:43.329311 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:43.328056 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 117080us
I20260709 12:11:43.336148 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2215) took 1137 ms. Trace:
I20260709 12:11:43.302223 17234 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:43.337086 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:43.254215 17134 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.352540 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:43.336716 12678 rpcz_store.cc:276] 0709 12:11:42.199042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:42.199208 (+   166us) service_pool.cc:224] Handling call
0709 12:11:42.227582 (+ 28374us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:42.445067 (+217485us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:42.445199 (+   132us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:42.584667 (+139468us) log.cc:844] Serialized 52 byte log entry
0709 12:11:42.795062 (+210395us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:11:42.838464 (+ 43402us) log.cc:844] Serialized 14 byte log entry
0709 12:11:43.114803 (+276339us) log.cc:844] Serialized 14 byte log entry
0709 12:11:43.115028 (+   225us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:43.115158 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:43.335881 (+220723us) raft_consensus.cc:1685] finished
0709 12:11:43.335915 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:43.336085 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":57207,"prepare.queue_time_us":56535,"prepare.run_cpu_time_us":132,"prepare.run_wall_time_us":5398,"raft.queue_time_us":272935,"raft.run_cpu_time_us":201,"raft.run_wall_time_us":32001,"spinlock_wait_cycles":14411264,"thread_start_us":471784,"threads_started":3,"wal-append.queue_time_us":277457}
I20260709 12:11:43.254084 16719 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.325476 17227 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:43.367547 12643 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:43.359762 17056 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:43.361388 16919 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:43.362799 17317 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:11:43.363282 17334 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:11:43.363781 16962 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:43.353653 17287 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:43.366086 17315 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:43.366341 17062 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:43.336661 17252 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:43.325819 16760 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:43.367024 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2600) took 2119 ms. Trace:
I20260709 12:11:43.334956 17168 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:43.348347 17234 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:43.342121 17284 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:43.393887 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:43.359810 17304 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.225s	user 0.002s	sys 0.000s
I20260709 12:11:43.366762 17230 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:11:43.387681 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:43.397950 12675 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:11:43.404033 12531 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:43.433096 12526 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:43.424615 12674 rpcz_store.cc:276] 0709 12:11:41.247141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.409762 (+162621us) service_pool.cc:224] Handling call
0709 12:11:41.472519 (+ 62757us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:41.814897 (+342378us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:11:42.164170 (+349273us) log.cc:844] Serialized 14 byte log entry
0709 12:11:42.374714 (+210544us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:43.329155 (+954441us) spinlock_profiling.cc:243] Waited 5.52 ms on lock 0x7b2c00037868. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:11:43.342832 (+ 13677us) log.cc:844] Serialized 26 byte log entry
0709 12:11:43.343028 (+   196us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:43.343078 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:43.343159 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:43.366680 (+ 23521us) raft_consensus.cc:1685] finished
0709 12:11:43.366779 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:43.366936 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22613,"prepare.queue_time_us":360714,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":5207,"raft.queue_time_us":263037,"raft.run_cpu_time_us":312,"raft.run_wall_time_us":32925,"spinlock_wait_cycles":13532416,"thread_start_us":616064,"threads_started":4,"wal-append.queue_time_us":491816}
I20260709 12:11:43.372453 16671 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:43.445721 12526 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:43.398072 17267 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:43.405494 16919 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:43.425293 16760 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:43.450554 17312 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:43.451242 17312 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:43.452960 16919 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:43.453563 17312 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:43.410656 17227 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:43.455090 17227 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.398811 17287 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.458300 17314 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 12:11:43.459860 17287 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:43.398218 16962 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.064s
I20260709 12:11:43.462844 16962 raft_consensus.cc:3055] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:43.463228 16962 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:43.463940 17285 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:43.464074 16962 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.466104 16719 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:43.420513 17315 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
W20260709 12:11:43.468309 17328 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:11:43.468559 17134 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:43.468918 17326 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:43.469363 17326 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:43.469726 17326 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:43.473196 17222 raft_consensus.cc:3055] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:43.474818 16211 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:43.475455 16211 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:43.476727 16211 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.473882 17222 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:43.480217 17222 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:43.489017 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 146158us
I20260709 12:11:43.460054 17336 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:43.499090 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:43.503741 12600 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:11:43.504817 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:43.512609 16719 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:43.513440 16719 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.515489 17333 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:43.516256 16719 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:43.516126 17333 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:43.517073 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:43.518447 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:43.519140 12605 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:43.520069 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:43.520675 12597 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:43.522358 17272 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.028s
I20260709 12:11:43.523413 17333 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:43.525224 17172 consensus_meta.cc:287] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.607s	user 0.008s	sys 0.002s
I20260709 12:11:43.525899 17122 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.526856 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:43.527580 12604 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 11: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:43.528263 17122 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:43.516454 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:43.530702 12527 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:43.535737 12602 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:43.530870 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:43.528258 12671 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:43.531692 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:43.534716 12526 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:43.536898 12523 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 12: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:43.533381 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:43.539283 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:43.525650 17172 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:43.583432 17306 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:43.593351 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:43.606490 13339 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
W20260709 12:11:43.582456 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 334009us
W20260709 12:11:43.603731 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 51566.9us
I20260709 12:11:43.609237 17234 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.495774 17263 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:43.593739 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:43.490996 17062 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:43.589644 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:43.611649 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 408600us
I20260709 12:11:43.625581 12597 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:43.574394 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:43.598701 12526 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:43.611151 13305 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:11:43.620002 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 232454us
I20260709 12:11:43.612507 12523 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: [11.8-11.8]
W20260709 12:11:43.614403 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:43.623080 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:43.625444 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:43.626768 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:43.620779 12524 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 6. Current term is 7. Ops: []
I20260709 12:11:43.629442 12600 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:43.636430 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:43.636065 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:43.636166 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:43.640019 12527 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 11: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:43.641083 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:43.641716 12597 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:43.644554 12494 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:43.645376 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:43.646281 12525 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:11:43.648677 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:43.649714 12600 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:43.655519 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:43.656826 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:43.659252 16719 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:43.659786 16719 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.660821 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:43.663501 17172 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:43.663817 17287 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:43.664438 17287 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:43.664788 17230 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:43.665298 17230 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.666483 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:43.667092 13338 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:43.655856 16943 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:43.668049 17235 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:43.668519 17235 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:43.668859 17235 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:43.671223 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:43.670995 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:43.675576 17340 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:43.680701 17340 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:43.670436 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:43.676261 17227 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:43.643959 17122 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:43.706276 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:11:43.710520 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 96937.6us
I20260709 12:11:43.631490 17333 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.068s
I20260709 12:11:43.714278 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:43.673468 17267 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:43.717200 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:43.612555 17197 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:43.718175 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 45361.9us
I20260709 12:11:43.722023 12494 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:43.631968 17338 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.099s
I20260709 12:11:43.732546 17338 raft_consensus.cc:3055] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:43.732930 17338 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:43.686241 16719 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:43.614705 17337 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.097s
I20260709 12:11:43.713970 17252 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:43.730300 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:43.731482 12494 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:43.749228 12494 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:43.727427 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:43.744911 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:43.752334 12597 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:43.753866 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:43.756613 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:43.759778 12494 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:43.733758 17338 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:43.764441 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:43.770826 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:43.771947 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:43.740875 16671 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:43.717741 17284 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.044s
I20260709 12:11:43.717841 17122 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.793154 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:43.799464 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:43.800558 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:43.800920 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:43.801501 12597 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:43.794739 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:43.803830 12600 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:43.797413 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:43.755427 17340 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:43.766862 17267 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:11:43.810364 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 93935.4us
I20260709 12:11:43.767089 17062 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:43.764118 17330 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:43.765280 17168 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:43.774780 17287 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:43.774765 17337 consensus_queue.cc:1243] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 9.5 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 4
I20260709 12:11:43.767468 16719 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:43.735173 17306 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 12:11:43.801735 17321 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
W20260709 12:11:43.832408 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:43.842490 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:43.828284 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:43.812983 17318 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:43.777762 17344 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.055s
I20260709 12:11:43.769404 17227 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 3, Time since last communication: 0.057s
I20260709 12:11:43.767640 16760 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.820843 17134 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:11:43.846839 12600 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:43.821429 13338 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:11:43.817512 17168 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:43.861876 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:43.863417 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:43.871968 12524 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:43.872618 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:43.862524 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:43.861275 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:43.867787 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:43.835291 17338 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:43.868525 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:43.869905 12597 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:43.873323 12527 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:43.877151 12527 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 13. Ops: []
I20260709 12:11:43.843384 17287 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:43.830547 16719 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:43.879102 12527 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:43.825206 17314 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260709 12:11:43.861442 17252 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:43.884374 17356 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:43.885113 17122 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:43.880430 16719 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:43.886962 17348 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.109s	user 0.000s	sys 0.001s
I20260709 12:11:43.885197 17356 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:43.882093 17287 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.890929 17356 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:43.892930 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:43.893142 16760 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:43.894441 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:43.895298 12600 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 11: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:43.895892 17287 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:43.896226 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:43.883137 12524 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:43.824591 17296 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:11:43.867662 17230 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:43.873975 17168 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:43.902783 12524 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
W20260709 12:11:43.903584 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 88139.6us
I20260709 12:11:43.975684 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:43.975867 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:43.846460 17326 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:43.988698 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:43.987847 17230 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:43.991772 17230 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:43.882301 17263 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:43.993054 17346 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 12:11:43.901155 17353 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:44.018724 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:44.023895 13339 raft_consensus.cc:2368] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12.
I20260709 12:11:44.027868 17235 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:44.031672 17230 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:44.022636 12674 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
W20260709 12:11:44.046788 17351 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
W20260709 12:11:44.054603 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 78819.1us
I20260709 12:11:44.056852 17234 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:44.057643 17126 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:44.058178 17126 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:44.058583 17126 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:44.061218 17197 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:44.017452 17240 consensus_meta.cc:287] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.865s	user 0.014s	sys 0.001s
I20260709 12:11:44.062443 17333 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:11:44.062980 17240 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:44.064819 17349 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.200s	user 0.001s	sys 0.000s
I20260709 12:11:44.071826 12530 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:44.065310 17240 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:44.063637 17333 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:11:44.068205 12526 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:44.020496 17353 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:11:44.086272 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 326671us
I20260709 12:11:44.087262 13331 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 12. Ops: []
I20260709 12:11:44.087478 12494 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:44.093894 12494 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:44.079133 17354 raft_consensus.cc:3055] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:44.095873 12494 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:44.102672 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:44.104658 12494 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:44.105988 17252 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:44.107388 17197 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:44.108366 17122 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:44.108860 17122 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:44.109293 17122 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:44.109185 12494 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:44.110616 17353 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.111120 12494 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:44.112007 17353 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:44.112319 12494 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:44.113139 17267 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:44.113291 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:44.113762 17267 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:44.114894 17267 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.116850 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:44.118320 16671 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:44.118578 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:44.119218 16671 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:44.119675 16671 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:44.120069 16671 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:44.121981 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2294) took 3075 ms. Trace:
I20260709 12:11:44.122520 12603 rpcz_store.cc:276] 0709 12:11:41.046707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.055900 (+  9193us) service_pool.cc:224] Handling call
0709 12:11:41.056217 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:41.411966 (+355749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:44.121704 (+2709738us) spinlock_profiling.cc:243] Waited 347 ms on lock 0x7b54003110d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:44.121900 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":797919488}
I20260709 12:11:44.123899 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:44.124492 12603 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:44.135802 17253 log.cc:927] Time spent T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 2.333s	user 0.015s	sys 0.006s
I20260709 12:11:44.137436 17270 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:44.138067 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2207) took 3108 ms. Trace:
I20260709 12:11:44.138525 12672 rpcz_store.cc:276] 0709 12:11:41.029839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.046649 (+ 16810us) service_pool.cc:224] Handling call
0709 12:11:41.046946 (+   297us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:41.050283 (+  3337us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:41.050317 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:41.050880 (+   563us) log.cc:844] Serialized 26 byte log entry
0709 12:11:41.398548 (+347668us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:41.662594 (+264046us) log.cc:844] Serialized 14 byte log entry
0709 12:11:42.115734 (+453140us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:42.115833 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:44.106827 (+1990994us) spinlock_profiling.cc:243] Waited 5.28 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:44.137740 (+ 30913us) raft_consensus.cc:1685] finished
0709 12:11:44.137844 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:44.138001 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16160,"prepare.queue_time_us":687650,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":388246,"raft.run_cpu_time_us":434,"raft.run_wall_time_us":55492,"spinlock_wait_cycles":12142336,"thread_start_us":1015491,"threads_started":3,"wal-append.queue_time_us":587738}
I20260709 12:11:44.140530 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:44.148027 17308 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:44.152654 13304 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:44.149468 17126 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.162082 17320 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:44.163379 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.163740 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.164371 12674 raft_consensus.cc:2410] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 8.
W20260709 12:11:44.139357 17350 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.266s	user 0.000s	sys 0.001s
I20260709 12:11:44.164616 12671 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:44.137977 17270 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:44.167218 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:44.160480 17126 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:44.099661 17240 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:44.190243 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.098708 17354 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:44.187819 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.188344 12606 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:44.103601 17306 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:44.186751 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.182174 17320 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:44.135840 17134 raft_consensus.cc:3055] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:44.149551 16719 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.196429 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.197448 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.211930 12679 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
W20260709 12:11:44.210415 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 89137.8us
I20260709 12:11:44.195451 12672 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
W20260709 12:11:44.212777 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:44.202347 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 145219us
I20260709 12:11:44.182675 17252 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:44.204258 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:44.218839 12603 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:11:44.190922 12676 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:44.205814 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:44.220924 12674 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:44.196219 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2322) took 2514 ms. Trace:
I20260709 12:11:44.215711 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:44.221639 12606 rpcz_store.cc:276] 0709 12:11:41.682908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.706843 (+ 23935us) service_pool.cc:224] Handling call
0709 12:11:44.187953 (+2481110us) spinlock_profiling.cc:243] Waited 20.3 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:11:44.188144 (+   191us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:44.195920 (+  7776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:44.196147 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":46739200}
I20260709 12:11:44.221199 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:44.225479 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.226395 12603 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 8.
I20260709 12:11:44.230073 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:44.230686 12671 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:44.232290 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.208974 17134 raft_consensus.cc:740] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:44.236863 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:44.237493 12676 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:44.241365 12678 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:11:44.223928 17252 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:44.245229 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.245664 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:44.228806 17056 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:44.222484 17306 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:44.247874 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:44.248544 17354 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.250662 16719 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:44.254323 17365 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:44.255028 17235 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:44.237203 17361 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:44.256590 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.235251 17270 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.258960 17168 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.239773 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:44.262976 17261 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.111s	user 0.000s	sys 0.000s
I20260709 12:11:44.292274 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:44.292853 12527 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:44.266396 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:44.267858 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:44.267241 12671 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:44.270780 12606 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:44.273617 12674 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:44.273669 12676 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:11:44.248116 17252 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:44.274554 17320 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:44.278766 17306 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:44.279870 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:44.269384 17122 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.306171 12678 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 11: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:11:44.263109 17293 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:11:44.308007 12530 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:44.287237 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:44.288853 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.290215 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2256) took 3040 ms. Trace:
I20260709 12:11:44.290607 17354 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:44.294454 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:44.308763 12603 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:44.309980 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:44.296402 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.299840 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:44.302690 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:44.306012 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:44.286264 17056 raft_consensus.cc:697] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:44.308830 12672 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:44.308799 12529 rpcz_store.cc:276] 0709 12:11:41.249955 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:41.415243 (+165288us) service_pool.cc:224] Handling call
0709 12:11:41.444787 (+ 29544us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:43.159879 (+1715092us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:44.289875 (+1129996us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b54003f7354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:44.290143 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3942071552}
W20260709 12:11:44.318408 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:44.333421 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:44.333837 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:44.334160 12531 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:44.335014 12605 raft_consensus.cc:2410] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 6.
I20260709 12:11:44.310621 17270 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:44.338793 12678 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:44.339334 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:44.357384 12603 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:11:44.340308 12671 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: [11.6-13.7]
I20260709 12:11:44.343056 17357 consensus_queue.cc:1059] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:11:44.343763 16719 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:44.353278 12494 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:44.287112 17365 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:44.294530 17361 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.347309 17298 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:44.371135 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:44.383559 12606 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:11:44.375612 17197 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:44.385041 17197 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:44.386304 17168 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:44.386067 17197 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:44.387328 17056 consensus_queue.cc:237] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.377542 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:44.384238 12527 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:11:44.390059 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:44.390466 12531 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:44.373565 17365 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:44.307116 17320 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:44.429895 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:44.303210 17134 consensus_queue.cc:260] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:44.435953 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:44.441094 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:44.437000 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:44.373747 17372 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:44.445883 17359 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:44.453202 12597 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:44.457190 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:44.458827 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2605) took 1667 ms. Trace:
I20260709 12:11:44.484615 12673 rpcz_store.cc:276] 0709 12:11:42.791251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:42.991288 (+200037us) service_pool.cc:224] Handling call
0709 12:11:42.991669 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:43.012382 (+ 20713us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:43.012435 (+    53us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:43.012485 (+    50us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:43.012521 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:44.458437 (+1445916us) spinlock_profiling.cc:243] Waited 20.5 ms on lock 0x7b54003d4fd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:44.458507 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:44.458736 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":47058176}
W20260709 12:11:44.460431 12528 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2271) took 1726 ms (client timeout 2096 ms). Trace:
W20260709 12:11:44.486493 12528 rpcz_store.cc:269] 0709 12:11:42.738648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:42.739148 (+   500us) service_pool.cc:224] Handling call
0709 12:11:44.460144 (+1720996us) spinlock_profiling.cc:243] Waited 131 ms on lock 0x7b54003f7854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:44.460334 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":302116096}
I20260709 12:11:44.488016 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:44.464366 17122 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:44.490650 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:44.467975 16211 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:44.471026 17222 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:44.471952 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:44.474722 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2265) took 2213 ms. Trace:
I20260709 12:11:44.402935 17333 raft_consensus.cc:3055] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:44.463996 16806 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:44.489375 12528 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 13: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:44.490453 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:44.400123 17252 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:44.491590 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:44.393304 16671 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.484146 12680 raft_consensus.cc:2376] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14. Re-sending same reply.
W20260709 12:11:44.505529 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 152327us
I20260709 12:11:44.497788 12522 rpcz_store.cc:276] 0709 12:11:42.261779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:42.295866 (+ 34087us) service_pool.cc:224] Handling call
0709 12:11:42.316708 (+ 20842us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:42.653208 (+336500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:42.653258 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:42.653322 (+    64us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:11:42.653371 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:44.474384 (+1821013us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b54003ca5d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:44.474475 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:44.474643 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":422110592}
I20260709 12:11:44.481813 17240 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:44.511850 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:44.515066 12496 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:44.529232 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:44.531127 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:44.531443 12600 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:44.527407 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:44.508864 16806 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:44.505079 17333 raft_consensus.cc:740] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:44.519290 17134 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:44.538800 17122 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:44.547861 16875 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:44.547514 16806 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.548393 16875 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:44.542570 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:44.550587 17252 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:44.551043 12605 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:44.552606 16671 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:44.492214 17359 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:44.555083 17359 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:44.555686 17359 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:44.556268 17359 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:44.548758 16806 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:44.539340 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:44.560660 12528 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 7.
I20260709 12:11:44.520499 17126 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:44.567723 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:44.534621 17372 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:44.544667 17241 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:44.543459 12496 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:44.581928 12496 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:44.527901 16935 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:44.561811 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:44.583673 12528 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:44.538519 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:44.585311 12522 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 14: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:11:44.544466 17338 consensus_meta.cc:287] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.664s	user 0.009s	sys 0.006s
I20260709 12:11:44.576433 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:44.547645 17374 log.cc:927] Time spent T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:11:44.560038 17240 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:44.612394 17240 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:44.601936 16935 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:44.598856 16671 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:44.568567 17361 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:44.571368 17252 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:44.575982 17357 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:44.547550 17122 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.616703 16671 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:44.599552 17241 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.575402 17372 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:44.631520 17122 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:44.593374 17267 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:44.632421 17267 raft_consensus.cc:697] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:44.613845 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:44.633828 17267 consensus_queue.cc:237] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.634603 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:44.636025 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:44.637140 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:44.631580 17372 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:44.630725 16935 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:44.632719 17241 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:44.640942 17320 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:44.641623 12528 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:11:44.588308 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:44.623013 16875 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
W20260709 12:11:44.642959 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54809.2us
I20260709 12:11:44.643702 12496 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:44.573175 17126 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:44.628995 17357 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.646512 17357 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:44.630693 16671 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:44.612818 17240 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:44.648564 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:44.624285 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:44.649034 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 72692us
I20260709 12:11:44.649453 12530 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:44.649791 12494 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:44.646651 17126 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:44.620011 12605 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:44.629235 17252 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.610265 17359 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:44.644025 17168 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:44.666932 12572 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:44.649950 12496 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:44.643671 12528 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:11:44.670050 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:44.676196 12528 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:11:44.676317 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:44.677259 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:44.675818 12496 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:44.681028 12530 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:44.611256 17333 consensus_queue.cc:260] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.682770 12494 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:44.683215 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:44.684212 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:44.650981 17240 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:44.687162 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:44.688040 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:44.651209 17122 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:44.666960 17361 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:44.680421 17359 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:44.686932 17357 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:11:44.678315 17241 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:44.698616 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:44.699600 12605 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
W20260709 12:11:44.701273 17377 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:11:44.691145 17354 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:44.689989 12496 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:44.703313 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:44.728130 12572 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:44.705679 17337 raft_consensus.cc:3055] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:44.707095 12679 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
W20260709 12:11:44.709352 17375 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:11:44.679078 17376 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:44.714564 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:44.779558 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:44.780800 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:44.781595 12605 raft_consensus.cc:2410] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 14 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:11:44.703677 12604 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:44.782755 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:44.783447 12605 raft_consensus.cc:2376] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8. Re-sending same reply.
I20260709 12:11:44.784974 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:44.785737 12605 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:44.786808 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:44.787489 12605 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:44.680545 17126 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:11:44.689577 17287 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:44.719414 17252 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:44.685794 17343 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:44.695693 17361 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:44.800046 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:44.800340 17361 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:44.800760 12601 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:44.802932 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:44.803783 12598 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:44.804641 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:44.805419 12598 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 11: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:44.691222 17168 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:44.806691 17168 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:44.807160 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2230) took 1655 ms. Trace:
I20260709 12:11:44.807950 17168 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.724999 17122 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260709 12:11:44.750758 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:44.810391 17122 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:44.810842 17122 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:44.726306 17363 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:44.756743 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.815877 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.815963 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:44.816538 12676 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:11:44.758599 17337 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:44.817559 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.757095 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:11:44.818718 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 61746.3us
I20260709 12:11:44.818492 17337 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.819521 17337 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:44.820691 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.820850 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:44.776244 17354 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:44.741009 17241 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:44.822638 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:44.823156 17241 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:44.824100 17322 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:11:44.824661 17322 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:44.824201 17241 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.825057 17322 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:11:44.827534 12572 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:44.816233 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:44.832547 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:44.833431 12602 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 13. Current term is 14. Ops: [11.8-13.9]
I20260709 12:11:44.834589 12676 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 7.
I20260709 12:11:44.835733 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 2489) took 1235 ms. Trace:
I20260709 12:11:44.835981 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:44.836138 12600 rpcz_store.cc:276] 0709 12:11:43.599959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:43.895938 (+295979us) service_pool.cc:224] Handling call
0709 12:11:44.835546 (+939608us) spinlock_profiling.cc:243] Waited 634 ms on lock 0x7b54001617d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:44.835670 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1459020672}
I20260709 12:11:44.836901 12676 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:44.838517 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:44.839964 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:44.840917 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:44.827759 12528 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:44.841750 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:44.797459 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:44.750196 17222 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:44.807600 12675 rpcz_store.cc:276] 0709 12:11:43.152251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:43.295375 (+143124us) service_pool.cc:224] Handling call
0709 12:11:43.295732 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:43.440945 (+145213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:44.806902 (+1365957us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b5400013bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:44.807082 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":234203392}
I20260709 12:11:44.843664 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.828246 17354 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:11:44.829526 17126 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:44.829862 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:44.830416 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:44.851414 12598 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:44.830308 17385 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:44.831051 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:44.831755 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:44.828282 13331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 627) took 1188 ms. Trace:
I20260709 12:11:44.844736 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.852583 13331 rpcz_store.cc:276] 0709 12:11:43.640290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:43.640458 (+   168us) service_pool.cc:224] Handling call
0709 12:11:43.647625 (+  7167us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:44.087910 (+440285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:44.828040 (+740130us) spinlock_profiling.cc:243] Waited 439 ms on lock 0x7b540055ee54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:44.828221 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1010614656}
I20260709 12:11:44.854405 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.821974 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.855489 13334 raft_consensus.cc:2368] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12.
I20260709 12:11:44.845284 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:44.828428 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:44.857105 12530 raft_consensus.cc:2368] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12.
I20260709 12:11:44.833917 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:44.837976 12600 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:44.848177 12643 leader_election.cc:400] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12.
W20260709 12:11:44.849146 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:44.844524 13304 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:44.861181 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:44.863169 12572 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:44.863539 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:44.864749 13335 raft_consensus.cc:2368] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12.
I20260709 12:11:44.864898 12675 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:44.866911 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:44.813779 17134 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:11:44.874017 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:44.874730 12646 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:44.879527 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:44.886807 12600 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260709 12:11:44.887297 12601 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:44.745821 17372 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.892263 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:44.893741 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 211078us
I20260709 12:11:44.896672 12605 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
W20260709 12:11:44.898413 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:44.869992 17289 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:44.868996 17359 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:44.901165 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 479) took 1258 ms. Trace:
I20260709 12:11:45.093256 12526 rpcz_store.cc:276] 0709 12:11:43.642334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:43.889533 (+247199us) service_pool.cc:224] Handling call
0709 12:11:43.889823 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:44.080152 (+190329us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:44.900899 (+820747us) spinlock_profiling.cc:243] Waited 176 ms on lock 0x7b5400106dd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:44.901105 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":405854976}
I20260709 12:11:44.904006 12494 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:45.090137 13305 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:11:45.094925 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 191026us
W20260709 12:11:45.097167 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 93266.2us
I20260709 12:11:44.910584 12675 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:11:45.098783 13305 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:11:44.928906 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 54280.7us
I20260709 12:11:45.099165 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:45.099849 12675 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:45.100672 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:45.101712 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:44.938032 12643 leader_election.cc:400] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12.
I20260709 12:11:44.928596 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:45.102396 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:44.940383 12676 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:11:44.941079 12572 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:44.900791 17058 raft_consensus.cc:3055] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:44.947748 12680 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:44.950471 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:44.932188 17298 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:44.952795 12528 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:44.958410 17372 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:44.963423 17240 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:44.972075 16671 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:44.975386 17339 log.cc:927] Time spent T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.369s	user 0.000s	sys 0.004s
I20260709 12:11:44.969501 17322 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:45.058785 17358 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.012s	user 0.000s	sys 0.004s
I20260709 12:11:45.026103 17357 raft_consensus.cc:2764] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:11:45.026381 17289 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:45.097352 12494 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:44.927718 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:45.098896 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 94550.9us
I20260709 12:11:45.102310 12643 leader_election.cc:403] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:11:44.888593 17222 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:44.889894 17363 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:45.140240 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:45.140552 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2288) took 1410 ms. Trace:
I20260709 12:11:45.140136 17222 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:45.103068 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:45.140928 12522 rpcz_store.cc:276] 0709 12:11:43.730649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:44.638033 (+907384us) service_pool.cc:224] Handling call
0709 12:11:44.640718 (+  2685us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:44.682153 (+ 41435us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:11:44.694540 (+ 12387us) log.cc:844] Serialized 14 byte log entry
0709 12:11:44.947412 (+252872us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:44.947480 (+    68us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:44.947539 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:45.140277 (+192738us) spinlock_profiling.cc:243] Waited 41.2 ms on lock 0x7b5400407c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:45.140336 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:45.140489 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":12682,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":1360,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":745,"spinlock_wait_cycles":94754688,"thread_start_us":252476,"threads_started":1}
I20260709 12:11:45.013006 17287 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:45.101954 12646 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:45.107216 17058 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:45.160773 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:45.120599 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 179628us
I20260709 12:11:45.126113 16935 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:45.105620 17320 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:11:45.111960 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 95727.8us
I20260709 12:11:45.112818 16943 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:11:45.146287 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 208405us
W20260709 12:11:45.171404 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:45.114394 17252 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:45.158176 12675 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:45.159664 13339 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: []
I20260709 12:11:45.142880 17222 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:45.162689 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 65410.2us
I20260709 12:11:45.163717 12572 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:45.039537 17361 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:45.166390 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 64556us
I20260709 12:11:45.134284 17309 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:45.143658 17289 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:45.183437 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:45.128656 17376 raft_consensus.cc:697] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:11:45.184724 17390 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:11:45.184963 17244 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 12:11:45.160931 16806 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12.
W20260709 12:11:45.188524 17389 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.094s	user 0.000s	sys 0.001s
I20260709 12:11:45.189522 17058 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:45.189343 17391 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.070s	user 0.000s	sys 0.001s
I20260709 12:11:45.189510 17134 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:45.212319 17058 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:45.212472 17134 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:45.213091 17353 raft_consensus.cc:3055] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:45.213588 17353 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:45.215080 17134 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:45.166675 17288 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:45.176069 16943 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:45.190917 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:45.219151 12526 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 11: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:45.194150 12494 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:45.172828 12496 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:45.175894 17300 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:45.189369 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:45.175789 17252 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:45.185057 17126 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:11:45.252378 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 58338.5us
I20260709 12:11:45.253278 12494 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:45.191790 17320 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:11:45.254292 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 114285us
I20260709 12:11:45.258668 13302 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:45.254518 12494 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:11:45.255019 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 82305.3us
I20260709 12:11:45.204255 17272 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:11:45.262502 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 130941us
I20260709 12:11:45.205101 17309 raft_consensus.cc:697] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:11:45.288416 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 99154.5us
I20260709 12:11:45.289776 12643 leader_election.cc:400] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12.
I20260709 12:11:45.291661 16211 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:45.293135 12643 leader_election.cc:403] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:11:45.248215 16943 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:45.254395 17300 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:45.205703 17397 consensus_queue.cc:1059] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.066s
I20260709 12:11:45.303978 12643 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:45.252298 17252 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:45.259512 17168 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
W20260709 12:11:45.324646 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 152733us
I20260709 12:11:45.267863 17272 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:45.256992 17226 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
W20260709 12:11:45.315395 17398 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
I20260709 12:11:45.344862 17326 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:45.346343 17320 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:45.346974 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:45.347901 17322 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:45.338269 17267 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:11:45.359371 17402 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:11:45.294718 16806 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:45.341598 17376 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:45.306982 17134 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12.
W20260709 12:11:45.379012 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:45.381980 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:45.311097 17300 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:45.371435 17322 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:45.370147 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 178092us
I20260709 12:11:45.333983 17271 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:45.388211 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:45.364655 17272 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:45.301054 17309 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:45.412487 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:45.412727 12601 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:45.413265 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:45.413833 12597 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: [8.4-12.6]
I20260709 12:11:45.411165 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:45.381986 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 177221us
I20260709 12:11:45.416532 12597 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:11:45.417047 12597 pending_rounds.cc:85] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 4
I20260709 12:11:45.417367 12597 pending_rounds.cc:107] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:11:45.417694 12597 raft_consensus.cc:2889] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:45.392900 16806 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:45.419219 16806 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:45.420277 16806 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:45.421474 16806 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:45.301986 17287 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:45.414983 12646 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:45.425778 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:45.427537 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 159299us
I20260709 12:11:45.423878 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:45.429661 12597 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:45.390374 17326 raft_consensus.cc:2764] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:11:45.431880 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:45.384097 17267 raft_consensus.cc:697] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:45.399290 17134 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:45.384346 17395 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:11:45.437343 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 182887us
I20260709 12:11:45.440425 12529 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:45.443557 12526 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: []
W20260709 12:11:45.431017 17122 consensus_meta.cc:287] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.620s	user 0.014s	sys 0.000s
I20260709 12:11:45.450330 12494 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:45.425733 17240 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:45.467341 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:45.470178 17315 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.470889 17315 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:45.475701 17400 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.291s	user 0.002s	sys 0.000s
W20260709 12:11:45.477617 17387 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.289s	user 0.001s	sys 0.000s
W20260709 12:11:45.485567 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 44308.3us
W20260709 12:11:45.487439 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 159446us
I20260709 12:11:45.486229 17413 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:11:45.490784 17399 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 12:11:45.493036 17168 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:45.467612 17420 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:45.494352 17168 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:45.495085 17420 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:45.495740 17168 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 12.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:45.499476 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 240897us
I20260709 12:11:45.444854 17361 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:45.434983 17322 consensus_queue.cc:1059] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.065s
W20260709 12:11:45.512347 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 78957.1us
I20260709 12:11:45.483850 17271 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:45.505085 17326 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:45.489220 17383 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:45.535964 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:45.510475 17284 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.477s	user 0.019s	sys 0.011s
I20260709 12:11:45.501720 17241 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:45.483731 17122 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:45.573022 17284 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.543s	user 0.020s	sys 0.011s
I20260709 12:11:45.506472 17414 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:11:45.595912 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 181026us
I20260709 12:11:45.573191 17413 raft_consensus.cc:3055] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:45.560106 17326 raft_consensus.cc:697] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:45.573684 17361 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.573541 17416 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:11:45.615468 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 148214us
W20260709 12:11:45.630268 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 217059us
I20260709 12:11:45.600975 12646 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:45.603634 17413 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:45.596868 17406 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:11:45.646857 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:45.642750 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:45.648028 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:45.647090 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:45.602552 17414 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:45.661614 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:45.649201 12599 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 11: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:45.599713 17395 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:45.672341 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:45.675314 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:45.677042 12643 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:45.680182 17420 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:45.681208 12646 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:45.648275 17289 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:45.683480 12646 leader_election.cc:400] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12.
I20260709 12:11:45.684356 12646 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:45.575487 17383 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:45.686223 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:45.671227 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:45.689805 12597 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:45.689890 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:45.690901 12598 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 11
W20260709 12:11:45.688421 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:45.694785 12599 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:11:45.650168 17271 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:11:45.674662 17423 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
W20260709 12:11:45.703454 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 140044us
I20260709 12:11:45.665634 17414 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:45.719883 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:45.720746 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:45.698051 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:45.724217 12527 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 8. Current term is 9. Ops: []
I20260709 12:11:45.724668 12528 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:45.597667 17267 consensus_queue.cc:237] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:45.676388 17406 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.641538 17424 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.086s
I20260709 12:11:45.642144 17361 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:45.689234 17315 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:45.702930 17226 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:11:45.744732 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:45.746802 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:45.742838 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:45.747536 12523 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:11:45.750381 12599 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:11:45.753196 17331 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:45.753464 17313 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 12.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.062s
I20260709 12:11:45.754508 17313 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:11:45.754817 17331 raft_consensus.cc:3055] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:45.755218 17331 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:45.755403 17432 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:45.756296 17411 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:11:45.756139 17331 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:45.757351 17331 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:45.761072 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 64679.8us
I20260709 12:11:45.762421 12494 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:45.745947 17395 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:45.777611 17320 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:45.807937 12525 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:11:45.800979 12597 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
W20260709 12:11:45.812816 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50499.6us
I20260709 12:11:45.716114 17297 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.075s
I20260709 12:11:45.813716 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:45.756155 17426 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:45.746908 17226 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:45.821935 12571 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:45.760356 17429 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:11:45.818229 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 302796us
I20260709 12:11:45.809172 17333 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:45.842136 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 122317us
I20260709 12:11:45.836196 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:45.842636 12494 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:45.843146 12671 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:45.843936 12494 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:45.845745 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:45.822835 17284 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.851811 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:45.824028 17226 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:45.740939 17326 consensus_queue.cc:237] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:45.821566 17416 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:45.748839 17406 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:45.844554 12572 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:45.841434 17372 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:45.876322 12572 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:45.867157 17436 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:45.843484 17271 raft_consensus.cc:697] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:45.849350 17420 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:45.884146 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:45.894577 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:45.900419 12572 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:45.901495 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 67996.8us
W20260709 12:11:45.865757 17337 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.046s	user 0.013s	sys 0.001s
I20260709 12:11:45.825105 17432 consensus_queue.cc:1059] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 12:11:45.907040 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:45.816641 17320 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:45.842368 17429 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:45.899766 17431 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260709 12:11:45.862926 17284 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:45.938915 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:45.953428 12674 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
W20260709 12:11:45.954813 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48009.6us
I20260709 12:11:45.955395 12598 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:45.894639 17361 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:45.963501 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 219203us
I20260709 12:11:45.879855 17122 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:45.946324 17443 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:45.952169 17431 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:45.946511 17432 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:45.959578 17397 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:45.914422 12606 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
W20260709 12:11:45.958734 17385 consensus_meta.cc:287] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.098s	user 0.011s	sys 0.005s
I20260709 12:11:45.907594 17372 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
W20260709 12:11:45.948542 16806 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.527s	user 0.010s	sys 0.000s
I20260709 12:11:45.940832 17437 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:45.908596 17297 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
W20260709 12:11:45.990856 17433 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 12:11:45.991094 17445 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:45.991524 17284 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:45.991235 17443 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:45.989073 17432 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:45.983454 17449 raft_consensus.cc:3055] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:45.938400 17435 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 4, Time since last communication: 0.055s
I20260709 12:11:45.998142 17385 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:11:46.009189 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 162162us
W20260709 12:11:45.997228 17252 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.663s	user 0.011s	sys 0.001s
I20260709 12:11:46.012883 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:46.034237 12598 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:45.989146 17431 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:45.964881 17289 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:46.016000 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 63447.8us
W20260709 12:11:46.018175 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 62563.7us
I20260709 12:11:45.999220 17383 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:45.938027 17271 consensus_queue.cc:237] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.8, Last appended by leader: 8, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:46.021275 16943 consensus_meta.cc:287] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.693s	user 0.012s	sys 0.001s
I20260709 12:11:45.998660 17134 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260709 12:11:46.001595 17427 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:46.060304 12569 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:46.019112 17449 raft_consensus.cc:740] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:45.995823 17397 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.056070 17284 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:46.057919 17450 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.092s
I20260709 12:11:46.070585 17424 consensus_queue.cc:1243] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 12.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:11:45.993177 17416 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.075737 17289 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:46.086345 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 92061.8us
I20260709 12:11:46.055759 17252 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:46.098949 12604 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2335) took 1527 ms (client timeout 1695 ms). Trace:
W20260709 12:11:46.104555 12604 rpcz_store.cc:269] 0709 12:11:44.571215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:44.679936 (+108721us) service_pool.cc:224] Handling call
0709 12:11:46.098615 (+1418679us) spinlock_profiling.cc:243] Waited 5.4 ms on lock 0x7b5400375d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:46.098817 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":12415232}
W20260709 12:11:46.104918 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 44739.2us
I20260709 12:11:46.077311 12598 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:46.095510 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:46.109711 12599 raft_consensus.cc:2410] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 6.
I20260709 12:11:46.111418 12597 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:46.097872 17443 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:46.097090 17446 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.138s	user 0.000s	sys 0.001s
I20260709 12:11:46.096470 17425 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:46.110054 17289 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:46.128590 17449 consensus_queue.cc:260] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.122957 17385 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.012s
I20260709 12:11:46.134465 17414 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:46.135439 17289 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:46.136722 17397 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:46.096331 17436 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 10.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
W20260709 12:11:46.138931 17058 consensus_meta.cc:287] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.926s	user 0.012s	sys 0.003s
W20260709 12:11:46.139350 17300 consensus_meta.cc:287] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.729s	user 0.009s	sys 0.003s
I20260709 12:11:46.139829 17300 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:46.146416 17422 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.391s	user 0.001s	sys 0.000s
I20260709 12:11:46.116523 17452 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:11:46.148448 17441 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 12:11:46.150921 17226 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.136722 17414 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:46.158020 17444 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:11:46.154884 12494 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:46.160379 12494 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:46.127044 17372 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:46.163385 12599 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:11:46.074766 16943 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:46.154711 17397 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.003s
I20260709 12:11:46.117199 17451 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:46.169018 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:46.169862 17453 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:11:46.169965 12677 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 13. Ops: []
I20260709 12:11:46.170425 17337 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:46.180365 17462 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:11:46.180402 17337 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:46.181731 17337 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.183033 17337 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:46.188043 12676 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:11:46.146447 17421 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.289s	user 0.001s	sys 0.001s
I20260709 12:11:46.173655 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:46.177085 17442 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:46.177620 17122 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:11:46.173214 17168 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:46.209082 17168 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:46.210188 17168 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.162045 17372 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:46.172889 17448 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
I20260709 12:11:46.180491 17226 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:46.130860 17424 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:46.222431 12600 raft_consensus.cc:2410] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 11 because replica has last-logged OpId of term: 10 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:11:46.223884 17372 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:46.226821 16962 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:46.229525 16943 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:46.141978 17300 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:46.193099 17272 consensus_meta.cc:287] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.764s	user 0.010s	sys 0.001s
I20260709 12:11:46.206735 17442 raft_consensus.cc:697] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:46.220180 17413 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:46.206724 17438 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:46.214368 17429 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:46.091360 17432 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.249502 17252 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:46.259958 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:46.257745 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 155144us
I20260709 12:11:46.267799 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:46.265331 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 208984us
I20260709 12:11:46.257340 17413 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:46.240226 17272 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:46.264343 17432 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:46.295324 17309 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.002s	sys 0.009s
I20260709 12:11:46.255400 17438 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.311388 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:46.290843 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:46.308468 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:46.308279 17309 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.002s	sys 0.010s
I20260709 12:11:46.321507 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:11:46.311880 17459 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.166s	user 0.000s	sys 0.001s
I20260709 12:11:46.325342 12675 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:46.326473 12674 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:46.326392 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:46.329483 12676 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:46.360160 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:46.360433 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:46.361390 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:46.366816 12646 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:46.369776 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:46.387036 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:46.387988 12524 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:46.401405 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:46.402415 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 345424us
I20260709 12:11:46.403318 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:46.407701 12528 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: []
W20260709 12:11:46.409127 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:46.409668 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:46.403478 12643 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:46.328267 16943 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:46.357898 17462 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:46.293196 17451 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260709 12:11:46.334658 17429 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:46.359594 17372 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:46.416707 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 50030.3us
I20260709 12:11:46.365656 17397 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:46.336737 17252 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:46.421658 12675 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 10. Current term is 11. Ops: [10.9-10.9]
I20260709 12:11:46.425071 12676 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:11:46.427354 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:46.358065 17465 raft_consensus.cc:2764] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:11:46.437566 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:46.441907 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:46.442200 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:46.437701 17471 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:46.450780 17354 raft_consensus.cc:3055] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:11:46.450866 17471 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:46.451301 17354 raft_consensus.cc:740] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:46.416527 17359 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:46.451491 17471 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:46.451499 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:46.451614 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:46.439311 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:46.445910 12676 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:11:46.462309 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:46.467499 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:46.359386 17454 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.088s
I20260709 12:11:46.416311 17461 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:46.486588 13302 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:46.441684 17326 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:46.297251 17442 consensus_queue.cc:237] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.452127 17359 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.467105 12572 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:46.434548 16943 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.443653 17322 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:11:46.496342 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:46.496675 17461 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.511304 17354 consensus_queue.cc:260] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.511353 17409 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:46.433053 17463 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:46.513921 17359 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:46.440696 17462 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:46.471465 17300 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:46.496284 17434 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:11:46.458361 17397 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:46.447329 17372 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.543004 17467 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:46.459190 17252 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:46.587678 12676 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: [11.6-13.7]
I20260709 12:11:46.452831 17429 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:46.587723 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 98896.2us
W20260709 12:11:46.587057 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 120014us
I20260709 12:11:46.606702 17409 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:46.494462 17326 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:46.522558 17426 consensus_meta.cc:287] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.677s	user 0.010s	sys 0.004s
I20260709 12:11:46.634280 17466 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:46.634438 17426 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:46.507619 17122 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:46.635385 17429 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:46.607798 12571 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.603s (SENT) [suppressed 11 similar messages]
I20260709 12:11:46.635634 17122 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:46.613550 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:46.608553 17475 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
W20260709 12:11:46.611778 17473 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.234s	user 0.001s	sys 0.001s
I20260709 12:11:46.638005 17122 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:46.638866 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:46.640615 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:46.631913 12676 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:11:46.633682 17326 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:46.648710 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:46.658751 17472 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 12:11:46.661021 17326 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:11:46.663805 17445 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.642s	user 0.010s	sys 0.004s
I20260709 12:11:46.664372 17445 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:46.665169 17466 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:46.665966 17466 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.670557 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:46.671414 17252 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:46.672200 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:46.672590 17461 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:46.673581 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 7 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:46.675148 17372 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:46.676167 12527 raft_consensus.cc:2393] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:46.676807 17466 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:46.677264 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:46.678537 12522 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:46.680159 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:46.680315 12572 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:46.681042 12601 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:11:46.681531 12601 pending_rounds.cc:85] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 6
I20260709 12:11:46.623124 17354 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:46.681761 17466 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:46.682521 17466 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:46.683241 17466 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:46.686034 17397 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:46.666812 17438 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:46.687371 17461 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:46.681797 12601 pending_rounds.cc:107] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 10.7
I20260709 12:11:46.687897 17462 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:46.688086 12601 raft_consensus.cc:2889] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:46.688165 17461 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:46.688925 17343 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:46.689783 17343 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.691661 17461 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:46.692837 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:46.693567 16943 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:46.614319 17463 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.699124 17326 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:11:46.697366 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:46.707918 12522 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:46.699489 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:46.708861 12525 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:46.617888 17272 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:46.718415 12524 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:46.719525 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:46.720299 12524 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:11:46.699898 17463 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:46.724653 12522 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:11:46.727694 12524 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:46.730700 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:46.713016 17477 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.191s	user 0.001s	sys 0.000s
W20260709 12:11:46.766341 17469 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.141s	user 0.005s	sys 0.000s
I20260709 12:11:46.757076 16962 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:46.767131 16962 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:46.767701 16962 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:46.776202 12674 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:46.708351 17397 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:46.778533 17397 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.755384 17461 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260709 12:11:46.723242 17409 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:46.803088 17409 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:46.756073 17462 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:46.807969 17462 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:46.810046 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:46.814301 17397 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:46.816206 17403 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.447s	user 0.006s	sys 0.002s
I20260709 12:11:46.820874 13339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 492) took 1959 ms. Trace:
I20260709 12:11:46.821466 13339 rpcz_store.cc:276] 0709 12:11:44.861357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:44.861568 (+   211us) service_pool.cc:224] Handling call
0709 12:11:44.862004 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:45.177256 (+315252us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:46.820510 (+1643254us) spinlock_profiling.cc:243] Waited 297 ms on lock 0x7b54006b0154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:46.820784 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":684009856}
W20260709 12:11:46.830668 17337 consensus_meta.cc:287] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.647s	user 0.008s	sys 0.005s
W20260709 12:11:46.833334 17431 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.770s	user 0.005s	sys 0.010s
I20260709 12:11:46.834195 17431 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:46.862711 17372 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:46.863600 17372 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:46.864571 17288 raft_consensus.cc:3055] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:46.865151 17288 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:46.866003 17288 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.867233 17288 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:46.875916 17353 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.980262 17353 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:46.879179 17407 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:46.983482 17407 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:46.984534 17407 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:46.985730 17407 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:46.912186 16211 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:46.986526 16211 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.959892 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:46.988768 16211 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:46.989421 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:46.990268 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:46.962433 17431 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:46.927325 17462 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:46.927989 12605 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
W20260709 12:11:46.942394 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 311985us
W20260709 12:11:46.943452 17427 consensus_meta.cc:287] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.850s	user 0.012s	sys 0.001s
I20260709 12:11:46.993003 16671 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:46.993285 17427 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:46.993714 17336 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:46.942412 17372 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:46.948683 17354 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:11:46.994210 17336 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:46.994460 12526 raft_consensus.cc:2410] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 11 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 7 index: 5.
I20260709 12:11:46.995157 17336 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.996450 17336 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:46.947799 12604 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 8: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:46.999275 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:46.999822 12604 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:47.000855 17431 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:46.950624 17471 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.949508 17449 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:47.002977 12604 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:11:47.003896 12604 pending_rounds.cc:85] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 8
I20260709 12:11:47.004175 12604 pending_rounds.cc:107] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:11:47.004441 12604 raft_consensus.cc:2889] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:47.005290 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:46.957356 12525 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:11:47.007951 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:47.009744 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:46.925202 17466 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:46.997694 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:47.011477 12526 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:11:47.012586 17471 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:46.963933 17252 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:46.960356 12598 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.010s	sys 0.028s
W20260709 12:11:47.013403 12598 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.010s	sys 0.028s
I20260709 12:11:47.013325 17252 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.014317 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:46.964699 16719 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:45.337845 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:47.015178 16719 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.015710 17252 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:47.017330 16719 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:46.968354 16962 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:47.019125 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:47.019424 17454 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:46.970263 17397 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:46.975127 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:47.020735 16962 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:47.021348 17431 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:47.012297 17466 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:46.956827 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.022548 17397 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:47.024425 12602 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:47.024784 12602 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:47.008008 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:46.959391 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:47.019811 13335 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
W20260709 12:11:47.020993 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46036.1us
I20260709 12:11:47.019810 17454 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:47.027146 12599 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:47.027318 12571 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:47.027513 17454 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:46.991514 12606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2351) took 2340 ms (client timeout 2556 ms). Trace:
I20260709 12:11:47.028538 17454 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:47.028451 12606 rpcz_store.cc:269] 0709 12:11:44.657372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:44.815946 (+158574us) service_pool.cc:224] Handling call
0709 12:11:46.991334 (+2175388us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b5400230d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:46.991454 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2487119616}
I20260709 12:11:47.029838 12606 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:47.023121 12597 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:11:47.026618 12530 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 7 FOLLOWER]: Advancing to term 8
I20260709 12:11:46.945457 17406 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:47.030478 17425 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:47.025514 12602 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.031508 12602 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:47.031436 12571 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:47.033986 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:47.034403 17425 raft_consensus.cc:2764] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election decision vote started in defunct term 9: won
I20260709 12:11:47.035400 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:47.009282 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:47.038735 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:47.012348 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:47.039691 12525 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:47.040109 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:46.919888 17413 consensus_meta.cc:287] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.627s	user 0.011s	sys 0.003s
I20260709 12:11:47.042017 12530 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 8.
I20260709 12:11:47.040874 12524 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:47.005872 12604 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:11:47.035563 17425 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:11:47.049154 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:47.055570 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:47.056120 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:47.057293 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.057996 12597 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:47.058091 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.059685 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:47.060199 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:47.061393 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:47.063838 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.064770 12604 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
W20260709 12:11:47.041280 17222 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.663s	user 0.009s	sys 0.008s
I20260709 12:11:47.068027 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:47.069115 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:47.069850 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:47.070073 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:47.071121 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.071983 12602 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 7. (term mismatch)
I20260709 12:11:47.073277 12602 pending_rounds.cc:85] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 6
I20260709 12:11:47.073530 12602 pending_rounds.cc:107] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 13.7
I20260709 12:11:47.073801 12602 raft_consensus.cc:2889] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:47.074250 12530 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:47.074599 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2521) took 1428 ms. Trace:
I20260709 12:11:47.075361 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.076076 12676 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:47.075621 12602 rpcz_store.cc:276] 0709 12:11:45.646020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:45.764283 (+118263us) service_pool.cc:224] Handling call
0709 12:11:45.765678 (+  1395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:47.024331 (+1258653us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b540002e520. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488997dc0 00007f1488a221c0 00007f1488a23231 00007f1488a1fd8f 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c
0709 12:11:47.074297 (+ 49966us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:47.074538 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":756831104}
I20260709 12:11:47.076522 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.078557 12525 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:11:47.042412 17372 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.042560 17474 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:47.041409 17471 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:47.080737 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:47.108040 12606 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:47.108683 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:11:47.081307 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:47.049310 16719 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:47.091989 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:47.085256 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:47.082693 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:47.067716 17222 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.698s	user 0.010s	sys 0.009s
I20260709 12:11:47.025521 17397 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:47.103528 12643 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:47.058885 17417 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.059101 17298 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:47.053625 16962 consensus_queue.cc:1059] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.049s
I20260709 12:11:47.071466 16943 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.072181 17432 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.081283 17322 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:47.121070 16719 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:47.110806 17372 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.129168 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:47.132606 12606 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:11:47.080224 17431 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:47.059437 17406 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:11:47.080286 17426 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:47.135700 12530 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: [11.6-13.7]
W20260709 12:11:47.158672 17412 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.800s	user 0.006s	sys 0.011s
I20260709 12:11:47.080184 17467 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260709 12:11:47.159329 17354 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:11:47.164800 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:47.165943 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 453) took 2868 ms. Trace:
I20260709 12:11:47.114599 17326 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
W20260709 12:11:47.171056 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 85918.1us
I20260709 12:11:47.184597 17271 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:11:47.179234 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 75831.5us
I20260709 12:11:47.180647 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.115386 17471 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:47.207758 12679 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:47.208392 16719 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:47.208909 17471 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:47.165892 17315 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:47.189563 13305 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:47.181638 17474 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:47.191779 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:47.193686 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:47.197906 12603 rpcz_store.cc:276] 0709 12:11:44.297975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:44.500761 (+202786us) service_pool.cc:224] Handling call
0709 12:11:44.518471 (+ 17710us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:44.782357 (+263886us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:44.782392 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:44.782922 (+   530us) log.cc:844] Serialized 26 byte log entry
0709 12:11:45.013428 (+230506us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:11:45.121776 (+108348us) log.cc:844] Serialized 14 byte log entry
0709 12:11:46.849877 (+1728101us) spinlock_profiling.cc:243] Waited 9.29 ms on lock 0x7b180047f0e0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:11:47.058846 (+208969us) spinlock_profiling.cc:243] Waited 8.52 ms on lock 0x7b180047f0e0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:11:47.165397 (+106551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:47.165503 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:47.165575 (+    72us) raft_consensus.cc:1685] finished
0709 12:11:47.165592 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:47.165873 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":70195,"prepare.queue_time_us":107740,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":12771,"raft.queue_time_us":81738,"raft.run_cpu_time_us":391,"raft.run_wall_time_us":17789,"spinlock_wait_cycles":40961152,"thread_start_us":542210,"threads_started":3,"wal-append.queue_time_us":381051}
I20260709 12:11:47.214946 13335 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 8: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:11:47.203630 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:47.183468 17298 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:47.218992 17298 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:47.213528 17415 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:47.235728 13339 raft_consensus.cc:2410] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:11:47.183084 17417 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:47.183706 17322 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:47.209060 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:47.212059 12643 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:47.242976 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.243645 12598 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:47.172089 12524 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:47.244004 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:47.244280 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:47.244856 12524 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:47.245471 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:47.246969 12603 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:11:47.248087 17426 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.248535 13335 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:47.248639 17426 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:47.252009 12524 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:11:47.185807 17432 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.253392 12524 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 9. Current term is 10. Ops: []
W20260709 12:11:47.239243 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 105883us
I20260709 12:11:47.205999 17271 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:47.257170 17467 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:47.184747 16943 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.252804 17322 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:47.262966 17432 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:47.268572 17372 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:47.279748 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:47.269824 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.283609 12606 raft_consensus.cc:2410] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 15 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 5.
I20260709 12:11:47.265964 12526 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:47.273514 13305 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:47.205274 17354 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:47.256547 17298 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:47.292637 12673 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 8: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:47.298779 12680 raft_consensus.cc:2393] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 13: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:47.272631 17271 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:47.302323 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:47.272181 17467 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:47.305126 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2230) took 2121 ms. Trace:
I20260709 12:11:47.268604 17474 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:47.311517 12531 rpcz_store.cc:276] 0709 12:11:45.183596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:45.183910 (+   314us) service_pool.cc:224] Handling call
0709 12:11:45.190399 (+  6489us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:45.196097 (+  5698us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:45.196139 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:45.203295 (+  7156us) log.cc:844] Serialized 52 byte log entry
0709 12:11:45.290421 (+ 87126us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:45.500726 (+210305us) log.cc:844] Serialized 14 byte log entry
0709 12:11:45.500980 (+   254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:45.501091 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:47.156200 (+1655109us) spinlock_profiling.cc:243] Waited 6.73 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:47.298199 (+141999us) raft_consensus.cc:1685] finished
0709 12:11:47.304913 (+  6714us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:47.305064 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16341,"prepare.queue_time_us":344032,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":4731,"raft.queue_time_us":101671,"raft.run_cpu_time_us":287,"raft.run_wall_time_us":11583,"spinlock_wait_cycles":15469952,"thread_start_us":205936,"threads_started":2,"wal-append.queue_time_us":147574}
I20260709 12:11:47.309449 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2246) took 1488 ms. Trace:
I20260709 12:11:47.314409 12677 rpcz_store.cc:276] 0709 12:11:45.821114 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:45.821270 (+   156us) service_pool.cc:224] Handling call
0709 12:11:45.821625 (+   355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:46.180022 (+358397us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:47.309201 (+1129179us) spinlock_profiling.cc:243] Waited 348 ms on lock 0x7b54000131d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:47.309394 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":800593664}
I20260709 12:11:47.316992 12598 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:11:47.317540 16943 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:47.282542 17383 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:47.318455 17383 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.320746 17383 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:47.289940 17333 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.307971 17454 consensus_queue.cc:1059] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 12:11:47.323006 17333 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.295948 17447 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:47.290728 17321 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.042s
W20260709 12:11:47.324993 17404 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.971s	user 0.002s	sys 0.007s
I20260709 12:11:47.325686 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.326354 12677 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:47.328267 12673 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:47.334558 17354 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:47.336460 17393 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 2.149s	user 0.001s	sys 0.010s
W20260709 12:11:47.339713 12600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2355) took 1929 ms (client timeout 2127 ms). Trace:
W20260709 12:11:47.340186 12600 rpcz_store.cc:269] 0709 12:11:45.410599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:45.412195 (+  1596us) service_pool.cc:224] Handling call
0709 12:11:47.339476 (+1927281us) spinlock_profiling.cc:243] Waited 809 ms on lock 0x7b540007b054. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:47.339626 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1860347136}
I20260709 12:11:47.343349 17426 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:47.307003 17298 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:47.345567 17298 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:47.348527 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:47.349656 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:47.352145 13332 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 13: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:47.351097 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:47.356578 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.357400 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2363) took 1935 ms. Trace:
I20260709 12:11:47.357798 12601 rpcz_store.cc:276] 0709 12:11:45.422780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:45.425349 (+  2569us) service_pool.cc:224] Handling call
0709 12:11:45.425793 (+   444us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:46.688705 (+1262912us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:47.357141 (+668436us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b540022ecd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:47.357328 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2885874944}
I20260709 12:11:47.358942 13339 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 8: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:47.360581 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.361395 12601 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:47.290622 17300 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 12:11:47.346809 17467 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:47.364248 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:47.378048 12677 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:11:47.378964 12677 pending_rounds.cc:85] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 7
I20260709 12:11:47.379582 12677 pending_rounds.cc:107] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:11:47.379868 12677 raft_consensus.cc:2889] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:47.380183 12677 pending_rounds.cc:107] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:11:47.380419 12677 raft_consensus.cc:2889] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:47.381217 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:47.381865 12671 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:47.378921 12673 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:47.387068 17270 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:11:47.387490 17270 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:47.388319 17270 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:47.389859 17270 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:11:47.396741 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:47.370787 17326 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:47.370744 17426 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:47.384464 17321 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.393242 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:47.408624 12673 raft_consensus.cc:2410] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 9 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:11:47.391849 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.384137 17464 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.407366 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:47.413399 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:47.413679 12671 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:47.370947 17415 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:11:47.417788 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.417115 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:47.420296 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:47.419329 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.421521 12673 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:47.422495 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.408859 17321 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.415530 17354 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.432190 17321 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:47.432601 17354 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.409498 17426 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.437808 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.439728 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.424027 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.421712 17471 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:47.412250 17326 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.426445 17411 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:47.427505 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.450425 12678 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 8: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:47.427898 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:47.429404 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.448829 12673 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:47.452270 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:47.452498 12671 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:47.454020 12571 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:47.449733 17471 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:47.460708 17471 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:47.461995 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:47.462447 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:47.463043 12672 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:47.457461 17474 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:47.457288 17383 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:47.467401 17383 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:47.467535 17320 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:47.467845 17383 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:47.468937 17326 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:47.469617 12677 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:47.418823 17464 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.416038 16943 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.036s
I20260709 12:11:47.454622 17411 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:47.459802 12571 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:47.447019 17315 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:47.476214 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:47.480638 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:47.459275 16935 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260709 12:11:47.485734 12571 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:47.486550 17492 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:11:47.496627 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 77285.8us
I20260709 12:11:47.502790 17432 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.503366 17432 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.488621 17315 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.507071 17315 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:47.485440 17365 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.016s
I20260709 12:11:47.508220 17322 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:47.508951 17449 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:47.509428 17449 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:47.510378 17449 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:47.510792 17343 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:47.487236 16962 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:47.521944 12672 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:47.510903 17322 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:47.495222 17466 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:47.530596 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.531661 17497 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.532606 17244 raft_consensus.cc:3055] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:47.532838 12646 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:47.536913 17354 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:47.537511 17467 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:47.540180 12672 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 13: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:47.540628 17336 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.545787 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.508997 17365 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:47.512562 17492 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:47.511590 17411 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.556267 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2310) took 1223 ms. Trace:
I20260709 12:11:47.556876 12528 rpcz_store.cc:276] 0709 12:11:46.332564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:46.332763 (+   199us) service_pool.cc:224] Handling call
0709 12:11:46.333108 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:46.417476 (+ 84368us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:47.555866 (+1138390us) spinlock_profiling.cc:243] Waited 74.4 ms on lock 0x7b54003f6e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:47.556173 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":171070336}
I20260709 12:11:47.543866 17466 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:47.569540 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.571275 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2368) took 1109 ms. Trace:
I20260709 12:11:47.567687 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:47.571976 12605 rpcz_store.cc:276] 0709 12:11:46.462048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:46.485392 (+ 23344us) service_pool.cc:224] Handling call
0709 12:11:47.571022 (+1085630us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b5400231754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:47.571187 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":781990656}
I20260709 12:11:47.572455 12672 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 11 index: 8. (index mismatch)
I20260709 12:11:47.580025 12672 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 10 index: 5. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:11:47.545258 17244 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:47.585417 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.587909 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:47.588850 12602 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:47.548000 17333 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:47.551863 17498 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.616350 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.592514 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.593034 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.618158 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.559919 17298 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:47.595323 12605 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:47.596248 17440 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 14 current term: 14 is higher.
I20260709 12:11:47.548555 17497 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.552814 17336 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.631260 17336 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:47.552203 17365 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:47.632325 17464 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:47.632486 17365 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.585356 17315 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.633945 17432 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:47.634438 17315 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.637020 17315 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:47.637625 17463 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.638227 17463 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.640487 17463 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:47.641007 17463 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.641441 17463 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.645411 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2241) took 1229 ms. Trace:
I20260709 12:11:47.646032 17467 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:47.583388 17300 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.063s
W20260709 12:11:47.647332 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 453701us
I20260709 12:11:47.648684 16962 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:47.648769 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:47.651265 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:47.654597 17244 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:47.655745 17244 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:47.656615 17392 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:47.657215 17383 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.625216 17499 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:47.658829 17499 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:47.659345 17392 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:11:47.660552 17383 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:47.609869 17426 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:47.612145 17417 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:47.615582 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.661929 12676 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:11:47.662571 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:47.616519 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 49023.6us
I20260709 12:11:47.662932 17497 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:47.554427 17492 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:47.663602 12646 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:47.658653 17372 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:47.664397 17372 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:47.664789 17372 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:47.665274 13337 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 9. (term mismatch)
I20260709 12:11:47.665683 13337 pending_rounds.cc:85] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 8
I20260709 12:11:47.665931 13337 pending_rounds.cc:107] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 10.9
I20260709 12:11:47.662985 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.629012 17498 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:47.667722 17425 raft_consensus.cc:3055] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:47.607749 17340 raft_consensus.cc:3055] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:47.668179 17425 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:47.668499 17340 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:47.669521 17340 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:47.618836 12601 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:47.625433 12602 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
I20260709 12:11:47.627089 12604 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:11:47.730515 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.731336 12602 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 8: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:47.731417 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.732299 12604 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:47.628281 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.733026 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.730516 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.583300 17466 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:47.591272 17413 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:47.660022 17499 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:47.618541 12606 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 14: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:47.737211 12643 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:47.658888 17502 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:11:47.682504 17365 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:47.589406 17271 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.646057 12522 rpcz_store.cc:276] 0709 12:11:46.416099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:47.042587 (+626488us) service_pool.cc:224] Handling call
0709 12:11:47.042921 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:47.043149 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:11:47.065576 (+ 22427us) log.cc:844] Serialized 14 byte log entry
0709 12:11:47.190251 (+124675us) log.cc:844] Serialized 14 byte log entry
0709 12:11:47.644611 (+454360us) spinlock_profiling.cc:243] Waited 6.28 ms on lock 0x7b180049a440. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:11:47.644876 (+   265us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:47.644951 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:47.645036 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:47.645137 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:47.645322 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":450425,"prepare.queue_time_us":22350,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":5093,"raft.queue_time_us":455776,"raft.run_cpu_time_us":252,"raft.run_wall_time_us":8232,"spinlock_wait_cycles":14440832,"thread_start_us":117429,"threads_started":1,"wal-append.queue_time_us":144720}
I20260709 12:11:47.766788 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:47.700706 17498 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:47.701231 12599 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260709 12:11:47.701583 12597 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:11:47.769812 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.770241 12597 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:47.770473 12599 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:47.772104 12597 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:11:47.666344 17497 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:47.774471 17497 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:47.775457 17497 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.703315 17315 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:47.778260 17315 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.780402 17315 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:47.703533 17288 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:47.703729 17471 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.706178 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:47.616380 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.782160 12676 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:47.782877 17471 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:47.712016 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:47.716288 12524 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:47.783747 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.717295 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:47.784060 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 72308.8us
I20260709 12:11:47.666272 13337 raft_consensus.cc:2889] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:47.784759 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:47.658782 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.785864 12680 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:47.787034 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.788481 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.727874 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:47.780483 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:47.791966 12677 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:11:47.728046 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:11:47.735466 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 79108.8us
I20260709 12:11:47.586057 17411 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:47.794767 17440 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:47.795392 17440 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:47.795858 17440 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:47.744546 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.803207 12602 raft_consensus.cc:2410] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 12 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:11:47.804629 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.805531 16962 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:47.806224 16962 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:47.805856 17440 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.744264 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.807265 16962 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:47.808094 12601 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:11:47.809087 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.809773 12601 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:47.742981 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:47.733948 12673 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:11:47.810432 12528 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:47.805570 12602 raft_consensus.cc:2410] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:11:47.813184 12572 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:47.813400 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:47.749878 12427 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47524 (request call id 98) took 19825 ms (client timeout 15000 ms). Trace:
I20260709 12:11:47.756136 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.764963 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 10 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:47.777597 17498 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:47.780439 17431 consensus_queue.cc:1059] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:47.781332 17288 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.783084 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:47.890851 12672 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:11:47.671300 17306 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:47.785830 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:47.740998 12605 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 13 index: 6. (index mismatch)
I20260709 12:11:47.813853 12673 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:11:47.832131 17333 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:47.832926 12604 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:47.832396 12680 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:11:47.834934 16935 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:47.898800 16211 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:47.793983 12677 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:11:47.841061 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:47.842085 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:47.900771 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:47.915145 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 73236.5us
I20260709 12:11:47.921061 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:47.923000 12525 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 11: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:11:47.928207 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 142433us
I20260709 12:11:47.929284 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:47.935431 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:47.936159 12605 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:47.936439 12674 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 11: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:47.883729 17426 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:47.940485 12528 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:11:47.843981 12597 raft_consensus.cc:2410] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 6.
W20260709 12:11:47.814240 12427 rpcz_store.cc:269] 0709 12:11:27.829071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:27.837686 (+  8615us) service_pool.cc:224] Handling call
0709 12:11:41.153487 (+13315801us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b54000100a8. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148c38a161 00007f148c4143af 00007f148c415b11 00007f148e915fa1 00007f148e9167ff 00007f148e7d5a1b 00007f148e8dd90e 00007f148cd9e2a4 00007f148cd9e232 00007f148cd9e161 00007f148cd9e0dc
0709 12:11:47.654066 (+6500579us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:11:42.476371 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:11:42.477132 (+   761us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:11:42.477188 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:11:42.477208 (+    20us) tablet.cc:662] Decoding operations
0709 12:11:42.478437 (+  1229us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:11:42.478541 (+   104us) write_op.cc:641] Partition lock acquired for write op
0709 12:11:42.478574 (+    33us) tablet.cc:685] Acquiring locks for 9 operations
0709 12:11:42.479487 (+   913us) tablet.cc:701] Row locks acquired
0709 12:11:42.479542 (+    55us) write_op.cc:260] PREPARE: finished
0709 12:11:42.479725 (+   183us) write_op.cc:270] Start()
0709 12:11:42.479893 (+   168us) write_op.cc:276] Timestamp: P: 1783599102479652 usec, L: 0
0709 12:11:42.479926 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 12:11:42.480738 (+   812us) log.cc:844] Serialized 3664 byte log entry
0709 12:11:43.473797 (+993059us) op_driver.cc:464] REPLICATION: finished
0709 12:11:43.875652 (+401855us) write_op.cc:301] APPLY: starting
0709 12:11:43.899897 (+ 24245us) tablet.cc:1367] starting BulkCheckPresence
0709 12:11:44.264161 (+364264us) tablet.cc:1370] finished BulkCheckPresence
0709 12:11:44.264319 (+   158us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:11:44.825162 (+560843us) spinlock_profiling.cc:243] Waited 8.4 ms on lock 0x7b1800013a80. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f148875f0ec 00007f14887abc0a 00007f14887ab4f7 00007f14887ab490 00007f1488762f48 00007f148c4771bb 00007f148c4be5d4 00007f148c36a8da 00007f148c36d3cb
0709 12:11:44.829445 (+  4283us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:11:44.829788 (+   343us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=18,key_file_lookups=18,delta_file_lookups=18,mrs_lookups=0
0709 12:11:44.829836 (+    48us) write_op.cc:312] APPLY: finished
0709 12:11:44.830253 (+   417us) log.cc:844] Serialized 89 byte log entry
0709 12:11:45.161869 (+331616us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:45.181820 (+ 19951us) write_op.cc:454] Released schema lock
0709 12:11:45.202199 (+ 20379us) write_op.cc:341] FINISH: Updating metrics
0709 12:11:45.332454 (+130255us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:45.353689 (+ 21235us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1197994,"reported_tablets":14,"spinlock_wait_cycles":30456064,"child_traces":[["op",{"apply.queue_time_us":368812,"apply.run_cpu_time_us":69503,"apply.run_wall_time_us":1452314,"cfile_cache_hit":38,"cfile_cache_hit_bytes":41693,"num_ops":9,"prepare.queue_time_us":831099,"prepare.run_cpu_time_us":21298,"prepare.run_wall_time_us":267586,"raft.queue_time_us":265234,"raft.run_cpu_time_us":8770,"raft.run_wall_time_us":139164,"replication_time_us":993670,"spinlock_wait_cycles":19319552,"thread_start_us":955594,"threads_started":5,"wal-append.queue_time_us":789894,"wal-append.run_cpu_time_us":84773,"wal-append.run_wall_time_us":2736355}]]}
I20260709 12:11:47.808921 17440 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:47.851020 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:47.851306 17503 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:47.851964 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:47.854437 12527 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:47.947130 12606 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 11: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:47.855618 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:47.856212 17415 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:11:47.726649 17432 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:47.857524 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:47.861847 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:47.862246 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:47.868750 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:47.869717 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:47.877039 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.878703 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:47.880353 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:47.842840 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:47.820093 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:47.939363 12673 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
W20260709 12:11:47.950042 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:47.937546 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:47.925732 17306 consensus_queue.cc:1243] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:11:47.887988 17498 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:47.756888 17413 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:47.925333 17406 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:48.052898 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:48.054384 12525 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
W20260709 12:11:48.011583 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 168838us
W20260709 12:11:48.002787 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:48.003327 12531 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:48.013664 17222 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:48.013837 17430 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:48.014590 17463 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:48.001837 17503 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:48.016973 17271 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:48.008940 17333 raft_consensus.cc:697] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:48.006570 12598 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:48.070724 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:48.008765 12671 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 11: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:48.002360 12526 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:48.006950 12600 raft_consensus.cc:2368] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14.
I20260709 12:11:48.025682 17392 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:48.027657 17340 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:47.669122 17425 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:47.924925 16935 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:48.036369 17498 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:47.928967 17372 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:48.037181 17432 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:48.039428 17270 raft_consensus.cc:3055] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:48.040920 17492 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:48.041318 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 112085us
I20260709 12:11:48.042308 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:47.937263 17343 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:48.048975 12605 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:11:47.948690 17426 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:11:48.060952 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:48.098376 17463 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:48.168919 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:48.066380 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 360299us
I20260709 12:11:48.007689 13335 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:48.059562 17126 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:48.059635 17222 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:48.169710 12523 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:48.170737 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:48.177068 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:48.177453 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:48.178059 12524 raft_consensus.cc:2410] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:11:48.178369 12523 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:48.180261 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:48.105865 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:48.155161 12571 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:48.157090 12527 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:48.160928 12526 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:48.185514 12526 raft_consensus.cc:606] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:48.186053 12526 raft_consensus.cc:614] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:11:48.064002 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 269250us
I20260709 12:11:48.050843 17464 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:48.193953 12599 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:48.195344 16211 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:48.196033 16211 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:48.197863 12494 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:48.199417 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 93707.4us
W20260709 12:11:48.200404 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:48.148185 17270 raft_consensus.cc:740] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:48.159011 17343 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:48.205266 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:48.206015 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:48.218982 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:48.220649 12527 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 11: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:48.090726 17430 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:48.174196 17463 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:48.213588 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:48.160405 17426 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:48.213689 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:48.205060 16962 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:48.208467 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:48.245651 12680 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:48.220531 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:48.227377 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:48.171983 17126 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:48.192976 17222 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:48.214474 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:48.241267 16211 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:48.191136 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:48.208051 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:48.206398 17411 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:48.247550 12674 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:11:48.234561 17440 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.248569 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:11:48.254559 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46586.1us
I20260709 12:11:48.228660 17343 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:48.232983 17415 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:48.263398 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 72395.5us
I20260709 12:11:48.230450 17306 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:48.269686 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:48.262194 12527 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:48.238201 17463 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:48.249819 16962 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:48.282415 12572 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:48.288470 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:48.284574 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 73186.5us
I20260709 12:11:48.214716 17464 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:48.278326 12496 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:48.260285 17126 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:48.261250 17411 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:48.267856 17498 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:48.312433 12572 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:48.233244 16719 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 12:11:48.304051 12496 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:48.266202 17507 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:48.311964 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:48.218878 17333 consensus_queue.cc:237] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:48.317603 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:48.327320 12673 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:11:48.318153 12496 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:48.319545 17512 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:48.335409 17512 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:48.320132 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 122376us
I20260709 12:11:48.322769 13304 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:48.343212 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:48.345254 17406 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:48.353262 17406 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:48.353822 17406 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:48.357049 12494 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:48.347354 17320 consensus_queue.cc:1059] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:11:48.348938 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 57345.2us
I20260709 12:11:48.359086 17407 raft_consensus.cc:3055] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:48.359903 17407 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:48.361368 17407 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:48.364387 17407 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:48.352152 17376 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:48.300755 16962 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:48.374086 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:48.376398 12527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10395d6e75b84c3982521323f92e3309"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:48.381513 12527 raft_consensus.cc:606] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:11:48.388298 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 70285.1us
I20260709 12:11:48.389662 12496 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:48.375844 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:48.301268 17514 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:48.310866 17392 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:48.421432 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:48.313055 17411 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:48.273959 17440 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:48.409264 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:48.410398 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 65566.1us
I20260709 12:11:48.416181 17509 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:48.350431 17512 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:48.319061 17507 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:48.316810 17498 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:48.317713 16943 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:48.370889 17298 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:48.370854 17306 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:48.327385 17413 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:48.436403 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:48.462389 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:48.285279 17415 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:48.284083 17270 consensus_queue.cc:260] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:48.394472 17432 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:48.447633 12525 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 11 index: 9. (index mismatch)
I20260709 12:11:48.421608 17428 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260709 12:11:48.431841 17503 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:48.482388 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 92842.2us
I20260709 12:11:48.489265 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:48.458532 17498 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:48.466848 17298 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:48.490823 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:48.489420 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:48.491833 12525 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:48.458984 17512 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:48.452870 17514 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:48.453395 17507 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:48.461933 17383 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:48.456305 17372 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:48.461040 17528 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:48.461160 17306 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:48.466455 16935 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:48.498694 17376 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:48.526957 12531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "502d264440884ecda3d4641918b20d0c"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:48.527745 12531 raft_consensus.cc:606] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:48.475975 17492 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:48.510651 17513 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
W20260709 12:11:48.517692 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:48.535650 12680 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:11:48.535873 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:48.518669 12525 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:48.538465 17464 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:48.541147 17524 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.541766 17524 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:48.543437 17365 consensus_meta.cc:287] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.667s	user 0.007s	sys 0.003s
I20260709 12:11:48.544104 17365 consensus_queue.cc:1059] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:11:48.468106 17392 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.038s
I20260709 12:11:48.551518 17503 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.552201 17503 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:48.553943 17244 consensus_meta.cc:287] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.898s	user 0.004s	sys 0.008s
W20260709 12:11:48.558166 17505 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.327s	user 0.001s	sys 0.000s
I20260709 12:11:48.558596 17449 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:48.560019 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2253) took 1775 ms. Trace:
I20260709 12:11:48.560456 12530 rpcz_store.cc:276] 0709 12:11:46.784393 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:47.206185 (+421792us) service_pool.cc:224] Handling call
0709 12:11:47.242036 (+ 35851us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:47.649295 (+407259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:47.649328 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:47.649934 (+   606us) log.cc:844] Serialized 26 byte log entry
0709 12:11:47.915135 (+265201us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:47.915219 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:47.915341 (+   122us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:48.559788 (+644447us) raft_consensus.cc:1685] finished
0709 12:11:48.559812 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:48.559956 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":403850,"raft.queue_time_us":412970,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":133,"thread_start_us":264265,"threads_started":1,"wal-append.queue_time_us":499500}
I20260709 12:11:48.562494 17430 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:48.563176 12527 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:11:48.563575 17506 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
W20260709 12:11:48.564592 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:48.563587 12527 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:48.551600 17415 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:48.567190 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:48.567245 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:48.567263 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:48.573678 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:48.574607 12680 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:11:48.576383 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:11:48.569139 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 80054.9us
I20260709 12:11:48.579710 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:48.580701 16943 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:48.581168 16943 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:48.581700 16943 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:48.582911 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:48.583638 13337 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 17: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:48.586980 17298 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.5, Last appended by leader: 5, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:48.566692 12527 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:48.588497 12527 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:48.590896 17271 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:48.591476 17271 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:48.593832 17271 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:48.595273 17440 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:48.597179 17507 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:48.600220 16943 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:48.565197 17515 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 12:11:48.581050 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:48.608134 12528 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 10 index: 8. Preceding OpId from leader: term: 11 index: 8. (term mismatch)
I20260709 12:11:48.526927 17306 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:48.614281 12527 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
W20260709 12:11:48.635844 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 265950us
I20260709 12:11:48.636029 12528 pending_rounds.cc:85] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 7
I20260709 12:11:48.636793 12528 pending_rounds.cc:107] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 10.8
I20260709 12:11:48.637151 12528 raft_consensus.cc:2889] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:48.641269 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:48.643090 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:48.644050 12676 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:48.645277 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:48.606046 12496 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:48.636204 12527 pending_rounds.cc:85] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 6
I20260709 12:11:48.647428 12527 pending_rounds.cc:107] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:11:48.647701 12527 raft_consensus.cc:2889] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:11:48.573431 17510 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.322s	user 0.004s	sys 0.000s
I20260709 12:11:48.632776 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:48.734769 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:48.675493 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:48.706602 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 100630us
I20260709 12:11:48.662775 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:48.808440 12496 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:48.804392 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 403094us
W20260709 12:11:48.811484 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 178812us
I20260709 12:11:48.803941 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:48.814329 12601 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
W20260709 12:11:48.756354 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 323539us
I20260709 12:11:48.815102 12528 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 13: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:48.815384 12494 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:48.570677 17464 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:48.864189 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50382.9us
I20260709 12:11:48.864931 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:48.865612 12676 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
W20260709 12:11:48.870302 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52659.1us
W20260709 12:11:48.870420 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 55167.7us
I20260709 12:11:48.872032 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:48.873011 12494 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:48.631836 17244 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:11:48.707358 17466 consensus_meta.cc:287] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.830s	user 0.008s	sys 0.004s
I20260709 12:11:48.871362 12496 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:48.718254 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:48.873971 12597 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:48.884578 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:48.631433 17376 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:48.727545 17418 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.885273 12601 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:48.747757 17507 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:48.730037 17503 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:48.757732 16962 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:48.749085 17222 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:48.743570 17126 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:48.728961 17406 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:49.165979 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:48.967424 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:48.890853 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:49.175383 12522 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:11:49.176141 12525 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 13: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:11:48.951656 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:49.176584 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:49.177099 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:49.177740 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:49.031975 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:48.953979 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2398) took 1212 ms. Trace:
W20260709 12:11:48.987557 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:49.147507 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 261632us
I20260709 12:11:48.887657 12494 leader_election.cc:400] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14.
W20260709 12:11:49.029068 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:49.182415 13302 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:49.182442 12494 leader_election.cc:403] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:11:48.873392 17497 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.806896 17300 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.816637 17271 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:49.183732 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:48.940416 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:49.198963 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 311428us
I20260709 12:11:48.874431 17413 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:49.206943 12522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:49.177938 12525 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:48.806406 17407 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:49.178740 12606 rpcz_store.cc:276] 0709 12:11:47.741869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:48.005158 (+263289us) service_pool.cc:224] Handling call
0709 12:11:48.005505 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:48.034607 (+ 29102us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:11:48.300457 (+265850us) log.cc:844] Serialized 14 byte log entry
0709 12:11:48.455497 (+155040us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:48.456031 (+   534us) log.cc:844] Serialized 26 byte log entry
0709 12:11:48.456310 (+   279us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:48.456353 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:48.456440 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:48.953747 (+497307us) raft_consensus.cc:1685] finished
0709 12:11:48.953771 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:48.953913 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":56707,"prepare.queue_time_us":399882,"prepare.run_cpu_time_us":134,"prepare.run_wall_time_us":132,"raft.queue_time_us":131274,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"spinlock_wait_cycles":34193792,"thread_start_us":348682,"threads_started":2,"wal-append.queue_time_us":459103}
I20260709 12:11:49.207026 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:49.211189 13337 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:11:49.211246 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:48.887290 17376 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:48.727258 17534 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election won for term 11
W20260709 12:11:48.878068 17426 consensus_meta.cc:287] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.613s	user 0.012s	sys 0.000s
I20260709 12:11:49.193212 17454 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:49.071394 17530 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.130s	user 0.000s	sys 0.001s
W20260709 12:11:48.943624 17411 consensus_meta.cc:287] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.000s
I20260709 12:11:48.938237 17430 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:48.953553 17546 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:48.934031 16719 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.272s
I20260709 12:11:48.879876 17244 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:48.884217 17326 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:49.247354 17326 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:49.247994 17244 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:11:49.248309 16962 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:48.880587 17528 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:49.249087 17528 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:49.251408 17528 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:49.252148 17524 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:49.208962 12525 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:49.149402 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:49.213214 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:49.255398 17222 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:49.255752 12606 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:49.256278 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:49.214047 12569 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:49.257624 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 43663.2us
I20260709 12:11:49.204650 17271 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:49.215145 17535 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:11:48.873169 17509 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.333s
W20260709 12:11:49.215663 17545 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 12:11:48.801368 17343 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:49.260514 17271 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:49.261175 12600 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:49.261443 17500 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:49.261894 17500 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:49.262271 17500 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:49.262297 17343 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:49.263216 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 474) took 1052 ms. Trace:
I20260709 12:11:49.263638 12600 rpcz_store.cc:276] 0709 12:11:48.210704 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:48.211578 (+   874us) service_pool.cc:224] Handling call
0709 12:11:48.211908 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:49.261546 (+1049638us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:49.262924 (+  1378us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b54001167d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:49.263126 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2412674432}
I20260709 12:11:49.264962 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:49.265647 12600 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:11:49.166277 17320 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:49.217963 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:49.266566 17320 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:48.650539 17463 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:49.268996 17320 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:49.269110 17463 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:48.963792 17512 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:49.269953 17512 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:49.270404 17512 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:49.271580 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2391) took 1547 ms. Trace:
I20260709 12:11:49.271975 12604 rpcz_store.cc:276] 0709 12:11:47.724115 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:47.897315 (+173200us) service_pool.cc:224] Handling call
0709 12:11:47.897654 (+   339us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:48.150140 (+252486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:48.150184 (+    44us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:48.150231 (+    47us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:48.150274 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:49.271338 (+1121064us) spinlock_profiling.cc:243] Waited 252 ms on lock 0x7b54002ffb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:49.271384 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:49.271517 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":580058496}
I20260709 12:11:49.272763 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:49.273442 12671 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:49.274106 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:49.274828 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:49.273396 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:49.193037 17354 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:49.202904 12676 raft_consensus.cc:3055] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:49.222846 17534 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:49.220177 13335 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:11:49.220439 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:49.291546 12531 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:48.948302 17536 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:49.291872 17534 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.6, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:48.952850 17519 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:48.952313 17539 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:49.197237 17503 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:49.293804 17519 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:49.293938 17539 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:49.295007 17440 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:11:48.940043 17372 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.280s
I20260709 12:11:49.225241 17426 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:49.296811 17539 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:49.297243 17126 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:49.236495 17508 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:49.297531 17503 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:49.298158 17508 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:49.298588 17426 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:49.203413 17300 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:49.300220 17503 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:49.300882 17508 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:49.304044 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:49.305199 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:49.305771 12680 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:49.306054 17406 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:49.306870 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:49.307025 17425 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:48.884837 17533 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.350s
I20260709 12:11:49.308483 12680 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 14.
I20260709 12:11:48.936697 17518 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.338s
I20260709 12:11:49.316790 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:49.217710 17507 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 14
W20260709 12:11:49.240622 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 369326us
I20260709 12:11:49.221756 17376 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:49.188472 17418 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:48.827178 17532 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:49.214627 17407 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:49.236498 17430 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:49.255402 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:49.256532 17270 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:49.258948 17288 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:49.275823 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:49.270434 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:49.277380 12601 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:49.315670 17519 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:49.277664 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:49.275770 12604 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:11:49.355424 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:49.278335 12676 raft_consensus.cc:740] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:49.077391 16943 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:49.282274 17354 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:49.289480 12671 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:11:49.244354 17546 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:49.236436 17411 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:49.204770 17497 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:49.295429 17541 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.355s	user 0.000s	sys 0.003s
I20260709 12:11:48.885864 17464 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:48.893980 17466 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:49.387683 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 205865us
I20260709 12:11:49.388041 17466 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:49.240942 17551 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:49.388895 12603 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:49.385433 17244 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:49.393589 12646 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:49.324731 12606 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:49.395136 12606 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:49.239509 17544 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:49.270112 17449 consensus_meta.cc:287] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.711s	user 0.008s	sys 0.004s
I20260709 12:11:49.328815 12496 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:49.300670 17300 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:49.400573 17222 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:49.401590 17222 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:49.402288 17326 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:49.402738 17551 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:49.341408 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:49.405336 17551 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:49.405823 13335 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 12: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:49.351027 12600 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 13: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:49.407476 17354 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:49.408075 17533 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:11:49.349901 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 152754us
I20260709 12:11:49.408643 17533 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:49.408958 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2339) took 1105 ms. Trace:
I20260709 12:11:49.409068 17533 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:49.351847 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:49.410730 12601 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:49.351064 12675 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 13: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:49.411258 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:49.359292 12494 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:49.357105 12676 consensus_queue.cc:260] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:49.411906 12677 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:49.408675 12600 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:48.949932 17543 raft_consensus.cc:2764] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:11:49.374396 12678 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:49.385833 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2394) took 1660 ms. Trace:
I20260709 12:11:49.415954 12597 rpcz_store.cc:276] 0709 12:11:47.724959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:48.005489 (+280530us) service_pool.cc:224] Handling call
0709 12:11:48.005756 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:48.881770 (+876014us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:49.385565 (+503795us) spinlock_profiling.cc:243] Waited 868 ms on lock 0x7b540007b054. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:49.385745 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1996299136}
I20260709 12:11:49.409391 12524 rpcz_store.cc:276] 0709 12:11:48.303995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:48.304172 (+   177us) service_pool.cc:224] Handling call
0709 12:11:48.304529 (+   357us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:11:48.305005 (+   476us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:48.305047 (+    42us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:11:48.330146 (+ 25099us) log.cc:844] Serialized 78 byte log entry
0709 12:11:48.637295 (+307149us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:48.877751 (+240456us) log.cc:844] Serialized 14 byte log entry
0709 12:11:48.945988 (+ 68237us) log.cc:844] Serialized 14 byte log entry
0709 12:11:48.946245 (+   257us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:48.946348 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:49.259762 (+313414us) spinlock_profiling.cc:243] Waited 1.07 ms on lock 0x7b4800002958. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:11:49.260852 (+  1090us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b1800367180. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485933522 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:11:49.265315 (+  4463us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b1800367180. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240
0709 12:11:49.265783 (+   468us) spinlock_profiling.cc:243] Waited 2.47 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:49.408736 (+142953us) raft_consensus.cc:1685] finished
0709 12:11:49.408759 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:49.408904 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67626,"prepare.queue_time_us":194555,"prepare.run_wall_time_us":6112,"raft.queue_time_us":72559,"raft.run_cpu_time_us":8092,"raft.run_wall_time_us":315331,"spinlock_wait_cycles":74447360,"thread_start_us":210786,"threads_started":1,"wal-append.queue_time_us":326943}
I20260709 12:11:49.387773 17407 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:49.309544 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:11:49.390689 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 84614.8us
I20260709 12:11:49.421020 12679 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
I20260709 12:11:49.399443 12606 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:11:49.408138 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2549) took 1706 ms. Trace:
I20260709 12:11:49.344255 17413 raft_consensus.cc:697] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 13 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:49.412637 12676 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:49.349246 17430 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:49.401969 17222 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:49.326237 17372 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:11:49.421838 12599 rpcz_store.cc:276] 0709 12:11:47.701923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:47.771210 (+ 69287us) service_pool.cc:224] Handling call
0709 12:11:47.771453 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:48.200446 (+428993us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:49.407912 (+1207466us) spinlock_profiling.cc:243] Waited 422 ms on lock 0x7b54002e6dd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:49.408079 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":971247488}
I20260709 12:11:49.423416 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:49.432874 12599 pending_rounds.cc:85] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 5
I20260709 12:11:49.433264 12599 pending_rounds.cc:107] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:11:49.413136 12646 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:49.346762 17532 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:49.472942 12599 raft_consensus.cc:2889] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:11:49.473842 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:49.474324 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:49.420753 17449 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:49.402494 17544 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:49.489112 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:49.489784 12671 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
I20260709 12:11:49.424240 17464 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:49.484297 12600 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
W20260709 12:11:49.438715 17498 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.942s	user 0.006s	sys 0.007s
I20260709 12:11:49.504784 12598 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:11:49.506763 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:49.505945 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:49.509933 13335 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:49.510695 12674 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 12: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:11:49.497730 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 85482.4us
I20260709 12:11:49.515087 12646 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:49.439633 17411 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:49.516883 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:49.437739 17447 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:49.529887 12646 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:49.525923 12676 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:49.518465 16211 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:49.478816 17546 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:49.534420 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:49.535406 12675 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:11:49.509130 17426 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:49.530771 17447 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:49.542205 17321 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:11:49.542366 17553 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:49.557574 17553 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:49.558184 17553 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:49.554260 17306 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.673s	user 0.005s	sys 0.006s
I20260709 12:11:49.562702 17306 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:49.516245 17498 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:49.555137 17447 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:49.574678 17512 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:49.537209 17413 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:49.567800 17376 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:49.565742 17418 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:49.568992 17507 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election lost for term 13. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14.
I20260709 12:11:49.591259 17500 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:49.592676 17544 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:49.596570 17457 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:49.597227 17457 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:49.599078 17532 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:49.600302 17512 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:49.399303 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 70637us
W20260709 12:11:49.323683 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:49.638705 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2390) took 1914 ms. Trace:
I20260709 12:11:49.639540 12602 rpcz_store.cc:276] 0709 12:11:47.723864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:47.827965 (+104101us) service_pool.cc:224] Handling call
0709 12:11:47.828479 (+   514us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:47.880989 (+ 52510us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:11:48.055728 (+174739us) log.cc:844] Serialized 14 byte log entry
0709 12:11:48.327927 (+272199us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:48.557361 (+229434us) spinlock_profiling.cc:243] Waited 12.5 ms on lock 0x7b1800449d40. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485933557 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:11:49.517239 (+959878us) spinlock_profiling.cc:243] Waited 44 ms on lock 0x7b2c00000e98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:11:49.517841 (+   602us) log.cc:844] Serialized 26 byte log entry
0709 12:11:49.518120 (+   279us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:49.518200 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:49.518333 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:49.637613 (+119280us) raft_consensus.cc:1685] finished
0709 12:11:49.637724 (+   111us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:49.638364 (+   640us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41623,"prepare.queue_time_us":175124,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":53,"raft.queue_time_us":167697,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":5759,"spinlock_wait_cycles":129910016,"thread_start_us":223282,"threads_started":1,"wal-append.queue_time_us":434861,"wal-append.run_cpu_time_us":1395,"wal-append.run_wall_time_us":1009053}
I20260709 12:11:49.643311 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:49.638767 12522 raft_consensus.cc:606] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:49.645722 12522 raft_consensus.cc:614] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:49.647558 12527 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
W20260709 12:11:49.652964 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 123051us
I20260709 12:11:49.655953 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:49.659430 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 475) took 1087 ms. Trace:
I20260709 12:11:49.662305 12680 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:49.663247 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:49.664773 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 254480us
I20260709 12:11:49.668401 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:11:49.670688 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:49.671821 13337 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
W20260709 12:11:49.633420 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 273944us
I20260709 12:11:49.675292 16935 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:49.676471 16935 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:49.660082 12673 rpcz_store.cc:276] 0709 12:11:48.572169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:48.572607 (+   438us) service_pool.cc:224] Handling call
0709 12:11:48.584135 (+ 11528us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:48.584356 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:48.584381 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:49.208551 (+624170us) spinlock_profiling.cc:243] Waited 5.74 ms on lock 0x7b2c00037868. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:11:49.222240 (+ 13689us) log.cc:844] Serialized 26 byte log entry
0709 12:11:49.367064 (+144824us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:11:49.367105 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:49.367177 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:49.658506 (+291329us) raft_consensus.cc:1685] finished
0709 12:11:49.658550 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:49.659099 (+   549us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":13205376,"thread_start_us":88522,"threads_started":1,"wal-append.queue_time_us":196970}
I20260709 12:11:49.680389 16935 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:49.683733 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47694.4us
I20260709 12:11:49.685012 12569 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:49.538370 17222 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:49.706619 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 51641.8us
I20260709 12:11:49.709878 12528 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 9. Current term is 11. Ops: []
I20260709 12:11:49.685173 12496 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:49.713253 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:49.686344 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:49.697278 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:49.687108 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:49.691964 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:49.687140 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:49.690846 12571 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:49.696544 17298 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:49.750797 17372 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:49.751756 17372 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:49.752348 17372 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:49.752874 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 67915.1us
I20260709 12:11:49.754666 17397 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:49.755000 12494 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:49.755649 17397 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:49.698201 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:49.714252 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:49.798275 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 49865.6us
I20260709 12:11:49.769218 12672 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:49.690477 17340 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:49.760306 12646 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:49.763734 13332 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:11:49.694615 17306 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:49.707768 17326 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:49.794704 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2300) took 1155 ms. Trace:
I20260709 12:11:49.757923 12494 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:49.764422 13335 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:11:49.773275 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:49.791502 12679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 467) took 2031 ms (client timeout 1855 ms). Trace:
I20260709 12:11:49.709239 17376 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:49.691677 17354 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:49.838893 12680 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:49.794927 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:49.770208 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:49.776087 12604 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:49.819420 16935 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:49.841655 12676 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 18
W20260709 12:11:49.842231 17568 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
W20260709 12:11:49.843158 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 193288us
I20260709 12:11:49.847085 17397 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:49.759609 17512 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.089s
I20260709 12:11:49.768554 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:49.848590 17340 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:49.837390 12679 rpcz_store.cc:269] 0709 12:11:47.760371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:47.763841 (+  3470us) service_pool.cc:224] Handling call
0709 12:11:49.790927 (+2027086us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54002c5e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:49.791208 (+   281us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3340368256}
I20260709 12:11:49.776647 17288 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.074s
I20260709 12:11:49.851140 12529 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 13: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:49.762217 17550 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.090s
I20260709 12:11:49.760435 17241 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:49.874591 17241 raft_consensus.cc:697] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:49.875058 17507 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:49.875608 17507 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:49.876138 17507 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:49.878691 12671 raft_consensus.cc:2410] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 10 because replica has last-logged OpId of term: 8 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 4.
I20260709 12:11:49.879112 12524 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:11:49.880914 12571 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:49.858690 13337 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:49.836683 12678 rpcz_store.cc:276] 0709 12:11:48.639121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:48.646713 (+  7592us) service_pool.cc:224] Handling call
0709 12:11:49.794260 (+1147547us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b54002c6854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:49.794454 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1131050240}
W20260709 12:11:49.861972 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 104495us
I20260709 12:11:49.759485 17392 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:49.771754 17222 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:49.867416 12601 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 12: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:49.868151 12569 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:49.869175 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:49.871372 12526 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 14.
I20260709 12:11:49.871883 12531 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:11:49.874337 17426 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:49.775451 17508 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:49.857230 17326 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:49.876811 17241 consensus_queue.cc:237] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:49.884189 17354 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:49.866164 16935 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:49.903465 12679 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:11:49.905187 12674 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:11:49.902240 12675 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260709 12:11:49.907061 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:49.767246 17534 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.130s
I20260709 12:11:49.765113 17543 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.128s
I20260709 12:11:49.917013 12675 pending_rounds.cc:85] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 4
I20260709 12:11:49.894059 17426 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:49.908630 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 43971us
I20260709 12:11:49.922515 12675 pending_rounds.cc:107] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 8.5
I20260709 12:11:49.896833 17326 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:49.927083 12675 raft_consensus.cc:2889] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:49.930588 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48476 (request call id 35) took 1016 ms. Trace:
I20260709 12:11:49.931129 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:49.935022 12671 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
W20260709 12:11:49.936685 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 56055.9us
I20260709 12:11:49.935151 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:49.930918 12522 rpcz_store.cc:276] 0709 12:11:48.906600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:49.205584 (+298984us) service_pool.cc:224] Handling call
0709 12:11:49.907762 (+702178us) spinlock_profiling.cc:243] Waited 431 ms on lock 0x7b5400015c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1a853 00007f148d9e860a 00007f1486f6e7d4 00007f1486f6e762 00007f1486f6e691 00007f1486f6e60c 00007f1486f6d8c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:49.923169 (+ 15407us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":990468992}
I20260709 12:11:49.899658 17499 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:49.940512 17315 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:49.924232 17449 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:49.947033 12526 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:11:49.949060 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:49.950810 17270 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:49.954994 17270 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:49.955395 17340 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:49.954525 17553 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:49.951339 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2536) took 2338 ms. Trace:
I20260709 12:11:49.956595 12603 rpcz_store.cc:276] 0709 12:11:47.584176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:47.584493 (+   317us) service_pool.cc:224] Handling call
0709 12:11:47.627710 (+ 43217us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:49.391864 (+1764154us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:49.907606 (+515742us) spinlock_profiling.cc:243] Waited 1.76 s on lock 0x7b5400376254. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:49.922189 (+ 14583us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4049895296}
I20260709 12:11:49.963464 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:49.899571 17354 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:49.947489 17315 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:49.967474 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:49.965090 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:49.972318 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2265) took 1790 ms. Trace:
I20260709 12:11:49.972532 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:49.973733 12531 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 12: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:49.972885 12523 rpcz_store.cc:276] 0709 12:11:48.182349 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:48.183658 (+  1309us) service_pool.cc:224] Handling call
0709 12:11:48.191325 (+  7667us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:48.191562 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:48.191588 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:48.191631 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:48.191675 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:48.191755 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:49.957755 (+1766000us) spinlock_profiling.cc:243] Waited 7.09 ms on lock 0x7b5400408650. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:49.972034 (+ 14279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":16312576}
W20260709 12:11:49.976228 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 216300us
I20260709 12:11:49.970181 17315 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:49.947376 17508 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:49.976893 17534 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.012s
I20260709 12:11:49.994529 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:50.001351 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:50.007337 12524 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:11:49.982928 17499 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:50.012075 17430 consensus_meta.cc:287] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.520s	user 0.011s	sys 0.009s
I20260709 12:11:50.014189 17430 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:50.015969 17557 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.460s	user 0.002s	sys 0.002s
I20260709 12:11:50.017050 17392 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:50.017590 17392 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:50.017706 17457 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.014205 17333 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:50.023281 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:50.023991 17440 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:50.023501 12603 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
W20260709 12:11:50.028053 17533 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.618s	user 0.000s	sys 0.017s
I20260709 12:11:50.032253 17241 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:50.033777 17288 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:50.034217 17288 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:50.028609 17533 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.034941 17532 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:50.037552 12675 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:50.040222 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 503) took 1385 ms. Trace:
I20260709 12:11:50.042318 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:50.036849 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:50.040803 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:11:50.041584 12677 consensus_meta.cc:287] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.565s	user 0.008s	sys 0.015s
I20260709 12:11:50.047525 12674 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 12: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:50.036392 17440 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:50.040680 12527 rpcz_store.cc:276] 0709 12:11:48.655030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:48.655214 (+   184us) service_pool.cc:224] Handling call
0709 12:11:48.655695 (+   481us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:49.648714 (+993019us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:50.039894 (+391180us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b5400015c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:50.040145 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2279449344}
W20260709 12:11:50.050691 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 93461.9us
I20260709 12:11:50.038029 17430 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.053361 12676 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:11:50.054212 12677 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:11:50.035504 17288 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.057075 17298 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:50.057437 12673 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:50.058072 17426 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:50.059587 17244 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.058956 17447 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:50.060716 17392 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.062054 17499 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:50.062337 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:50.048954 17457 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:50.063882 17508 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.064693 12678 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 14: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:50.034248 12526 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:50.049934 17270 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:11:50.069571 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:50.063292 16211 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:50.075302 12526 raft_consensus.cc:606] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:50.074704 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:50.079769 12526 raft_consensus.cc:614] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:50.081028 16211 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:50.083812 12571 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:50.086243 17466 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:50.086963 17466 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:50.087347 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:50.088575 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:50.092330 17497 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:50.094339 17241 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:50.094481 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:50.095229 17241 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.095608 13337 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:50.086292 17498 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:50.066838 17298 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:50.087399 17466 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:50.098598 17425 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:50.086436 17500 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:50.109931 17533 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.111261 13332 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
W20260709 12:11:50.121269 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:50.077363 17553 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.124501 13330 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:50.135277 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:50.136227 12530 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 13: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:50.136742 12597 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260709 12:11:50.137319 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:11:50.137754 17558 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.579s	user 0.000s	sys 0.002s
I20260709 12:11:50.117225 17425 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:50.138785 17425 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:50.139755 16935 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:50.141114 17570 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.267s	user 0.000s	sys 0.002s
I20260709 12:11:50.142704 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:50.143509 13332 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:50.144899 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:50.146586 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:50.147147 12529 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:50.076297 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:50.137701 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2406) took 1236 ms. Trace:
I20260709 12:11:50.148537 12598 rpcz_store.cc:276] 0709 12:11:48.866140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:48.871728 (+  5588us) service_pool.cc:224] Handling call
0709 12:11:50.101843 (+1230115us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b54002710d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:50.102079 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1453129600}
I20260709 12:11:50.158579 17430 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:11:50.159122 17569 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.175s	user 0.001s	sys 0.001s
I20260709 12:11:50.159471 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:50.159327 17430 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.166286 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:50.104697 17298 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:50.168224 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:50.169211 12676 raft_consensus.cc:2410] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:11:50.174835 12675 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:11:50.168992 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 468) took 2386 ms. Trace:
I20260709 12:11:50.168164 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:50.177173 17440 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.189180 12677 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 12: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:50.189942 12529 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:11:50.176712 12672 rpcz_store.cc:276] 0709 12:11:47.760678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:47.893335 (+132657us) service_pool.cc:224] Handling call
0709 12:11:47.893660 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:49.849097 (+1955437us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:50.146871 (+297774us) spinlock_profiling.cc:243] Waited 1.87 s on lock 0x7b540015d6d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:50.147085 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4312101248}
I20260709 12:11:50.111246 17241 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:50.193904 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 197277us
I20260709 12:11:50.194949 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:50.195896 12530 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:11:50.195796 16935 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:50.202286 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:50.205715 12601 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: [13.6-13.6]
W20260709 12:11:50.210613 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 120688us
I20260709 12:11:50.215896 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:50.213901 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:50.202663 17392 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:50.223659 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:11:50.210613 17572 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.196s	user 0.002s	sys 0.000s
I20260709 12:11:50.221251 16211 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.228891 16935 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:50.205338 17430 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.249607 17532 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:50.266495 17532 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:50.234946 12672 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:11:50.239125 12679 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:11:50.242964 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 506) took 1376 ms. Trace:
I20260709 12:11:50.243973 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:50.247239 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2555) took 1047 ms. Trace:
I20260709 12:11:50.249872 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:50.250628 17333 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:50.211156 17270 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:50.278379 17270 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:50.281185 17270 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:50.282004 12677 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:11:50.284518 13304 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:50.252326 17534 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:50.188656 17457 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.286595 17534 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.252817 17376 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:50.254742 13332 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 11: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:11:50.262003 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:50.263458 17543 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:50.225605 17553 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:50.251314 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:50.228109 17288 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:50.277009 17333 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:50.277393 17447 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.287135 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:50.277455 12602 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 12: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:11:50.313244 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:50.316622 16211 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.316880 12678 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:11:50.269197 12597 rpcz_store.cc:276] 0709 12:11:49.200329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:49.419178 (+218849us) service_pool.cc:224] Handling call
0709 12:11:50.111420 (+692242us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:50.137188 (+ 25768us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:50.246976 (+109788us) spinlock_profiling.cc:243] Waited 683 ms on lock 0x7b54002710d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:50.247172 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1570082688}
W20260709 12:11:50.303414 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 73913.9us
W20260709 12:11:50.264472 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:50.268607 12528 rpcz_store.cc:276] 0709 12:11:48.866443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:48.866576 (+   133us) service_pool.cc:224] Handling call
0709 12:11:48.866898 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:49.834582 (+967684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:50.242558 (+407976us) spinlock_profiling.cc:243] Waited 842 ms on lock 0x7b5400105ed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:50.242872 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1937469824}
I20260709 12:11:50.302407 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:50.312973 12604 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:11:50.313264 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:50.319927 12604 pending_rounds.cc:85] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 6
I20260709 12:11:50.324415 12604 pending_rounds.cc:107] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:11:50.324852 12604 raft_consensus.cc:2889] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:50.328541 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:50.324936 17426 raft_consensus.cc:2764] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:11:50.329530 12605 raft_consensus.cc:2410] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 14 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:11:50.291941 17430 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260709 12:11:50.339433 17449 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:50.341208 12680 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:50.346410 17418 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:50.346855 17326 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:50.347574 17553 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:50.349604 17326 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:50.350381 17519 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:50.351058 17519 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.352002 17534 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.352362 17376 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.354570 17376 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.355046 13330 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
W20260709 12:11:50.356868 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 70280.2us
I20260709 12:11:50.357290 17426 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:50.358747 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:50.358850 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:11:50.359861 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 60012.2us
I20260709 12:11:50.360417 17425 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.361652 12572 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:50.362615 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:50.350052 17553 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.363512 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:50.364357 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:50.365053 17411 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:50.365605 17244 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:50.367188 17322 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:11:50.368434 12673 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 12. Ops: []
W20260709 12:11:50.352654 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 54005.1us
W20260709 12:11:50.362615 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 152431us
W20260709 12:11:50.369696 17575 log.cc:927] Time spent T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.240s	user 0.000s	sys 0.003s
I20260709 12:11:50.296978 17543 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:50.369952 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:50.359879 12569 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:50.379673 13337 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:11:50.381971 12680 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:50.361799 12675 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 7 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
W20260709 12:11:50.391867 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 44050.9us
W20260709 12:11:50.409219 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 158010us
I20260709 12:11:50.411183 17270 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:50.401372 17315 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:50.417151 17315 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:50.419334 12571 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:50.389638 17244 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:50.427721 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 67970.2us
I20260709 12:11:50.433535 16211 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:50.402225 17430 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:50.434500 12569 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:50.442426 16211 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:50.406014 17322 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:50.370404 17426 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.409865 17449 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:50.448028 12571 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:50.446995 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:50.434397 17244 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:50.454025 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 55424.7us
I20260709 12:11:50.458017 13330 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:50.400671 17298 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:50.441470 17533 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:50.468068 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 218378us
I20260709 12:11:50.476176 17426 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:50.472607 17397 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.012s
W20260709 12:11:50.482785 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 121243us
I20260709 12:11:50.484926 17543 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.487200 17322 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:50.488044 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47263.6us
I20260709 12:11:50.467589 17376 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
W20260709 12:11:50.488971 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 83783.1us
I20260709 12:11:50.436434 17534 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:11:50.489632 17466 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.490592 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2567) took 1177 ms. Trace:
I20260709 12:11:50.534374 12600 rpcz_store.cc:276] 0709 12:11:49.313395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:49.989202 (+675807us) service_pool.cc:224] Handling call
0709 12:11:50.000448 (+ 11246us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:50.039148 (+ 38700us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:11:50.052769 (+ 13621us) log.cc:844] Serialized 14 byte log entry
0709 12:11:50.225108 (+172339us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:50.321712 (+ 96604us) log.cc:844] Serialized 26 byte log entry
0709 12:11:50.331799 (+ 10087us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:50.331835 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:50.331915 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:50.490199 (+158284us) raft_consensus.cc:1685] finished
0709 12:11:50.490222 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:50.490387 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12532,"prepare.queue_time_us":10914,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":7207,"raft.queue_time_us":44964,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":1104,"thread_start_us":108639,"threads_started":1,"wal-append.queue_time_us":333565}
I20260709 12:11:50.535961 17466 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.537041 17466 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:50.472978 12494 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:50.538364 17532 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.498999 17584 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:50.539525 12527 raft_consensus.cc:2410] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 13 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:11:50.499858 12643 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:50.540467 17426 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:50.540956 17532 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.541033 17426 raft_consensus.cc:697] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:50.541786 17532 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:50.546487 17532 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.547788 17222 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:50.549180 17425 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.494249 16211 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:50.472944 16935 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:50.506350 17582 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:50.514436 17533 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.515316 13330 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:11:50.467857 17411 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:50.519471 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:50.525473 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:50.490619 17449 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:50.538599 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 81752us
I20260709 12:11:50.542222 17457 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.543859 17534 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:50.575508 17534 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:50.575970 17534 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:50.579725 17288 raft_consensus.cc:3055] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:50.580214 17288 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:50.581254 17288 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:50.544749 17497 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:11:50.524760 17507 consensus_meta.cc:287] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.646s	user 0.014s	sys 0.004s
W20260709 12:11:50.522063 17372 consensus_meta.cc:287] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.769s	user 0.012s	sys 0.002s
W20260709 12:11:50.558842 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:50.577876 17519 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.553828 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:50.542263 17426 consensus_queue.cc:237] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.592670 17376 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:50.593256 17241 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:50.595506 12494 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:50.500519 12646 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:50.600978 12569 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:50.537988 17466 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:50.539724 17270 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:50.553561 17543 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:50.505277 17244 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:50.626083 17440 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:50.632673 17440 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:50.631417 12673 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:50.632004 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:50.635964 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 106737us
I20260709 12:11:50.646571 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:50.607545 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:50.654410 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:50.563416 17582 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:50.650341 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:50.629063 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:50.611253 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:50.646479 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:50.647595 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:50.656975 13330 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
W20260709 12:11:50.646610 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 79040.8us
W20260709 12:11:50.640645 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 140232us
I20260709 12:11:50.657956 12672 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 11: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:50.647964 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:50.629541 17580 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:11:50.660832 17580 consensus_queue.cc:1243] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:11:50.629848 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 159251us
W20260709 12:11:50.663378 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 68001us
I20260709 12:11:50.568504 17333 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:50.667492 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:50.599673 17392 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:11:50.549265 17532 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.610993 17315 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:11:50.694434 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 65480.5us
I20260709 12:11:50.629127 17376 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:50.697526 17411 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:50.633776 17440 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:50.698899 17447 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:11:50.628954 17449 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:50.630020 17298 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:50.640751 17241 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:50.704695 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:50.634061 17244 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.705739 17550 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:50.707325 12525 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:11:50.655586 17582 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:50.710253 17498 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:50.718669 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:50.718866 12675 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:50.721261 12677 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
W20260709 12:11:50.723709 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 60017.7us
I20260709 12:11:50.737764 12496 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:50.635854 17553 raft_consensus.cc:2764] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:11:50.634603 17288 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:50.727741 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:50.728142 12572 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:50.606178 17507 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.728917 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:50.714008 17500 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:50.746731 12600 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:50.729789 12569 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:50.609427 17372 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.765283 12572 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:50.769107 12646 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:50.770685 17270 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:50.771407 17270 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:50.772161 17270 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:50.772634 17508 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:50.773119 17508 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:50.774564 17519 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:50.775610 17533 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.775053 17519 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:50.776171 17498 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:50.776189 17519 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:50.778950 17500 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:50.779425 17500 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:11:50.779691 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52169.3us
I20260709 12:11:50.780563 12571 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:50.781420 16935 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:50.782394 17457 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:50.782650 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:50.783008 17457 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:50.783551 12597 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:11:50.784101 17457 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.782049 16935 raft_consensus.cc:697] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:50.786556 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:50.657330 17397 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:50.722070 12671 raft_consensus.cc:2376] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13. Re-sending same reply.
I20260709 12:11:50.673449 17222 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:50.694866 17418 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:50.761202 12605 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:50.761969 17241 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:50.763567 17333 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.763810 17580 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:50.765252 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2570) took 1082 ms. Trace:
I20260709 12:11:50.765358 17376 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:50.766580 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:50.630692 17466 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.766494 17589 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:11:50.763610 17507 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.793360 17322 raft_consensus.cc:3055] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:50.777850 12496 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:50.729429 17550 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:50.787513 16935 consensus_queue.cc:237] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.858868 17550 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:50.846779 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:50.765921 12598 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:50.813814 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:11:50.863118 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:50.861490 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:50.866223 17589 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:50.872887 17315 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:50.770730 17372 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:50.881961 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:50.810585 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:50.810245 12675 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:11:50.815634 17533 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:50.807166 17584 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:50.807464 12606 rpcz_store.cc:276] 0709 12:11:49.682818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:50.049563 (+366745us) service_pool.cc:224] Handling call
0709 12:11:50.075824 (+ 26261us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:50.227060 (+151236us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:50.227108 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:50.270196 (+ 43088us) log.cc:844] Serialized 26 byte log entry
0709 12:11:50.607614 (+337418us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:50.607665 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:50.607747 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:50.764916 (+157169us) raft_consensus.cc:1685] finished
0709 12:11:50.764937 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:50.765066 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14336,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":111,"thread_start_us":324550,"threads_started":2,"wal-append.queue_time_us":417720}
I20260709 12:11:50.816946 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:50.875466 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 97687.6us
I20260709 12:11:50.878733 12597 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:11:50.794466 17222 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:50.803061 17392 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:50.797178 17418 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:50.886682 12604 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:50.847527 17241 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
W20260709 12:11:50.888200 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 71354.7us
I20260709 12:11:50.797406 17298 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:50.876173 17322 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:50.883095 17315 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:50.855327 17498 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:50.847597 17553 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:11:50.886341 17376 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:50.863466 17397 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:11:50.898644 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:50.900333 13305 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:50.888520 17222 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:50.891297 17340 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:11:50.905890 17508 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:50.906219 17244 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:11:50.900717 17533 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:50.936120 17533 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:50.936584 17533 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:50.936388 17322 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:50.908306 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:50.908895 12428 catalog_manager.cc:5697] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 7 to 8, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 8 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:50.910534 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:50.890753 17418 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:50.942586 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:50.918156 12603 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:50.925312 17589 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:50.927026 17553 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:11:50.920531 12601 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:11:50.932495 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:50.933481 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:50.901065 17392 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:50.916848 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 147827us
W20260709 12:11:50.956166 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 74281.3us
W20260709 12:11:50.946828 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 158187us
W20260709 12:11:50.942893 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 57137.1us
I20260709 12:11:50.949343 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:50.945957 12674 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:11:50.938069 17322 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:50.939100 17508 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:50.978771 12597 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 8.
I20260709 12:11:50.979527 12572 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:50.981613 17354 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:50.982672 12646 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:50.988386 17584 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:50.990551 17603 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:50.993175 17392 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:50.989584 12494 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:50.999085 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:51.012861 12671 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
W20260709 12:11:50.997591 17598 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:11:51.006433 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:51.015511 12677 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
W20260709 12:11:51.009276 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 154503us
I20260709 12:11:50.987653 17222 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:51.016940 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 55002.3us
I20260709 12:11:51.001322 17580 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:51.025290 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:51.025768 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:51.027424 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:51.028115 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 48678.3us
I20260709 12:11:51.028312 12675 raft_consensus.cc:2368] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12.
I20260709 12:11:51.025101 13304 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:51.031718 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:51.036042 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:51.011811 17532 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:51.038693 13304 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:51.038841 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:51.039429 17553 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:51.010419 17354 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.027041 17498 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.016s
I20260709 12:11:51.030083 17497 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:51.048974 17500 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.022585 17466 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:51.050084 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:51.050645 17544 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:11:51.051039 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:51.051772 12597 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:51.113278 13337 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:11:51.052615 12428 catalog_manager.cc:5697] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 4 to 8, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 8 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:51.053947 17397 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:11:51.050668 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 68076.9us
I20260709 12:11:51.054863 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:51.060496 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:51.035605 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:51.065317 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:51.066473 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:51.068845 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:51.100430 12572 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:51.104664 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:51.117201 12606 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:50.998142 17589 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.045835 17519 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:51.123241 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:51.126344 12675 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:51.033668 17603 raft_consensus.cc:3055] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:51.061698 17532 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:51.095314 17553 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:51.134641 12605 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:51.108004 17298 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:51.014279 17606 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.100s
I20260709 12:11:51.143309 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:51.143956 12675 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:51.123212 12428 catalog_manager.cc:5697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:51.138592 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:51.144356 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:51.134840 12603 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:51.142002 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:51.050045 17270 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.132869 17603 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:51.121793 17397 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:51.148188 12494 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:51.153704 17298 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.139680 17532 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:51.033881 17580 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:51.141074 17553 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:51.140647 17593 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:51.179800 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:51.184501 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:11:51.185689 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 120661us
I20260709 12:11:51.183434 12428 catalog_manager.cc:5697] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 6 to 7, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 7 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:51.156575 17497 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:51.157131 17397 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:51.195574 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 60866.9us
I20260709 12:11:51.197530 17519 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:51.200062 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:51.222455 13332 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:51.195245 17603 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.223754 17603 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:51.223526 17533 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.200472 17612 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:51.226406 17612 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:51.226583 17497 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:51.227133 17497 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:51.227348 17612 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.228734 17612 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:51.201511 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 56214.3us
I20260709 12:11:51.202241 17606 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:51.210742 17499 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:51.150329 17376 consensus_queue.cc:1059] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.066s
I20260709 12:11:51.232837 17499 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:51.213516 17604 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:11:51.190930 17610 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.013s
I20260709 12:11:51.121201 17466 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:51.195202 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 159831us
I20260709 12:11:51.195261 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:51.191947 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:51.136234 17418 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:51.248550 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 148721us
I20260709 12:11:51.254007 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:51.234319 12428 catalog_manager.cc:5697] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 7 to 10, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 10 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:51.266673 12598 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
W20260709 12:11:51.271627 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 49876.8us
I20260709 12:11:51.272789 13332 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:51.277325 12676 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:51.300853 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:51.191744 17244 raft_consensus.cc:2764] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 12:11:51.222469 17298 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:51.226027 17533 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:51.313648 12674 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 14: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:51.310973 12598 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: []
W20260709 12:11:51.305042 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49402.1us
I20260709 12:11:51.322804 13337 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:11:51.322965 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:51.331511 12602 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:51.326416 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:51.333069 12494 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:51.333379 12428 catalog_manager.cc:5697] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:51.276113 17411 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:11:51.343087 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 156440us
I20260709 12:11:51.345494 12428 catalog_manager.cc:5697] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:11:51.328621 17582 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.521s	user 0.009s	sys 0.001s
I20260709 12:11:51.302311 17606 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:51.350844 12646 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:51.337226 12675 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
W20260709 12:11:51.359637 17609 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 12:11:51.353744 12676 pending_rounds.cc:85] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 6
I20260709 12:11:51.361058 12676 pending_rounds.cc:107] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:11:51.361373 12676 raft_consensus.cc:2889] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:51.363389 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:51.364035 12674 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:51.332109 17498 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.031s
I20260709 12:11:51.348189 17244 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:51.354105 17519 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:51.354837 17610 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:51.362452 17544 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:51.355815 17222 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:51.391944 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 46601us
W20260709 12:11:51.409225 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:51.416782 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 162853us
I20260709 12:11:51.359972 17270 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:51.438336 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 106971us
I20260709 12:11:51.355582 17582 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:51.371789 17298 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:51.439224 12496 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:51.371165 17534 consensus_meta.cc:287] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.795s	user 0.014s	sys 0.000s
I20260709 12:11:51.454052 12677 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:11:51.351926 17616 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:51.460592 12600 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
W20260709 12:11:51.464236 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:51.413162 17610 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:51.465065 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:51.400038 12428 catalog_manager.cc:5697] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:51.367007 17315 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.472236 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:51.476857 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:51.436764 17544 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:51.437589 17500 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:51.472833 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 127358us
I20260709 12:11:51.391268 17430 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 12:11:51.371354 17606 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.492059 17606 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:51.400652 17244 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:51.496946 12604 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:51.485452 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:51.498407 12597 raft_consensus.cc:2368] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10.
I20260709 12:11:51.488426 17197 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.156s
W20260709 12:11:51.489804 17550 consensus_meta.cc:287] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.630s	user 0.001s	sys 0.014s
I20260709 12:11:51.493602 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:51.467607 17610 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:51.431069 17411 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.507443 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:51.505753 17606 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:11:51.518076 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 78950.2us
I20260709 12:11:51.465888 17534 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:51.523607 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:51.525164 12428 catalog_manager.cc:5697] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 4 to 6, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 6 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:51.527451 12428 catalog_manager.cc:5697] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:51.494328 17507 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:51.529778 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:51.516522 17543 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.878s	user 0.013s	sys 0.000s
W20260709 12:11:51.547175 17618 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:11:51.549281 12646 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:51.550078 17298 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:51.550978 17426 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:11:51.552896 17619 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:11:51.553579 17466 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:51.554579 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:51.632311 12598 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:51.558724 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:51.633049 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:51.559640 17544 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:51.561662 17582 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:51.571487 12526 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 12: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:51.571957 17534 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:51.522879 17418 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:51.586417 12603 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:51.572000 17498 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:51.588387 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:51.517155 17372 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:51.589183 17620 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:11:51.590361 12428 catalog_manager.cc:5697] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 7 to 10, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 10 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:51.594518 17626 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:51.594699 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 75809us
I20260709 12:11:51.594817 17508 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:51.594524 17603 raft_consensus.cc:3055] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:51.598397 17270 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:51.596678 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:51.658324 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:51.658427 17270 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:51.574018 17430 raft_consensus.cc:3055] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:51.659551 17430 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:51.659452 17532 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.660686 17430 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.600581 17315 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:51.649348 17222 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:51.597733 12680 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:11:51.602283 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:51.610553 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:51.663491 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 53081.4us
I20260709 12:11:51.663697 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:51.664044 17411 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:51.664184 12571 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:51.664909 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:51.665285 12571 leader_election.cc:400] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12.
I20260709 12:11:51.664803 12428 catalog_manager.cc:5697] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:51.612282 12674 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:11:51.594736 17631 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:11:51.627105 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 293150us
I20260709 12:11:51.666486 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:51.666954 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:51.506310 17550 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.661801 17430 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:51.667963 17582 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:51.668383 17582 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:51.668433 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:51.668792 17582 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:51.669123 17550 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:51.669631 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:51.644340 17498 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.661804 17532 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:51.674546 17498 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:51.658684 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:51.644621 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:51.637663 17397 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.680565 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:51.681401 17397 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:51.647687 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:51.681782 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:51.682500 12679 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 12: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:51.650466 12496 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:51.682963 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:51.656482 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:51.657747 12530 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:11:51.683859 12524 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:11:51.684604 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:51.684823 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:51.604007 17497 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:51.686663 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:51.687754 17497 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:51.688024 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:51.688383 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:51.688709 12675 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:51.652901 12602 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:51.666169 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:51.649013 12643 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.525s (SENT) [suppressed 14 similar messages]
I20260709 12:11:51.533349 12606 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 11: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:51.690474 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:51.690913 12606 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:51.676224 12527 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:51.692538 12606 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 12:11:51.666625 12428 catalog_manager.cc:5697] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 1 to 10. New cstate: current_term: 10 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:51.658393 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:51.603729 17244 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:51.670080 17315 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:51.693722 12428 catalog_manager.cc:5697] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 6 to 9, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 9 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:51.694646 17315 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:51.695187 17315 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:51.666699 17631 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:51.695706 12428 catalog_manager.cc:5697] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 7 to 9, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 9 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:11:51.670080 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 120964us
I20260709 12:11:51.697670 17631 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.667339 12529 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 12: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:51.672286 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:51.672688 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:51.651280 17603 raft_consensus.cc:740] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:51.641206 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:51.646358 17615 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:11:51.643743 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:51.698860 12646 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:51.667410 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:51.682397 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:51.631064 17322 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:51.682879 17372 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.688887 12605 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 11: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:51.661082 17270 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:51.677484 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:51.698364 12428 catalog_manager.cc:5697] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 5 to 7, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 7 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
W20260709 12:11:51.730365 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 79980.1us
I20260709 12:11:51.759217 12527 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
W20260709 12:11:51.730376 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 63036.8us
I20260709 12:11:51.676254 17626 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.760452 12606 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:51.735193 17322 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:51.727775 17507 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.733784 17497 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:51.744525 17288 raft_consensus.cc:3055] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:51.701189 17603 consensus_queue.cc:260] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.630676 17553 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:51.765391 17610 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:51.732020 17333 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:51.759402 17241 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:51.767494 17411 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:51.786803 17533 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:11:51.787690 17612 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.559s	user 0.009s	sys 0.010s
I20260709 12:11:51.788127 17533 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:51.788614 17533 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:51.788838 17603 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:51.789129 17372 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:51.790320 12528 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:11:51.791379 17507 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:51.792083 17333 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:51.789523 17533 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:51.771209 17550 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:51.771885 17498 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:51.794631 17498 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:51.795325 17498 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:51.795764 12525 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 14: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:51.797178 17610 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:51.778568 17288 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:51.797966 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:51.799139 17288 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:51.800206 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:51.769970 17626 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:51.759292 17532 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:51.814879 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:51.816012 12571 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:51.760696 12494 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:51.817373 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 56801.3us
I20260709 12:11:51.817492 12530 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:11:51.778676 17222 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:51.699185 17631 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:51.784256 12605 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:51.785301 12496 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:51.786863 17411 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.921326 17411 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:51.769011 12522 raft_consensus.cc:2410] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 15 because replica has last-logged OpId of term: 13 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 5.
I20260709 12:11:51.921625 17354 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:51.792541 17333 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:51.755010 12428 catalog_manager.cc:5697] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 7 to 10, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 10 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:11:51.776391 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 113463us
I20260709 12:11:51.925892 12678 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:11:51.816170 12525 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:51.777523 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:51.844184 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:51.819741 17532 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.845000 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:51.930526 12677 raft_consensus.cc:2368] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12.
I20260709 12:11:51.846385 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:51.930944 12597 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:11:51.931206 12679 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:51.780072 17497 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.759897 17582 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.790057 17322 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:51.866700 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 48670.8us
I20260709 12:11:51.825814 17288 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:51.821982 17626 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:51.822343 17500 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:51.878018 17553 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:51.823750 17315 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.917788 12680 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:51.825107 12674 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:11:51.780536 17584 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.045s
I20260709 12:11:51.927350 17222 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.954852 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:51.822029 17466 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 4, Time since last communication: 0.032s
I20260709 12:11:51.822708 12643 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:51.794099 17533 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:51.926855 12671 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:51.927673 12428 catalog_manager.cc:5697] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 7 to 8, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 8 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:11:51.928494 13302 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:51.925776 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:51.931654 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 480) took 1150 ms. Trace:
W20260709 12:11:51.932642 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 147408us
I20260709 12:11:51.962646 12600 rpcz_store.cc:276] 0709 12:11:50.781627 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:50.786226 (+  4599us) service_pool.cc:224] Handling call
0709 12:11:51.931505 (+1145279us) spinlock_profiling.cc:243] Waited 673 ms on lock 0x7b5400374e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:51.931595 (+    90us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1548126976}
I20260709 12:11:51.963346 12496 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:51.933976 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 156559us
I20260709 12:11:51.926541 12569 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:51.964937 12572 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:51.939159 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:51.941255 17610 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:51.966259 12530 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:51.966466 17626 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:51.966751 12598 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 13: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:51.967420 12428 catalog_manager.cc:5697] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 5 to 7. New cstate: current_term: 7 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:51.967607 17626 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:51.968144 12673 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:11:51.945993 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:51.968868 12428 catalog_manager.cc:5697] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:51.937359 17354 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:11:51.949854 12599 raft_consensus.cc:2368] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10.
W20260709 12:11:51.961357 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 262613us
I20260709 12:11:51.973235 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:51.975481 12679 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:51.977669 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:11:51.979583 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 156984us
I20260709 12:11:51.978631 12428 catalog_manager.cc:5697] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 6 to 9, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 9 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:51.984441 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:51.985110 12601 raft_consensus.cc:2368] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 10. Current term is 11.
I20260709 12:11:51.985272 12600 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:51.989651 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:51.990026 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:51.954985 17222 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:51.974480 12494 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:51.992997 12646 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:51.996523 17582 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:51.996832 17241 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:51.977613 17500 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:51.998270 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:51.998974 12522 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:51.998572 17500 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:52.000241 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:51.973549 17550 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.033s
I20260709 12:11:52.003175 17612 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:52.003573 17612 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:52.004503 17612 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:52.005851 17612 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:52.007236 17333 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:52.009521 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:52.010154 17632 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:51.974911 17553 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:52.075984 12525 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:11:52.011438 16935 raft_consensus.cc:3055] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:52.020820 12524 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:52.017434 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:51.997364 12428 catalog_manager.cc:5697] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 5 to 9. New cstate: current_term: 9 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:11:52.021721 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:52.019016 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:11:52.029443 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:52.095496 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:52.011400 17497 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:51.981477 17411 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:52.075940 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:52.098913 12597 raft_consensus.cc:2368] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13.
I20260709 12:11:52.077026 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:52.099697 13339 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 12: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:52.080509 17499 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:52.082880 17589 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:51.984036 17593 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:11:52.107462 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:52.108208 12598 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:11:52.014099 17610 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:52.097735 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:52.034787 17288 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:52.103407 17333 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:51.994078 17354 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:11:52.098233 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:52.109537 12525 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:52.118893 12526 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:11:52.110154 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:52.076236 17397 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:51.994259 17298 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.115s
I20260709 12:11:52.119315 12526 pending_rounds.cc:85] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 7
I20260709 12:11:52.116259 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:52.126421 12526 pending_rounds.cc:107] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:11:52.126735 12526 raft_consensus.cc:2889] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:52.123456 12528 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:52.120776 12428 catalog_manager.cc:5697] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 1 to 7, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 7 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:52.127148 12522 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:51.989096 17426 consensus_queue.cc:1059] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.122s
I20260709 12:11:52.099572 17632 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:11:52.124694 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 105799us
I20260709 12:11:52.125011 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:52.089527 16935 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:52.055747 12530 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:11:52.111495 17497 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:51.997408 17241 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:52.128938 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 144528us
I20260709 12:11:52.136539 12428 catalog_manager.cc:5697] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:11:52.075848 17430 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.047s
I20260709 12:11:52.139093 12671 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:52.115593 17508 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.144714 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:52.146617 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:52.152325 12679 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:52.148175 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:52.150455 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:52.139650 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 158301us
I20260709 12:11:52.157884 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:52.158326 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:52.134744 17507 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:52.162436 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:52.130082 17354 raft_consensus.cc:697] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:52.164538 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:52.190399 13304 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:52.165313 12494 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:52.193874 12494 leader_election.cc:400] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13.
I20260709 12:11:52.194335 12494 leader_election.cc:403] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:11:52.195083 12525 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
I20260709 12:11:52.197266 12600 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
I20260709 12:11:52.140115 17397 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:52.204428 12675 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 12: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:52.207401 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:52.212062 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:52.212131 12679 raft_consensus.cc:2410] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 11 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 6.
I20260709 12:11:52.161410 17553 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.222182 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:52.223291 12679 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 12: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:52.194619 17392 raft_consensus.cc:3055] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:52.231617 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:52.161000 17508 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:52.204947 17430 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:52.238615 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:52.264823 12675 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 12: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:52.244585 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:52.261260 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 49279.6us
I20260709 12:11:52.237921 17376 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.262090 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:52.250486 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:52.237114 12602 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:52.269281 13335 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:52.272150 12680 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:52.273465 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:52.258622 17632 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:52.258373 17372 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:52.282367 17298 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 10.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.014s
I20260709 12:11:52.283242 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:52.284111 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:52.285050 12672 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:11:52.268581 17649 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:52.247371 17634 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.316s	user 0.004s	sys 0.000s
I20260709 12:11:52.288249 13335 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:11:52.288928 17498 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:52.292147 17241 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:52.238066 17413 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:52.296552 17508 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:52.227357 17507 raft_consensus.cc:697] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:52.239066 17397 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:52.224700 16935 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:52.306169 17315 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:52.254020 17641 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:11:52.231863 17392 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:52.259711 17430 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:52.233613 17553 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:52.253078 17639 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.165s	user 0.000s	sys 0.001s
I20260709 12:11:52.252709 17411 raft_consensus.cc:3055] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:52.368016 17411 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:11:52.252640 17636 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:11:52.308650 12531 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 7. (term mismatch)
I20260709 12:11:52.311666 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:52.370920 12531 pending_rounds.cc:85] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 6
W20260709 12:11:52.371299 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 59696.6us
I20260709 12:11:52.372094 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:52.315573 12604 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:52.319069 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:52.336212 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:52.373682 12676 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
W20260709 12:11:52.314599 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 388142us
I20260709 12:11:52.377097 12496 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:52.321717 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:52.323319 13302 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:52.378647 12679 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:52.325598 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:11:52.327201 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 362366us
I20260709 12:11:52.371346 12531 pending_rounds.cc:107] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 12.7
I20260709 12:11:52.379690 12531 raft_consensus.cc:2889] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:52.333601 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:52.317830 17376 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:52.357993 12599 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:52.379347 12672 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:52.382272 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:52.385733 12676 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
I20260709 12:11:52.388046 17392 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:52.392127 17553 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:52.394794 12673 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 14: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:52.395002 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:52.398983 17315 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:52.400897 17315 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:52.401692 17626 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:52.402688 17508 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 10.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:11:52.403525 17288 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.404032 17288 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:52.404433 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:52.405843 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:52.406327 17631 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.407094 17631 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:52.373339 17619 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:11:52.414399 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:52.415277 12496 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:52.374936 17430 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:52.423169 17532 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:52.326848 17241 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.084s
I20260709 12:11:52.425160 17507 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:52.374385 17544 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.085s
I20260709 12:11:52.419868 17644 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:52.369287 17411 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:52.488935 12677 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:11:52.517865 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:52.237946 17354 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 9.10, Last appended by leader: 10, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:52.490366 12529 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:11:52.491526 12679 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 9.
W20260709 12:11:52.493084 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 77869.3us
I20260709 12:11:52.519922 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:52.501029 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 86780.3us
I20260709 12:11:52.521360 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:52.506922 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:52.522634 12524 raft_consensus.cc:2410] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:11:52.506346 12672 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:11:52.509493 12678 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: [10.6-10.6]
I20260709 12:11:52.496308 17413 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:11:52.411790 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 180631us
I20260709 12:11:52.495895 17533 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:52.530148 17645 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:52.534374 17411 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:52.536098 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:52.537297 12526 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 11: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:52.538587 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:52.539503 17498 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:52.540006 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:52.484609 17508 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:52.543432 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:52.545692 12572 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:52.549002 12496 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:52.552943 12496 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:52.554319 17457 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:52.553882 17532 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:52.562374 17646 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:52.551143 12494 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:52.563730 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:52.505035 17652 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:52.566574 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 193810us
I20260709 12:11:52.575908 13304 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:52.596915 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:52.598402 13335 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:52.598452 12572 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:52.599009 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 47962.8us
I20260709 12:11:52.558008 17372 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13.
I20260709 12:11:52.599865 17589 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:52.599581 12572 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:52.552139 17644 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:11:52.605837 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 282630us
I20260709 12:11:52.611214 12572 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:52.570977 17508 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:11:52.562417 17241 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:52.570428 17533 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:52.597021 17593 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:52.626368 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:52.627025 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:52.597980 17397 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:52.631212 12674 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:11:52.635444 17533 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:52.635860 17653 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:11:52.636811 17659 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:11:52.614356 17646 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:52.645011 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:52.647295 12676 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:52.663059 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:52.665877 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:52.625275 17241 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:52.668116 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:52.668995 12522 raft_consensus.cc:2410] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:11:52.633036 17397 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:52.675987 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:52.676697 12674 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:52.679553 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:52.680707 12675 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 11
I20260709 12:11:52.646674 17497 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.686401 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:52.687358 12525 raft_consensus.cc:2410] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 13 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:11:52.601105 17652 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:52.666193 17354 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:52.663902 17647 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:52.702988 17647 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:52.703156 17657 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:11:52.703544 17647 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:52.682399 17397 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:52.674243 17241 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:52.689100 17654 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:52.697495 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:52.691311 17497 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:52.709981 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:52.678735 17580 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:52.684873 17646 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:52.712146 12525 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:52.709786 12494 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:52.714376 13339 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:11:52.724562 12675 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:52.692057 17660 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:11:52.727435 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:52.727614 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:52.728154 12522 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 13: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:52.735215 12526 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:52.711760 17354 raft_consensus.cc:697] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:52.737323 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:52.716012 17532 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:52.751012 17652 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:52.752084 17392 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:52.756016 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2313) took 1010 ms. Trace:
I20260709 12:11:52.756505 12531 rpcz_store.cc:276] 0709 12:11:51.745683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:52.117380 (+371697us) service_pool.cc:224] Handling call
0709 12:11:52.122352 (+  4972us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:52.380162 (+257810us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:52.755715 (+375553us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54003f7d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:52.755944 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":415920384}
I20260709 12:11:52.759675 12671 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:11:52.760483 17497 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:52.724946 17519 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.763592 17519 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:52.766422 17631 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:52.767033 17354 consensus_queue.cc:237] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:52.724298 17626 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:52.770637 17626 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:52.771045 17626 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:52.771654 17663 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:11:52.772377 17646 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:52.738827 17644 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:52.742369 17660 raft_consensus.cc:697] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 9 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:52.776757 17660 consensus_queue.cc:237] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 9, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:52.780273 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:52.781742 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:52.782451 12522 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:11:52.783806 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:52.758028 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:52.715345 17550 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:52.789624 12531 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:11:52.803939 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:52.804857 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:52.805552 12522 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:52.802943 17550 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:11:52.807169 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 186340us
I20260709 12:11:52.806471 12571 leader_election.cc:400] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12.
I20260709 12:11:52.804651 12531 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:11:52.766104 17519 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:52.810928 12571 leader_election.cc:403] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:11:52.811690 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:52.812480 12600 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:52.812502 12646 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:52.816773 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:52.821591 12531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:52.824235 12531 raft_consensus.cc:606] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:52.824635 12531 raft_consensus.cc:614] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:52.821269 12571 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:52.826429 17645 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:52.827077 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:52.827939 13335 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
W20260709 12:11:52.819496 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 243692us
I20260709 12:11:52.820008 17582 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:11:52.807514 17550 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:52.832118 12571 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:52.944103 17593 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:52.947728 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 115710us
I20260709 12:11:52.937223 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:52.947926 17593 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:52.948721 12680 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:52.948844 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:52.926728 17288 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:52.949730 17270 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:52.893688 12529 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:11:52.934423 17397 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:52.950292 17270 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:52.950778 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:52.951123 12525 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 15: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:52.895193 12527 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:52.952332 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:52.944154 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:52.890105 17662 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.077s	user 0.000s	sys 0.001s
I20260709 12:11:52.953063 12529 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15.
I20260709 12:11:52.953171 12598 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:52.954108 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:52.881289 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:52.879732 17647 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:52.954880 17270 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:52.820963 17580 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.134s
I20260709 12:11:52.952702 17593 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:52.832758 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:52.879987 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:52.968858 12674 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:11:52.969274 12524 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:52.970448 12524 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: [11.9-11.9]
I20260709 12:11:52.934175 17533 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:52.970574 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:52.936921 17658 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.267s	user 0.000s	sys 0.000s
I20260709 12:11:52.971451 17647 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:52.864233 17644 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:11:52.879978 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:52.984732 12571 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:53.001197 12531 raft_consensus.cc:2393] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 12: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:52.834702 17645 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:52.828907 17532 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.136s
I20260709 12:11:52.821233 17333 consensus_queue.cc:1059] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.137s
I20260709 12:11:52.883222 13305 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:11:52.914655 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 66655.7us
I20260709 12:11:52.937304 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:53.011144 12569 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:53.012127 17322 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.012689 17322 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:52.956427 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 345327us
I20260709 12:11:53.013531 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:53.013850 12572 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:53.014962 17322 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:52.941020 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:52.941480 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:53.016677 12599 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:52.941893 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:52.960063 12643 leader_election.cc:336] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.712s (SENT)
I20260709 12:11:52.940689 12600 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:11:52.858508 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 46066.3us
I20260709 12:11:52.851531 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:52.858019 13304 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:52.962992 12494 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:53.019641 12646 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:53.020637 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 57740us
I20260709 12:11:53.020975 17519 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12.
I20260709 12:11:52.951489 12530 raft_consensus.cc:2368] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12.
W20260709 12:11:53.018033 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 58021.4us
I20260709 12:11:52.954025 12526 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:53.024703 12526 raft_consensus.cc:606] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:53.025311 12526 raft_consensus.cc:614] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:11:52.935477 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:52.955914 17397 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:52.971791 12524 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:11:53.024758 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:11:53.027391 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:52.928071 17626 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:53.027966 17508 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:53.028468 17508 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:53.028613 12529 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 13: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:53.028893 17508 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:53.029479 17626 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:52.852928 17649 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:53.030375 17649 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:53.031366 17649 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:53.032346 13339 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: [11.10-11.10]
I20260709 12:11:53.032528 17649 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:53.033614 12526 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:53.034324 17626 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:11:52.973788 12601 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:11:53.034905 12496 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:52.931075 17376 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:52.974912 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:52.978389 17270 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:52.982795 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:53.039036 17612 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:53.039057 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:53.039763 17612 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:53.040181 17612 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:53.042361 17376 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:53.042941 17376 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:53.035228 17322 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:52.983976 12598 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:52.997931 12525 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:53.044430 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:52.895104 17430 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:53.048640 17544 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:53.049146 17544 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:53.049542 17544 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:53.051110 17430 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:52.956250 16935 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:52.999660 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:52.909219 17637 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.792s	user 0.000s	sys 0.004s
I20260709 12:11:52.817514 17669 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:53.005087 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:53.053339 17669 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:53.054415 12528 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:53.056449 17669 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:53.007385 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2309) took 1270 ms. Trace:
I20260709 12:11:53.057040 12527 rpcz_store.cc:276] 0709 12:11:51.737384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:51.993955 (+256571us) service_pool.cc:224] Handling call
0709 12:11:52.076095 (+ 82140us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:52.952000 (+875905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:53.006985 (+ 54985us) spinlock_profiling.cc:243] Waited 816 ms on lock 0x7b5400408654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:53.007258 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1877764864}
W20260709 12:11:53.019717 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 161832us
I20260709 12:11:52.942869 12602 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:11:53.069067 17672 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:11:53.069178 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 49640.2us
I20260709 12:11:53.070461 17333 raft_consensus.cc:3055] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:53.070950 17333 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:53.071790 17333 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:53.073158 17333 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:53.070461 17498 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.078802 17498 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:53.076957 17612 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:53.044523 17589 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.072s
I20260709 12:11:53.082222 17612 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:53.034587 12526 raft_consensus.cc:606] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:53.043495 17376 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:52.986048 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:52.899397 17241 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:53.045850 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 162773us
I20260709 12:11:53.103924 12673 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:11:53.109229 12526 raft_consensus.cc:614] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:53.042812 12494 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:53.037622 17270 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:53.083151 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 58546.3us
I20260709 12:11:52.981397 17671 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:53.086915 12679 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 14: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:53.115397 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:52.828912 17668 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.118980 12677 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 12: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:11:53.088884 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 77879.5us
W20260709 12:11:53.089263 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 75493.4us
I20260709 12:11:53.120854 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:53.121687 12675 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 10 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:11:53.071424 12646 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:53.005663 17457 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
W20260709 12:11:53.114297 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 71596.3us
I20260709 12:11:53.009244 17646 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:53.097224 17411 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:53.116037 12673 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:53.096946 17647 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:53.122772 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:53.142740 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:53.144122 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:53.144824 12675 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
I20260709 12:11:53.123608 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:53.147207 12671 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
W20260709 12:11:53.147804 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:53.090327 17322 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:53.111232 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:53.118819 17612 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:53.139860 12604 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:53.109560 17532 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:53.097101 17645 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:53.131620 12494 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:11:53.155036 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 83728.2us
I20260709 12:11:53.160128 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:53.161939 12646 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:53.162340 17615 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:11:53.166335 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 46157.1us
I20260709 12:11:53.138856 17500 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:53.140726 17411 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:53.175011 12526 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b825dff40dc4290a25d71534b598330"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:53.138650 17671 raft_consensus.cc:697] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:53.158859 17322 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:53.168315 12569 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:53.178467 12526 raft_consensus.cc:606] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:53.185133 12526 raft_consensus.cc:614] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:53.120988 17430 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
W20260709 12:11:53.179564 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:53.177647 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46163.7us
I20260709 12:11:53.188879 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:53.132982 17315 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:11:53.133757 17582 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.041s
I20260709 12:11:53.162952 17615 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:53.164112 17669 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:53.133989 17626 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260709 12:11:53.182083 17270 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:53.193056 12646 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:53.162955 17652 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 12:11:53.201830 17669 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:53.202489 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:53.150420 17674 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:11:53.176232 17672 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:53.180029 17411 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:53.204595 12526 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:53.192842 17532 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:53.193071 17500 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:53.243324 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:53.249930 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:53.276446 12494 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:53.277155 12602 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:53.176538 17647 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:11:53.129391 17668 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:53.205055 17610 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:11:53.197722 17615 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:53.273862 12526 raft_consensus.cc:606] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:53.291323 12526 raft_consensus.cc:614] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:11:53.293311 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 170555us
I20260709 12:11:53.203146 17603 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.076s
I20260709 12:11:53.272652 17669 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:53.182981 17632 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:53.195446 17670 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.077s
I20260709 12:11:53.320307 12677 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 11 index: 7. (term mismatch)
I20260709 12:11:53.320914 12677 pending_rounds.cc:85] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 6
I20260709 12:11:53.321349 12677 pending_rounds.cc:107] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 9.7
I20260709 12:11:53.321797 12677 raft_consensus.cc:2889] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:53.242408 17667 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.253803 17582 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:11:53.274235 17634 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:11:53.281355 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 88372.4us
I20260709 12:11:53.336814 12646 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:53.256764 17612 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.037s
I20260709 12:11:53.270934 17315 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:11:53.312376 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:53.276484 17498 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:53.287249 17646 raft_consensus.cc:697] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:53.342074 17670 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:11:53.343381 17684 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.072s	user 0.002s	sys 0.000s
I20260709 12:11:53.342904 17670 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:53.343209 17646 consensus_queue.cc:237] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:53.345544 17670 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:53.315201 17631 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:11:53.347795 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:53.320125 17632 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:53.348791 12598 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:53.330345 17694 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.335414 17649 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:53.355259 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:53.360033 17632 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:53.337571 17582 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:53.371672 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:53.384779 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 377774us
I20260709 12:11:53.392508 12602 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:11:53.277093 17645 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 10.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:53.410741 17659 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:11:53.414464 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:53.415277 12677 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:53.419006 17668 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:53.379788 17500 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:53.363402 17498 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:11:53.316529 17671 consensus_queue.cc:237] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:53.368916 17582 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:53.374353 17241 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:53.397354 17670 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:53.482877 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:11:53.385552 17631 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.085s
I20260709 12:11:53.370167 17457 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:53.563382 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 226695us
I20260709 12:11:53.348940 17667 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:53.564493 12677 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:53.426726 13337 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:11:53.377971 17544 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:53.568822 12673 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:53.571636 17670 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:53.591915 17670 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:53.590457 17550 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:11:53.591292 17333 consensus_meta.cc:287] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.518s	user 0.009s	sys 0.005s
I20260709 12:11:53.583909 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:53.400367 17532 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:53.489764 17499 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:11:53.583515 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:53.605345 12496 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:53.568953 12522 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:11:53.567875 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:53.384222 17694 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:53.606405 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:53.571206 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:11:53.608240 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:53.584225 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:53.608708 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:11:53.637918 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:53.644438 12646 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:53.650970 17632 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:53.658236 12527 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 9: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:53.666749 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:53.660399 12646 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:53.669149 12646 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:53.572964 17508 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.543s	user 0.013s	sys 0.004s
I20260709 12:11:53.676057 17508 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:53.678226 17649 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:53.678718 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:53.679227 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:53.680429 12598 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15. Ops: [14.6-14.6]
I20260709 12:11:53.674221 12528 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 12: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:11:53.655493 17376 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.523s	user 0.007s	sys 0.004s
I20260709 12:11:53.679981 12600 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 9: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:53.690320 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:53.691511 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:53.692592 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:53.690985 12602 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:11:53.614615 17553 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.657984 17457 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:53.657490 17654 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:53.678758 17649 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:53.658378 17582 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.659264 17673 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:53.790990 12531 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:11:53.701125 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:53.701166 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:53.695735 12604 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 12: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:53.701406 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:53.793082 12671 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.011s	sys 0.015s
I20260709 12:11:53.740073 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:53.604848 17675 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:53.671398 17698 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:53.743372 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:53.751178 17241 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.758369 17626 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.799530 12599 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 15: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:53.798491 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:53.656621 17646 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.695786 12601 raft_consensus.cc:2410] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:11:53.689498 17544 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:53.692029 17376 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:53.689818 17593 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:53.656538 17397 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:53.752465 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:53.761272 17669 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:53.763346 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:53.682669 12646 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:53.695720 17702 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:53.758285 16935 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.706s	user 0.008s	sys 0.006s
I20260709 12:11:53.699968 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:53.880671 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 198045us
I20260709 12:11:53.822613 17673 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:53.786371 17649 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:53.844158 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:53.832136 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:53.865712 12494 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:53.867842 12530 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:53.757822 12526 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
W20260709 12:11:53.825911 17700 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
I20260709 12:11:53.875130 12599 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 9 index: 4. Preceding OpId from leader: term: 11 index: 5. (index mismatch)
I20260709 12:11:53.899046 13335 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:11:53.881626 12646 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:53.899716 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:53.793535 12671 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.011s	sys 0.015s
I20260709 12:11:53.794834 17582 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:53.827013 17508 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:53.820858 17675 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:53.892313 17544 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.892777 12598 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:53.895258 17691 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.895874 17705 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:11:53.896857 17706 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:11:53.745069 17553 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:53.792553 17580 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.091s
I20260709 12:11:53.904644 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:53.904985 12530 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:53.906704 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:53.959810 12606 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:53.826210 17241 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:53.830873 17626 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:53.881626 17646 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:53.869657 17593 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:53.960387 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:53.960201 12526 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:53.906116 17644 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:53.907593 17659 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.912956 17376 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:53.918411 17654 raft_consensus.cc:697] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:53.958843 17631 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:53.978348 13337 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:11:53.994491 12605 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 12: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:53.993760 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:53.997072 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 131507us
I20260709 12:11:53.913595 17698 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.064s
I20260709 12:11:53.885308 17702 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:53.997867 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 97563.3us
I20260709 12:11:53.996611 17708 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:53.904031 17673 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:53.945590 17544 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:54.016017 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:54.017277 12526 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 14: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:53.995244 17644 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:53.949327 17691 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.024662 12597 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:11:53.999812 17694 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.004688 17631 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:11:54.065505 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 45656.1us
I20260709 12:11:54.018970 17701 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:54.071017 12646 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:54.022184 17644 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:53.999689 17671 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:11:53.957036 17705 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:54.085135 17615 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.774s	user 0.010s	sys 0.000s
I20260709 12:11:54.051467 17667 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.011024 17708 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:53.999812 17659 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:54.070302 17701 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:54.142452 17660 raft_consensus.cc:3055] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:54.144119 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:54.152968 12600 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:54.144335 17670 consensus_meta.cc:287] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.552s	user 0.010s	sys 0.000s
I20260709 12:11:54.108593 17397 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:54.145052 17715 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:54.146437 17714 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:11:54.146162 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:54.148648 17632 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.164206 12605 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:54.165524 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:54.168962 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:54.168972 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:54.172644 12605 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:11:54.179551 17722 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.108s	user 0.000s	sys 0.000s
I20260709 12:11:54.179261 17721 consensus_queue.cc:1059] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 9.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.089s
I20260709 12:11:54.180568 17721 raft_consensus.cc:3055] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:54.180958 17721 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:54.097888 17707 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.182471 17707 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.183430 17397 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.158005 17660 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:54.187047 17660 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:54.188333 17660 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:54.023257 17654 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:54.190613 17719 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.191205 17719 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.192106 12530 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
W20260709 12:11:54.194303 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 123664us
I20260709 12:11:54.199796 17544 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.203069 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:54.204255 17689 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.896s	user 0.009s	sys 0.000s
I20260709 12:11:54.205523 17646 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.166776 17715 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.206903 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2367) took 1220 ms. Trace:
I20260709 12:11:54.207970 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 537) took 1325 ms. Trace:
I20260709 12:11:54.163723 17670 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.208542 12529 rpcz_store.cc:276] 0709 12:11:52.882353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:52.953854 (+ 71501us) service_pool.cc:224] Handling call
0709 12:11:54.207741 (+1253887us) spinlock_profiling.cc:243] Waited 61.5 ms on lock 0x7b5400106dd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:54.207887 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":141492480}
I20260709 12:11:54.160301 17723 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:54.210290 12522 raft_consensus.cc:2393] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 14: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:54.211061 17691 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.159737 17701 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:54.214488 17582 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.215548 17593 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.189914 17649 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.218068 17649 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:54.218804 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 504) took 1082 ms. Trace:
I20260709 12:11:54.219565 17673 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:54.208369 17715 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.207378 12672 rpcz_store.cc:276] 0709 12:11:52.986365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:52.986501 (+   136us) service_pool.cc:224] Handling call
0709 12:11:52.986818 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:53.045408 (+ 58590us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:53.045442 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:53.046402 (+   960us) log.cc:844] Serialized 26 byte log entry
0709 12:11:53.175544 (+129142us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:53.175603 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:53.175739 (+   136us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:54.203518 (+1027779us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:54.206141 (+  2623us) raft_consensus.cc:1685] finished
0709 12:11:54.206584 (+   443us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:54.206834 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6805,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":71565,"spinlock_wait_cycles":4748800,"thread_start_us":108181,"threads_started":1,"wal-append.queue_time_us":182240}
I20260709 12:11:54.223565 12529 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:11:54.224480 12600 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:11:54.225072 12529 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:11:54.159070 17669 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:54.245227 12529 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:11:54.247031 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:54.248612 17672 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:54.250792 12606 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:54.230486 17702 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.231020 17694 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.231314 17724 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.231496 17667 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:54.231551 17727 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:54.232420 17508 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.232493 17632 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.233196 17659 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.234599 17705 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:54.235128 17322 consensus_meta.cc:287] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.039s	user 0.013s	sys 0.000s
I20260709 12:11:54.238704 17698 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:54.239444 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:54.240072 17457 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.242267 17649 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:54.221240 17631 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.6, Last appended by leader: 6, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.244136 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:54.229194 12646 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:54.321376 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 123121us
I20260709 12:11:54.252337 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:54.219187 12671 rpcz_store.cc:276] 0709 12:11:53.136636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.149311 (+ 12675us) service_pool.cc:224] Handling call
0709 12:11:53.149581 (+   270us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:53.149772 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:11:53.900895 (+751123us) log.cc:844] Serialized 14 byte log entry
0709 12:11:54.218335 (+317440us) spinlock_profiling.cc:243] Waited 1.86 ms on lock 0x7b18002b8160. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287
0709 12:11:54.218442 (+   107us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:54.218490 (+    48us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:11:54.218541 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:54.218616 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:54.218745 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":100039,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":3327,"raft.queue_time_us":150813,"raft.run_cpu_time_us":215,"raft.run_wall_time_us":9481,"spinlock_wait_cycles":4287232,"thread_start_us":926948,"threads_started":2}
I20260709 12:11:54.260489 12643 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:54.228567 17673 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.136629 17615 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:54.181767 17721 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.267055 17694 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:54.283857 17550 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.691s	user 0.012s	sys 0.000s
I20260709 12:11:54.299408 17241 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:54.287572 17713 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.279s	user 0.002s	sys 0.000s
I20260709 12:11:54.281191 17626 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.272459 17632 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:54.267820 17497 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.021s
I20260709 12:11:54.269407 17667 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.271135 17508 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.300140 17603 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.328490 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:54.316176 17718 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.332006 12680 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 12: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:54.279315 17322 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:54.270336 17724 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.358227 12646 leader_election.cc:400] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12.
I20260709 12:11:54.386158 12646 leader_election.cc:403] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
W20260709 12:11:54.368238 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 107854us
I20260709 12:11:54.269464 17727 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:54.385632 12527 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:54.359267 17593 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.379014 17498 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:54.363449 17730 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:54.361742 17723 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.393988 12680 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
W20260709 12:11:54.377265 17695 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.173s	user 0.000s	sys 0.018s
I20260709 12:11:54.397517 12671 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:54.408514 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:54.390434 17411 consensus_meta.cc:287] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.097s	user 0.014s	sys 0.000s
I20260709 12:11:54.412485 13339 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 13: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:54.415575 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:54.416533 12671 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:11:54.321695 17698 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.393179 17721 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:54.358966 17673 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:54.362242 17649 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:54.425520 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:54.428778 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:54.424216 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:54.434926 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:54.432972 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:54.446012 12524 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15.
I20260709 12:11:54.446175 12525 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:54.445369 12522 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 14.
I20260709 12:11:54.443513 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:54.447980 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:54.392262 17603 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.399487 17593 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.488092 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:54.436003 17626 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.489651 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:54.445549 12529 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
I20260709 12:11:54.491230 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:54.503870 12522 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:11:54.492236 12672 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 14.
W20260709 12:11:54.492208 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 104786us
I20260709 12:11:54.490378 13305 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:11:54.378214 17672 consensus_queue.cc:1175] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" responder_term: 12 status { last_received { term: 11 index: 9 } last_committed_idx: 8 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:11:54.498560 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 164886us
I20260709 12:11:54.499963 12525 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:11:54.512955 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:54.506078 12522 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260709 12:11:54.516047 12522 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:11:54.517618 12522 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:54.509096 13335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 538) took 1099 ms. Trace:
I20260709 12:11:54.509418 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:54.521188 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:54.394737 17718 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:54.421538 17411 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:54.513180 12525 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:54.527511 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:54.527946 12525 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: [11.5-11.5]
I20260709 12:11:54.500910 12671 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:11:54.518365 13335 rpcz_store.cc:276] 0709 12:11:53.399727 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.399947 (+   220us) service_pool.cc:224] Handling call
0709 12:11:53.409847 (+  9900us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:53.899580 (+489733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:54.498795 (+599215us) spinlock_profiling.cc:243] Waited 489 ms on lock 0x7b54004439d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:54.499014 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1124438784}
I20260709 12:11:54.524109 12529 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:54.497064 17333 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:54.493397 17744 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:11:54.510521 17695 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:11:54.419731 17723 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.541989 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:54.526051 17544 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:54.554527 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:54.555124 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:54.445258 17701 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.552871 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:54.574018 12673 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:11:54.574683 12680 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:54.575256 17519 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:54.575330 17669 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:54.570887 12679 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 12. Ops: []
W20260709 12:11:54.538448 17693 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.040s	user 0.009s	sys 0.033s
I20260709 12:11:54.572389 12524 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:11:54.575837 17519 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:54.538918 17508 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.570565 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:54.579002 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:54.580283 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 55656.8us
I20260709 12:11:54.543627 17649 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:54.536824 17660 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:11:54.546224 17333 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.546492 17736 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.570153 12528 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:11:54.597429 12673 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:54.547871 17603 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.539685 17322 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.543272 17667 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.541592 17652 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:54.533812 17659 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:54.512408 17626 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:54.579751 17693 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.201s	user 0.011s	sys 0.033s
I20260709 12:11:54.572634 17544 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:54.610306 17457 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.606932 12571 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:54.556846 17744 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12.
I20260709 12:11:54.609158 12671 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:11:54.609880 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 102588us
I20260709 12:11:54.590567 17519 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:54.615767 17727 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:54.597462 17376 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:54.616369 17411 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:54.612279 17392 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:54.614699 12673 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:54.620116 13305 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:54.611191 17740 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:54.619513 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:54.630203 17652 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:54.630471 17659 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:54.636256 17724 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.630731 17603 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.631011 17746 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.640659 17723 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:54.641064 17333 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.665055 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2354) took 2022 ms. Trace:
I20260709 12:11:54.690567 12522 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:54.690932 12673 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.662674 17544 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:11:54.732266 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 126991us
I20260709 12:11:54.664004 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:54.687726 17742 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
W20260709 12:11:54.692232 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 72213.6us
W20260709 12:11:54.706981 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 100149us
I20260709 12:11:54.736146 12673 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:54.688850 17652 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:54.633328 17593 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.617971 17736 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.746430 17718 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.747360 12671 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:54.689060 17659 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:54.690363 12678 rpcz_store.cc:276] 0709 12:11:52.642515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:52.644856 (+  2341us) service_pool.cc:224] Handling call
0709 12:11:52.645259 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:52.895197 (+249938us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:11:53.139926 (+244729us) log.cc:844] Serialized 14 byte log entry
0709 12:11:53.351219 (+211293us) log.cc:844] Serialized 14 byte log entry
0709 12:11:53.360272 (+  9053us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:53.364786 (+  4514us) log.cc:844] Serialized 26 byte log entry
0709 12:11:53.364943 (+   157us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:53.364997 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:53.365097 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:54.319940 (+954843us) spinlock_profiling.cc:243] Waited 2.98 ms on lock 0x7b2400201a28. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14857b99c4 00007f14857b7acb 00007f1485972dc9 00007f1485972a55 00007f148597209d 00007f14885a410e 00007f14885bb88c 00007f14885babb6 00007f14885ba7d8 00007f14885c4221
0709 12:11:54.664729 (+344789us) raft_consensus.cc:1685] finished
0709 12:11:54.664770 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:54.664969 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10311,"prepare.queue_time_us":320665,"prepare.run_cpu_time_us":123,"prepare.run_wall_time_us":11469,"raft.queue_time_us":94819,"raft.run_cpu_time_us":477,"raft.run_wall_time_us":71456,"spinlock_wait_cycles":6849408,"thread_start_us":600867,"threads_started":4,"wal-append.queue_time_us":31471}
I20260709 12:11:54.663843 17632 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:54.693099 17701 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:54.693323 17615 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.734450 17333 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:54.689335 17740 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:54.732220 17669 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:54.692472 17660 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:54.690359 17603 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.749645 17693 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.067s
I20260709 12:11:54.734519 17698 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:54.688762 17497 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.066s
I20260709 12:11:54.691468 17376 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:54.866762 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 113908us
I20260709 12:11:54.744976 17673 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:54.793530 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 129676us
I20260709 12:11:54.867349 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:54.852278 12673 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:11:54.706302 17392 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:54.889120 12671 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:54.890036 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:54.890101 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:54.896026 12643 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.525s (SENT)
W20260709 12:11:54.866899 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 82226.8us
I20260709 12:11:54.891820 12522 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 14.
I20260709 12:11:54.689618 17746 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:54.901093 13305 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:54.896976 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:54.904923 12678 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:11:54.813856 17738 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:11:54.907955 12673 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:54.914013 17756 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:11:54.867393 17411 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:54.915530 17497 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:11:54.916031 17497 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:54.915691 17757 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:11:54.793334 17333 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:54.916242 17644 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.826s	user 0.008s	sys 0.005s
I20260709 12:11:54.791553 17593 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:54.789743 17754 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:54.915623 17756 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:54.816551 17730 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:54.887951 17669 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:54.908346 17694 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:54.936954 17669 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:54.897914 17701 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260709 12:11:54.938591 17701 consensus_queue.cc:1243] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 13.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:11:54.938824 17411 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 9.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260709 12:11:54.940238 17701 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:54.936688 17756 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:11:54.940155 17411 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:11:54.940768 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:54.949582 17708 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.917024 17497 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:54.956120 13339 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 665) took 1759 ms (client timeout 1745 ms). Trace:
I20260709 12:11:54.941646 17756 raft_consensus.cc:3055] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:54.974784 17756 raft_consensus.cc:740] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:11:54.974690 13339 rpcz_store.cc:269] 0709 12:11:53.196720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.197116 (+   396us) service_pool.cc:224] Handling call
0709 12:11:54.955813 (+1758697us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400145fd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:54.955994 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2611015552}
I20260709 12:11:54.976220 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:54.977852 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:54.977994 17750 log.cc:927] Time spent T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.244s	user 0.000s	sys 0.000s
I20260709 12:11:54.978590 12597 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
W20260709 12:11:54.982815 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 88185.8us
I20260709 12:11:54.942863 17753 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:11:54.957276 12527 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2325) took 1284 ms (client timeout 1629 ms). Trace:
I20260709 12:11:54.941814 17411 raft_consensus.cc:3055] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:54.940892 17701 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:54.961561 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:55.011305 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 121592us
I20260709 12:11:54.962520 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:54.962965 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2462) took 1283 ms. Trace:
I20260709 12:11:54.963196 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:54.956178 12678 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:11:55.013861 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:54.868167 17723 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.094s
I20260709 12:11:55.013120 12601 rpcz_store.cc:276] 0709 12:11:53.679814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.831039 (+151225us) service_pool.cc:224] Handling call
0709 12:11:53.831363 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:54.002473 (+171110us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:54.002508 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:54.002555 (+    47us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:54.002585 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:54.962739 (+960154us) spinlock_profiling.cc:243] Waited 171 ms on lock 0x7b5400310bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:54.962787 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:54.962911 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":392999808}
I20260709 12:11:54.792703 17740 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:54.966909 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:54.967458 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:55.020434 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:55.020695 12526 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:11:55.021003 12525 raft_consensus.cc:2410] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 12 because replica has last-logged OpId of term: 10 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 8 index: 5.
I20260709 12:11:54.968096 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:54.965634 12646 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:54.970971 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:54.971665 12605 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
W20260709 12:11:55.022178 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 56618.1us
I20260709 12:11:54.975848 17756 consensus_queue.cc:260] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 11.5, Last appended by leader: 5, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.984076 17718 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:54.923005 17333 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:54.851815 17755 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:54.991837 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:54.997718 12671 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:55.014598 12597 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:54.992779 17753 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:55.010133 17411 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:54.969141 17673 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 11.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.056s
I20260709 12:11:55.039319 12678 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:54.942025 17376 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:55.024660 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:55.024874 12646 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:54.927909 17644 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:54.935554 17593 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:55.027284 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2369) took 2040 ms. Trace:
I20260709 12:11:55.018332 17626 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:54.967483 17754 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:55.039708 12678 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:11:55.010483 12527 rpcz_store.cc:269] 0709 12:11:53.673466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.674218 (+   752us) service_pool.cc:224] Handling call
0709 12:11:54.957068 (+1282850us) spinlock_profiling.cc:243] Waited 485 ms on lock 0x7b54003caad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:54.957205 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1114892544}
I20260709 12:11:55.048408 12676 rpcz_store.cc:276] 0709 12:11:52.986974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:52.997194 (+ 10220us) service_pool.cc:224] Handling call
0709 12:11:52.997611 (+   417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:52.997967 (+   356us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:52.998000 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:53.003897 (+  5897us) log.cc:844] Serialized 26 byte log entry
0709 12:11:54.213459 (+1209562us) spinlock_profiling.cc:243] Waited 5.04 ms on lock 0x7b1800318a60. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:11:54.367264 (+153805us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:54.367328 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:54.367400 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:55.022942 (+655542us) raft_consensus.cc:1685] finished
0709 12:11:55.022965 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:55.027216 (+  4251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38333,"raft.queue_time_us":104,"raft.run_cpu_time_us":210,"raft.run_wall_time_us":5018,"spinlock_wait_cycles":11592448,"thread_start_us":153623,"threads_started":1,"wal-append.queue_time_us":1609211}
I20260709 12:11:55.027024 17354 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:11:55.047755 12531 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:55.021895 12524 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:11:55.027194 17649 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:55.111487 12677 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 14: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:55.042517 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:55.054409 12676 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 10 index: 11. (index mismatch)
I20260709 12:11:55.038159 17718 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 14 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:55.119226 17718 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:55.122948 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 96452.5us
I20260709 12:11:55.077358 12529 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
W20260709 12:11:55.051534 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 132240us
W20260709 12:11:55.127601 12643 leader_election.cc:336] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.158s (SENT)
I20260709 12:11:55.128194 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:55.158056 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 133244us
I20260709 12:11:55.027786 17693 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:11:55.164448 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:55.166388 17411 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:55.167016 17741 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
W20260709 12:11:55.171841 17758 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:11:55.037904 17755 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:55.175019 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2745) took 1865 ms. Trace:
I20260709 12:11:55.021451 17660 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:55.190696 12679 rpcz_store.cc:276] 0709 12:11:53.309964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.375170 (+ 65206us) service_pool.cc:224] Handling call
0709 12:11:53.375531 (+   361us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:54.576641 (+1201110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:55.174714 (+598073us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54000c26d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:55.174946 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2717659520}
I20260709 12:11:55.271505 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:55.297784 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:55.164346 17649 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:55.190848 17760 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:55.267212 17736 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:55.274482 17603 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:55.321597 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2333) took 1344 ms. Trace:
I20260709 12:11:55.322202 12530 rpcz_store.cc:276] 0709 12:11:53.976730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.976949 (+   219us) service_pool.cc:224] Handling call
0709 12:11:55.321357 (+1344408us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b54000eef54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:55.321511 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":429776128}
I20260709 12:11:55.369004 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:55.370126 12679 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
W20260709 12:11:55.394702 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 267110us
W20260709 12:11:55.397954 12643 leader_election.cc:336] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.935s (SENT)
I20260709 12:11:55.398753 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:55.311183 17760 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:55.311715 17698 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
W20260709 12:11:55.401978 12643 leader_election.cc:336] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.124s (SENT)
I20260709 12:11:55.315040 17411 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:55.403228 12643 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:55.194459 17753 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:55.301815 17761 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.106s
I20260709 12:11:55.441256 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2337) took 1390 ms. Trace:
I20260709 12:11:55.439646 17736 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:55.448055 17764 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.259s	user 0.000s	sys 0.000s
I20260709 12:11:55.448182 17736 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:55.406622 17760 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:55.444841 12523 rpcz_store.cc:276] 0709 12:11:54.051298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.204192 (+152894us) service_pool.cc:224] Handling call
0709 12:11:54.204542 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:54.204951 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:11:54.499609 (+294658us) log.cc:844] Serialized 14 byte log entry
0709 12:11:54.749539 (+249930us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:54.789802 (+ 40263us) log.cc:844] Serialized 26 byte log entry
0709 12:11:54.790019 (+   217us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:54.790057 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:54.791095 (+  1038us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:55.440938 (+649843us) raft_consensus.cc:1685] finished
0709 12:11:55.440967 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:55.441172 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3122,"prepare.queue_time_us":332982,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":928,"raft.run_cpu_time_us":220,"raft.run_wall_time_us":34179,"thread_start_us":489865,"threads_started":2,"wal-append.queue_time_us":387675}
I20260709 12:11:55.413776 17746 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:55.414088 17626 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:55.412873 17603 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.454495 17736 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:55.469666 17765 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:11:55.480777 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:55.480373 17753 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:55.478834 17698 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:55.477042 17769 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:11:55.502517 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:55.488559 17593 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:55.479763 17626 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:55.480846 17603 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:55.499917 17644 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:55.514389 17767 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:11:55.525044 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2329) took 1703 ms. Trace:
I20260709 12:11:55.525550 12531 rpcz_store.cc:276] 0709 12:11:53.822210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.822410 (+   200us) service_pool.cc:224] Handling call
0709 12:11:53.822868 (+   458us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:55.061279 (+1238411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:55.524775 (+463496us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54003cc3d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:55.524974 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2816133504}
I20260709 12:11:55.512333 17674 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:55.481190 17649 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:55.516400 17736 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:11:55.549793 12675 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.021s	sys 0.021s
I20260709 12:11:55.551131 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2390) took 1010 ms. Trace:
I20260709 12:11:55.557045 17392 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:55.528199 17754 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:55.527449 17772 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:55.534288 17755 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:55.551577 17674 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:55.551581 12675 rpcz_store.cc:276] 0709 12:11:54.541344 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.951908 (+410564us) service_pool.cc:224] Handling call
0709 12:11:54.952349 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:54.952636 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:11:54.953005 (+   369us) log.cc:844] Serialized 14 byte log entry
0709 12:11:55.550722 (+597717us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:55.550792 (+    70us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:55.550848 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:55.550918 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:55.551063 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19142,"prepare.queue_time_us":2249,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":123948,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":23336,"thread_start_us":593812,"threads_started":2,"wal-append.queue_time_us":360568}
I20260709 12:11:55.587598 12646 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:55.588492 12646 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:55.554217 17736 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:55.592805 17736 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:55.595877 17754 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:55.598956 12531 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15. Ops: []
W20260709 12:11:55.584636 17770 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:11:55.570284 17354 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:55.600994 12526 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15. Ops: [14.6-14.6]
W20260709 12:11:55.604853 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:55.595492 17519 consensus_meta.cc:287] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.910s	user 0.013s	sys 0.000s
I20260709 12:11:55.614315 17675 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:55.614786 17519 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:55.617427 17519 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:55.618369 12601 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:55.578560 17772 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:55.621018 17772 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:55.623178 17652 consensus_meta.cc:287] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.868s	user 0.011s	sys 0.001s
I20260709 12:11:55.626494 12679 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:55.623764 17652 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:55.627780 17544 consensus_meta.cc:287] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.885s	user 0.012s	sys 0.001s
I20260709 12:11:55.628044 17354 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:55.608114 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:55.629160 17603 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:55.629333 17652 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:55.608242 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:55.629710 12530 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:55.630003 12523 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:55.602690 17776 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:55.610639 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:55.606982 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:55.614140 17626 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:55.631093 17776 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:55.631345 12528 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 14: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:55.632133 17776 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:55.631711 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:55.634382 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:55.603502 17754 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:55.634461 17354 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:55.634919 17754 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:11:55.612504 17659 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.820s	user 0.008s	sys 0.004s
I20260709 12:11:55.639408 17659 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:55.645638 17626 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:55.646713 17392 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:55.646322 12494 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:55.647315 17392 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:55.648276 17397 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:11:55.649159 17733 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.187s	user 0.000s	sys 0.006s
I20260709 12:11:55.648891 17397 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:55.646342 17603 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.650744 17603 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:55.651145 17397 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:55.651711 17397 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:55.652079 17397 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:55.652402 17397 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:55.652878 17603 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:55.606252 17755 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:55.654953 17755 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:55.657847 12494 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:55.658823 12494 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:55.660456 17376 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:55.661002 17376 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:55.662330 17376 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:55.646399 17626 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:55.628492 17544 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:55.669664 17736 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:55.669808 17701 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:55.665485 17740 consensus_meta.cc:287] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.629s	user 0.009s	sys 0.003s
I20260709 12:11:55.672739 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:11:55.675343 17716 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.663s	user 0.023s	sys 0.005s
I20260709 12:11:55.662349 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:55.675566 17603 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:55.649972 17392 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:55.696470 12646 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:55.697749 17333 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.660s	user 0.013s	sys 0.001s
I20260709 12:11:55.707295 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:55.697801 17753 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:11:55.700439 17760 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:55.702746 17755 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:55.704617 17626 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:55.715035 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:55.728283 12530 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:11:55.719511 12529 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:11:55.720916 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:55.722806 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:55.730564 12523 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:55.727244 12528 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:11:55.734843 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:55.734910 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:55.737534 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:55.738590 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:55.738891 12528 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 12: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:55.741952 12526 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:11:55.726282 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:55.745301 12531 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:11:55.764003 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:55.764955 12523 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:55.765551 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:55.705250 17778 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.766489 12529 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:11:55.693747 17754 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:55.721804 17392 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:55.768280 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:55.722182 17411 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:55.791721 12530 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:11:55.767606 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:55.790726 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 103085us
I20260709 12:11:55.793032 12527 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:55.764269 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:55.691488 17740 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:55.740852 17777 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:55.707074 17644 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
I20260709 12:11:55.710363 17603 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:55.721927 17333 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:55.827893 17775 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:11:55.829368 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:55.829690 17775 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:11:55.830543 17775 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:55.833562 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:55.834754 12572 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:55.807655 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:55.848875 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 47079us
I20260709 12:11:55.795506 17392 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:55.796481 17772 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.864252 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:55.791224 17755 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:55.851354 12494 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:55.795980 17411 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:55.868893 12602 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 12:11:55.869547 12602 pending_rounds.cc:85] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 11
I20260709 12:11:55.869804 12602 pending_rounds.cc:107] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 12:11:55.870097 12602 raft_consensus.cc:2889] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:55.799998 17519 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:55.800925 17777 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:55.871594 12572 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:55.874341 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:55.874642 12598 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 12: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:55.881791 12605 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 12: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:55.799286 17659 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:55.941344 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:55.886061 12597 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:11:55.867321 17703 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.953s	user 0.044s	sys 0.038s
I20260709 12:11:55.835279 17779 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:55.865377 17644 consensus_queue.cc:1243] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 15.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:11:55.882899 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:55.796146 17778 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:55.950528 17498 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:55.950665 17270 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:55.879381 17698 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.879076 17593 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:11:55.954015 17784 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:11:55.954636 17674 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:55.878738 17721 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:55.955804 17544 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:55.957908 17754 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:55.963032 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:55.976436 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:55.866513 17392 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:55.971426 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:55.974810 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:55.986537 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:55.853890 17736 raft_consensus.cc:697] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:55.867765 17626 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.083s
I20260709 12:11:55.873122 17772 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:56.039675 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:56.136234 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 264689us
I20260709 12:11:56.076876 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:56.184360 12523 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:11:56.114475 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2372) took 1870 ms. Trace:
I20260709 12:11:56.117920 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 505) took 1879 ms. Trace:
I20260709 12:11:56.182988 12605 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
W20260709 12:11:56.082818 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:56.187490 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:56.187901 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:56.188251 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:56.188344 12598 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:55.872691 17411 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:55.945412 17703 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 2.038s	user 0.046s	sys 0.040s
I20260709 12:11:55.983745 17498 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:56.186244 12672 rpcz_store.cc:276] 0709 12:11:54.239053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.239179 (+   126us) service_pool.cc:224] Handling call
0709 12:11:56.117666 (+1878487us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b540016f8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:56.117833 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":421372672}
I20260709 12:11:56.192618 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:55.879702 17792 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:56.201263 12572 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:56.198331 12523 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:55.883266 17519 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:55.882390 17777 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:56.185621 12677 rpcz_store.cc:276] 0709 12:11:54.243767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.243893 (+   126us) service_pool.cc:224] Handling call
0709 12:11:56.114104 (+1870211us) spinlock_profiling.cc:243] Waited 791 ms on lock 0x7b54002c4a54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:56.114378 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1818174336}
I20260709 12:11:56.206112 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:55.881068 17755 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:56.211854 12572 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:56.101338 17785 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
W20260709 12:11:56.165046 17784 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:11:56.214457 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:56.111763 17745 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.601s	user 0.004s	sys 0.027s
I20260709 12:11:56.200057 17794 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.009s
I20260709 12:11:56.124495 17603 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:56.163226 17333 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:56.008523 17593 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:56.175725 17707 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:56.213904 12528 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:11:56.075783 17778 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:56.205901 12646 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:56.214428 12523 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:11:56.190687 17791 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.195014 17270 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:11:56.077719 17397 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:56.193137 17721 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:56.230922 17740 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:55.968472 17779 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:56.201705 17792 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:56.200884 17626 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:11:56.219440 17736 consensus_queue.cc:237] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 9.4, Last appended by leader: 4, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:56.268388 17519 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:56.269326 17772 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:11:56.270076 17603 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:11:56.271629 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:56.293184 17673 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.217692 17457 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:56.033396 17698 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:56.270493 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 58738.9us
I20260709 12:11:56.282822 17778 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:56.283298 17791 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:56.272946 17593 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:56.294109 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2625) took 2422 ms. Trace:
I20260709 12:11:56.332609 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:56.333782 13335 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:11:56.271541 17795 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:56.305181 17779 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:56.305716 17792 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:56.282259 17760 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:56.270545 17652 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260709 12:11:56.319113 12606 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
W20260709 12:11:56.281131 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 75305.9us
I20260709 12:11:56.326148 12572 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:11:56.327394 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 113016us
I20260709 12:11:56.328408 13337 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:11:56.345393 12680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2382) took 1941 ms (client timeout 2142 ms). Trace:
I20260709 12:11:56.357628 12646 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:56.360630 12646 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:56.361354 12523 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:56.332753 12604 rpcz_store.cc:276] 0709 12:11:53.871485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.170049 (+298564us) service_pool.cc:224] Handling call
0709 12:11:54.436237 (+266188us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:54.443508 (+  7271us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:54.443538 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:54.452588 (+  9050us) log.cc:844] Serialized 26 byte log entry
0709 12:11:54.541557 (+ 88969us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:54.541687 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:54.541860 (+   173us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:54.639348 (+ 97488us) raft_consensus.cc:1685] finished
0709 12:11:54.639371 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:56.274368 (+1634997us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54002f2450. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:56.294004 (+ 19636us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3630,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":2777,"spinlock_wait_cycles":611148160,"thread_start_us":39067,"threads_started":1,"wal-append.queue_time_us":85111,"wal-append.run_cpu_time_us":2716,"wal-append.run_wall_time_us":1042021}
I20260709 12:11:56.367789 12572 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:56.368505 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:56.369803 12604 raft_consensus.cc:2410] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:11:56.324867 17778 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:56.358846 12680 rpcz_store.cc:269] 0709 12:11:54.403804 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.408169 (+  4365us) service_pool.cc:224] Handling call
0709 12:11:56.345122 (+1936953us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54000933d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:56.345309 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":377139456}
I20260709 12:11:56.373051 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:56.376525 13337 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:56.383387 12673 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:56.383220 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:56.384651 12680 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 14: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:56.392184 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.396344 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:56.314517 17626 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:56.386895 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:56.407330 13330 raft_consensus.cc:2368] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16.
I20260709 12:11:56.319191 17498 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:56.413803 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.414774 12673 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:56.412267 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.406392 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:56.416025 12606 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:56.329876 17772 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:11:56.431208 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:56.435432 12604 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:56.436663 12604 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:56.344659 17781 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:56.432590 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.437793 17781 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:56.412961 17707 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.439476 17707 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:56.441788 17707 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:56.442389 17707 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.442893 17707 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:56.445446 17707 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:56.374197 17746 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:56.450925 13337 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:56.330029 17673 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:56.374699 17774 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:56.387367 17778 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:56.386807 17457 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:56.457674 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:56.459698 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:56.458654 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:56.413196 17779 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:56.463595 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:56.465097 13330 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:11:56.423938 17553 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:56.403904 17793 consensus_queue.cc:1059] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.073s
I20260709 12:11:56.470922 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.474288 12672 raft_consensus.cc:2410] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260709 12:11:56.464900 17672 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.463718 17746 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:56.456413 17740 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:11:56.488624 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.491566 17397 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:56.472247 17755 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:56.463376 17354 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.024s
I20260709 12:11:56.508520 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2635) took 1547 ms. Trace:
I20260709 12:11:56.490629 17707 raft_consensus.cc:3055] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:11:56.512951 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:56.513639 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.515143 17805 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:56.515764 17805 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:56.510512 12600 rpcz_store.cc:276] 0709 12:11:54.961455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:54.963626 (+  2171us) service_pool.cc:224] Handling call
0709 12:11:54.970779 (+  7153us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:54.971002 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:11:56.386314 (+1415312us) log.cc:844] Serialized 14 byte log entry
0709 12:11:56.508053 (+121739us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:56.508126 (+    73us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:56.508186 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:56.508258 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:56.508428 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1334329,"prepare.queue_time_us":1392866,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":3791,"thread_start_us":201625,"threads_started":2}
I20260709 12:11:56.517674 17754 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:11:56.518558 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:56.519264 13334 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:11:56.520596 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:56.498252 17790 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 12:11:56.522817 17698 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:56.524916 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:56.525596 12523 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:56.512888 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:56.523433 17721 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:56.531253 17498 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:56.533107 17626 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:56.534621 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:56.536321 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2401) took 1080 ms. Trace:
I20260709 12:11:56.536738 17791 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:56.524160 17707 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:56.539996 17707 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:56.541360 17707 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:56.544081 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:56.488866 17672 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:56.547250 17673 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:56.536856 12676 rpcz_store.cc:276] 0709 12:11:55.456231 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:55.484163 (+ 27932us) service_pool.cc:224] Handling call
0709 12:11:55.506528 (+ 22365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:55.506849 (+   321us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:55.506869 (+    20us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:55.507239 (+   370us) log.cc:844] Serialized 26 byte log entry
0709 12:11:55.682649 (+175410us) spinlock_profiling.cc:243] Waited 1.23 ms on lock 0x7b1800278b60. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:11:56.033328 (+350679us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:56.033388 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:56.033488 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:56.523636 (+490148us) raft_consensus.cc:1685] finished
0709 12:11:56.523655 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:56.536230 (+ 12575us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11735,"raft.queue_time_us":11506,"raft.run_cpu_time_us":285,"raft.run_wall_time_us":13591,"spinlock_wait_cycles":2833792,"thread_start_us":350442,"threads_started":1,"wal-append.queue_time_us":677423}
I20260709 12:11:56.550042 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:56.550973 12528 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
W20260709 12:11:56.568863 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:56.506287 17740 consensus_queue.cc:1243] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:11:56.579698 12569 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:56.581359 12673 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:11:56.583455 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:56.584928 12527 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:56.585405 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.589650 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.590607 12679 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:56.616601 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.617580 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:56.620812 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:56.583266 17498 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.578874 17673 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:11:56.627612 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:56.629403 12523 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:11:56.631311 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.611524 17740 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:56.634778 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.638007 12673 raft_consensus.cc:2393] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 13: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:56.640004 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2644) took 1467 ms. Trace:
I20260709 12:11:56.588600 17647 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:11:56.616406 17457 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:56.627362 17774 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:56.672112 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.640108 17740 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:56.640480 12599 rpcz_store.cc:276] 0709 12:11:55.173098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:55.198007 (+ 24909us) service_pool.cc:224] Handling call
0709 12:11:55.198584 (+   577us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:55.478828 (+280244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:55.478859 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:55.491347 (+ 12488us) log.cc:844] Serialized 52 byte log entry
0709 12:11:55.584204 (+ 92857us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:56.450604 (+866400us) log.cc:844] Serialized 14 byte log entry
0709 12:11:56.639616 (+189012us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:56.639720 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:56.639769 (+    49us) raft_consensus.cc:1685] finished
0709 12:11:56.639780 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:56.639942 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":735478,"prepare.queue_time_us":839396,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":3576,"raft.queue_time_us":240915,"raft.run_cpu_time_us":319,"raft.run_wall_time_us":29929,"thread_start_us":577051,"threads_started":4,"wal-append.queue_time_us":95942}
I20260709 12:11:56.633783 17791 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.639247 17626 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:56.643903 17670 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:56.755831 12679 raft_consensus.cc:2410] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 13 because replica has last-logged OpId of term: 11 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 8.
I20260709 12:11:56.737958 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.640669 17519 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:56.774941 17626 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:56.795966 17392 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.599s	user 0.012s	sys 0.000s
I20260709 12:11:56.796471 17392 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:56.797605 17626 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:56.775170 17647 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:11:56.802172 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:56.802644 17647 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:56.671967 17705 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:56.803687 17705 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:56.803560 17647 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:56.804996 17647 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:56.633168 17673 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:56.628026 17746 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:56.792536 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 333963us
W20260709 12:11:56.805682 12494 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.623s	user 0.017s	sys 0.017s
W20260709 12:11:56.807404 12494 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.019s	sys 0.019s
I20260709 12:11:56.731043 17740 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:56.804626 17705 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:56.630453 17498 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:56.806737 12676 raft_consensus.cc:2410] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 12:11:56.807726 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 681809us
I20260709 12:11:56.807729 12572 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:56.808274 12673 raft_consensus.cc:2410] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
W20260709 12:11:56.775012 17810 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260709 12:11:56.808928 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:56.794171 17672 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:56.723523 17777 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:56.822389 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.825181 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.825965 12673 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:56.809211 17795 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:56.815470 17593 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:56.830708 17816 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:56.831312 17593 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:56.832594 17805 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:56.820785 17672 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:56.814267 17719 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:56.775445 17791 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:56.832994 17813 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:56.859802 17813 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:56.834568 17774 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:56.862649 12496 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:56.842089 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 500) took 3145 ms. Trace:
W20260709 12:11:56.864598 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:56.856604 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.865483 12679 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:56.865796 12675 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:56.838711 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.866811 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.850262 12572 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:56.871029 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:56.860278 17813 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:56.971527 13334 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:11:56.983839 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:56.985061 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:56.822959 17705 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:56.892889 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.990727 17644 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:56.991525 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.871016 12496 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:56.863377 12603 rpcz_store.cc:276] 0709 12:11:53.696912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:53.908872 (+211960us) service_pool.cc:224] Handling call
0709 12:11:53.947153 (+ 38281us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:54.262555 (+315402us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:54.262611 (+    56us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:54.264501 (+  1890us) log.cc:844] Serialized 26 byte log entry
0709 12:11:54.430972 (+166471us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:11:56.284376 (+1853404us) log.cc:844] Serialized 14 byte log entry
0709 12:11:56.338752 (+ 54376us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:56.840821 (+502069us) spinlock_profiling.cc:243] Waited 4.83 ms on lock 0x7b180043c3c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419
0709 12:11:56.841775 (+   954us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54002e68d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:56.841811 (+    36us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:56.841858 (+    47us) raft_consensus.cc:1685] finished
0709 12:11:56.841870 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:56.842028 (+   158us) inbound_call.cc:177] Queueing success response
0709 12:11:56.850691 (+  8663us) spinlock_profiling.cc:243] Waited 33.1 ms on lock 0x7b180043c3c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485933522 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
Metrics: {"mutex_wait_us":1901925,"prepare.queue_time_us":1891318,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":57,"raft.queue_time_us":48758,"raft.run_cpu_time_us":356,"raft.run_wall_time_us":14659,"spinlock_wait_cycles":717207040,"thread_start_us":201404,"threads_started":2,"wal-append.queue_time_us":425662,"wal-append.run_cpu_time_us":11380,"wal-append.run_wall_time_us":1382991}
I20260709 12:11:56.987044 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:56.867368 12680 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:56.982261 12679 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:56.994592 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.898840 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:56.944610 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:56.932065 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:56.955598 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:56.917815 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 67661us
I20260709 12:11:56.955139 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:56.992203 12673 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:56.939905 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:56.998230 12572 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:56.873368 13337 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 12. Ops: []
W20260709 12:11:56.993299 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 122374us
I20260709 12:11:56.999727 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:57.000043 12496 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:57.000434 12680 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:56.998088 13335 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 16: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:11:56.942727 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:57.002774 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:57.003631 13334 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 13: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:57.005241 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:57.006835 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:57.005928 13334 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:56.837591 17672 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:57.015753 17672 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:57.015853 17411 consensus_meta.cc:287] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.812s	user 0.011s	sys 0.005s
I20260709 12:11:56.978811 17777 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:57.016428 17411 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:57.021971 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:57.025209 13338 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 15: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:56.835121 17673 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:56.856501 17791 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:57.031289 13330 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.024s	sys 0.072s
W20260709 12:11:57.032719 13330 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.024s	sys 0.072s
I20260709 12:11:57.033252 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:56.866952 17696 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:11:56.857494 17701 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:57.035518 13333 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:56.842691 17392 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:56.839844 17498 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:56.894243 17626 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:11:57.000288 17603 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:57.055459 12679 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:11:57.056823 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:56.842811 17795 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.197s
I20260709 12:11:57.040886 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:57.032733 17647 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:56.839998 17457 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:57.061136 17411 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:57.067245 17740 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:56.831355 17816 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:57.069645 17376 raft_consensus.cc:3055] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:11:57.039999 17719 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:57.070195 17376 raft_consensus.cc:740] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:57.071568 17816 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:57.076009 17789 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.800s	user 0.001s	sys 0.000s
I20260709 12:11:57.040215 17794 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.041s
I20260709 12:11:57.039994 17645 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:57.078387 17645 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:57.078791 17645 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:57.078564 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:57.079370 12494 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:57.080171 17626 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:57.081009 17626 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:57.036006 17707 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:11:57.049578 17792 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.676s	user 0.006s	sys 0.006s
I20260709 12:11:57.083930 17792 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:56.854359 17778 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:56.991323 17644 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:56.857172 17797 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 10.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.184s
I20260709 12:11:56.931474 17819 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:11:57.035526 17717 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:11:57.034371 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:57.094676 17795 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:57.095194 17795 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:57.095664 17795 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:57.058571 17603 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:57.098526 17603 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:57.103833 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:57.104986 12673 raft_consensus.cc:2410] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
W20260709 12:11:57.094034 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 59782.4us
I20260709 12:11:57.048027 12496 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:57.079339 13336 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:57.112285 12679 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:11:57.090461 12680 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:57.091799 12494 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:57.115274 12494 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:57.069692 17740 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:57.037582 17696 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:57.117187 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:57.093752 17792 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:57.118942 17717 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:11:57.127332 17776 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:57.129365 17776 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:57.131778 17776 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:57.121114 12494 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:57.117475 17797 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:57.126536 17707 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:11:57.121397 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 73440.9us
I20260709 12:11:57.106601 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:57.133854 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:57.106534 17791 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.023s
I20260709 12:11:57.146427 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:57.146914 12673 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:57.148213 12673 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
I20260709 12:11:57.126142 17816 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:57.149780 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:57.150580 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:57.168344 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:57.169071 12527 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:57.149444 17580 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:57.149466 17717 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260709 12:11:57.170997 17580 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:57.168296 17708 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:11:57.157766 17647 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:11:57.161252 17819 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
W20260709 12:11:57.152288 12494 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.080s (SENT) [suppressed 11 similar messages]
I20260709 12:11:57.145895 17797 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:57.167618 12496 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:57.167932 12523 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: [13.11-15.12]
I20260709 12:11:57.169414 13336 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:11:57.166649 17793 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:57.144438 17778 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:57.205089 17793 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:57.205639 17793 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:57.208441 17781 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:57.188176 17800 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.716s	user 0.002s	sys 0.000s
I20260709 12:11:57.150683 17756 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:57.194593 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:57.217465 12523 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:57.195633 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:57.218634 12529 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:11:57.220556 12529 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:57.198061 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:57.223464 12522 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:57.162657 17740 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:57.132872 17626 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:57.192018 12496 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:57.211609 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:57.195245 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:57.226370 17778 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:57.226984 17778 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:57.228713 17392 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:57.229512 17392 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:57.151196 17707 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:57.228895 12496 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:57.173823 17497 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:57.229964 17392 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:57.237942 12528 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:11:57.232795 12523 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:11:57.173859 17708 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:57.216975 12678 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:57.237066 17717 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:57.251389 12523 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:57.210985 17797 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:57.288481 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:57.289484 12523 raft_consensus.cc:2368] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13.
W20260709 12:11:57.257062 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 104798us
I20260709 12:11:57.293181 12678 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:57.231377 17740 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:57.296839 12528 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:11:57.235697 17707 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:57.233659 17644 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:57.257897 17744 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.072s	user 0.011s	sys 0.016s
I20260709 12:11:57.300731 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:57.174032 17580 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:57.316077 17580 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:57.246690 17768 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:57.308506 12572 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:57.309979 17457 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:11:57.299038 17740 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:57.313329 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2394) took 2561 ms. Trace:
I20260709 12:11:57.169799 17696 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:57.319860 12572 leader_election.cc:400] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13.
I20260709 12:11:57.346802 12572 leader_election.cc:403] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Cancelling election due to peer responding with higher term
I20260709 12:11:57.323779 12671 rpcz_store.cc:276] 0709 12:11:54.751665 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:55.030775 (+279110us) service_pool.cc:224] Handling call
0709 12:11:55.031108 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:55.296976 (+265868us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:55.297008 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:55.297051 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:55.297092 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:57.312972 (+2015880us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54000ac8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:57.313039 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:57.313232 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":610851584}
I20260709 12:11:57.348320 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:57.349144 12671 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:57.350064 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:57.360708 12671 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:11:57.306103 17744 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.133s	user 0.012s	sys 0.016s
W20260709 12:11:57.367954 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 48244.5us
I20260709 12:11:57.318092 17813 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:57.320361 17457 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:11:57.318348 17768 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:57.329208 17647 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:57.319242 17778 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:57.389580 17644 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:57.371456 17774 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:57.369568 17723 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:57.384563 17649 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:57.401062 12671 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:11:57.384382 17693 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:57.400610 17768 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:57.402474 17647 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:57.366829 17772 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:11:57.368060 17717 raft_consensus.cc:697] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:57.389065 17792 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:57.327769 17376 consensus_queue.cc:260] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:57.412240 17793 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:57.418063 12601 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 2641) took 2392 ms (client timeout 2080 ms). Trace:
I20260709 12:11:57.417101 17819 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:57.425487 12496 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:11:57.455466 12496 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:57.456533 12496 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:57.457231 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:57.431243 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:57.460240 12526 raft_consensus.cc:2393] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 14: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:57.435565 12679 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:11:57.445894 12671 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:11:57.416975 17719 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:11:57.452680 12601 rpcz_store.cc:269] 0709 12:11:55.023447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:55.024335 (+   888us) service_pool.cc:224] Handling call
0709 12:11:57.415112 (+2390777us) spinlock_profiling.cc:243] Waited 577 ms on lock 0x7b54001946d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:57.415277 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1326857984}
I20260709 12:11:57.401671 17457 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:57.465025 12496 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:57.430354 17647 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:11:57.467783 12526 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2357) took 1704 ms (client timeout 2036 ms). Trace:
I20260709 12:11:57.394490 17708 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 14.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:57.416329 17644 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:57.469905 12526 rpcz_store.cc:269] 0709 12:11:55.763094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.430951 (+1667857us) service_pool.cc:224] Handling call
0709 12:11:57.467720 (+ 36769us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:11:57.418238 17723 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:57.419580 17649 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:57.451877 17772 consensus_queue.cc:1243] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:11:57.466395 17670 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:57.417346 17774 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:57.506754 12604 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:57.476740 17376 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:57.538561 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:57.539378 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:57.539803 12606 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 15: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:57.531589 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:57.540601 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:57.458925 17792 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:57.490571 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:57.491622 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:57.509105 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:57.515018 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:57.521234 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:57.531540 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:57.533366 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:57.582165 12600 raft_consensus.cc:2368] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13.
I20260709 12:11:57.531544 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:57.582234 12674 raft_consensus.cc:2376] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14. Re-sending same reply.
I20260709 12:11:57.469521 17837 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:57.489169 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:57.541751 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:57.540632 13339 raft_consensus.cc:2368] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16.
I20260709 12:11:57.537619 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:57.572824 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:57.516525 17826 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
W20260709 12:11:57.584340 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:57.584383 12604 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:11:57.583145 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2760) took 1174 ms. Trace:
W20260709 12:11:57.574213 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:57.544974 13338 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 13: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:57.514863 17776 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:57.599113 12603 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 14: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:57.580894 12601 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:57.602010 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:57.604934 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:57.588662 12674 rpcz_store.cc:276] 0709 12:11:56.409188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:56.409310 (+   122us) service_pool.cc:224] Handling call
0709 12:11:57.491408 (+1082098us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b58001f2c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:11:57.583067 (+ 91659us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3817216}
W20260709 12:11:57.623883 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:57.626027 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:57.627300 12602 raft_consensus.cc:2368] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15.
I20260709 12:11:57.469965 17819 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:57.634896 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:57.651733 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:57.670444 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:57.573489 17813 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:57.651068 13331 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:11:57.674952 12678 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:11:57.676721 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2391) took 2934 ms. Trace:
I20260709 12:11:57.677260 12678 rpcz_store.cc:276] 0709 12:11:54.742680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:55.022558 (+279878us) service_pool.cc:224] Handling call
0709 12:11:55.035996 (+ 13438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:57.216566 (+2180570us) spinlock_profiling.cc:243] Waited 3.03 ms on lock 0x7b4c00332c80. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1486ceca20 00007f1486cfda29 00007f1486cc3644 00007f1486ceaf2d 00007f1486ceab93 00007f1488a1b18b 00007f1488a1bcb5 00007f1488a16f30 00007f1488a187de 00007f1488a22b49 00007f1488a23416
0709 12:11:57.676329 (+459763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:57.676663 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6967552}
W20260709 12:11:57.680644 17831 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.228s	user 0.001s	sys 0.000s
W20260709 12:11:57.683105 17829 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.281s	user 0.001s	sys 0.000s
I20260709 12:11:57.683106 17840 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.004s
I20260709 12:11:57.686982 17840 consensus_queue.cc:1243] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 14.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:11:57.688172 17740 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:57.690605 17740 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:57.699707 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:57.702502 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:57.706430 17747 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 12:11:57.713245 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:57.758891 12605 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:11:57.766426 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:57.767227 12605 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:57.762431 17805 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:57.772605 17805 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:57.774448 17696 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:57.775060 17805 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:57.776399 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 311455us
I20260709 12:11:57.777107 12496 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:57.540706 17717 consensus_queue.cc:237] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:57.784034 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:57.620754 17776 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:57.799135 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:57.799656 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:57.794108 17670 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:11:57.764029 17817 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
I20260709 12:11:57.693738 17797 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:57.820840 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:57.853973 17670 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:57.856222 12597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 2643) took 2801 ms (client timeout 2239 ms). Trace:
W20260709 12:11:57.856647 12597 rpcz_store.cc:269] 0709 12:11:55.054821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:55.055084 (+   263us) service_pool.cc:224] Handling call
0709 12:11:57.855977 (+2800893us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b54002706d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:57.856138 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1632241024}
I20260709 12:11:57.858382 12597 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 11 index: 5. Preceding OpId from leader: term: 15 index: 7. (index mismatch)
W20260709 12:11:57.858709 17835 log.cc:927] Time spent T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.259s	user 0.001s	sys 0.000s
I20260709 12:11:57.858994 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2764) took 1372 ms. Trace:
I20260709 12:11:57.859403 12672 rpcz_store.cc:276] 0709 12:11:56.486754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:56.488335 (+  1581us) service_pool.cc:224] Handling call
0709 12:11:57.858832 (+1370497us) spinlock_profiling.cc:243] Waited 637 ms on lock 0x7b540008eb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:57.858937 (+   105us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1464442752}
W20260709 12:11:57.854688 17838 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.223s	user 0.001s	sys 0.000s
I20260709 12:11:57.636372 17457 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:57.843600 17839 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.081s	user 0.001s	sys 0.000s
W20260709 12:11:57.854074 17834 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
I20260709 12:11:57.887411 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:57.886770 17795 consensus_meta.cc:287] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.787s	user 0.010s	sys 0.004s
I20260709 12:11:57.908880 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:57.909087 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:57.910386 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 133355us
I20260709 12:11:57.911113 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:57.912830 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:57.880769 17844 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:57.915640 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:57.919337 12679 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:57.920303 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:57.921389 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:57.911996 12605 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
I20260709 12:11:57.931866 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:11:57.878247 12643 connection.cc:256] RPC call timeout handler was delayed by -1.8826587519999975s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1517s for a response
I20260709 12:11:57.935914 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:57.984661 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:57.935436 13334 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 13: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:57.938385 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:57.950016 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:57.996572 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:57.937492 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:57.999148 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:57.999246 12672 raft_consensus.cc:2368] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13.
I20260709 12:11:57.944268 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:57.998307 13334 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: []
W20260709 12:11:58.002688 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:57.940447 12522 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.002s	sys 0.013s
I20260709 12:11:57.937654 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:58.003428 12522 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.002s	sys 0.014s
I20260709 12:11:58.003783 12671 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:11:57.939534 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:57.993685 17705 consensus_meta.cc:287] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.996s	user 0.013s	sys 0.000s
I20260709 12:11:58.007151 12678 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:11:58.083669 12678 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:57.999677 12605 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:58.022861 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:58.084949 12672 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:58.093681 12672 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:11:58.033545 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2772) took 1488 ms. Trace:
I20260709 12:11:58.120333 12676 rpcz_store.cc:276] 0709 12:11:56.545215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:56.585148 (+ 39933us) service_pool.cc:224] Handling call
0709 12:11:58.033312 (+1448164us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b54003d40d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:58.033477 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434478976}
I20260709 12:11:58.035216 12646 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:58.037612 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:58.041127 17644 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:58.043277 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:58.123581 17644 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:58.124183 17644 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:58.055721 17843 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 12:11:58.126420 17644 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:11:58.049587 17832 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.758s	user 0.002s	sys 0.000s
I20260709 12:11:58.078240 17797 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:58.095218 17819 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:58.027314 17645 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.948s	user 0.007s	sys 0.005s
I20260709 12:11:58.100023 17553 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:58.102231 17840 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:11:58.005650 17626 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:58.104007 17776 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:58.121978 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 86930.5us
I20260709 12:11:57.985080 17723 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:58.050709 17749 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:58.013821 17707 consensus_meta.cc:287] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.690s	user 0.010s	sys 0.000s
I20260709 12:11:58.127638 17744 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:11:58.127943 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2758) took 1719 ms. Trace:
I20260709 12:11:58.059937 17792 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:11:58.280329 17376 consensus_meta.cc:287] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.724s	user 0.007s	sys 0.004s
I20260709 12:11:58.190474 17553 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:58.285310 17553 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:58.271341 12677 rpcz_store.cc:276] 0709 12:11:56.408483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:56.408596 (+   113us) service_pool.cc:224] Handling call
0709 12:11:57.472720 (+1064124us) spinlock_profiling.cc:243] Waited 3.34 ms on lock 0x7b58001f2c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:11:57.472932 (+   212us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:57.475945 (+  3013us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:57.475980 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:57.478663 (+  2683us) log.cc:844] Serialized 26 byte log entry
0709 12:11:57.721419 (+242756us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:57.721465 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:57.721543 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:58.127324 (+405781us) raft_consensus.cc:1685] finished
0709 12:11:58.127358 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:58.127827 (+   469us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2536,"raft.queue_time_us":3220,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":5005,"spinlock_wait_cycles":7689984,"thread_start_us":239903,"threads_started":1,"wal-append.queue_time_us":362719}
W20260709 12:11:58.073391 17647 consensus_meta.cc:287] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.593s	user 0.003s	sys 0.006s
I20260709 12:11:58.271729 17749 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:58.286319 17707 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:58.130846 12602 raft_consensus.cc:2410] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 16 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:11:58.287799 12526 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:58.135254 12598 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:11:57.914996 17795 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:58.143719 12529 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:11:57.936827 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:58.150470 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:58.083263 17834 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 12:11:58.152823 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:58.296609 13334 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 13: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:58.153384 12605 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
W20260709 12:11:58.178145 12530 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s	user 0.024s	sys 0.061s
W20260709 12:11:58.297498 12530 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.024s	sys 0.061s
I20260709 12:11:58.298283 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2379) took 1002 ms. Trace:
I20260709 12:11:58.298774 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:58.298655 12530 rpcz_store.cc:276] 0709 12:11:57.295945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.297563 (+  1618us) service_pool.cc:224] Handling call
0709 12:11:57.297945 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:57.494626 (+196681us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:57.494674 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:57.495335 (+   661us) log.cc:844] Serialized 26 byte log entry
0709 12:11:58.297915 (+802580us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:58.297957 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:58.298030 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:58.298078 (+    48us) raft_consensus.cc:1685] finished
0709 12:11:58.298089 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:58.298229 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3493,"raft.queue_time_us":32090,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":20019,"thread_start_us":866607,"threads_started":2,"wal-append.queue_time_us":738469}
I20260709 12:11:58.172686 17837 raft_consensus.cc:697] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:11:58.187206 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:57.944407 17649 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:57.934741 17844 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:58.199896 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:11:58.222694 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:58.318318 12646 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:58.319281 12646 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:58.225390 17840 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:11:58.247440 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 1.16926e+06us
W20260709 12:11:58.326647 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:58.327024 12522 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:11:58.250146 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:58.327487 12530 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:11:58.190218 17645 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:11:58.187821 17779 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.711s	user 0.031s	sys 0.067s
W20260709 12:11:58.188091 17816 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.022s	user 0.014s	sys 0.000s
I20260709 12:11:58.271134 17744 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:58.341707 17840 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:58.343200 17744 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:58.343670 17644 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:58.344141 17707 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:58.187937 17774 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:58.275063 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:58.295019 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:58.348446 12529 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:58.356259 13305 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:58.356709 12529 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
I20260709 12:11:58.322777 12678 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 9. (index mismatch)
I20260709 12:11:58.323884 12526 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 12 index: 11. (index mismatch)
W20260709 12:11:58.248999 17828 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.705s	user 0.001s	sys 0.001s
I20260709 12:11:58.357785 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:58.358086 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:58.358505 12529 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 8.
I20260709 12:11:58.358714 13332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 678) took 1206 ms. Trace:
W20260709 12:11:58.289987 17580 consensus_meta.cc:287] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.973s	user 0.013s	sys 0.001s
I20260709 12:11:58.328665 12530 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:11:58.330078 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:58.361358 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 435619us
W20260709 12:11:58.361871 12571 connection.cc:256] RPC call timeout handler was delayed by -1.6644610690000263s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16770000000000002s for a response
I20260709 12:11:58.332732 12569 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:11:58.362774 12571 connection.cc:256] RPC call timeout handler was delayed by -1.5866264789999605s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2142s for a response
I20260709 12:11:58.359419 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:58.339694 13336 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:58.341569 17807 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:58.366372 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:11:58.346836 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:58.344551 17744 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:58.359288 17553 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:11:58.359426 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:58.360666 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:58.359143 13332 rpcz_store.cc:276] 0709 12:11:57.152294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.152419 (+   125us) service_pool.cc:224] Handling call
0709 12:11:57.154397 (+  1978us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:57.154803 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:57.154834 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:57.155491 (+   657us) log.cc:844] Serialized 52 byte log entry
0709 12:11:57.367148 (+211657us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:57.367192 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:57.367289 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:58.248316 (+881027us) spinlock_profiling.cc:243] Waited 32.1 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:11:58.358392 (+110076us) raft_consensus.cc:1685] finished
0709 12:11:58.358416 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:58.358653 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7554,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":116,"spinlock_wait_cycles":73941760,"thread_start_us":211255,"threads_started":1,"wal-append.queue_time_us":298289}
W20260709 12:11:58.364533 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:58.365839 12646 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:58.331835 17779 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.856s	user 0.033s	sys 0.072s
I20260709 12:11:58.286322 17647 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:11:58.318962 17844 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:58.348737 17816 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:58.374281 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:58.422720 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:58.367967 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:58.404269 12526 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 13: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:58.330830 17649 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:11:58.405831 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:58.416376 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:58.417538 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:11:58.413611 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 473537us
I20260709 12:11:58.423347 12530 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:58.388103 17773 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:11:58.368188 17705 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:58.368014 17775 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:11:58.412014 17693 consensus_meta.cc:287] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.988s	user 0.012s	sys 0.001s
W20260709 12:11:58.413833 17603 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.315s	user 0.009s	sys 0.009s
I20260709 12:11:58.385666 17749 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:58.439554 12601 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 15: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:11:58.413635 17774 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:11:58.345655 17761 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.075s
I20260709 12:11:58.440294 12531 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:58.432381 17649 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:11:58.436105 17392 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.148s	user 0.009s	sys 0.006s
I20260709 12:11:58.413467 17779 raft_consensus.cc:697] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:58.411044 17807 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:58.444821 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:58.433987 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:58.540333 12529 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:58.541520 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:58.542243 12529 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:11:58.543216 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:58.543854 12529 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:58.434813 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:58.435571 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2368) took 1240 ms. Trace:
I20260709 12:11:58.546115 12526 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 14: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:11:58.547374 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:58.548040 12526 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 14: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:11:58.548882 12526 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:58.545555 12524 rpcz_store.cc:276] 0709 12:11:57.195515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.195785 (+   270us) service_pool.cc:224] Handling call
0709 12:11:57.196205 (+   420us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:57.196488 (+   283us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:57.196526 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:57.222450 (+ 25924us) log.cc:844] Serialized 26 byte log entry
0709 12:11:57.419085 (+196635us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:57.448914 (+ 29829us) log.cc:844] Serialized 14 byte log entry
0709 12:11:57.449149 (+   235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:57.449228 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:11:58.052825 (+603597us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b180034bfa0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:11:58.319884 (+267059us) spinlock_profiling.cc:243] Waited 31.9 ms on lock 0x7b4800002958. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:11:58.435267 (+115383us) raft_consensus.cc:1685] finished
0709 12:11:58.435298 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:58.435494 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29147,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":2293,"raft.queue_time_us":540,"raft.run_cpu_time_us":3073,"raft.run_wall_time_us":276960,"spinlock_wait_cycles":78999808,"thread_start_us":123081,"threads_started":1,"wal-append.queue_time_us":240614}
I20260709 12:11:58.557888 12530 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 13. Current term is 14. Ops: [13.7-13.7]
I20260709 12:11:58.432484 17776 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 12:11:58.421584 17553 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:58.438048 12646 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:58.435868 17497 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.192s	user 0.005s	sys 0.010s
W20260709 12:11:58.512173 13304 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.872s	user 0.124s	sys 0.200s
W20260709 12:11:58.623852 13304 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.984s	user 0.124s	sys 0.200s
W20260709 12:11:58.624361 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 984632us
I20260709 12:11:58.501577 17773 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:11:58.628749 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 190782us
W20260709 12:11:58.630009 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:58.630535 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:11:58.631100 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:58.450351 17693 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election lost for term 12. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13.
W20260709 12:11:58.574716 12496 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.663s	user 0.049s	sys 0.114s
I20260709 12:11:58.446341 17705 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:58.606851 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:58.636224 12679 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
W20260709 12:11:58.538990 17696 consensus_meta.cc:287] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.764s	user 0.005s	sys 0.009s
I20260709 12:11:58.504397 17774 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:58.451831 17603 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:58.509723 17649 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:11:58.431187 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:11:58.446184 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 312021us
W20260709 12:11:58.652606 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:58.653147 12494 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:58.558387 17768 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.096s	user 0.010s	sys 0.003s
I20260709 12:11:58.625978 12529 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
W20260709 12:11:58.632754 12496 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.722s	user 0.049s	sys 0.114s
W20260709 12:11:58.655212 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 744118us
I20260709 12:11:58.449272 17775 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:58.633854 12646 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:58.630575 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:11:58.433440 12643 connection.cc:256] RPC call timeout handler was delayed by -1.8691587519999757s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1532s for a response
I20260709 12:11:58.657182 12646 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:11:58.657482 12643 connection.cc:256] RPC call timeout handler was delayed by -1.3768587519998619s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20790000000000003s for a response
I20260709 12:11:58.658598 12597 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:11:58.658845 12530 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 13: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:11:58.658241 12643 connection.cc:256] RPC call timeout handler was delayed by -1.2214737069998591s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1522s for a response
W20260709 12:11:58.659998 12643 connection.cc:256] RPC call timeout handler was delayed by -1.1377737069997238s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
W20260709 12:11:58.660488 12643 connection.cc:256] RPC call timeout handler was delayed by -1.1377737069997238s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1615s for a response
I20260709 12:11:58.663064 17553 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:58.671869 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:58.681855 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2356) took 2919 ms. Trace:
I20260709 12:11:58.688632 12531 rpcz_store.cc:276] 0709 12:11:55.762712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.311774 (+1549062us) service_pool.cc:224] Handling call
0709 12:11:57.312245 (+   471us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:58.505051 (+1192806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:58.681610 (+176559us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400407754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:58.681790 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2589671168}
I20260709 12:11:58.674075 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:58.686180 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:58.702380 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49331.6us
I20260709 12:11:58.706498 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:58.684978 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:58.438813 17837 consensus_queue.cc:237] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:58.742338 12528 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:58.554050 17392 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:58.650629 17774 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:11:58.749157 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:58.717563 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:58.750223 12494 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:58.755870 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 399653us
I20260709 12:11:58.641036 17723 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:58.735729 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:58.764961 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 107876us
I20260709 12:11:58.686836 17776 raft_consensus.cc:2764] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election decision vote started in defunct term 12: won
I20260709 12:11:58.639453 17760 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.775600 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:58.712548 17836 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.069s
W20260709 12:11:58.751322 17691 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.017s	sys 0.054s
I20260709 12:11:58.752514 12599 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:11:58.640712 17457 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.834399 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:58.835930 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:58.712278 17376 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:11:58.837366 12600 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:58.683729 17644 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:58.845541 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:58.845543 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:11:58.835243 17691 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.593s	user 0.017s	sys 0.055s
I20260709 12:11:58.778359 17723 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:58.778275 17771 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.779021 17497 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:11:58.776052 17795 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:58.875198 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:58.712673 17717 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.135s
I20260709 12:11:58.881834 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:58.878042 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:11:58.882668 12606 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:58.883139 12599 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:11:58.694736 17768 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:58.843113 17645 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:58.877116 17497 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:58.848208 17457 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:58.856743 17816 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.877475 17723 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:58.838922 17760 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:58.919282 17816 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:58.683535 17807 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:58.935252 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:11:58.936163 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46155us
I20260709 12:11:58.947661 12571 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:58.933346 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 172381us
I20260709 12:11:58.882853 17647 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:11:58.891134 17853 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:58.887460 17376 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:11:58.892254 17776 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:58.913448 17497 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:11:58.756028 17779 consensus_queue.cc:237] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:58.952739 12606 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:58.955631 12599 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:11:58.839959 17773 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:58.937021 17645 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:59.012593 17645 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:58.956871 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2782) took 2112 ms. Trace:
I20260709 12:11:58.884779 17771 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:58.975069 12606 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:59.011169 12571 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:59.010605 12599 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:11:59.021643 12599 pending_rounds.cc:85] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 11
I20260709 12:11:59.015223 12673 rpcz_store.cc:276] 0709 12:11:56.844516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.164505 (+319989us) service_pool.cc:224] Handling call
0709 12:11:57.164804 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:57.170227 (+  5423us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:11:57.352318 (+182091us) log.cc:844] Serialized 14 byte log entry
0709 12:11:58.956161 (+1603843us) spinlock_profiling.cc:243] Waited 1.69 ms on lock 0x7b1800476860. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a
0709 12:11:58.956422 (+   261us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:58.956497 (+    75us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:58.956559 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:58.956623 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:58.956785 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4378,"prepare.queue_time_us":253606,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":2244,"spinlock_wait_cycles":3892480,"thread_start_us":237089,"threads_started":2,"wal-append.queue_time_us":266139,"wal-append.run_cpu_time_us":2353,"wal-append.run_wall_time_us":1256904}
I20260709 12:11:58.949883 17836 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:59.021917 12599 pending_rounds.cc:107] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:11:59.023936 12599 raft_consensus.cc:2889] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:58.961393 17797 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:11:58.975719 17865 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:59.028707 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2415) took 1102 ms. Trace:
I20260709 12:11:59.029136 12679 rpcz_store.cc:276] 0709 12:11:57.926664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.935642 (+  8978us) service_pool.cc:224] Handling call
0709 12:11:59.028535 (+1092893us) spinlock_profiling.cc:243] Waited 695 ms on lock 0x7b54001e2154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:59.028647 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1597902720}
I20260709 12:11:58.947322 17708 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.067s
I20260709 12:11:59.051373 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:11:59.011348 17691 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:11:58.976127 17776 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:59.047982 17852 raft_consensus.cc:2764] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:11:59.025799 17603 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:58.965667 17853 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:59.113510 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 61091.1us
W20260709 12:11:59.125802 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 341979us
I20260709 12:11:59.046567 17705 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:59.114172 17855 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.363s	user 0.001s	sys 0.001s
I20260709 12:11:59.019894 17674 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.099s
I20260709 12:11:59.108564 17775 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:59.140080 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 190125us
W20260709 12:11:59.116420 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 366262us
I20260709 12:11:59.149889 12494 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:59.122053 17864 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 12:11:59.159158 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 2661) took 1457 ms. Trace:
I20260709 12:11:59.172376 12600 rpcz_store.cc:276] 0709 12:11:57.738215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.740237 (+  2022us) service_pool.cc:224] Handling call
0709 12:11:59.158896 (+1418659us) spinlock_profiling.cc:243] Waited 875 ms on lock 0x7b54002e63d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:11:59.159062 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2013587200}
I20260709 12:11:59.163463 17793 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:59.165661 12598 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 13. Current term is 14. Ops: []
W20260709 12:11:59.169986 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:59.052726 17836 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:59.185976 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:59.124478 17392 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:59.183960 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:11:59.187614 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:59.121661 17691 raft_consensus.cc:697] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:59.196986 12675 pending_rounds.cc:85] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 5
W20260709 12:11:59.184465 17856 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.325s	user 0.000s	sys 0.002s
I20260709 12:11:59.215924 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:59.216325 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:59.222950 12598 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 13. Current term is 14. Ops: []
I20260709 12:11:59.216673 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:59.227932 12674 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:11:59.217821 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:59.229192 12672 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:59.218228 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:59.231508 12671 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 13
I20260709 12:11:59.213532 12675 pending_rounds.cc:107] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 14.6
I20260709 12:11:59.232287 12675 raft_consensus.cc:2889] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:11:59.235250 12494 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:59.223266 17747 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:59.239780 12673 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 13: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:11:59.241025 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:11:59.049856 17865 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:59.262192 12676 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 13: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:11:59.246158 13304 leader_election.cc:336] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.814s (SENT)
I20260709 12:11:59.190510 17376 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:11:59.211014 17857 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.325s	user 0.000s	sys 0.001s
I20260709 12:11:59.219038 17840 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:59.281253 17840 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:59.282027 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 100350us
I20260709 12:11:59.283283 17873 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:59.283936 17873 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:59.284415 17873 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:11:59.285205 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:11:59.285797 17863 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.260s	user 0.001s	sys 0.000s
I20260709 12:11:59.286212 17696 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:11:59.286880 17866 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
W20260709 12:11:59.287207 17870 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 12:11:59.285476 17852 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:11:59.215380 17603 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:59.210927 17878 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:59.259011 17392 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:11:59.266094 17858 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.350s	user 0.001s	sys 0.000s
I20260709 12:11:59.293449 12571 leader_election.cc:400] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13.
I20260709 12:11:59.307271 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:11:59.294109 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 148776us
I20260709 12:11:59.309731 12646 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:59.311896 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2363) took 2139 ms. Trace:
I20260709 12:11:59.312470 12527 rpcz_store.cc:276] 0709 12:11:57.172485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.174453 (+  1968us) service_pool.cc:224] Handling call
0709 12:11:57.174766 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:57.174969 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:11:57.175292 (+   323us) log.cc:844] Serialized 14 byte log entry
0709 12:11:58.874336 (+1699044us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b1800262e40. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287
0709 12:11:59.127110 (+252774us) log.cc:844] Serialized 14 byte log entry
0709 12:11:59.311143 (+184033us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:11:59.311322 (+   179us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:11:59.311400 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:59.311592 (+   192us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:11:59.311777 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6799,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":23016,"raft.run_cpu_time_us":155,"raft.run_wall_time_us":20931,"spinlock_wait_cycles":6662400,"thread_start_us":627802,"threads_started":4,"wal-append.queue_time_us":191353,"wal-append.run_cpu_time_us":2151,"wal-append.run_wall_time_us":1315229}
W20260709 12:11:59.290300 17862 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.347s	user 0.001s	sys 0.000s
I20260709 12:11:59.348515 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:11:59.353215 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 199060us
I20260709 12:11:59.355008 17553 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:11:59.358611 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:11:59.359648 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 118698us
I20260709 12:11:59.364409 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:59.362468 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 138612us
I20260709 12:11:59.366429 12643 leader_election.cc:400] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16.
I20260709 12:11:59.367761 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:59.374197 17553 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:59.375051 17553 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 13 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:11:59.376430 17553 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:59.388154 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:11:59.393529 17553 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:11:59.403120 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 93451.9us
I20260709 12:11:59.436326 12672 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:59.438079 12527 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:11:59.441744 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:59.442204 12671 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:11:59.413710 12643 leader_election.cc:400] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16.
I20260709 12:11:59.443373 12527 raft_consensus.cc:2410] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 13 because replica has last-logged OpId of term: 11 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
W20260709 12:11:59.446344 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 44450.5us
I20260709 12:11:59.444320 12674 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:11:59.432737 12676 raft_consensus.cc:2410] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:11:59.448285 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:59.346879 17392 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:11:59.452427 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 145281us
I20260709 12:11:59.458732 12571 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:11:59.244809 17874 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.105s
I20260709 12:11:59.294910 17884 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:11:59.466405 12571 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:59.265785 17747 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:59.452070 17696 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:59.473822 17696 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:59.474898 17868 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.287s	user 0.000s	sys 0.000s
I20260709 12:11:59.475100 17747 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:59.479429 17876 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:11:59.481279 17836 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:59.482009 17903 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:59.482614 17903 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:11:59.483518 17903 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:59.345510 17792 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:59.427538 17813 raft_consensus.cc:3055] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:11:59.495599 12571 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:11:59.510614 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:11:59.430035 17776 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:59.429093 17865 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:11:59.508606 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 103439us
I20260709 12:11:59.299031 17603 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:59.466693 17693 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:11:59.634609 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 270320us
I20260709 12:11:59.636494 12494 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:59.636710 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:59.637717 12529 raft_consensus.cc:2410] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 16 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
W20260709 12:11:59.523092 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 74910.8us
I20260709 12:11:59.639005 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:59.639310 12522 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:11:59.639824 12643 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:59.642149 12527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:11:59.642983 12527 raft_consensus.cc:606] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:59.643431 12527 raft_consensus.cc:614] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:11:59.639705 12530 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:11:59.650689 12529 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13. Ops: []
I20260709 12:11:59.652431 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:59.653970 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:59.655426 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:59.544948 12674 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13. Ops: []
W20260709 12:11:59.522472 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 325541us
I20260709 12:11:59.464898 17392 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:59.568714 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:59.295490 17875 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.209s
I20260709 12:11:59.545958 12671 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
I20260709 12:11:59.546654 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:59.487885 17708 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:11:59.631299 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:59.345783 17878 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:59.704907 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:59.504067 17647 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:59.657757 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:59.638594 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:59.707695 12522 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:59.708101 12531 raft_consensus.cc:2410] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:11:59.684609 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:59.709194 12530 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:59.662251 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:59.487310 17457 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:59.525350 17705 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:59.502827 17792 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:59.608527 17795 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:11:59.726461 17795 raft_consensus.cc:697] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:11:59.726751 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 87071.6us
I20260709 12:11:59.523913 17747 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:11:59.727720 17747 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:11:59.727445 17795 consensus_queue.cc:237] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:59.728056 17816 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:11:59.728209 12643 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:11:59.730645 17875 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:11:59.731266 17875 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:11:59.471894 17884 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:59.523582 17805 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:11:59.699888 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:59.728164 17747 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:11:59.704659 17708 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:59.465703 17691 consensus_queue.cc:237] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:11:59.718042 17844 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.270s	user 0.014s	sys 0.000s
I20260709 12:11:59.630646 17580 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:59.504122 17813 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:11:59.706761 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:59.732096 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:11:59.753729 17744 consensus_meta.cc:287] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.350s	user 0.008s	sys 0.003s
I20260709 12:11:59.715551 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:59.758826 12672 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:11:59.622640 17771 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:59.708101 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:59.692965 12676 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:11:59.644294 17693 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:11:59.649835 17853 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:59.601629 17819 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.103s
I20260709 12:11:59.755553 17769 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:11:59.756632 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2380) took 2456 ms. Trace:
I20260709 12:11:59.760478 12674 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:11:59.764613 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:59.720770 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:11:59.752077 12527 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
I20260709 12:11:59.856706 12531 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:59.734560 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:59.748241 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2668) took 1950 ms. Trace:
I20260709 12:11:59.768674 12524 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:11:59.856231 12528 rpcz_store.cc:276] 0709 12:11:57.300008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.303134 (+  3126us) service_pool.cc:224] Handling call
0709 12:11:57.345346 (+ 42212us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:58.746887 (+1401541us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:59.756358 (+1009471us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54002107d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:59.756557 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3212204416}
I20260709 12:11:59.859652 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:59.858536 12597 rpcz_store.cc:276] 0709 12:11:57.798272 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.859159 (+ 60887us) service_pool.cc:224] Handling call
0709 12:11:57.859451 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:11:58.659087 (+799636us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:59.747972 (+1088885us) spinlock_profiling.cc:243] Waited 788 ms on lock 0x7b5400271ad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:11:59.748176 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1812483968}
I20260709 12:11:59.721848 17773 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:11:59.790047 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 155597us
I20260709 12:11:59.846378 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:11:59.676730 17553 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 12:11:59.719211 17696 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:11:59.910704 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:11:59.732223 17884 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:59.925366 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:11:59.925976 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 51126.6us
I20260709 12:11:59.739379 17891 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:59.845299 17693 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:11:59.885213 12522 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
W20260709 12:11:59.840201 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 203753us
I20260709 12:11:59.736323 17708 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:11:59.721155 17705 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:11:59.738911 17649 consensus_meta.cc:287] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.055s	user 0.013s	sys 0.000s
I20260709 12:11:59.926040 12676 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:11:59.926856 12646 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:00.012754 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 85995.6us
W20260709 12:11:59.921579 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 189556us
I20260709 12:11:59.874471 12530 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:11:59.747768 17876 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:11:59.750594 17626 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:11:59.735924 17878 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:11:59.878756 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:00.038138 12672 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:11:59.842700 17892 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:11:59.768882 17875 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:11:59.910171 17870 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.150s	user 0.000s	sys 0.000s
I20260709 12:11:59.732417 17805 raft_consensus.cc:697] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:00.015857 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:00.017601 12530 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b825dff40dc4290a25d71534b598330"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
W20260709 12:12:00.018326 13305 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.008s	sys 0.007s
W20260709 12:12:00.063592 13305 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.008s	sys 0.007s
I20260709 12:12:00.062892 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:00.064918 12672 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:12:00.036734 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:00.041061 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:11:59.749198 17747 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:11:59.748538 17580 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:00.091223 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 54589.8us
I20260709 12:12:00.027961 17705 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:12:00.063532 12530 raft_consensus.cc:606] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:11:59.746766 17397 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.191s
W20260709 12:12:00.064035 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 619347us
I20260709 12:11:59.937762 17519 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.125s
I20260709 12:11:59.923570 17773 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:00.074615 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:00.129474 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:00.075688 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 59951.4us
I20260709 12:12:00.130384 12600 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 14: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:00.075176 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:00.053968 12679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.004s	sys 0.013s
W20260709 12:12:00.131644 12679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.004s	sys 0.013s
I20260709 12:12:00.036931 17626 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:00.096683 12530 raft_consensus.cc:614] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:12:00.100009 12494 connection.cc:256] RPC call timeout handler was delayed by -1.7843826470000295s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1569s for a response
I20260709 12:12:00.029080 17876 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:00.034801 12496 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:11:59.742946 17844 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:00.082072 17878 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:11:59.920011 17894 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.202s
I20260709 12:12:00.129699 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:00.131524 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:11:59.749503 17813 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:00.028542 17891 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:00.230010 17913 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:00.411319 12427 catalog_manager.cc:5697] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 10 to 12, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 12 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:00.455809 17813 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:00.455714 17913 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:00.224226 17904 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.089s	user 0.003s	sys 0.000s
I20260709 12:12:00.416769 12525 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13. Ops: []
W20260709 12:12:00.226589 17888 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.171s	user 0.001s	sys 0.000s
I20260709 12:12:00.195544 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:00.261013 17884 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:00.225272 17912 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
I20260709 12:12:00.244016 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 515) took 1623 ms. Trace:
I20260709 12:12:00.424522 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:00.463904 17647 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:00.462543 12599 rpcz_store.cc:276] 0709 12:11:58.621207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:59.025668 (+404461us) service_pool.cc:224] Handling call
0709 12:11:59.025995 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:59.048761 (+ 22766us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:59.048824 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:59.076582 (+ 27758us) log.cc:844] Serialized 26 byte log entry
0709 12:11:59.351287 (+274705us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:11:59.351352 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:59.351452 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:00.243713 (+892261us) raft_consensus.cc:1685] finished
0709 12:12:00.243741 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:00.243940 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2974,"raft.queue_time_us":19819,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":57681,"thread_start_us":207191,"threads_started":1,"wal-append.queue_time_us":1120567}
I20260709 12:12:00.465334 12673 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11. Ops: [10.11-10.11]
I20260709 12:12:00.279722 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2802) took 1063 ms. Trace:
W20260709 12:12:00.425293 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 412893us
I20260709 12:12:00.166841 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:00.468329 12600 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 13: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:00.268803 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2797) took 1054 ms. Trace:
W20260709 12:12:00.316996 17392 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.594s	user 0.010s	sys 0.002s
I20260709 12:12:00.177814 13339 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
W20260709 12:12:00.407696 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 372704us
W20260709 12:12:00.234827 17497 consensus_meta.cc:287] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.229s	user 0.013s	sys 0.000s
I20260709 12:12:00.168951 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:00.472528 17497 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:00.474021 12601 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 2659) took 2749 ms (client timeout 1834 ms). Trace:
I20260709 12:12:00.036409 17649 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:00.469851 17392 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:00.474720 12601 rpcz_store.cc:269] 0709 12:11:57.701691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.712931 (+ 11240us) service_pool.cc:224] Handling call
0709 12:12:00.451129 (+2738198us) spinlock_profiling.cc:243] Waited 724 ms on lock 0x7b5400079c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:00.451474 (+   345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1665021696}
I20260709 12:12:00.477828 12522 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:00.478920 12678 raft_consensus.cc:2368] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14.
I20260709 12:12:00.479481 12674 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:00.437234 17519 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:12:00.440781 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:00.248961 17580 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:00.312913 17626 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:00.320964 17693 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:00.440680 17719 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:00.490868 17719 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:00.233441 17723 consensus_meta.cc:287] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.287s	user 0.010s	sys 0.003s
I20260709 12:12:00.492848 17723 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:00.497107 17723 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:00.016559 17769 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.079s
I20260709 12:12:00.017022 17771 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:00.501514 12604 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:00.502485 17797 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:00.503104 17884 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:00.244092 17774 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.397s	user 0.011s	sys 0.002s
I20260709 12:12:00.179239 17852 raft_consensus.cc:3055] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:12:00.074373 17892 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:00.227403 17893 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.062s
I20260709 12:12:00.224879 17894 raft_consensus.cc:3055] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:00.244585 17805 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 9.9, Last appended by leader: 9, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:00.179163 17807 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:00.537405 17894 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:00.144184 17816 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:00.063190 17853 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
I20260709 12:12:00.142658 17878 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:00.575867 17878 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:00.576220 17894 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:00.225270 17906 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:00.225674 17905 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:12:00.229104 17900 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:12:00.226959 17877 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:00.409761 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 566) took 1025 ms. Trace:
I20260709 12:12:00.142125 17876 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:12:00.169468 12530 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 10 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:12:00.630786 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:00.631798 12530 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:12:00.632261 17553 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:00.633002 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:00.259004 17844 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:00.457896 17891 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:00.634896 12674 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:00.204798 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:00.466532 12677 rpcz_store.cc:276] 0709 12:11:59.216623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:59.216788 (+   165us) service_pool.cc:224] Handling call
0709 12:11:59.217163 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:59.351723 (+134560us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:59.351771 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:59.352504 (+   733us) log.cc:844] Serialized 26 byte log entry
0709 12:11:59.386371 (+ 33867us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:11:59.386408 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:59.386481 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:00.279459 (+892978us) raft_consensus.cc:1685] finished
0709 12:12:00.279486 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:00.279646 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49519,"raft.queue_time_us":263846,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":108,"thread_start_us":117916,"threads_started":2,"wal-append.queue_time_us":921092}
I20260709 12:12:00.247175 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2803) took 1030 ms. Trace:
I20260709 12:12:00.644994 12680 rpcz_store.cc:276] 0709 12:11:59.216989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:59.217111 (+   122us) service_pool.cc:224] Handling call
0709 12:11:59.217404 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:11:59.360511 (+143107us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:59.360556 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:11:59.361565 (+  1009us) log.cc:844] Serialized 26 byte log entry
0709 12:11:59.373655 (+ 12090us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:11:59.373711 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:59.373814 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:00.246916 (+873102us) raft_consensus.cc:1685] finished
0709 12:12:00.246939 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:00.247108 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49664,"raft.queue_time_us":263890,"raft.run_cpu_time_us":60,"raft.run_wall_time_us":60,"thread_start_us":104438,"threads_started":2,"wal-append.queue_time_us":883117}
I20260709 12:12:00.472108 12496 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:00.649303 12603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 2666) took 2878 ms (client timeout 1569 ms). Trace:
I20260709 12:12:00.290514 12676 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:00.650897 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2427) took 1081 ms. Trace:
I20260709 12:12:00.651311 12676 rpcz_store.cc:276] 0709 12:11:59.569457 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:59.855944 (+286487us) service_pool.cc:224] Handling call
0709 12:12:00.650656 (+794712us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:00.651932 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 180167us
I20260709 12:12:00.439394 17670 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:00.656734 17670 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:00.277653 17705 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:00.658596 17670 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:00.315590 17708 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:00.662076 17705 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:00.664242 17708 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:00.440462 12523 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:00.469834 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:00.461732 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:00.465670 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:00.415915 12528 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:12:00.653391 12496 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:00.469064 12679 rpcz_store.cc:276] 0709 12:11:59.214438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:59.214635 (+   197us) service_pool.cc:224] Handling call
0709 12:11:59.215179 (+   544us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:59.266998 (+ 51819us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:59.267032 (+    34us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:59.290655 (+ 23623us) log.cc:844] Serialized 52 byte log entry
0709 12:12:00.132101 (+841446us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:00.132142 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:00.132210 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:00.268485 (+136275us) raft_consensus.cc:1685] finished
0709 12:12:00.268516 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:00.268704 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77306,"raft.queue_time_us":214793,"raft.run_cpu_time_us":205,"raft.run_wall_time_us":26547,"thread_start_us":681172,"threads_started":1,"wal-append.queue_time_us":921643}
I20260709 12:12:00.503178 17797 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:00.277511 17773 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:00.477730 17649 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:00.282534 17918 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:00.669564 17918 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:00.671139 12678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2800) took 1455 ms (client timeout 1811 ms). Trace:
W20260709 12:12:00.671665 12678 rpcz_store.cc:269] 0709 12:11:59.215887 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:59.216036 (+   149us) service_pool.cc:224] Handling call
0709 12:12:00.670866 (+1454830us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b54002c4f54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:00.671046 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2876787072}
I20260709 12:12:00.493907 17693 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:00.515408 17852 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:00.523236 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 49774.9us
I20260709 12:12:00.482828 17875 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:00.675674 17852 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:00.480620 17913 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:00.504720 17774 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:00.532775 17760 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:00.679142 12600 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
W20260709 12:12:00.550359 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 126266us
I20260709 12:12:00.553581 17626 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:00.547858 17392 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:00.560070 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 391048us
I20260709 12:12:00.552171 17497 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:00.584496 12494 connection.cc:256] RPC call timeout handler was delayed by -1.5458826470000986s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1834s for a response
I20260709 12:12:00.595728 17894 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:00.756943 12494 connection.cc:256] RPC call timeout handler was delayed by -1.4027826470000946s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19930000000000003s for a response
I20260709 12:12:00.758339 12494 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:00.761575 12676 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 8. Current term is 9. Ops: []
I20260709 12:12:00.601305 12529 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:00.765290 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:00.766921 17392 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:00.768134 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2796) took 1447 ms. Trace:
I20260709 12:12:00.768679 12675 rpcz_store.cc:276] 0709 12:11:59.195974 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:59.196205 (+   231us) service_pool.cc:224] Handling call
0709 12:11:59.196680 (+   475us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:59.320166 (+123486us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:59.320219 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:59.321070 (+   851us) log.cc:844] Serialized 52 byte log entry
0709 12:11:59.327340 (+  6270us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:11:59.327395 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:11:59.327501 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:00.618824 (+1291323us) raft_consensus.cc:1685] finished
0709 12:12:00.622782 (+  3958us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:00.643157 (+ 20375us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42086,"raft.queue_time_us":234138,"raft.run_cpu_time_us":49,"raft.run_wall_time_us":49,"thread_start_us":35979,"threads_started":2,"wal-append.queue_time_us":727653}
I20260709 12:12:00.600054 12526 rpcz_store.cc:276] 0709 12:11:59.384464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:59.384844 (+   380us) service_pool.cc:224] Handling call
0709 12:11:59.385383 (+   539us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:11:59.757394 (+372011us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:11:59.757425 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:11:59.775781 (+ 18356us) log.cc:844] Serialized 52 byte log entry
0709 12:12:00.143444 (+367663us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:00.408161 (+264717us) log.cc:844] Serialized 14 byte log entry
0709 12:12:00.408850 (+   689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:00.409073 (+   223us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:00.409222 (+   149us) raft_consensus.cc:1685] finished
0709 12:12:00.409251 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:00.409524 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":330237,"prepare.queue_time_us":309798,"prepare.run_cpu_time_us":184,"prepare.run_wall_time_us":180,"raft.queue_time_us":130942,"raft.run_cpu_time_us":412,"raft.run_wall_time_us":411,"spinlock_wait_cycles":28416,"thread_start_us":436850,"threads_started":2,"wal-append.queue_time_us":348247}
I20260709 12:12:00.628202 17645 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:00.632694 17905 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:12:00.214598 17873 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.922s	user 0.010s	sys 0.003s
W20260709 12:12:00.621140 13304 leader_election.cc:336] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.652s (SENT)
I20260709 12:12:00.643775 12530 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:12:00.644388 17761 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.248s	user 0.006s	sys 0.010s
I20260709 12:12:00.653373 17840 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:00.653769 17891 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:00.654351 12646 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:00.654569 12598 raft_consensus.cc:3055] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:00.247037 17747 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:00.649814 12603 rpcz_store.cc:269] 0709 12:11:57.770633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:57.777346 (+  6713us) service_pool.cc:224] Handling call
0709 12:12:00.648857 (+2871511us) spinlock_profiling.cc:243] Waited 2.26 s on lock 0x7b5400374954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:00.649092 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5206588160}
I20260709 12:12:00.756960 12643 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:00.669981 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:00.936108 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 282983us
W20260709 12:12:00.941234 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 260309us
I20260709 12:12:00.933236 12529 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:00.665606 12523 raft_consensus.cc:606] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:00.946429 12523 raft_consensus.cc:614] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:12:00.870437 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 140578us
W20260709 12:12:00.943929 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 185871us
I20260709 12:12:00.945437 12598 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:00.943679 17761 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.646s	user 0.011s	sys 0.018s
W20260709 12:12:00.960935 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 340241us
I20260709 12:12:00.944283 17852 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:00.790987 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:00.869266 17392 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:00.966320 17392 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:00.935770 17626 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:12:00.870100 17719 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:00.975220 13302 connection.cc:256] RPC call timeout handler was delayed by -1.0108376200000748s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15080000000000002s for a response
W20260709 12:12:00.979318 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 344120us
I20260709 12:12:00.978300 12528 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:00.940860 17693 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260709 12:12:00.912241 17876 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:00.973771 12494 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:00.983109 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:00.972640 17891 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:00.985069 17497 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:12:00.998001 12496 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:00.998708 17392 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:00.998155 17918 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:00.984256 17774 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:01.014586 12526 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:12:00.978111 17890 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.032s
I20260709 12:12:00.983707 17626 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:01.016256 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 44873.4us
I20260709 12:12:01.013377 12674 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:12:01.000741 17869 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 12:12:01.009100 17497 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:01.027994 12531 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.112s	user 0.011s	sys 0.015s
I20260709 12:12:01.012460 17645 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:01.005906 17807 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:01.030411 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:12:01.035014 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 278441us
I20260709 12:12:01.037469 17644 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:01.038466 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2391) took 1389 ms. Trace:
I20260709 12:12:01.038190 17644 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.038878 12525 rpcz_store.cc:276] 0709 12:11:59.649121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:59.804306 (+155185us) service_pool.cc:224] Handling call
0709 12:11:59.804626 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:00.458773 (+654147us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:01.020045 (+561272us) spinlock_profiling.cc:243] Waited 611 ms on lock 0x7b540042a9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:01.038381 (+ 18336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1406211328}
I20260709 12:12:01.039886 12531 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:01.031117 17497 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:01.033000 12598 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.049428 17719 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:01.077736 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2395) took 1390 ms. Trace:
I20260709 12:12:01.078223 12531 rpcz_store.cc:276] 0709 12:11:59.684822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:59.720489 (+ 35667us) service_pool.cc:224] Handling call
0709 12:12:01.072643 (+1352154us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:01.079407 17708 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:01.082520 12527 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:01.074800 17906 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:12:01.090029 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 88065.6us
I20260709 12:12:01.084596 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:01.084728 17875 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:01.094889 12494 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:01.100368 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:01.117331 12598 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:01.116449 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 63707.3us
I20260709 12:12:01.113216 17891 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.116335 17553 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:01.130897 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:01.140206 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 168109us
I20260709 12:12:01.142514 12525 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:12:01.109535 17869 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.028s
W20260709 12:12:01.149672 17944 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
W20260709 12:12:01.152639 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 78631.8us
I20260709 12:12:01.132884 17876 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:01.156682 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 153682us
I20260709 12:12:01.087568 17873 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:01.158264 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 97197.1us
I20260709 12:12:01.135177 17553 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:01.124233 17797 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.135730 17773 consensus_queue.cc:1059] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 9.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.035s
W20260709 12:12:01.163811 17950 log.cc:927] Time spent T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
W20260709 12:12:01.163981 17942 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.091s	user 0.000s	sys 0.001s
I20260709 12:12:01.166061 13337 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
W20260709 12:12:01.158214 17945 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:12:01.155144 17519 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:01.139503 17875 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:01.098661 17905 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:01.155882 17719 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:01.126099 17747 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.127s
I20260709 12:12:01.181015 17603 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:01.186635 12524 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 9. Current term is 10. Ops: []
I20260709 12:12:01.187778 17797 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:01.193645 17705 raft_consensus.cc:3055] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:12:01.194172 17705 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:01.194455 17768 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:01.174899 17781 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:12:01.196197 17781 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:01.196662 17781 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:01.198938 17644 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:01.138849 17947 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:12:01.205631 17792 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:01.171636 17519 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:01.137110 17626 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.211443 17873 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:01.222541 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:01.222744 17519 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.223433 12526 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:12:01.224380 12526 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:01.224572 17519 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:12:01.205978 17878 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.629s	user 0.004s	sys 0.008s
I20260709 12:12:01.231194 12528 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
W20260709 12:12:01.241382 17925 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.942s	user 0.019s	sys 0.000s
I20260709 12:12:01.242256 12530 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:12:01.175282 17719 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:01.249608 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 71574us
W20260709 12:12:01.268669 13302 leader_election.cc:336] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.652s (SENT)
I20260709 12:12:01.252656 17719 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:01.269341 13302 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:01.217223 17918 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.045s
W20260709 12:12:01.272217 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 117937us
I20260709 12:12:01.242209 17869 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:12:01.277055 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:01.279986 12526 raft_consensus.cc:606] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:01.283455 12529 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:01.195372 17705 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:01.284475 17392 raft_consensus.cc:2764] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election decision vote started in defunct term 13: won
I20260709 12:12:01.295576 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 572) took 1190 ms. Trace:
I20260709 12:12:01.296144 12524 rpcz_store.cc:276] 0709 12:12:00.105544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:00.107691 (+  2147us) service_pool.cc:224] Handling call
0709 12:12:00.108017 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:01.187348 (+1079331us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:01.295259 (+107911us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54003f6954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:01.295497 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2464292736}
I20260709 12:12:01.297441 17691 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:01.298298 17691 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.299506 12646 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:01.300525 12646 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:01.300727 17691 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:01.301698 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:01.302210 12677 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:01.303426 17644 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:12:01.304309 17644 consensus_queue.cc:1059] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:12:01.305256 17644 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:01.305605 17644 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:01.306406 17894 raft_consensus.cc:3055] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:12:01.306668 17899 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:12:01.306869 17894 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:01.307665 17899 raft_consensus.cc:697] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:01.308272 17894 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.309118 17899 consensus_queue.cc:237] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:01.284451 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 83267.8us
I20260709 12:12:01.256305 17878 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.311071 12646 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:01.319775 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 102182us
I20260709 12:12:01.281778 17947 raft_consensus.cc:3055] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:01.279771 17869 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:01.324805 12676 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:01.327957 17747 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:01.328459 17747 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:01.328859 17747 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:01.337746 17768 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:01.339368 12603 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:01.339540 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:01.339066 12605 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:01.343377 17705 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:01.310084 17894 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:12:01.339005 17768 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.322860 17947 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:01.349660 17947 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:01.350558 17768 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:01.351415 17947 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:01.355460 17869 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:01.357717 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 56165.7us
I20260709 12:12:01.361385 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:01.361917 12643 leader_election.cc:400] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13.
I20260709 12:12:01.363318 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:01.365661 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:01.366433 12674 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:01.367235 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:01.367997 12675 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:12:01.369689 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 570) took 1264 ms. Trace:
I20260709 12:12:01.369760 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:01.358428 12572 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:01.361406 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:01.370227 12527 rpcz_store.cc:276] 0709 12:12:00.104971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:00.105077 (+   106us) service_pool.cc:224] Handling call
0709 12:12:01.082182 (+977105us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:01.083032 (+   850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:01.369487 (+286455us) spinlock_profiling.cc:243] Waited 944 ms on lock 0x7b54003cb9d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:01.369630 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2170229504}
I20260709 12:12:01.360237 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:01.371553 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:01.371778 12605 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:01.372219 12597 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:12:01.360319 17840 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:01.372879 17840 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:01.374433 17645 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:01.374454 17691 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:12:01.375128 17645 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.305994 17644 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:01.359095 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 78090.9us
I20260709 12:12:01.373236 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:01.382228 17792 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:01.365631 12643 leader_election.cc:400] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15.
W20260709 12:12:01.387001 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 76048us
I20260709 12:12:01.390367 12677 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:12:01.388844 12646 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:01.391942 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:01.345628 17497 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.389658 17854 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:01.377540 17645 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:01.402141 12675 raft_consensus.cc:2410] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 17 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:12:01.415591 17949 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:01.401124 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:01.390189 17875 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:01.424876 12599 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 13: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:01.426290 12571 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:01.431746 12572 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:01.435592 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:01.395042 17854 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:01.440490 17761 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:01.421692 17774 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:12:01.443593 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 265306us
W20260709 12:12:01.444880 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49128.8us
W20260709 12:12:01.450917 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 52313.7us
I20260709 12:12:01.453819 12605 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:12:01.441211 17854 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:01.457105 12674 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
W20260709 12:12:01.458523 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:01.474620 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 109080us
I20260709 12:12:01.416810 17792 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.448908 17875 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:12:01.476974 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 214206us
I20260709 12:12:01.481384 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:01.458468 17774 raft_consensus.cc:697] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:01.483382 12530 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:12:01.485929 12597 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:12:01.489530 12674 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
W20260709 12:12:01.491091 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 258174us
I20260709 12:12:01.467720 17693 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:01.493419 12676 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:12:01.460497 17747 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:01.495182 12571 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:01.478758 17645 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:12:01.497077 12494 leader_election.cc:336] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.080s (SENT)
I20260709 12:12:01.498803 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:01.453871 17781 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.504269 12494 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:01.508460 12572 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:01.503947 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:01.511087 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:01.512835 12672 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:01.513422 12672 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:01.513824 17747 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:01.514390 17840 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:01.518221 17768 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:12:01.519168 17768 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:01.519835 17768 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:12:01.522277 17878 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:01.522711 17774 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:01.525223 17696 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.028s
I20260709 12:12:01.526940 17696 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:01.527446 17696 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:01.527989 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:01.528119 17875 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.528479 17696 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.530373 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:01.531906 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:01.500531 17645 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:01.546278 12530 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:12:01.551319 13304 leader_election.cc:336] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.878s (SENT)
I20260709 12:12:01.492380 17761 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.561218 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:01.568758 12524 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:01.509696 17693 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:01.564355 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:01.536115 17696 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:01.568094 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2678) took 1468 ms. Trace:
I20260709 12:12:01.570935 12531 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:01.571880 17693 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:01.573546 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:01.575657 17781 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:01.532763 17719 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:01.579107 17956 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
I20260709 12:12:01.610879 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:01.578876 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 90768.4us
W20260709 12:12:01.579614 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 97672.1us
I20260709 12:12:01.580109 17892 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:01.580240 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 71873.9us
I20260709 12:12:01.570680 17840 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:01.582659 17792 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:01.583155 17906 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:12:01.585553 17761 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:01.554788 17645 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:01.589349 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:01.590191 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:01.599370 12598 rpcz_store.cc:276] 0709 12:12:00.095319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:00.096955 (+  1636us) service_pool.cc:224] Handling call
0709 12:12:00.654010 (+557055us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:01.444407 (+790397us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:12:01.444446 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:01.444489 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:01.448258 (+  3769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:01.448334 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:01.531336 (+ 83002us) spinlock_profiling.cc:243] Waited 2.62 ms on lock 0x7b1800342b80. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485933557 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:12:01.563323 (+ 31987us) spinlock_profiling.cc:243] Waited 557 ms on lock 0x7b5400374950. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:01.563470 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7951,"raft.queue_time_us":8309,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":4298,"spinlock_wait_cycles":1286191488}
I20260709 12:12:01.564087 13304 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:01.610641 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 101624us
W20260709 12:12:01.590461 17852 consensus_meta.cc:287] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.616s	user 0.004s	sys 0.009s
I20260709 12:12:01.616818 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:01.620790 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 574) took 1514 ms. Trace:
I20260709 12:12:01.622016 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:01.625095 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:12:01.631038 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 88617.2us
I20260709 12:12:01.623085 17949 consensus_queue.cc:1059] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 9.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.018s
I20260709 12:12:01.637375 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:01.638485 17949 raft_consensus.cc:3055] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:12:01.638972 17949 raft_consensus.cc:740] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:01.639658 17878 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:01.631129 17890 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:12:01.640278 17949 consensus_queue.cc:260] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:01.627717 12522 rpcz_store.cc:276] 0709 12:12:00.106052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:00.107157 (+  1105us) service_pool.cc:224] Handling call
0709 12:12:00.107458 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:00.478567 (+371109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:01.609181 (+1130614us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b5400054954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:01.620713 (+ 11532us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":850630912}
I20260709 12:12:01.609596 17497 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:01.642450 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:01.642318 12680 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:12:01.657557 12674 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:12:01.650509 12675 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 13: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:01.672607 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:01.674383 17906 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:01.674791 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:01.675472 17906 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:01.676473 17852 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:01.677110 17852 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:01.677469 17963 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:12:01.678157 17719 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:01.644429 17890 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:12:01.643968 17819 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:01.679385 17852 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:01.645502 17955 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.163s	user 0.001s	sys 0.000s
I20260709 12:12:01.648130 17957 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
W20260709 12:12:01.686986 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 59070.1us
I20260709 12:12:01.689059 12494 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:01.690276 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:01.693053 17905 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:01.693228 17840 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:01.694185 17840 raft_consensus.cc:697] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:01.695703 17840 consensus_queue.cc:237] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:01.702560 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 43700.4us
I20260709 12:12:01.704743 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:01.705847 12643 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:01.708266 17807 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:12:01.616098 17644 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.714366 17773 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:01.715232 12676 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:12:01.716115 17644 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:01.716315 17773 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:01.716588 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:01.719712 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:01.719887 17691 raft_consensus.cc:3055] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:01.720566 17691 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:01.723094 12496 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:01.723526 17691 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.724491 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:01.725517 17691 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:01.713939 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:01.726102 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:01.732455 12674 raft_consensus.cc:2410] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 15 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:12:01.733642 12530 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
I20260709 12:12:01.734795 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:01.771080 12529 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:01.748533 12678 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:01.642112 17949 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:12:01.748947 12522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:01.749464 12524 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:12:01.749747 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:01.743395 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:01.764173 12531 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:12:01.776139 12522 raft_consensus.cc:606] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:01.754796 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:01.772267 12678 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:12:01.774391 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48422.1us
I20260709 12:12:01.752875 12676 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:01.777436 12522 raft_consensus.cc:614] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:01.775347 12672 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.760994 12680 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:12:01.795152 12675 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13. Ops: [12.9-12.9]
I20260709 12:12:01.750103 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:01.802067 12677 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:01.808503 12678 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.809749 12678 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:01.777839 17891 raft_consensus.cc:2764] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election decision vote started in defunct term 10: won
I20260709 12:12:01.735404 17707 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:01.814460 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:01.814824 17707 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.816845 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:01.818325 17899 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:01.818866 17707 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:01.819234 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:01.816340 12598 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:01.820257 12674 raft_consensus.cc:2368] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11.
I20260709 12:12:01.819257 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:01.822160 17852 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:01.822752 12605 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:01.822990 12600 raft_consensus.cc:2410] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
I20260709 12:12:01.822752 12606 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
W20260709 12:12:01.815920 17965 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.129s	user 0.003s	sys 0.000s
I20260709 12:12:01.828733 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:01.829672 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:01.673910 17768 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.730628 17913 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:01.832031 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:01.833588 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:01.730990 17773 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:01.840775 13304 leader_election.cc:336] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.508s (SENT)
I20260709 12:12:01.841398 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:01.778113 17670 raft_consensus.cc:2764] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 12:12:01.856366 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52870.3us
I20260709 12:12:01.857664 12599 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:01.827165 17708 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:01.856841 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:01.859644 12672 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:01.729437 17775 consensus_queue.cc:1059] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.106s
I20260709 12:12:01.843523 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:01.803824 17719 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:12:01.871239 12527 raft_consensus.cc:2368] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11.
I20260709 12:12:01.733814 17854 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:01.873970 12523 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:12:01.874760 12678 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:12:01.876242 17899 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:12:01.740155 17645 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:01.822789 17852 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.881908 12678 pending_rounds.cc:85] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 8
I20260709 12:12:01.883203 17966 consensus_queue.cc:1059] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.027s
I20260709 12:12:01.888657 17890 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:12:01.889655 17968 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:12:01.889688 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 71612.8us
I20260709 12:12:01.890054 17773 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:01.892765 17836 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.032s
W20260709 12:12:01.894217 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:01.894490 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:01.896456 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 55720.2us
I20260709 12:12:01.900128 12675 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 19: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:12:01.899960 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50904.8us
I20260709 12:12:01.886850 12678 pending_rounds.cc:107] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:12:01.909838 12680 raft_consensus.cc:2393] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 13: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:01.910461 12597 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:12:01.882953 17873 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:01.914052 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:01.910286 12678 raft_consensus.cc:2889] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:01.911055 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:01.920621 12529 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
W20260709 12:12:01.921164 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:01.924427 17968 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:01.925266 17968 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:01.926403 17968 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:01.924659 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 80069.1us
W20260709 12:12:01.931190 13304 leader_election.cc:336] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.981s (SENT)
I20260709 12:12:01.924674 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 9 index: 4 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:01.870640 17708 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:01.936872 12571 leader_election.cc:400] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11.
I20260709 12:12:01.938436 12571 leader_election.cc:403] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
I20260709 12:12:01.937628 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:01.940923 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:01.938902 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 256643us
I20260709 12:12:01.939603 12572 leader_election.cc:400] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11.
I20260709 12:12:01.940472 12522 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:01.943922 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:01.944875 12678 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:01.927855 17968 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:01.943661 12571 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:01.955998 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:01.976539 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54776.1us
W20260709 12:12:01.991297 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47714.8us
I20260709 12:12:01.992708 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:12:01.995891 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 44295.3us
I20260709 12:12:02.000358 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:01.936587 17957 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 12:12:02.005409 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:02.008314 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:02.011312 12571 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:02.014138 17892 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:02.014746 17892 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:02.015512 17819 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:12:02.017009 17892 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:12:01.943454 17946 consensus_queue.cc:1059] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.061s
I20260709 12:12:01.999660 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
W20260709 12:12:02.021199 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.020833 17553 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:02.021921 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:02.025847 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.018167 17854 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:01.951617 17707 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.053s
W20260709 12:12:02.023231 17958 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.355s	user 0.001s	sys 0.000s
I20260709 12:12:02.030191 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 581) took 1073 ms. Trace:
I20260709 12:12:02.031473 17708 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:02.030555 12525 rpcz_store.cc:276] 0709 12:12:00.956778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:01.159752 (+202974us) service_pool.cc:224] Handling call
0709 12:12:01.168828 (+  9076us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:01.331082 (+162254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:01.331123 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:01.332151 (+  1028us) log.cc:844] Serialized 26 byte log entry
0709 12:12:01.484385 (+152234us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:01.484431 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:01.484513 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:02.022448 (+537935us) spinlock_profiling.cc:243] Waited 31 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:02.029892 (+  7444us) raft_consensus.cc:1685] finished
0709 12:12:02.029913 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:02.030114 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":105588,"raft.queue_time_us":107486,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":137,"spinlock_wait_cycles":71195008,"thread_start_us":151801,"threads_started":1,"wal-append.queue_time_us":260779}
I20260709 12:12:02.032100 13339 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:12:02.032649 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:02.022835 17852 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:02.034493 17708 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:02.033870 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:02.039248 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:02.017062 17819 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:02.040235 17819 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:02.044333 17819 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:02.043808 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:02.050580 13304 leader_election.cc:336] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.038s (SENT)
I20260709 12:12:02.063787 12603 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15.
W20260709 12:12:02.068154 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 140016us
I20260709 12:12:02.070508 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:02.048661 17761 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:12:02.069124 12496 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:02.046277 17736 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.127s
I20260709 12:12:02.072973 12525 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:02.069180 17553 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:02.097915 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:02.101991 12598 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:02.075032 17768 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:02.104986 12527 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: []
W20260709 12:12:02.116863 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 106963us
I20260709 12:12:02.129496 17982 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:02.130259 17457 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:02.130740 17457 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:02.131124 17457 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:02.130972 17982 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:12:02.133528 13304 leader_election.cc:336] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.165s (SENT)
W20260709 12:12:02.134075 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 122853us
W20260709 12:12:02.136061 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 67048.8us
I20260709 12:12:02.132282 17792 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:02.140581 12603 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:02.121565 17582 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:02.143272 12527 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:12:02.144712 12675 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 10.
I20260709 12:12:02.149102 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:02.150795 12675 raft_consensus.cc:2410] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:12:02.151962 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:02.160523 13302 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:02.124362 17645 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:02.168045 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 126007us
W20260709 12:12:02.178192 17974 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:12:02.179733 17774 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:02.180864 17774 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:02.182737 17971 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.180s	user 0.000s	sys 0.001s
W20260709 12:12:02.184211 17972 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
W20260709 12:12:02.190431 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 56929.4us
I20260709 12:12:02.145409 17792 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:02.199152 13304 leader_election.cc:336] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.422s (SENT)
I20260709 12:12:02.200531 13304 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:02.201519 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:02.207674 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 63200us
I20260709 12:12:02.224613 12672 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 12:12:02.225111 12672 pending_rounds.cc:85] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 7
I20260709 12:12:02.225373 12672 pending_rounds.cc:107] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 12:12:02.225682 12672 raft_consensus.cc:2889] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:02.239187 17774 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:02.242432 17984 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:02.245853 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2428) took 2676 ms. Trace:
W20260709 12:12:02.246286 17973 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:12:02.246371 12672 rpcz_store.cc:276] 0709 12:11:59.569810 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:00.138776 (+568966us) service_pool.cc:224] Handling call
0709 12:12:00.142931 (+  4155us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:01.453785 (+1310854us) spinlock_profiling.cc:243] Waited 357 ms on lock 0x7b5400289c20. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488997dc0 00007f1488a221c0 00007f1488a23231 00007f1488a1fd8f 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c
0709 12:12:01.745851 (+292066us) spinlock_profiling.cc:243] Waited 18.9 ms on lock 0x7b4c00333000. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1486ceca20 00007f1486cfda29 00007f1486cc3644 00007f1486ceaf2d 00007f1486ceab93 00007f1488a1b18b 00007f1488a1bcb5 00007f1488a16f30 00007f1488a187de 00007f1488a22b49 00007f1488a23416
0709 12:12:02.245423 (+499572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:02.245757 (+   334us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":863467264}
I20260709 12:12:02.248023 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:02.248924 12672 raft_consensus.cc:2410] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
W20260709 12:12:02.249933 17749 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:12:02.252061 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:02.253198 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:02.261193 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:02.254287 17977 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:12:02.255659 17975 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.109s	user 0.000s	sys 0.000s
I20260709 12:12:02.257169 17457 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:02.262097 12679 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:02.265857 12672 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 13: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:12:02.280237 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.281946 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:02.292286 12678 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 11: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:02.284126 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:02.268666 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 69507.1us
W20260709 12:12:02.311861 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.274057 12494 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:02.335265 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:02.285969 12671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2810) took 2622 ms (client timeout 2222 ms). Trace:
I20260709 12:12:02.327216 12680 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:12:02.334820 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:02.339005 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.339252 12678 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 10.
W20260709 12:12:02.337205 12671 rpcz_store.cc:269] 0709 12:11:59.663517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:59.757160 (+ 93643us) service_pool.cc:224] Handling call
0709 12:12:02.285582 (+2528422us) spinlock_profiling.cc:243] Waited 1.75 s on lock 0x7b54000c26d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:02.285757 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4024589696}
I20260709 12:12:02.266321 17946 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:12:02.346071 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 72081.7us
I20260709 12:12:02.347345 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:02.348729 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:02.277388 17674 raft_consensus.cc:3055] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Stepping down as leader of term 8
I20260709 12:12:02.264511 17977 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 15 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:02.356094 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:02.218600 17957 consensus_queue.cc:1059] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 12.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.130s
I20260709 12:12:02.271273 17457 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:02.365304 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:02.360077 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:02.357580 12494 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:02.265281 17891 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.137s
I20260709 12:12:02.265411 17984 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.099s
I20260709 12:12:02.262777 17792 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:02.327299 17966 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 12:12:02.367995 12571 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:02.530450 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:02.544414 12523 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:02.551678 12528 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:02.432749 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:02.536851 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 599272us
I20260709 12:12:02.556991 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:02.557706 12672 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:12:02.558049 13336 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:02.558526 12679 raft_consensus.cc:2410] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:12:02.368743 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:02.551134 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:02.417331 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.560424 12678 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 13: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:12:02.368974 13304 leader_election.cc:336] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.183s (SENT)
I20260709 12:12:02.561949 13304 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:02.560652 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:02.563606 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.368712 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:02.355582 17674 raft_consensus.cc:740] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:02.408735 17953 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.732s	user 0.002s	sys 0.003s
W20260709 12:12:02.495820 17905 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.802s	user 0.017s	sys 0.000s
I20260709 12:12:02.542529 17947 raft_consensus.cc:3055] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:02.363878 17913 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:02.510380 17707 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:12:02.568216 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:02.558837 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:02.411684 17691 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.640s	user 0.011s	sys 0.000s
I20260709 12:12:02.571786 17988 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:12:02.569927 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:02.528025 17983 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:02.528048 17819 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:02.565995 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 208721us
I20260709 12:12:02.533186 17708 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:02.418666 17991 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:12:02.521013 17760 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 16 current term: 16 is higher.
I20260709 12:12:02.407531 17873 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:02.572165 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:02.603583 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:02.606364 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:02.581542 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:02.583438 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 224823us
W20260709 12:12:02.608260 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:02.567755 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 216990us
I20260709 12:12:02.589988 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:02.568724 17670 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:12:02.613301 12676 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 11: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:02.567752 12496 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:02.572216 17947 raft_consensus.cc:740] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:02.585913 17957 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:02.621397 12523 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:02.580314 17992 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:02.607605 12680 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 13: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:02.582023 12526 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:02.605504 17792 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:02.614086 12671 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:02.623373 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:02.601768 12679 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:12:02.602103 17991 raft_consensus.cc:697] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:02.411039 17977 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:02.623430 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:02.627696 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:02.611987 12672 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:02.628798 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:02.628965 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:02.629647 12678 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 13: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:02.629058 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:02.630004 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:02.631040 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:02.632058 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:02.634618 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:02.635326 12599 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:12:02.635597 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 9 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:02.636404 12680 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 13: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:02.639137 13339 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:02.639935 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:02.641350 13304 leader_election.cc:336] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.697s (SENT)
I20260709 12:12:02.643653 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:02.627451 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:02.647517 12604 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 7. (term mismatch)
I20260709 12:12:02.647972 12604 pending_rounds.cc:85] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 6
I20260709 12:12:02.648221 12604 pending_rounds.cc:107] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:12:02.648484 12604 raft_consensus.cc:2889] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:12:02.648409 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 438379us
I20260709 12:12:02.648846 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:02.649696 12597 raft_consensus.cc:2368] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15.
I20260709 12:12:02.650607 13304 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:02.649516 12496 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:02.599346 17708 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:02.647471 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:02.659647 12676 raft_consensus.cc:2410] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 16 because replica has last-logged OpId of term: 15 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 10.
I20260709 12:12:02.654778 12526 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:02.656108 12678 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:02.658600 12671 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:02.662102 12604 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 8. (term mismatch)
I20260709 12:12:02.662531 12604 pending_rounds.cc:85] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 7
I20260709 12:12:02.662833 12604 pending_rounds.cc:107] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 12.8
I20260709 12:12:02.663079 12604 raft_consensus.cc:2889] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:02.663900 12494 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:02.632809 17985 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:02.627883 17670 raft_consensus.cc:697] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:02.667714 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:02.668843 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:02.672081 12603 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:02.645754 17873 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:02.647150 17988 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:02.675303 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2432) took 3089 ms. Trace:
I20260709 12:12:02.654013 17957 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:12:02.600157 17674 consensus_queue.cc:260] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:02.681123 12526 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:12:02.681211 12672 raft_consensus.cc:3055] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:02.683722 12672 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:02.670964 17736 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:02.686813 17736 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:02.687266 17736 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:02.687388 12671 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:02.647311 17691 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.030s
W20260709 12:12:02.690883 17997 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:12:02.691953 12671 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:02.693594 12671 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
W20260709 12:12:02.693897 17996 log.cc:927] Time spent T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.081s	user 0.000s	sys 0.001s
I20260709 12:12:02.695472 12676 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 18. Current term is 19. Ops: []
W20260709 12:12:02.696362 12675 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.544s	user 0.014s	sys 0.001s
I20260709 12:12:02.696764 12675 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 14.
I20260709 12:12:02.698750 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:02.633831 17992 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:02.676730 12673 rpcz_store.cc:276] 0709 12:11:59.586048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:00.466879 (+880831us) service_pool.cc:224] Handling call
0709 12:12:00.467618 (+   739us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:00.519473 (+ 51855us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:00.519522 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:01.190807 (+671285us) spinlock_profiling.cc:243] Waited 7.62 ms on lock 0x7b2c00037868. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:12:01.191533 (+   726us) log.cc:844] Serialized 52 byte log entry
0709 12:12:01.457034 (+265501us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:01.457091 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:01.457186 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:02.408024 (+950838us) spinlock_profiling.cc:243] Waited 45.8 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:02.675055 (+267031us) raft_consensus.cc:1685] finished
0709 12:12:02.675085 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:02.675240 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12471,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":186,"spinlock_wait_cycles":122893312,"thread_start_us":264322,"threads_started":1,"wal-append.queue_time_us":429960}
W20260709 12:12:02.704129 17582 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.539s	user 0.012s	sys 0.000s
I20260709 12:12:02.716566 17869 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:02.678249 17988 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:02.717728 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:02.718940 17707 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:02.671056 17913 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:02.704731 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:12:02.716610 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.728145 13335 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:12:02.729960 12672 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:02.723008 12427 catalog_manager.cc:5697] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 9 to 12. New cstate: current_term: 12 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:02.676029 17891 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.073s
I20260709 12:12:02.751277 12676 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 9. Current term is 10. Ops: [9.7-9.7]
I20260709 12:12:02.755002 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2399) took 1007 ms. Trace:
I20260709 12:12:02.755537 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:02.756760 12605 raft_consensus.cc:2368] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15.
I20260709 12:12:02.755580 12525 rpcz_store.cc:276] 0709 12:12:01.746754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:02.086912 (+340158us) service_pool.cc:224] Handling call
0709 12:12:02.104323 (+ 17411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:02.125195 (+ 20872us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:12:02.622281 (+497086us) log.cc:844] Serialized 14 byte log entry
0709 12:12:02.753796 (+131515us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:02.753857 (+    61us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:02.753908 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:02.753986 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:02.754158 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33186,"prepare.queue_time_us":44823,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"thread_start_us":536460,"threads_started":2}
W20260709 12:12:02.759837 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:02.760483 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:02.755911 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 86242.9us
I20260709 12:12:02.763057 12496 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:02.653714 17947 consensus_queue.cc:260] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:02.765440 12496 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:02.748111 17707 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:02.799288 12676 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
W20260709 12:12:02.800443 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 166763us
I20260709 12:12:02.808382 12675 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
W20260709 12:12:02.773869 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46620.3us
I20260709 12:12:02.802151 17519 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:02.812085 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 148254us
I20260709 12:12:02.745448 17582 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 11.
I20260709 12:12:02.728992 17988 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:02.773836 12672 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:02.750659 17990 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:02.772852 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:02.702347 17873 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:02.822618 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:02.819147 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 53779.4us
I20260709 12:12:02.829566 12494 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:02.800045 17773 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:02.834755 12671 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
W20260709 12:12:02.833400 13304 leader_election.cc:336] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.779s (SENT)
I20260709 12:12:02.835755 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:02.837325 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.803755 17768 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:02.762207 17913 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:02.810253 17691 consensus_queue.cc:1059] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
I20260709 12:12:02.816561 18000 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:02.918080 12599 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:12:02.860141 17670 consensus_queue.cc:237] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:02.931102 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:02.864605 12671 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:12:02.896411 12525 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
W20260709 12:12:02.864713 12569 leader_election.cc:336] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Timed out: RequestConsensusVote RPC to 127.11.13.132:45801 timed out after 1.566s (SENT)
I20260709 12:12:02.897925 12602 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 15: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:02.927098 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:02.932871 12672 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 12:12:02.934953 12672 pending_rounds.cc:85] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 11
I20260709 12:12:02.935451 12672 pending_rounds.cc:107] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:12:02.935204 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:02.939769 12672 raft_consensus.cc:2889] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:02.937088 12569 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:02.834347 17947 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:02.948060 12671 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 12:12:02.948431 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2447) took 1200 ms. Trace:
I20260709 12:12:02.948802 12671 pending_rounds.cc:85] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 10
I20260709 12:12:02.949412 12671 pending_rounds.cc:107] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 12:12:02.949833 12671 raft_consensus.cc:2889] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:02.952826 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2451) took 1090 ms. Trace:
I20260709 12:12:02.953462 12672 rpcz_store.cc:276] 0709 12:12:01.862419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:02.654141 (+791722us) service_pool.cc:224] Handling call
0709 12:12:02.654479 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:02.952460 (+297981us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:02.952743 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:02.961004 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:02.961450 12672 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 10. (term mismatch)
I20260709 12:12:02.962069 12672 pending_rounds.cc:85] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 9
I20260709 12:12:02.961982 12522 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:02.962685 12672 pending_rounds.cc:107] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 11.10
I20260709 12:12:02.963153 12672 raft_consensus.cc:2889] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:02.964139 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:02.964980 12672 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:02.965061 12524 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:02.967990 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 527) took 1147 ms. Trace:
I20260709 12:12:02.968554 12598 rpcz_store.cc:276] 0709 12:12:01.820761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:01.820998 (+   237us) service_pool.cc:224] Handling call
0709 12:12:01.821403 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:02.102553 (+281150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:02.967652 (+865099us) spinlock_profiling.cc:243] Waited 280 ms on lock 0x7b540007b554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:02.967902 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":644755712}
W20260709 12:12:02.977316 17964 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.333s	user 0.003s	sys 0.000s
I20260709 12:12:02.988857 12600 pending_rounds.cc:85] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 8
I20260709 12:12:02.989419 12600 pending_rounds.cc:107] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:12:02.989773 12600 raft_consensus.cc:2889] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:02.993139 12604 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:02.993053 12598 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 12:12:02.997149 12598 pending_rounds.cc:85] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 8
I20260709 12:12:02.997519 12598 pending_rounds.cc:107] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 12:12:02.997824 12598 raft_consensus.cc:2889] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:03.003593 12524 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:03.004606 12598 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:03.024046 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:03.008236 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2402) took 1260 ms. Trace:
I20260709 12:12:02.920298 17981 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:03.025516 12528 rpcz_store.cc:276] 0709 12:12:01.747534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:02.100331 (+352797us) service_pool.cc:224] Handling call
0709 12:12:02.551153 (+450822us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:02.555309 (+  4156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:03.007942 (+452633us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b54003f7350. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:03.008139 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":997335936}
I20260709 12:12:02.821000 17519 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:02.911671 17819 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:02.833787 17457 consensus_queue.cc:1059] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.112s
W20260709 12:12:03.012985 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 191573us
I20260709 12:12:03.018014 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:02.803684 17991 consensus_queue.cc:237] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:03.047996 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:03.027031 12602 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260709 12:12:03.017630 12496 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.222s (SENT) [suppressed 29 similar messages]
W20260709 12:12:03.028618 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 199152us
I20260709 12:12:02.896416 17582 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.116s
W20260709 12:12:03.029963 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:02.936626 17768 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:02.920089 17773 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:03.053332 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:03.058363 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:03.058861 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 236137us
I20260709 12:12:03.062639 12528 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 16: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:03.022204 17392 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:03.074864 12599 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 13: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:03.075881 12572 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:03.077418 12572 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:02.957026 18002 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.113s
W20260709 12:12:03.096544 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.050354 17691 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:12:03.134549 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 120528us
W20260709 12:12:03.145474 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 90135.7us
I20260709 12:12:03.146946 17670 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:03.147770 17670 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:03.150871 17670 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:03.123497 17989 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260709 12:12:03.154543 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 104426us
I20260709 12:12:03.119522 17819 raft_consensus.cc:3055] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:12:03.161798 13339 consensus_meta.cc:287] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.516s	user 0.009s	sys 0.001s
I20260709 12:12:03.152036 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:03.165823 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.125645 17708 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:03.215468 12679 consensus_meta.cc:287] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.575s	user 0.012s	sys 0.000s
W20260709 12:12:03.215962 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:03.196631 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 136756us
I20260709 12:12:03.223719 13339 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:12:03.143913 17582 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:03.231138 12679 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:03.124974 17392 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:03.204661 17781 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:12:03.252249 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:03.253114 13339 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:03.222299 17819 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:03.269395 18006 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.200s	user 0.000s	sys 0.000s
I20260709 12:12:03.271293 17768 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:03.279040 13334 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
W20260709 12:12:03.286005 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 239179us
W20260709 12:12:03.288838 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.248064 17989 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:12:03.288882 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 71606.8us
W20260709 12:12:03.284795 18007 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
I20260709 12:12:03.266515 17873 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:03.305359 17946 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.950s	user 0.014s	sys 0.000s
W20260709 12:12:03.299069 18001 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.352s	user 0.000s	sys 0.002s
I20260709 12:12:03.315656 17946 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:12:03.323227 17519 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:03.297326 17989 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:03.326355 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:03.327713 13332 raft_consensus.cc:2410] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 11 because replica has last-logged OpId of term: 10 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 10.
W20260709 12:12:03.315251 18005 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.283s	user 0.002s	sys 0.000s
W20260709 12:12:03.342938 13337 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.614s	user 0.013s	sys 0.000s
I20260709 12:12:03.314200 17992 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:03.345494 13337 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:12:03.348243 17968 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:03.348809 17968 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:03.349324 17392 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:03.352999 12494 leader_election.cc:400] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15.
W20260709 12:12:03.353459 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:03.353879 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.359404 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:03.305581 17819 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:03.349835 17968 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:03.364512 12529 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 10 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:12:03.357307 17519 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.009s
I20260709 12:12:03.365483 12531 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: []
W20260709 12:12:03.370391 13336 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38472 (request call id 562) took 1761 ms (client timeout 2014 ms). Trace:
W20260709 12:12:03.370960 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 506273us
I20260709 12:12:03.374852 12494 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:03.374109 13336 rpcz_store.cc:269] 0709 12:12:01.609506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:01.612565 (+  3059us) service_pool.cc:224] Handling call
0709 12:12:03.370012 (+1757447us) spinlock_profiling.cc:243] Waited 892 ms on lock 0x7b54001261d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:03.370305 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2052466560}
I20260709 12:12:03.371142 13339 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
W20260709 12:12:03.358009 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 280680us
I20260709 12:12:03.363452 12646 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:03.364564 12530 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:12:03.386874 12572 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:03.354883 17773 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:03.462008 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 98646.5us
W20260709 12:12:03.462297 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 87539.9us
I20260709 12:12:03.462838 12646 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:03.462997 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:03.464160 12494 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:03.469413 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:03.470456 12646 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:03.470912 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:03.472041 13339 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:03.473271 12494 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:03.469568 13304 leader_election.cc:336] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.625s (SENT)
I20260709 12:12:03.476410 12523 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:03.377810 17392 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.097s
I20260709 12:12:03.475181 17781 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:03.480819 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:03.480988 17781 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:03.481521 13339 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:12:03.483338 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:03.483559 17781 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:03.486754 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:03.474707 18020 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:03.484241 17670 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.015s
I20260709 12:12:03.499120 18020 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:03.499847 18020 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:03.492688 18025 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.131s	user 0.000s	sys 0.000s
I20260709 12:12:03.472368 17760 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:12:03.475968 17989 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:03.479686 17981 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader election won for term 10
I20260709 12:12:03.474098 17519 consensus_queue.cc:1243] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 12.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:12:03.504681 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 129693us
W20260709 12:12:03.506727 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 119949us
I20260709 12:12:03.507884 12572 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:03.508786 17707 consensus_meta.cc:287] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.683s	user 0.013s	sys 0.000s
I20260709 12:12:03.475298 18002 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:03.510735 18002 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:03.511132 18002 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:03.511557 17582 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:03.511896 17781 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:03.461498 17819 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:03.514618 17708 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:03.515375 12571 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:03.516899 17708 consensus_queue.cc:1059] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:03.518363 17781 raft_consensus.cc:3055] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:03.518915 17781 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:03.521652 17708 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:03.522503 17708 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:03.523226 17519 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:03.524030 17736 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.836s	user 0.008s	sys 0.004s
I20260709 12:12:03.523834 17519 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:03.524537 17736 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:03.509649 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:03.528456 18009 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:03.509460 17707 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:03.537817 13332 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:03.475452 17968 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:03.474922 17891 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:03.475056 17899 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:03.559967 18034 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:03.561930 18036 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:03.563308 13339 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:12:03.543303 17992 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:03.511333 18023 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.054s
W20260709 12:12:03.583572 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 114032us
I20260709 12:12:03.520165 17781 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:03.543402 17392 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:03.582437 18036 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:03.556231 17981 raft_consensus.cc:697] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:03.542995 17983 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.039s
W20260709 12:12:03.591946 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 101315us
I20260709 12:12:03.594275 17760 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:03.576346 17989 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.017s
I20260709 12:12:03.588408 18009 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:03.523801 17708 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:03.604409 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.603797 17736 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:03.608685 17774 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:03.608071 18032 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.016s
I20260709 12:12:03.600757 17519 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:03.607575 17837 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:03.598044 17781 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:03.637288 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.622406 17981 consensus_queue.cc:237] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:02.949028 12680 rpcz_store.cc:276] 0709 12:12:01.747777 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:02.642414 (+894637us) service_pool.cc:224] Handling call
0709 12:12:02.642706 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:02.666832 (+ 24126us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:12:02.766626 (+ 99794us) log.cc:844] Serialized 14 byte log entry
0709 12:12:02.922438 (+155812us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:02.923510 (+  1072us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:02.948064 (+ 24554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:02.948186 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:02.948354 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":225471,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":2848,"raft.run_cpu_time_us":440,"raft.run_wall_time_us":30305,"thread_start_us":157836,"threads_started":2}
I20260709 12:12:03.608085 17760 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:03.647110 12601 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:12:03.648150 12606 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:12:03.622984 17774 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:03.658635 12599 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:03.663173 17553 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:12:03.656558 17999 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.906s	user 0.006s	sys 0.010s
I20260709 12:12:03.664496 17890 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:03.628626 17707 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:03.620628 17792 consensus_meta.cc:287] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.985s	user 0.006s	sys 0.007s
W20260709 12:12:03.652963 17988 consensus_meta.cc:287] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.733s	user 0.014s	sys 0.000s
I20260709 12:12:03.617684 18031 consensus_queue.cc:1059] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.048s
I20260709 12:12:03.617245 18009 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:03.671057 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.665908 17774 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:03.591403 17392 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:03.674453 12494 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:03.699122 17691 raft_consensus.cc:3055] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Stepping down as leader of term 11
W20260709 12:12:03.650856 17947 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.640s	user 0.015s	sys 0.000s
I20260709 12:12:03.718329 17775 raft_consensus.cc:3055] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:03.715723 17691 raft_consensus.cc:740] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:03.710433 17988 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:03.631485 18036 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:03.678854 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 588) took 1649 ms. Trace:
I20260709 12:12:03.679996 17873 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:12:03.719946 17873 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:03.698551 17992 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 12:12:03.720160 12605 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:12:03.721100 17988 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:03.720907 17873 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:03.680655 17457 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:03.722282 17457 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:03.723433 17457 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:03.724660 17457 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:03.683615 18020 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:03.727985 18020 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
W20260709 12:12:03.728268 17998 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.046s	user 0.005s	sys 0.013s
I20260709 12:12:03.729091 17891 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:03.700197 17890 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:03.732939 17890 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:03.719228 17392 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:03.738159 13335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 687) took 1120 ms. Trace:
I20260709 12:12:03.738692 13335 rpcz_store.cc:276] 0709 12:12:02.617973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:02.623540 (+  5567us) service_pool.cc:224] Handling call
0709 12:12:03.737889 (+1114349us) spinlock_profiling.cc:243] Waited 79.7 ms on lock 0x7b54001b49d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:03.738066 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":183345920}
I20260709 12:12:03.719774 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 586) took 1774 ms. Trace:
I20260709 12:12:03.740255 12527 rpcz_store.cc:276] 0709 12:12:01.945487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:02.569886 (+624399us) service_pool.cc:224] Handling call
0709 12:12:02.587942 (+ 18056us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:02.601660 (+ 13718us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:02.601692 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:02.622026 (+ 20334us) log.cc:844] Serialized 26 byte log entry
0709 12:12:02.749518 (+127492us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:02.749584 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:02.749662 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:03.655891 (+906229us) spinlock_profiling.cc:243] Waited 8.62 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:03.719417 (+ 63526us) raft_consensus.cc:1685] finished
0709 12:12:03.719443 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:03.719701 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3618,"raft.queue_time_us":7761,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":8772,"spinlock_wait_cycles":19829632,"thread_start_us":84517,"threads_started":1,"wal-append.queue_time_us":103523}
I20260709 12:12:03.719451 12523 rpcz_store.cc:276] 0709 12:12:02.028968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:02.623083 (+594115us) service_pool.cc:224] Handling call
0709 12:12:02.640179 (+ 17096us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:03.476930 (+836751us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:03.678540 (+201610us) spinlock_profiling.cc:243] Waited 836 ms on lock 0x7b5400054954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:03.678778 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1922795776}
I20260709 12:12:03.685113 18002 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:03.691475 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:03.695322 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 166363us
W20260709 12:12:03.699112 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 105459us
I20260709 12:12:03.743930 12525 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
W20260709 12:12:03.703529 12643 connection.cc:256] RPC call timeout handler was delayed by -1.8599290590000237s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 12:12:03.710031 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:03.744870 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.717519 17707 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:03.718891 17775 raft_consensus.cc:740] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:03.675946 17674 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:12:03.779335 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:03.781100 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:03.785456 13332 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:03.790268 13332 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
W20260709 12:12:03.791745 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:03.754752 17869 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.009s	user 0.011s	sys 0.003s
I20260709 12:12:03.716199 18031 consensus_queue.cc:1243] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 11.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:12:03.793399 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:03.798674 12530 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:03.790750 13332 pending_rounds.cc:85] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 10
I20260709 12:12:03.742834 18009 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:03.754581 17392 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:03.800036 13332 pending_rounds.cc:107] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:12:03.810962 13332 raft_consensus.cc:2889] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:03.781997 12525 raft_consensus.cc:2376] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12. Re-sending same reply.
I20260709 12:12:03.799962 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:12:03.802695 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.754804 18020 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:03.796628 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:03.758697 17899 raft_consensus.cc:697] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:03.814615 13339 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:03.793638 17890 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:03.794298 17873 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:03.816582 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:03.796991 17707 raft_consensus.cc:697] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:03.831267 12530 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:03.830940 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:12:03.831174 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 47410.9us
I20260709 12:12:03.035637 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:03.834426 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:03.801653 17985 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.072s	user 0.022s	sys 0.032s
I20260709 12:12:03.719750 17691 consensus_queue.cc:260] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.8, Last appended by leader: 8, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:03.837322 12646 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:03.762987 17774 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 12.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:12:03.793406 18033 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.075s
W20260709 12:12:03.846689 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 269118us
I20260709 12:12:03.822448 17968 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:03.849469 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:12:03.842199 17985 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.117s	user 0.023s	sys 0.032s
I20260709 12:12:03.851303 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:03.861245 18041 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:03.861955 18041 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:03.865849 18041 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:03.866482 18041 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:03.866987 18041 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:03.868589 17749 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:03.816612 17773 consensus_queue.cc:1059] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.031s
I20260709 12:12:03.844018 17519 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:12:03.841743 17913 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:03.824493 18020 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:03.821902 18038 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.059s
I20260709 12:12:03.892489 13330 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:03.869511 17749 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:03.894439 12572 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:03.885412 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:03.891472 13305 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:03.890591 12427 catalog_manager.cc:5697] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 6 to 9, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 9 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:12:03.826175 17392 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:03.928419 13339 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:03.870167 17774 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:12:03.935264 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 43864us
I20260709 12:12:03.936510 12494 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:03.880005 18040 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:03.870954 18037 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.095s
I20260709 12:12:03.943296 17392 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:03.891815 18041 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:03.879237 17968 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:03.948519 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:03.926057 17957 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:03.949265 17719 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:03.949772 17707 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:03.950425 17553 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:03.950778 17899 consensus_queue.cc:237] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:03.952754 18002 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:03.953720 18031 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:03.953814 17760 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:03.943348 18020 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:03.939122 17989 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
W20260709 12:12:03.925710 18035 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:12:03.825666 17775 consensus_queue.cc:260] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:03.967665 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 70081.9us
I20260709 12:12:03.897046 18044 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:04.005931 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:04.006770 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:04.006098 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 75024.8us
I20260709 12:12:04.007085 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:04.007259 12602 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
W20260709 12:12:03.967681 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 130428us
I20260709 12:12:03.973091 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:03.875501 17985 raft_consensus.cc:697] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:04.009258 12646 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:03.896629 17981 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.072s
I20260709 12:12:03.963713 17774 raft_consensus.cc:697] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:04.010476 17985 consensus_queue.cc:237] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:04.010874 17774 consensus_queue.cc:237] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:03.927282 17781 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260709 12:12:03.995885 13339 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
W20260709 12:12:03.996809 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 102543us
I20260709 12:12:04.009253 12598 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 9. Current term is 10. Ops: []
I20260709 12:12:04.015767 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:04.016964 12522 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:04.017484 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:03.986907 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:04.020294 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:04.021394 12522 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:12:03.986894 17913 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:12:04.006644 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 125982us
I20260709 12:12:04.033114 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:03.987746 17553 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:04.035146 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:04.036196 12603 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 17: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:04.036924 12597 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 14: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:04.024001 12602 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:12:04.040513 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:04.040513 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:04.034366 12601 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 9.
W20260709 12:12:04.044982 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:03.992812 17760 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:04.045406 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:04.048074 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:04.056282 12569 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:04.005555 17519 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:04.060551 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:04.062431 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 53247us
I20260709 12:12:04.064707 12531 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:04.071419 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:04.072484 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:04.065330 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:04.073470 12528 raft_consensus.cc:2410] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:12:04.005932 18044 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:04.076726 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:04.072707 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:04.077869 12569 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:03.965101 18040 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:04.083612 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:04.086969 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:04.082784 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:04.040115 17913 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:04.086586 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:04.086614 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:04.041529 17553 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:03.973953 17957 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:04.093178 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:04.094772 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:04.095745 12603 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:12:04.104727 12524 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:04.065399 17781 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:04.062232 18038 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:04.112057 12569 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:04.112911 12528 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:03.995570 18031 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:04.100358 17913 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:04.120590 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:04.150257 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 63697.3us
I20260709 12:12:04.087121 18044 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:04.090893 18047 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:04.076433 18009 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:04.155632 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:04.156257 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:04.098522 17674 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:12:04.091104 17947 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.071s
I20260709 12:12:04.091216 17792 consensus_queue.cc:1059] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260709 12:12:04.088065 18042 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:04.164371 12603 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 12: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:04.161651 18002 raft_consensus.cc:3055] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:04.105698 18051 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:04.161320 18032 raft_consensus.cc:3055] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:04.117630 18000 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.060s
I20260709 12:12:04.175817 13339 raft_consensus.cc:2376] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13. Re-sending same reply.
W20260709 12:12:04.253042 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 89223.8us
I20260709 12:12:04.165226 18023 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:04.170848 17837 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:04.161876 18052 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:04.292619 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:04.292857 12672 raft_consensus.cc:1217] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Deduplicated request from leader. Original: 8.6->[10.7-11.8]   Dedup: 10.7->[11.8-11.8]
I20260709 12:12:04.166285 18041 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:04.274166 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:04.299460 12680 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:04.275738 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:04.279158 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:04.123524 18036 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 12:12:04.301626 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:04.329116 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:04.174044 18047 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:04.333920 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:04.264261 17869 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:04.293610 12671 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:04.294679 12678 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:12:04.357999 18054 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.006s
I20260709 12:12:04.293640 12672 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: [10.7-11.8]
I20260709 12:12:04.372913 12672 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:04.300519 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:04.300467 12676 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:12:04.175812 17774 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:04.172616 18020 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:04.177244 17674 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:04.154163 18038 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:04.353603 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:04.356067 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:04.265492 17840 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:04.269901 18066 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:04.270533 18065 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:04.134719 17781 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:04.297093 18023 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:04.268006 18053 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:12:04.272617 18002 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:04.280246 18032 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:12:04.377091 13333 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.587s	user 0.005s	sys 0.007s
I20260709 12:12:04.432932 13333 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:12:04.303479 17749 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:04.392665 12675 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:04.433362 18032 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:04.434600 18032 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:04.408296 12673 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:12:04.407928 12679 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:04.278743 17873 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 12:12:04.264039 18009 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:04.446655 18009 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:04.406250 17674 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:12:04.452817 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 394429us
I20260709 12:12:04.427608 12680 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:04.273559 17760 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:04.454752 12675 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:12:04.390865 18062 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:04.456216 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:04.539448 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:04.539999 12676 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:04.414559 17670 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:04.430908 18023 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:04.515039 18021 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:04.538864 18048 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.455s	user 0.002s	sys 0.000s
I20260709 12:12:04.410171 18020 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:04.453001 17749 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:12:04.523840 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:04.570391 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:04.528930 18032 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:12:04.374621 17999 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:12:04.484769 18000 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:04.333174 18041 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:04.426558 18003 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:04.329464 18052 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:04.551241 12676 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:04.364943 17869 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:04.406383 17774 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:04.431932 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:04.422380 17840 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:04.426596 18066 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:04.580543 12674 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:04.595845 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:04.574261 17760 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:12:04.581456 18049 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.489s	user 0.000s	sys 0.001s
I20260709 12:12:04.573803 18051 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:04.569311 17645 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:04.602205 12571 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:04.561995 17670 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:04.610558 17977 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.000s	sys 0.028s
W20260709 12:12:04.692940 17977 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.000s	sys 0.029s
I20260709 12:12:04.611006 12525 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 13. (index mismatch)
I20260709 12:12:04.694392 17774 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:04.611454 17992 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.005s	sys 0.019s
I20260709 12:12:04.457798 18065 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:04.695538 17992 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.006s	sys 0.019s
I20260709 12:12:04.695950 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:04.697217 18065 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:04.692384 17670 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:04.649132 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 214427us
I20260709 12:12:04.697659 18052 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:04.697208 17992 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:12:04.605302 18050 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.438s	user 0.000s	sys 0.001s
I20260709 12:12:04.670861 17840 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:04.671273 18074 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:04.699105 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:04.699353 18020 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:04.699451 18074 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:04.671494 17719 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:04.672767 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:04.675287 18066 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:04.675468 17582 raft_consensus.cc:3055] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:04.722810 17582 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:04.724002 18074 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:04.724025 17582 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:04.725426 17582 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:04.725754 18064 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
I20260709 12:12:04.709025 17719 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:04.727047 17719 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:04.727707 18017 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:04.728204 17719 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:04.731773 12428 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58992 (request call id 172) took 38203 ms (client timeout 15000 ms). Trace:
I20260709 12:12:04.678282 17819 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:04.665390 17645 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:04.734558 17819 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:04.735129 17819 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:04.736702 17645 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:04.683976 18038 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:04.684825 17781 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:04.732240 12428 rpcz_store.cc:269] 0709 12:11:26.533322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:26.558518 (+ 25196us) service_pool.cc:224] Handling call
0709 12:12:04.714581 (+38156063us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:11:52.753641 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:11:52.778609 (+ 24968us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:11:52.778661 (+    52us) write_op.cc:435] Acquired schema lock
0709 12:11:52.778695 (+    34us) tablet.cc:662] Decoding operations
0709 12:11:52.827359 (+ 48664us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:11:52.827478 (+   119us) write_op.cc:641] Partition lock acquired for write op
0709 12:11:52.827512 (+    34us) tablet.cc:685] Acquiring locks for 24 operations
0709 12:11:52.906592 (+ 79080us) tablet.cc:701] Row locks acquired
0709 12:11:52.906644 (+    52us) write_op.cc:260] PREPARE: finished
0709 12:11:52.907710 (+  1066us) write_op.cc:270] Start()
0709 12:11:52.907876 (+   166us) write_op.cc:276] Timestamp: P: 1783599112907394 usec, L: 0
0709 12:11:52.907921 (+    45us) op_driver.cc:348] REPLICATION: starting
0709 12:11:52.908767 (+   846us) log.cc:844] Serialized 9457 byte log entry
0709 12:11:53.795633 (+886866us) op_driver.cc:464] REPLICATION: finished
0709 12:11:55.660088 (+1864455us) spinlock_profiling.cc:243] Waited 27.1 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14859160ac 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f148c456ef0
0709 12:11:56.059071 (+398983us) write_op.cc:301] APPLY: starting
0709 12:11:56.059206 (+   135us) tablet.cc:1367] starting BulkCheckPresence
0709 12:11:56.534705 (+475499us) spinlock_profiling.cc:243] Waited 6.18 ms on lock 0x7b18002a4720. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f148875f0ec 00007f14887abc0a 00007f14887ab4f7 00007f14887ab490 00007f1488762f48 00007f148c4771bb 00007f148c477532 00007f148c4beb93 00007f148c36bde3
0709 12:11:57.007395 (+472690us) tablet.cc:1370] finished BulkCheckPresence
0709 12:11:57.007443 (+    48us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:11:57.872108 (+864665us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:11:57.889793 (+ 17685us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=48,key_file_lookups=48,delta_file_lookups=48,mrs_lookups=0
0709 12:11:57.889882 (+    89us) write_op.cc:312] APPLY: finished
0709 12:11:58.074432 (+184550us) log.cc:844] Serialized 212 byte log entry
0709 12:11:58.536521 (+462089us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:58.560441 (+ 23920us) write_op.cc:454] Released schema lock
0709 12:11:58.638231 (+ 77790us) write_op.cc:341] FINISH: Updating metrics
0709 12:11:58.876695 (+238464us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:11:58.882203 (+  5508us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":12907,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":2226020,"apply.run_cpu_time_us":74312,"apply.run_wall_time_us":2785510,"cfile_cache_hit":98,"cfile_cache_hit_bytes":104303,"num_ops":24,"prepare.queue_time_us":301026,"prepare.run_cpu_time_us":60388,"prepare.run_wall_time_us":389972,"raft.queue_time_us":346284,"raft.run_cpu_time_us":100294,"raft.run_wall_time_us":2504198,"replication_time_us":841369,"spinlock_wait_cycles":76621440,"thread_start_us":3071200,"threads_started":5,"wal-append.queue_time_us":962974,"wal-append.run_cpu_time_us":10325,"wal-append.run_wall_time_us":2818946}]]}
I20260709 12:12:04.682646 12427 catalog_manager.cc:5697] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:04.742965 12427 catalog_manager.cc:5697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:04.744441 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:04.685873 18051 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:12:04.748672 18059 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.293s	user 0.001s	sys 0.000s
I20260709 12:12:04.549273 18062 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:04.696128 18047 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:04.613428 12603 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:04.764189 13333 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 547) took 1302 ms. Trace:
I20260709 12:12:04.764657 13333 rpcz_store.cc:276] 0709 12:12:03.462235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:03.462374 (+   139us) service_pool.cc:224] Handling call
0709 12:12:04.763994 (+1301620us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b54004781d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:04.764115 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":646182272}
I20260709 12:12:04.766711 13334 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:12:04.767246 17775 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:04.768237 17674 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:04.769841 12606 raft_consensus.cc:2368] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12.
I20260709 12:12:04.586869 18003 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:04.770733 18047 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:04.771265 18047 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:04.772531 18003 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:04.773383 18047 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:04.773034 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:04.777696 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:04.574553 17749 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:04.703245 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:04.577461 18002 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:04.706389 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:04.669417 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 67199.3us
I20260709 12:12:04.710901 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:04.579161 18000 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:04.660693 18040 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:04.797111 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:04.689711 12572 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:04.745747 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:04.689451 18042 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:12:04.750486 12529 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:12:04.745221 12427 catalog_manager.cc:5697] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:04.782629 17836 raft_consensus.cc:3055] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:12:04.784044 17691 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:04.565738 18021 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:04.776913 17674 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:04.793565 12571 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:04.787235 18041 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:04.799242 12675 raft_consensus.cc:2410] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 13 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:12:04.829622 12677 raft_consensus.cc:2410] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 14 because replica has last-logged OpId of term: 12 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:12:04.801851 12494 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:04.794297 12606 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:12:04.832052 18071 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:04.832154 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:04.832567 18071 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:04.809473 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 119846us
I20260709 12:12:04.789783 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2474) took 1039 ms. Trace:
I20260709 12:12:04.805835 18003 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:04.819814 12529 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:04.833904 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:04.787925 18074 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:04.795475 12597 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 9.
I20260709 12:12:04.823987 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:04.835616 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:04.845939 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:04.844183 12571 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:04.853066 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:04.853438 12675 raft_consensus.cc:2393] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 9: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:04.855170 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:04.855157 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:04.859105 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:04.828636 17890 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.996s	user 0.011s	sys 0.002s
I20260709 12:12:04.807443 17836 raft_consensus.cc:740] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:04.824275 18002 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:04.819998 18036 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:04.836108 12523 rpcz_store.cc:276] 0709 12:12:03.750357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:03.750523 (+   166us) service_pool.cc:224] Handling call
0709 12:12:03.780477 (+ 29954us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:03.805552 (+ 25075us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:03.805601 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:03.806252 (+   651us) log.cc:844] Serialized 26 byte log entry
0709 12:12:03.997689 (+191437us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:03.997742 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:03.997815 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:04.789486 (+791671us) raft_consensus.cc:1685] finished
0709 12:12:04.789515 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:04.789704 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16646,"raft.queue_time_us":17013,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":8359,"thread_start_us":160816,"threads_started":1,"wal-append.queue_time_us":249454}
I20260709 12:12:04.832864 18000 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:04.904675 13337 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:04.906896 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:04.830667 18009 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:12:04.864326 18003 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:04.917227 18003 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:04.859212 18037 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:04.918581 18037 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:04.919052 18037 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:04.807237 17840 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:12:04.910481 17890 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:04.907251 18036 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:04.833595 18071 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:12:04.892462 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:04.899503 18042 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:12:04.924947 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:04.932385 18009 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:04.860548 12427 catalog_manager.cc:5697] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 8 to 13, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 13 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:12:04.961436 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:04.944276 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:04.882226 18075 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260709 12:12:04.902873 18039 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:04.899240 17674 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:04.922521 17670 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:04.900781 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:04.924563 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:04.967233 12679 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.013s	sys 0.067s
I20260709 12:12:04.890435 18074 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:04.982717 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:04.995110 12679 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.014s	sys 0.067s
W20260709 12:12:04.963755 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 76878.2us
I20260709 12:12:04.978808 12496 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:04.996942 12606 raft_consensus.cc:2410] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
I20260709 12:12:04.986655 12601 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:12:04.999022 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:04.983949 12427 catalog_manager.cc:5697] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 8 to 10. New cstate: current_term: 10 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:12:05.033864 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 55159.1us
I20260709 12:12:05.034507 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:05.037532 12496 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:05.035997 17645 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:05.035245 12675 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:05.038931 12673 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: [15.8-15.8]
W20260709 12:12:04.972934 17553 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.855s	user 0.014s	sys 0.006s
W20260709 12:12:04.967370 18054 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.029s	sys 0.050s
I20260709 12:12:05.064427 13333 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:12:04.984743 18009 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:04.974212 18036 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:05.084626 12571 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:04.990206 18075 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:04.997643 17674 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:05.068619 18039 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:05.070698 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:05.029876 17670 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:04.990480 18042 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:05.029531 18071 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:05.141073 17519 consensus_meta.cc:287] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.058s	user 0.008s	sys 0.005s
I20260709 12:12:05.039315 18047 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:05.038582 18082 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:05.068673 12643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.906s	user 0.009s	sys 0.009s
I20260709 12:12:05.082206 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:05.090595 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:05.183285 12675 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:05.081259 12600 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:12:05.184540 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:05.168119 17890 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:05.091964 13304 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.542s	user 0.013s	sys 0.005s
W20260709 12:12:05.186030 13304 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.636s	user 0.014s	sys 0.005s
W20260709 12:12:05.088263 18054 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.029s	sys 0.055s
I20260709 12:12:05.041465 18003 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.048s
W20260709 12:12:05.186393 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 636441us
W20260709 12:12:05.092552 17457 consensus_meta.cc:287] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.367s	user 0.013s	sys 0.005s
I20260709 12:12:05.186378 17890 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:05.182641 12643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.032s	user 0.009s	sys 0.009s
W20260709 12:12:05.189999 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 1.18291e+06us
I20260709 12:12:05.190721 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:05.153786 17775 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:12:05.170941 12569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.948s	user 0.017s	sys 0.016s
W20260709 12:12:05.224323 12569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.010s	user 0.018s	sys 0.016s
I20260709 12:12:05.088500 18068 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:05.181128 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2527) took 1009 ms. Trace:
I20260709 12:12:05.228142 12603 rpcz_store.cc:276] 0709 12:12:04.171324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:04.274660 (+103336us) service_pool.cc:224] Handling call
0709 12:12:05.176323 (+901663us) spinlock_profiling.cc:243] Waited 284 ms on lock 0x7b54002e68d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:05.180999 (+  4676us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":652260096}
I20260709 12:12:05.185446 12600 raft_consensus.cc:2410] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 14 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 9.
W20260709 12:12:05.087226 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49767.3us
I20260709 12:12:05.154050 17674 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:05.001204 17836 consensus_queue.cc:260] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:05.071524 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:05.150377 18075 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 8 FOLLOWER]: Advancing to term 9
I20260709 12:12:05.090342 17553 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:05.226315 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:05.067441 12427 catalog_manager.cc:5697] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 8 to 13. New cstate: current_term: 13 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:05.086485 17869 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:05.245175 12678 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.008s	sys 0.012s
I20260709 12:12:05.092579 17645 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:05.182626 17519 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:05.182163 18047 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:12:05.209533 17707 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.758s	user 0.016s	sys 0.020s
I20260709 12:12:05.257552 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:05.258997 12678 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.008s	sys 0.013s
I20260709 12:12:05.262949 12603 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
W20260709 12:12:05.264405 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:05.184427 17749 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:05.212790 17775 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:05.164271 18039 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:05.222858 18076 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 12:12:05.249142 12427 catalog_manager.cc:5697] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 5 to 9, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 9 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:05.329118 12606 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:05.267622 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:05.376031 12600 raft_consensus.cc:2393] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 13: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:12:05.376289 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 141930us
I20260709 12:12:05.377105 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:05.377357 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:05.353420 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2484) took 1066 ms. Trace:
I20260709 12:12:05.378187 12600 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:05.379230 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:05.378074 12678 rpcz_store.cc:276] 0709 12:12:04.287011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:04.364338 (+ 77327us) service_pool.cc:224] Handling call
0709 12:12:04.364681 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:04.364908 (+   227us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:12:04.568335 (+203427us) log.cc:844] Serialized 14 byte log entry
0709 12:12:05.319132 (+750797us) log.cc:844] Serialized 14 byte log entry
0709 12:12:05.352744 (+ 33612us) log.cc:844] Serialized 14 byte log entry
0709 12:12:05.352940 (+   196us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:05.352997 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:05.353069 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:05.353161 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:05.353338 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":67628,"prepare.queue_time_us":339031,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":84,"raft.queue_time_us":93665,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":116,"thread_start_us":853215,"threads_started":2}
I20260709 12:12:05.382439 18021 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:05.383498 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:05.384665 12678 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:05.385692 18074 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:05.226766 17457 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:05.386756 17457 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:05.388995 17457 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:05.271478 18094 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.117s
W20260709 12:12:05.271138 17707 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.820s	user 0.016s	sys 0.020s
I20260709 12:12:05.380985 17775 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:05.391369 12530 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: [12.10-12.10]
W20260709 12:12:05.338418 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:05.229929 18054 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:05.235644 18051 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.146s
I20260709 12:12:05.260383 18031 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:05.399821 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:05.401108 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:05.279868 18085 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.235s	user 0.001s	sys 0.000s
I20260709 12:12:05.271598 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:05.210049 18082 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:05.251175 18070 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.449s
I20260709 12:12:05.283787 18047 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:05.418900 18103 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:12:05.420783 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2483) took 1138 ms. Trace:
I20260709 12:12:05.421342 12671 rpcz_store.cc:276] 0709 12:12:04.282867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:04.365550 (+ 82683us) service_pool.cc:224] Handling call
0709 12:12:04.375207 (+  9657us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:04.658666 (+283459us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:04.658711 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:04.711127 (+ 52416us) log.cc:844] Serialized 26 byte log entry
0709 12:12:04.989197 (+278070us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:04.989235 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:04.989310 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:05.420469 (+431159us) raft_consensus.cc:1685] finished
0709 12:12:05.420508 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:05.420700 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":33923,"raft.queue_time_us":204152,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":16786,"thread_start_us":433706,"threads_started":2,"wal-append.queue_time_us":468398}
I20260709 12:12:05.423548 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:05.425065 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:05.426430 18047 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:05.430711 18047 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:05.428198 12678 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:12:05.441095 17645 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:05.439859 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:05.445115 18044 consensus_meta.cc:287] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.271s	user 0.014s	sys 0.000s
I20260709 12:12:05.444097 17674 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:05.435832 18039 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:05.436550 12427 catalog_manager.cc:5697] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 5 to 11. New cstate: current_term: 11 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:12:05.437559 18082 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:05.443125 18084 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.367s	user 0.000s	sys 0.001s
I20260709 12:12:05.445760 18044 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:05.448318 17553 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:05.446374 12678 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:12:05.450273 12598 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:05.449527 18044 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:05.451527 12675 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:12:05.425757 12671 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:05.455564 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:05.408879 12603 raft_consensus.cc:2410] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:12:05.453802 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:05.435056 18083 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.104s	user 0.001s	sys 0.000s
I20260709 12:12:05.460999 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:05.401724 12600 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
W20260709 12:12:05.462724 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 85718us
I20260709 12:12:05.455844 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:05.465346 12678 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:05.340332 18102 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.088s	user 0.001s	sys 0.000s
I20260709 12:12:05.468691 12427 catalog_manager.cc:5697] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 7 to 10, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 10 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:12:05.418812 17719 consensus_meta.cc:287] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.690s	user 0.015s	sys 0.000s
I20260709 12:12:05.514991 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:05.458177 18042 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:05.471151 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:05.471351 17749 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:05.473637 18062 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:05.474238 18054 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:12:05.474472 13330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38472 (request call id 566) took 3794 ms (client timeout 1566 ms). Trace:
I20260709 12:12:05.477449 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:05.478202 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:05.478782 13335 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 14 index: 12. (index mismatch)
I20260709 12:12:05.477449 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 10 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:05.485447 17890 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:05.487267 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:05.534528 12525 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
W20260709 12:12:05.535250 13334 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.039s	sys 0.050s
I20260709 12:12:05.488150 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:05.488955 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:05.490657 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:05.491735 18053 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:12:05.499058 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:05.536579 12524 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 14.
W20260709 12:12:05.528107 13330 rpcz_store.cc:269] 0709 12:12:01.679812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:01.754458 (+ 74646us) service_pool.cc:224] Handling call
0709 12:12:05.474253 (+3719795us) spinlock_profiling.cc:243] Waited 2 s on lock 0x7b5400469954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:05.474367 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4608801664}
I20260709 12:12:05.507084 18076 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:05.416863 18091 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.206s	user 0.000s	sys 0.000s
W20260709 12:12:05.420890 18092 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.247s	user 0.001s	sys 0.000s
W20260709 12:12:05.513283 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:05.271129 18081 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:05.514796 17781 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:05.516903 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:05.457521 12602 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
W20260709 12:12:05.519233 12643 connection.cc:256] RPC call timeout handler was delayed by -1.7077928829999109s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15000000000000002s for a response
W20260709 12:12:05.552786 12643 connection.cc:256] RPC call timeout handler was delayed by -1.6951928829998906s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1514s for a response
W20260709 12:12:05.553172 12643 connection.cc:256] RPC call timeout handler was delayed by -1.6600928829998338s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1553s for a response
W20260709 12:12:05.467283 18023 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.901s	user 0.012s	sys 0.001s
W20260709 12:12:05.553550 12643 connection.cc:256] RPC call timeout handler was delayed by -1.6474928829998134s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1567s for a response
W20260709 12:12:05.553895 12643 connection.cc:256] RPC call timeout handler was delayed by -1.5862928829997145s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1635s for a response
I20260709 12:12:05.553781 18023 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:05.554483 12643 connection.cc:256] RPC call timeout handler was delayed by -1.52329288299984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1705s for a response
W20260709 12:12:05.554831 12643 connection.cc:256] RPC call timeout handler was delayed by -1.5052928829998109s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17250000000000001s for a response
W20260709 12:12:05.555133 12643 connection.cc:256] RPC call timeout handler was delayed by -1.4818928829997731s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1751s for a response
W20260709 12:12:05.555408 12643 connection.cc:256] RPC call timeout handler was delayed by -1.4791928829997687s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1754s for a response
W20260709 12:12:05.555678 12643 connection.cc:256] RPC call timeout handler was delayed by -1.4629928829997425s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
W20260709 12:12:05.555966 12643 connection.cc:256] RPC call timeout handler was delayed by -1.4044928829998753s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1837s for a response
I20260709 12:12:05.556177 18023 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:05.524943 13336 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 13. Current term is 14. Ops: [13.11-13.11]
I20260709 12:12:05.521142 12427 catalog_manager.cc:5697] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 7 to 11, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 11 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:12:05.550619 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 535) took 1823 ms. Trace:
I20260709 12:12:05.532382 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2478) took 1255 ms. Trace:
I20260709 12:12:05.464883 18094 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:05.487900 17957 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:05.536398 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:05.539525 13305 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:05.557185 12605 rpcz_store.cc:276] 0709 12:12:03.726953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:03.734229 (+  7276us) service_pool.cc:224] Handling call
0709 12:12:03.734620 (+   391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:03.735119 (+   499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:03.735156 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:03.735789 (+   633us) log.cc:844] Serialized 26 byte log entry
0709 12:12:03.963935 (+228146us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:03.963973 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:03.964042 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:05.482953 (+1518911us) spinlock_profiling.cc:243] Waited 41.2 ms on lock 0x7b240033f848. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14857b99c4 00007f14857b7acb 00007f1485972dc9 00007f1485972a55 00007f1485971fb0 00007f14885e8488 00007f14885e8be8 00007f14885c0398 00007f14885bb76a 00007f14885babb6
0709 12:12:05.544109 (+ 61156us) raft_consensus.cc:1685] finished
0709 12:12:05.544132 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:05.550520 (+  6388us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":136,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":98,"spinlock_wait_cycles":94732672,"thread_start_us":227719,"threads_started":1,"wal-append.queue_time_us":287299}
I20260709 12:12:05.562878 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:05.539697 13334 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.635s	user 0.039s	sys 0.050s
I20260709 12:12:05.542724 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2843) took 1135 ms. Trace:
I20260709 12:12:05.372278 18068 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:05.544700 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:05.570863 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:05.556265 12643 connection.cc:256] RPC call timeout handler was delayed by -1.3558928829997967s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18910000000000002s for a response
W20260709 12:12:05.571686 12643 connection.cc:256] RPC call timeout handler was delayed by -1.3486928829997851s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1899s for a response
W20260709 12:12:05.572144 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:05.572544 12643 connection.cc:256] RPC call timeout handler was delayed by -1.2892928829999164s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1965s for a response
W20260709 12:12:05.572883 12643 connection.cc:256] RPC call timeout handler was delayed by -1.237092882999832s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20230000000000004s for a response
W20260709 12:12:05.573226 12643 connection.cc:256] RPC call timeout handler was delayed by -1.2352928829998291s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2025s for a response
W20260709 12:12:05.573467 12643 connection.cc:256] RPC call timeout handler was delayed by -1.2100928829997883s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2053s for a response
W20260709 12:12:05.573724 12643 connection.cc:256] RPC call timeout handler was delayed by -1.1866928829997505s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20790000000000003s for a response
I20260709 12:12:05.525664 12600 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 10: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:05.558903 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:05.559017 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:05.557495 12674 rpcz_store.cc:276] 0709 12:12:04.276676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:04.277744 (+  1068us) service_pool.cc:224] Handling call
0709 12:12:05.532089 (+1254345us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b54003bd654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:05.532285 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":474625280}
W20260709 12:12:05.574031 12643 connection.cc:256] RPC call timeout handler was delayed by -1.1803928829997403s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2086s for a response
I20260709 12:12:05.575690 12603 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
W20260709 12:12:05.576272 12643 connection.cc:256] RPC call timeout handler was delayed by -1.1623928829997112s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2106s for a response
W20260709 12:12:05.576769 12643 connection.cc:256] RPC call timeout handler was delayed by -1.1380928829998993s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21330000000000002s for a response
W20260709 12:12:05.577162 12643 connection.cc:256] RPC call timeout handler was delayed by -1.1299928829998862s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2142s for a response
W20260709 12:12:05.577487 12643 connection.cc:256] RPC call timeout handler was delayed by -1.1047928829998455s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.217s for a response
W20260709 12:12:05.577891 12643 connection.cc:256] RPC call timeout handler was delayed by -1.103892882999844s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2171s for a response
W20260709 12:12:05.578598 12643 connection.cc:256] RPC call timeout handler was delayed by -1.1029928829998425s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21720000000000003s for a response
I20260709 12:12:05.532523 17670 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:05.581399 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2529) took 1247 ms. Trace:
W20260709 12:12:05.585145 12679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.012s	sys 0.017s
W20260709 12:12:05.586546 12679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.012s	sys 0.017s
I20260709 12:12:05.587793 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2483) took 1505 ms. Trace:
I20260709 12:12:05.588261 12522 rpcz_store.cc:276] 0709 12:12:04.082389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:04.114645 (+ 32256us) service_pool.cc:224] Handling call
0709 12:12:04.115142 (+   497us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:04.371231 (+256089us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:04.371261 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:04.375490 (+  4229us) log.cc:844] Serialized 26 byte log entry
0709 12:12:04.615093 (+239603us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:04.615346 (+   253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:04.658364 (+ 43018us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:05.587588 (+929224us) raft_consensus.cc:1685] finished
0709 12:12:05.587605 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:05.587728 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":156555,"raft.queue_time_us":178191,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":34693,"thread_start_us":289596,"threads_started":2,"wal-append.queue_time_us":413682}
W20260709 12:12:05.591017 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 51628.4us
I20260709 12:12:05.591777 13305 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:05.592553 18096 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:05.593006 18096 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:05.593391 18096 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:05.548595 17749 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:05.594514 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:05.595232 12603 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 14.
I20260709 12:12:05.595063 17749 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:05.597194 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2733) took 1586 ms. Trace:
I20260709 12:12:05.598637 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:05.586181 18068 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:05.599541 17691 consensus_meta.cc:287] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.776s	user 0.007s	sys 0.004s
I20260709 12:12:05.599852 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:05.600294 12599 raft_consensus.cc:2410] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 13 because replica has last-logged OpId of term: 11 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 7.
I20260709 12:12:05.600939 18068 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:05.561668 17781 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:05.585687 12600 rpcz_store.cc:276] 0709 12:12:04.334474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:05.470839 (+1136365us) service_pool.cc:224] Handling call
0709 12:12:05.581322 (+110483us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:05.597529 12603 rpcz_store.cc:276] 0709 12:12:04.010483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:05.596296 (+1585813us) service_pool.cc:224] Handling call
0709 12:12:05.596602 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:05.596773 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:05.596798 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:05.596843 (+    45us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:05.596884 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:05.596939 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:05.597065 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:05.542431 18053 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:05.606547 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:05.610632 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2842) took 1223 ms. Trace:
I20260709 12:12:05.644686 12679 rpcz_store.cc:276] 0709 12:12:04.387352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:04.440496 (+ 53144us) service_pool.cc:224] Handling call
0709 12:12:04.440903 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:04.441204 (+   301us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:12:04.995612 (+554408us) log.cc:844] Serialized 14 byte log entry
0709 12:12:05.587284 (+591672us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:05.587343 (+    59us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:05.587399 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:05.587467 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:05.610493 (+ 23026us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":237093,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":30,"raft.queue_time_us":89,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":106,"thread_start_us":1077500,"threads_started":2}
W20260709 12:12:05.603627 18037 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.684s	user 0.013s	sys 0.000s
I20260709 12:12:05.647128 18037 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:05.608386 17781 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:05.651731 17913 consensus_meta.cc:287] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.479s	user 0.015s	sys 0.000s
I20260709 12:12:05.652207 17913 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:05.653744 12528 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:05.638499 12678 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:05.638540 12597 raft_consensus.cc:2410] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:12:05.586380 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:05.602795 18084 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.052s	user 0.000s	sys 0.001s
W20260709 12:12:05.602617 17819 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.867s	user 0.013s	sys 0.001s
I20260709 12:12:05.550518 18076 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:05.586046 17519 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:12:05.661754 17913 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:05.609073 18036 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:05.607076 17775 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:05.663295 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 77005us
I20260709 12:12:05.663926 18054 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:05.665129 18099 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:05.602097 17674 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:05.603498 17957 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:05.694159 17957 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:05.694698 17957 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:05.557943 18081 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:05.611167 18023 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:05.675770 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:05.704849 12671 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:05.677404 12603 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:05.569592 18094 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:05.688923 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:05.716401 18094 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:05.716476 12643 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:05.667213 18053 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:05.688231 13304 leader_election.cc:336] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.796s (SENT)
I20260709 12:12:05.675267 17819 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:05.719867 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:05.693354 18076 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:05.674410 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:05.686254 17977 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:05.729275 12674 raft_consensus.cc:2410] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:12:05.702310 12600 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:12:05.684674 17519 raft_consensus.cc:697] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:05.721004 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2736) took 1709 ms. Trace:
I20260709 12:12:05.732975 12603 rpcz_store.cc:276] 0709 12:12:04.011391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:05.619775 (+1608384us) service_pool.cc:224] Handling call
0709 12:12:05.674994 (+ 55219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:05.720566 (+ 45572us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:05.720912 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:05.738505 12569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.017s	sys 0.040s
I20260709 12:12:05.704027 18023 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:05.739089 12569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.017s	sys 0.040s
I20260709 12:12:05.722543 18099 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:05.722703 18088 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:12:05.752071 18104 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:12:05.754252 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:12:05.760795 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 353883us
I20260709 12:12:05.744848 12643 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:05.761543 13333 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:05.735641 18076 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:05.762758 12643 leader_election.cc:400] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15.
I20260709 12:12:05.763131 12643 leader_election.cc:403] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:12:05.764374 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:05.767323 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:05.742066 18023 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:05.746033 18002 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.839s	user 0.010s	sys 0.004s
I20260709 12:12:05.783178 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:05.784086 13333 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:05.782694 18034 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:05.788400 18034 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:05.782939 17674 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:05.790030 17674 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:05.785645 18062 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15.
I20260709 12:12:05.795254 17519 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 10.11, Last appended by leader: 11, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:05.764307 18053 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:05.798686 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2840) took 1462 ms. Trace:
I20260709 12:12:05.799139 12672 rpcz_store.cc:276] 0709 12:12:04.335969 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:04.390775 (+ 54806us) service_pool.cc:224] Handling call
0709 12:12:04.391285 (+   510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:04.626073 (+234788us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:04.668395 (+ 42322us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:04.668925 (+   530us) log.cc:844] Serialized 26 byte log entry
0709 12:12:05.252599 (+583674us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:05.252661 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:05.252765 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:05.798407 (+545642us) raft_consensus.cc:1685] finished
0709 12:12:05.798433 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:05.798618 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":82402,"raft.queue_time_us":222692,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":43511,"thread_start_us":593213,"threads_started":2,"wal-append.queue_time_us":855654}
I20260709 12:12:05.800971 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:05.803130 18036 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:05.774482 18031 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:05.803908 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:05.804706 13338 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:12:05.801918 12672 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:12:05.806180 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:05.806645 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:05.803210 18081 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:05.773453 17989 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:05.808611 17989 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:05.809022 17989 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:05.809490 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:05.806857 12672 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:05.805771 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:05.810164 13331 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:12:05.814756 18074 raft_consensus.cc:3055] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:05.815479 18074 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:05.816431 18074 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:05.821050 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:05.824723 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:05.825898 12674 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:05.814846 17968 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:05.829365 13305 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:05.836516 13333 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
W20260709 12:12:05.841521 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:05.842938 18031 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:05.844762 18031 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:05.847901 17819 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:05.848116 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 1.01226e+06us
W20260709 12:12:05.850592 12599 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 2723) took 1856 ms (client timeout 1737 ms). Trace:
W20260709 12:12:05.851074 12599 rpcz_store.cc:269] 0709 12:12:03.993806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:03.994223 (+   417us) service_pool.cc:224] Handling call
0709 12:12:05.850324 (+1856101us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b5400231754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:05.850471 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3584982272}
I20260709 12:12:05.852530 12599 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:05.852805 18037 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:05.853155 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2519) took 1791 ms. Trace:
I20260709 12:12:05.853562 12599 rpcz_store.cc:276] 0709 12:12:04.061353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:05.851969 (+1790616us) service_pool.cc:224] Handling call
0709 12:12:05.852340 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:05.852908 (+   568us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:05.853100 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:05.854647 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:05.855335 12678 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:05.843060 17992 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:12:05.855985 12604 raft_consensus.cc:1217] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Deduplicated request from leader. Original: 12.11->[14.12-14.12]   Dedup: 14.12->[]
I20260709 12:12:05.856572 12604 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:12:05.857165 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2523) took 1793 ms. Trace:
I20260709 12:12:05.857602 18031 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:05.862879 17760 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:05.864351 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2844) took 1437 ms. Trace:
I20260709 12:12:05.875937 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:05.864614 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2488) took 1494 ms. Trace:
I20260709 12:12:05.876744 12673 rpcz_store.cc:276] 0709 12:12:04.369881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:04.438770 (+ 68889us) service_pool.cc:224] Handling call
0709 12:12:04.439262 (+   492us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:05.083886 (+644624us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:05.864279 (+780393us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b540016eed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:05.864535 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1378182528}
I20260709 12:12:05.867596 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2486) took 1612 ms. Trace:
W20260709 12:12:05.868961 12602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2521) took 1807 ms (client timeout 1754 ms). Trace:
I20260709 12:12:05.876901 12678 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:05.857492 12604 rpcz_store.cc:276] 0709 12:12:04.063509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:05.855490 (+1791981us) service_pool.cc:224] Handling call
0709 12:12:05.855874 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:05.856917 (+  1043us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:05.857110 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:05.874366 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2486) took 1543 ms. Trace:
I20260709 12:12:05.881325 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:05.874257 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 111666us
I20260709 12:12:05.881496 12677 rpcz_store.cc:276] 0709 12:12:04.330582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:04.333548 (+  2966us) service_pool.cc:224] Handling call
0709 12:12:05.863498 (+1529950us) spinlock_profiling.cc:243] Waited 491 ms on lock 0x7b54000933d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:05.874287 (+ 10789us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1129901440}
W20260709 12:12:05.879066 12602 rpcz_store.cc:269] 0709 12:12:04.061905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:04.693873 (+631968us) service_pool.cc:224] Handling call
0709 12:12:05.865973 (+1172100us) spinlock_profiling.cc:243] Waited 401 ms on lock 0x7b5400231254. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:05.868857 (+  2884us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":922025472}
I20260709 12:12:05.736579 17977 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:05.886541 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2531) took 1464 ms. Trace:
I20260709 12:12:05.878003 12530 rpcz_store.cc:276] 0709 12:12:04.254869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:04.688838 (+433969us) service_pool.cc:224] Handling call
0709 12:12:04.706535 (+ 17697us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:05.391780 (+685245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:05.867356 (+475576us) spinlock_profiling.cc:243] Waited 684 ms on lock 0x7b5400105ed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:05.867536 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1574286208}
I20260709 12:12:05.876191 12676 rpcz_store.cc:276] 0709 12:12:04.427443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:04.578005 (+150562us) service_pool.cc:224] Handling call
0709 12:12:04.601483 (+ 23478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:04.700784 (+ 99301us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:12:04.701265 (+   481us) log.cc:844] Serialized 14 byte log entry
0709 12:12:05.082755 (+381490us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:05.082816 (+    61us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:05.082878 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:05.864104 (+781226us) spinlock_profiling.cc:243] Waited 99 ms on lock 0x7b5400010c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:05.864152 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:05.864295 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15376,"prepare.queue_time_us":7804,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":73,"raft.queue_time_us":9979,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":12130,"spinlock_wait_cycles":227684096,"thread_start_us":380807,"threads_started":1,"wal-append.queue_time_us":878891}
I20260709 12:12:05.894302 12602 rpcz_store.cc:276] 0709 12:12:04.424514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:05.885128 (+1460614us) service_pool.cc:224] Handling call
0709 12:12:05.885561 (+   433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:05.885998 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:05.886038 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:05.886105 (+    67us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:05.886174 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:05.886296 (+   122us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:05.886461 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:05.898561 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:05.899626 13333 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:05.907779 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:05.907841 12496 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:05.911438 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:05.911674 12525 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
W20260709 12:12:05.877817 18075 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.619s	user 0.006s	sys 0.005s
I20260709 12:12:05.857978 18034 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:05.934625 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:05.884374 17981 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:05.912057 18009 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.760s	user 0.012s	sys 0.000s
I20260709 12:12:05.920020 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:05.926326 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:05.946269 12524 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:05.939769 12525 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 16: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:05.938557 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:05.938560 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:05.947249 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:05.947463 12673 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:12:05.913041 12602 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 12 index: 8. (term mismatch)
I20260709 12:12:05.938555 17670 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:05.943429 17982 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:05.953620 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:05.954901 12571 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:05.955689 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:05.957454 17977 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:05.958963 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:05.565400 12680 rpcz_store.cc:276] 0709 12:12:04.407256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:04.552720 (+145464us) service_pool.cc:224] Handling call
0709 12:12:04.553111 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:04.564396 (+ 11285us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:12:04.980764 (+416368us) log.cc:844] Serialized 14 byte log entry
0709 12:12:05.542027 (+561263us) log.cc:844] Serialized 14 byte log entry
0709 12:12:05.542267 (+   240us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:05.542323 (+    56us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:05.542394 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:05.542493 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:05.542649 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":121065,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":76,"raft.queue_time_us":20946,"spinlock_wait_cycles":12185728,"thread_start_us":899422,"threads_started":2}
I20260709 12:12:05.959865 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:05.962452 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:05.962996 17981 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:05.964830 18101 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.686s	user 0.001s	sys 0.002s
I20260709 12:12:05.965977 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:06.038019 12679 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
I20260709 12:12:05.913990 18044 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.056s
W20260709 12:12:05.973140 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:05.964880 12643 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:05.898187 17957 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:05.976325 17989 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:05.977105 17775 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:05.889083 17760 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:05.985095 13337 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 692) took 2188 ms (client timeout 1835 ms). Trace:
I20260709 12:12:06.003404 17773 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:12:05.988648 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:06.000981 17708 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:05.932886 18075 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:05.991669 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 263709us
I20260709 12:12:05.996594 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:05.992628 12602 pending_rounds.cc:85] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 7
I20260709 12:12:06.073230 12602 pending_rounds.cc:107] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 11.8
I20260709 12:12:06.073573 12602 raft_consensus.cc:2889] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:06.001447 12678 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
W20260709 12:12:06.001654 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46849us
I20260709 12:12:06.073976 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:05.977638 18009 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:06.030221 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:06.074656 12571 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:06.037905 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:06.075647 13331 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 11 index: 10. (index mismatch)
I20260709 12:12:06.075834 12523 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
W20260709 12:12:05.970549 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:06.017670 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:06.039951 12672 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:06.045459 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:05.978863 18032 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:06.001343 17819 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:06.000052 17982 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:05.985311 12671 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
W20260709 12:12:06.058921 13337 rpcz_store.cc:269] 0709 12:12:03.796439 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:03.796663 (+   224us) service_pool.cc:224] Handling call
0709 12:12:05.984920 (+2188257us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54004439d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:05.985024 (+   104us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2479233920}
I20260709 12:12:05.990429 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:06.074296 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2870) took 1074 ms. Trace:
I20260709 12:12:06.000458 12677 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:12:06.096969 12678 rpcz_store.cc:276] 0709 12:12:04.999551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:05.955351 (+955800us) service_pool.cc:224] Handling call
0709 12:12:06.074229 (+118878us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:06.078404 12496 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:06.053755 17781 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:06.079594 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:06.100538 12680 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:06.084133 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:06.045310 18037 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:06.047072 17691 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:06.103539 17691 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:06.106645 17781 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:06.098995 18032 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:06.107286 18037 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:06.108403 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:06.109130 17781 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:06.088200 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:06.109316 13337 raft_consensus.cc:2410] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:12:06.090864 13338 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 15: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:06.091240 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2533) took 1544 ms. Trace:
I20260709 12:12:06.109429 18037 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:06.110751 18032 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:06.110903 17957 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:05.985139 18096 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:06.110369 12602 rpcz_store.cc:276] 0709 12:12:04.547337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:05.912305 (+1364968us) service_pool.cc:224] Handling call
0709 12:12:05.912663 (+   358us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.090963 (+178300us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.091179 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:06.112352 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:06.112664 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:06.112815 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:06.113322 18096 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:06.113528 12602 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:12:06.094775 17708 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:06.114395 13334 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: [13.13-13.13]
I20260709 12:12:06.099504 17819 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:06.114028 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:06.114913 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:06.115509 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:06.119024 17819 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:06.119686 17819 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:06.120209 17819 raft_consensus.cc:697] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:06.115279 17773 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:06.099771 17982 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:06.121413 17773 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:06.122550 17773 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:12:06.128161 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 163384us
I20260709 12:12:06.128880 17819 consensus_queue.cc:237] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:06.137758 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2492) took 1586 ms. Trace:
I20260709 12:12:06.138286 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:06.138607 12672 rpcz_store.cc:276] 0709 12:12:04.551365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.091874 (+1540509us) service_pool.cc:224] Handling call
0709 12:12:06.136944 (+ 45070us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.137249 (+   305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:06.137290 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:06.137355 (+    65us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:06.137415 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.137495 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:06.137667 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:06.139914 12523 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 12: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:06.140801 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:06.141465 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:06.150803 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:06.115602 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:06.151087 13339 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:06.105888 17691 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:06.156392 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:06.161087 12672 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:12:06.143150 18002 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:05.995790 12674 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:06.162322 12525 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:06.165971 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:06.185075 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:06.003865 17670 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:06.181206 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2442) took 1488 ms. Trace:
I20260709 12:12:06.087528 18081 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:06.188550 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:06.182386 17781 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:06.193065 12528 rpcz_store.cc:276] 0709 12:12:04.692404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:04.705455 (+ 13051us) service_pool.cc:224] Handling call
0709 12:12:06.180922 (+1475467us) spinlock_profiling.cc:243] Waited 860 ms on lock 0x7b5400015754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:06.181121 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1977522560}
I20260709 12:12:06.187659 17708 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:06.194096 12602 raft_consensus.cc:2393] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 11: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:06.193986 12678 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:12:06.222329 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2494) took 1657 ms. Trace:
I20260709 12:12:06.222743 12678 rpcz_store.cc:276] 0709 12:12:04.565266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.097899 (+1532633us) service_pool.cc:224] Handling call
0709 12:12:06.193622 (+ 95723us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.221988 (+ 28366us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.222262 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:06.090679 18068 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:06.200120 12572 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:06.215549 12571 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:06.218048 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:06.230222 12674 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:06.230607 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:06.098330 17775 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:06.226538 12569 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:06.233222 12569 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:06.234110 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:06.234866 12569 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:06.238348 12680 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:06.246039 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:06.246856 12523 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
W20260709 12:12:06.251403 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 121597us
I20260709 12:12:06.252311 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:06.194556 18053 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:06.184900 17749 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:06.260598 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:06.265588 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:06.209523 17836 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:12:06.261250 17760 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:06.261857 18034 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:06.262969 18112 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:06.261330 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:06.276309 12528 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:06.245297 17691 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:06.260882 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:06.270654 12601 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.009s	sys 0.013s
I20260709 12:12:06.275131 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:06.300670 12601 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.010s	sys 0.013s
I20260709 12:12:06.301254 12678 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:06.302521 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:06.303267 12678 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:12:06.266873 12602 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:12:06.342482 12598 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 2737) took 2330 ms (client timeout 2151 ms). Trace:
W20260709 12:12:06.343106 12598 rpcz_store.cc:269] 0709 12:12:04.011712 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:04.019958 (+  8246us) service_pool.cc:224] Handling call
0709 12:12:06.342049 (+2322091us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b5400375354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:06.342371 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3286296320}
I20260709 12:12:06.344612 12603 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:06.347376 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:06.351326 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:06.352667 12602 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:06.270630 17674 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:06.230139 17781 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:06.355885 12674 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:06.275350 18034 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:06.347524 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2739) took 2335 ms. Trace:
I20260709 12:12:06.279997 18112 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:06.299293 18037 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:06.371484 12605 rpcz_store.cc:276] 0709 12:12:04.012356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:05.642260 (+1629904us) service_pool.cc:224] Handling call
0709 12:12:05.642680 (+   420us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:05.642970 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:12:05.896741 (+253771us) log.cc:844] Serialized 14 byte log entry
0709 12:12:06.347145 (+450404us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:06.347217 (+    72us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:06.347269 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.347335 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:06.347467 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":163516,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":2913,"thread_start_us":684670,"threads_started":2}
I20260709 12:12:06.340939 17519 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:12:06.346228 18088 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:12:06.380699 12673 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:12:06.378095 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:06.384788 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2495) took 1800 ms. Trace:
I20260709 12:12:06.394574 12676 rpcz_store.cc:276] 0709 12:12:04.584930 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.354234 (+1769304us) service_pool.cc:224] Handling call
0709 12:12:06.383643 (+ 29409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.383902 (+   259us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:06.383940 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:06.383998 (+    58us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:06.384052 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.384346 (+   294us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:06.384674 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:06.245434 18068 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:06.401562 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:06.388128 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:06.375267 12674 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:12:06.272395 18044 consensus_queue.cc:1059] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.102s
I20260709 12:12:06.341228 17957 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:06.354825 17708 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:12:06.392727 18023 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.581s	user 0.011s	sys 0.000s
I20260709 12:12:06.393292 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2848) took 1807 ms. Trace:
I20260709 12:12:06.500247 12673 rpcz_store.cc:276] 0709 12:12:04.585746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.340573 (+1754827us) service_pool.cc:224] Handling call
0709 12:12:06.373802 (+ 33229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.393009 (+ 19207us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.393235 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:06.373512 18034 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:06.403841 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2851) took 1690 ms. Trace:
I20260709 12:12:06.504961 12674 rpcz_store.cc:276] 0709 12:12:04.713541 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.362610 (+1649069us) service_pool.cc:224] Handling call
0709 12:12:06.375046 (+ 12436us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:06.403520 (+ 28474us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.403773 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:06.407846 12606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2524) took 2340 ms (client timeout 2524 ms). Trace:
I20260709 12:12:06.408413 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2517) took 2347 ms. Trace:
I20260709 12:12:06.507041 12603 rpcz_store.cc:276] 0709 12:12:04.060640 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:05.733974 (+1673334us) service_pool.cc:224] Handling call
0709 12:12:06.344329 (+610355us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.349303 (+  4974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.408145 (+ 58842us) spinlock_profiling.cc:243] Waited 598 ms on lock 0x7b5400375350. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:06.408332 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1374342912}
I20260709 12:12:06.411170 17781 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:06.414301 17989 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:06.414619 12646 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:06.520042 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 105522us
I20260709 12:12:06.520963 12646 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:06.415613 18009 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:06.272553 17749 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:06.386761 18112 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:06.440392 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:06.526011 12674 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 10 index: 7. (index mismatch)
I20260709 12:12:06.526701 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:06.437752 12528 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15.
W20260709 12:12:06.434514 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 182276us
I20260709 12:12:06.390301 18037 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:06.465901 17957 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:06.468463 17582 raft_consensus.cc:3055] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:12:06.470026 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:06.471230 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:06.471971 12525 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
I20260709 12:12:06.392333 17519 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:06.473984 17670 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:06.471235 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2735) took 2460 ms. Trace:
I20260709 12:12:06.486713 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2853) took 1677 ms. Trace:
W20260709 12:12:06.490284 12597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2539) took 1804 ms (client timeout 1571 ms). Trace:
I20260709 12:12:06.492412 12529 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:12:06.493924 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:06.300997 17691 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:06.501786 18023 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:06.505578 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 437969us
I20260709 12:12:06.597868 12494 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:06.599408 12494 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:06.538488 18037 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:06.517998 12672 pending_rounds.cc:85] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 4
I20260709 12:12:06.522883 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2856) took 1695 ms. Trace:
I20260709 12:12:06.529129 13305 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:12:06.523550 12599 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.021s	sys 0.018s
I20260709 12:12:06.530308 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:06.531304 18090 raft_consensus.cc:2764] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:12:06.531944 18054 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:06.534663 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:06.501363 17981 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:06.526700 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2861) took 1665 ms. Trace:
W20260709 12:12:06.536702 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 458425us
I20260709 12:12:06.550714 12598 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:06.551524 12524 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:06.564700 12525 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 9 index: 8. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
W20260709 12:12:06.538909 18031 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.560s	user 0.014s	sys 0.021s
W20260709 12:12:06.582228 12522 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.021s	sys 0.013s
W20260709 12:12:06.728260 12522 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.779s	user 0.021s	sys 0.013s
I20260709 12:12:06.581619 12600 rpcz_store.cc:276] 0709 12:12:04.011075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:05.611846 (+1600771us) service_pool.cc:224] Handling call
0709 12:12:05.678602 (+ 66756us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:05.731902 (+ 53300us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.470920 (+739018us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b540023fd54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:06.471166 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53574400}
I20260709 12:12:06.581828 17519 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:12:06.582607 12597 rpcz_store.cc:269] 0709 12:12:04.685721 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:04.845576 (+159855us) service_pool.cc:224] Handling call
0709 12:12:06.490012 (+1644436us) spinlock_profiling.cc:243] Waited 764 ms on lock 0x7b5400079c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:06.490178 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1757649152}
I20260709 12:12:06.499504 12676 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:06.595760 17691 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:06.506273 12606 rpcz_store.cc:269] 0709 12:12:04.067062 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:05.066198 (+999136us) service_pool.cc:224] Handling call
0709 12:12:06.407540 (+1341342us) spinlock_profiling.cc:243] Waited 115 ms on lock 0x7b54002f2954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:06.407740 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":263891200}
W20260709 12:12:06.602854 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:06.542950 17582 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:06.647980 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:06.615940 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:06.649727 12672 pending_rounds.cc:107] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 11.5
I20260709 12:12:06.650314 12673 rpcz_store.cc:276] 0709 12:12:04.827079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.501582 (+1674503us) service_pool.cc:224] Handling call
0709 12:12:06.522059 (+ 20477us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.522392 (+   333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:06.522446 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:06.522508 (+    62us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:06.522558 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.522633 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:06.522784 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:06.651428 12599 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.025s	sys 0.021s
I20260709 12:12:06.660768 12674 rpcz_store.cc:276] 0709 12:12:04.860940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.506038 (+1645098us) service_pool.cc:224] Handling call
0709 12:12:06.525658 (+ 19620us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.526456 (+   798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.526647 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:06.724169 12604 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:06.595724 17670 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:06.747550 17670 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:06.735527 17957 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:06.735988 17760 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:12:06.727059 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 198050us
I20260709 12:12:06.729946 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:06.752074 12598 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:12:06.756772 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2437) took 2672 ms. Trace:
I20260709 12:12:06.757318 12598 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:12:06.757978 17749 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:06.730041 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:06.708688 17553 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:06.758778 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2744) took 1995 ms. Trace:
W20260709 12:12:06.723229 17982 consensus_meta.cc:287] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.601s	user 0.017s	sys 0.001s
I20260709 12:12:06.593767 18053 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.129s
I20260709 12:12:06.754923 12597 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:06.734735 18068 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:06.759397 12598 rpcz_store.cc:276] 0709 12:12:04.763741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.756550 (+1992809us) service_pool.cc:224] Handling call
0709 12:12:06.756999 (+   449us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:06.757905 (+   906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.758683 (+   778us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:06.761049 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2537) took 2076 ms. Trace:
I20260709 12:12:06.759028 17553 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:06.759737 17982 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:06.763132 12598 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: [10.7-10.7]
I20260709 12:12:06.657195 18054 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:06.763958 18054 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:06.764158 17982 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:06.764317 17553 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:06.764846 17982 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:06.765271 17982 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:06.765532 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2746) took 1967 ms. Trace:
I20260709 12:12:06.765628 17982 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:06.648082 18088 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.087s
I20260709 12:12:06.766423 12523 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 14: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:06.767995 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:06.768361 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2741) took 2394 ms. Trace:
I20260709 12:12:06.769251 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:06.584190 12527 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:12:06.770017 18023 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:06.769001 12599 rpcz_store.cc:276] 0709 12:12:04.373622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:05.884446 (+1510824us) service_pool.cc:224] Handling call
0709 12:12:05.925898 (+ 41452us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:05.927046 (+  1148us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:12:05.949920 (+ 22874us) log.cc:844] Serialized 14 byte log entry
0709 12:12:06.767911 (+817991us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:06.767976 (+    65us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:06.768025 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.768104 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:06.768288 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20815,"prepare.queue_time_us":27003,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":21786,"raft.run_cpu_time_us":81,"raft.run_wall_time_us":82,"thread_start_us":796719,"threads_started":1}
I20260709 12:12:06.770982 12527 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:06.772250 12527 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:06.750108 17670 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:06.582059 12680 rpcz_store.cc:276] 0709 12:12:04.808918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.485763 (+1676845us) service_pool.cc:224] Handling call
0709 12:12:06.486166 (+   403us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.486380 (+   214us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:06.486403 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:06.486442 (+    39us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:06.486477 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.486526 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:06.486650 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:06.772030 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2747) took 1963 ms. Trace:
I20260709 12:12:06.773540 12680 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:06.772950 12599 rpcz_store.cc:276] 0709 12:12:04.808560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.770930 (+1962370us) service_pool.cc:224] Handling call
0709 12:12:06.771244 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.771511 (+   267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:06.771561 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:06.771629 (+    68us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:06.771705 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.771786 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:06.771954 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:06.774617 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2867) took 1853 ms. Trace:
W20260709 12:12:06.774914 18125 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
I20260709 12:12:06.774984 12680 rpcz_store.cc:276] 0709 12:12:04.920872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.773063 (+1852191us) service_pool.cc:224] Handling call
0709 12:12:06.773362 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.774299 (+   937us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.774546 (+   247us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:06.746297 12672 raft_consensus.cc:2889] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:06.738688 17869 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:06.776613 12680 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:06.776751 17869 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 15 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:06.759290 17582 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:06.550603 17773 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:06.777276 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2501) took 1781 ms. Trace:
I20260709 12:12:06.792416 12646 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:06.792937 12680 rpcz_store.cc:276] 0709 12:12:04.996170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.776131 (+1779961us) service_pool.cc:224] Handling call
0709 12:12:06.776418 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.777034 (+   616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.777222 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:06.759847 12600 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:12:06.793941 12597 rpcz_store.cc:276] 0709 12:12:04.685129 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.742272 (+2057143us) service_pool.cc:224] Handling call
0709 12:12:06.754592 (+ 12320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.760749 (+  6157us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.760979 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:06.794884 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2745) took 2030 ms. Trace:
W20260709 12:12:06.780932 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:06.795765 12524 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:06.788311 12603 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 13: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:12:06.784732 12675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2496) took 1894 ms (client timeout 2182 ms). Trace:
I20260709 12:12:06.795437 12600 rpcz_store.cc:276] 0709 12:12:04.763993 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.759238 (+1995245us) service_pool.cc:224] Handling call
0709 12:12:06.759591 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:06.794485 (+ 34894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.794805 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:06.736955 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:06.797169 12675 rpcz_store.cc:269] 0709 12:12:04.890826 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:05.460750 (+569924us) service_pool.cc:224] Handling call
0709 12:12:06.784346 (+1323596us) spinlock_profiling.cc:243] Waited 799 ms on lock 0x7b5400092ed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:06.784622 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1837930880}
I20260709 12:12:06.799207 12675 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:12:06.800067 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2871) took 1567 ms. Trace:
I20260709 12:12:06.766004 12598 rpcz_store.cc:276] 0709 12:12:04.798055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.762494 (+1964439us) service_pool.cc:224] Handling call
0709 12:12:06.762883 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:06.763544 (+   661us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.765475 (+  1931us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:06.757179 12526 rpcz_store.cc:276] 0709 12:12:04.084109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:04.118676 (+ 34567us) service_pool.cc:224] Handling call
0709 12:12:04.165436 (+ 46760us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:04.362534 (+197098us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:04.362570 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:04.376255 (+ 13685us) log.cc:844] Serialized 26 byte log entry
0709 12:12:05.949210 (+1572955us) spinlock_profiling.cc:243] Waited 5.58 ms on lock 0x7b18003d5d00. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:12:06.371702 (+422492us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:06.371753 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.475692 (+103939us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54003f7354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:06.475781 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:06.756534 (+280753us) raft_consensus.cc:1685] finished
0709 12:12:06.756555 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:06.756686 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":208,"raft.queue_time_us":77597,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":33818,"spinlock_wait_cycles":398041216,"thread_start_us":422251,"threads_started":1,"wal-append.queue_time_us":2230038}
W20260709 12:12:06.737898 18031 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.749s	user 0.014s	sys 0.021s
I20260709 12:12:06.800769 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2757) took 1535 ms. Trace:
I20260709 12:12:06.808024 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2852) took 2019 ms. Trace:
I20260709 12:12:06.808724 12676 rpcz_store.cc:276] 0709 12:12:04.788693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.396775 (+1608082us) service_pool.cc:224] Handling call
0709 12:12:06.472099 (+ 75324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.736730 (+264631us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.807729 (+ 70999us) spinlock_profiling.cc:243] Waited 24.9 ms on lock 0x7b5400091fd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:06.807929 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":57197824}
I20260709 12:12:06.809633 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2742) took 2435 ms. Trace:
I20260709 12:12:06.800452 12675 rpcz_store.cc:276] 0709 12:12:05.232324 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.798621 (+1566297us) service_pool.cc:224] Handling call
0709 12:12:06.798961 (+   340us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:06.799798 (+   837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.800010 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:06.771023 12674 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260709 12:12:06.811833 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2863) took 1892 ms. Trace:
I20260709 12:12:06.800088 12680 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 11 index: 10. (term mismatch)
W20260709 12:12:06.805406 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 189584us
I20260709 12:12:06.807523 12600 rpcz_store.cc:276] 0709 12:12:05.265959 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.797812 (+1531853us) service_pool.cc:224] Handling call
0709 12:12:06.800185 (+  2373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.800440 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:06.800472 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:06.800521 (+    49us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:06.800557 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.800614 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:06.800715 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:06.812871 12680 pending_rounds.cc:85] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 9
I20260709 12:12:06.813282 12680 pending_rounds.cc:107] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 12.10
I20260709 12:12:06.813666 12494 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:06.814270 12496 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:06.810119 12604 rpcz_store.cc:276] 0709 12:12:04.373971 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:05.888147 (+1514176us) service_pool.cc:224] Handling call
0709 12:12:05.913260 (+ 25113us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.747014 (+833754us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.809335 (+ 62321us) spinlock_profiling.cc:243] Waited 768 ms on lock 0x7b5400374954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:06.809559 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1766468864}
I20260709 12:12:06.816685 12496 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:06.779912 17582 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:12:06.788038 18124 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.131s	user 0.002s	sys 0.001s
I20260709 12:12:06.827648 12597 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:06.832762 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2756) took 1578 ms. Trace:
I20260709 12:12:06.832914 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2504) took 1336 ms. Trace:
I20260709 12:12:06.761221 18068 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:12:06.782907 18128 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:12:06.830242 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:06.787977 17773 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:06.831723 12676 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:06.833243 12597 rpcz_store.cc:276] 0709 12:12:05.254871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.794841 (+1539970us) service_pool.cc:224] Handling call
0709 12:12:06.827438 (+ 32597us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.832433 (+  4995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.832686 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:06.849669 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 319453us
I20260709 12:12:06.850890 12643 leader_election.cc:400] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12.
I20260709 12:12:06.851434 12643 leader_election.cc:403] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Cancelling election due to peer responding with higher term
I20260709 12:12:06.813818 18074 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:06.778191 18075 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:06.861702 18068 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:06.862257 18068 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:06.862622 18068 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:06.862984 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:06.817104 17553 consensus_queue.cc:1175] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" responder_term: 15 status { last_received { term: 14 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 14. Current term is 15. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:12:06.864776 18031 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:06.865213 18031 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:06.865587 18031 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:06.864501 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:06.871804 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:06.833479 12675 rpcz_store.cc:276] 0709 12:12:05.496916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.830991 (+1334075us) service_pool.cc:224] Handling call
0709 12:12:06.832328 (+  1337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.832509 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:06.832545 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:06.832596 (+    51us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:06.832642 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.832715 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:06.832843 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:06.816653 18088 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:06.817246 18081 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:12:06.856415 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2503) took 1359 ms. Trace:
I20260709 12:12:06.897969 12676 rpcz_store.cc:276] 0709 12:12:05.496639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.827568 (+1330929us) service_pool.cc:224] Handling call
0709 12:12:06.831524 (+  3956us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.853706 (+ 22182us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.856331 (+  2625us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:06.813922 17670 raft_consensus.cc:2764] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:12:06.913472 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:06.882336 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2507) took 1384 ms. Trace:
W20260709 12:12:06.884083 12671 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.034s	sys 0.046s
W20260709 12:12:06.954630 12671 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.709s	user 0.034s	sys 0.046s
I20260709 12:12:06.813576 12680 raft_consensus.cc:2889] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:06.806345 17947 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.050s
W20260709 12:12:06.856030 12677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.516s	user 0.011s	sys 0.013s
I20260709 12:12:06.954504 12675 rpcz_store.cc:276] 0709 12:12:05.497745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.881096 (+1383351us) service_pool.cc:224] Handling call
0709 12:12:06.881551 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.881811 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:06.881850 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:06.881913 (+    63us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:06.881983 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.882059 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:06.882255 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:06.902601 17836 consensus_meta.cc:287] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.630s	user 0.010s	sys 0.001s
I20260709 12:12:06.957288 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:06.957962 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2869) took 1959 ms. Trace:
I20260709 12:12:06.958752 12680 rpcz_store.cc:276] 0709 12:12:04.999132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.798902 (+1799770us) service_pool.cc:224] Handling call
0709 12:12:06.799287 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.957612 (+158325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.957887 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:06.812378 12674 rpcz_store.cc:276] 0709 12:12:04.919191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.770386 (+1851195us) service_pool.cc:224] Handling call
0709 12:12:06.770717 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.811523 (+ 40806us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.811766 (+   243us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:06.858804 18002 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.598s	user 0.013s	sys 0.017s
W20260709 12:12:06.915714 12572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.648s	user 0.009s	sys 0.021s
W20260709 12:12:06.960342 12572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.009s	sys 0.021s
W20260709 12:12:06.960753 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 695194us
I20260709 12:12:06.777834 17869 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:06.859534 18053 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election lost for term 11. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12.
I20260709 12:12:06.891443 17749 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:06.891147 18088 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:06.932178 12597 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:12:06.935595 17519 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:06.940254 17982 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:06.915254 18075 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:06.903848 18096 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:06.943918 12603 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 10. (term mismatch)
I20260709 12:12:07.008853 12603 pending_rounds.cc:85] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 9
I20260709 12:12:07.009181 12603 pending_rounds.cc:107] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 13.10
I20260709 12:12:07.009486 12603 raft_consensus.cc:2889] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:06.916172 17760 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:07.025219 12606 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: []
I20260709 12:12:07.027002 12523 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:06.925463 18090 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader election won for term 9
I20260709 12:12:06.948168 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2509) took 1362 ms. Trace:
I20260709 12:12:06.961658 12572 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:07.029716 18090 raft_consensus.cc:697] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:06.998641 18088 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:07.030658 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 69158.5us
I20260709 12:12:07.030828 18090 consensus_queue.cc:237] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:07.031751 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:07.034075 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:07.035135 18081 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:07.035933 18081 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:07.038780 17708 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:06.971275 17773 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:07.039877 17773 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:06.937767 18037 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:07.044363 18037 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:06.970993 18002 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.014s	sys 0.018s
I20260709 12:12:07.045233 18002 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:07.046578 18002 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:07.047856 13335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 702) took 1060 ms. Trace:
I20260709 12:12:06.974431 18132 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:07.050570 18132 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:07.054340 12531 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
W20260709 12:12:07.057574 12531 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2488) took 1182 ms (client timeout 1500 ms). Trace:
W20260709 12:12:06.971406 17775 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.024s	sys 0.063s
I20260709 12:12:07.075114 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2743) took 2311 ms. Trace:
I20260709 12:12:07.075809 12606 rpcz_store.cc:276] 0709 12:12:04.763422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.745077 (+1981655us) service_pool.cc:224] Handling call
0709 12:12:06.745479 (+   402us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:07.025722 (+280243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:07.074753 (+ 49031us) spinlock_profiling.cc:243] Waited 279 ms on lock 0x7b540023f854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:07.075026 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":642723584}
I20260709 12:12:06.949805 17913 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:12:06.999212 12678 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.005s	sys 0.021s
I20260709 12:12:07.002318 12600 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 11 index: 6. (index mismatch)
W20260709 12:12:07.078365 12678 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.005s	sys 0.021s
W20260709 12:12:07.021582 18136 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
W20260709 12:12:07.022642 18135 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:12:07.023554 17982 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:07.062551 18129 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:12:07.034499 17781 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:07.048448 13335 rpcz_store.cc:276] 0709 12:12:05.987499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:05.987718 (+   219us) service_pool.cc:224] Handling call
0709 12:12:06.043206 (+ 55488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:06.043716 (+   510us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:12:06.043758 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:06.055511 (+ 11753us) log.cc:844] Serialized 26 byte log entry
0709 12:12:06.574701 (+519190us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:06.574773 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.574888 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:07.047528 (+472640us) raft_consensus.cc:1685] finished
0709 12:12:07.047557 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:07.047744 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":174,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":9334,"thread_start_us":483897,"threads_started":1,"wal-append.queue_time_us":902843}
W20260709 12:12:06.949234 17819 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.685s	user 0.030s	sys 0.010s
I20260709 12:12:07.029567 12676 rpcz_store.cc:276] 0709 12:12:05.585597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.929093 (+1343496us) service_pool.cc:224] Handling call
0709 12:12:06.947587 (+ 18494us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.947769 (+   182us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:06.947801 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:06.947857 (+    56us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:06.947893 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.947944 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:06.948094 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:07.067622 12531 rpcz_store.cc:269] 0709 12:12:05.875489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:05.875633 (+   144us) service_pool.cc:224] Handling call
0709 12:12:07.057281 (+1181648us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400053054. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:07.057475 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2708319104}
I20260709 12:12:07.101178 18054 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:07.101526 12646 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:07.074658 12569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.808s	user 0.015s	sys 0.032s
W20260709 12:12:07.103298 12569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.837s	user 0.016s	sys 0.032s
W20260709 12:12:07.104286 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 869453us
I20260709 12:12:07.062933 18053 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:12:07.021139 17519 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:07.008260 18075 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:07.075133 17775 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.805s	user 0.024s	sys 0.064s
I20260709 12:12:07.063061 18096 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:07.034111 17957 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:07.037173 18081 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:07.034344 18036 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:07.090643 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2445) took 1197 ms. Trace:
I20260709 12:12:07.110756 17760 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:12:07.091434 18032 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:12:07.149730 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 44271.5us
W20260709 12:12:07.098614 18127 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.245s	user 0.002s	sys 0.000s
I20260709 12:12:07.081552 17773 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:07.079005 12571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.817s	user 0.000s	sys 0.047s
W20260709 12:12:07.186190 12571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.925s	user 0.000s	sys 0.048s
W20260709 12:12:07.187079 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 971568us
I20260709 12:12:07.188124 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:07.106681 17819 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.030s	sys 0.010s
I20260709 12:12:07.129536 12675 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:07.130627 18068 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:07.139782 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:07.192935 12531 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:07.140791 18054 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:07.143810 17957 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:07.194263 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:07.141785 18096 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:07.147856 12522 rpcz_store.cc:276] 0709 12:12:05.893187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:05.893334 (+   147us) service_pool.cc:224] Handling call
0709 12:12:05.893702 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:05.897457 (+  3755us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:05.897495 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:05.929498 (+ 32003us) log.cc:844] Serialized 26 byte log entry
0709 12:12:06.728775 (+799277us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:06.728835 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.728929 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:07.090402 (+361473us) raft_consensus.cc:1685] finished
0709 12:12:07.090421 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:07.090577 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":176,"raft.queue_time_us":325,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":15942,"thread_start_us":779826,"threads_started":1,"wal-append.queue_time_us":993052}
I20260709 12:12:07.092341 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2845) took 2635 ms. Trace:
I20260709 12:12:07.197204 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:07.128975 18075 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:07.171631 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:07.202282 12601 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.854s	user 0.018s	sys 0.036s
I20260709 12:12:07.130455 17775 raft_consensus.cc:697] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:07.202725 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:07.197450 12679 rpcz_store.cc:276] 0709 12:12:04.456675 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.072541 (+1615866us) service_pool.cc:224] Handling call
0709 12:12:06.130646 (+ 58105us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:06.131156 (+   510us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:06.131191 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:06.131881 (+   690us) log.cc:844] Serialized 52 byte log entry
0709 12:12:06.499480 (+367599us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:06.524268 (+ 24788us) log.cc:844] Serialized 14 byte log entry
0709 12:12:06.536693 (+ 12425us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:06.536894 (+   201us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:07.092069 (+555175us) raft_consensus.cc:1685] finished
0709 12:12:07.092108 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:07.092276 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14222,"prepare.queue_time_us":18285,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":91,"raft.queue_time_us":45253,"raft.run_cpu_time_us":499,"raft.run_wall_time_us":23043,"thread_start_us":367097,"threads_started":1,"wal-append.queue_time_us":783660}
W20260709 12:12:07.208204 12601 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.906s	user 0.019s	sys 0.036s
W20260709 12:12:07.149480 17899 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.889s	user 0.017s	sys 0.030s
W20260709 12:12:07.190613 18112 consensus_meta.cc:287] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.665s	user 0.006s	sys 0.005s
W20260709 12:12:07.203856 12495 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.738s	user 0.022s	sys 0.034s
I20260709 12:12:07.150872 17760 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:07.217916 12495 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.752s	user 0.022s	sys 0.034s
W20260709 12:12:07.209216 12530 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.725s	user 0.015s	sys 0.054s
I20260709 12:12:07.212199 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:07.212746 12599 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:07.150895 12569 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:07.219802 12522 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
W20260709 12:12:07.220908 12530 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.015s	sys 0.054s
I20260709 12:12:07.200299 17773 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:12:07.227970 12528 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.009s	sys 0.017s
W20260709 12:12:07.290617 12528 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.644s	user 0.009s	sys 0.029s
W20260709 12:12:07.291678 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 94573.8us
I20260709 12:12:07.294260 12571 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:07.295593 12571 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:07.298375 17819 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:12:07.298839 17819 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:07.299223 17819 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:07.299446 17781 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:07.300179 17781 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:07.293011 18142 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:07.302819 18142 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:07.303414 18142 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:07.292757 17775 consensus_queue.cc:237] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:07.306479 17957 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:07.298390 18009 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:07.307711 17957 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:07.307852 18009 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:07.309266 18009 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 11.8, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:07.227154 17760 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:12:07.326370 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2740) took 3313 ms. Trace:
I20260709 12:12:07.327858 12601 rpcz_store.cc:276] 0709 12:12:04.012796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:05.712776 (+1699980us) service_pool.cc:224] Handling call
0709 12:12:05.730944 (+ 18168us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:05.731230 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:12:06.302112 (+570882us) log.cc:844] Serialized 14 byte log entry
0709 12:12:07.325137 (+1023025us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:07.325228 (+    91us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:07.325301 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:07.326064 (+   763us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:07.326287 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":75486,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":83813,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":132,"thread_start_us":1466173,"threads_started":2}
I20260709 12:12:07.252903 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2513) took 1492 ms. Trace:
I20260709 12:12:07.333745 12675 rpcz_store.cc:276] 0709 12:12:05.760530 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.974025 (+1213495us) service_pool.cc:224] Handling call
0709 12:12:06.974397 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:07.191153 (+216756us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:07.252612 (+ 61459us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b54000c2bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:07.252838 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":356135680}
W20260709 12:12:07.259042 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:07.206816 17519 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:07.290944 17691 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:07.201776 18036 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.089s
I20260709 12:12:07.317543 17749 raft_consensus.cc:3055] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:12:07.322868 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:07.310292 18132 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:07.341887 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 191116us
I20260709 12:12:07.361963 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:07.301542 18031 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:07.352525 12531 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:07.374670 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:07.252928 17913 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:07.356468 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:07.357961 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2751) took 2527 ms. Trace:
I20260709 12:12:07.379554 12599 rpcz_store.cc:276] 0709 12:12:04.830420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.774645 (+1944225us) service_pool.cc:224] Handling call
0709 12:12:06.775190 (+   545us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:07.222790 (+447600us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:07.357595 (+134805us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b54003a0cd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:07.357868 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1005759744}
W20260709 12:12:07.213514 17899 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.953s	user 0.017s	sys 0.031s
I20260709 12:12:07.325482 17708 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:07.252379 18140 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:12:07.377875 12672 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.008s	sys 0.027s
I20260709 12:12:07.380806 12602 raft_consensus.cc:2368] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15.
W20260709 12:12:07.386253 12525 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.572s	user 0.008s	sys 0.008s
W20260709 12:12:07.394243 12525 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.009s	sys 0.008s
W20260709 12:12:07.394477 12672 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.008s	sys 0.027s
I20260709 12:12:07.380393 12646 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:07.227332 18112 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:07.313898 17781 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:07.372586 18118 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.026s	sys 0.026s
I20260709 12:12:07.366279 17749 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:07.403911 17749 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:07.399070 12602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 2762) took 1410 ms (client timeout 1852 ms). Trace:
W20260709 12:12:07.406600 12602 rpcz_store.cc:269] 0709 12:12:05.989044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.360818 (+371774us) service_pool.cc:224] Handling call
0709 12:12:07.398783 (+1037965us) spinlock_profiling.cc:243] Waited 983 ms on lock 0x7b5400115dd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:07.398985 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2261830528}
I20260709 12:12:07.405771 17749 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:07.414589 18118 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.027s	sys 0.026s
W20260709 12:12:07.374693 17989 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.029s	sys 0.014s
W20260709 12:12:07.438946 17989 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.840s	user 0.029s	sys 0.014s
I20260709 12:12:07.355494 18044 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:12:07.364466 18143 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:07.394862 17708 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:07.379022 18074 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:07.445978 18032 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:07.460139 12605 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 13: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:07.448094 18143 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:07.463347 18143 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:07.466426 18032 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:07.466784 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2847) took 2897 ms. Trace:
I20260709 12:12:07.467550 18032 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:07.467152 12678 rpcz_store.cc:276] 0709 12:12:04.569840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.340165 (+1770325us) service_pool.cc:224] Handling call
0709 12:12:06.340643 (+   478us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:06.382096 (+ 41453us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:06.382197 (+   101us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:06.390952 (+  8755us) log.cc:844] Serialized 52 byte log entry
0709 12:12:07.079520 (+688568us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:07.089986 (+ 10466us) log.cc:844] Serialized 14 byte log entry
0709 12:12:07.090251 (+   265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:07.090326 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:07.466463 (+376137us) raft_consensus.cc:1685] finished
0709 12:12:07.466490 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:07.466708 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13732,"prepare.queue_time_us":12347,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":49,"raft.queue_time_us":21529,"raft.run_cpu_time_us":184,"raft.run_wall_time_us":1778,"thread_start_us":611156,"threads_started":1,"wal-append.queue_time_us":896453}
I20260709 12:12:07.482987 18141 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:07.397243 18140 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:07.487614 18140 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:07.467145 18138 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.214s	user 0.001s	sys 0.000s
I20260709 12:12:07.389128 17913 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:07.496006 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:07.450481 18074 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:07.451894 18037 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
W20260709 12:12:07.512092 12526 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.669s	user 0.003s	sys 0.021s
W20260709 12:12:07.512558 12526 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.004s	sys 0.022s
W20260709 12:12:07.513300 12529 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.005s	sys 0.020s
W20260709 12:12:07.513736 12529 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.660s	user 0.005s	sys 0.020s
I20260709 12:12:07.505849 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:07.517376 12673 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.769s	user 0.002s	sys 0.051s
W20260709 12:12:07.518006 12673 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.769s	user 0.003s	sys 0.052s
I20260709 12:12:07.522862 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:07.519532 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2500) took 2627 ms. Trace:
I20260709 12:12:07.534857 12673 rpcz_store.cc:276] 0709 12:12:04.891952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.747507 (+1855555us) service_pool.cc:224] Handling call
0709 12:12:06.747873 (+   366us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.748120 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:12:06.748427 (+   307us) log.cc:844] Serialized 14 byte log entry
0709 12:12:07.518937 (+770510us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:07.519045 (+   108us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:07.519134 (+    89us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:07.519247 (+   113us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:07.519440 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1985,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":292,"thread_start_us":769791,"threads_started":1}
I20260709 12:12:07.500891 18074 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:07.538765 12531 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 14: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:07.540270 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:07.541710 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:07.545359 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:07.543645 13304 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.009s	sys 0.017s
W20260709 12:12:07.552703 13304 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.009s	sys 0.017s
W20260709 12:12:07.553464 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 701248us
W20260709 12:12:07.554468 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:07.557497 12643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.002s	sys 0.027s
W20260709 12:12:07.558167 12643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.002s	sys 0.027s
W20260709 12:12:07.559247 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 602030us
W20260709 12:12:07.560732 12677 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.604s	user 0.014s	sys 0.015s
W20260709 12:12:07.561460 12677 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.015s	sys 0.015s
I20260709 12:12:07.562768 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:07.551138 12531 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 10.
W20260709 12:12:07.572038 12671 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.007s	sys 0.022s
W20260709 12:12:07.575595 12671 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.008s	sys 0.022s
I20260709 12:12:07.581128 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2846) took 3029 ms. Trace:
I20260709 12:12:07.511096 18146 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:07.590678 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:07.615875 12522 raft_consensus.cc:2410] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:12:07.590660 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:07.583902 12671 rpcz_store.cc:276] 0709 12:12:04.560592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.096223 (+1535631us) service_pool.cc:224] Handling call
0709 12:12:06.114846 (+ 18623us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.115101 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:12:06.164672 (+ 49571us) log.cc:844] Serialized 14 byte log entry
0709 12:12:07.096912 (+932240us) spinlock_profiling.cc:243] Waited 38.6 ms on lock 0x7b4800060858. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:12:07.578252 (+481340us) log.cc:844] Serialized 14 byte log entry
0709 12:12:07.580568 (+  2316us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:07.580658 (+    90us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:07.580740 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:07.580844 (+   104us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:07.581044 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":38225,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":12656,"raft.queue_time_us":67376,"raft.run_cpu_time_us":2998,"raft.run_wall_time_us":72463,"spinlock_wait_cycles":90141696,"thread_start_us":1330675,"threads_started":2,"wal-append.queue_time_us":996614}
I20260709 12:12:07.618875 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:07.619217 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:07.620271 12643 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:07.631604 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:07.613538 18034 consensus_meta.cc:287] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.092s	user 0.007s	sys 0.007s
I20260709 12:12:07.555297 18068 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:07.558665 18096 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:07.628031 17582 consensus_meta.cc:287] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.788s	user 0.011s	sys 0.002s
I20260709 12:12:07.635428 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:07.630774 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:07.556054 17553 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:07.653211 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:07.634851 18088 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.602s	user 0.013s	sys 0.001s
W20260709 12:12:07.650930 12524 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.752s	user 0.011s	sys 0.015s
I20260709 12:12:07.641952 17899 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:07.655090 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:07.664865 12572 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:07.665676 12522 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:07.651270 12494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.806s	user 0.016s	sys 0.011s
W20260709 12:12:07.667018 12494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.822s	user 0.016s	sys 0.011s
W20260709 12:12:07.659734 12524 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.011s	sys 0.016s
W20260709 12:12:07.668752 12527 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.798s	user 0.028s	sys 0.063s
W20260709 12:12:07.670372 12527 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.028s	sys 0.063s
I20260709 12:12:07.570851 18143 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:07.674168 12680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.041s	sys 0.035s
W20260709 12:12:07.667855 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 837702us
I20260709 12:12:07.629005 18053 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:12:07.654896 17691 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:07.652520 17869 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.006s	sys 0.027s
I20260709 12:12:07.631709 17819 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:07.675263 12680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.041s	sys 0.035s
W20260709 12:12:07.709298 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:07.710170 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2517) took 1594 ms. Trace:
I20260709 12:12:07.710716 12680 rpcz_store.cc:276] 0709 12:12:06.115998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.973844 (+857846us) service_pool.cc:224] Handling call
0709 12:12:06.974260 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.974498 (+   238us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:12:07.001383 (+ 26885us) log.cc:844] Serialized 14 byte log entry
0709 12:12:07.709286 (+707903us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:07.709397 (+   111us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:07.709700 (+   303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:07.709850 (+   150us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:07.710054 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":45115,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":61,"thread_start_us":707370,"threads_started":1}
I20260709 12:12:07.679406 12494 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:07.632157 18146 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:07.658090 17582 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:07.659441 18150 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:07.691113 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2546) took 1641 ms. Trace:
W20260709 12:12:07.703521 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 81683.6us
I20260709 12:12:07.650024 18096 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:07.770691 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:07.771912 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:07.772254 12643 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:07.718967 12644 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.980s	user 0.011s	sys 0.033s
W20260709 12:12:07.755353 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 76045.5us
W20260709 12:12:07.774194 12644 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.035s	user 0.012s	sys 0.033s
I20260709 12:12:07.776104 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:07.645061 18034 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:07.674297 18054 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:07.767120 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:07.692348 17869 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.007s	sys 0.027s
I20260709 12:12:07.783396 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2493) took 3223 ms. Trace:
I20260709 12:12:07.784607 12531 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 13: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:07.783907 12677 rpcz_store.cc:276] 0709 12:12:04.559984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.097798 (+1537814us) service_pool.cc:224] Handling call
0709 12:12:06.099958 (+  2160us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.274221 (+174263us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:12:06.339656 (+ 65435us) log.cc:844] Serialized 14 byte log entry
0709 12:12:06.956136 (+616480us) log.cc:844] Serialized 14 byte log entry
0709 12:12:07.094373 (+138237us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b1800322a20. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:12:07.561890 (+467517us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:07.561960 (+    70us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:07.562008 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:07.783128 (+221120us) spinlock_profiling.cc:243] Waited 174 ms on lock 0x7b5400013bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:07.783187 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:07.783334 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22304,"prepare.queue_time_us":66329,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":107,"raft.queue_time_us":88745,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":56827,"spinlock_wait_cycles":423172608,"thread_start_us":1221199,"threads_started":2,"wal-append.queue_time_us":877197}
I20260709 12:12:07.711665 17899 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:07.754561 17691 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:07.756603 18112 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:07.791747 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:07.794334 12523 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.628s	user 0.012s	sys 0.024s
I20260709 12:12:07.662148 18088 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:07.663640 17553 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:07.790563 12605 rpcz_store.cc:276] 0709 12:12:06.049929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.526369 (+476440us) service_pool.cc:224] Handling call
0709 12:12:07.690822 (+1164453us) spinlock_profiling.cc:243] Waited 927 ms on lock 0x7b5400232154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:07.691040 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2132044544}
W20260709 12:12:07.801769 18002 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.665s	user 0.010s	sys 0.013s
I20260709 12:12:07.758670 17582 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:07.815773 12523 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.013s	sys 0.024s
W20260709 12:12:07.833742 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 481512us
I20260709 12:12:07.836999 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:07.842976 12531 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 15: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:07.845933 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:07.848830 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:07.852113 12680 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:12:07.854956 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:07.754688 18149 consensus_queue.cc:1059] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.139s
I20260709 12:12:07.846729 12677 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:07.859630 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:07.864032 12678 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:07.860684 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:07.804251 17691 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:07.787941 18132 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.075s
I20260709 12:12:07.891955 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:07.892558 12677 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:07.888003 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:07.864284 12680 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 10. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:12:07.894239 12673 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:12:07.841503 18002 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.011s	sys 0.014s
I20260709 12:12:07.890789 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:07.836628 18112 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:07.783893 18150 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:12:07.921691 12525 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.013s	sys 0.002s
W20260709 12:12:07.922370 12525 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.527s	user 0.014s	sys 0.002s
I20260709 12:12:07.864398 18053 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:07.825130 17869 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:07.924247 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:07.915659 12678 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:12:07.836700 17519 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:07.936667 18149 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260709 12:12:07.940644 17519 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:07.942044 18155 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:12:07.943920 18002 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:07.944891 18002 raft_consensus.cc:3055] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:07.945349 18002 raft_consensus.cc:740] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:07.946409 18002 consensus_queue.cc:260] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:07.948110 18002 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:07.918211 18147 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 12:12:07.935451 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2506) took 1830 ms. Trace:
I20260709 12:12:07.893276 17913 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:07.833137 18068 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 12:12:07.969901 12531 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:07.894456 18032 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:12:07.924993 18090 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.008s	sys 0.007s
I20260709 12:12:07.988848 12531 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:07.918097 18112 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:07.962500 12525 rpcz_store.cc:276] 0709 12:12:06.105035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.738859 (+633824us) service_pool.cc:224] Handling call
0709 12:12:06.739174 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.739424 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:12:07.394885 (+655461us) log.cc:844] Serialized 14 byte log entry
0709 12:12:07.934886 (+540001us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:07.934964 (+    78us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:07.935071 (+   107us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:07.935185 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:07.935366 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17315,"prepare.queue_time_us":1019632,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"thread_start_us":1107910,"threads_started":2}
I20260709 12:12:07.965893 18162 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:12:07.986917 18165 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:12:07.993016 18090 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.009s	sys 0.008s
W20260709 12:12:08.018021 12528 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.675s	user 0.013s	sys 0.036s
I20260709 12:12:08.008563 18068 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Stepping down as leader of term 13
W20260709 12:12:08.025789 12569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.008s	sys 0.034s
W20260709 12:12:08.030295 12529 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s	user 0.004s	sys 0.041s
W20260709 12:12:08.012380 17775 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.011s	sys 0.006s
I20260709 12:12:07.963250 17869 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:08.013095 18044 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.000s	sys 0.017s
I20260709 12:12:08.037639 12531 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
W20260709 12:12:08.033087 17773 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.002s	sys 0.059s
I20260709 12:12:07.956027 17582 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:08.041548 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2509) took 1251 ms. Trace:
W20260709 12:12:08.042078 12569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.008s	sys 0.034s
W20260709 12:12:08.112697 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 743547us
W20260709 12:12:08.035717 12528 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.014s	sys 0.036s
I20260709 12:12:08.112537 12524 rpcz_store.cc:276] 0709 12:12:06.789864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.807225 (+ 17361us) service_pool.cc:224] Handling call
0709 12:12:06.807607 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:06.814790 (+  7183us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:06.814820 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:06.845800 (+ 30980us) log.cc:844] Serialized 26 byte log entry
0709 12:12:07.668888 (+823088us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:07.668932 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:07.669012 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:08.041222 (+372210us) raft_consensus.cc:1685] finished
0709 12:12:08.041258 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:08.041457 (+   199us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6818,"raft.queue_time_us":53528,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":86,"thread_start_us":769968,"threads_started":1,"wal-append.queue_time_us":1123319}
W20260709 12:12:08.046804 13302 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.822s (SENT) [suppressed 10 similar messages]
W20260709 12:12:08.115043 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 68237.5us
I20260709 12:12:08.115521 12524 raft_consensus.cc:3055] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:07.964548 17749 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:12:08.031805 17989 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.005s	sys 0.015s
I20260709 12:12:08.025395 17981 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:08.130630 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2510) took 1340 ms. Trace:
I20260709 12:12:08.131089 12526 rpcz_store.cc:276] 0709 12:12:06.790169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.802060 (+ 11891us) service_pool.cc:224] Handling call
0709 12:12:06.837526 (+ 35466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:07.513057 (+675531us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:07.513108 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:07.639370 (+126262us) log.cc:844] Serialized 26 byte log entry
0709 12:12:08.015982 (+376612us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:08.016047 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:08.016136 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:08.130380 (+114244us) raft_consensus.cc:1685] finished
0709 12:12:08.130401 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:08.130561 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":88495,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":13528,"thread_start_us":1015631,"threads_started":2,"wal-append.queue_time_us":475659}
I20260709 12:12:08.133793 18174 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:08.134197 18182 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:08.134865 18031 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:08.135059 18182 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:08.135666 18182 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:08.137130 12531 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 9. (term mismatch)
I20260709 12:12:08.137640 12531 pending_rounds.cc:85] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 8
I20260709 12:12:08.137914 12531 pending_rounds.cc:107] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 14.9
I20260709 12:12:08.138173 12531 raft_consensus.cc:2889] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:12:08.139643 18156 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.241s	user 0.000s	sys 0.001s
I20260709 12:12:08.141335 17582 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:08.080852 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 304818us
W20260709 12:12:08.096881 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 315139us
I20260709 12:12:08.029012 18096 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:08.091126 17775 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.011s	sys 0.006s
I20260709 12:12:08.116071 12524 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:08.114413 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2501) took 2010 ms. Trace:
I20260709 12:12:08.120376 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 605) took 2009 ms. Trace:
I20260709 12:12:07.974593 17691 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:08.122869 17989 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s	user 0.005s	sys 0.016s
W20260709 12:12:08.141834 18154 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
I20260709 12:12:08.161788 12527 rpcz_store.cc:276] 0709 12:12:06.111293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.773106 (+661813us) service_pool.cc:224] Handling call
0709 12:12:06.773389 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.773601 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:12:07.673823 (+900222us) log.cc:844] Serialized 14 byte log entry
0709 12:12:08.120006 (+446183us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:08.120065 (+    59us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:12:08.120126 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:08.120192 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:08.120316 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":114480,"prepare.queue_time_us":1019007,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":2584,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":97,"thread_start_us":1245068,"threads_started":2}
I20260709 12:12:08.119131 18140 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:08.141335 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2854) took 3314 ms. Trace:
I20260709 12:12:08.161679 12528 rpcz_store.cc:276] 0709 12:12:06.103599 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.528136 (+424537us) service_pool.cc:224] Handling call
0709 12:12:06.562895 (+ 34759us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.563139 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:12:07.342595 (+779456us) log.cc:844] Serialized 14 byte log entry
0709 12:12:08.114018 (+771423us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:08.114075 (+    57us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:08.114157 (+    82us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:08.114229 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:08.114351 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":107979,"prepare.queue_time_us":1095972,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":5886,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":27807,"thread_start_us":1449589,"threads_started":2}
I20260709 12:12:08.164927 12672 rpcz_store.cc:276] 0709 12:12:04.826475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.496223 (+1669748us) service_pool.cc:224] Handling call
0709 12:12:06.517733 (+ 21510us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:06.776990 (+259257us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:06.777039 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:06.777497 (+   458us) log.cc:844] Serialized 26 byte log entry
0709 12:12:07.395438 (+617941us) raft_consensus.cc:1645] Marking committed up to 4
0709 12:12:07.395486 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:07.395567 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:08.141036 (+745469us) raft_consensus.cc:1685] finished
0709 12:12:08.141063 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:08.141256 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":617479,"threads_started":1,"wal-append.queue_time_us":1013653}
I20260709 12:12:08.041531 18068 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:08.111027 17773 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.734s	user 0.003s	sys 0.059s
W20260709 12:12:08.110800 18044 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.000s	sys 0.018s
W20260709 12:12:08.154011 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:08.080195 17819 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:08.121336 18034 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:08.093472 12522 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
W20260709 12:12:08.193933 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 402265us
I20260709 12:12:08.194271 12598 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:12:08.195036 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:08.161579 18146 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:08.197415 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:08.199832 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:08.201116 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:08.205673 12602 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 20: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:08.146082 17749 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:08.167987 18088 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:08.213786 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:08.189116 17691 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:08.169992 17773 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:08.226171 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:08.213800 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:08.230721 12599 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:08.193215 18187 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:08.226847 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:08.232573 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:08.207633 18031 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:08.233515 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:08.212518 18146 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:08.169546 17913 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260709 12:12:08.193614 18132 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:12:08.243837 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:08.277237 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:08.277945 12678 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
I20260709 12:12:08.278528 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:08.279090 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:08.279750 12678 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:12:08.280678 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:08.280843 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:08.281595 12676 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 11: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:08.237517 12602 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 15: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:08.209467 18034 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:08.240845 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:08.215126 18081 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:08.216861 18044 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:08.257949 12673 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:08.260087 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:08.200126 17989 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:12:08.179240 18167 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 11.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:12:08.294940 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:08.300972 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:08.298676 12597 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:08.243486 18031 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:08.224982 18090 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:08.298754 12599 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:08.304349 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:08.304525 12677 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:08.307989 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:08.264642 18150 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:08.309283 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:08.309684 13330 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
W20260709 12:12:08.310331 12643 connection.cc:256] RPC call timeout handler was delayed by -1.101134342000023s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18510000000000001s for a response
W20260709 12:12:08.314381 18169 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.233s	user 0.000s	sys 0.004s
I20260709 12:12:08.319696 18143 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:08.320111 18075 consensus_meta.cc:287] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.108s	user 0.010s	sys 0.005s
I20260709 12:12:08.323819 13336 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:08.324700 13338 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 12. (index mismatch)
I20260709 12:12:08.300460 13334 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:08.327656 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:08.321616 12603 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:12:08.326560 12678 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:12:08.314858 18088 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:08.308665 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:08.231922 18153 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.075s
I20260709 12:12:08.338164 12571 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:08.334462 13335 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
W20260709 12:12:08.339975 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 176491us
I20260709 12:12:08.236745 18054 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.091s
W20260709 12:12:08.317598 18168 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.345s	user 0.000s	sys 0.002s
I20260709 12:12:08.344265 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:08.348547 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:08.228140 18162 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.094s
I20260709 12:12:08.353281 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:08.346328 18112 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:08.355942 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:08.330408 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 216941us
I20260709 12:12:08.336143 18031 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:08.344740 18074 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.774s	user 0.011s	sys 0.003s
I20260709 12:12:08.239111 18164 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.116s
I20260709 12:12:08.329699 18044 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:08.391099 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:08.246061 18187 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:08.339867 18032 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:08.373980 12599 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
I20260709 12:12:08.371699 17989 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:08.381815 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:08.305404 18081 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:08.387750 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:08.321318 18034 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:08.389835 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:08.370232 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 206773us
I20260709 12:12:08.432468 12603 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:12:08.391134 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 47008us
I20260709 12:12:08.433835 12643 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:08.373798 17913 raft_consensus.cc:3055] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:08.282747 18146 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:08.407163 18189 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.195s	user 0.000s	sys 0.001s
W20260709 12:12:08.405220 18183 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 12:12:08.450865 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:08.391155 17519 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.006s
I20260709 12:12:08.453979 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:08.400882 18186 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:12:08.404543 18178 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 12:12:08.399809 18179 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 12:12:08.406512 18171 log.cc:927] Time spent T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
W20260709 12:12:08.363857 18142 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.060s	user 0.015s	sys 0.002s
I20260709 12:12:08.428443 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:08.472711 12597 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 15: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:08.430451 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:08.431054 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 44232.7us
I20260709 12:12:08.473336 12606 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:08.440680 17869 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:08.405241 18036 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:08.388582 18143 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:08.367918 18075 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:08.394817 18184 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
I20260709 12:12:08.458631 12601 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 14: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:08.372216 18088 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:08.447129 18170 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.270s	user 0.001s	sys 0.001s
I20260709 12:12:08.439121 18081 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:08.437180 17989 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:08.387921 18150 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.069s
I20260709 12:12:08.342476 18191 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.069s
I20260709 12:12:08.438624 17913 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:08.522734 17913 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:08.524307 17913 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:08.361092 17773 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:08.439972 17708 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:08.484496 12606 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:08.487437 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2470) took 1054 ms. Trace:
I20260709 12:12:08.487665 12603 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:12:08.415841 18074 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:08.529897 12525 rpcz_store.cc:276] 0709 12:12:07.432733 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:08.027844 (+595111us) service_pool.cc:224] Handling call
0709 12:12:08.028505 (+   661us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:08.038367 (+  9862us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:08.038411 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:08.039351 (+   940us) log.cc:844] Serialized 52 byte log entry
0709 12:12:08.190797 (+151446us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b1800405a00. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:12:08.356326 (+165529us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:08.371608 (+ 15282us) log.cc:844] Serialized 14 byte log entry
0709 12:12:08.486955 (+115347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:08.487038 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:08.487085 (+    47us) raft_consensus.cc:1685] finished
0709 12:12:08.487174 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:08.487354 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":41979,"prepare.queue_time_us":22775,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":1866,"raft.queue_time_us":133682,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":2818,"spinlock_wait_cycles":24226560,"thread_start_us":238048,"threads_started":2,"wal-append.queue_time_us":352844}
I20260709 12:12:08.532289 18074 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:08.489451 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:08.467788 18090 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:12:08.499444 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 117717us
I20260709 12:12:08.534070 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:08.600455 17819 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:08.455860 18153 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:12:08.478670 18036 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:08.619874 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 186537us
W20260709 12:12:08.627390 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 169012us
I20260709 12:12:08.633661 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:08.472272 18193 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.152s
I20260709 12:12:08.481398 18142 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:08.498456 18162 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.133s
I20260709 12:12:08.646162 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:08.634752 17819 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:08.663064 17819 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:08.675305 12496 leader_election.cc:336] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.748s (SENT)
I20260709 12:12:08.675941 12496 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:08.647861 17957 consensus_meta.cc:287] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.340s	user 0.014s	sys 0.005s
I20260709 12:12:08.645699 18192 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:08.680722 18181 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:08.681444 18181 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:08.681859 18181 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:08.682313 18191 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:12:08.684630 12494 leader_election.cc:400] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15.
I20260709 12:12:08.684970 12494 leader_election.cc:403] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
W20260709 12:12:08.687060 18205 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:12:08.689463 18187 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:08.689640 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2504) took 2585 ms. Trace:
I20260709 12:12:08.691174 18203 consensus_queue.cc:1059] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:12:08.691627 18182 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:12:08.692116 18203 consensus_queue.cc:1059] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:12:08.693470 18203 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260709 12:12:08.695103 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:08.699069 18201 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:12:08.700912 18198 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.310s	user 0.000s	sys 0.001s
I20260709 12:12:08.703027 18195 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15.
I20260709 12:12:08.704077 18195 raft_consensus.cc:3055] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:12:08.704551 18195 raft_consensus.cc:740] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:08.705231 18187 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:08.705847 18187 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:08.658305 17989 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:08.707310 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2512) took 1646 ms. Trace:
W20260709 12:12:08.704622 13304 leader_election.cc:336] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.999s (SENT)
W20260709 12:12:08.710577 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:08.685715 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:08.711517 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:08.606926 18199 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.166s
I20260709 12:12:08.707799 12523 rpcz_store.cc:276] 0709 12:12:07.060536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:07.068174 (+  7638us) service_pool.cc:224] Handling call
0709 12:12:07.083240 (+ 15066us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:07.104209 (+ 20969us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:07.104258 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:07.126289 (+ 22031us) log.cc:844] Serialized 52 byte log entry
0709 12:12:07.841967 (+715678us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:07.897650 (+ 55683us) log.cc:844] Serialized 14 byte log entry
0709 12:12:08.168245 (+270595us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:08.168612 (+   367us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:08.707072 (+538460us) raft_consensus.cc:1685] finished
0709 12:12:08.707092 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:08.707247 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":74246,"prepare.queue_time_us":55729,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":60,"raft.queue_time_us":75354,"raft.run_cpu_time_us":356,"raft.run_wall_time_us":21535,"thread_start_us":933898,"threads_started":2,"wal-append.queue_time_us":887383}
I20260709 12:12:08.612639 18194 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.151s
I20260709 12:12:08.659364 18164 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:08.690003 12529 rpcz_store.cc:276] 0709 12:12:06.104429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.666604 (+562175us) service_pool.cc:224] Handling call
0709 12:12:06.723793 (+ 57189us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:06.753975 (+ 30182us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:12:07.517573 (+763598us) log.cc:844] Serialized 14 byte log entry
0709 12:12:08.042451 (+524878us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:08.042505 (+    54us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:08.042553 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:08.687617 (+645064us) spinlock_profiling.cc:243] Waited 2.86 ms on lock 0x7b1800212a40. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419
0709 12:12:08.689390 (+  1773us) spinlock_profiling.cc:243] Waited 29.9 ms on lock 0x7b540038e5d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:08.689438 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:08.689578 (+   140us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":109500,"prepare.queue_time_us":1024704,"prepare.run_cpu_time_us":93,"prepare.run_wall_time_us":11870,"raft.queue_time_us":103,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":49194,"spinlock_wait_cycles":75309184,"thread_start_us":1185188,"threads_started":2,"wal-append.queue_time_us":690298}
I20260709 12:12:08.708431 17989 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:08.637409 18167 consensus_queue.cc:1059] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.074s
I20260709 12:12:08.637959 18090 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.075s
W20260709 12:12:08.719339 18202 consensus_queue.cc:1175] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" responder_term: 14 status { last_received { term: 13 index: 8 } last_committed_idx: 7 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:12:08.719158 18164 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:08.733587 18164 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:08.734490 18212 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:08.734972 18212 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:08.735366 18212 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:08.736245 18181 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:08.737591 18194 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:08.738207 18194 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:08.739488 18181 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:08.740511 18194 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:08.713619 18074 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:08.741501 18074 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:08.744369 18074 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:08.744971 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:08.745770 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:08.745795 17773 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:08.746599 17773 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:08.747259 18214 raft_consensus.cc:3055] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:12:08.747713 18214 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:08.748999 17773 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:08.748661 18214 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:08.750288 18214 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:08.759661 18142 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:08.760171 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:08.761157 18075 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:08.762225 18209 raft_consensus.cc:3055] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:08.762873 18209 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:08.765638 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:08.766330 12529 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:08.768155 12525 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:12:08.735203 18164 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:08.764091 18209 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:08.777419 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:08.782449 12525 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:08.778661 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:08.711778 18203 consensus_queue.cc:1243] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 14.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:12:08.784191 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:08.784938 12523 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15.
I20260709 12:12:08.708576 18187 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:08.769860 18173 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.585s	user 0.002s	sys 0.000s
W20260709 12:12:08.784500 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 58939.9us
I20260709 12:12:08.791501 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:08.793298 13305 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:08.795420 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:08.796361 12531 raft_consensus.cc:2410] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:12:08.774883 18195 consensus_queue.cc:260] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:08.802253 18209 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:08.806461 18187 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:08.806980 18187 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:08.807384 18187 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:08.807731 18031 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:08.727545 17869 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:08.809991 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:08.814471 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:08.817972 12525 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:12:08.820227 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 286255us
I20260709 12:12:08.823096 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:08.825243 12571 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:08.825562 12526 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:12:08.829973 12529 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:08.785951 18199 consensus_queue.cc:1243] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:12:08.786715 18146 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:08.810591 17760 consensus_meta.cc:287] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.439s	user 0.014s	sys 0.002s
I20260709 12:12:08.858501 17760 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:08.860486 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:08.861539 13330 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:08.811399 18034 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:08.839916 18068 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:08.875965 18068 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:08.824836 18075 consensus_queue.cc:1175] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e56d487365394ac5ac5f1002926a5645" responder_term: 10 status { last_received { term: 10 index: 8 } last_committed_idx: 6 last_received_current_leader { term: 10 index: 8 } }
I20260709 12:12:08.837352 18216 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:08.880657 18216 raft_consensus.cc:697] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:08.843683 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:12:08.844287 12496 leader_election.cc:336] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.997s (SENT)
I20260709 12:12:08.885672 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:08.885524 12496 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:08.840687 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:08.816419 18150 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:12:08.816980 18002 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.863s	user 0.017s	sys 0.000s
I20260709 12:12:08.850495 18167 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:08.850703 18217 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:08.817058 17869 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:08.914211 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:08.925865 12531 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 14: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:08.870877 18199 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:12:08.838029 18190 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:08.849279 18112 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:12:08.888137 17899 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.060s	user 0.009s	sys 0.006s
I20260709 12:12:08.816504 18203 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.104s
I20260709 12:12:08.928864 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:08.927901 18217 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:08.936946 18031 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:08.938244 18081 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:08.938877 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2497) took 2836 ms. Trace:
I20260709 12:12:08.939157 18217 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:08.939561 12530 rpcz_store.cc:276] 0709 12:12:06.102376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.483490 (+381114us) service_pool.cc:224] Handling call
0709 12:12:06.484018 (+   528us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:07.229432 (+745414us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:07.229482 (+    50us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:07.291549 (+ 62067us) log.cc:844] Serialized 52 byte log entry
0709 12:12:07.833940 (+542391us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:07.834007 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:07.846809 (+ 12802us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:08.769137 (+922328us) spinlock_profiling.cc:243] Waited 5.45 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:08.938276 (+169139us) raft_consensus.cc:1685] finished
0709 12:12:08.938326 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:08.938714 (+   388us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":276508,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"spinlock_wait_cycles":12530944,"thread_start_us":1199662,"threads_started":2,"wal-append.queue_time_us":731578}
I20260709 12:12:08.941340 17981 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:08.942266 17981 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:08.915045 18150 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:08.944850 18150 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:08.950062 18216 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:08.953946 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:08.957064 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:08.930799 12494 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:08.959460 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:08.960376 13332 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:12:08.928933 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 84688.9us
W20260709 12:12:08.910224 18009 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.044s	user 0.008s	sys 0.045s
I20260709 12:12:08.846040 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:08.941136 18081 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:08.861881 17760 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:08.964658 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:08.966161 13336 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:08.971644 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:08.972316 12531 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:08.928355 17869 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:08.973315 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:08.980286 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 137874us
I20260709 12:12:08.981333 12571 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:08.982375 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:08.984045 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:08.984879 12531 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 15: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:08.915263 18132 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:08.952814 12524 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:08.989040 12524 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:08.928247 18191 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:08.987500 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:08.990571 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:08.993125 13332 raft_consensus.cc:2410] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 11 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
W20260709 12:12:08.966346 18009 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.119s	user 0.009s	sys 0.045s
I20260709 12:12:08.916741 18074 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:12:08.976194 12494 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:08.994417 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:09.000837 12531 raft_consensus.cc:2410] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:12:08.996336 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:08.965745 17899 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:12:08.998637 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:08.970160 18203 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:08.990660 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:09.007558 12523 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 14: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:08.998903 12524 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:12:09.008719 12524 pending_rounds.cc:85] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 7
I20260709 12:12:09.009438 12524 pending_rounds.cc:107] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:12:09.009249 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:08.969844 18088 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:09.011430 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:09.016608 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:08.991948 18215 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.149s	user 0.001s	sys 0.001s
I20260709 12:12:09.014786 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:09.018476 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:09.014889 12524 raft_consensus.cc:2889] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:09.015949 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:08.924796 18167 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:08.991704 18112 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:12:09.012933 17582 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.844s	user 0.017s	sys 0.004s
I20260709 12:12:09.030880 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:09.031159 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:09.032148 12675 raft_consensus.cc:2410] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:12:08.954298 18190 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:09.032435 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:09.023769 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2481) took 1356 ms. Trace:
I20260709 12:12:09.033231 12601 raft_consensus.cc:2393] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 14: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:12:09.012714 17691 consensus_meta.cc:287] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.778s	user 0.011s	sys 0.008s
I20260709 12:12:09.035363 12600 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:09.036664 18167 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:09.025996 12599 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 11: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:09.037667 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:09.018950 18221 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:09.038419 12680 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:09.028905 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:09.039863 12604 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:09.018875 18158 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:09.014565 18174 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:09.058499 18174 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:09.061456 18174 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:09.030185 12597 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:12:09.023051 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:09.063947 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:09.064945 12671 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:12:09.065045 12678 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:09.022315 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:09.033715 12602 raft_consensus.cc:2410] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 14 because replica has last-logged OpId of term: 13 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 12:12:09.011829 18203 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:09.066745 12679 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:09.036689 12672 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:12:09.044309 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:09.068226 13339 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 14: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:09.044656 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:09.069702 13330 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:09.050803 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:09.071113 12677 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:09.043921 12675 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:12:09.046074 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:09.047034 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:09.029348 13332 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 10 index: 11. Preceding OpId from leader: term: 12 index: 12. (index mismatch)
I20260709 12:12:09.049975 18112 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:09.053308 12531 raft_consensus.cc:2410] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:12:09.038599 18221 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:09.054793 18190 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:09.066855 18203 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:09.070339 18044 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:09.081744 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:09.072898 12676 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 14: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:09.047586 18158 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:09.033643 12524 rpcz_store.cc:276] 0709 12:12:07.667847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:08.114904 (+447057us) service_pool.cc:224] Handling call
0709 12:12:08.115367 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:08.952330 (+836963us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b54001f61a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f148899e06f 00007f1488a16f60 00007f1488a187de 00007f1488a22b49 00007f1488a23416 00007f1488a1fd8f 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62
0709 12:12:09.023428 (+ 71098us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:09.023707 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":328682240}
I20260709 12:12:09.105310 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:09.106045 13334 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:09.108191 13330 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:09.084781 12525 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:12:09.085731 12679 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:12:09.074331 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:09.074409 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:09.074887 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:09.077001 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:09.078640 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:09.110141 12525 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:09.110636 12603 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 14: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:09.080478 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:09.061309 18187 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:09.071648 18180 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.743s	user 0.000s	sys 0.001s
I20260709 12:12:09.103965 12605 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 14: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:09.110667 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:09.116727 12597 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 20: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:09.111515 12599 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
W20260709 12:12:09.079545 17913 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.555s	user 0.014s	sys 0.000s
I20260709 12:12:09.114504 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:09.121030 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:09.122591 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:08.999655 18191 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:09.125771 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:09.127351 12599 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:09.128826 18187 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:09.134945 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:09.118474 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:09.136229 12597 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 11: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:09.138414 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:09.143183 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:09.079310 18221 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:09.148429 18150 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:09.149672 12527 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:09.155141 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:09.155974 12606 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:09.166481 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:09.167194 12528 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:09.172469 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:09.173341 12603 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:09.174960 12530 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:09.175643 12530 raft_consensus.cc:606] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:09.175984 12530 raft_consensus.cc:614] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:09.180610 17553 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:09.183674 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:09.187482 18224 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:12:09.188995 18195 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
W20260709 12:12:09.191339 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:09.194772 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2758) took 3208 ms. Trace:
I20260709 12:12:09.195252 12598 rpcz_store.cc:276] 0709 12:12:05.986648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:06.900195 (+913547us) service_pool.cc:224] Handling call
0709 12:12:06.932949 (+ 32754us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:08.194761 (+1261812us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:09.194391 (+999630us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b540022ecd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:09.194645 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2900073344}
I20260709 12:12:09.208626 18036 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:09.212208 17819 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:09.215343 12605 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:09.223198 17749 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:09.229043 18191 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:09.230522 12599 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:09.230011 18158 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:09.079969 18212 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:09.232990 18212 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:09.144017 13338 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:09.238569 12528 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
W20260709 12:12:09.240175 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:09.241006 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:09.253890 12598 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:09.240893 13305 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:09.256660 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:09.267478 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:09.268805 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:09.269114 13332 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 14: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:09.273078 12605 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:12:09.291814 13334 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:12:09.073158 17981 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:09.270515 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:09.303695 12597 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 15: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:09.306089 12569 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:09.307273 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:09.308715 12601 raft_consensus.cc:2410] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 17 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:12:09.311175 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:09.312536 12569 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:09.317569 12527 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:12:09.076541 18112 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:09.325528 12569 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:09.332713 12599 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: []
W20260709 12:12:09.309291 17989 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.543s	user 0.011s	sys 0.001s
I20260709 12:12:09.356050 12606 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
W20260709 12:12:09.344259 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:09.304772 17553 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:09.357372 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:09.365993 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:09.367064 12606 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:12:09.368438 12571 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:09.326525 18187 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:09.359611 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:09.328526 18075 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:09.331211 17819 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:12:09.335470 18172 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:09.387602 12525 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "502d264440884ecda3d4641918b20d0c"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:09.390619 12525 raft_consensus.cc:606] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:09.391018 12525 raft_consensus.cc:614] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:09.395443 12604 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:12:09.360780 18174 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:09.372248 17691 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:09.379390 18075 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:09.410562 12602 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:09.416033 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:09.495411 12523 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:12:09.375882 18175 raft_consensus.cc:2764] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:12:09.420428 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:09.366235 18176 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
W20260709 12:12:09.489769 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 178616us
W20260709 12:12:09.444648 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:09.407080 18034 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:09.497316 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:09.509197 12606 raft_consensus.cc:2368] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18.
I20260709 12:12:09.421037 12572 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:09.510931 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 92788.5us
I20260709 12:12:09.511937 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:09.514885 12569 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:09.516642 12569 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:09.519155 12572 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:09.401232 17957 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:09.498972 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:09.523540 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:09.526912 12494 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:09.381121 18199 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.097s
I20260709 12:12:09.422869 17691 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:09.398701 18009 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.104s
I20260709 12:12:09.533866 17989 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:09.542614 17989 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:09.543072 17989 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:09.540694 17582 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:09.535756 17519 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:09.535480 18146 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:12:09.544615 17869 consensus_meta.cc:287] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.550s	user 0.010s	sys 0.002s
I20260709 12:12:09.543637 17691 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:09.558730 12572 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:09.559814 18009 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:09.560760 12598 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "816b6f34d124434882148e507cf5b612"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:32832
I20260709 12:12:09.584347 12598 raft_consensus.cc:606] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:09.560812 18158 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:09.546799 17819 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:09.561753 18054 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:09.571589 18203 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:09.534799 18034 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:09.573640 18162 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:09.574106 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:12:09.574190 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 928065us
I20260709 12:12:09.603528 13330 raft_consensus.cc:2368] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18.
I20260709 12:12:09.603801 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:09.575316 18193 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:09.575489 18192 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:09.576046 18153 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:09.576581 18032 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:09.577349 18090 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:09.579528 18075 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:09.562567 18150 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:09.534618 18187 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:12:09.582930 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:09.590811 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:09.592164 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:09.582271 18210 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:12:09.599018 12604 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:09.563283 18146 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:09.639197 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:09.640228 12601 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:09.586884 18163 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:09.613510 18090 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:09.613463 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:09.592967 18194 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:09.592837 17553 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:09.673429 17553 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:09.620179 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:09.601524 18158 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:09.602968 18162 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:09.645015 17899 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:09.604286 17819 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:09.636711 13338 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:09.637208 12606 raft_consensus.cc:2376] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18. Re-sending same reply.
I20260709 12:12:09.587239 17869 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:09.610422 18031 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:09.696161 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:09.697777 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:09.698627 12680 raft_consensus.cc:2376] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12. Re-sending same reply.
I20260709 12:12:09.611769 18192 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:09.700984 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:09.703874 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:09.705032 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:09.620272 18032 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:09.709064 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:09.657718 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:09.633394 18200 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:09.626680 18193 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:09.664289 17760 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:09.664767 18081 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:09.666543 18213 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:12:09.661139 18088 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:09.613304 18174 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:12:09.722810 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:09.676167 13330 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:12:09.644737 18146 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:09.725396 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:09.726300 12529 raft_consensus.cc:2368] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 10. Current term is 11.
I20260709 12:12:09.727145 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:09.636230 12599 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 14: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:12:09.662884 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50258.1us
I20260709 12:12:09.614109 18153 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:09.707347 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:09.712661 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 55061us
I20260709 12:12:09.731810 12675 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:09.733141 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:09.714136 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:09.715019 13336 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: [11.10-13.11]
I20260709 12:12:09.611822 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:09.734668 12679 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:09.711266 18200 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:09.717002 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:09.737449 12677 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 14: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:09.719143 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2525) took 1869 ms. Trace:
I20260709 12:12:09.737898 12671 rpcz_store.cc:276] 0709 12:12:07.849951 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:07.850081 (+   130us) service_pool.cc:224] Handling call
0709 12:12:09.718944 (+1868863us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54003d4ad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:09.719065 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2574537472}
I20260709 12:12:09.712276 18031 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:09.718017 17749 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:09.697047 17957 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader election won for term 18
W20260709 12:12:09.725018 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 142186us
W20260709 12:12:09.748162 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:09.719738 12601 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:12:09.693568 17899 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:09.754940 12680 raft_consensus.cc:2376] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15. Re-sending same reply.
I20260709 12:12:09.731776 12678 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:12:09.735183 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 123425us
I20260709 12:12:09.737449 12494 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:09.744565 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:09.744781 12600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2557) took 1504 ms (client timeout 1976 ms). Trace:
I20260709 12:12:09.746657 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:09.677629 13304 leader_election.cc:336] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.659s (SENT)
I20260709 12:12:09.756929 12528 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 14: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:09.757283 13304 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:09.750530 12569 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:12:09.756711 12600 rpcz_store.cc:269] 0709 12:12:08.240430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:08.240564 (+   134us) service_pool.cc:224] Handling call
0709 12:12:09.744534 (+1503970us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b54001946d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:09.744695 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1683750784}
I20260709 12:12:09.751158 13302 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:09.759320 18176 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:09.759821 18034 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:09.759891 18176 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:09.760198 18075 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:09.760934 18075 raft_consensus.cc:697] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:09.761533 18195 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.572s	user 0.006s	sys 0.006s
I20260709 12:12:09.761066 18176 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:09.740477 18239 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:12:09.761972 18075 consensus_queue.cc:237] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:09.762605 18239 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:09.752120 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:09.763095 18239 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:09.763671 12671 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 13: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:09.764285 18153 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:09.764807 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:09.711395 18162 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:09.753700 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:09.711933 17989 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:09.766161 12522 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:12:09.769587 12494 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:09.740073 18009 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:12:09.771682 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:09.775596 12677 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 11: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:09.715823 17760 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:09.706934 18158 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:09.715986 18081 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:09.778417 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:09.778721 12677 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 10. Current term is 11. Ops: [9.7-10.8]
I20260709 12:12:09.779556 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:09.743465 17749 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:09.785974 12671 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: [10.6-12.7]
I20260709 12:12:09.728377 18088 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:09.772107 17582 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.083s
I20260709 12:12:09.855419 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:09.856263 12598 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 14: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:12:09.785535 18153 consensus_queue.cc:1175] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e56d487365394ac5ac5f1002926a5645" responder_term: 19 status { last_received { term: 18 index: 14 } last_committed_idx: 14 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:12:09.856422 18088 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:09.857299 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:09.790835 17749 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:09.858646 17760 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:09.859833 18140 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:09.793679 12494 leader_election.cc:400] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18.
I20260709 12:12:09.860699 12494 leader_election.cc:403] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
W20260709 12:12:09.861260 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 67782.5us
I20260709 12:12:09.862565 18192 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:09.780781 18210 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:12:09.863473 18210 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:09.863921 18210 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:09.864406 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:09.865269 12604 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 15: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:09.796612 12601 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:12:09.808359 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:09.868014 12605 raft_consensus.cc:2393] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 15: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:09.868633 12675 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:09.802335 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:09.854612 12523 raft_consensus.cc:2368] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16.
I20260709 12:12:09.869972 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:09.852473 12677 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:12:09.873946 12677 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:12:09.874709 12679 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:09.742098 18212 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:12:09.888600 12494 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:09.901767 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:09.890830 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 150743us
W20260709 12:12:09.923239 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 188887us
I20260709 12:12:09.924258 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:09.925947 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:09.928821 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:09.932090 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 180989us
I20260709 12:12:09.794685 18009 raft_consensus.cc:697] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:09.893774 18140 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18.
I20260709 12:12:09.885699 18162 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:09.918968 18203 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:09.918666 18195 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:09.922537 18237 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:12:09.950819 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 273228us
I20260709 12:12:09.953496 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:09.956789 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:09.964684 12598 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:12:09.965036 12672 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 10.
I20260709 12:12:09.968927 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:09.969165 13338 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:12:09.975695 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 617) took 1196 ms. Trace:
I20260709 12:12:09.976361 12527 rpcz_store.cc:276] 0709 12:12:08.779362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:08.779580 (+   218us) service_pool.cc:224] Handling call
0709 12:12:09.975421 (+1195841us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b54003c1254. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:09.975609 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":763766016}
I20260709 12:12:09.780784 17819 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:09.983573 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:09.984241 12672 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:12:10.014434 17899 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:10.015906 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:09.880647 18193 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.10, Last appended by leader: 10, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:09.976169 18203 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:10.022681 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:10.024408 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 48137.3us
I20260709 12:12:09.985556 18195 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:10.003554 18237 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:10.049587 13332 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:12:10.051313 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:09.955965 18031 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:10.116915 12672 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:10.118774 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:10.032831 18032 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:10.041136 18203 raft_consensus.cc:697] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:10.038676 17869 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:10.221833 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:10.231383 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:10.029536 18088 consensus_queue.cc:1059] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.056s
W20260709 12:12:10.256762 18146 consensus_meta.cc:287] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.502s	user 0.006s	sys 0.006s
I20260709 12:12:10.224089 17582 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:10.239473 17989 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:10.101775 18172 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.171s
I20260709 12:12:10.039116 18009 consensus_queue.cc:237] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:10.258874 18200 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:10.230976 12478 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 129833us
W20260709 12:12:10.295771 18245 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.287s	user 0.001s	sys 0.000s
I20260709 12:12:10.296721 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:10.298648 18250 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:10.299413 18250 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:10.302439 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:10.303473 12601 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:10.304726 18203 consensus_queue.cc:237] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:10.275369 18081 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:10.308805 18081 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:12:10.313724 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:10.283595 18146 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:12:10.315747 18257 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:12:10.317265 18146 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:12:10.317735 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:10.317947 12597 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:10.318562 12671 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 11: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:10.319438 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:10.323845 18253 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.055s	user 0.000s	sys 0.001s
I20260709 12:12:10.342231 18088 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:10.275872 18195 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:10.345801 18227 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.812s	user 0.002s	sys 0.007s
W20260709 12:12:10.347980 18229 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.968s	user 0.001s	sys 0.002s
W20260709 12:12:10.292761 18239 consensus_meta.cc:287] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.529s	user 0.013s	sys 0.000s
I20260709 12:12:10.408071 17749 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:10.418485 12677 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 16: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:12:10.422427 12531 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.503s	user 0.000s	sys 0.006s
I20260709 12:12:10.355677 17899 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:10.289077 17582 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:10.449093 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 420609us
W20260709 12:12:10.451006 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 407919us
I20260709 12:12:10.453392 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:10.417971 17749 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:10.471762 12600 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:12:10.450177 12646 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:10.476662 12646 leader_election.cc:400] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 10. Current term is 11.
W20260709 12:12:10.464367 12679 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.025s	sys 0.011s
W20260709 12:12:10.477954 12679 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.025s	sys 0.011s
W20260709 12:12:10.457170 12494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.002s	sys 0.007s
W20260709 12:12:10.479705 12494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.560s	user 0.002s	sys 0.007s
W20260709 12:12:10.480122 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 591607us
I20260709 12:12:10.483115 12599 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 12:12:10.485078 13305 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.007s	sys 0.006s
I20260709 12:12:10.484388 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:10.485450 13305 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.007s	sys 0.006s
W20260709 12:12:10.477181 17957 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.004s	sys 0.005s
I20260709 12:12:10.489522 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:12:10.477347 18075 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.003s	sys 0.010s
W20260709 12:12:10.486487 17691 consensus_meta.cc:287] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.888s	user 0.010s	sys 0.003s
I20260709 12:12:10.515810 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:10.483790 18262 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:10.448071 17899 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:10.422912 18239 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:10.492592 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:10.494160 17957 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.005s	sys 0.005s
I20260709 12:12:10.502312 12646 leader_election.cc:403] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: Cancelling election due to peer responding with higher term
W20260709 12:12:10.993290 13330 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.006s	sys 0.010s
W20260709 12:12:11.247633 13330 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.007s	sys 0.015s
I20260709 12:12:10.901892 18213 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:10.849730 18164 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:11.203914 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:11.250171 13336 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.686s	user 0.008s	sys 0.003s
W20260709 12:12:10.508311 18075 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.004s	sys 0.010s
W20260709 12:12:11.206709 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 730220us
W20260709 12:12:10.570257 18265 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
W20260709 12:12:10.576867 18162 consensus_meta.cc:287] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.618s	user 0.012s	sys 0.000s
I20260709 12:12:10.543224 18262 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:12:10.743535 18210 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.879s	user 0.013s	sys 0.000s
I20260709 12:12:10.542725 17691 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:10.925884 18276 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:10.740123 17582 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:10.537770 18192 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.721s
I20260709 12:12:11.238201 17708 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:11.193331 17957 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:11.191892 12676 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
W20260709 12:12:11.248764 12679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.770s	user 0.017s	sys 0.086s
W20260709 12:12:11.286029 12679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.018s	sys 0.086s
W20260709 12:12:11.286907 12676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2903) took 1362 ms (client timeout 1772 ms). Trace:
W20260709 12:12:11.287523 12676 rpcz_store.cc:269] 0709 12:12:09.924801 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:10.492252 (+567451us) service_pool.cc:224] Handling call
0709 12:12:11.286817 (+794565us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:11.288908 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:11.289996 12676 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 11: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:12:11.250545 13336 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.008s	sys 0.003s
W20260709 12:12:11.249174 18193 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.006s	sys 0.011s
W20260709 12:12:11.037401 18237 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.988s	user 0.014s	sys 0.000s
I20260709 12:12:11.265950 12671 raft_consensus.cc:2410] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:12:11.254710 12646 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:11.297583 17708 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:11.348965 17691 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:11.350679 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2900) took 1562 ms. Trace:
I20260709 12:12:11.261473 18213 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:11.261921 18164 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:11.265079 18262 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:11.369153 12679 rpcz_store.cc:276] 0709 12:12:09.788214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.880163 (+ 91949us) service_pool.cc:224] Handling call
0709 12:12:09.880677 (+   514us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:09.881021 (+   344us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:12:10.478696 (+597675us) log.cc:844] Serialized 14 byte log entry
0709 12:12:11.349710 (+871014us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:11.349979 (+   269us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:11.350296 (+   317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:11.350412 (+   116us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:11.350584 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3290,"prepare.queue_time_us":381990,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":3063,"thread_start_us":1404460,"threads_started":2}
I20260709 12:12:11.270615 18162 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election lost for term 10. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 10. Current term is 11.
W20260709 12:12:11.371264 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 116670us
I20260709 12:12:11.281668 17582 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:11.269009 18210 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:11.366778 18193 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.006s	sys 0.011s
I20260709 12:12:11.426725 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:11.427922 12679 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
W20260709 12:12:11.428715 12679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 556) took 1505 ms (client timeout 1787 ms). Trace:
W20260709 12:12:11.429246 12679 rpcz_store.cc:269] 0709 12:12:09.923657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:11.394709 (+1471052us) service_pool.cc:224] Handling call
0709 12:12:11.428634 (+ 33925us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:11.372571 18237 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:11.395092 18270 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:11.281740 18276 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:11.471743 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:11.474700 13335 raft_consensus.cc:2410] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
W20260709 12:12:11.493018 12679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2907) took 1546 ms (client timeout 1783 ms). Trace:
W20260709 12:12:11.493698 12679 rpcz_store.cc:269] 0709 12:12:09.946581 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:11.430216 (+1483635us) service_pool.cc:224] Handling call
0709 12:12:11.492909 (+ 62693us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:11.501163 17708 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:11.561854 18164 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:11.562762 18284 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 12:12:11.460704 18239 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:11.423075 17582 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:11.576184 17899 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:11.571717 18269 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:11.467402 18270 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:11.486927 17957 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:11.618206 18282 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.245s	user 0.000s	sys 0.000s
I20260709 12:12:11.621106 18276 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:11.637272 18270 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:11.679059 18289 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
W20260709 12:12:11.681294 18285 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
W20260709 12:12:11.682355 18280 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.175s	user 0.001s	sys 0.000s
I20260709 12:12:11.683295 18210 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:11.185065 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:11.738799 12569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.582s	user 0.001s	sys 0.010s
W20260709 12:12:11.739596 12569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.001s	sys 0.010s
I20260709 12:12:11.742580 13336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 603) took 1708 ms. Trace:
W20260709 12:12:11.744838 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:11.743358 13336 rpcz_store.cc:276] 0709 12:12:10.033894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:10.036541 (+  2647us) service_pool.cc:224] Handling call
0709 12:12:10.056026 (+ 19485us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:10.481944 (+425918us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:10.481995 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:10.484986 (+  2991us) log.cc:844] Serialized 26 byte log entry
0709 12:12:11.296572 (+811586us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:11.296640 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:11.296746 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:11.741500 (+444754us) raft_consensus.cc:1685] finished
0709 12:12:11.741586 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:11.742091 (+   505us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47473,"raft.queue_time_us":427585,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":23830,"thread_start_us":732059,"threads_started":1,"wal-append.queue_time_us":1079279}
W20260709 12:12:11.403452 12676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 567) took 1467 ms (client timeout 1765 ms). Trace:
W20260709 12:12:11.759316 12676 rpcz_store.cc:269] 0709 12:12:09.936331 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:11.288561 (+1352230us) service_pool.cc:224] Handling call
0709 12:12:11.403343 (+114782us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:11.763813 12523 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2519) took 2041 ms (client timeout 2188 ms). Trace:
W20260709 12:12:11.764428 12523 rpcz_store.cc:269] 0709 12:12:09.722268 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.722536 (+   268us) service_pool.cc:224] Handling call
0709 12:12:11.763231 (+2040695us) spinlock_profiling.cc:243] Waited 68.8 ms on lock 0x7b5400407c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:11.763553 (+   322us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":158172544}
W20260709 12:12:11.767364 12675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2891) took 2014 ms (client timeout 2153 ms). Trace:
W20260709 12:12:11.768039 12675 rpcz_store.cc:269] 0709 12:12:09.753181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.753398 (+   217us) service_pool.cc:224] Handling call
0709 12:12:11.766738 (+2013340us) spinlock_profiling.cc:243] Waited 101 ms on lock 0x7b5400013454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:11.767058 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":233367936}
I20260709 12:12:11.770082 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:11.788591 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:11.788911 12671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 563) took 1860 ms (client timeout 1743 ms). Trace:
I20260709 12:12:11.779184 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2768) took 2701 ms. Trace:
W20260709 12:12:11.789546 12671 rpcz_store.cc:269] 0709 12:12:09.928630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:10.319164 (+390534us) service_pool.cc:224] Handling call
0709 12:12:11.788306 (+1469142us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b5400092ed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:11.788649 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2174395136}
W20260709 12:12:11.780357 12678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2879) took 2081 ms (client timeout 2028 ms). Trace:
W20260709 12:12:11.781324 12673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2882) took 2078 ms (client timeout 2036 ms). Trace:
W20260709 12:12:11.787459 12522 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 615) took 3009 ms (client timeout 1765 ms). Trace:
W20260709 12:12:12.024158 12572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.014s	sys 0.025s
W20260709 12:12:12.024863 12572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.820s	user 0.014s	sys 0.025s
I20260709 12:12:12.025153 17708 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.026364 17708 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:11.772799 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:12.009486 12602 rpcz_store.cc:276] 0709 12:12:09.077864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.172673 (+ 94809us) service_pool.cc:224] Handling call
0709 12:12:09.172975 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:09.506550 (+333575us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:11.778361 (+2271811us) spinlock_profiling.cc:243] Waited 237 ms on lock 0x7b5400232154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:11.778869 (+   508us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":545911552}
W20260709 12:12:12.023110 12522 rpcz_store.cc:269] 0709 12:12:08.778082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:08.778410 (+   328us) service_pool.cc:224] Handling call
0709 12:12:11.786893 (+3008483us) spinlock_profiling.cc:243] Waited 982 ms on lock 0x7b54003f6954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:11.787173 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2259410176}
I20260709 12:12:12.042641 18181 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:12.042800 18270 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:12.043849 18181 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.044873 18270 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:12.019856 12673 rpcz_store.cc:269] 0709 12:12:09.703280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.703594 (+   314us) service_pool.cc:224] Handling call
0709 12:12:11.780760 (+2077166us) spinlock_profiling.cc:243] Waited 811 ms on lock 0x7b54000933d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:11.781083 (+   323us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1865270016}
W20260709 12:12:12.049465 12673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2910) took 2060 ms (client timeout 1962 ms). Trace:
W20260709 12:12:12.049923 12673 rpcz_store.cc:269] 0709 12:12:09.989065 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:12.049139 (+2060074us) service_pool.cc:211] Skipping call since client already timed out
0709 12:12:12.049374 (+   235us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:12:11.771241 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:12.059532 18239 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.060654 18239 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.062600 18269 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:12.063735 18269 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.064056 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:12.065336 12599 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:12.066850 17899 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:11.789893 12675 raft_consensus.cc:2376] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14. Re-sending same reply.
I20260709 12:12:11.691646 18262 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:12.018764 12678 rpcz_store.cc:269] 0709 12:12:09.698988 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.699296 (+   308us) service_pool.cc:224] Handling call
0709 12:12:11.779408 (+2080112us) spinlock_profiling.cc:243] Waited 22.6 ms on lock 0x7b54003bd654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:11.780044 (+   636us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":52029184}
W20260709 12:12:12.092801 12675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2913) took 2070 ms (client timeout 1908 ms). Trace:
W20260709 12:12:12.093461 12675 rpcz_store.cc:269] 0709 12:12:10.015747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:11.769345 (+1753598us) service_pool.cc:224] Handling call
0709 12:12:12.086482 (+317137us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:12.103963 12673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2911) took 2105 ms (client timeout 2028 ms). Trace:
W20260709 12:12:12.102073 12676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 565) took 2172 ms (client timeout 1884 ms). Trace:
W20260709 12:12:12.108793 12676 rpcz_store.cc:269] 0709 12:12:09.929179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:11.761728 (+1832549us) service_pool.cc:224] Handling call
0709 12:12:12.101914 (+340186us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:12.107744 12673 rpcz_store.cc:269] 0709 12:12:09.991271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:12.087486 (+2096215us) service_pool.cc:211] Skipping call since client already timed out
0709 12:12:12.096345 (+  8859us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:12:12.111914 18293 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:12.112780 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:12.068087 17899 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:12.116110 12675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 562) took 2187 ms (client timeout 2092 ms). Trace:
W20260709 12:12:12.123767 12678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2912) took 2103 ms (client timeout 2036 ms). Trace:
W20260709 12:12:12.124614 12678 rpcz_store.cc:269] 0709 12:12:10.015384 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:12.108110 (+2092726us) service_pool.cc:211] Skipping call since client already timed out
0709 12:12:12.118772 (+ 10662us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:12:12.126592 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:12.127815 12678 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
W20260709 12:12:12.128625 12678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2915) took 2112 ms (client timeout 2743 ms). Trace:
W20260709 12:12:12.129087 12678 rpcz_store.cc:269] 0709 12:12:10.016255 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:12.125916 (+2109661us) service_pool.cc:224] Handling call
0709 12:12:12.128499 (+  2583us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:12.083582 18291 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:12.117870 12671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2908) took 2156 ms (client timeout 2046 ms). Trace:
W20260709 12:12:12.130584 12671 rpcz_store.cc:269] 0709 12:12:09.958638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:12.100615 (+2141977us) service_pool.cc:211] Skipping call since client already timed out
0709 12:12:12.114652 (+ 14037us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:12:12.123158 12675 rpcz_store.cc:269] 0709 12:12:09.928378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:12.106276 (+2177898us) service_pool.cc:211] Skipping call since client already timed out
0709 12:12:12.116022 (+  9746us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:12:12.087841 17708 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:12.113164 18293 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:12.141997 12679 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:12:12.152376 18175 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.153582 18175 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:12.157407 18175 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:12.135207 18181 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:12.158262 17708 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.158419 18295 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:12.159437 17708 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.160244 18295 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.162396 12678 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:12.162673 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 10 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:12.163928 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2904) took 2235 ms. Trace:
I20260709 12:12:12.164288 12676 raft_consensus.cc:2410] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 13 because replica has last-logged OpId of term: 12 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 11.
I20260709 12:12:12.164952 17899 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:12.165197 18276 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:12.166345 12679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 564) took 2237 ms (client timeout 1822 ms). Trace:
W20260709 12:12:12.167044 12679 rpcz_store.cc:269] 0709 12:12:09.928893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:11.495215 (+1566322us) service_pool.cc:224] Handling call
0709 12:12:12.166178 (+670963us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:12.166590 18276 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.168797 18074 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:12.169961 18074 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:12.170431 18210 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.171818 18210 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.173372 18074 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:12.174399 18291 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:12.165009 12676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 559) took 2237 ms (client timeout 2520 ms). Trace:
I20260709 12:12:12.179131 18298 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:12.179829 12676 rpcz_store.cc:269] 0709 12:12:09.927472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:12.138261 (+2210789us) service_pool.cc:224] Handling call
0709 12:12:12.164938 (+ 26677us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:12.164565 12678 rpcz_store.cc:276] 0709 12:12:09.928544 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:12.134639 (+2206095us) service_pool.cc:224] Handling call
0709 12:12:12.135408 (+   769us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:12.163255 (+ 27847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:12.163678 (+   423us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:12.180513 18298 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:12.203570 18291 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:12.182361 17708 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:12.174052 18074 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.218228 12599 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
W20260709 12:12:12.221765 12599 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 2792) took 1478 ms (client timeout 1637 ms). Trace:
W20260709 12:12:12.222314 12599 rpcz_store.cc:269] 0709 12:12:10.742048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:12.062975 (+1320927us) service_pool.cc:224] Handling call
0709 12:12:12.219815 (+156840us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:12.214442 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:12.158330 18292 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.241660 18292 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:12.243674 18293 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:12.245421 18292 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:12.231561 18296 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.246006 17582 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:12.246766 18296 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:12.247633 18270 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:12.232576 18297 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.250383 18297 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.233053 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2571) took 2350 ms. Trace:
I20260709 12:12:12.234653 17899 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:12:12.238607 12677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 555) took 2267 ms (client timeout 1659 ms). Trace:
I20260709 12:12:12.239430 18239 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:12.240664 18298 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:12.229066 18269 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:12.263130 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:12.254554 12671 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: [13.9-13.9]
W20260709 12:12:12.255167 12673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 557) took 2328 ms (client timeout 2357 ms). Trace:
W20260709 12:12:12.282648 12673 rpcz_store.cc:269] 0709 12:12:09.926806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:12.118844 (+2192038us) service_pool.cc:224] Handling call
0709 12:12:12.255041 (+136197us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:12.291396 18162 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.283982 18296 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:12.291751 18074 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:12.280831 12677 rpcz_store.cc:269] 0709 12:12:09.923390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.925633 (+  2243us) service_pool.cc:224] Handling call
0709 12:12:12.129976 (+2204343us) spinlock_profiling.cc:243] Waited 484 ms on lock 0x7b54000c3ad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:12.190569 (+ 60593us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1113896320}
I20260709 12:12:12.281306 12597 rpcz_store.cc:276] 0709 12:12:09.800944 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.801101 (+   157us) service_pool.cc:224] Handling call
0709 12:12:09.801393 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:10.318366 (+516973us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:12.084490 (+1766124us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b5400079c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:12.150787 (+ 66297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1187541888}
I20260709 12:12:12.254021 18297 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:12.308676 13304 connection.cc:256] RPC call timeout handler was delayed by -2.1693892180001058s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23570000000000002s for a response
I20260709 12:12:12.312757 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2490) took 3492 ms. Trace:
I20260709 12:12:12.299862 12675 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 12:12:12.453848 13304 connection.cc:256] RPC call timeout handler was delayed by -2.0594780850001371s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1537s for a response
W20260709 12:12:12.455281 13304 connection.cc:256] RPC call timeout handler was delayed by -2.0555557250002039s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1705s for a response
W20260709 12:12:12.317935 18230 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 2.767s	user 0.026s	sys 0.041s
I20260709 12:12:12.457578 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2914) took 2438 ms. Trace:
I20260709 12:12:12.462656 18308 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.463464 18308 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.466399 12675 rpcz_store.cc:276] 0709 12:12:10.016017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:12.138451 (+2122434us) service_pool.cc:224] Handling call
0709 12:12:12.261599 (+123148us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:12.454469 (+192870us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:12.454758 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:12.290717 18276 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:12.298640 18075 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:12.450842 18295 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:12.483029 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:12.347976 18299 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:12.446535 17519 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:12.304725 17981 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:12.492060 18297 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:12.492690 18276 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.494338 18210 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:12.521591 18276 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:12.522591 12680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2880) took 2822 ms (client timeout 1997 ms). Trace:
I20260709 12:12:12.498324 18301 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:12.523033 12680 rpcz_store.cc:269] 0709 12:12:09.700394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.700696 (+   302us) service_pool.cc:224] Handling call
0709 12:12:12.522196 (+2821500us) spinlock_profiling.cc:243] Waited 53.1 ms on lock 0x7b540015dbd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:12.522389 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":122229120}
I20260709 12:12:12.523765 18301 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.525080 18276 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:12.501468 18305 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.450518 17899 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.505486 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2495) took 3683 ms. Trace:
I20260709 12:12:12.298537 18162 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.521450 17691 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.522253 18297 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.503732 18075 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.541661 18297 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:12.537946 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:12.552431 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:12.528707 12524 rpcz_store.cc:276] 0709 12:12:08.821684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.106365 (+284681us) service_pool.cc:224] Handling call
0709 12:12:09.106848 (+   483us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:09.107498 (+   650us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:09.107557 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:09.108239 (+   682us) log.cc:844] Serialized 26 byte log entry
0709 12:12:09.362339 (+254100us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:09.362401 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:09.362538 (+   137us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:12.316600 (+2954062us) spinlock_profiling.cc:243] Waited 26.6 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:12.504777 (+188177us) raft_consensus.cc:1685] finished
0709 12:12:12.504813 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:12.505221 (+   408us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8147,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":7226,"spinlock_wait_cycles":61140736,"thread_start_us":140405,"threads_started":1,"wal-append.queue_time_us":303508}
W20260709 12:12:12.553061 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:12.454020 12531 rpcz_store.cc:276] 0709 12:12:08.820280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.090578 (+270298us) service_pool.cc:224] Handling call
0709 12:12:09.102958 (+ 12380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:09.245429 (+142471us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:12:09.720245 (+474816us) log.cc:844] Serialized 14 byte log entry
0709 12:12:10.423136 (+702891us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:10.423234 (+    98us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:10.423312 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:12.312065 (+1888753us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b54003f7354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:12.312147 (+    82us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:12.312477 (+   330us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":164360,"prepare.run_cpu_time_us":9176,"prepare.run_wall_time_us":59914,"raft.queue_time_us":57626,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":7431,"spinlock_wait_cycles":326723328,"thread_start_us":1117263,"threads_started":3,"wal-append.queue_time_us":235558,"wal-append.run_cpu_time_us":8306,"wal-append.run_wall_time_us":1341377}
W20260709 12:12:12.555346 13335 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38472 (request call id 605) took 2520 ms (client timeout 2136 ms). Trace:
W20260709 12:12:12.555860 13335 rpcz_store.cc:269] 0709 12:12:10.034546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:10.050948 (+ 16402us) service_pool.cc:224] Handling call
0709 12:12:12.554752 (+2503804us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54000d7854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:12.555052 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3262216448}
I20260709 12:12:12.556660 18162 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:12.537488 17519 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:12.542399 17691 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:12.543743 18075 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:12.545192 17899 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:12.526262 18295 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.484766 18308 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:12.546446 17957 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.547426 18305 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.566672 18305 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:12.549818 18313 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.550894 17981 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:12.558831 13304 connection.cc:256] RPC call timeout handler was delayed by -1.8956780850000996s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17190000000000003s for a response
W20260709 12:12:12.570824 13304 connection.cc:256] RPC call timeout handler was delayed by -1.8740780850000647s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1743s for a response
W20260709 12:12:12.571266 13304 connection.cc:256] RPC call timeout handler was delayed by -1.8029780850001771s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18220000000000003s for a response
W20260709 12:12:12.571666 13304 connection.cc:256] RPC call timeout handler was delayed by -1.7471780850000869s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1884s for a response
W20260709 12:12:12.572055 13304 connection.cc:256] RPC call timeout handler was delayed by -1.7446278859999893s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1593s for a response
W20260709 12:12:12.572415 13304 connection.cc:256] RPC call timeout handler was delayed by -1.5599780850000116s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20920000000000002s for a response
W20260709 12:12:12.572790 13304 connection.cc:256] RPC call timeout handler was delayed by -1.539201574000117s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1765s for a response
W20260709 12:12:12.573180 13304 connection.cc:256] RPC call timeout handler was delayed by -1.2959665800001403s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.252s for a response
W20260709 12:12:12.510804 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 107223us
W20260709 12:12:12.578464 12572 connection.cc:256] RPC call timeout handler was delayed by -1.3154638499997873s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20550000000000002s for a response
I20260709 12:12:12.581171 17981 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.584488 17981 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:12.585382 18301 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:12.586637 18295 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.589664 18074 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:12.577315 18075 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:12.549821 18299 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.591754 18295 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:12.590739 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:12.592065 17691 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:12.598477 17957 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:12.604379 18308 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:12.605468 17899 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.606582 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:12.607209 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:12.608422 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:12.609390 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:12.611528 12526 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:12:12.612860 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:12.613191 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:12.607250 18313 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.621639 13333 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 13. Current term is 14. Ops: [13.12-13.12]
I20260709 12:12:12.623210 13337 raft_consensus.cc:2410] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 15 because replica has last-logged OpId of term: 13 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 9.
I20260709 12:12:12.623903 12531 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 16: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:12.628836 18299 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:12.644526 17519 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:12.621311 18308 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:12.625833 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:12.621680 18303 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.653635 13338 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:12:12.622354 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:12.653944 18303 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.631585 18075 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.635123 12572 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:12.620699 17981 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:12.632863 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:12.647924 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:12.648702 18308 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:12.645022 18301 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.676970 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:12.649142 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:12.652194 12526 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:12.626551 18074 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:12.632275 18313 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:12.638912 13304 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:12.681047 12524 raft_consensus.cc:2410] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:12:12.662840 12527 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:12.669044 12597 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:12:12.662039 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:12.622092 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:12.621704 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:12.693917 18112 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 3.129s	user 0.014s	sys 0.044s
I20260709 12:12:12.694442 12522 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: []
W20260709 12:12:12.694857 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 84278.1us
I20260709 12:12:12.695694 12529 raft_consensus.cc:3055] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:12.695605 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:12.697005 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:12.664641 18299 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:12.703109 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:12.751830 12494 leader_election.cc:400] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18.
I20260709 12:12:12.753194 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:12.754681 18317 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:12.754504 18299 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:12.755610 18317 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.757803 18299 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:12.758342 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 119649us
I20260709 12:12:12.758544 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:12.759501 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:12.753587 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:12.762522 18210 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:12.762925 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:12.763293 12527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:12.763731 18317 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:12.764147 12527 raft_consensus.cc:606] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:12.764364 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:12.765622 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:12.766618 12526 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:12:12.767920 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:12.768038 18317 consensus_queue.cc:1059] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 9.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:12.768836 12526 raft_consensus.cc:2410] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 15 because replica has last-logged OpId of term: 13 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:12:12.769233 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:12.700917 18150 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:12.701839 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:12.664322 18303 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:12.634315 17957 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:12.670893 17981 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.763038 18210 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:12.700680 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:12.761603 13304 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:12.769966 12522 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:12.792714 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:12.780840 12680 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:12.769901 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:12.780720 18112 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 3.238s	user 0.016s	sys 0.046s
I20260709 12:12:12.765663 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:12.787322 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 95967us
I20260709 12:12:12.665926 17519 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:12.782850 12529 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:12.771113 18313 raft_consensus.cc:3055] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:12.807818 13337 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 14: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:12.804867 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:12.811368 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:12.692569 18295 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.813212 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:12.781903 18150 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:12.626605 17899 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.795279 18210 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:12.825186 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:12.827973 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:12.828855 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:12.805327 17519 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:12.834193 18249 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:12.833528 12601 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:12:12.834952 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:12.835426 18150 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.836928 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:12.837721 13333 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:12.838917 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:12.840353 12599 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:12.810890 18313 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:12.843310 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 81865.4us
I20260709 12:12:12.847508 12527 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:12:12.801184 18270 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:12.863574 12528 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:12.823823 18301 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.860245 12597 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:12:12.862651 12602 raft_consensus.cc:2410] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 12 because replica has last-logged OpId of term: 9 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:12:12.821512 18075 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:12.864773 12680 raft_consensus.cc:2410] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:12:12.865533 12522 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:12:12.856634 13336 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:12.633911 18090 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.12, Last appended by leader: 12, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:12.856068 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 61194.5us
I20260709 12:12:12.873121 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:12.877200 12676 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 16: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:12.855788 17519 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:12.880842 18150 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:12.885519 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:12.886963 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:12.887699 12530 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:12:12.887883 18299 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:12.888707 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:12.888619 18299 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:12.889465 12530 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:12.878213 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:12.880373 12569 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:12.880800 18009 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.881278 18311 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:12:12.913144 12529 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.914740 12529 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:12.915983 18307 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:12:12.917752 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 57494.3us
I20260709 12:12:12.918582 12646 leader_election.cc:400] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16.
I20260709 12:12:12.919036 12646 leader_election.cc:403] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:12:12.920384 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 46667.9us
W20260709 12:12:12.890538 13304 leader_election.cc:336] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Remote error: Timed out: Call waited in the queue past client deadline
I20260709 12:12:12.923993 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:12.876163 17708 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:12.940583 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:12.878952 18074 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:12.921932 12646 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:12.969271 13304 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:12.967697 18090 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:12.857527 18295 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:12.986910 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:12.983714 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:12.916941 18307 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.994513 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:12.922382 18308 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:12:12.922773 18270 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:12.969256 18009 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:13.159035 12677 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 11 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:12.995610 12602 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:13.282241 13337 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
W20260709 12:12:13.305706 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:12.998773 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 94483.9us
I20260709 12:12:13.158766 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2906) took 3218 ms. Trace:
I20260709 12:12:13.307356 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:13.307521 17957 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:13.072575 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:13.308473 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:13.308918 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:13.308281 17957 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:13.311041 12643 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.583s (SENT) [suppressed 27 similar messages]
I20260709 12:12:13.311676 17957 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:13.149689 18140 raft_consensus.cc:3055] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:13.312877 18140 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:13.314327 18140 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:13.315855 18140 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:13.318403 18220 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:12:13.318470 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 427978us
I20260709 12:12:13.319406 18220 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:13.320077 18220 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:13.320142 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:13.182852 18305 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.139s
I20260709 12:12:13.166844 18174 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:13.324378 18174 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:13.324880 18174 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:13.172991 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:13.327710 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:13.169762 18036 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:13.091284 17981 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:13.065006 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:13.337709 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:13.339368 12528 raft_consensus.cc:2410] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 15 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
I20260709 12:12:13.337837 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:13.346971 12598 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:13.348805 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:13.340819 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:13.257740 12673 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15.
I20260709 12:12:13.351518 12671 rpcz_store.cc:276] 0709 12:12:09.940570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:12.138226 (+2197656us) service_pool.cc:224] Handling call
0709 12:12:12.225867 (+ 87641us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:12.285287 (+ 59420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:13.158255 (+872968us) spinlock_profiling.cc:243] Waited 28.2 ms on lock 0x7b540036cc54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:13.158633 (+   378us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":66521728}
I20260709 12:12:13.352715 12528 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:12:13.356635 18270 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:13.360985 12571 connection.cc:256] RPC call timeout handler was delayed by -2.417823750000025s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2638s for a response
I20260709 12:12:13.362653 12599 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:12:13.091035 13338 raft_consensus.cc:2410] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
W20260709 12:12:13.088204 12606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2572) took 3286 ms (client timeout 2236 ms). Trace:
W20260709 12:12:13.375231 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 453375us
W20260709 12:12:13.374024 12571 connection.cc:256] RPC call timeout handler was delayed by -2.1164815199999794s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16920000000000002s for a response
W20260709 12:12:13.380430 12571 connection.cc:256] RPC call timeout handler was delayed by -1.9526407189998736s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1573s for a response
W20260709 12:12:13.380985 12571 connection.cc:256] RPC call timeout handler was delayed by -1.8410815199999888s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1998s for a response
W20260709 12:12:13.381472 12571 connection.cc:256] RPC call timeout handler was delayed by -1.6511815199999091s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2209s for a response
W20260709 12:12:13.382642 12571 connection.cc:256] RPC call timeout handler was delayed by -1.5188407189998543s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20550000000000002s for a response
I20260709 12:12:13.172420 18302 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:13.014945 18162 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:13.055454 18311 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
W20260709 12:12:13.375402 12606 rpcz_store.cc:269] 0709 12:12:09.801250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.801764 (+   514us) service_pool.cc:224] Handling call
0709 12:12:13.087608 (+3285844us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b54002710d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:13.087933 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2018614528}
I20260709 12:12:13.356616 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:13.405879 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:13.389582 12571 connection.cc:256] RPC call timeout handler was delayed by -1.410840718999907s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2175s for a response
I20260709 12:12:12.986210 17708 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:13.379868 18303 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:13.410471 12571 connection.cc:256] RPC call timeout handler was delayed by -1.3100815200000397s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.25880000000000003s for a response
I20260709 12:12:13.386480 17819 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:13.393476 17957 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:12:13.420251 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:13.251516 18313 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:13.440923 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:13.434481 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:13.457095 12529 consensus_meta.cc:287] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.541s	user 0.013s	sys 0.001s
W20260709 12:12:13.459340 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 139547us
I20260709 12:12:13.461725 12671 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:12:13.423548 18299 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:13.472813 18233 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.034s	sys 0.037s
W20260709 12:12:13.473239 18266 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 2.825s	user 0.007s	sys 0.015s
I20260709 12:12:13.473547 18262 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:13.427947 17957 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:12:13.478616 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 50749.1us
I20260709 12:12:13.469135 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:13.487241 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:13.487711 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:13.488466 18153 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:13.489566 12646 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:13.483825 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:13.484750 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:13.489485 18088 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:12:13.490942 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:13.418648 18302 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:13.481736 18233 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s	user 0.034s	sys 0.037s
I20260709 12:12:13.491887 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:13.466979 18162 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:13.495975 12524 raft_consensus.cc:2410] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 6.
I20260709 12:12:13.493673 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:13.428139 18009 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:13.500911 12529 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:12:13.495319 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:13.494083 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:13.502913 13304 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:13.499433 12606 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:13.480756 18301 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:13.486032 17899 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:13.493043 17957 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:12:13.503196 18150 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.621s	user 0.009s	sys 0.005s
I20260709 12:12:13.510295 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:13.458747 17819 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:13.506376 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 195353us
W20260709 12:12:13.518085 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 63492.5us
W20260709 12:12:13.519749 12525 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.006s	sys 0.017s
W20260709 12:12:13.520139 12525 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.006s	sys 0.017s
I20260709 12:12:13.482959 18303 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:13.520741 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:13.518766 12597 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:12:13.520926 12599 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:13.524598 12524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:13.525207 12524 raft_consensus.cc:606] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:12:13.501353 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 174337us
W20260709 12:12:13.519469 12569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.535s	user 0.009s	sys 0.004s
W20260709 12:12:13.529255 12569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.545s	user 0.009s	sys 0.004s
W20260709 12:12:13.529732 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 649441us
I20260709 12:12:13.514649 18311 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:13.525564 12524 raft_consensus.cc:614] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:13.539743 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:13.537956 12494 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:13.521256 18295 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:13.556362 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:13.550836 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:13.559054 18032 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:13.559588 13335 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:13.559880 18032 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:13.560424 18032 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:13.550693 18317 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:13.539971 18088 consensus_queue.cc:1059] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.021s
I20260709 12:12:13.561086 18317 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:13.561425 18088 consensus_queue.cc:1243] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 13.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:12:13.561691 18317 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:13.564039 12494 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:13.566540 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:13.554492 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:13.537643 18313 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:13.579101 12597 raft_consensus.cc:2410] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 15 because replica has last-logged OpId of term: 13 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 12.
I20260709 12:12:13.581321 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:13.581395 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:13.514072 17760 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.088s
I20260709 12:12:13.584944 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:13.586462 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:13.549916 18301 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:13.592326 18303 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:13.592638 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:13.601620 18299 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:12:13.603837 13330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38482 (request call id 560) took 3854 ms. Trace:
I20260709 12:12:13.607007 18295 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:13.607674 18295 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:13.586714 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:13.611338 18324 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:12:13.608850 12494 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:13.615041 12606 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:13.527297 18162 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:13.620970 18233 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:12:13.623060 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 86523us
I20260709 12:12:13.585374 18307 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:13.629559 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:13.634600 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:13.643141 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:13.647208 12606 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:13.604708 13330 rpcz_store.cc:276] 0709 12:12:09.749329 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.749491 (+   162us) service_pool.cc:224] Handling call
0709 12:12:09.749844 (+   353us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:09.927499 (+177655us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:09.927536 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:09.933767 (+  6231us) log.cc:844] Serialized 26 byte log entry
0709 12:12:10.220415 (+286648us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:11.248436 (+1028021us) log.cc:844] Serialized 14 byte log entry
0709 12:12:11.248911 (+   475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:11.249042 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:13.161608 (+1912566us) spinlock_profiling.cc:243] Waited 8.82 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:13.603017 (+441409us) raft_consensus.cc:1685] finished
0709 12:12:13.603153 (+   136us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:13.603419 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1023365,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":67,"raft.queue_time_us":36469,"raft.run_cpu_time_us":347,"raft.run_wall_time_us":91857,"spinlock_wait_cycles":20284800,"thread_start_us":1200237,"threads_started":3,"wal-append.queue_time_us":509067}
W20260709 12:12:13.644287 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 66616.8us
I20260709 12:12:13.616119 18295 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:13.658996 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:13.616617 18299 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:13.624563 18301 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:13.667131 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 52265.3us
I20260709 12:12:13.678467 18174 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:13.651882 18302 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:13.681926 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:13.642002 17981 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:12:13.694254 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:13.690452 12531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
W20260709 12:12:13.754928 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:13.762346 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 171753us
W20260709 12:12:13.767462 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 101902us
I20260709 12:12:13.696923 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:13.776079 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:13.694556 18210 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.856s	user 0.007s	sys 0.009s
W20260709 12:12:13.673794 18090 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.677s	user 0.011s	sys 0.002s
W20260709 12:12:13.836725 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:13.858877 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:13.664268 18326 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
W20260709 12:12:13.792199 18196 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.774s	user 0.026s	sys 0.097s
W20260709 12:12:13.863788 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:13.863952 12531 raft_consensus.cc:606] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:13.865797 12531 raft_consensus.cc:614] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:13.861640 12606 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:13.867333 12597 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 14.
W20260709 12:12:13.868466 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 53555.5us
I20260709 12:12:13.869737 12679 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 16: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:13.770159 17981 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:13.871613 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:13.873437 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:13.863755 18036 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.534s	user 0.016s	sys 0.008s
I20260709 12:12:13.866888 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:12:13.868665 18196 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.884s	user 0.034s	sys 0.098s
I20260709 12:12:13.882246 18274 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:13.869037 18220 consensus_meta.cc:287] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.548s	user 0.020s	sys 0.003s
I20260709 12:12:13.885720 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:13.888303 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:13.874294 12600 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:13.890530 12671 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:12:13.861907 18307 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:13.888396 13304 leader_election.cc:336] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.681s (SENT)
I20260709 12:12:13.891626 12646 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:13.895941 12646 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:13.894549 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:13.896502 13339 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.031s	sys 0.013s
W20260709 12:12:13.896853 13339 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.807s	user 0.033s	sys 0.013s
I20260709 12:12:13.896917 12679 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:13.861780 18210 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:13.932948 18210 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:13.933349 12530 raft_consensus.cc:2410] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 14 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:12:13.933830 12527 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 15: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:13.935112 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:13.939883 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:13.940593 13339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 608) took 3467 ms. Trace:
I20260709 12:12:13.941115 13339 rpcz_store.cc:276] 0709 12:12:10.472926 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:10.473163 (+   237us) service_pool.cc:224] Handling call
0709 12:12:10.473563 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:10.493420 (+ 19857us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:10.493454 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:10.493963 (+   509us) log.cc:844] Serialized 26 byte log entry
0709 12:12:13.089972 (+2596009us) spinlock_profiling.cc:243] Waited 9.27 ms on lock 0x7b18004b6160. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:12:13.927812 (+837840us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:13.927884 (+    72us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:13.928029 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:13.940142 (+ 12113us) raft_consensus.cc:1685] finished
0709 12:12:13.940169 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:13.940507 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":636295,"raft.queue_time_us":43061,"raft.run_cpu_time_us":186,"raft.run_wall_time_us":24922,"spinlock_wait_cycles":21316992,"thread_start_us":807096,"threads_started":1,"wal-append.queue_time_us":3156655}
I20260709 12:12:13.945577 12523 pending_rounds.cc:85] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 7
I20260709 12:12:13.874648 17981 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:13.899008 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:13.950011 12597 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:13.948863 12523 pending_rounds.cc:107] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:12:13.902472 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:12:13.895439 18274 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:13.903147 18081 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:13.897925 17899 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:13.905642 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:13.906289 12646 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:13.951972 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:13.950644 12523 raft_consensus.cc:2889] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:13.911581 18250 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:13.914431 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:13.914551 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:13.915664 17708 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:13.893334 18220 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:13.921571 18213 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:13.924458 18090 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:13.987522 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:12:13.926990 18334 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:12:13.953536 18262 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:12:13.930195 18187 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:13.872905 17760 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:13.929962 18317 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:13.949247 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 87578us
I20260709 12:12:13.957907 12672 raft_consensus.cc:2410] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 14 because replica has last-logged OpId of term: 13 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:12:13.957123 18274 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:13.960125 18303 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:13.960165 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:13.965792 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:13.967746 12600 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:12:13.950984 18236 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:13.961019 18081 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:13.959854 18150 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:13.972287 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:13.964653 17899 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:13.974002 18232 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 4.431s	user 0.025s	sys 0.048s
I20260709 12:12:13.969017 18032 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:13.974881 12679 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
W20260709 12:12:14.037956 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 158144us
I20260709 12:12:13.957777 17957 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:13.975489 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 77002.7us
I20260709 12:12:13.975885 18174 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:13.976034 12671 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:13.977430 12598 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 17: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:13.978179 18220 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:13.979485 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2507) took 1284 ms. Trace:
I20260709 12:12:13.980181 12569 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:13.980885 18088 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:12:13.953729 18308 consensus_meta.cc:287] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.582s	user 0.002s	sys 0.009s
I20260709 12:12:14.051352 12646 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:13.925704 18162 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:13.967075 18307 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:13.992659 18140 consensus_meta.cc:287] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.676s	user 0.016s	sys 0.008s
I20260709 12:12:13.994567 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 623) took 1241 ms. Trace:
W20260709 12:12:13.929584 17519 consensus_meta.cc:287] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.047s	user 0.006s	sys 0.007s
I20260709 12:12:14.062820 12530 rpcz_store.cc:276] 0709 12:12:12.753436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:12.891702 (+138266us) service_pool.cc:224] Handling call
0709 12:12:13.982952 (+1091250us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b5400105ed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:13.994489 (+ 11537us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1357623552}
I20260709 12:12:14.006982 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:14.011566 13338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 570) took 1393 ms. Trace:
I20260709 12:12:14.012391 17760 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:14.068077 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:13.988821 18090 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.025617 12522 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:14.013038 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:14.028023 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:14.030272 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:14.030629 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2537) took 1329 ms. Trace:
W20260709 12:12:14.038043 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 149666us
I20260709 12:12:14.004856 18262 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:14.043882 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:14.051277 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 71209.4us
I20260709 12:12:14.052838 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:14.049029 12527 rpcz_store.cc:276] 0709 12:12:12.695075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:12.869849 (+174774us) service_pool.cc:224] Handling call
0709 12:12:13.979193 (+1109344us) spinlock_profiling.cc:243] Waited 944 ms on lock 0x7b5400054954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:13.979389 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2170452352}
I20260709 12:12:14.053445 18187 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:14.092811 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2547) took 1340 ms. Trace:
I20260709 12:12:14.093533 12522 rpcz_store.cc:276] 0709 12:12:12.751939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:12.876444 (+124505us) service_pool.cc:224] Handling call
0709 12:12:13.960008 (+1083564us) spinlock_profiling.cc:243] Waited 33.3 ms on lock 0x7b5800002a98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:12:14.092727 (+132719us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76480768}
W20260709 12:12:14.055213 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:14.089094 12529 rpcz_store.cc:276] 0709 12:12:12.695419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:13.525852 (+830433us) service_pool.cc:224] Handling call
0709 12:12:13.551436 (+ 25584us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:13.627338 (+ 75902us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:12:13.687181 (+ 59843us) log.cc:844] Serialized 14 byte log entry
0709 12:12:14.024576 (+337395us) log.cc:844] Serialized 14 byte log entry
0709 12:12:14.024795 (+   219us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:14.024869 (+    74us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:14.024944 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:14.025041 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:14.025202 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":38965,"prepare.run_cpu_time_us":113,"prepare.run_wall_time_us":6085,"raft.queue_time_us":457,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":10728,"spinlock_wait_cycles":43603456,"thread_start_us":126427,"threads_started":1,"wal-append.queue_time_us":182220}
I20260709 12:12:14.057790 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:14.091487 18199 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:14.095721 12526 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:14.095913 18199 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:14.096140 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 66095.6us
I20260709 12:12:14.096457 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2544) took 1398 ms. Trace:
I20260709 12:12:14.097335 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:14.097855 18342 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:14.088088 18090 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:14.030938 18236 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:14.099769 17957 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:14.096561 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:14.103122 18262 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:14.104228 12605 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
I20260709 12:12:14.104902 18262 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:14.107074 12597 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:14.107798 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2510) took 1411 ms. Trace:
I20260709 12:12:14.064172 18274 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:14.096938 12526 rpcz_store.cc:276] 0709 12:12:12.697880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:14.050883 (+1353003us) service_pool.cc:224] Handling call
0709 12:12:14.073413 (+ 22530us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:14.096185 (+ 22772us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:14.096400 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:14.098315 18174 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:14.109018 17760 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:14.061105 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:14.063232 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:14.065019 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:14.068404 12524 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:12:14.112946 18274 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.134918 18274 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:14.135720 13339 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:14.011469 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:14.051962 17819 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:14.089000 18140 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:14.074340 13337 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:14.072320 13338 rpcz_store.cc:276] 0709 12:12:12.612615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:12.834601 (+221986us) service_pool.cc:224] Handling call
0709 12:12:13.998780 (+1164179us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b54006f8454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:14.006063 (+  7283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":313147264}
I20260709 12:12:14.096355 18199 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:13.966490 18153 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:14.073462 12646 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:14.131424 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 547) took 5895 ms. Trace:
W20260709 12:12:13.950349 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 58372.1us
I20260709 12:12:14.079484 13331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38482 (request call id 564) took 1390 ms. Trace:
W20260709 12:12:14.079099 18339 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:12:14.012251 18299 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:14.051842 18088 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:14.026561 18303 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:14.054898 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:14.026705 18081 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:14.058287 13336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 715) took 1209 ms. Trace:
I20260709 12:12:14.098577 18342 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:14.060904 18308 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16.
I20260709 12:12:14.052614 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:14.169028 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:14.073177 18295 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:14.169799 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 117299us
I20260709 12:12:14.106081 12605 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
W20260709 12:12:14.092278 12496 connection.cc:256] RPC call timeout handler was delayed by -2.1920916230001239s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1504s for a response
W20260709 12:12:14.171135 12496 connection.cc:256] RPC call timeout handler was delayed by -2.1641916230000788s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1535s for a response
W20260709 12:12:14.171536 12496 connection.cc:256] RPC call timeout handler was delayed by -2.0834096959999897s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1584s for a response
W20260709 12:12:14.171888 12496 connection.cc:256] RPC call timeout handler was delayed by -1.9620321780000722s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21530000000000002s for a response
W20260709 12:12:14.172223 12496 connection.cc:256] RPC call timeout handler was delayed by -1.9502244739999242s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17720000000000002s for a response
W20260709 12:12:14.172668 12496 connection.cc:256] RPC call timeout handler was delayed by -1.8565433989999747s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17830000000000001s for a response
W20260709 12:12:14.173066 12496 connection.cc:256] RPC call timeout handler was delayed by -1.7649988729999677s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1603s for a response
W20260709 12:12:14.173494 12496 connection.cc:256] RPC call timeout handler was delayed by -1.4025677180000002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2046s for a response
I20260709 12:12:14.031268 17899 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:14.089659 12528 raft_consensus.cc:2376] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18. Re-sending same reply.
I20260709 12:12:14.175266 18236 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:14.176803 12672 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 558) took 4249 ms (client timeout 1705 ms). Trace:
W20260709 12:12:14.177273 12672 rpcz_store.cc:269] 0709 12:12:09.927209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:10.231024 (+303815us) service_pool.cc:224] Handling call
0709 12:12:14.176500 (+3945476us) spinlock_profiling.cc:243] Waited 3.72 s on lock 0x7b540036d154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:14.176722 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8567186304}
I20260709 12:12:14.182265 18150 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:14.182634 18274 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:14.116663 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:14.185165 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:14.078306 18301 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:14.143994 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:14.140704 18220 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:14.131915 12676 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:14.080267 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:14.143667 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.133785 18032 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:14.150399 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2509) took 1454 ms. Trace:
W20260709 12:12:14.151062 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 91762.9us
I20260709 12:12:14.151015 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:14.154649 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:14.146589 13331 rpcz_store.cc:276] 0709 12:12:12.604012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:12.611507 (+  7495us) service_pool.cc:224] Handling call
0709 12:12:13.084658 (+473151us) spinlock_profiling.cc:243] Waited 3.64 ms on lock 0x7b5800163298. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:12:13.084944 (+   286us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:13.085283 (+   339us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:13.085325 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:13.336756 (+251431us) log.cc:844] Serialized 26 byte log entry
0709 12:12:13.538743 (+201987us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:13.538795 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:13.538892 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:13.993852 (+454960us) raft_consensus.cc:1685] finished
0709 12:12:13.993888 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:13.994265 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8364928,"thread_start_us":201493,"threads_started":1,"wal-append.queue_time_us":298354}
I20260709 12:12:14.138412 12675 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:12:14.154523 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 624) took 1400 ms. Trace:
I20260709 12:12:14.165107 13336 rpcz_store.cc:276] 0709 12:12:12.848431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:12.983103 (+134672us) service_pool.cc:224] Handling call
0709 12:12:13.983349 (+1000246us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b5800163298. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:12:13.994332 (+ 10983us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:14.057739 (+ 63407us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:14.057772 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:14.057823 (+    51us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:14.057866 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:14.057937 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:14.058075 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":33489664}
I20260709 12:12:14.077942 17519 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:14.167493 12529 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:12:14.163794 17760 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:14.170645 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 626) took 1410 ms. Trace:
I20260709 12:12:14.108395 12523 rpcz_store.cc:276] 0709 12:12:12.696144 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:13.944751 (+1248607us) service_pool.cc:224] Handling call
0709 12:12:13.945215 (+   464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:13.970899 (+ 25684us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:13.970932 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:13.971467 (+   535us) log.cc:844] Serialized 26 byte log entry
0709 12:12:14.024469 (+ 53002us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:14.024505 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:14.024578 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:14.107536 (+ 82958us) raft_consensus.cc:1685] finished
0709 12:12:14.107560 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:14.107725 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":52559,"threads_started":1,"wal-append.queue_time_us":115100}
I20260709 12:12:14.182951 18150 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:14.158408 12597 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:12:14.107134 18317 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:14.173933 12496 connection.cc:256] RPC call timeout handler was delayed by -1.3860405560001254s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1908s for a response
I20260709 12:12:14.194631 12680 raft_consensus.cc:2410] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
W20260709 12:12:14.190385 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:14.191226 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 94153.3us
W20260709 12:12:14.237916 12496 connection.cc:256] RPC call timeout handler was delayed by -1.338758271000188s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1962s for a response
I20260709 12:12:14.194594 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2508) took 1499 ms. Trace:
I20260709 12:12:14.225693 12600 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:14.238373 12496 connection.cc:256] RPC call timeout handler was delayed by -1.2787692430001698s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2028s for a response
W20260709 12:12:14.227535 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 129840us
I20260709 12:12:14.227218 12527 rpcz_store.cc:276] 0709 12:12:12.753746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:14.092190 (+1338444us) service_pool.cc:224] Handling call
0709 12:12:14.141918 (+ 49728us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:14.148432 (+  6514us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:14.148483 (+    51us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:14.154074 (+  5591us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:14.154188 (+   114us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:14.154278 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:14.154448 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:14.225936 12526 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:14.227006 12531 rpcz_store.cc:276] 0709 12:12:12.695829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:13.898523 (+1202694us) service_pool.cc:224] Handling call
0709 12:12:13.930988 (+ 32465us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:13.987127 (+ 56139us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:13.987189 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:13.991513 (+  4324us) log.cc:844] Serialized 26 byte log entry
0709 12:12:14.075556 (+ 84043us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:14.075618 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:14.075708 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:14.150049 (+ 74341us) raft_consensus.cc:1685] finished
0709 12:12:14.150085 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:14.150306 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30299,"raft.queue_time_us":51595,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":4242,"thread_start_us":70370,"threads_started":1,"wal-append.queue_time_us":116149}
W20260709 12:12:14.234660 12427 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47524 (request call id 99) took 25617 ms (client timeout 15000 ms). Trace:
W20260709 12:12:14.225234 12675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2557) took 1726 ms (client timeout 2055 ms). Trace:
W20260709 12:12:14.240979 12675 rpcz_store.cc:269] 0709 12:12:12.500851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:14.051422 (+1550571us) service_pool.cc:224] Handling call
0709 12:12:14.225158 (+173736us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:14.161618 18088 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:14.145467 12674 rpcz_store.cc:276] 0709 12:12:08.235693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:08.235834 (+   141us) service_pool.cc:224] Handling call
0709 12:12:08.279914 (+ 44080us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:08.315156 (+ 35242us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:08.315193 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:09.193187 (+877994us) spinlock_profiling.cc:243] Waited 13.5 ms on lock 0x7b2c00037868. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:12:09.193712 (+   525us) log.cc:844] Serialized 26 byte log entry
0709 12:12:09.353851 (+160139us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:09.549144 (+195293us) log.cc:844] Serialized 14 byte log entry
0709 12:12:09.584278 (+ 35134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:09.584368 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:13.973049 (+4388681us) spinlock_profiling.cc:243] Waited 27.4 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:14.130842 (+157793us) raft_consensus.cc:1685] finished
0709 12:12:14.130875 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:14.131188 (+   313us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65637,"prepare.queue_time_us":259565,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":4577,"raft.queue_time_us":77012,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":14462,"spinlock_wait_cycles":93988864,"thread_start_us":336584,"threads_started":2,"wal-append.queue_time_us":348996}
I20260709 12:12:14.231264 12522 rpcz_store.cc:276] 0709 12:12:12.760770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:14.146173 (+1385403us) service_pool.cc:224] Handling call
0709 12:12:14.170104 (+ 23931us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:14.170336 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:14.170359 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:14.170402 (+    43us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:12:14.170437 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:14.170489 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:14.170587 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:14.200853 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:14.238759 12496 connection.cc:256] RPC call timeout handler was delayed by -1.2712411770000926s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2036s for a response
I20260709 12:12:14.225715 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 625) took 1471 ms. Trace:
I20260709 12:12:14.245663 12529 rpcz_store.cc:276] 0709 12:12:12.754071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:14.113784 (+1359713us) service_pool.cc:224] Handling call
0709 12:12:14.152415 (+ 38631us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:14.225462 (+ 73047us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:14.225660 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:14.248422 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:14.248898 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 48170us
I20260709 12:12:14.249302 12528 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:14.249811 12599 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:12:14.151083 18140 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:14.249152 12679 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:14.250073 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:14.197299 18303 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:14.256057 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:14.257433 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:14.236385 18150 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:14.260196 12601 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 13: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:14.258072 12671 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:14.260326 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:14.231082 17760 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:14.262305 17760 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:14.262408 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:14.264768 18274 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:14.266402 12680 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 15: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:14.238512 12525 rpcz_store.cc:276] 0709 12:12:12.695478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:13.544471 (+848993us) service_pool.cc:224] Handling call
0709 12:12:13.563094 (+ 18623us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:13.579800 (+ 16706us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:13.579843 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:13.587730 (+  7887us) log.cc:844] Serialized 26 byte log entry
0709 12:12:13.866591 (+278861us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:13.866652 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:13.866768 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:14.194255 (+327487us) raft_consensus.cc:1685] finished
0709 12:12:14.194281 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:14.194507 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15083,"raft.queue_time_us":20361,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":3644,"thread_start_us":275078,"threads_started":1,"wal-append.queue_time_us":403232}
W20260709 12:12:14.267195 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:14.261351 12675 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:14.266806 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:14.272025 12571 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:14.253759 18313 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:14.272486 12494 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:14.272958 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:14.286278 18308 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:14.292163 12673 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:14.270937 18303 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:14.307488 18342 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:14.311177 18140 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:14.313874 12602 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: []
W20260709 12:12:14.318001 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:14.322085 12600 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:14.323719 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:14.279166 18313 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:14.326663 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 70323us
I20260709 12:12:14.280475 18341 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:12:14.327235 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 81857.9us
W20260709 12:12:14.331497 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:14.292524 18220 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:12:14.330744 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2556) took 1839 ms. Trace:
I20260709 12:12:14.333168 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2802) took 1491 ms. Trace:
I20260709 12:12:14.346215 18032 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:14.320415 18303 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.385637 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:14.347781 18220 raft_consensus.cc:697] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:14.390393 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.354266 12679 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:14.332214 18140 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.023s
W20260709 12:12:14.358413 18345 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:12:14.363780 13305 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:14.375277 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.347169 17519 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:14.363027 18295 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:14.325047 18317 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.067s
W20260709 12:12:14.240731 12427 rpcz_store.cc:269] 0709 12:11:48.564597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:48.564814 (+   217us) service_pool.cc:224] Handling call
0709 12:12:14.181780 (+25616966us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:06.132735 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:06.133676 (+   941us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:06.133749 (+    73us) write_op.cc:435] Acquired schema lock
0709 12:12:06.133786 (+    37us) tablet.cc:662] Decoding operations
0709 12:12:06.135575 (+  1789us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:06.135746 (+   171us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:06.135793 (+    47us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:12:06.532374 (+396581us) tablet.cc:701] Row locks acquired
0709 12:12:06.532451 (+    77us) write_op.cc:260] PREPARE: finished
0709 12:12:06.532729 (+   278us) write_op.cc:270] Start()
0709 12:12:06.532959 (+   230us) write_op.cc:276] Timestamp: P: 1783599126532634 usec, L: 0
0709 12:12:06.533014 (+    55us) op_driver.cc:348] REPLICATION: starting
0709 12:12:06.726537 (+193523us) log.cc:844] Serialized 5133 byte log entry
0709 12:12:08.395812 (+1669275us) op_driver.cc:464] REPLICATION: finished
0709 12:12:08.669573 (+273761us) write_op.cc:301] APPLY: starting
0709 12:12:08.716106 (+ 46533us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:09.075237 (+359131us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:09.075375 (+   138us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:09.668686 (+593311us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:09.669041 (+   355us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0709 12:12:09.669092 (+    51us) write_op.cc:312] APPLY: finished
0709 12:12:09.669532 (+   440us) log.cc:844] Serialized 121 byte log entry
0709 12:12:09.924777 (+255245us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:09.941532 (+ 16755us) write_op.cc:454] Released schema lock
0709 12:12:09.962287 (+ 20755us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:10.039291 (+ 77004us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:10.053561 (+ 14270us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4114410,"reported_tablets":21,"child_traces":[["op",{"apply.queue_time_us":234636,"apply.run_cpu_time_us":51225,"apply.run_wall_time_us":1378975,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58389,"num_ops":13,"prepare.queue_time_us":538375,"prepare.run_cpu_time_us":63132,"prepare.run_wall_time_us":1283097,"raft.queue_time_us":451891,"raft.run_cpu_time_us":17562,"raft.run_wall_time_us":136016,"replication_time_us":1862605,"thread_start_us":1551927,"threads_started":5,"wal-append.queue_time_us":1296475,"wal-append.run_cpu_time_us":13208,"wal-append.run_wall_time_us":2476317}]]}
I20260709 12:12:14.361256 18090 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:14.377713 13339 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:14.377594 12602 rpcz_store.cc:276] 0709 12:12:12.836961 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:13.313382 (+476421us) service_pool.cc:224] Handling call
0709 12:12:14.313190 (+999808us) spinlock_profiling.cc:243] Waited 32.7 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:12:14.313419 (+   229us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:14.328152 (+ 14733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:14.328427 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":75188480}
I20260709 12:12:14.295710 18307 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:14.377554 12677 rpcz_store.cc:276] 0709 12:12:12.486732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:12.488408 (+  1676us) service_pool.cc:224] Handling call
0709 12:12:13.158566 (+670158us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:13.303528 (+144962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:14.320036 (+1016508us) spinlock_profiling.cc:243] Waited 670 ms on lock 0x7b540036cc50. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:14.325957 (+  5921us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1540142336}
I20260709 12:12:14.390578 18303 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:14.351482 17981 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:14.296123 18301 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:14.391175 12527 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:12:14.394464 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.332444 18313 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:14.393298 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.376821 12676 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:12:14.449041 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:14.416219 12646 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:14.393750 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.332001 18341 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:12:14.378069 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 80443.3us
I20260709 12:12:14.332870 18342 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.083s
W20260709 12:12:14.451154 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:14.440322 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 76783.1us
I20260709 12:12:14.440228 18199 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:14.441437 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:14.442039 12522 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:14.419898 17957 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:14.457379 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:14.446405 18301 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.456327 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.390831 18262 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 14.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:12:14.458736 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:14.381184 18088 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:14.454176 18341 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:14.464999 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.412897 18032 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:14.467772 12605 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:14.469624 18301 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:14.445690 18081 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:14.471426 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:14.474715 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 58736.5us
I20260709 12:12:14.462261 12522 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:14.509157 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.477725 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:14.482236 12526 raft_consensus.cc:2393] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 12: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:14.467233 18199 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:14.461423 18140 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:14.483816 12604 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:14.486657 17760 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:14.516352 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 628) took 1754 ms. Trace:
I20260709 12:12:14.487032 12572 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:14.487116 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:14.487870 18290 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:12:14.488881 17981 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:14.526554 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:14.437670 18220 consensus_queue.cc:237] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:14.527529 13339 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:14.499056 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:14.483263 12676 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:12:14.507555 12646 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:14.529204 12672 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:14.503911 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 117300us
I20260709 12:12:14.514686 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:14.485903 18088 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:14.516921 12528 rpcz_store.cc:276] 0709 12:12:12.763532 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:14.175118 (+1411586us) service_pool.cc:224] Handling call
0709 12:12:14.175524 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:14.254048 (+ 78524us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:14.515983 (+261935us) spinlock_profiling.cc:243] Waited 68.9 ms on lock 0x7b540027fbd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:14.516263 (+   280us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":158480768}
I20260709 12:12:14.528381 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:14.533548 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2796) took 1690 ms. Trace:
I20260709 12:12:14.526649 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:14.535466 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:14.535418 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:14.536312 12679 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:14.535661 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:14.537737 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:14.538576 12522 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:14.461576 18150 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:14.534847 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 65141us
I20260709 12:12:14.534004 12604 rpcz_store.cc:276] 0709 12:12:12.834514 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:13.135857 (+301343us) service_pool.cc:224] Handling call
0709 12:12:14.415640 (+1279783us) spinlock_profiling.cc:243] Waited 43.9 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:12:14.415870 (+   230us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:14.515858 (+ 99988us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:14.524413 (+  8555us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100996992}
I20260709 12:12:14.536917 12646 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:14.551003 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:14.553509 13333 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:14.515129 18153 raft_consensus.cc:3055] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:14.555804 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.557654 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.559957 18199 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:12:14.560412 12643 connection.cc:256] RPC call timeout handler was delayed by -1.0662594930001887s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1841s for a response
I20260709 12:12:14.560825 18164 raft_consensus.cc:2764] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
W20260709 12:12:14.560827 12643 connection.cc:256] RPC call timeout handler was delayed by -1.0527594930001669s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18560000000000001s for a response
I20260709 12:12:14.551424 18262 raft_consensus.cc:3055] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:14.562814 18262 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:14.562858 12496 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:14.564069 17760 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:14.554842 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2538) took 1859 ms. Trace:
I20260709 12:12:14.563978 18262 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:14.565200 18262 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:14.566542 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:14.558669 12527 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:14.569013 18236 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:12:14.574186 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 97157.3us
I20260709 12:12:14.533740 18140 raft_consensus.cc:697] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:14.566879 18153 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:14.575775 18140 consensus_queue.cc:237] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:14.576197 18153 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:14.570562 12529 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 14: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:14.564847 12524 rpcz_store.cc:276] 0709 12:12:12.695821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:13.615908 (+920087us) service_pool.cc:224] Handling call
0709 12:12:13.630904 (+ 14996us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:14.134641 (+503737us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:14.542649 (+408008us) spinlock_profiling.cc:243] Waited 437 ms on lock 0x7b540038f4d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:14.554773 (+ 12124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1005316864}
I20260709 12:12:14.574301 18150 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:14.535398 17519 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:14.568738 12646 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:14.579157 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 50917.4us
I20260709 12:12:14.551600 18213 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:14.580760 18213 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.581504 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:14.582639 12676 raft_consensus.cc:2410] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:12:14.578904 18088 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:12:14.584607 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 78330.3us
I20260709 12:12:14.585244 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.540829 12678 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:14.588027 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.536141 13335 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:14.584800 18088 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:14.581135 18213 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:14.596719 12646 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:14.577260 18317 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:14.599283 12676 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:12:14.581631 17519 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:14.598176 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:14.608536 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:14.608911 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:14.600188 18090 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:14.600952 12677 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 10. (index mismatch)
I20260709 12:12:14.594178 18081 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:14.602327 17760 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:14.602391 18274 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:12:14.602353 18193 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:14.593911 18164 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:12:14.634577 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47356.4us
I20260709 12:12:14.613834 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.635879 12643 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:14.617302 12522 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
W20260709 12:12:14.628542 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:14.633136 13339 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:14.634526 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:14.614931 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48780.7us
I20260709 12:12:14.638891 12524 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:12:14.607781 18088 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:14.640386 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.610558 17519 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:14.616649 17957 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.014s
I20260709 12:12:14.626509 18153 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:14.587719 17899 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:14.653215 12677 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:14.647715 18250 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260709 12:12:14.659514 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.660394 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:14.661211 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.662591 12522 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 9.
I20260709 12:12:14.663810 12680 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:14.664520 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.627264 18187 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:14.637557 18164 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:14.639724 18246 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:14.648711 18274 consensus_queue.cc:1243] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 15.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
W20260709 12:12:14.722608 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 48965.9us
W20260709 12:12:14.711949 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 72477.1us
W20260709 12:12:14.722201 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 68304.6us
I20260709 12:12:14.724813 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:14.726574 12679 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:14.724136 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:14.676553 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2817) took 1021 ms. Trace:
I20260709 12:12:14.679512 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2918) took 1207 ms. Trace:
I20260709 12:12:14.728979 12673 rpcz_store.cc:276] 0709 12:12:13.471699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:13.471868 (+   169us) service_pool.cc:224] Handling call
0709 12:12:14.679274 (+1207406us) spinlock_profiling.cc:243] Waited 683 ms on lock 0x7b54000ac3d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:14.679424 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1571456768}
I20260709 12:12:14.682006 12528 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 15: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:14.672029 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 2927) took 1055 ms. Trace:
I20260709 12:12:14.732035 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.731863 12678 rpcz_store.cc:276] 0709 12:12:13.616864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:14.043102 (+426238us) service_pool.cc:224] Handling call
0709 12:12:14.671756 (+628654us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b540016f3d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:14.671943 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":923533056}
W20260709 12:12:14.732901 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:14.675578 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:14.731062 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 134400us
I20260709 12:12:14.665712 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.735589 12646 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:14.736358 18252 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.153s
I20260709 12:12:14.725596 18081 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:12:14.716013 18308 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:14.749714 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:14.752542 12673 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:14.728128 12605 rpcz_store.cc:276] 0709 12:12:13.655086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:14.236869 (+581783us) service_pool.cc:224] Handling call
0709 12:12:14.251743 (+ 14874us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:14.491962 (+240219us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:14.676213 (+184251us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b540007b554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:14.676474 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496316672}
I20260709 12:12:14.730634 18164 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:14.715430 18364 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:14.765149 13333 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:14.766212 12678 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:12:14.673161 18090 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:14.720362 18341 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:15.057698 18341 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:15.057806 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2772) took 5969 ms. Trace:
I20260709 12:12:15.059016 12603 rpcz_store.cc:276] 0709 12:12:09.088205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:09.173923 (+ 85718us) service_pool.cc:224] Handling call
0709 12:12:09.174260 (+   337us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:13.950689 (+4776429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:13.950748 (+    59us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:13.950797 (+    49us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:13.950839 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:14.192170 (+241331us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b1800470e60. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485933522 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:12:14.606324 (+414154us) spinlock_profiling.cc:243] Waited 8.2 ms on lock 0x7b1800470e60. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419
0709 12:12:15.057289 (+450965us) spinlock_profiling.cc:243] Waited 4.71 s on lock 0x7b540007a654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:15.057342 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:15.057593 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":46131,"raft.queue_time_us":46336,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":7726,"spinlock_wait_cycles":10913339264}
I20260709 12:12:15.061230 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.778036 12643 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:14.761888 18274 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:12:15.062826 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 284867us
I20260709 12:12:14.765602 18250 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:12:15.064092 18090 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:15.064628 18090 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:14.791631 18358 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.148s	user 0.000s	sys 0.001s
I20260709 12:12:14.794478 12674 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:12:15.054476 18364 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:14.794906 17519 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:15.067152 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2932) took 1434 ms. Trace:
I20260709 12:12:15.066771 18364 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:15.067945 17519 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:14.797242 17899 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:14.802201 12672 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:15.070978 12678 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:12:15.067987 18364 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:15.067598 12674 rpcz_store.cc:276] 0709 12:12:13.632794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:14.572021 (+939227us) service_pool.cc:224] Handling call
0709 12:12:14.572356 (+   335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:15.065941 (+493585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:15.066693 (+   752us) spinlock_profiling.cc:243] Waited 222 ms on lock 0x7b54000ab4d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:15.067045 (+   352us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":509885824}
I20260709 12:12:15.072523 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:15.065475 18090 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:15.073962 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:15.075644 12677 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:14.765098 18313 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:14.844981 18301 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:15.079391 18313 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:14.772938 18246 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:14.849118 18303 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:14.867524 18213 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:15.080264 18246 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:15.080793 18246 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:14.886968 12671 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:12:15.081893 18213 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:14.880393 12524 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:15.083839 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:14.932494 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:15.087109 12531 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15.
I20260709 12:12:15.087355 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:15.088101 12526 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:15.088251 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:15.089202 12531 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 16: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:15.088971 18164 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:14.937664 12675 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 13 index: 12. Preceding OpId from leader: term: 14 index: 12. (term mismatch)
I20260709 12:12:14.977938 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:15.090286 12523 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:15.090678 12528 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:12:15.091612 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 630) took 2327 ms. Trace:
W20260709 12:12:15.003686 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:14.996646 18187 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:15.092175 12523 rpcz_store.cc:276] 0709 12:12:12.764266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:14.241078 (+1476812us) service_pool.cc:224] Handling call
0709 12:12:14.258644 (+ 17566us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:15.090766 (+832122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:15.091388 (+   622us) spinlock_profiling.cc:243] Waited 831 ms on lock 0x7b54003c1254. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:15.091554 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1912104320}
W20260709 12:12:15.093694 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:14.888890 18365 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.150s	user 0.000s	sys 0.001s
I20260709 12:12:15.091792 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:15.021073 18053 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:15.094729 18053 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:15.089061 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:15.079695 18301 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:14.859922 18088 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:15.117309 12526 raft_consensus.cc:2410] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 12 because replica has last-logged OpId of term: 10 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 7.
I20260709 12:12:15.119184 18088 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:15.080890 12525 raft_consensus.cc:2368] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17.
I20260709 12:12:15.120882 12526 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:15.120805 12680 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:15.091698 18164 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:15.093011 18187 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:15.122895 12526 raft_consensus.cc:1217] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Deduplicated request from leader. Original: 11.6->[12.7-12.7]   Dedup: 12.7->[]
I20260709 12:12:15.123777 12526 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:12:15.124861 18187 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:15.090286 12676 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:15.127285 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:15.127800 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2570) took 1023 ms. Trace:
I20260709 12:12:15.128237 12676 rpcz_store.cc:276] 0709 12:12:14.104847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:14.889594 (+784747us) service_pool.cc:224] Handling call
0709 12:12:14.889956 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:15.125914 (+235958us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:15.127546 (+  1632us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b54003d40d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:15.127741 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":460243200}
I20260709 12:12:15.090315 12675 pending_rounds.cc:85] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 11
I20260709 12:12:15.130067 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:15.130396 12675 pending_rounds.cc:107] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 13.12
I20260709 12:12:15.130741 12675 raft_consensus.cc:2889] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:15.131521 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2573) took 1025 ms. Trace:
I20260709 12:12:15.131852 12675 rpcz_store.cc:276] 0709 12:12:14.105754 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:14.936856 (+831102us) service_pool.cc:224] Handling call
0709 12:12:14.937247 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:15.131267 (+194020us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:15.131461 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:15.032882 12679 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:15.097595 13305 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:15.134243 18187 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:15.134735 18187 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:15.135201 18187 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:15.046763 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:15.137547 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:15.064656 18250 raft_consensus.cc:697] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:15.140038 18250 consensus_queue.cc:237] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:14.767464 18032 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:15.144486 12601 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 2800) took 2308 ms (client timeout 1969 ms). Trace:
W20260709 12:12:15.145042 12601 rpcz_store.cc:269] 0709 12:12:12.836098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:12.995574 (+159476us) service_pool.cc:224] Handling call
0709 12:12:15.144193 (+2148619us) spinlock_profiling.cc:243] Waited 923 ms on lock 0x7b54001176d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:15.144391 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2123507456}
I20260709 12:12:15.097882 12646 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:15.146660 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 48866.7us
I20260709 12:12:15.147487 12646 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:15.081889 18303 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:15.085891 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:15.099426 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:15.152648 12531 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:15.093990 12523 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 12 index: 10. Preceding OpId from leader: term: 14 index: 11. (index mismatch)
I20260709 12:12:15.073350 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:15.094322 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:15.107780 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:15.113561 12494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.695s	user 0.005s	sys 0.004s
I20260709 12:12:15.062011 12530 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 11.
W20260709 12:12:15.243886 12494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.826s	user 0.006s	sys 0.006s
W20260709 12:12:15.244324 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 858805us
I20260709 12:12:15.244920 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:15.165709 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 438937us
I20260709 12:12:15.246356 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:15.168110 18032 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:15.104884 18090 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:15.107192 18140 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:15.165728 18164 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:15.070919 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:15.674698 12672 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 14.
I20260709 12:12:15.096331 18053 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:15.181221 12531 raft_consensus.cc:2410] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:12:15.678609 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2574) took 1087 ms. Trace:
I20260709 12:12:15.679057 12531 rpcz_store.cc:276] 0709 12:12:14.591590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:15.090057 (+498467us) service_pool.cc:224] Handling call
0709 12:12:15.678496 (+588439us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:15.680980 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:15.182231 12527 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 18: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:15.682355 18381 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:15.682991 18381 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:15.683741 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2537) took 1133 ms. Trace:
I20260709 12:12:15.684163 12527 rpcz_store.cc:276] 0709 12:12:14.554012 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:14.715784 (+161772us) service_pool.cc:224] Handling call
0709 12:12:15.683476 (+967692us) spinlock_profiling.cc:243] Waited 449 ms on lock 0x7b54003cafd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:15.683673 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1032360832}
I20260709 12:12:15.685714 12527 pending_rounds.cc:85] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 8
I20260709 12:12:15.185386 12526 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:15.689896 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2529) took 1141 ms. Trace:
I20260709 12:12:15.690330 12526 rpcz_store.cc:276] 0709 12:12:14.548104 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:15.177663 (+629559us) service_pool.cc:224] Handling call
0709 12:12:15.184975 (+  7312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:15.689527 (+504552us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:15.689818 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:15.188344 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 464058us
W20260709 12:12:15.195539 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 471457us
I20260709 12:12:15.073333 12678 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
I20260709 12:12:15.193476 18326 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:15.697345 18326 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:15.216328 12523 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:15.698719 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2536) took 1148 ms. Trace:
I20260709 12:12:15.699373 12523 rpcz_store.cc:276] 0709 12:12:14.550077 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:15.215771 (+665694us) service_pool.cc:224] Handling call
0709 12:12:15.216113 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:15.698394 (+482281us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:15.698653 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:15.701128 12523 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:15.108402 18364 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:15.703161 18364 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:15.247620 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:15.707695 12530 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:15.247521 12496 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:15.249930 18032 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:15.710999 18032 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:15.255690 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:15.714525 12601 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:15.150311 18187 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:15.722172 18364 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:15.722586 18308 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:15.723228 18308 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:15.724151 18187 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:15.261504 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:15.263510 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:15.726022 18307 consensus_queue.cc:1059] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:15.265291 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:15.726961 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2553) took 1352 ms. Trace:
I20260709 12:12:15.727327 12525 rpcz_store.cc:276] 0709 12:12:14.374368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:14.374495 (+   127us) service_pool.cc:224] Handling call
0709 12:12:15.726751 (+1352256us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b54000eef54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:15.726906 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1448931712}
W20260709 12:12:15.263460 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 134255us
I20260709 12:12:15.268647 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:15.268471 18246 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:15.728338 12643 leader_election.cc:336] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.672s (SENT)
I20260709 12:12:15.270812 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:15.730181 12522 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:15.272650 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:15.274197 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:15.731366 12599 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:15.277772 18362 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.518s	user 0.000s	sys 0.002s
W20260709 12:12:15.281563 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:15.307260 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:15.335290 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:15.743835 12529 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
W20260709 12:12:15.616186 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:15.745695 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:15.748862 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:15.686679 12528 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:12:15.687291 18381 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:15.687543 18164 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:15.688324 18053 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:15.688722 18140 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:15.681950 12531 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:15.685961 12527 pending_rounds.cc:107] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:12:15.730408 18317 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260709 12:12:15.697865 18326 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:15.720651 18090 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:15.768838 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:15.752180 18187 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:15.727602 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:15.267161 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:15.728349 12604 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:15.725812 18308 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:15.752849 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:15.758828 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 92103.6us
I20260709 12:12:15.762770 12527 raft_consensus.cc:2889] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:15.693960 12571 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:15.763441 12522 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:12:15.773798 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:15.764611 12601 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
W20260709 12:12:15.766616 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 519143us
I20260709 12:12:15.770619 12606 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 17: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:15.771114 12524 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:15.772254 12605 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:15.726078 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:15.776630 12603 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:15.773082 12599 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:15.773263 12531 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:15.773378 18317 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:15.778765 18317 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:15.779588 18140 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:15.780262 18140 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:12:15.781208 18317 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:15.783846 12496 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:15.785435 12496 leader_election.cc:336] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Remote error: Timed out: Call waited in the queue past client deadline
I20260709 12:12:15.785883 12496 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:15.786051 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:15.786819 12606 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:15.791505 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:15.792187 12646 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:15.794103 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:15.791177 18246 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:15.758620 18381 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.015s
I20260709 12:12:15.799501 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:15.778787 18250 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.003s	sys 0.013s
I20260709 12:12:15.806365 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:15.806638 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:15.811425 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:15.798636 18140 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:15.803579 18384 raft_consensus.cc:3055] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:12:15.810705 18250 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.004s	sys 0.014s
I20260709 12:12:15.798525 18364 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:15.794283 18363 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:12:15.773597 18187 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:15.836956 13339 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:12:15.832795 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 59085us
I20260709 12:12:15.844904 12673 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:15.845606 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:15.835444 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:15.812731 18307 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:15.819751 18246 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:15.855036 12605 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:15.856456 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:15.858069 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:15.847776 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:15.852819 12646 leader_election.cc:400] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17.
I20260709 12:12:15.859900 12605 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:12:15.860714 12646 leader_election.cc:403] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:15.836169 18383 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:12:15.838256 18317 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:15.833469 18140 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:15.870100 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:12:15.875917 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 76487.7us
I20260709 12:12:15.871135 18308 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:15.834748 18384 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:15.874046 12646 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:15.863476 12601 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:12:15.874336 12605 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:12:15.876693 12494 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:15.880422 12646 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:15.862674 18357 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:15.871829 18250 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:15.872238 18383 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:15.872581 18317 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:15.849695 18364 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:15.878219 18308 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:15.889384 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:15.887465 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:15.818368 18164 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:15.910312 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:15.910900 18313 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:15.911130 12531 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:15.890049 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:15.911988 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:15.867762 18246 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:15.888300 18090 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election won for term 13
W20260709 12:12:15.925900 13304 leader_election.cc:336] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.787s (SENT)
W20260709 12:12:15.929039 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 143664us
I20260709 12:12:15.929032 18391 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:15.890110 18383 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:15.884604 18381 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 17
I20260709 12:12:15.884399 18307 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:15.935843 13304 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:15.928920 18140 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.14, Last appended by leader: 14, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:15.961223 18187 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:15.961942 18140 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:15.961876 18187 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 15.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:12:15.931363 18317 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:15.945851 18391 raft_consensus.cc:697] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:15.909899 18308 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:15.944402 12646 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:15.890300 18380 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:15.911547 18313 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:15.935022 18053 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:15.965824 12603 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
W20260709 12:12:15.981726 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 111735us
I20260709 12:12:15.982326 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2519) took 1528 ms. Trace:
I20260709 12:12:15.982738 12524 rpcz_store.cc:276] 0709 12:12:14.455319 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:15.085513 (+630194us) service_pool.cc:224] Handling call
0709 12:12:15.982109 (+896596us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b540038f4d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:15.982263 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1422852864}
I20260709 12:12:15.969398 18388 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:15.991021 12604 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:15.946017 18390 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:12:15.977279 18380 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:15.977144 18250 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:16.005725 18380 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:16.010541 18388 raft_consensus.cc:3055] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:16.011366 18388 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:16.012349 18388 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:16.022413 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50307us
W20260709 12:12:16.023833 12496 leader_election.cc:336] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Remote error: Timed out: Call waited in the queue past client deadline
I20260709 12:12:16.024345 18303 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:16.025023 18303 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:16.026253 18053 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:16.026813 18053 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:16.027918 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2540) took 1424 ms. Trace:
I20260709 12:12:16.028395 12527 rpcz_store.cc:276] 0709 12:12:14.603063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:15.685121 (+1082058us) service_pool.cc:224] Handling call
0709 12:12:15.685524 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:15.786483 (+100959us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:15.786522 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:15.787116 (+   594us) log.cc:844] Serialized 26 byte log entry
0709 12:12:15.848333 (+ 61217us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:15.848375 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:15.848741 (+   366us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:16.027656 (+178915us) raft_consensus.cc:1685] finished
0709 12:12:16.027680 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:16.027850 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":60745,"threads_started":1,"wal-append.queue_time_us":95788}
I20260709 12:12:16.024305 12496 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:15.967617 18090 raft_consensus.cc:697] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:15.969760 18307 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:15.997844 18389 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:15.950294 18384 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:16.053952 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 180698us
I20260709 12:12:16.059791 12606 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
W20260709 12:12:16.061050 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 66099.7us
W20260709 12:12:16.054544 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 167199us
I20260709 12:12:16.054683 18401 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:16.062742 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:16.002957 18250 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:16.066327 18250 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:16.080574 13302 leader_election.cc:336] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.713s (SENT)
I20260709 12:12:16.081300 13302 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:16.065100 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:16.091859 18402 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:12:16.100004 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 113250us
W20260709 12:12:16.061733 12643 leader_election.cc:336] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 10 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.236s (SENT)
I20260709 12:12:16.013814 18313 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:16.058733 18389 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:15.982573 18391 consensus_queue.cc:237] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:16.046144 18393 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.056s
W20260709 12:12:16.134491 13304 leader_election.cc:336] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.705s (SENT)
I20260709 12:12:16.139816 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:16.120278 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 96498.7us
I20260709 12:12:16.143592 13337 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
W20260709 12:12:16.153870 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:16.162679 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:16.110090 18390 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:12:16.165218 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 84703us
I20260709 12:12:16.144345 18384 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:16.172444 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:16.154486 18246 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:16.179807 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 45479.2us
W20260709 12:12:16.181087 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 116135us
I20260709 12:12:16.120186 18032 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:16.185998 12496 leader_election.cc:336] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Remote error: Timed out: Call waited in the queue past client deadline
I20260709 12:12:16.186726 12496 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:16.168059 18390 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:12:16.203743 18396 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:12:16.210695 12627 debug-util.cc:398] Leaking SignalData structure 0x7b08003b6680 after lost signal to thread 11321
W20260709 12:12:16.212389 12627 debug-util.cc:398] Leaking SignalData structure 0x7b0800337860 after lost signal to thread 12463
I20260709 12:12:16.219813 18390 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:16.196189 18053 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:16.223631 18303 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:16.224507 18303 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:16.145794 18090 consensus_queue.cc:237] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:16.236140 18307 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:16.245401 18407 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:12:16.267295 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:16.269546 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 86086.7us
W20260709 12:12:16.272574 12627 debug-util.cc:398] Leaking SignalData structure 0x7b0800360720 after lost signal to thread 12555
W20260709 12:12:16.272943 12627 debug-util.cc:398] Leaking SignalData structure 0x7b0800394060 after lost signal to thread 12556
W20260709 12:12:16.273941 12627 debug-util.cc:398] Leaking SignalData structure 0x7b0800340ea0 after lost signal to thread 12630
W20260709 12:12:16.274251 12627 debug-util.cc:398] Leaking SignalData structure 0x7b08002e57a0 after lost signal to thread 12631
I20260709 12:12:16.290450 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:16.184480 18397 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:16.221837 18390 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:16.296528 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:16.297976 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:16.304582 12529 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:16.259555 18388 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:16.309357 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:16.248430 18393 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:16.254875 18187 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:16.261967 18032 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:16.198899 18401 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:16.256816 18363 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:16.248127 18357 raft_consensus.cc:3055] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:16.306577 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:16.245991 18308 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:16.316890 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:16.315217 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:12:16.307948 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 246230us
I20260709 12:12:16.326552 18032 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:16.326606 18053 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:16.327257 18032 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:16.327286 18053 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:16.329025 18341 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:16.323024 12523 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:16.329586 18341 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:16.330580 18341 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:16.333647 18381 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election lost for term 16. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17.
I20260709 12:12:16.334745 18032 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:16.335276 18053 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:16.335594 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:16.337116 12678 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:12:16.337378 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:16.266655 18389 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 15.13, Last appended by leader: 13, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:16.339310 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:16.339934 12672 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
W20260709 12:12:16.340656 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:16.342475 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:16.343350 12672 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 14.
I20260709 12:12:16.305603 18397 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:16.348439 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:16.348620 18397 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:16.318593 18187 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:16.354279 18187 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:16.354919 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:16.355809 13333 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:16.359865 13304 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:12:16.360381 12643 leader_election.cc:336] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.583s (SENT)
I20260709 12:12:16.360985 12643 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:16.362083 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:16.364593 13333 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:16.368275 18053 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:16.374311 12523 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:16.256851 18326 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:16.318843 18402 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:16.380246 13304 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:16.322672 18357 raft_consensus.cc:740] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:16.382555 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:16.386646 12672 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15.
I20260709 12:12:16.384829 13304 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:16.393416 18401 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:16.398882 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:16.375775 18053 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:16.407400 12674 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 11: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:16.404783 12600 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:16.390179 18397 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:12:16.410456 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:16.401901 18393 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:16.362797 18250 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:16.398240 18411 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:16.413589 18187 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:16.383877 18317 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:16.437336 12672 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:16.412467 18397 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:16.464391 18397 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:16.440521 18357 consensus_queue.cc:260] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:16.441808 12627 debug-util.cc:398] Leaking SignalData structure 0x7b08002bbc80 after lost signal to thread 12704
W20260709 12:12:16.469022 12627 debug-util.cc:398] Leaking SignalData structure 0x7b0800207780 after lost signal to thread 13363
W20260709 12:12:16.469344 12627 debug-util.cc:398] Leaking SignalData structure 0x7b080038c8c0 after lost signal to thread 13364
I20260709 12:12:16.443902 18317 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:16.446031 18308 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:16.435729 18187 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:16.447342 18326 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:16.428074 18393 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:16.431337 18411 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:16.410984 18053 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:16.460335 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:16.479473 13333 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:16.461665 18405 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:12:16.463819 18383 consensus_meta.cc:287] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.504s	user 0.009s	sys 0.003s
W20260709 12:12:16.485494 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 314769us
W20260709 12:12:16.491638 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:16.473680 18187 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:16.477053 18393 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:16.495018 13333 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:16.480047 18411 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:16.502967 18053 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:16.506428 17519 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:16.507102 17519 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:16.508803 12496 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:16.526628 12569 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:16.527575 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:16.528324 13333 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:16.529889 17519 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:12:16.530961 17519 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:16.531412 17519 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:16.526383 17819 consensus_queue.cc:1059] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 11.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:16.532222 17519 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:16.494303 18383 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:16.542448 12643 leader_election.cc:336] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.947s (SENT)
I20260709 12:12:16.546342 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:16.550685 17819 raft_consensus.cc:3055] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:12:16.551220 17819 raft_consensus.cc:740] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:16.552138 17819 consensus_queue.cc:260] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:16.553510 17819 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:16.555899 18250 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:16.568709 18406 raft_consensus.cc:3055] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:16.569237 18406 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:16.570829 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2561) took 2192 ms. Trace:
I20260709 12:12:16.571249 12528 rpcz_store.cc:276] 0709 12:12:14.378786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:15.095252 (+716466us) service_pool.cc:224] Handling call
0709 12:12:15.686246 (+590994us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:15.751603 (+ 65357us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:16.570572 (+818969us) spinlock_profiling.cc:243] Waited 590 ms on lock 0x7b54003cafd0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:16.570737 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1358071552}
I20260709 12:12:16.572011 18383 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:16.537119 18317 raft_consensus.cc:2764] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:12:16.573230 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2565) took 2470 ms. Trace:
I20260709 12:12:16.573628 12680 rpcz_store.cc:276] 0709 12:12:14.102787 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:14.722922 (+620135us) service_pool.cc:224] Handling call
0709 12:12:14.755444 (+ 32522us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:15.121837 (+366393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:16.572960 (+1451123us) spinlock_profiling.cc:243] Waited 365 ms on lock 0x7b540008eb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:16.573150 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":839637760}
W20260709 12:12:16.607003 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 64601.3us
W20260709 12:12:16.615687 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 115611us
I20260709 12:12:16.617492 13337 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 11 index: 10. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:12:16.605755 18384 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:16.607503 18392 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
W20260709 12:12:16.634477 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 941497us
I20260709 12:12:16.644450 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:12:16.646569 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:16.656596 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 962643us
I20260709 12:12:16.654515 12572 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:16.622372 18187 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:16.662075 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:16.605383 18406 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:16.632714 18384 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:16.658412 18426 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:12:16.692143 12643 leader_election.cc:336] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.225s (SENT)
I20260709 12:12:16.698553 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:16.686568 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:16.706516 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 44531.7us
I20260709 12:12:16.707876 12572 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:12:16.708472 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 86245.3us
I20260709 12:12:16.702808 18390 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:16.717190 18390 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:16.721041 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:16.722539 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 57341.1us
W20260709 12:12:16.722833 18421 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:12:16.723459 12571 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:16.717406 18426 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:16.725683 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2831) took 1450 ms. Trace:
I20260709 12:12:16.686604 18429 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260709 12:12:16.726626 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:16.721802 18390 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:16.710680 18290 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:16.733110 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2828) took 1461 ms. Trace:
I20260709 12:12:16.726068 12597 rpcz_store.cc:276] 0709 12:12:15.275352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:15.275476 (+   124us) service_pool.cc:224] Handling call
0709 12:12:15.275841 (+   365us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:16.495840 (+1219999us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:16.495880 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:16.498609 (+  2729us) log.cc:844] Serialized 26 byte log entry
0709 12:12:16.641217 (+142608us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:16.641284 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:16.641384 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:16.725429 (+ 84045us) raft_consensus.cc:1685] finished
0709 12:12:16.725446 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:16.725616 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1217028,"raft.queue_time_us":1232491,"spinlock_wait_cycles":29566848,"thread_start_us":103875,"threads_started":1,"wal-append.queue_time_us":198579}
I20260709 12:12:16.730366 18290 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:16.733717 12602 rpcz_store.cc:276] 0709 12:12:15.271485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:15.310835 (+ 39350us) service_pool.cc:224] Handling call
0709 12:12:15.311334 (+   499us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:16.608147 (+1296813us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:16.608199 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:16.620844 (+ 12645us) log.cc:844] Serialized 52 byte log entry
0709 12:12:16.709748 (+ 88904us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:16.709784 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:16.709866 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:16.732756 (+ 22890us) raft_consensus.cc:1685] finished
0709 12:12:16.732790 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:16.733021 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1184369,"raft.queue_time_us":1200308,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":14745,"thread_start_us":176388,"threads_started":2,"wal-append.queue_time_us":96043}
I20260709 12:12:16.714406 18392 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:16.711723 18428 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:16.729507 18429 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:16.696645 18397 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:16.746644 18430 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:16.749037 18402 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:12:16.751120 18422 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.225s	user 0.005s	sys 0.000s
I20260709 12:12:16.753962 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:16.752923 12572 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:16.729967 18413 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:16.692998 18393 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:16.760349 17519 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:16.760588 12572 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:16.751787 18426 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:16.791095 18426 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:16.767795 18044 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:16.744038 18053 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:16.759027 18402 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:16.771071 12424 catalog_manager.cc:5697] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 9 to 10, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 10 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:12:16.773952 17519 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:16.806473 17519 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:16.807164 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:16.807816 12531 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:16.807878 12527 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:12:16.808665 18433 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.050s	user 0.000s	sys 0.001s
I20260709 12:12:16.774940 12643 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:16.809058 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:16.777012 18364 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:16.810356 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:16.772231 18140 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:12:16.811169 12524 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:16.779953 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2568) took 2675 ms. Trace:
I20260709 12:12:16.812382 12673 rpcz_store.cc:276] 0709 12:12:14.104215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:14.765712 (+661497us) service_pool.cc:224] Handling call
0709 12:12:14.775395 (+  9683us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:15.845377 (+1069982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:16.779683 (+934306us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b540015e0d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:16.779888 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2459197696}
I20260709 12:12:16.780527 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 635) took 1547 ms. Trace:
I20260709 12:12:16.781158 18397 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:16.782742 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2577) took 1545 ms. Trace:
W20260709 12:12:16.788031 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 44697.6us
I20260709 12:12:16.788714 18220 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:16.757900 18429 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:12:16.790295 17582 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:16.790633 18384 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:16.796227 12646 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:16.798099 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:16.807369 18423 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.281s	user 0.000s	sys 0.004s
I20260709 12:12:16.810420 18392 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:12:16.757994 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 95285.5us
I20260709 12:12:16.778628 18187 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:16.814302 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:16.958880 13333 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 10.
I20260709 12:12:16.822700 12531 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:16.791517 18426 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:16.831771 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:16.966418 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:16.932209 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:16.939239 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 133048us
I20260709 12:12:16.813481 12526 rpcz_store.cc:276] 0709 12:12:15.233610 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:15.691740 (+458130us) service_pool.cc:224] Handling call
0709 12:12:15.692155 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:15.716555 (+ 24400us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:15.716589 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:15.721253 (+  4664us) log.cc:844] Serialized 26 byte log entry
0709 12:12:16.479001 (+757748us) spinlock_profiling.cc:243] Waited 9.82 ms on lock 0x7b180049b220. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:12:16.485092 (+  6091us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:16.485137 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:16.485208 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:16.743410 (+258202us) spinlock_profiling.cc:243] Waited 9.31 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:16.780223 (+ 36813us) raft_consensus.cc:1685] finished
0709 12:12:16.780255 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:16.780431 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":24065,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"spinlock_wait_cycles":43991168,"thread_start_us":5863,"threads_started":1,"wal-append.queue_time_us":763817}
I20260709 12:12:16.969388 13332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38482 (request call id 578) took 1840 ms. Trace:
I20260709 12:12:16.941303 12571 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:16.934587 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:16.947490 12524 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:12:16.948787 12529 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:12:16.978092 12528 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:16.817906 12530 rpcz_store.cc:276] 0709 12:12:15.239797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:15.708765 (+468968us) service_pool.cc:224] Handling call
0709 12:12:15.982787 (+274022us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:15.986458 (+  3671us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:15.986520 (+    62us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:16.000919 (+ 14399us) log.cc:844] Serialized 26 byte log entry
0709 12:12:16.317116 (+316197us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:16.317164 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:16.317263 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:16.356670 (+ 39407us) raft_consensus.cc:1685] finished
0709 12:12:16.356699 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:16.782411 (+425712us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b540038f4d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:16.782675 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3223,"raft.queue_time_us":41056,"raft.run_cpu_time_us":50,"raft.run_wall_time_us":50,"spinlock_wait_cycles":629297536,"thread_start_us":256780,"threads_started":1,"wal-append.queue_time_us":278915}
I20260709 12:12:16.952603 17519 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:16.939276 18220 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:16.962787 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:16.987432 12531 raft_consensus.cc:2393] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 15: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:16.981449 12496 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:16.951395 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 155271us
I20260709 12:12:16.989403 12646 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:16.984272 12571 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:16.954984 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:16.991609 12527 raft_consensus.cc:2393] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 15: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:16.805406 18053 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:16.961424 18384 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:17.000713 17519 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:16.975466 13332 rpcz_store.cc:276] 0709 12:12:15.129204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:15.131447 (+  2243us) service_pool.cc:224] Handling call
0709 12:12:15.131843 (+   396us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:15.709446 (+577603us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:12:15.769975 (+ 60529us) log.cc:844] Serialized 14 byte log entry
0709 12:12:15.957443 (+187468us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:15.957517 (+    74us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:15.957558 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:16.755514 (+797956us) spinlock_profiling.cc:243] Waited 9.56 ms on lock 0x7b1800346e40. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419
0709 12:12:16.968787 (+213273us) spinlock_profiling.cc:243] Waited 577 ms on lock 0x7b540061f654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:16.968853 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:16.969036 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":68969,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":135990,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":14603,"spinlock_wait_cycles":1349787904,"thread_start_us":239010,"threads_started":3,"wal-append.queue_time_us":64121,"wal-append.run_cpu_time_us":1736,"wal-append.run_wall_time_us":1097928}
I20260709 12:12:17.005095 12522 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:16.775787 18413 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:17.011866 12646 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:17.012869 17519 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:17.016633 17519 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:17.017357 17519 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:17.021248 12646 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:16.994060 18220 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:17.023908 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:17.031041 12526 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:16.988121 17582 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:16.957441 18044 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:12:17.006433 18384 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:17.007886 18386 raft_consensus.cc:2764] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election decision vote started in defunct term 11: won
W20260709 12:12:17.046518 13302 leader_election.cc:336] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.094s (SENT)
I20260709 12:12:17.063136 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:17.056435 18250 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:17.027638 18383 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:17.008769 18392 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:17.007678 18308 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.055s
W20260709 12:12:17.075817 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 296374us
W20260709 12:12:17.078244 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 96906.9us
I20260709 12:12:17.079419 12496 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:17.063582 12526 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:12:17.023367 18090 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:12:17.080295 17519 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:17.064692 18384 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:17.083933 12531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "10395d6e75b84c3982521323f92e3309"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:17.072750 18383 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:17.084383 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:17.088810 12531 raft_consensus.cc:606] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:17.095534 18413 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:17.099246 18044 consensus_queue.cc:1059] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:12:17.101943 18424 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.571s	user 0.001s	sys 0.001s
I20260709 12:12:17.075786 18397 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:17.073071 18246 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.032s
I20260709 12:12:17.105379 18393 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:17.076524 18250 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:17.114187 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:17.087311 17819 consensus_meta.cc:287] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.533s	user 0.015s	sys 0.000s
I20260709 12:12:17.114531 17519 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:17.115926 18443 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:17.107894 18383 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:17.124249 12496 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:17.121129 17819 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.006s
W20260709 12:12:17.127224 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:17.127384 18053 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:17.130985 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 341181us
I20260709 12:12:17.133021 12496 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:17.118456 18389 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:17.118530 17582 raft_consensus.cc:3055] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:17.140255 12496 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:17.167012 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:17.134415 18386 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:17.169152 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:17.139050 18413 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:17.128460 18402 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:17.174603 12496 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:17.128367 18250 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:17.138412 17582 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:17.180401 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 91308.1us
I20260709 12:12:17.188097 12496 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:17.183569 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 147783us
W20260709 12:12:17.197345 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 92277.8us
I20260709 12:12:17.177213 18389 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:17.179872 18313 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:17.215265 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:17.217324 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 83865.5us
I20260709 12:12:17.189357 18391 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:17.196888 18392 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:17.203192 18386 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:12:17.215629 18427 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.683s	user 0.000s	sys 0.015s
I20260709 12:12:17.243033 12528 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:12:17.208630 18389 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:17.125010 12424 catalog_manager.cc:5697] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:17.214850 18313 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:17.249101 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:17.220643 18391 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:17.237219 18411 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.702s	user 0.015s	sys 0.000s
W20260709 12:12:17.247339 18427 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.714s	user 0.000s	sys 0.015s
I20260709 12:12:17.266845 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:17.270210 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 71112.7us
I20260709 12:12:17.254989 18389 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:17.271916 12496 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:17.260891 18313 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:17.235152 18221 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.155s
I20260709 12:12:17.232491 18428 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260709 12:12:17.267217 18411 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:17.268206 18439 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:12:17.242703 18140 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:12:17.261072 18391 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:17.246781 18441 consensus_queue.cc:1059] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260709 12:12:17.271574 18112 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:17.253945 18220 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:17.283533 17819 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:17.256661 18446 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:12:17.340552 18053 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:17.341555 18053 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:17.342955 18053 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:17.345714 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:17.347906 18406 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:17.363420 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:17.364346 12529 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 17: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:17.297880 18112 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:17.365355 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:17.366896 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:17.367794 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2592) took 1095 ms. Trace:
I20260709 12:12:17.374979 12529 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 16: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:17.374871 12525 rpcz_store.cc:276] 0709 12:12:16.272126 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:16.272376 (+   250us) service_pool.cc:224] Handling call
0709 12:12:16.307590 (+ 35214us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:16.378875 (+ 71285us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:12:16.396785 (+ 17910us) log.cc:844] Serialized 14 byte log entry
0709 12:12:16.406354 (+  9569us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:16.406435 (+    81us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:16.406493 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:17.367480 (+960987us) spinlock_profiling.cc:243] Waited 70.9 ms on lock 0x7b54003f8c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:17.367545 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:17.367716 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3029,"prepare.queue_time_us":17891,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":11634,"raft.run_cpu_time_us":115,"raft.run_wall_time_us":5006,"spinlock_wait_cycles":163159808,"thread_start_us":13421,"threads_started":1}
I20260709 12:12:17.261741 17582 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:17.372529 18053 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:17.406410 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:17.398293 18440 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.078s	user 0.000s	sys 0.001s
I20260709 12:12:17.409590 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:17.288650 18448 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.094s
I20260709 12:12:17.405946 17582 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:17.402793 12597 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:32832
I20260709 12:12:17.296886 18392 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:17.419925 18411 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:17.357844 18444 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:17.377038 18112 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:17.422674 18444 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:17.425120 18444 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:12:17.372305 17819 raft_consensus.cc:697] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:17.426972 17819 consensus_queue.cc:237] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:17.427417 18443 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:12:17.430545 18402 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:17.431818 18449 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:12:17.432340 13333 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:17.432861 13332 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:17.418535 12597 raft_consensus.cc:606] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:17.435253 12597 raft_consensus.cc:614] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:17.441562 18407 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:17.446475 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:17.446696 12494 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:17.478861 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2586) took 1547 ms. Trace:
I20260709 12:12:17.479499 12522 rpcz_store.cc:276] 0709 12:12:15.931447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:15.931577 (+   130us) service_pool.cc:224] Handling call
0709 12:12:15.932023 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:17.005868 (+1073845us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:17.478491 (+472623us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b5400054454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:17.478770 (+   279us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2467105536}
I20260709 12:12:17.481158 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:17.481980 12522 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:17.490794 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:17.494828 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:17.499864 12522 raft_consensus.cc:2410] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 15 because replica has last-logged OpId of term: 14 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 13.
W20260709 12:12:17.502313 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 55702.3us
I20260709 12:12:17.510881 13333 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:12:17.517681 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:17.519876 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 706740us
I20260709 12:12:17.520906 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:17.526646 18446 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:17.527145 18220 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:17.528033 18220 raft_consensus.cc:3055] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:17.528419 18220 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:17.528589 18290 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:17.378639 12424 catalog_manager.cc:5697] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:17.529678 18290 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 12:12:17.532336 18290 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:17.532101 12424 catalog_manager.cc:5697] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 10 to 12. New cstate: current_term: 12 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:17.476043 18140 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260709 12:12:17.534533 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:17.535454 12529 raft_consensus.cc:2410] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 18 because replica has last-logged OpId of term: 16 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:12:17.540634 18444 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:17.541278 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2827) took 2271 ms. Trace:
I20260709 12:12:17.541931 18446 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:17.542279 18199 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:17.543183 18236 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:17.543651 18236 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:17.543370 18446 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 12, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:17.544471 18199 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:17.545059 18199 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:17.545742 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:17.547324 12522 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:17.548548 12522 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:17.551818 13333 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:12:17.553442 18467 raft_consensus.cc:3055] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:17.549925 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:17.539328 18313 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:17.577877 18461 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260709 12:12:17.578624 18461 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:17.579077 18461 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:17.580777 18389 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:17.582717 18392 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:17.583492 18392 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:17.584946 18413 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:17.588487 18392 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:17.575289 18469 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.014s
I20260709 12:12:17.588467 18112 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:17.550411 12424 catalog_manager.cc:5697] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 7 to 9. New cstate: current_term: 9 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:12:17.592505 18112 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:17.529412 18220 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:17.598871 18220 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:17.541729 12598 rpcz_store.cc:276] 0709 12:12:15.269995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:15.270141 (+   146us) service_pool.cc:224] Handling call
0709 12:12:15.270501 (+   360us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:16.492735 (+1222234us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:16.492766 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:16.497618 (+  4852us) log.cc:844] Serialized 26 byte log entry
0709 12:12:17.101318 (+603700us) spinlock_profiling.cc:243] Waited 6.61 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:17.540755 (+439437us) spinlock_profiling.cc:243] Waited 1.74 ms on lock 0x7b18003aae00. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1
0709 12:12:17.540920 (+   165us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:17.540958 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:17.541040 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:17.541085 (+    45us) raft_consensus.cc:1685] finished
0709 12:12:17.541101 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:17.541219 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1221802,"raft.queue_time_us":1237624,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":23121,"spinlock_wait_cycles":19207808,"thread_start_us":7025,"threads_started":1,"wal-append.queue_time_us":26589}
I20260709 12:12:17.553915 18467 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:17.602840 12600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 2825) took 2336 ms (client timeout 1600 ms). Trace:
W20260709 12:12:17.607908 12600 rpcz_store.cc:269] 0709 12:12:15.266738 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:15.266880 (+   142us) service_pool.cc:224] Handling call
0709 12:12:17.602628 (+2335748us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b54002ffb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:17.602765 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1453786880}
I20260709 12:12:17.567410 18402 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:17.544615 18236 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:17.593146 12424 catalog_manager.cc:5697] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 8 to 9, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 9 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:12:17.630692 12527 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
W20260709 12:12:17.630853 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:17.631115 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 356859us
I20260709 12:12:17.634254 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:17.638449 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 351306us
I20260709 12:12:17.606779 18053 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:17.657166 12646 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:17.668161 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:17.681615 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49146.1us
I20260709 12:12:17.682579 12496 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:17.682679 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:17.683719 12600 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 17: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:17.686172 12602 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 14: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:17.691730 12597 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:12:17.630925 18402 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:17.668907 18384 consensus_meta.cc:287] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.556s	user 0.008s	sys 0.004s
I20260709 12:12:17.702440 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:17.703642 12523 raft_consensus.cc:2393] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 15: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:12:17.708714 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:17.694397 18392 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:12:17.713403 18413 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:17.713776 18467 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:17.714926 12597 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:12:17.715317 18461 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:17.716617 18313 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:17.717439 18477 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:17.718358 17519 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:12:17.724417 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:17.740204 12602 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:17.699888 18402 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:17.738493 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:17.741916 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:17.742864 12672 raft_consensus.cc:2410] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
W20260709 12:12:17.711283 18429 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.750s	user 0.013s	sys 0.000s
I20260709 12:12:17.739252 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:17.687659 18053 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:17.709157 18236 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:17.746515 12673 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: []
W20260709 12:12:17.750638 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 79425.6us
I20260709 12:12:17.748525 12523 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:17.750734 17519 raft_consensus.cc:3055] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:17.752696 17519 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:17.746297 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 9 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:17.690476 18391 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:17.752396 12523 raft_consensus.cc:606] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:17.755034 12678 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 12. Ops: []
I20260709 12:12:17.753393 12674 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:12:17.757174 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:17.759956 12494 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:17.751565 12646 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:17.775893 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:17.707628 18384 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:17.780578 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:17.791718 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:17.792235 12597 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 13: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:17.784552 17519 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:17.757148 18426 consensus_meta.cc:287] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.777s	user 0.015s	sys 0.000s
I20260709 12:12:17.729656 18326 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:12:17.752408 18250 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:17.727865 18090 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.051s
I20260709 12:12:17.797103 12531 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 15: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:17.775010 18467 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:17.800227 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:17.806824 12602 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 17: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:17.806738 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:17.786319 18475 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:17.748188 18468 consensus_queue.cc:1059] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.066s
I20260709 12:12:17.806945 12597 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:17.824429 12523 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:17.749948 18429 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:17.818964 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:17.828732 12680 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:17.827975 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:17.799718 18389 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:17.830699 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:17.797832 18313 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:17.834658 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:17.747133 18477 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:17.843631 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:17.814608 18090 consensus_queue.cc:1243] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 17.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:12:17.831089 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:17.848089 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 82142.4us
I20260709 12:12:17.845643 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:17.847014 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:17.852584 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:17.801414 18426 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:17.854799 12606 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:17.855008 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:17.838099 18475 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:17.863547 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:17.864562 12673 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:17.805735 18250 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:17.843487 18468 consensus_queue.cc:1243] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 12.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:12:17.883777 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:17.869350 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:17.882632 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 122738us
I20260709 12:12:17.891986 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:17.832970 18413 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:12:17.845057 18392 raft_consensus.cc:697] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:17.856596 18090 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:17.896546 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:17.900818 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:17.865402 18480 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:17.906193 12680 raft_consensus.cc:2410] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:12:17.803609 12424 catalog_manager.cc:5697] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 2 to 12. New cstate: current_term: 12 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:12:17.906738 12606 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:17.837523 18476 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.064s
I20260709 12:12:17.909440 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:17.911818 12599 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 11.
I20260709 12:12:17.849542 18313 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:17.906898 18199 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:17.916589 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:17.917133 18477 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:17.933635 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 251142us
I20260709 12:12:17.889204 18475 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:17.894902 18481 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:17.941534 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:17.902904 18090 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:17.949927 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:17.950598 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:17.904523 18411 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:17.956089 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:17.968806 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:17.904495 18480 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:18.007902 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:18.007820 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.013604 12599 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:18.011489 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:18.017316 12601 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:12:18.019898 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.032545 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:17.941519 18466 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 12:12:18.038781 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:17.939064 18479 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:12:18.045691 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 152524us
I20260709 12:12:18.049952 12680 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:18.051657 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:18.052064 12603 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:18.053745 12604 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:12:18.055495 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:17.960666 18090 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:18.064234 12679 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:12:18.073554 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:18.074630 12529 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:18.071822 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:18.060307 18053 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:18.003901 18481 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:18.066021 18391 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:18.096486 18483 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.197s	user 0.000s	sys 0.001s
W20260709 12:12:18.098824 18474 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.216s	user 0.000s	sys 0.001s
I20260709 12:12:18.098349 18480 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:18.099948 18480 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:18.100462 18487 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:12:18.078812 18384 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:18.102021 18426 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:18.102545 18489 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:18.103287 18489 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:18.105700 18383 consensus_meta.cc:287] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.967s	user 0.011s	sys 0.000s
I20260709 12:12:18.075925 18479 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:18.067750 18461 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:18.086784 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:18.075809 18246 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:18.064507 18488 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260709 12:12:18.125372 13333 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:18.094383 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:18.095348 18250 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:18.102604 18429 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:18.019486 18411 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:18.136747 18489 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:18.138331 18411 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:18.142252 18246 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:18.145134 18246 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:18.147801 18481 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:18.122934 18407 consensus_meta.cc:287] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.679s	user 0.005s	sys 0.006s
I20260709 12:12:18.149159 18407 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:18.090564 12523 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:18.150898 12523 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:18.151499 12523 raft_consensus.cc:606] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:18.151432 18491 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:18.152047 12523 raft_consensus.cc:614] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:12:18.079385 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 480640us
I20260709 12:12:18.130964 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:18.108907 18475 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:18.132164 18313 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:18.132612 18473 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.317s	user 0.000s	sys 0.001s
W20260709 12:12:18.134188 18478 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
I20260709 12:12:18.135274 18479 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:18.166383 13333 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:12:18.178747 12526 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:18.168679 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:18.174466 12496 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:18.180704 12674 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:18.176182 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:18.180620 12529 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:18.182760 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:18.183418 12675 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:18.188377 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:18.190107 12643 leader_election.cc:336] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.573s (SENT)
I20260709 12:12:18.155550 18199 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:18.194716 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:18.193351 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:18.205735 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:18.206925 12528 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 14: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:18.207760 12602 raft_consensus.cc:2410] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 15 because replica has last-logged OpId of term: 14 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 12.
I20260709 12:12:18.207907 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:18.208796 12496 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:18.210314 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:18.210574 18492 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:18.210863 12677 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21. Ops: []
I20260709 12:12:18.211017 12671 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:18.211342 18492 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:12:18.211920 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:18.164544 18411 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:18.220283 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:18.078524 18392 consensus_queue.cc:237] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:18.221397 12675 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:18.110421 18383 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:18.216667 12597 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:12:18.233027 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.234314 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.175034 18489 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:18.191740 18220 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.583s	user 0.008s	sys 0.004s
I20260709 12:12:18.235080 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:18.243669 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:18.245316 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:18.247102 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:18.225406 12529 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 14 index: 14. Preceding OpId from leader: term: 15 index: 14. (term mismatch)
I20260709 12:12:18.248437 12529 pending_rounds.cc:85] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 13
W20260709 12:12:18.250267 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 60474.9us
I20260709 12:12:18.257953 12529 pending_rounds.cc:107] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 14.14
I20260709 12:12:18.259457 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:18.259707 12606 raft_consensus.cc:2410] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:12:18.258674 12671 raft_consensus.cc:2368] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14.
I20260709 12:12:18.259817 12529 raft_consensus.cc:2889] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:18.260365 12675 raft_consensus.cc:2410] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
W20260709 12:12:18.259100 13302 leader_election.cc:336] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.307s (SENT)
I20260709 12:12:18.262729 13302 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:18.264197 18220 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:18.257522 12598 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:18.264818 18220 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:18.266583 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:18.266896 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:18.268603 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:18.267637 12525 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:18.270943 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:18.274338 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:18.289403 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:18.289465 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:18.257247 18481 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:18.296598 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:18.294000 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.295915 12671 raft_consensus.cc:2393] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:18.297786 12598 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
W20260709 12:12:18.290737 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 116889us
I20260709 12:12:18.299243 12600 raft_consensus.cc:2393] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 14: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:18.091930 12424 catalog_manager.cc:5697] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 10 to 11, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 11 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:12:18.232539 18411 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:18.307264 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:18.312378 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:18.318604 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:18.319701 12671 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:18.320757 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:18.321697 12671 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 17: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:18.323372 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:18.324180 12672 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:12:18.325095 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:18.327560 18383 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:18.333777 18140 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:18.334635 18140 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:18.335050 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:18.336391 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.342998 12606 raft_consensus.cc:2393] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 10: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:18.274030 18499 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:18.338025 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:18.338873 12424 catalog_manager.cc:5697] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 6 to 12. New cstate: current_term: 12 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:18.340446 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:18.273459 18044 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
I20260709 12:12:18.359259 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:18.345255 18500 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.119s	user 0.000s	sys 0.001s
I20260709 12:12:18.353428 12598 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 15: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:18.257481 18489 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:18.337420 18140 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:18.363160 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:18.371222 12524 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:18.352010 12674 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:18.373150 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:18.362531 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:18.367055 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:18.386013 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 12 candidate_status { last_received { term: 10 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:18.387035 12671 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:18.388226 12674 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:18.394027 12525 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b65110412db1492aa47f4a845ec89242"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:18.403700 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:18.409705 12525 raft_consensus.cc:606] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:18.410494 12525 raft_consensus.cc:614] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:18.415611 12524 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:18.417251 12672 raft_consensus.cc:2393] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 14: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:12:18.375451 18450 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.962s	user 0.006s	sys 0.000s
W20260709 12:12:18.419667 18497 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:12:18.440310 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:18.420784 12424 catalog_manager.cc:5697] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 6 to 9, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 9 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:12:18.462745 12529 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:18.463913 12677 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:18.425546 18489 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:18.426862 18411 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:18.442852 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.466141 12598 raft_consensus.cc:2368] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17.
I20260709 12:12:18.445034 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.467417 12604 raft_consensus.cc:2410] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:12:18.446528 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2606) took 1073 ms. Trace:
I20260709 12:12:18.358268 18499 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:18.415206 18467 consensus_meta.cc:287] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.606s	user 0.008s	sys 0.004s
I20260709 12:12:18.440189 18427 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:18.455406 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2541) took 1151 ms. Trace:
I20260709 12:12:18.476894 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.478458 12526 rpcz_store.cc:276] 0709 12:12:17.303742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:17.517361 (+213619us) service_pool.cc:224] Handling call
0709 12:12:18.455217 (+937856us) spinlock_profiling.cc:243] Waited 631 ms on lock 0x7b54003f6e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:18.455336 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1452053248}
I20260709 12:12:18.479342 12598 raft_consensus.cc:2410] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 14 because replica has last-logged OpId of term: 13 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 7.
I20260709 12:12:18.459054 18458 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:18.464896 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:18.467902 12522 rpcz_store.cc:276] 0709 12:12:17.373364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:17.741967 (+368603us) service_pool.cc:224] Handling call
0709 12:12:17.742298 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:17.859777 (+117479us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:17.859811 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:17.859855 (+    44us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:17.859887 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:18.446218 (+586331us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b540038f9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:18.446279 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:18.446449 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":269716736}
I20260709 12:12:18.481983 12605 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:12:18.482689 12677 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:18.480302 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:18.459859 18044 consensus_queue.cc:1059] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:12:18.483990 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:18.472695 12525 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:12:18.473152 12524 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:18.468259 12674 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
W20260709 12:12:18.449952 18402 consensus_meta.cc:287] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.704s	user 0.010s	sys 0.001s
I20260709 12:12:18.469861 12675 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:18.488363 12529 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:18.488826 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:18.488984 12529 raft_consensus.cc:606] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:18.489356 12529 raft_consensus.cc:614] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:18.491823 18499 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:18.492583 18044 raft_consensus.cc:3055] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:12:18.484617 18427 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:18.492975 18044 raft_consensus.cc:740] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:18.478598 18505 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:18.493867 18044 consensus_queue.cc:260] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:18.494583 18505 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:18.495178 18044 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:18.498893 18469 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:18.499431 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:18.499482 18469 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:18.501088 12672 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:18.503407 12672 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:18.505807 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:18.506703 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:18.512447 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.512596 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:18.512867 18505 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:18.513773 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:18.514820 18430 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:18.515039 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.515987 12600 raft_consensus.cc:2410] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:12:18.520418 12672 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 12:12:18.485857 18463 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.012s	sys 0.004s
I20260709 12:12:18.516763 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.526494 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:18.527053 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.528097 12604 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:12:18.528889 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:12:18.526856 12600 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:18.530427 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.531121 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:18.532016 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:12:18.532963 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:18.533149 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:12:18.533608 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:18.482930 18509 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:18.534540 12571 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:18.538875 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:18.519451 18467 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:12:18.527699 18463 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.013s	sys 0.004s
I20260709 12:12:18.540840 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.537329 12571 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:18.549213 12496 leader_election.cc:400] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14.
I20260709 12:12:18.552789 12496 leader_election.cc:403] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Cancelling election due to peer responding with higher term
I20260709 12:12:18.516165 18427 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:18.553324 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.555533 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.556895 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:18.558734 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.559520 12600 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
I20260709 12:12:18.560154 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:18.560643 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:18.583693 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:18.593595 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2602) took 1399 ms. Trace:
I20260709 12:12:18.594233 12531 rpcz_store.cc:276] 0709 12:12:17.194418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:17.194637 (+   219us) service_pool.cc:224] Handling call
0709 12:12:18.593246 (+1398609us) spinlock_profiling.cc:243] Waited 581 ms on lock 0x7b54003caad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:18.593483 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1336041344}
I20260709 12:12:18.596273 12523 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:12:18.642994 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:18.528630 18221 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 12:12:18.644562 12599 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 13: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:18.537881 17819 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:18.646438 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 10 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:18.502907 18140 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
W20260709 12:12:18.676905 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 139646us
I20260709 12:12:18.547848 18505 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:18.648792 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:18.651841 12600 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 15: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:18.684026 12599 raft_consensus.cc:2376] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12. Re-sending same reply.
I20260709 12:12:18.684778 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.541743 18199 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:18.535488 18389 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:18.517127 18485 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.035s
I20260709 12:12:18.508667 18402 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:18.644495 18467 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:18.646366 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:18.729310 12677 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:18.522684 18090 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:18.517390 18452 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.042s
I20260709 12:12:18.685496 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.696651 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:18.695822 18465 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:18.723402 12600 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:18.678076 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:18.725502 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 186674us
I20260709 12:12:18.696305 18428 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:18.740922 12677 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:18.759281 12679 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 18: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:18.737618 13333 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:18.760246 12496 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:18.768375 12672 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
W20260709 12:12:18.734869 18451 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.309s	user 0.030s	sys 0.035s
I20260709 12:12:18.772187 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.776098 12496 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:18.741390 18508 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:18.778461 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.764781 18427 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:18.759598 18428 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:18.780963 18427 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:18.783847 18427 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:18.784521 18504 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.302s	user 0.001s	sys 0.000s
I20260709 12:12:18.779171 18452 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:18.758991 18465 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:18.739800 18090 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:12:18.788149 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:12:18.778263 18451 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.349s	user 0.030s	sys 0.036s
I20260709 12:12:18.800405 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.800786 12671 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:12:18.801559 12599 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:12:18.802954 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:18.803673 12599 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:18.808740 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:18.813642 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 135659us
I20260709 12:12:18.818037 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:18.800930 18140 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:18.802783 18465 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:18.822997 18140 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:18.815311 18426 consensus_queue.cc:1059] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:12:18.838485 18140 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:18.838907 18426 raft_consensus.cc:3055] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:18.839644 18426 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:18.840919 18426 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:18.844751 12599 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:12:18.805626 18428 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:18.849143 18428 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:18.826483 18467 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:18.790656 18508 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:18.815925 18485 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:18.832024 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2589) took 1107 ms. Trace:
I20260709 12:12:18.851100 12678 rpcz_store.cc:276] 0709 12:12:17.724918 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:17.725055 (+   137us) service_pool.cc:224] Handling call
0709 12:12:17.729422 (+  4367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:17.757958 (+ 28536us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:18.831712 (+1073754us) spinlock_profiling.cc:243] Waited 25.4 ms on lock 0x7b54000924d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:18.831937 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":58319744}
I20260709 12:12:18.853274 18508 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:18.854923 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:18.845989 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:18.834312 18220 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:18.818071 18516 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:12:18.823362 18479 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:18.836824 12604 raft_consensus.cc:2410] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
W20260709 12:12:18.837194 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49171.8us
I20260709 12:12:18.849968 18451 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:18.788697 12424 catalog_manager.cc:5697] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 6 to 11, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 11 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:18.869833 12496 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:18.871253 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2545) took 1565 ms. Trace:
I20260709 12:12:18.871827 12527 rpcz_store.cc:276] 0709 12:12:17.306160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:17.537800 (+231640us) service_pool.cc:224] Handling call
0709 12:12:17.538067 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:17.634420 (+ 96353us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:18.870901 (+1236481us) spinlock_profiling.cc:243] Waited 78.9 ms on lock 0x7b540034fb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:18.871164 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":181503360}
I20260709 12:12:18.874027 12571 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:18.875977 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:18.876227 12496 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:18.879002 12496 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:18.882246 12496 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:18.887022 18430 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:12:18.887912 17819 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:18.888360 17819 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:18.888772 17819 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:18.898759 13302 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.125s (SENT) [suppressed 30 similar messages]
I20260709 12:12:18.874707 18426 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:18.873019 18044 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:18.883750 18488 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:18.884174 18485 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:18.886344 18341 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:18.910626 12571 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:18.895576 18221 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:12:18.890515 18451 raft_consensus.cc:697] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:18.913677 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:18.914783 18427 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:18.915740 12571 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:18.918511 12571 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:18.924908 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:18.901213 18402 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:18.931869 12606 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:18.894793 18220 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.026s
I20260709 12:12:18.927007 18525 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:18.927452 18526 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:18.927452 18427 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:18.925853 18455 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:18.942298 18455 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:18.943550 18455 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:18.944792 18455 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:18.901965 18430 consensus_queue.cc:1243] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:12:18.948709 12424 catalog_manager.cc:5697] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:18.951644 12424 catalog_manager.cc:5697] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 6 to 10. New cstate: current_term: 10 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:18.930291 18467 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:18.938395 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.931767 18488 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:18.929098 12571 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:18.950044 18452 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:12:18.958627 18471 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.273s	user 0.008s	sys 0.011s
I20260709 12:12:18.954363 18442 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:18.931402 18341 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:18.961560 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.964480 12571 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:18.974330 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 92236.7us
I20260709 12:12:18.975744 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.976985 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.977584 12606 raft_consensus.cc:2376] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12. Re-sending same reply.
I20260709 12:12:18.978932 18430 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:18.970525 18467 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:18.965987 18442 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:18.981267 12674 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:18.987722 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:18.988519 12674 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:18.989434 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:18.989945 12674 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:12:18.991257 12674 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:12:18.991377 12600 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
I20260709 12:12:18.993122 12600 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:18.997830 12606 raft_consensus.cc:3055] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:12:18.998370 12606 raft_consensus.cc:740] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:18.998342 12600 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:18.999328 12600 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:19.005254 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:18.978498 18220 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260709 12:12:19.011796 18303 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:19.011446 18442 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:19.012684 18442 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:19.012794 18523 consensus_queue.cc:1059] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:12:19.013520 18524 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:12:19.013229 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:19.015913 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:19.018361 18451 consensus_queue.cc:237] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:18.995870 18452 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.034s
I20260709 12:12:18.994737 12424 catalog_manager.cc:5697] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 3 to 9, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 9 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:12:19.028870 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:19.028120 12424 catalog_manager.cc:5697] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 7 to 11, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 11 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:19.004194 18525 raft_consensus.cc:697] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:18.997097 18467 raft_consensus.cc:3055] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:18.981340 18140 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:19.014318 18521 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:12:19.010043 18044 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:19.039167 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:19.045133 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 805605us
I20260709 12:12:19.052878 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2597) took 1148 ms. Trace:
I20260709 12:12:19.046916 12527 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:19.047937 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:19.054876 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:19.104758 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:19.073190 12673 rpcz_store.cc:276] 0709 12:12:17.904293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:17.905501 (+  1208us) service_pool.cc:224] Handling call
0709 12:12:17.917494 (+ 11993us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:17.917755 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:12:17.924209 (+  6454us) log.cc:844] Serialized 14 byte log entry
0709 12:12:18.331111 (+406902us) spinlock_profiling.cc:243] Waited 24 ms on lock 0x7b180033f1c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:12:19.041945 (+710834us) spinlock_profiling.cc:243] Waited 5.04 ms on lock 0x7b180033f1c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:12:19.050407 (+  8462us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:19.050487 (+    80us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:19.050560 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:19.050645 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:19.052809 (+  2164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27640,"prepare.queue_time_us":6432,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":46,"raft.queue_time_us":822271,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":108,"spinlock_wait_cycles":66853504,"thread_start_us":134854,"threads_started":1,"wal-append.queue_time_us":184261}
I20260709 12:12:19.109601 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 572) took 1206 ms. Trace:
I20260709 12:12:19.114809 12605 rpcz_store.cc:276] 0709 12:12:17.854325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:17.854737 (+   412us) service_pool.cc:224] Handling call
0709 12:12:19.060703 (+1205966us) spinlock_profiling.cc:243] Waited 622 ms on lock 0x7b5400115dd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:19.060874 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1430312832}
W20260709 12:12:19.116331 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 217604us
I20260709 12:12:19.116853 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:12:19.117383 13302 leader_election.cc:336] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.187s (SENT)
I20260709 12:12:19.117596 13339 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:19.117828 12529 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:19.117911 13302 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:19.120128 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:19.122164 13302 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:19.124361 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:19.123554 13302 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:19.109102 12599 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
I20260709 12:12:19.131047 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:19.131943 12529 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:12:19.042219 18524 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:19.042917 18452 raft_consensus.cc:3055] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:19.135340 13305 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:19.136080 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:19.147408 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:19.147954 12524 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:19.108979 18526 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:19.108695 17582 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:19.052567 18407 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:19.050405 18250 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:19.040917 18140 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:19.128680 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:19.150823 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 143492us
I20260709 12:12:19.163661 12601 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:12:19.149387 18509 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:12:19.109185 18383 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260709 12:12:19.188808 12601 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: []
W20260709 12:12:19.166961 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 110626us
I20260709 12:12:19.190564 12524 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:19.150641 18452 raft_consensus.cc:740] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:19.171052 18491 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:19.172394 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:19.148700 18524 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:19.136971 18485 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:19.128794 18489 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:12:19.174933 17582 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:19.037539 18476 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.112s
I20260709 12:12:19.038079 18341 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:19.128623 18505 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:19.182665 12496 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:19.182695 12525 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:19.035871 18467 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:19.051357 18384 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.106s
I20260709 12:12:19.051553 18044 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:19.186609 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 57992us
I20260709 12:12:19.168637 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:19.197184 12524 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:19.205893 13339 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
W20260709 12:12:19.206403 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:19.278162 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:19.278687 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:19.036561 18427 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:19.178660 18407 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:19.282416 18407 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:19.283784 18407 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:19.284888 18199 consensus_queue.cc:1059] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:12:19.286759 18389 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:19.288316 18389 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:12:19.176954 12522 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:19.290597 18527 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:19.192958 18508 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:19.185312 18250 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:19.193972 18221 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:19.297875 18221 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:19.298971 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2601) took 1228 ms. Trace:
I20260709 12:12:19.165607 17819 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:19.299574 12679 rpcz_store.cc:276] 0709 12:12:18.070901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:18.071582 (+   681us) service_pool.cc:224] Handling call
0709 12:12:19.298678 (+1227096us) spinlock_profiling.cc:243] Waited 666 ms on lock 0x7b540016eed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:19.298885 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1531084160}
I20260709 12:12:19.302903 17819 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:19.228721 18455 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:12:19.250310 18090 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:19.250883 18386 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:19.304476 18455 raft_consensus.cc:3055] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:19.305025 18386 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:19.305593 18199 consensus_queue.cc:1243] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 10.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:12:19.251757 18463 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:19.305052 18455 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:19.306533 18463 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:19.307909 18516 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:12:19.252048 12424 catalog_manager.cc:5697] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 9 to 11. New cstate: current_term: 11 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:19.252398 18456 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:19.293793 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2621) took 1477 ms. Trace:
I20260709 12:12:19.309469 12424 catalog_manager.cc:5697] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 4 to 7, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 7 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:12:19.311101 18250 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:19.254252 17582 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:19.312567 17582 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:19.034559 18525 consensus_queue.cc:237] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:19.314198 18112 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:12:19.315127 18523 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Stepping down as leader of term 14
W20260709 12:12:19.271958 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 89386.4us
I20260709 12:12:19.316172 12496 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:19.176697 18443 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
I20260709 12:12:19.307412 18455 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:19.308344 18516 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:19.308943 18456 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:19.229856 18430 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.026s
I20260709 12:12:19.293054 18508 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:19.251112 18308 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:12:19.280966 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:19.318559 12496 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:19.289671 18389 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:12:19.379591 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 221407us
I20260709 12:12:19.310004 12602 rpcz_store.cc:276] 0709 12:12:17.816516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:17.830635 (+ 14119us) service_pool.cc:224] Handling call
0709 12:12:19.293453 (+1462818us) spinlock_profiling.cc:243] Waited 357 ms on lock 0x7b5400374e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:19.293704 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":821825408}
I20260709 12:12:19.171696 18526 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:19.259078 18505 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:19.315603 18523 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:19.495067 12494 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:12:19.494217 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:19.528761 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 468713us
I20260709 12:12:19.237967 18465 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:19.394884 18508 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:19.563261 12604 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:12:19.542076 13302 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:19.529764 18498 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.209s	user 0.001s	sys 0.000s
W20260709 12:12:19.494236 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:19.544685 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:19.569427 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:19.502207 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:19.534319 18455 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:19.571034 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 584) took 1256 ms. Trace:
I20260709 12:12:19.571568 12599 rpcz_store.cc:276] 0709 12:12:18.314249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:18.845700 (+531451us) service_pool.cc:224] Handling call
0709 12:12:19.427161 (+581461us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54001171d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:19.570971 (+143810us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":407732096}
I20260709 12:12:19.536177 18505 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:19.572903 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:19.575649 12602 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:19.320030 18485 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:19.529428 18511 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:19.571965 18308 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.117s
I20260709 12:12:19.573168 18430 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:19.494160 18452 consensus_queue.cc:260] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:19.533605 18536 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:19.774925 18509 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.006s	sys 0.025s
I20260709 12:12:19.833839 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:12:19.834394 12676 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.678s	user 0.009s	sys 0.029s
I20260709 12:12:19.567935 12646 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:19.893219 12676 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.743s	user 0.009s	sys 0.030s
W20260709 12:12:19.893625 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 325751us
W20260709 12:12:19.892411 12645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.719s	user 0.020s	sys 0.029s
I20260709 12:12:19.894591 12646 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:19.838963 13337 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.005s	sys 0.008s
W20260709 12:12:19.611043 18481 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.000s	sys 0.011s
I20260709 12:12:19.842490 13334 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:19.320865 18479 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:12:19.573776 18467 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:19.895730 13337 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.005s	sys 0.009s
W20260709 12:12:19.896826 12645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.021s	sys 0.029s
W20260709 12:12:19.954754 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 636268us
W20260709 12:12:19.834192 18313 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.010s	sys 0.023s
I20260709 12:12:19.388496 18524 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:19.896175 18430 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:19.947635 18308 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:19.948467 18452 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:19.955037 13337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 635) took 1095 ms. Trace:
I20260709 12:12:19.956077 12646 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:19.956409 12496 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:20.024487 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:20.026831 12673 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:12:20.024791 12680 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:19.964655 17519 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:19.892652 18442 consensus_meta.cc:287] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.879s	user 0.011s	sys 0.003s
W20260709 12:12:19.901971 18481 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.850s	user 0.000s	sys 0.020s
W20260709 12:12:19.972628 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:19.971215 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:19.970074 12679 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
W20260709 12:12:19.999061 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:19.868420 12603 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.008s	sys 0.008s
W20260709 12:12:20.004060 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:19.675010 18523 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:19.896723 18463 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:19.897923 18389 consensus_queue.cc:1243] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 14.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
W20260709 12:12:19.891660 18509 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.006s	sys 0.026s
W20260709 12:12:20.028908 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 72612.3us
I20260709 12:12:19.574098 12424 catalog_manager.cc:5697] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 3 to 9, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 9 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:12:20.031757 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:20.031123 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 75115us
I20260709 12:12:19.984542 18477 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:12:19.996737 18313 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.799s	user 0.017s	sys 0.034s
I20260709 12:12:20.025961 13337 rpcz_store.cc:276] 0709 12:12:18.859305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:18.859550 (+   245us) service_pool.cc:224] Handling call
0709 12:12:18.860036 (+   486us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:18.860349 (+   313us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:12:19.130028 (+269679us) log.cc:844] Serialized 14 byte log entry
0709 12:12:19.205385 (+ 75357us) log.cc:844] Serialized 14 byte log entry
0709 12:12:19.954617 (+749232us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:19.954674 (+    57us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:12:19.954729 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:19.954814 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:19.954973 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":558311,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":1047,"raft.queue_time_us":511906,"raft.run_cpu_time_us":303,"raft.run_wall_time_us":77606,"thread_start_us":1044252,"threads_started":3}
W20260709 12:12:20.092344 12603 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.009s	sys 0.009s
W20260709 12:12:20.093065 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:20.091459 12643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.898s	user 0.038s	sys 0.032s
W20260709 12:12:20.266347 12643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.073s	user 0.038s	sys 0.032s
I20260709 12:12:19.993021 18221 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:19.991902 18476 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:20.130716 12597 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:20.015061 18308 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:20.230069 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:20.231093 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:20.359731 18546 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:20.247407 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 592) took 1244 ms. Trace:
I20260709 12:12:20.029874 18544 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:19.995044 18485 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.131s
W20260709 12:12:20.113720 18532 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.793s	user 0.001s	sys 0.000s
I20260709 12:12:20.256572 17519 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.459182 17519 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:20.113565 18246 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:20.080606 18303 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:20.302906 18386 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:20.235702 18509 raft_consensus.cc:697] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:12:20.295439 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 1.24054e+06us
I20260709 12:12:20.467741 12643 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:20.438601 18559 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.265760 18044 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:20.301971 18534 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.765s	user 0.002s	sys 0.000s
I20260709 12:12:20.132481 18524 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:20.266271 18389 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
W20260709 12:12:20.359433 12671 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.019s	sys 0.027s
W20260709 12:12:20.474401 12671 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.021s	sys 0.032s
I20260709 12:12:20.256584 18442 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:20.396234 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 991512us
I20260709 12:12:20.397341 18543 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.068s
W20260709 12:12:20.393841 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:20.342517 18456 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:20.350809 18522 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:12:20.326659 18538 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:12:20.358924 13303 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.017s	sys 0.017s
W20260709 12:12:20.484699 13303 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.811s	user 0.023s	sys 0.018s
W20260709 12:12:20.470263 18455 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.899s	user 0.014s	sys 0.000s
W20260709 12:12:20.239440 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58024.9us
I20260709 12:12:20.485746 12496 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:20.439591 12673 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:20.450640 12604 rpcz_store.cc:276] 0709 12:12:19.003391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:19.047555 (+ 44164us) service_pool.cc:224] Handling call
0709 12:12:20.247065 (+1199510us) spinlock_profiling.cc:243] Waited 485 ms on lock 0x7b54002e59d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:20.247300 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1116087552}
I20260709 12:12:20.292316 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:20.489435 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:20.302470 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 464058us
I20260709 12:12:20.465356 18246 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:20.456105 18476 raft_consensus.cc:697] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:20.464968 18556 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:12:20.483830 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:20.483661 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:20.518383 12604 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:20.489187 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:20.491227 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:20.492938 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:20.493866 13302 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:20.449103 18308 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:20.497377 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:20.498055 18554 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
W20260709 12:12:20.498750 18537 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.266s	user 0.000s	sys 0.001s
I20260709 12:12:20.498982 18496 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:20.501127 12606 consensus_queue.cc:260] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.6, Last appended by leader: 6, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:20.482215 18442 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:20.600659 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 80040.3us
I20260709 12:12:20.506305 12673 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:20.506565 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:20.507843 13332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 736) took 1996 ms. Trace:
I20260709 12:12:20.458798 18546 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:20.487555 18538 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:20.602270 13332 rpcz_store.cc:276] 0709 12:12:18.510995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:18.511211 (+   216us) service_pool.cc:224] Handling call
0709 12:12:18.511518 (+   307us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:18.562192 (+ 50674us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:12:18.779645 (+217453us) log.cc:844] Serialized 14 byte log entry
0709 12:12:18.820731 (+ 41086us) log.cc:844] Serialized 14 byte log entry
0709 12:12:18.823335 (+  2604us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:18.823408 (+    73us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:18.823478 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:20.505672 (+1682194us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b18001f3000. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240
0709 12:12:20.507092 (+  1420us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b18001f3000. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419
0709 12:12:20.507584 (+   492us) spinlock_profiling.cc:243] Waited 50.4 ms on lock 0x7b54000d7854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:20.507616 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:20.507772 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2318,"prepare.queue_time_us":271965,"raft.queue_time_us":2662,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":222,"spinlock_wait_cycles":208807168,"thread_start_us":137542,"threads_started":2,"wal-append.queue_time_us":129332,"wal-append.run_cpu_time_us":2128,"wal-append.run_wall_time_us":1041101}
I20260709 12:12:20.508977 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2960) took 2233 ms. Trace:
I20260709 12:12:20.509114 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:20.467916 18544 raft_consensus.cc:697] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:20.606002 12677 rpcz_store.cc:276] 0709 12:12:18.275580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:18.775578 (+499998us) service_pool.cc:224] Handling call
0709 12:12:19.053465 (+277887us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:19.185756 (+132291us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:19.185795 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:19.185862 (+    67us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:19.185915 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:19.186005 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:20.508795 (+1322790us) spinlock_profiling.cc:243] Waited 277 ms on lock 0x7b54003be050. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:20.508911 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":132033,"raft.queue_time_us":152661,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":20478,"spinlock_wait_cycles":637509888}
I20260709 12:12:20.517728 18246 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:20.519804 12524 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:20.518376 18430 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:20.655570 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2854) took 2422 ms. Trace:
I20260709 12:12:20.524856 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:20.525274 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:20.596563 18455 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.551704 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:20.524785 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:20.553517 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:20.470556 18540 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:20.336928 12424 catalog_manager.cc:5697] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 8 to 11, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 11 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:12:20.599586 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:20.476114 18559 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:20.607848 12606 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
W20260709 12:12:20.609740 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:20.614012 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:20.671482 12673 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 18: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:20.614244 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:20.672227 12677 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:20.483323 17519 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:20.604827 18538 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:20.617353 12526 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:20.521569 18303 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:20.618963 18565 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:12:20.619773 18549 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.597239 18489 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:20.596534 18475 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:20.649565 17582 consensus_meta.cc:287] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.337s	user 0.019s	sys 0.003s
I20260709 12:12:20.531184 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2860) took 2007 ms. Trace:
I20260709 12:12:20.600471 18429 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.597486 18496 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:20.660923 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2560) took 2493 ms. Trace:
W20260709 12:12:20.663511 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 138336us
I20260709 12:12:20.665304 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:20.656215 12603 rpcz_store.cc:276] 0709 12:12:18.233195 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:19.296616 (+1063421us) service_pool.cc:224] Handling call
0709 12:12:19.297123 (+   507us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:19.297672 (+   549us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:12:19.297716 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:19.315341 (+ 17625us) log.cc:844] Serialized 26 byte log entry
0709 12:12:20.265613 (+950272us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:20.265655 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:20.265726 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:20.655243 (+389517us) raft_consensus.cc:1685] finished
0709 12:12:20.655269 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:20.655484 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6189,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"thread_start_us":949863,"threads_started":1,"wal-append.queue_time_us":1304723}
I20260709 12:12:20.732053 12528 rpcz_store.cc:276] 0709 12:12:18.167957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:18.209136 (+ 41179us) service_pool.cc:224] Handling call
0709 12:12:18.209503 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:18.209748 (+   245us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:18.209778 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:18.959581 (+749803us) spinlock_profiling.cc:243] Waited 4.65 ms on lock 0x7b2c00058028. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:12:18.960078 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 12:12:19.167573 (+207495us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:19.167638 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:19.167874 (+   236us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:20.652539 (+1484665us) raft_consensus.cc:1685] finished
0709 12:12:20.652561 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:20.660849 (+  8288us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10703104,"thread_start_us":204694,"threads_started":1,"wal-append.queue_time_us":315271}
I20260709 12:12:20.605024 18543 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.599260 18389 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.103s
I20260709 12:12:20.612931 18456 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:20.674227 13336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 739) took 1622 ms. Trace:
I20260709 12:12:20.738412 13336 rpcz_store.cc:276] 0709 12:12:19.052157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:19.052753 (+   596us) service_pool.cc:224] Handling call
0709 12:12:19.120378 (+ 67625us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:19.120840 (+   462us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:19.120885 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:19.146043 (+ 25158us) log.cc:844] Serialized 26 byte log entry
0709 12:12:19.493656 (+347613us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:12:19.493710 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:19.493796 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:20.673908 (+1180112us) raft_consensus.cc:1685] finished
0709 12:12:20.673939 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:20.674146 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":730,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":22070,"thread_start_us":347082,"threads_started":1,"wal-append.queue_time_us":752338}
I20260709 12:12:20.701876 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:20.695400 17519 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.622227 18509 consensus_queue.cc:237] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:20.715382 12606 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 12 index: 7. (index mismatch)
I20260709 12:12:20.669196 18384 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:20.718187 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2963) took 2345 ms. Trace:
I20260709 12:12:20.720000 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2856) took 2216 ms. Trace:
I20260709 12:12:20.670547 18570 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:20.673446 18463 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:20.731675 12604 rpcz_store.cc:276] 0709 12:12:18.523893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:20.517719 (+1993826us) service_pool.cc:224] Handling call
0709 12:12:20.518199 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:20.530887 (+ 12688us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:20.531113 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:20.746737 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2647) took 2648 ms. Trace:
I20260709 12:12:20.756983 12604 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:20.731689 17582 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.552503 18556 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:20.741356 17519 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:20.761338 12524 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:20.744791 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:20.746258 12678 rpcz_store.cc:276] 0709 12:12:18.372636 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:18.852288 (+479652us) service_pool.cc:224] Handling call
0709 12:12:18.852667 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:19.170893 (+318226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:19.170923 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:19.187899 (+ 16976us) log.cc:844] Serialized 26 byte log entry
0709 12:12:19.573594 (+385695us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:19.573656 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:19.573761 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:20.717890 (+1144129us) raft_consensus.cc:1685] finished
0709 12:12:20.717912 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:20.718083 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":317791,"raft.queue_time_us":332429,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":4469,"thread_start_us":367458,"threads_started":1,"wal-append.queue_time_us":1343169}
I20260709 12:12:20.766747 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:20.669400 18455 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:20.746428 12602 rpcz_store.cc:276] 0709 12:12:18.503986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:19.401725 (+897739us) service_pool.cc:224] Handling call
0709 12:12:19.494685 (+ 92960us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:19.891077 (+396392us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:20.719723 (+828646us) spinlock_profiling.cc:243] Waited 80.7 ms on lock 0x7b540007a654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:20.719938 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":185613184}
I20260709 12:12:20.718947 18489 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:20.772116 12603 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:20.792397 12673 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:20.804081 17519 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:20.806404 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2861) took 2282 ms. Trace:
I20260709 12:12:20.808101 12603 rpcz_store.cc:276] 0709 12:12:18.524143 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:20.752640 (+2228497us) service_pool.cc:224] Handling call
0709 12:12:20.767862 (+ 15222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:20.786648 (+ 18786us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:20.806306 (+ 19658us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:20.718875 18303 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:20.717232 18386 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:20.713049 18402 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.848943 18386 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:20.849516 18402 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:20.757990 17582 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:20.851190 18386 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:20.851835 17582 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:20.768916 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:20.768908 12602 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:20.852759 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2864) took 2323 ms. Trace:
I20260709 12:12:20.853080 12602 rpcz_store.cc:276] 0709 12:12:18.529515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:20.768461 (+2238946us) service_pool.cc:224] Handling call
0709 12:12:20.768716 (+   255us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:20.852483 (+ 83767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:20.852704 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:20.768816 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2862) took 2244 ms. Trace:
I20260709 12:12:20.742761 18540 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:20.755302 18541 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.485s	user 0.000s	sys 0.001s
I20260709 12:12:20.736651 18561 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.818714 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2859) took 2295 ms. Trace:
I20260709 12:12:20.758736 12606 rpcz_store.cc:276] 0709 12:12:18.098419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:18.990429 (+892010us) service_pool.cc:224] Handling call
0709 12:12:18.990712 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:20.500691 (+1509979us) spinlock_profiling.cc:243] Waited 75 ms on lock 0x7b5400142ba0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f148899e06f 00007f1488a16f60 00007f1488a187de 00007f1488a22b49 00007f1488a23416 00007f1488a1fd8f 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62
0709 12:12:20.746170 (+245479us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:20.746623 (+   453us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":172517760}
W20260709 12:12:20.757040 18448 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 12 current term: 12 is higher.
I20260709 12:12:20.844384 12603 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:20.861739 12598 rpcz_store.cc:276] 0709 12:12:18.523625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:20.316105 (+1792480us) service_pool.cc:224] Handling call
0709 12:12:20.316498 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:20.316748 (+   250us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:12:20.481893 (+165145us) log.cc:844] Serialized 14 byte log entry
0709 12:12:20.818316 (+336423us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:20.818378 (+    62us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:20.818435 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:20.818506 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:20.818648 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38379,"prepare.queue_time_us":287000,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":63,"raft.queue_time_us":122532,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":113,"thread_start_us":435325,"threads_started":3,"wal-append.queue_time_us":295422}
I20260709 12:12:20.885212 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2865) took 1315 ms. Trace:
I20260709 12:12:20.885699 12603 rpcz_store.cc:276] 0709 12:12:19.570071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:20.809308 (+1239237us) service_pool.cc:224] Handling call
0709 12:12:20.844204 (+ 34896us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:20.884888 (+ 40684us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:20.885152 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:20.887646 12603 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 13. Preceding OpId from leader: term: 16 index: 14. (index mismatch)
I20260709 12:12:20.888568 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2868) took 1317 ms. Trace:
I20260709 12:12:20.731601 18549 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:20.743323 18429 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:20.752007 18481 consensus_queue.cc:1059] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 9.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.060s
W20260709 12:12:20.846308 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:20.744966 18475 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:20.813256 18565 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.050s	user 0.001s	sys 0.000s
W20260709 12:12:20.960712 18568 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.177s	user 0.000s	sys 0.000s
W20260709 12:12:20.961488 18569 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.288s	user 0.000s	sys 0.001s
W20260709 12:12:20.962332 18574 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:12:20.763916 18544 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:20.963663 18575 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:12:20.694052 18476 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:20.854583 12604 rpcz_store.cc:276] 0709 12:12:18.524432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:20.756403 (+2231971us) service_pool.cc:224] Handling call
0709 12:12:20.756753 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:20.762282 (+  5529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:20.768737 (+  6455us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:20.755585 18456 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:20.854197 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:20.888931 12603 rpcz_store.cc:276] 0709 12:12:19.570888 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:20.886749 (+1315861us) service_pool.cc:224] Handling call
0709 12:12:20.887238 (+   489us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:20.888243 (+  1005us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:20.888496 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:20.786386 18430 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.901530 12606 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 12:12:21.067435 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 300744us
I20260709 12:12:21.067978 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2866) took 1497 ms. Trace:
I20260709 12:12:21.068410 12606 rpcz_store.cc:276] 0709 12:12:19.570409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:20.868816 (+1298407us) service_pool.cc:224] Handling call
0709 12:12:20.869279 (+   463us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:21.067652 (+198373us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:21.067913 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:20.812896 18570 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:20.812294 18563 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader election won for term 12
I20260709 12:12:20.945415 12529 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 9 index: 5. Preceding OpId from leader: term: 12 index: 6. (index mismatch)
W20260709 12:12:20.857127 18491 consensus_meta.cc:287] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.648s	user 0.008s	sys 0.006s
I20260709 12:12:20.771297 18463 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:20.952704 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:21.088568 12598 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:20.818622 18489 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:20.745041 18543 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:20.957610 18553 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.308s	user 0.000s	sys 0.001s
I20260709 12:12:20.958273 18578 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:20.958384 18573 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:20.959439 18542 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:20.959810 18548 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.243s	user 0.000s	sys 0.000s
W20260709 12:12:20.960417 18560 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.300s	user 0.000s	sys 0.001s
I20260709 12:12:20.944414 18561 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:20.822898 17519 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:12:20.974819 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 229067us
I20260709 12:12:20.992789 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2619) took 1025 ms. Trace:
I20260709 12:12:21.115833 17519 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:20.884867 18540 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:21.048928 12602 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:21.050869 13339 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 16: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:21.069447 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:21.097052 18563 raft_consensus.cc:697] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:21.075696 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 314807us
W20260709 12:12:21.081218 18575 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.114s	user 0.000s	sys 0.002s
I20260709 12:12:21.229065 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:21.086513 18341 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:21.230028 12602 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:12:21.230376 18341 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:21.199519 18563 consensus_queue.cc:237] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.7, Last appended by leader: 7, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:21.208251 18544 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.023s
I20260709 12:12:21.233078 18341 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:21.234747 18590 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:21.211112 18303 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:21.089705 18491 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election lost for term 13. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14.
I20260709 12:12:21.235476 18590 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:21.088768 18389 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:21.116454 17519 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:12:21.117550 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 452319us
I20260709 12:12:21.116022 12672 rpcz_store.cc:276] 0709 12:12:19.967739 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:19.992664 (+ 24925us) service_pool.cc:224] Handling call
0709 12:12:19.993291 (+   627us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:12:20.023821 (+ 30530us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:20.023867 (+    46us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:12:20.079281 (+ 55414us) log.cc:844] Serialized 78 byte log entry
0709 12:12:20.660555 (+581274us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:20.772494 (+111939us) log.cc:844] Serialized 14 byte log entry
0709 12:12:20.817549 (+ 45055us) log.cc:844] Serialized 14 byte log entry
0709 12:12:20.843561 (+ 26012us) log.cc:844] Serialized 14 byte log entry
0709 12:12:20.991875 (+148314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:20.992024 (+   149us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:20.992285 (+   261us) raft_consensus.cc:1685] finished
0709 12:12:20.992327 (+    42us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:20.992654 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63452,"prepare.queue_time_us":420406,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":597343,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":12769,"spinlock_wait_cycles":6069504,"thread_start_us":649950,"threads_started":3,"wal-append.queue_time_us":631783}
I20260709 12:12:21.070883 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:21.130780 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 61424.7us
I20260709 12:12:21.083573 18430 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:21.102545 18546 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:21.134994 12646 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:21.119669 18442 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:21.184103 18508 consensus_meta.cc:287] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.605s	user 0.013s	sys 0.001s
I20260709 12:12:21.118328 18542 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:21.136783 18578 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:21.265954 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:21.267282 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:21.264227 18590 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:21.167299 18576 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.280s	user 0.000s	sys 0.000s
I20260709 12:12:21.268764 18590 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:12:21.269430 18578 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:21.154434 18567 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:21.179145 18559 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:21.158550 18572 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.366s	user 0.001s	sys 0.000s
I20260709 12:12:21.192523 18455 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:21.170898 18573 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:12:21.194276 18564 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:21.170550 18556 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:21.199896 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2961) took 2900 ms. Trace:
I20260709 12:12:21.282673 12671 rpcz_store.cc:276] 0709 12:12:18.299271 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:18.802056 (+502785us) service_pool.cc:224] Handling call
0709 12:12:18.802374 (+   318us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:19.306695 (+504321us) spinlock_profiling.cc:243] Waited 4.95 ms on lock 0x7b4800060858. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:12:19.566020 (+259325us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:19.566056 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:19.566623 (+   567us) log.cc:844] Serialized 26 byte log entry
0709 12:12:20.474822 (+908199us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:20.474871 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:20.474968 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:21.199533 (+724565us) raft_consensus.cc:1685] finished
0709 12:12:21.199556 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:21.199800 (+   244us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":376808,"raft.queue_time_us":372994,"raft.run_cpu_time_us":2170,"raft.run_wall_time_us":125294,"spinlock_wait_cycles":11374848,"thread_start_us":1096093,"threads_started":2,"wal-append.queue_time_us":870593}
I20260709 12:12:21.285290 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:21.162621 18456 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:21.193367 18549 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:21.159936 18543 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:21.172075 18463 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:21.168514 18475 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:21.197834 18524 consensus_meta.cc:287] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.718s	user 0.007s	sys 0.003s
I20260709 12:12:21.198855 18429 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:21.158012 18540 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:21.207338 18505 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.593s	user 0.008s	sys 0.008s
W20260709 12:12:21.003778 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:21.195070 18536 consensus_meta.cc:287] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.295s	user 0.010s	sys 0.003s
W20260709 12:12:21.202445 18452 consensus_meta.cc:287] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.171s	user 0.010s	sys 0.004s
I20260709 12:12:21.267728 18542 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:21.191126 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2569) took 2185 ms. Trace:
I20260709 12:12:21.268010 12672 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 17: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:21.265791 18508 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:21.271144 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:21.271562 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:21.351692 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:21.272373 12604 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
W20260709 12:12:21.352348 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 80959.9us
W20260709 12:12:21.273422 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 138567us
I20260709 12:12:21.353149 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:21.351737 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 80665.1us
W20260709 12:12:21.205461 18246 consensus_meta.cc:287] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.550s	user 0.009s	sys 0.002s
W20260709 12:12:21.217551 18479 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.273s	user 0.014s	sys 0.000s
W20260709 12:12:21.359123 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 59780.9us
I20260709 12:12:21.278637 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:21.278395 17519 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:21.359932 12598 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
W20260709 12:12:21.280278 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:21.286092 12671 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:12:21.287736 18567 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:21.290556 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:21.290823 18430 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:21.291517 18588 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:21.292239 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:21.369576 13336 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
W20260709 12:12:21.321203 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:21.133505 18469 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:21.339627 18505 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:21.340588 18536 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:12:21.347080 18308 consensus_meta.cc:287] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.721s	user 0.012s	sys 0.000s
I20260709 12:12:21.280884 18389 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:21.349325 12523 rpcz_store.cc:276] 0709 12:12:19.005735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:19.005862 (+   127us) service_pool.cc:224] Handling call
0709 12:12:19.011680 (+  5818us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:19.293109 (+281429us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:12:19.293144 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:19.293599 (+   455us) log.cc:844] Serialized 26 byte log entry
0709 12:12:19.788879 (+495280us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:19.788940 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:20.251288 (+462348us) spinlock_profiling.cc:243] Waited 281 ms on lock 0x7b54003cb9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:20.251358 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:21.189885 (+938527us) raft_consensus.cc:1685] finished
0709 12:12:21.189907 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:21.191047 (+  1140us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":202,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"spinlock_wait_cycles":646304000,"thread_start_us":494788,"threads_started":1,"wal-append.queue_time_us":1048674}
I20260709 12:12:21.277650 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:21.355052 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:21.361354 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:21.378976 12602 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:21.379755 12604 raft_consensus.cc:2410] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 13 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:12:21.365243 12643 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:21.371321 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:21.380409 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:21.369617 17519 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:21.361120 18386 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:21.360710 18429 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:21.360517 18540 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:21.379540 12571 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:21.362463 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:21.364056 18479 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260709 12:12:21.336866 18564 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:21.390954 17519 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:21.366770 18573 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:21.384543 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:21.420279 17519 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:21.427759 18389 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:21.384747 18386 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:21.429701 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:21.391577 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:21.424273 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:21.430430 12525 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15.
I20260709 12:12:21.424477 13337 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:21.364269 18246 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:21.434203 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46018.1us
I20260709 12:12:21.435997 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:21.436165 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:21.441352 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:21.442462 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:21.442644 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:21.443209 12523 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:21.383829 18469 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:21.443730 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:21.444052 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:21.444600 12523 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:21.442051 12598 raft_consensus.cc:2376] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13. Re-sending same reply.
I20260709 12:12:21.419257 18481 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:21.419799 18536 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:21.419486 18588 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:21.381134 18308 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:21.449723 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:21.447240 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 62764.4us
I20260709 12:12:21.452225 17519 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:21.453640 13302 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:21.456902 12643 leader_election.cc:400] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17.
I20260709 12:12:21.434643 18429 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:21.459370 12672 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:12:21.458381 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:21.458660 12643 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:21.454066 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:21.466604 13302 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:21.464816 18489 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:21.446427 18479 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 14.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.023s
I20260709 12:12:21.459136 18509 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:21.489722 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:21.495172 12494 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:21.498093 12494 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:21.470278 18573 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:21.457039 18469 raft_consensus.cc:697] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:21.507158 18581 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:21.507757 18581 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:21.508198 18581 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:21.490684 18549 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:21.457150 18524 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:12:21.499734 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:21.509703 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2869) took 1938 ms. Trace:
I20260709 12:12:21.516224 12603 rpcz_store.cc:276] 0709 12:12:19.571130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:21.076329 (+1505199us) service_pool.cc:224] Handling call
0709 12:12:21.076676 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:21.079443 (+  2767us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:21.079478 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:21.080020 (+   542us) log.cc:844] Serialized 26 byte log entry
0709 12:12:21.360785 (+280765us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:21.360821 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:21.360886 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:21.509454 (+148568us) raft_consensus.cc:1685] finished
0709 12:12:21.509473 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:21.509639 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2320,"raft.queue_time_us":5401,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":97,"thread_start_us":251557,"threads_started":1,"wal-append.queue_time_us":363172}
I20260709 12:12:21.459172 18475 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.039s
I20260709 12:12:21.459278 18536 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:21.521337 12494 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:21.489342 18563 consensus_queue.cc:1059] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.029s
I20260709 12:12:21.509575 18479 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:21.489138 18303 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:12:21.530570 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:21.531414 18444 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.453s	user 0.009s	sys 0.005s
W20260709 12:12:21.531967 18444 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.500s	user 0.009s	sys 0.006s
I20260709 12:12:21.531656 12603 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:21.488938 18456 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.027s
I20260709 12:12:21.544047 18585 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:21.495486 18341 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:12:21.550323 12494 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:21.553150 18429 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:21.520720 18549 raft_consensus.cc:697] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 13 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:21.556651 12494 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:21.566810 18246 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:21.546062 18538 consensus_meta.cc:287] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.843s	user 0.013s	sys 0.000s
I20260709 12:12:21.582885 18308 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:21.578023 18593 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:21.578763 18586 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:12:21.580232 18600 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
W20260709 12:12:21.581238 18601 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.122s	user 0.001s	sys 0.000s
I20260709 12:12:21.577626 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:21.540102 18479 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:21.544680 18585 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:21.575574 18563 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:21.582602 18538 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:21.636065 13334 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:21.632337 18456 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:21.605651 18593 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:21.646557 18593 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:21.648656 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:21.649531 12603 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:21.639541 18479 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:21.632433 18586 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:21.545675 18588 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:21.641858 18563 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:21.642577 18585 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:21.662637 12603 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
I20260709 12:12:21.647928 18456 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:21.664954 13302 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:21.661026 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 202455us
I20260709 12:12:21.666854 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:21.669425 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2624) took 1697 ms. Trace:
I20260709 12:12:21.658080 18567 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:21.669816 12671 rpcz_store.cc:276] 0709 12:12:19.972135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:21.367172 (+1395037us) service_pool.cc:224] Handling call
0709 12:12:21.367464 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:21.367686 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:12:21.636300 (+268614us) log.cc:844] Serialized 14 byte log entry
0709 12:12:21.669049 (+ 32749us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:21.669104 (+    55us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:21.669153 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:21.669229 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:21.669352 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":188250,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":1887,"raft.queue_time_us":220,"thread_start_us":285731,"threads_started":2}
I20260709 12:12:21.661904 18563 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:21.661854 18549 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 12.12, Last appended by leader: 12, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:21.660197 18586 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:21.632359 18469 consensus_queue.cc:237] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:21.675148 12643 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:21.665539 18456 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:21.679898 12671 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: [11.7-11.7]
W20260709 12:12:21.708020 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 151449us
I20260709 12:12:21.709514 12494 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:21.710882 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:21.711743 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2625) took 1739 ms. Trace:
I20260709 12:12:21.712150 12671 rpcz_store.cc:276] 0709 12:12:19.972523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:21.678991 (+1706468us) service_pool.cc:224] Handling call
0709 12:12:21.679556 (+   565us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:21.708330 (+ 28774us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:21.711686 (+  3356us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:21.670478 18544 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:12:21.678656 18606 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:12:21.714179 12643 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:21.671404 18538 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:21.671805 18595 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:21.716599 12671 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:21.715619 18524 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:21.673274 18429 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:21.723886 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2626) took 1751 ms. Trace:
I20260709 12:12:21.726816 12671 rpcz_store.cc:276] 0709 12:12:19.972852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:21.712918 (+1740066us) service_pool.cc:224] Handling call
0709 12:12:21.716416 (+  3498us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:21.723611 (+  7195us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:21.723826 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:21.716041 18308 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:21.721487 12600 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:21.728792 12671 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:21.718963 18446 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:21.729492 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2627) took 1756 ms. Trace:
I20260709 12:12:21.731914 12600 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:21.733173 12671 rpcz_store.cc:276] 0709 12:12:19.973101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:21.728310 (+1755209us) service_pool.cc:224] Handling call
0709 12:12:21.728624 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:21.729257 (+   633us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:21.729441 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:21.739006 18386 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:21.724510 18524 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:21.726415 18429 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:21.743022 18389 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:21.743386 18341 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:21.761751 18341 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:21.764715 18341 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:21.766271 18613 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:12:21.740002 13339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 745) took 1260 ms. Trace:
I20260709 12:12:21.767956 13339 rpcz_store.cc:276] 0709 12:12:20.480053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:20.480277 (+   224us) service_pool.cc:224] Handling call
0709 12:12:21.739752 (+1259475us) spinlock_profiling.cc:243] Waited 444 ms on lock 0x7b54006419d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:21.739926 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1020537728}
I20260709 12:12:21.750351 12599 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:21.746405 12671 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:12:21.747823 12643 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:21.727506 18595 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 17 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:21.732838 18446 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:21.725008 18544 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:21.770977 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:21.771755 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2628) took 1798 ms. Trace:
I20260709 12:12:21.731925 18617 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:21.775193 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2629) took 1028 ms. Trace:
I20260709 12:12:21.777801 18617 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:21.777611 12671 rpcz_store.cc:276] 0709 12:12:19.973422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:21.734012 (+1760590us) service_pool.cc:224] Handling call
0709 12:12:21.746036 (+ 12024us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:21.771463 (+ 25427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:21.771700 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:21.781348 18617 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:21.765309 18341 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:21.744776 18536 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:21.781365 12671 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 12. Ops: []
I20260709 12:12:21.778074 12528 rpcz_store.cc:276] 0709 12:12:20.747293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:21.337245 (+589952us) service_pool.cc:224] Handling call
0709 12:12:21.365871 (+ 28626us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:21.375775 (+  9904us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:21.375808 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:21.376325 (+   517us) log.cc:844] Serialized 26 byte log entry
0709 12:12:21.448119 (+ 71794us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:21.448165 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:21.448274 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:21.774923 (+326649us) raft_consensus.cc:1685] finished
0709 12:12:21.774977 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:21.775128 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1918,"raft.queue_time_us":9726,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":4820,"thread_start_us":18568,"threads_started":1,"wal-append.queue_time_us":128442}
I20260709 12:12:21.755977 18524 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:21.782582 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:21.808095 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2629) took 1834 ms. Trace:
I20260709 12:12:21.808550 12671 rpcz_store.cc:276] 0709 12:12:19.973778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:21.780266 (+1806488us) service_pool.cc:224] Handling call
0709 12:12:21.781012 (+   746us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:21.807783 (+ 26771us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:21.808024 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:21.812754 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:21.774511 18308 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:21.814617 12671 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:21.782410 18446 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:21.816287 13334 raft_consensus.cc:2368] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18.
I20260709 12:12:21.816533 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2632) took 1321 ms. Trace:
I20260709 12:12:21.817786 12671 rpcz_store.cc:276] 0709 12:12:20.495361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:21.809409 (+1314048us) service_pool.cc:224] Handling call
0709 12:12:21.810039 (+   630us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:21.816212 (+  6173us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:21.816452 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:21.773903 18595 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:21.804870 18544 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:21.845796 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:21.848076 12600 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:21.848722 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:21.848953 17582 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:21.849577 17582 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:21.851198 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2661) took 3710 ms. Trace:
I20260709 12:12:21.775024 18389 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:21.855156 12671 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 21: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:21.855150 12494 leader_election.cc:400] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18.
I20260709 12:12:21.858930 12494 leader_election.cc:403] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:21.852363 12600 rpcz_store.cc:276] 0709 12:12:18.140714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:18.994159 (+853445us) service_pool.cc:224] Handling call
0709 12:12:18.994594 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:21.721123 (+2726529us) spinlock_profiling.cc:243] Waited 904 us on lock 0x7b4c004ec380. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1486ceca20 00007f1486cfda29 00007f1486cc3644 00007f1486ceaf2d 00007f1486ceab93 00007f1488a1b18b 00007f1488a1bcb5 00007f1488a16f30 00007f1488a187de 00007f1488a22b49 00007f1488a23416
0709 12:12:21.848746 (+127623us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:21.851124 (+  2378us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2078464}
I20260709 12:12:21.810838 18246 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260709 12:12:21.860913 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:21.853196 18459 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 12:12:21.808410 18341 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:21.861930 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:21.865520 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:21.864595 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:21.854763 18586 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:21.851127 18544 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:21.930492 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:21.903744 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:21.930492 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:21.907482 18618 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 12:12:21.877143 18456 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:21.939718 18456 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:21.864615 18386 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:12:21.940956 18593 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:21.870023 12671 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 12:12:21.946959 12671 pending_rounds.cc:85] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 8
W20260709 12:12:21.872813 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:21.890339 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:21.950780 12671 pending_rounds.cc:107] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:12:21.951237 12671 raft_consensus.cc:2889] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:21.970064 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2633) took 1474 ms. Trace:
I20260709 12:12:21.970593 12671 rpcz_store.cc:276] 0709 12:12:20.495678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:21.867136 (+1371458us) service_pool.cc:224] Handling call
0709 12:12:21.867430 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:21.957601 (+ 90171us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:21.969976 (+ 12375us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:21.971728 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:21.929198 18429 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:21.974337 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:21.975801 12671 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:21.909562 18564 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:21.976006 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:21.977931 18389 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:21.976894 12603 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:21.997392 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:22.008685 18456 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:22.012370 18593 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:22.016270 18611 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.449s	user 0.003s	sys 0.000s
I20260709 12:12:22.018564 18505 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:22.021157 18443 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:22.021710 18443 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:22.023267 18547 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.657s	user 0.004s	sys 0.018s
I20260709 12:12:22.024986 18446 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:22.028041 18586 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:22.029160 18496 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:22.030750 18496 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:22.031042 18443 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:22.032313 18443 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:22.033128 18511 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:22.041553 12671 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
W20260709 12:12:22.050593 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:22.050685 12643 connection.cc:256] RPC call timeout handler was delayed by -4.0996488839998619s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
I20260709 12:12:22.050836 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:22.054078 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.054585 13333 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:22.057403 12673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2637) took 1447 ms (client timeout 1741 ms). Trace:
W20260709 12:12:22.058025 12673 rpcz_store.cc:269] 0709 12:12:20.610497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:20.701510 (+ 91013us) service_pool.cc:224] Handling call
0709 12:12:22.057197 (+1355687us) spinlock_profiling.cc:243] Waited 51.1 ms on lock 0x7b54003bcc54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:22.057316 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":117498112}
I20260709 12:12:22.059967 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.009744 18456 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:22.060441 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.060672 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:22.060786 12673 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 18: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:12:22.072540 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 85088us
I20260709 12:12:22.036960 18505 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:22.082859 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:22.086513 13332 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:22.067135 12603 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:22.042649 18511 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:22.091749 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:22.093374 13336 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:12:22.071839 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2853) took 3843 ms. Trace:
I20260709 12:12:22.096382 12494 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:22.089037 13339 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:22.055516 18246 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:22.099385 18246 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:22.071195 18250 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:22.106714 18250 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:22.081472 18615 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.029s
I20260709 12:12:22.098510 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:22.112121 12603 raft_consensus.cc:2393] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 12: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:22.111949 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.113269 12671 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 18: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:22.090405 18505 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:22.117089 18505 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:22.113300 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:22.122202 13333 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:22.124250 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:22.088340 18509 consensus_meta.cc:287] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.590s	user 0.010s	sys 0.002s
W20260709 12:12:22.135277 18628 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:12:22.131311 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:22.140439 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:22.087051 18593 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:22.143775 12643 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:22.145572 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:22.147774 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:22.148699 13336 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 15: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:22.126312 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:22.151134 12671 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:22.152742 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:22.153919 13333 raft_consensus.cc:2393] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 12: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:22.095146 12601 rpcz_store.cc:276] 0709 12:12:18.228828 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:19.190006 (+961178us) service_pool.cc:224] Handling call
0709 12:12:19.190400 (+   394us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:19.209846 (+ 19446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:19.209895 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:19.231076 (+ 21181us) log.cc:844] Serialized 52 byte log entry
0709 12:12:19.567564 (+336488us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:20.079061 (+511497us) log.cc:844] Serialized 14 byte log entry
0709 12:12:20.079523 (+   462us) log.cc:844] Serialized 14 byte log entry
0709 12:12:20.087992 (+  8469us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:20.088125 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:22.022473 (+1934348us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:22.071429 (+ 48956us) raft_consensus.cc:1685] finished
0709 12:12:22.071474 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:22.071750 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26937,"prepare.queue_time_us":1522971,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":103,"raft.queue_time_us":62592,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":123827,"spinlock_wait_cycles":54318848,"thread_start_us":804454,"threads_started":2,"wal-append.queue_time_us":1091212}
I20260709 12:12:22.155938 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:22.159994 12603 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:22.157328 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:22.160235 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:22.162071 12643 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:22.145396 18509 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:12:22.153725 18588 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:22.177416 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:12:22.164971 18581 consensus_meta.cc:287] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.656s	user 0.007s	sys 0.007s
I20260709 12:12:22.179940 13334 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:22.179337 13336 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:22.156898 18341 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:22.191193 12603 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:22.165815 18584 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:22.149511 18593 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:22.230824 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:22.190518 18581 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:22.196197 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:22.238076 12671 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15.
I20260709 12:12:22.235790 12601 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:22.229408 18236 raft_consensus.cc:3055] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Stepping down as leader of term 10
I20260709 12:12:22.178649 18528 raft_consensus.cc:3055] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:12:22.250337 18528 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:22.251341 18528 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:22.248378 18236 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:22.253101 18561 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:22.252844 18236 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:22.253934 18236 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 12
I20260709 12:12:22.256692 18465 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:22.259388 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.260262 12673 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 21: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:12:22.260951 18585 consensus_meta.cc:287] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.592s	user 0.000s	sys 0.009s
I20260709 12:12:22.261461 18585 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:22.225855 18633 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:22.263152 18633 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:22.263883 18585 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:22.222867 18446 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:22.247905 12603 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:22.268694 18528 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:22.269639 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:22.264206 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:22.249248 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:22.261219 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 11 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.229092 18489 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:22.200923 18341 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:22.291927 18456 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:22.292902 18516 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:22.293471 18581 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:22.294807 18516 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:22.295858 18593 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:22.303311 18625 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:12:22.243983 18584 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:22.301879 18557 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.938s	user 0.012s	sys 0.010s
I20260709 12:12:22.309684 18584 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:22.311981 18633 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:22.210882 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 114637us
I20260709 12:12:22.309363 18341 raft_consensus.cc:697] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:22.366973 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:22.368549 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:22.492466 12603 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:22.368563 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:22.494828 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:22.495105 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:22.496181 12603 raft_consensus.cc:2410] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 12 because replica has last-logged OpId of term: 11 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 5.
I20260709 12:12:22.496532 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:22.370632 12525 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260709 12:12:22.424026 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:22.409394 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 12 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:22.510183 12671 raft_consensus.cc:2376] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12. Re-sending same reply.
I20260709 12:12:22.433619 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:22.513330 18589 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:22.435081 18386 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18.
I20260709 12:12:22.411226 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2606) took 4238 ms. Trace:
I20260709 12:12:22.451813 18456 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:22.441023 18341 consensus_queue.cc:237] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:22.384964 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:22.513444 13336 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
W20260709 12:12:22.518924 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:22.455726 18479 consensus_meta.cc:287] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.804s	user 0.008s	sys 0.007s
I20260709 12:12:22.519639 18479 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:22.417049 13332 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:22.411870 12677 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:12:22.521791 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:22.409394 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:22.497474 12522 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:22.522845 18479 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:22.458110 18508 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:22.463670 18491 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:22.523674 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:22.463689 13334 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:12:22.477744 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.484091 12680 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:22.487321 12678 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:12:22.369608 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:22.517692 18456 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:22.460181 18391 raft_consensus.cc:3055] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:22.591660 18391 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:22.386530 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:22.595311 12601 raft_consensus.cc:2410] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:12:22.596976 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:22.598034 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2634) took 2074 ms. Trace:
I20260709 12:12:22.598531 12672 rpcz_store.cc:276] 0709 12:12:20.523976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:21.351418 (+827442us) service_pool.cc:224] Handling call
0709 12:12:22.597629 (+1246211us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54000ac8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:22.597839 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245467904}
I20260709 12:12:22.600554 12672 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:12:22.601310 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2980) took 1990 ms. Trace:
I20260709 12:12:22.601648 12672 rpcz_store.cc:276] 0709 12:12:20.610937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:22.599509 (+1988572us) service_pool.cc:224] Handling call
0709 12:12:22.600069 (+   560us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:22.600990 (+   921us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:22.601231 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:22.417773 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:22.602550 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:22.604156 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:22.605383 12522 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:22.606351 18526 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:22.444902 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2625) took 1957 ms. Trace:
I20260709 12:12:22.608147 18446 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:22.608362 12529 rpcz_store.cc:276] 0709 12:12:20.486846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:20.659129 (+172283us) service_pool.cc:224] Handling call
0709 12:12:20.659446 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:21.082972 (+423526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:22.444100 (+1361128us) spinlock_profiling.cc:243] Waited 285 ms on lock 0x7b5400053054. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:22.444617 (+   517us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":656561536}
I20260709 12:12:22.610647 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:22.612748 18496 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:22.614549 18542 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:22.517246 12676 rpcz_store.cc:276] 0709 12:12:18.172825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:18.173210 (+   385us) service_pool.cc:224] Handling call
0709 12:12:18.193052 (+ 19842us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:18.216292 (+ 23240us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:12:18.308151 (+ 91859us) log.cc:844] Serialized 14 byte log entry
0709 12:12:19.302050 (+993899us) spinlock_profiling.cc:243] Waited 2.93 ms on lock 0x7b18004bbe60. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1
0709 12:12:19.304486 (+  2436us) spinlock_profiling.cc:243] Waited 3.44 ms on lock 0x7b18004bbe60. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:12:19.901537 (+597051us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:22.262308 (+2360771us) spinlock_profiling.cc:243] Waited 10.1 ms on lock 0x7b2c00037868. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:12:22.262964 (+   656us) log.cc:844] Serialized 52 byte log entry
0709 12:12:22.410297 (+147333us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:22.410427 (+   130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:22.410610 (+   183us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:22.410709 (+    99us) raft_consensus.cc:1685] finished
0709 12:12:22.410721 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:22.410956 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1243549,"prepare.queue_time_us":38216,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":48,"raft.queue_time_us":483365,"raft.run_cpu_time_us":307,"raft.run_wall_time_us":18040,"spinlock_wait_cycles":37775616,"thread_start_us":1297116,"threads_started":4,"wal-append.queue_time_us":424755,"wal-append.run_cpu_time_us":11283,"wal-append.run_wall_time_us":1108209}
I20260709 12:12:22.615221 18542 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:22.372211 12531 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:12:22.617584 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:22.532295 18511 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:22.309041 18544 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:22.543090 18557 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.229s	user 0.001s	sys 0.000s
I20260709 12:12:22.370496 12528 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 11. Current term is 12. Ops: [11.7-11.7]
I20260709 12:12:22.545586 17582 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:22.383298 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:22.547273 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2628) took 1802 ms. Trace:
W20260709 12:12:22.323290 18626 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.159s	user 0.001s	sys 0.002s
W20260709 12:12:22.656268 18221 consensus_queue.cc:1175] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "193a40da6e364ead8ab561e14c3cc8d1" responder_term: 14 status { last_received { term: 14 index: 9 } last_committed_idx: 9 last_received_current_leader { term: 14 index: 9 } }
I20260709 12:12:22.554437 18592 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:22.383416 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 161913us
I20260709 12:12:22.547273 18585 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:22.559311 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2972) took 1953 ms. Trace:
W20260709 12:12:22.563174 12597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2654) took 4462 ms (client timeout 1623 ms). Trace:
I20260709 12:12:22.564095 12525 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:12:22.566931 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:22.569078 18463 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:22.569620 18386 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:22.593451 18391 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:22.570211 12494 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:22.663419 18386 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:22.663854 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 93911us
I20260709 12:12:22.664271 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:22.665181 12531 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:22.577097 18448 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:22.583639 18525 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:22.586835 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2858) took 4063 ms. Trace:
I20260709 12:12:22.386349 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2870) took 2814 ms. Trace:
I20260709 12:12:22.613732 18496 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:22.626726 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:22.626739 18542 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:22.612820 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:22.629910 18589 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:22.631999 18456 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:22.632702 12524 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:22.660215 12678 rpcz_store.cc:276] 0709 12:12:20.605769 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:22.480121 (+1874352us) service_pool.cc:224] Handling call
0709 12:12:22.480815 (+   694us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:22.558927 (+ 78112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:22.559222 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:22.636093 12675 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:12:22.625897 18584 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:22.639138 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:22.671865 12522 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:12:22.640748 18499 raft_consensus.cc:3055] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:22.645079 12677 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
I20260709 12:12:22.680291 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:22.577553 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:22.648475 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2975) took 2041 ms. Trace:
I20260709 12:12:22.650166 12676 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: []
W20260709 12:12:22.655690 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:22.655867 13333 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 16: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:22.657876 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.662953 12673 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:12:22.634413 18491 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:22.640204 18524 consensus_meta.cc:287] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.827s	user 0.016s	sys 0.013s
I20260709 12:12:22.661403 18463 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:22.660377 12597 rpcz_store.cc:269] 0709 12:12:18.100617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:18.234056 (+133439us) service_pool.cc:224] Handling call
0709 12:12:22.562690 (+4328634us) spinlock_profiling.cc:243] Waited 1.89 s on lock 0x7b54002f1554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:22.562923 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4350866688}
I20260709 12:12:22.682548 12680 rpcz_store.cc:276] 0709 12:12:20.606977 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:22.482786 (+1875809us) service_pool.cc:224] Handling call
0709 12:12:22.483442 (+   656us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:22.555939 (+ 72497us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:22.648362 (+ 92423us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:22.755163 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:22.669293 12599 rpcz_store.cc:276] 0709 12:12:18.523323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:19.943641 (+1420318us) service_pool.cc:224] Handling call
0709 12:12:19.944341 (+   700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:21.769106 (+1824765us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:22.586182 (+817076us) spinlock_profiling.cc:243] Waited 1.81 s on lock 0x7b54002ff654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:22.586595 (+   413us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4152787840}
I20260709 12:12:22.757210 18540 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:22.757740 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.758836 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:22.759725 12528 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:22.760495 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:22.671303 18496 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:22.670243 18592 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:22.754244 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:22.761925 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:22.655030 12526 rpcz_store.cc:276] 0709 12:12:20.744584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:20.787400 (+ 42816us) service_pool.cc:224] Handling call
0709 12:12:20.791811 (+  4411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:21.095188 (+303377us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:12:21.331309 (+236121us) log.cc:844] Serialized 14 byte log entry
0709 12:12:21.376965 (+ 45656us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:21.377023 (+    58us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:21.377059 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:22.546686 (+1169627us) spinlock_profiling.cc:243] Waited 303 ms on lock 0x7b5400053554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:22.546786 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:22.547057 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1716,"prepare.queue_time_us":235044,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":3389,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":37388,"spinlock_wait_cycles":696991616,"thread_start_us":264295,"threads_started":2,"wal-append.queue_time_us":132603,"wal-append.run_cpu_time_us":1955,"wal-append.run_wall_time_us":1077458}
I20260709 12:12:22.771929 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.758652 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:22.768996 13337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 744) took 2290 ms. Trace:
I20260709 12:12:22.776348 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:22.762029 12529 raft_consensus.cc:2410] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:12:22.774214 12527 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 17: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:12:22.766733 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:22.799551 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:22.760648 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:22.776199 12671 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:22.758675 12679 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:22.666072 18386 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:22.671926 18511 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:22.770932 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2970) took 4246 ms. Trace:
I20260709 12:12:22.811252 12528 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:22.812896 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:22.767568 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2977) took 2160 ms. Trace:
I20260709 12:12:22.797916 13337 rpcz_store.cc:276] 0709 12:12:20.478630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:20.479446 (+   816us) service_pool.cc:224] Handling call
0709 12:12:20.508724 (+ 29278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:21.431041 (+922317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:22.768475 (+1337434us) spinlock_profiling.cc:243] Waited 912 ms on lock 0x7b54004439d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:22.768762 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2096577280}
W20260709 12:12:22.764760 18563 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.091s	user 0.018s	sys 0.000s
I20260709 12:12:22.680320 18499 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:22.813851 12673 rpcz_store.cc:276] 0709 12:12:20.607573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:22.562876 (+1955303us) service_pool.cc:224] Handling call
0709 12:12:22.563478 (+   602us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:22.749813 (+186335us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:22.767139 (+ 17326us) spinlock_profiling.cc:243] Waited 95.3 ms on lock 0x7b54000ac3d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:22.767464 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":219246976}
I20260709 12:12:22.760432 18491 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:22.814069 12526 raft_consensus.cc:2410] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:12:22.666755 18640 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:22.816805 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:22.816946 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:22.823886 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 251013us
I20260709 12:12:22.816803 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.772526 18313 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:22.828686 12677 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:22.666815 18452 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.150s
I20260709 12:12:22.839116 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.850104 12675 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 17: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:12:22.851358 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:22.853729 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.854040 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.854408 12603 raft_consensus.cc:2376] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15. Re-sending same reply.
I20260709 12:12:22.855209 12675 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:22.855614 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.856529 12603 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 13 index: 7. (index mismatch)
I20260709 12:12:22.853763 18544 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:22.765017 18524 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:22.857084 18236 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:22.856429 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.857106 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.851112 12601 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:22.827557 18640 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:22.861575 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 11 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:22.811897 12674 rpcz_store.cc:276] 0709 12:12:18.524615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:18.992231 (+467616us) service_pool.cc:224] Handling call
0709 12:12:18.992619 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:19.175904 (+183285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:19.175934 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:19.176436 (+   502us) log.cc:844] Serialized 26 byte log entry
0709 12:12:21.991242 (+2814806us) spinlock_profiling.cc:243] Waited 1.62 ms on lock 0x7b18003be4e0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:12:22.148441 (+157199us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:22.161815 (+ 13374us) log.cc:844] Serialized 14 byte log entry
0709 12:12:22.296217 (+134402us) spinlock_profiling.cc:243] Waited 3.16 ms on lock 0x7b18003be4e0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:12:22.601484 (+305267us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b18003be4e0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:12:22.745632 (+144148us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:22.745766 (+   134us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:22.770403 (+ 24637us) raft_consensus.cc:1685] finished
0709 12:12:22.770439 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:22.770677 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":182834,"prepare.queue_time_us":14769,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":50,"raft.queue_time_us":647600,"raft.run_cpu_time_us":491,"raft.run_wall_time_us":23338,"spinlock_wait_cycles":16543744,"thread_start_us":157023,"threads_started":1,"wal-append.queue_time_us":2993269}
I20260709 12:12:22.835090 18313 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:22.864256 12675 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:22.849237 18391 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:22.848652 12529 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:22.876909 12678 raft_consensus.cc:2368] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18.
I20260709 12:12:22.775203 18463 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:22.870807 18640 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:22.883939 18236 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:22.881210 18357 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:22.886977 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.887970 12680 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:22.889127 18579 raft_consensus.cc:3055] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:22.889359 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.889686 18579 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:22.895159 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.896003 12678 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:22.878140 12679 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:12:22.901788 12523 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:22.903209 12603 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 15 index: 12. (term mismatch)
I20260709 12:12:22.903632 12603 pending_rounds.cc:85] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 11
I20260709 12:12:22.833798 18585 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:22.669421 12606 rpcz_store.cc:276] 0709 12:12:19.571381 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:21.076009 (+1504628us) service_pool.cc:224] Handling call
0709 12:12:21.076402 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:21.079090 (+  2688us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:12:21.095553 (+ 16463us) log.cc:844] Serialized 14 byte log entry
0709 12:12:21.427691 (+332138us) spinlock_profiling.cc:243] Waited 3.66 ms on lock 0x7b18003700c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485933522 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:12:21.911989 (+484298us) spinlock_profiling.cc:243] Waited 1.15 ms on lock 0x7b18003700c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a 00007f1488a2aa2a
0709 12:12:21.995575 (+ 83586us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:21.995969 (+   394us) log.cc:844] Serialized 26 byte log entry
0709 12:12:21.996070 (+   101us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:21.996090 (+    20us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:21.996137 (+    47us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:22.302698 (+306561us) spinlock_profiling.cc:243] Waited 43.9 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:22.385266 (+ 82568us) raft_consensus.cc:1685] finished
0709 12:12:22.385345 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:22.385839 (+   494us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9894,"prepare.queue_time_us":19500,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":48,"raft.queue_time_us":8517,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":15677,"spinlock_wait_cycles":112130048,"thread_start_us":72985,"threads_started":1,"wal-append.queue_time_us":1019662}
I20260709 12:12:22.825259 18563 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:22.907195 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.909765 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.903873 12603 pending_rounds.cc:107] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 14.12
I20260709 12:12:22.911649 12603 raft_consensus.cc:2889] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:22.912508 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:22.913785 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:22.916613 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.920956 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:22.909520 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:22.919831 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:22.923653 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:22.923702 12680 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:12:22.919641 13333 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:12:22.927187 12674 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:22.937752 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:22.940182 18544 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:22.940302 12676 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:22.940963 18544 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:22.942497 12523 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:22.943760 18544 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:22.944134 18463 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:22.944568 18645 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:22.945695 12597 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:22.945242 18645 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:22.947662 18585 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:22.948129 18645 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:22.932514 18452 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.017s
I20260709 12:12:22.949513 18452 consensus_queue.cc:1243] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:12:22.950183 18452 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:22.950554 18452 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:22.951728 18524 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:22.952944 18584 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:22.956877 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:22.957261 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2880) took 1241 ms. Trace:
I20260709 12:12:22.958449 18112 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:22.957655 12598 rpcz_store.cc:276] 0709 12:12:21.715358 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:22.452768 (+737410us) service_pool.cc:224] Handling call
0709 12:12:22.453683 (+   915us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:22.464896 (+ 11213us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:22.464936 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:22.482344 (+ 17408us) log.cc:844] Serialized 26 byte log entry
0709 12:12:22.746202 (+263858us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:12:22.822658 (+ 76456us) log.cc:844] Serialized 14 byte log entry
0709 12:12:22.834336 (+ 11678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:22.834438 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:22.947772 (+113334us) spinlock_profiling.cc:243] Waited 2.74 ms on lock 0x7b18003bb6c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f148c8f458f 00007f1485933571 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd
0709 12:12:22.956720 (+  8948us) raft_consensus.cc:1685] finished
0709 12:12:22.956869 (+   149us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:22.957044 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11830,"prepare.queue_time_us":107250,"raft.queue_time_us":12040,"raft.run_cpu_time_us":687,"raft.run_wall_time_us":10770,"spinlock_wait_cycles":19338880,"thread_start_us":190077,"threads_started":1,"wal-append.queue_time_us":365300}
I20260709 12:12:22.959272 18112 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:22.960443 18477 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:22.960561 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:22.961264 18477 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:22.939280 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.962395 12675 raft_consensus.cc:2393] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 17: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:22.962613 18112 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:22.941781 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:22.964555 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:22.945088 12523 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
W20260709 12:12:22.964061 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 64074.6us
I20260709 12:12:22.963444 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.969278 12679 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:22.971602 12676 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 13: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:22.983471 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:22.984192 12676 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:22.951786 18452 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:22.988787 18452 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:22.991825 12646 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:22.992934 18550 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:22.993520 18550 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:22.994107 18550 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:23.007287 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:23.008894 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:23.013315 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:23.014209 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:23.013278 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 45598.2us
W20260709 12:12:23.016431 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49593us
I20260709 12:12:23.023684 12675 raft_consensus.cc:2410] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 19 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:12:23.037772 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:23.036674 12676 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:23.036070 12603 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
W20260709 12:12:23.047693 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 173081us
I20260709 12:12:23.025120 18236 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:23.024591 18477 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.062160 13305 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:23.070350 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:23.071390 12675 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
W20260709 12:12:23.063138 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 114484us
W20260709 12:12:23.087366 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 44089.6us
I20260709 12:12:23.089600 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:23.091432 12674 raft_consensus.cc:2410] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:12:23.089668 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:23.093421 12597 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
W20260709 12:12:23.094543 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:23.097338 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:12:23.098099 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:23.098634 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:23.108878 13333 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:23.110435 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:23.096401 18563 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:23.112783 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:23.113502 12674 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:23.110582 12597 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:12:23.114023 12601 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:23.114712 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 2889) took 1026 ms. Trace:
W20260709 12:12:23.115586 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:23.112186 12496 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:23.115067 12601 rpcz_store.cc:276] 0709 12:12:22.088086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:22.651906 (+563820us) service_pool.cc:224] Handling call
0709 12:12:23.114642 (+462736us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:23.118333 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:23.100814 18511 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:23.111824 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:23.118094 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:23.130321 12646 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:23.128630 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:23.063457 18313 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.133096 18313 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:23.136356 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 888059us
I20260709 12:12:23.137540 18587 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:23.138208 18587 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:23.111816 18402 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:23.144131 18587 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:23.150938 12603 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:23.151587 12679 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:23.158515 13339 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:23.097687 18579 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.166301 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:23.146370 18479 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:12:23.168902 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58365.5us
I20260709 12:12:23.146651 18341 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:23.175010 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:23.174774 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:23.176054 12679 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:23.178062 12601 raft_consensus.cc:2393] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 13: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:12:23.178674 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 116695us
I20260709 12:12:23.178903 18199 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:23.179594 13305 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:12:23.181609 18623 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.173s	user 0.003s	sys 0.003s
I20260709 12:12:23.181797 18566 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:23.182986 18250 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:23.182680 18566 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.186836 12530 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:23.179555 18199 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.189570 12494 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:23.189561 18635 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:23.189857 18511 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:23.196722 18635 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:23.196925 18511 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:23.197198 18635 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:23.197355 18511 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:23.178789 18448 raft_consensus.cc:2764] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:12:23.207096 13337 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:12:23.209241 18579 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:23.212074 18511 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:23.214977 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 127758us
I20260709 12:12:23.216393 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2881) took 1496 ms. Trace:
I20260709 12:12:23.216784 12600 rpcz_store.cc:276] 0709 12:12:21.720038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:21.869126 (+149088us) service_pool.cc:224] Handling call
0709 12:12:21.876214 (+  7088us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:21.877414 (+  1200us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:12:22.579984 (+702570us) spinlock_profiling.cc:243] Waited 4.21 ms on lock 0x7b1800467500. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf 00007f1488a3a038
0709 12:12:22.853199 (+273215us) log.cc:844] Serialized 14 byte log entry
0709 12:12:23.111647 (+258448us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:23.164717 (+ 53070us) log.cc:844] Serialized 52 byte log entry
0709 12:12:23.165016 (+   299us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:23.173625 (+  8609us) log.cc:844] Serialized 14 byte log entry
0709 12:12:23.174505 (+   880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:23.188530 (+ 14025us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:23.215969 (+ 27439us) raft_consensus.cc:1685] finished
0709 12:12:23.215990 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:23.216225 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":100904,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":9811,"raft.run_cpu_time_us":474,"raft.run_wall_time_us":27704,"spinlock_wait_cycles":14681600,"thread_start_us":402509,"threads_started":2,"wal-append.queue_time_us":297338}
I20260709 12:12:23.219766 18199 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:23.220595 18199 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:23.219761 18511 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:23.221230 18199 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:23.222291 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:23.222463 18499 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.224932 18499 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:23.187634 18550 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.229255 18550 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:23.230155 18341 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.230542 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:23.232836 18341 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:23.170027 18479 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:12:23.175545 18480 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.078s	user 0.012s	sys 0.008s
I20260709 12:12:23.234302 18402 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:23.288612 12674 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
W20260709 12:12:23.234851 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 80266.9us
I20260709 12:12:23.238003 18476 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:23.298774 18476 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:23.299893 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:23.300083 18476 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.239176 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:23.301735 13332 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 17: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:23.238343 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:23.239491 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:23.242647 18585 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 11.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:12:23.243454 18563 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:12:23.243621 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52205.2us
I20260709 12:12:23.244652 18357 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:23.245906 13334 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 586) took 1202 ms (client timeout 1523 ms). Trace:
I20260709 12:12:23.246172 13335 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 10. Current term is 12. Ops: []
I20260709 12:12:23.246487 18586 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:23.248013 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2976) took 2640 ms. Trace:
I20260709 12:12:23.241650 12496 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:23.248347 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:23.279912 18427 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 6, Time since last communication: 0.013s
I20260709 12:12:23.255604 13339 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:23.248296 18566 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:23.327787 12671 rpcz_store.cc:276] 0709 12:12:20.607281 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:22.511243 (+1903962us) service_pool.cc:224] Handling call
0709 12:12:22.511945 (+   702us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:22.807861 (+295916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:23.247736 (+439875us) spinlock_profiling.cc:243] Waited 264 ms on lock 0x7b54003d45d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:23.247923 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":607278336}
I20260709 12:12:23.261406 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:23.262781 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:23.264907 12680 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:23.265352 18199 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:23.284354 18479 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:23.266947 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:23.197098 18528 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:23.271704 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:23.273103 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2857) took 4753 ms. Trace:
I20260709 12:12:23.340549 13338 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:12:23.273531 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2577) took 4099 ms. Trace:
I20260709 12:12:23.345171 13330 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 17: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:23.343420 12530 rpcz_store.cc:276] 0709 12:12:19.174523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:19.174669 (+   146us) service_pool.cc:224] Handling call
0709 12:12:19.192326 (+ 17657us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:23.187334 (+3995008us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:23.273084 (+ 85750us) spinlock_profiling.cc:243] Waited 3.99 s on lock 0x7b5400053a54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:23.273395 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9185031552}
I20260709 12:12:23.278612 12676 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:12:23.279104 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:23.284297 12606 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:23.290323 12598 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
W20260709 12:12:23.288621 18480 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.272s	user 0.013s	sys 0.008s
I20260709 12:12:23.300745 12679 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:23.301731 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:23.240902 18597 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:12:23.319937 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:23.319538 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:12:23.209980 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:23.322818 13334 rpcz_store.cc:269] 0709 12:12:22.043130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:22.043402 (+   272us) service_pool.cc:224] Handling call
0709 12:12:23.245585 (+1202183us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b540063ecd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:23.245814 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245844224}
W20260709 12:12:23.328904 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:23.265633 18570 raft_consensus.cc:3055] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Stepping down as leader of term 9
I20260709 12:12:23.380261 18570 raft_consensus.cc:740] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:23.381175 18563 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:12:23.382346 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 140796us
I20260709 12:12:23.382664 18430 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:23.383373 18430 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:23.387470 18631 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.175s	user 0.002s	sys 0.002s
I20260709 12:12:23.341964 12605 rpcz_store.cc:276] 0709 12:12:18.519860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:19.422307 (+902447us) service_pool.cc:224] Handling call
0709 12:12:20.247283 (+824976us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:20.247815 (+   532us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:20.247861 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:20.248572 (+   711us) log.cc:844] Serialized 26 byte log entry
0709 12:12:20.479522 (+230950us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:20.479593 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:20.484056 (+  4463us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:21.081786 (+597730us) raft_consensus.cc:1685] finished
0709 12:12:21.081810 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:23.272617 (+2190807us) spinlock_profiling.cc:243] Waited 379 ms on lock 0x7b54002e59d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:23.272917 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":46748,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":9578,"spinlock_wait_cycles":871856384,"thread_start_us":230521,"threads_started":1,"wal-append.queue_time_us":365239,"wal-append.run_cpu_time_us":3377,"wal-append.run_wall_time_us":1477478}
I20260709 12:12:23.390515 18443 raft_consensus.cc:3055] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:23.391412 18443 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:23.354579 12601 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:23.362521 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:23.392951 12606 raft_consensus.cc:2410] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:12:23.392736 18443 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.392750 13336 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 17: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:23.394292 12606 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: []
W20260709 12:12:23.369316 12602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 598) took 1927 ms (client timeout 1994 ms). Trace:
I20260709 12:12:23.359493 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:23.395015 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2676) took 1347 ms. Trace:
I20260709 12:12:23.395663 12598 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:23.395535 12606 rpcz_store.cc:276] 0709 12:12:22.047973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:23.393625 (+1345652us) service_pool.cc:224] Handling call
0709 12:12:23.394051 (+   426us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:23.394727 (+   676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:23.394953 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:23.367630 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:23.374679 13337 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:23.397320 12598 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:23.397678 12522 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 15: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:23.342361 18357 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.398816 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:23.399529 12522 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15.
I20260709 12:12:23.358160 12674 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:12:23.399971 18357 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:23.400549 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:23.401202 12674 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:23.375861 18635 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:23.401718 13304 connection.cc:256] RPC call timeout handler was delayed by -1.4382502609998937s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16100000000000003s for a response
I20260709 12:12:23.377410 18220 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:23.378042 18559 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:23.378811 18586 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:23.379673 18303 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:23.385953 18430 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:23.387322 12604 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 12.
I20260709 12:12:23.391549 18580 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:23.393235 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2586) took 1027 ms. Trace:
I20260709 12:12:23.430375 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 600) took 1754 ms. Trace:
W20260709 12:12:23.425544 18391 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.553s	user 0.007s	sys 0.006s
I20260709 12:12:23.402954 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:23.424459 13304 connection.cc:256] RPC call timeout handler was delayed by -1.3523648509999475s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19890000000000002s for a response
I20260709 12:12:23.322395 18402 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:23.372073 18592 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:23.405377 12676 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:12:23.406265 18443 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:23.374878 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:23.376677 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:23.397981 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2681) took 1348 ms. Trace:
W20260709 12:12:23.375746 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 45971.2us
I20260709 12:12:23.387226 12671 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:12:23.452126 12598 rpcz_store.cc:276] 0709 12:12:22.049423 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:23.396570 (+1347147us) service_pool.cc:224] Handling call
0709 12:12:23.396937 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:23.397718 (+   781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:23.397922 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:23.395064 12602 rpcz_store.cc:269] 0709 12:12:21.441937 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:23.266509 (+1824572us) service_pool.cc:224] Handling call
0709 12:12:23.369225 (+102716us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:23.395195 12672 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: [14.10-14.10]
I20260709 12:12:23.418680 12601 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:23.422447 12531 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
W20260709 12:12:23.391400 18640 consensus_meta.cc:287] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.509s	user 0.005s	sys 0.008s
I20260709 12:12:23.379585 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:23.390676 18427 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:23.402324 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:23.430898 12604 rpcz_store.cc:276] 0709 12:12:21.675657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:23.270991 (+1595334us) service_pool.cc:224] Handling call
0709 12:12:23.430284 (+159293us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:23.404222 18635 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:23.431535 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 210679us
I20260709 12:12:23.435904 12527 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
W20260709 12:12:23.435829 13304 connection.cc:256] RPC call timeout handler was delayed by -1.3482502609999756s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.171s for a response
I20260709 12:12:23.428227 18303 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:23.430668 12524 rpcz_store.cc:276] 0709 12:12:22.365539 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:22.367622 (+  2083us) service_pool.cc:224] Handling call
0709 12:12:23.393034 (+1025412us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b5400408154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:23.393151 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":320680448}
I20260709 12:12:23.449679 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:23.381513 18570 consensus_queue.cc:260] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.469538 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:23.460876 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 81358.6us
I20260709 12:12:23.447909 12679 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:23.470808 12531 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 18: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:23.475138 12672 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:23.410559 18528 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.468492 18592 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:23.490357 18592 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:23.490903 18592 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:23.494277 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:23.495707 12494 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:23.499866 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:23.500871 12528 raft_consensus.cc:2410] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:12:23.501562 18635 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:23.502036 18635 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:23.502529 18635 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:23.502765 12646 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:23.503207 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:23.504274 12646 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:23.504575 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:23.505182 18499 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:23.505913 18499 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:23.506924 12598 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:23.507001 18499 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:23.507855 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2683) took 1454 ms. Trace:
I20260709 12:12:23.508224 12598 rpcz_store.cc:276] 0709 12:12:22.053225 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:23.453387 (+1400162us) service_pool.cc:224] Handling call
0709 12:12:23.506691 (+ 53304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:23.507544 (+   853us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:23.507762 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:23.510326 18463 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.566s	user 0.014s	sys 0.000s
I20260709 12:12:23.511055 12529 raft_consensus.cc:606] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:23.511482 12529 raft_consensus.cc:614] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:23.480518 12496 leader_election.cc:400] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18.
I20260709 12:12:23.511863 18523 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:23.512773 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 76447.4us
I20260709 12:12:23.480639 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:23.514952 12602 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:23.515129 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:23.515662 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2887) took 1432 ms. Trace:
W20260709 12:12:23.514987 18642 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.609s	user 0.003s	sys 0.001s
W20260709 12:12:23.484740 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 110094us
I20260709 12:12:23.516942 18587 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.517640 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:23.485327 18430 raft_consensus.cc:2764] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:12:23.485215 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:23.485730 18402 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:23.487588 12643 leader_election.cc:336] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.623s (SENT)
I20260709 12:12:23.488618 18220 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.471820 18559 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.516294 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:23.470378 18303 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:23.483738 18586 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:23.501974 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:23.505214 12522 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:23.515991 12602 rpcz_store.cc:276] 0709 12:12:22.083186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:23.460210 (+1377024us) service_pool.cc:224] Handling call
0709 12:12:23.514736 (+ 54526us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:23.515417 (+   681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:23.515597 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:23.523121 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:23.532563 12527 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:23.516988 18491 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:23.523975 12676 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:12:23.476655 18570 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 9 FOLLOWER]: Advancing to term 10
I20260709 12:12:23.524428 12494 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:23.540489 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:12:23.540900 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:23.467020 18640 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:23.542435 12606 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:12:23.549885 13305 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:23.553362 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:23.558230 13336 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:23.558701 12605 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:12:23.478515 18427 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:23.538524 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:23.534005 18491 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:23.541988 18391 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:23.561132 12525 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:23.562800 12602 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:23.562706 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:23.564069 12598 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:12:23.480192 18580 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:23.567562 18428 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:23.568022 18428 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:23.568246 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 80731.1us
I20260709 12:12:23.570343 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:23.552340 18402 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:23.571745 18402 raft_consensus.cc:697] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:23.572661 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:23.572861 18402 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 14.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:23.573539 18480 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:23.574069 18480 raft_consensus.cc:697] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:23.574273 18430 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:23.574702 18430 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:23.575111 18430 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:23.578258 12571 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:23.579254 18587 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:23.579381 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:23.575070 18480 consensus_queue.cc:237] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.582348 18645 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:23.583132 18645 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:23.583719 18645 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:23.586038 18645 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:23.587594 18451 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:23.588163 18451 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:23.588313 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:23.588569 18451 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:23.592221 12522 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:23.593318 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:23.593950 12522 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:23.594224 18645 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:23.595778 18559 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:23.596544 18220 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:23.596937 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:23.560693 18448 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:23.602707 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:23.596678 18559 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:23.565100 12496 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:23.568418 18428 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:23.597316 12524 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 9. Current term is 10. Ops: []
I20260709 12:12:23.613860 12525 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
W20260709 12:12:23.615588 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 130513us
I20260709 12:12:23.624837 12602 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:12:23.625479 18476 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:23.626502 18597 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:23.628611 12602 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:23.629549 18467 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:23.630573 18463 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:23.631191 18463 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.631788 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2671) took 1586 ms. Trace:
I20260709 12:12:23.632288 12605 rpcz_store.cc:276] 0709 12:12:22.045416 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:23.390044 (+1344628us) service_pool.cc:224] Handling call
0709 12:12:23.407195 (+ 17151us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:23.559055 (+151860us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:23.631483 (+ 72428us) spinlock_profiling.cc:243] Waited 140 ms on lock 0x7b54002ffb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:23.631704 (+   221us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":321623424}
I20260709 12:12:23.635447 12604 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:12:23.634016 12605 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 12 index: 8. (index mismatch)
I20260709 12:12:23.645416 12527 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
I20260709 12:12:23.650894 18491 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:23.653290 18491 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:23.654803 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 57988us
I20260709 12:12:23.655992 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:23.647756 18250 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:23.659260 18528 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:23.634302 18617 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.513s	user 0.010s	sys 0.024s
W20260709 12:12:23.660066 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:23.660200 18617 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.634s	user 0.010s	sys 0.024s
W20260709 12:12:23.661409 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:23.663462 18452 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:23.664021 18452 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:23.669095 12522 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:23.676162 12524 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:23.620600 18559 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:23.678087 12525 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:23.694361 18463 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:23.707742 18246 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:23.712461 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48476 (request call id 57) took 1156 ms. Trace:
I20260709 12:12:23.713140 12529 rpcz_store.cc:276] 0709 12:12:22.646371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:22.823439 (+177068us) service_pool.cc:224] Handling call
0709 12:12:23.712088 (+888649us) spinlock_profiling.cc:243] Waited 635 ms on lock 0x7b5400052b54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1a853 00007f148d9e860a 00007f1486f6e7d4 00007f1486f6e762 00007f1486f6e691 00007f1486f6e60c 00007f1486f6d8c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:23.712293 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1460845312}
I20260709 12:12:23.716462 18511 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:23.727519 12646 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:23.731374 18645 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:23.732156 18645 raft_consensus.cc:697] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:23.734422 18587 consensus_queue.cc:1059] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:12:23.736187 18540 raft_consensus.cc:3055] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:23.736743 18540 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:23.738829 18540 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.7, Last appended by leader: 7, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.733568 18645 consensus_queue.cc:237] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.743639 18511 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:23.756536 18656 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:23.757269 18656 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:23.757797 18656 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:23.762789 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 214530us
I20260709 12:12:23.844405 18427 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:23.856549 18580 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:23.683410 18221 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.175s
I20260709 12:12:23.665063 18452 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.860934 18452 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:12:23.865691 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 186252us
I20260709 12:12:23.867774 12569 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:12:23.878211 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:23.879585 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:23.880875 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:23.804545 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 74559.5us
I20260709 12:12:23.745038 18511 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:23.945770 12524 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:23.657724 18479 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.684717 18559 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:23.688182 18597 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:23.986768 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:23.973054 12674 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.007s	sys 0.010s
I20260709 12:12:23.990571 12599 raft_consensus.cc:2410] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 8.
I20260709 12:12:23.755784 18303 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:23.960951 18221 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:23.973093 18584 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:23.996929 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:24.007068 12523 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: []
W20260709 12:12:24.017185 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 149557us
W20260709 12:12:24.020665 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:23.991235 18582 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:24.036074 18582 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:24.036478 18341 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:12:24.036944 18657 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.348s	user 0.000s	sys 0.001s
I20260709 12:12:24.037351 18341 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:24.007440 18221 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:24.038465 18221 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:24.038883 18341 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:24.041440 18582 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:24.043377 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 603) took 1189 ms. Trace:
I20260709 12:12:24.044870 18586 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:24.046836 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 2987) took 1426 ms. Trace:
I20260709 12:12:24.051020 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2682) took 2001 ms. Trace:
I20260709 12:12:24.052492 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 604) took 1175 ms. Trace:
I20260709 12:12:24.070014 12598 rpcz_store.cc:276] 0709 12:12:22.877432 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:23.508994 (+631562us) service_pool.cc:224] Handling call
0709 12:12:23.509350 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:23.566999 (+ 57649us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:24.052171 (+485172us) spinlock_profiling.cc:243] Waited 44.5 ms on lock 0x7b54001167d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:24.052413 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":102311808}
I20260709 12:12:24.054335 18221 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:24.064870 12528 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:24.073133 12606 rpcz_store.cc:276] 0709 12:12:22.049691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:23.396602 (+1346911us) service_pool.cc:224] Handling call
0709 12:12:23.397040 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:23.545630 (+148590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:24.050680 (+505050us) spinlock_profiling.cc:243] Waited 145 ms on lock 0x7b540007ba54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:24.050929 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":333826176}
W20260709 12:12:24.066944 12601 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.509s	user 0.005s	sys 0.020s
I20260709 12:12:24.064333 12604 rpcz_store.cc:276] 0709 12:12:22.854034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:23.470166 (+616132us) service_pool.cc:224] Handling call
0709 12:12:23.520354 (+ 50188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:23.635855 (+115501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:24.042902 (+407047us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54001171d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:24.043255 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":236815232}
W20260709 12:12:23.990728 12674 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.007s	sys 0.011s
W20260709 12:12:24.076572 12603 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.548s	user 0.003s	sys 0.024s
W20260709 12:12:24.076918 12603 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.003s	sys 0.024s
I20260709 12:12:24.065388 12673 rpcz_store.cc:276] 0709 12:12:22.620880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:23.266837 (+645957us) service_pool.cc:224] Handling call
0709 12:12:23.267262 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:23.267642 (+   380us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:23.267673 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:23.268287 (+   614us) log.cc:844] Serialized 26 byte log entry
0709 12:12:23.561930 (+293643us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:23.561964 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:23.562032 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:24.046520 (+484488us) raft_consensus.cc:1685] finished
0709 12:12:24.046549 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:24.046740 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":293164,"threads_started":1,"wal-append.queue_time_us":689911}
W20260709 12:12:24.056885 18658 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.489s	user 0.003s	sys 0.008s
W20260709 12:12:24.071831 13305 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.514s	user 0.015s	sys 0.011s
W20260709 12:12:24.076122 12601 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.518s	user 0.006s	sys 0.021s
W20260709 12:12:24.078317 13305 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.521s	user 0.016s	sys 0.011s
W20260709 12:12:24.082535 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 503029us
I20260709 12:12:24.088770 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2655) took 1231 ms. Trace:
I20260709 12:12:24.089380 12674 rpcz_store.cc:276] 0709 12:12:22.856914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:23.402326 (+545412us) service_pool.cc:224] Handling call
0709 12:12:23.402679 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:23.407794 (+  5115us) raft_consensus.cc:1534] Early marking committed up to index 5
0709 12:12:23.434494 (+ 26700us) log.cc:844] Serialized 14 byte log entry
0709 12:12:24.088237 (+653743us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:24.088322 (+    85us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:24.088408 (+    86us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:24.088506 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:24.088684 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11210,"prepare.queue_time_us":27050,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":58,"raft.queue_time_us":6884,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":2607,"thread_start_us":603929,"threads_started":1}
W20260709 12:12:24.079579 18499 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.007s	sys 0.020s
I20260709 12:12:24.071662 18586 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:24.111300 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:24.112336 13335 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:24.113734 13334 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:24.116660 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:24.119554 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:24.120591 13334 raft_consensus.cc:2410] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:12:24.122802 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:12:24.093632 18592 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.602s	user 0.012s	sys 0.001s
I20260709 12:12:24.031203 18584 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:24.130939 12526 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.502s	user 0.015s	sys 0.028s
W20260709 12:12:24.132889 12526 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.504s	user 0.015s	sys 0.028s
I20260709 12:12:24.134353 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:24.136799 18635 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.634s	user 0.014s	sys 0.000s
I20260709 12:12:24.137363 18635 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:24.141130 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2647) took 1303 ms. Trace:
I20260709 12:12:24.141757 12677 rpcz_store.cc:276] 0709 12:12:22.837346 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:23.395713 (+558367us) service_pool.cc:224] Handling call
0709 12:12:23.396057 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:23.396460 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:23.396490 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:23.397078 (+   588us) log.cc:844] Serialized 26 byte log entry
0709 12:12:23.634573 (+237495us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:23.634628 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:23.634760 (+   132us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:23.701272 (+ 66512us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b4800060858. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:12:24.133917 (+432645us) raft_consensus.cc:1685] finished
0709 12:12:24.133954 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:24.141041 (+  7087us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11315,"raft.run_cpu_time_us":4016,"raft.run_wall_time_us":291529,"spinlock_wait_cycles":8098688,"thread_start_us":237045,"threads_started":1,"wal-append.queue_time_us":734160}
W20260709 12:12:24.147078 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:24.147584 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2665) took 2116 ms. Trace:
I20260709 12:12:24.148056 12599 rpcz_store.cc:276] 0709 12:12:22.030819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:22.810188 (+779369us) service_pool.cc:224] Handling call
0709 12:12:24.147080 (+1336892us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540023fd54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:24.147445 (+   365us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2577762048}
W20260709 12:12:24.151356 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:24.153329 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:24.073505 18550 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.036s
W20260709 12:12:24.108251 18499 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.008s	sys 0.020s
I20260709 12:12:24.154971 18540 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:24.157626 18451 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.569s	user 0.007s	sys 0.007s
I20260709 12:12:24.159574 18391 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:24.160491 18391 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:24.066735 18430 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:24.164146 18430 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:24.164937 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 168150us
I20260709 12:12:24.167845 18499 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:24.169062 18499 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:24.173393 18584 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:24.175422 12522 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:24.165941 12572 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:12:24.180199 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:24.183285 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 633481us
I20260709 12:12:24.101948 18341 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:24.186573 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:24.101694 18580 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:12:24.193411 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:24.195569 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:24.125685 18586 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:24.193127 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 86086.2us
I20260709 12:12:24.202080 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:24.202764 13334 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:24.186072 18584 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:24.206630 18584 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:24.209548 18584 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:24.211941 12522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:24.214336 18528 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:24.215085 18528 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:24.215616 18528 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:24.217293 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:24.220794 18635 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:24.203542 18661 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:12:24.197948 18523 raft_consensus.cc:2764] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
W20260709 12:12:24.234915 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 706469us
I20260709 12:12:24.191977 18580 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:24.218097 12494 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.538s (SENT) [suppressed 23 similar messages]
W20260709 12:12:24.240417 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 74619.6us
I20260709 12:12:24.196856 18250 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:24.175019 18592 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:24.158062 18451 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:24.248060 18428 consensus_meta.cc:287] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.621s	user 0.007s	sys 0.008s
W20260709 12:12:24.268558 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 299156us
I20260709 12:12:24.265908 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:24.244913 18580 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:24.297591 18523 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:24.251423 18250 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:24.303987 18221 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:24.298312 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:24.307948 12572 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:24.317131 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:24.338846 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 94078.6us
W20260709 12:12:24.335879 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:24.343068 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:24.345422 12598 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:24.316856 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:24.343401 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:24.342730 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 114003us
I20260709 12:12:24.351400 12597 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:12:24.300262 18491 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.646s	user 0.014s	sys 0.000s
I20260709 12:12:24.347425 13338 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:24.342679 18674 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:12:24.353709 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 60500.5us
I20260709 12:12:24.348249 12600 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:24.356421 13334 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:24.355693 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:24.362257 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:24.364362 18443 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:12:24.364688 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:24.305828 18479 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:24.322494 18303 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:24.374716 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:24.364415 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:24.364651 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:24.326713 18499 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:24.376964 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:24.300089 18428 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:24.375712 12599 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:24.380898 18221 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:12:24.383651 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:24.384297 18582 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:24.385751 12605 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:12:24.387311 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 138979us
I20260709 12:12:24.370327 18511 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:24.333794 18523 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:24.401265 12597 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:24.401980 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:24.393700 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:24.403069 12606 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:12:24.321938 18587 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.061s
W20260709 12:12:24.408066 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 433148us
I20260709 12:12:24.390187 18582 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:24.411827 13305 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:12:24.393879 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 411794us
I20260709 12:12:24.425650 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2890) took 1511 ms. Trace:
I20260709 12:12:24.370579 18640 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:24.398702 12598 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:12:24.390578 12600 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:24.446877 12606 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:12:24.447151 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2684) took 2393 ms. Trace:
I20260709 12:12:24.447500 12601 rpcz_store.cc:276] 0709 12:12:22.053557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:23.454384 (+1400827us) service_pool.cc:224] Handling call
0709 12:12:23.474544 (+ 20160us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:23.527960 (+ 53416us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:23.528014 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:23.540929 (+ 12915us) log.cc:844] Serialized 26 byte log entry
0709 12:12:24.078892 (+537963us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:24.446505 (+367613us) log.cc:844] Serialized 14 byte log entry
0709 12:12:24.446813 (+   308us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:24.446895 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:24.446952 (+    57us) raft_consensus.cc:1685] finished
0709 12:12:24.446963 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:24.447095 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10444,"raft.queue_time_us":8146,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":11256,"thread_start_us":866350,"threads_started":2,"wal-append.queue_time_us":741901}
I20260709 12:12:24.404296 18303 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:24.452914 12646 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:24.430156 13304 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:24.458567 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:24.407542 18443 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:24.459658 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:24.464062 12528 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 18: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:24.445001 12603 rpcz_store.cc:276] 0709 12:12:22.914700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:23.513028 (+598328us) service_pool.cc:224] Handling call
0709 12:12:23.513411 (+   383us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:23.513699 (+   288us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:12:23.514058 (+   359us) log.cc:844] Serialized 14 byte log entry
0709 12:12:24.397839 (+883781us) log.cc:844] Serialized 14 byte log entry
0709 12:12:24.425273 (+ 27434us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:24.425331 (+    58us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:24.425380 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:24.425450 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:24.425585 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":91699,"prepare.queue_time_us":1378,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":48566,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":5053,"thread_start_us":847790,"threads_started":2,"wal-append.queue_time_us":610961}
I20260709 12:12:24.418449 18582 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:24.469993 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:24.472334 13334 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:24.478399 13331 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:12:24.444720 18427 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:24.430914 18511 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:12:24.462553 18665 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.055s	user 0.001s	sys 0.000s
W20260709 12:12:24.489872 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:24.473826 18656 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.715s	user 0.008s	sys 0.004s
W20260709 12:12:24.493204 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 62388.6us
I20260709 12:12:24.498256 12571 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:24.476483 18499 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:24.514851 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:24.516080 18592 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:24.516772 18443 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:24.517242 18443 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:24.517648 18443 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:24.518740 18250 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:24.521073 18250 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:24.521431 18540 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:12:24.522749 18669 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.157s	user 0.001s	sys 0.000s
W20260709 12:12:24.524642 18673 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:12:24.526329 18391 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:24.527097 18540 consensus_queue.cc:1243] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 15.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:12:24.527631 18679 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:24.528045 18679 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:24.528497 18391 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:24.488888 18480 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:24.529414 18679 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:24.530491 18480 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:24.531474 18480 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:24.537916 18443 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:24.540107 18580 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:24.540633 18443 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:24.544174 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 67076.1us
I20260709 12:12:24.503665 18656 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:24.488078 18584 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader election won for term 12
W20260709 12:12:24.546665 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:24.546787 12571 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:24.548255 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 330155us
I20260709 12:12:24.551769 13338 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
W20260709 12:12:24.559818 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 54604.9us
I20260709 12:12:24.472958 18303 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:24.575760 18303 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:24.578480 18586 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:24.581626 18467 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.952s	user 0.009s	sys 0.005s
W20260709 12:12:24.569501 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:24.572116 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:24.583431 18391 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:24.583901 18592 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:24.584764 18592 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:24.583957 18391 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:24.587038 18523 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:24.588099 12605 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:12:24.586597 18582 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:24.500857 18246 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.090s
I20260709 12:12:24.588582 18391 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:24.590396 18592 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:24.590921 18246 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:24.591308 18246 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:24.592207 18582 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:24.591631 12494 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:24.559320 18451 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:24.504076 18511 raft_consensus.cc:697] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 17 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:24.594094 18511 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:24.596406 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 187195us
I20260709 12:12:24.597361 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:24.598296 18666 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:24.598903 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:24.600170 13335 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:24.601564 18656 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:24.606455 12496 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:24.607383 18303 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:12:24.607826 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:24.607946 18303 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:24.608439 18303 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:24.611266 18580 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:24.614900 18559 consensus_meta.cc:287] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.589s	user 0.009s	sys 0.001s
I20260709 12:12:24.615521 18559 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:24.618618 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:24.619827 12523 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 16: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:24.622032 18592 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:24.627991 18391 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:24.628851 18391 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:24.631265 18391 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:24.633355 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:24.573240 13334 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:24.637764 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:24.638872 18391 raft_consensus.cc:2764] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election decision vote started in defunct term 16: won
I20260709 12:12:24.640729 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2653) took 1234 ms. Trace:
I20260709 12:12:24.642417 18586 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:24.462757 18674 consensus_queue.cc:1059] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.051s
I20260709 12:12:24.641384 12527 rpcz_store.cc:276] 0709 12:12:23.406360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:23.648638 (+242278us) service_pool.cc:224] Handling call
0709 12:12:23.649155 (+   517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:23.681196 (+ 32041us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:23.681230 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:23.787467 (+106237us) log.cc:844] Serialized 26 byte log entry
0709 12:12:24.374578 (+587111us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:24.374621 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:24.374705 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:24.640336 (+265631us) raft_consensus.cc:1685] finished
0709 12:12:24.640377 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:24.640632 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25371,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":10813,"thread_start_us":376557,"threads_started":1,"wal-append.queue_time_us":662750}
I20260709 12:12:24.641515 12494 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:24.642432 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 272585us
I20260709 12:12:24.659641 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:24.662267 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:24.642575 18675 raft_consensus.cc:2764] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won
W20260709 12:12:24.552002 18452 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.690s	user 0.017s	sys 0.000s
W20260709 12:12:24.665887 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 56233.9us
I20260709 12:12:24.668588 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:24.669545 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 160037us
I20260709 12:12:24.678334 18645 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:24.592291 18246 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:24.681542 18246 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:24.688774 13330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 762) took 1143 ms. Trace:
I20260709 12:12:24.689265 13330 rpcz_store.cc:276] 0709 12:12:23.545871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:23.546105 (+   234us) service_pool.cc:224] Handling call
0709 12:12:23.565739 (+ 19634us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:12:24.107626 (+541887us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:24.107672 (+    46us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:12:24.241629 (+133957us) log.cc:844] Serialized 78 byte log entry
0709 12:12:24.498337 (+256708us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:24.498403 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:24.498511 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:24.688471 (+189960us) raft_consensus.cc:1685] finished
0709 12:12:24.688494 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:24.688681 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36969,"raft.queue_time_us":151592,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":130,"thread_start_us":692571,"threads_started":2,"wal-append.queue_time_us":416569}
I20260709 12:12:24.665735 18250 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:24.692162 18250 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:24.692592 18250 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:24.671319 18199 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:24.694836 18199 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:24.696132 12602 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:24.676589 18587 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.021s
I20260709 12:12:24.698200 18587 raft_consensus.cc:3055] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:24.698580 18587 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:24.699498 18587 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:24.700728 18587 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:24.676761 18584 raft_consensus.cc:697] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:24.703321 18584 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:24.706308 18597 consensus_meta.cc:287] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.718s	user 0.015s	sys 0.000s
I20260709 12:12:24.706961 18597 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:24.711510 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2905) took 1757 ms. Trace:
I20260709 12:12:24.711987 12602 rpcz_store.cc:276] 0709 12:12:22.954001 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:23.629474 (+675473us) service_pool.cc:224] Handling call
0709 12:12:23.629799 (+   325us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:24.696522 (+1066723us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:24.711142 (+ 14620us) spinlock_profiling.cc:243] Waited 1.07 s on lock 0x7b54002715d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:24.711442 (+   300us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2451591424}
I20260709 12:12:24.545372 18499 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:24.721796 18672 raft_consensus.cc:3055] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:24.722371 18672 raft_consensus.cc:740] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:24.723429 18672 consensus_queue.cc:260] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.9, Last appended by leader: 9, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:24.724466 18580 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:12:24.767674 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 419456us
W20260709 12:12:24.768440 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 290405us
I20260709 12:12:24.651291 18656 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:24.769709 12569 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:24.782027 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 76562.6us
W20260709 12:12:24.783092 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 107661us
I20260709 12:12:24.816258 18582 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:24.793969 12569 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:24.818779 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 177377us
W20260709 12:12:24.821524 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 153069us
I20260709 12:12:24.822789 12494 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:12:24.817595 18528 consensus_meta.cc:287] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.601s	user 0.009s	sys 0.006s
W20260709 12:12:24.830027 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:24.812525 18428 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:24.833829 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:24.790185 18586 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:12:24.817039 18582 raft_consensus.cc:697] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:24.781726 18580 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:24.836292 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:24.844168 12531 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:12:24.845383 12523 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:24.789672 18476 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:24.848429 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:24.849442 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:24.824808 18523 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:24.853746 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:24.854676 12524 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:24.854866 12528 raft_consensus.cc:2410] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 16 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:12:24.849776 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:24.852327 13330 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
I20260709 12:12:24.859088 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:24.860198 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:24.860343 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:24.861733 12530 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:24.858896 12526 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:24.816342 18467 consensus_queue.cc:1059] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.031s
I20260709 12:12:24.865633 12646 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:24.819761 18597 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.031s
W20260709 12:12:24.869558 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 270747us
W20260709 12:12:24.878523 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 83782.4us
I20260709 12:12:24.826882 18391 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.037s
I20260709 12:12:24.855446 18221 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:24.822221 18452 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.044s
I20260709 12:12:24.860956 18499 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:24.855670 18559 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:24.842298 18680 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:24.838972 18676 consensus_queue.cc:1059] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:12:24.859578 18672 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:24.836699 18617 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.042s
I20260709 12:12:24.893468 13337 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: [15.12-15.12]
I20260709 12:12:24.892632 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:24.914757 12523 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:24.902765 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:24.848980 18430 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:12:24.840652 18528 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:24.921221 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:24.870285 18428 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:12:24.842330 18443 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.050s
W20260709 12:12:24.893311 18341 consensus_meta.cc:287] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.699s	user 0.013s	sys 0.004s
I20260709 12:12:24.872821 18580 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:24.933519 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:24.934196 12522 raft_consensus.cc:606] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:24.934635 12522 raft_consensus.cc:614] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:12:24.939656 18570 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.398s	user 0.015s	sys 0.002s
I20260709 12:12:24.942253 18528 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:24.944552 12677 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 12:12:24.945051 12677 pending_rounds.cc:85] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 8
I20260709 12:12:24.945264 12677 pending_rounds.cc:107] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 17.9
W20260709 12:12:24.924005 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 67133us
I20260709 12:12:24.945482 12677 raft_consensus.cc:2889] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:24.946260 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:24.947896 12572 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:24.949285 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:24.877830 18688 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:24.950076 18688 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:24.950748 18688 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:24.872696 18540 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:12:24.928211 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:24.928390 18692 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:24.946393 18199 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:24.929219 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:24.960052 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:24.963532 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:24.965323 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:24.878320 18480 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:12:24.892021 12569 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:25.147835 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:25.212318 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 365352us
I20260709 12:12:24.925478 18476 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.196s
I20260709 12:12:24.910876 18221 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:24.957697 18427 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:25.204454 18443 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:25.173681 18586 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:24.966403 18694 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:24.973351 12676 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 17: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:24.999359 18559 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:24.924379 18582 consensus_queue.cc:237] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 11.9, Last appended by leader: 9, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:25.207334 12524 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:12:24.912214 18467 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.050s
I20260709 12:12:25.030709 18303 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:24.999116 18676 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:12:25.212489 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 316768us
I20260709 12:12:25.215144 18695 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:25.218502 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:25.218878 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:25.217687 18428 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260709 12:12:25.195575 18666 consensus_meta.cc:287] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.597s	user 0.007s	sys 0.012s
I20260709 12:12:25.223773 18430 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
W20260709 12:12:25.224494 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 332574us
W20260709 12:12:25.270107 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:25.239485 12675 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
W20260709 12:12:25.249402 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:25.269290 18430 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:25.282784 18430 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:25.243067 18680 raft_consensus.cc:697] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:25.287698 18680 consensus_queue.cc:237] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 13.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.254675 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.290397 18480 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:25.265000 18587 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:25.291036 18480 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:25.291733 18587 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.293457 18480 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:25.265273 18676 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:25.294019 18587 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:25.294780 18676 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.296351 18706 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:12:25.296953 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:25.258872 18221 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 12.11, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:25.297287 18570 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:25.236537 18540 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:25.257817 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.250188 18499 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:25.261128 18586 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:25.258956 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:25.233443 18250 consensus_meta.cc:287] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.540s	user 0.010s	sys 0.003s
I20260709 12:12:25.260146 18688 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:25.236265 18592 consensus_meta.cc:287] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.613s	user 0.007s	sys 0.006s
I20260709 12:12:25.290617 12677 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:25.256508 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.279323 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 10 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:25.316948 12676 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:25.315584 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.296866 18706 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:25.311987 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:25.225653 12530 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:25.305271 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48476 (request call id 58) took 1114 ms. Trace:
I20260709 12:12:25.298550 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:25.317003 12673 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:25.318945 12522 rpcz_store.cc:276] 0709 12:12:24.190781 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:24.195441 (+  4660us) service_pool.cc:224] Handling call
0709 12:12:25.304994 (+1109553us) spinlock_profiling.cc:243] Waited 721 ms on lock 0x7b5400407c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1a853 00007f148d9e860a 00007f1486f6e7d4 00007f1486f6e762 00007f1486f6e691 00007f1486f6e60c 00007f1486f6d8c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:25.305150 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1659027328}
W20260709 12:12:25.321324 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 396128us
I20260709 12:12:25.333187 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:25.302099 18640 raft_consensus.cc:3055] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:25.301746 18443 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:25.335423 12675 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:25.280104 18666 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:25.336879 12672 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:25.299683 18199 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:25.297878 18695 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:25.315897 18587 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:25.320859 18676 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:25.338645 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 124090us
I20260709 12:12:25.340205 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.335412 12679 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:25.341063 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.321727 18467 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:25.343357 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.340567 18476 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.007s
I20260709 12:12:25.325420 18427 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:25.379722 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 45423.5us
I20260709 12:12:25.322448 18250 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:25.342049 18303 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:25.297919 18570 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:25.322770 18592 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:25.384315 13335 raft_consensus.cc:2410] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 15.
I20260709 12:12:25.337715 18640 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:25.325639 18672 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:12:25.321692 18499 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.325022 18692 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 12:12:25.331013 18540 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.396907 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:25.442646 12678 raft_consensus.cc:2410] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 12.
I20260709 12:12:25.426595 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:25.443490 12679 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:25.401669 18476 consensus_queue.cc:1243] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 11.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:12:25.432032 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:25.438916 12677 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: [12.7-12.7]
I20260709 12:12:25.427012 12494 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:25.446612 18523 raft_consensus.cc:3055] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:25.441838 18427 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.447727 18640 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:25.448311 18540 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:25.385891 18199 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.449108 12680 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:25.454147 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.434937 18679 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:25.385429 18443 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:25.456092 12671 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:25.456769 12527 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:25.461236 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:25.455646 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:25.464046 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:25.471104 12678 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:12:25.394219 18695 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.475101 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.475553 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:25.475859 12677 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:25.476189 12606 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:12:25.471042 12605 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:25.462688 12672 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:25.466150 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:25.472345 12604 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 17: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:25.467358 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:25.402740 18467 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.489269 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:25.489941 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.490317 18427 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:25.490720 18250 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:25.490921 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:25.498857 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 107596us
I20260709 12:12:25.499085 12677 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:12:25.434538 18706 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:25.453483 18523 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:25.491943 12602 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 17: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:25.396304 18587 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.503211 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:25.444967 18559 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.034s
I20260709 12:12:25.495537 12494 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:25.492828 12678 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
W20260709 12:12:25.503676 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 120111us
I20260709 12:12:25.470283 18592 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:25.495175 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:25.527509 12526 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:25.467031 18540 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:25.503834 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:25.533151 12530 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 16: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:25.476572 18303 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:25.534227 18303 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:25.534667 18303 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:25.538333 18688 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:25.511970 12680 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:25.512239 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:25.503054 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:25.541097 18467 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:25.546386 18540 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.549026 18540 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:25.549669 18592 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.550796 18523 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.552094 18592 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:25.552217 18523 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:25.546432 18695 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:25.537868 18694 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:25.516589 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.560719 12677 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:25.561699 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.562721 12677 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:12:25.562978 18592 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:25.564024 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.565557 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.470894 18705 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:25.516624 12531 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:25.566862 18705 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.568389 18680 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:25.569608 18705 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:25.571235 18199 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:25.572851 18443 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:25.572321 18303 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.573975 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:25.575265 18499 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:25.575424 18303 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:25.576148 18586 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:25.498950 18597 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:25.500469 18677 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:25.577677 12602 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:12:25.578059 18677 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:25.578783 18677 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:25.531137 18341 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:25.579526 18341 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:25.579942 18341 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:25.583022 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:25.584122 12529 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 14: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:25.521498 12494 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:25.528407 12526 raft_consensus.cc:606] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:25.527655 12603 raft_consensus.cc:2376] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17. Re-sending same reply.
I20260709 12:12:25.529891 12674 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 10.
I20260709 12:12:25.526608 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:25.531049 12572 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:25.506043 18645 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:25.559808 12496 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:25.577726 18246 raft_consensus.cc:3055] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:25.578059 18597 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.604430 18246 raft_consensus.cc:740] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:25.503049 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.606478 18528 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:25.606652 12672 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 14: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:25.606962 18528 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:25.607165 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:25.607720 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:25.607827 12671 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:25.608440 12672 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:25.608122 18528 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.609362 18528 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:25.610466 18597 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:25.586787 12677 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:25.606963 18570 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:25.611713 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.612591 12677 raft_consensus.cc:2410] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 16 because replica has last-logged OpId of term: 13 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 9.
I20260709 12:12:25.613451 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:25.589937 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 68509.1us
I20260709 12:12:25.591677 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:25.615166 18586 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:25.615628 18672 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:25.616108 18672 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:25.616464 18672 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:25.618510 12606 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:32832
I20260709 12:12:25.618963 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.619197 12606 raft_consensus.cc:606] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:25.619624 12606 raft_consensus.cc:614] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:25.620405 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:25.621356 13302 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:25.622314 18666 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:12:25.622623 18587 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:25.622987 12494 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:25.623813 18666 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:25.624383 18666 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 17 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:25.624732 18617 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:25.625196 18617 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:25.625674 18617 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:25.627381 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:25.628429 18703 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:25.628656 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:25.625308 18666 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:25.599810 18645 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:25.619649 12680 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:25.630285 18645 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.631376 18645 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:25.633297 18499 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:25.633893 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:25.614063 12677 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:25.635105 18250 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:25.635679 18250 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.636085 18703 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:25.635993 18476 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:25.636742 18703 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:25.637694 18476 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:25.650957 18582 raft_consensus.cc:2764] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:12:25.637799 18703 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:25.631510 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:25.641034 18640 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:25.630529 12527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:25.531522 12679 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:25.633857 18499 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.647742 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:25.519477 18706 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:25.650203 13304 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:25.598667 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.602834 18592 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:25.661629 18511 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:25.662050 13334 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:12:25.661600 12678 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:25.655527 12680 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:12:25.637985 18250 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:25.664129 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:25.664912 12678 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:25.603808 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:25.667892 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:25.668608 12674 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:25.605472 18246 consensus_queue.cc:260] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.639180 12496 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:25.670794 18246 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:25.671118 18592 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:12:25.671859 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:25.672575 12496 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:25.672515 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:25.660458 18499 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:25.675067 18677 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:25.676338 12523 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 15: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:25.677021 18303 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260709 12:12:25.677495 18303 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:25.677865 18303 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:25.515578 12675 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:25.678970 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.655504 18476 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:25.679777 18476 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:25.680317 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:25.650595 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 128327us
W20260709 12:12:25.679201 18704 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.176s	user 0.000s	sys 0.004s
I20260709 12:12:25.643500 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:25.685271 13331 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 16: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:25.649160 18692 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:12:25.602269 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:25.640719 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:12:25.594523 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:25.681880 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:25.695621 12675 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:25.695624 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:25.697679 12646 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:25.635491 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:25.601419 12524 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 14: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:25.695024 12605 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 17: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:25.711249 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:25.712423 12679 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:25.713235 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:25.713202 13335 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 16: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:25.726670 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:25.728560 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.730773 12677 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:25.625779 18430 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:25.741389 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:25.742383 12606 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:25.743211 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.745213 12677 raft_consensus.cc:2410] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 14 because replica has last-logged OpId of term: 13 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:12:25.694324 18592 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:25.749295 18680 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:25.726881 18586 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:25.753283 18680 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:25.751072 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:25.751727 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:25.731989 18711 log.cc:927] Time spent T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.062s	user 0.000s	sys 0.001s
I20260709 12:12:25.750381 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:25.754305 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:12:25.790313 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:25.799955 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:25.813108 18699 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 16.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:12:25.826241 13302 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:25.830381 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:25.838140 18692 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:25.753844 18586 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:25.842900 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:25.844247 12427 catalog_manager.cc:5697] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 12 to 16. New cstate: current_term: 16 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:25.846066 12427 catalog_manager.cc:5697] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 9 to 15, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 15 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:12:25.792966 12598 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:25.847533 18430 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:25.752449 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:25.852644 12676 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.530s	user 0.017s	sys 0.001s
I20260709 12:12:25.853222 12676 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:12:25.846877 18680 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:25.856343 12598 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
W20260709 12:12:25.852764 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 180261us
I20260709 12:12:25.858821 12672 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:25.860193 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:25.858008 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:25.885569 12675 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:25.885807 13334 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:25.866372 12678 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:25.896044 18511 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:25.873392 12673 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:25.875635 12606 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
W20260709 12:12:25.876834 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 226755us
I20260709 12:12:25.877213 12671 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:25.878793 18246 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:25.879552 12496 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:25.963548 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:25.964021 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 84611.4us
I20260709 12:12:25.964215 12606 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:25.964754 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:25.862011 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:25.856629 18617 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:12:25.897346 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 199740us
I20260709 12:12:25.897746 12672 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:25.896281 13334 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:25.914570 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:25.916957 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:25.972040 12526 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:25.922497 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:25.977811 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.979542 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.980264 12675 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:25.980707 18692 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:25.981325 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.926591 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:25.981388 18692 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:25.982534 12525 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:25.927664 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:25.984138 12524 raft_consensus.cc:2393] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 16: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:25.925696 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:25.985347 12599 raft_consensus.cc:2368] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18.
I20260709 12:12:25.930908 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:25.968043 12569 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:25.986604 12523 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 15: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:25.792464 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 13 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.940217 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:25.941203 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:25.944648 12530 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:25.945873 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:25.948832 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.990279 12604 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:25.922391 18676 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:25.989857 12671 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:25.949637 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:25.950013 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:25.996296 18246 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:25.998654 18246 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:25.999121 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:25.999853 12602 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:25.961495 12674 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:25.922441 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:26.000991 12676 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:26.001240 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:26.001987 12674 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:26.002019 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:26.002506 12601 raft_consensus.cc:2393] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 15: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:26.003818 12676 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 17: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:25.961264 12679 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:25.985702 12646 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:25.966189 12571 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:26.007975 18586 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:25.927192 18680 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 12:12:26.010599 12427 catalog_manager.cc:5697] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 12 to 14, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 14 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:12:26.011633 18680 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:26.000042 18676 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:26.013088 18676 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:25.951459 18677 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:26.018967 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:26.019739 18430 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:26.020577 18430 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:26.022505 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:26.024542 12671 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:26.027631 18617 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:26.028378 18617 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:26.027786 18672 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:26.028789 18617 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:26.029291 18617 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:26.030468 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 64081.4us
W20260709 12:12:26.030664 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 64570us
I20260709 12:12:26.031332 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:26.030489 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:26.031858 13338 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 17: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:26.025874 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:26.035143 12530 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 17: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:26.035791 12527 raft_consensus.cc:606] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:26.036046 12672 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
I20260709 12:12:26.036320 12527 raft_consensus.cc:614] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:26.012406 12427 catalog_manager.cc:5697] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 9 to 12, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 12 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:12:26.037712 12672 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 12 index: 12. Preceding OpId from leader: term: 13 index: 13. (index mismatch)
I20260709 12:12:26.040318 12531 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:26.042726 12677 raft_consensus.cc:2393] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 13: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:26.043648 12676 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 12:12:26.044169 12676 pending_rounds.cc:85] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 12
I20260709 12:12:26.044538 12676 pending_rounds.cc:107] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 12:12:26.044255 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:26.044924 12676 raft_consensus.cc:2889] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:26.045321 12675 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:26.049634 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:26.050566 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 82609us
W20260709 12:12:26.052057 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:26.052597 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:26.034770 12604 pending_rounds.cc:85] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 6
I20260709 12:12:26.053869 12604 pending_rounds.cc:107] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 11.7
I20260709 12:12:26.054234 12604 raft_consensus.cc:2889] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:26.061275 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:26.033417 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:26.068463 12526 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:12:26.071416 12673 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:26.071826 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:26.072588 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:26.077091 12523 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:26.069670 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:26.083737 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:26.085330 12526 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:26.001241 18692 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:26.083212 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:26.080624 12674 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:12:26.086596 12530 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:26.092152 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:26.093559 12600 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:12:26.050504 18677 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:12:26.119285 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:26.098004 12673 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:26.050252 18246 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:26.072220 18714 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:26.094859 18511 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:26.105893 12571 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:26.068472 18221 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:12:25.999776 18679 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.095s
I20260709 12:12:26.110173 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:26.139187 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:26.103255 18694 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:26.104530 12525 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 17: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:26.129328 12680 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:26.130882 12679 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 15 index: 12. (index mismatch)
I20260709 12:12:26.012401 18680 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:26.135864 12571 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:26.111142 12606 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:26.157492 12674 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 12:12:26.137252 18645 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.505s	user 0.010s	sys 0.002s
I20260709 12:12:26.166600 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:26.173218 12571 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:26.127281 18666 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:26.142678 18341 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.562s	user 0.017s	sys 0.003s
I20260709 12:12:26.181449 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:26.108935 18587 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:12:26.136821 18221 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:26.146327 18706 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:26.178465 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:26.182364 12606 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:26.177925 12673 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 12:12:26.168210 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:26.196063 12525 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:26.196573 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 231893us
I20260709 12:12:26.203711 12602 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:26.153759 18714 raft_consensus.cc:697] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:26.154670 18694 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:26.197309 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:26.205065 12527 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:26.158892 18580 raft_consensus.cc:2764] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election decision vote started in defunct term 15: won
I20260709 12:12:26.216871 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:26.217576 12600 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:26.155622 18672 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:26.219669 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:26.221125 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:26.221736 12598 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:26.222625 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:26.223039 18528 consensus_meta.cc:287] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.613s	user 0.016s	sys 0.007s
I20260709 12:12:26.222894 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:26.223899 12602 raft_consensus.cc:2368] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16.
I20260709 12:12:26.225136 12496 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:26.226552 12496 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:26.226904 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:26.227875 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:26.228881 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:26.229589 12496 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:26.230480 12496 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:26.232223 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:26.223827 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:26.234421 18303 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.556s	user 0.010s	sys 0.002s
I20260709 12:12:26.235025 18303 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:26.214322 12527 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: [13.6-13.6]
I20260709 12:12:26.224102 18694 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:26.235962 18680 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:26.236752 18680 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:26.234344 18703 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:26.237200 18680 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:26.237546 18703 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:26.223779 18528 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:26.225574 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:26.239017 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:26.222916 12673 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: [13.6-13.6]
I20260709 12:12:26.240165 12525 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:26.210340 12679 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:12:26.240967 12524 raft_consensus.cc:2393] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 11: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:26.235265 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:26.203128 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 334344us
I20260709 12:12:26.179807 18582 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:26.207417 12530 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:26.245325 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:26.194414 18706 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:26.272504 12600 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
I20260709 12:12:26.282824 12527 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 13: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:26.280095 12598 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 13: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:26.248191 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:26.195446 18221 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:26.284039 12606 raft_consensus.cc:2410] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
W20260709 12:12:26.251580 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:26.238193 18703 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:26.242799 18528 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:26.213717 18430 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:26.339062 12673 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:12:26.284911 18706 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:26.352942 18680 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:26.358867 18719 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.155s	user 0.000s	sys 0.000s
W20260709 12:12:26.360338 18718 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:12:26.360780 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:26.363413 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:26.365383 12522 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:12:26.237496 18303 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:26.366351 12598 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:26.366415 18303 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:26.366794 12525 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:12:26.367163 18303 raft_consensus.cc:697] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:26.367673 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:26.193490 18341 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:26.369251 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:26.353538 12673 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:12:26.370833 12598 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:26.195955 18666 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:26.242425 18726 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:26.369772 12525 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:12:26.350581 12606 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:12:26.366154 18528 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:26.340507 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:26.361975 12603 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:26.364686 18714 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:26.374991 12527 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:26.339140 12530 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: [16.8-16.8]
I20260709 12:12:26.246057 18582 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:26.422879 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:26.424069 12606 raft_consensus.cc:2410] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
W20260709 12:12:26.421866 12643 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.627s	user 0.018s	sys 0.026s
W20260709 12:12:26.425459 12643 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.019s	sys 0.027s
I20260709 12:12:26.425619 12529 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 16. Ops: []
W20260709 12:12:26.425729 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 739215us
I20260709 12:12:26.450075 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:26.451103 12603 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:26.454176 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:26.368300 18303 consensus_queue.cc:237] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 10.8, Last appended by leader: 8, Current term: 13, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:26.462137 12598 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 17: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:12:26.464833 18476 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.785s	user 0.016s	sys 0.008s
I20260709 12:12:26.464475 18676 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:26.467556 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:26.469015 12603 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:26.377779 18692 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:12:26.475960 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:26.419425 18582 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:26.477154 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:26.479588 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:26.477057 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:26.480656 12606 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:26.402840 18723 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:26.465577 18476 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:26.453801 18430 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:26.537310 18592 raft_consensus.cc:3055] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:26.539947 12523 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
W20260709 12:12:26.491770 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:26.493893 12678 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 16: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:12:26.540942 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:26.540648 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 45085.4us
W20260709 12:12:26.480568 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 530640us
I20260709 12:12:26.542544 12643 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:26.501026 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:26.503309 18640 consensus_meta.cc:287] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.847s	user 0.009s	sys 0.001s
I20260709 12:12:26.506922 12531 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 16: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:26.508603 18476 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:26.522446 18679 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 12.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:26.484592 18729 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election won for term 14
W20260709 12:12:26.531100 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:26.546926 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 314762us
I20260709 12:12:26.553988 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:26.554247 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:26.531589 12606 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:26.535012 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:26.540318 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:26.498139 18721 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.260s	user 0.000s	sys 0.001s
I20260709 12:12:26.493762 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:26.537935 18592 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:26.568117 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:26.563040 12523 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
W20260709 12:12:26.544991 18523 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.992s	user 0.012s	sys 0.003s
I20260709 12:12:26.513141 18706 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:26.565536 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:26.574615 12525 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:26.482389 18692 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:26.568645 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:26.565181 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:26.570750 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:26.570762 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:26.566605 12603 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:26.589722 13338 raft_consensus.cc:2410] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 16 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 11.
I20260709 12:12:26.552781 12496 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:26.580513 13334 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:26.547411 18597 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:26.580513 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:26.582582 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:26.593900 12523 raft_consensus.cc:2393] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 11: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:12:26.596046 12526 consensus_meta.cc:287] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.510s	user 0.014s	sys 0.000s
I20260709 12:12:26.596616 12526 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:26.599191 12599 raft_consensus.cc:2410] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
W20260709 12:12:26.599467 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:26.592492 12496 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:26.599694 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 426564us
I20260709 12:12:26.604863 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:26.618100 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:26.542346 18430 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:26.567190 18695 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:26.620003 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:26.572906 18499 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:26.615778 12571 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:26.617583 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47264.1us
I20260709 12:12:26.635636 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:26.633855 12598 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:26.576372 18246 raft_consensus.cc:697] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:26.600399 18699 raft_consensus.cc:2764] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:12:26.643086 12605 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:12:26.644807 12525 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
W20260709 12:12:26.633334 18617 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.604s	user 0.012s	sys 0.000s
W20260709 12:12:26.643163 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 398854us
I20260709 12:12:26.651664 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:26.638159 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:26.650810 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:26.658752 12525 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:12:26.642192 18695 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:26.660660 12605 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:12:26.619138 18732 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:26.587266 18672 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
W20260709 12:12:26.663635 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:26.634471 18499 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:26.644300 18341 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:26.662554 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 89616.9us
W20260709 12:12:26.661188 18731 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:12:26.672591 18729 raft_consensus.cc:697] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:26.672899 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3038) took 1161 ms. Trace:
I20260709 12:12:26.674039 12605 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 12:12:26.688588 12605 pending_rounds.cc:85] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 8
I20260709 12:12:26.688867 12605 pending_rounds.cc:107] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 12:12:26.689086 12605 raft_consensus.cc:2889] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:26.670863 18511 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.010s
I20260709 12:12:26.679926 18246 consensus_queue.cc:237] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:26.682649 18499 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:26.691016 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3042) took 1149 ms. Trace:
I20260709 12:12:26.653781 18617 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:26.690827 12671 rpcz_store.cc:276] 0709 12:12:25.511642 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:26.025570 (+513928us) service_pool.cc:224] Handling call
0709 12:12:26.025945 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:26.034492 (+  8547us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:12:26.405584 (+371092us) log.cc:844] Serialized 14 byte log entry
0709 12:12:26.668052 (+262468us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:26.668134 (+    82us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:26.672626 (+  4492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:26.672699 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:26.672836 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":611414,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":44,"raft.queue_time_us":14349,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":91,"thread_start_us":592223,"threads_started":2}
W20260709 12:12:26.698312 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 96305.9us
I20260709 12:12:26.652221 18730 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 12:12:26.695152 18723 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:26.710671 13334 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 11 index: 9. Preceding OpId from leader: term: 13 index: 10. (index mismatch)
I20260709 12:12:26.703965 12675 rpcz_store.cc:276] 0709 12:12:25.541569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:26.046271 (+504702us) service_pool.cc:224] Handling call
0709 12:12:26.046685 (+   414us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:26.087123 (+ 40438us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:26.087168 (+    45us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:26.108257 (+ 21089us) log.cc:844] Serialized 52 byte log entry
0709 12:12:26.359340 (+251083us) spinlock_profiling.cc:243] Waited 8.64 ms on lock 0x7b4800060858. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:12:26.550067 (+190727us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:26.550116 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:26.550224 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:26.685704 (+135480us) raft_consensus.cc:1685] finished
0709 12:12:26.685733 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:26.690947 (+  5214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14918,"raft.run_cpu_time_us":3174,"raft.run_wall_time_us":263428,"spinlock_wait_cycles":19878144,"thread_start_us":365853,"threads_started":1,"wal-append.queue_time_us":537001}
I20260709 12:12:26.716625 12525 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:26.738487 12525 raft_consensus.cc:606] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:26.738875 12525 raft_consensus.cc:614] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:26.685660 18726 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:26.681399 18703 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:26.734359 12524 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 15: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:26.747939 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:26.748828 12678 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 17: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:26.735695 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:26.744231 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:26.749459 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:26.664866 18592 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:26.735117 18745 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:26.751088 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3041) took 1209 ms. Trace:
I20260709 12:12:26.759964 12677 rpcz_store.cc:276] 0709 12:12:25.541265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:26.043685 (+502420us) service_pool.cc:224] Handling call
0709 12:12:26.059858 (+ 16173us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:26.060081 (+   223us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:12:26.546425 (+486344us) log.cc:844] Serialized 14 byte log entry
0709 12:12:26.750595 (+204170us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:26.750674 (+    79us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:26.750742 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:26.750839 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:26.751013 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39001,"prepare.queue_time_us":611020,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":2581,"raft.queue_time_us":17679,"thread_start_us":631956,"threads_started":2}
W20260709 12:12:26.754304 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 116242us
I20260709 12:12:26.720642 18744 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:12:26.739326 18736 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.252s	user 0.001s	sys 0.000s
I20260709 12:12:26.765959 12571 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:26.767267 12526 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:26.751310 12572 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:26.770547 12571 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:26.772089 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:26.769780 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 119086us
I20260709 12:12:26.775107 12494 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:26.783743 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:12:26.783946 18523 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:26.784658 18523 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:26.759670 18714 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:26.788216 18753 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.002s
I20260709 12:12:26.741905 18680 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:12:26.785748 18523 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:26.769727 18499 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:26.793040 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46294.8us
I20260709 12:12:26.770092 18756 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:12:26.810086 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 74579.7us
W20260709 12:12:26.810623 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:26.773999 18744 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:26.818598 13334 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
W20260709 12:12:26.831223 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:26.836417 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:26.791163 18706 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:26.847666 12643 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:26.851683 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:26.852865 12601 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:12:26.867945 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46329.5us
I20260709 12:12:26.806526 18692 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:26.827555 18756 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:26.878456 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3040) took 1337 ms. Trace:
I20260709 12:12:26.881628 12672 rpcz_store.cc:276] 0709 12:12:25.540933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:26.038627 (+497694us) service_pool.cc:224] Handling call
0709 12:12:26.051376 (+ 12749us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:26.073242 (+ 21866us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:12:26.073276 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:26.096381 (+ 23105us) log.cc:844] Serialized 26 byte log entry
0709 12:12:26.203437 (+107056us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:26.204186 (+   749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:26.204579 (+   393us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:26.877942 (+673363us) raft_consensus.cc:1685] finished
0709 12:12:26.877985 (+    43us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:26.878370 (+   385us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21442,"raft.queue_time_us":36266,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":7205,"thread_start_us":39577,"threads_started":1,"wal-append.queue_time_us":242712}
I20260709 12:12:26.782613 18221 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:26.874275 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:26.889012 12601 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:26.877977 18250 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:12:26.891335 18744 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:26.898805 18748 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:12:26.899171 18679 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:26.818042 18723 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.7, Last appended by leader: 7, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:26.848198 18755 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:26.866806 18726 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:26.884702 18756 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:26.870949 18699 raft_consensus.cc:993] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: : Instructing follower 193a40da6e364ead8ab561e14c3cc8d1 to start an election
I20260709 12:12:26.799985 18750 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.071s
W20260709 12:12:26.898571 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 51060us
I20260709 12:12:26.840495 18430 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:26.790331 18729 consensus_queue.cc:237] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.5, Last appended by leader: 5, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:26.897527 18528 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:26.917258 12606 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 22: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:26.880860 18476 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:26.923246 18617 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:26.925134 18694 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.689s	user 0.010s	sys 0.001s
I20260709 12:12:26.926344 18250 consensus_queue.cc:1059] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:12:26.927856 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:26.878901 18752 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 12:12:26.932444 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:26.935498 18666 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:26.935822 18743 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:26.936350 12603 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:12:26.939779 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2671) took 1437 ms. Trace:
I20260709 12:12:26.941854 18756 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:26.943404 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:26.957238 12643 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:26.958659 12598 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:26.958909 12600 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:12:26.956887 12599 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
W20260709 12:12:26.957615 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 185606us
I20260709 12:12:26.906960 18706 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:26.955015 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:26.907536 18759 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:26.928155 18695 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:26.920303 18680 raft_consensus.cc:1081] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Signalling peer 193a40da6e364ead8ab561e14c3cc8d1 to start an election
I20260709 12:12:26.955361 12531 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:12:26.961691 12571 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:26.957832 12603 pending_rounds.cc:85] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 9
I20260709 12:12:26.988528 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:26.991468 12603 pending_rounds.cc:107] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:12:26.888013 18739 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 12:12:26.991984 12603 raft_consensus.cc:2889] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:27.015475 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:27.017768 12599 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:26.958166 12528 rpcz_store.cc:276] 0709 12:12:25.502341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:25.502863 (+   522us) service_pool.cc:224] Handling call
0709 12:12:25.513993 (+ 11130us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:25.514262 (+   269us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:12:25.963353 (+449091us) log.cc:844] Serialized 14 byte log entry
0709 12:12:26.939126 (+975773us) spinlock_profiling.cc:243] Waited 3.21 ms on lock 0x7b18003833e0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287
0709 12:12:26.939270 (+   144us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:26.939342 (+    72us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:26.939418 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:26.939511 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:26.939695 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":769599,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":58,"raft.queue_time_us":302199,"spinlock_wait_cycles":24900992,"thread_start_us":1156159,"threads_started":3,"wal-append.queue_time_us":667829}
W20260709 12:12:27.021545 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 146146us
I20260709 12:12:27.016294 12605 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:12:27.022742 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 61144.5us
W20260709 12:12:27.026389 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 157059us
I20260709 12:12:27.023682 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:27.028623 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:27.031668 12598 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
I20260709 12:12:27.032044 12571 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:27.033214 12571 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:27.039193 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:26.987231 18250 consensus_queue.cc:1243] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:12:26.957765 18587 raft_consensus.cc:2764] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election decision vote started in defunct term 13: won
I20260709 12:12:26.976750 18706 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:27.014986 18570 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:27.056277 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:27.057405 12605 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:26.987893 18759 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:27.046842 12427 catalog_manager.cc:5697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 12 to 20, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 20 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:12:27.047722 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2614) took 1129 ms. Trace:
I20260709 12:12:26.946205 18755 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.115s
I20260709 12:12:27.020215 18430 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:26.955277 18769 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:27.057610 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:27.046429 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:27.034255 18745 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:27.063161 12529 rpcz_store.cc:276] 0709 12:12:25.918326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:25.926227 (+  7901us) service_pool.cc:224] Handling call
0709 12:12:25.964046 (+ 37819us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:26.446763 (+482717us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:27.047483 (+600720us) spinlock_profiling.cc:243] Waited 461 ms on lock 0x7b54002107d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:27.047661 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1060891520}
I20260709 12:12:26.976353 18694 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:27.075106 18714 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:27.042166 18749 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:27.042155 18695 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:27.052945 18250 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:27.056695 18706 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:27.058763 18570 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:27.060623 18739 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:27.110483 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52960.7us
I20260709 12:12:27.067767 18759 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:26.986524 18743 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:27.092046 18582 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.597s	user 0.011s	sys 0.002s
I20260709 12:12:27.117873 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:27.050629 18666 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:12:27.101477 18749 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:27.121794 18767 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:27.126838 12494 leader_election.cc:400] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18.
I20260709 12:12:27.124264 18677 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:27.127326 18763 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:27.116011 18570 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:27.130522 18714 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:27.135298 18781 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:27.116689 18737 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:27.135226 18745 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:27.138862 12494 leader_election.cc:403] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:12:27.133952 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:27.127453 18760 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:27.117784 18617 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:27.121543 18739 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:27.159423 18677 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:27.164942 18747 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:12:27.147454 18749 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:27.077827 18746 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.086s
W20260709 12:12:27.191774 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 66845.9us
I20260709 12:12:27.138540 18582 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:27.095942 18430 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:27.168174 18743 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:27.121749 18580 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:12:27.176146 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:27.110282 18695 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:27.193511 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 160080us
I20260709 12:12:27.212896 12494 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:27.214732 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:27.215497 13337 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:27.215791 18430 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:27.216468 18430 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.002s
I20260709 12:12:27.161507 18587 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
I20260709 12:12:27.159123 18250 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.032s
I20260709 12:12:27.250491 12674 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:27.177770 18737 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:27.219777 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:27.190683 18767 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:12:27.204695 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 430657us
I20260709 12:12:27.222862 12598 raft_consensus.cc:2393] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:27.185180 18597 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:27.222848 18743 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.131s
I20260709 12:12:27.186928 18781 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:27.322096 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 60195.6us
I20260709 12:12:27.197933 18676 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:27.283381 18706 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:27.362537 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2935) took 1399 ms. Trace:
W20260709 12:12:27.218636 12556 heartbeater.cc:646] Failed to heartbeat to 127.11.13.190:46629 (3 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.11.13.190:46629 timed out after 15.000s (SENT)
I20260709 12:12:27.375294 12604 rpcz_store.cc:276] 0709 12:12:25.963459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:26.034213 (+ 70754us) service_pool.cc:224] Handling call
0709 12:12:26.034549 (+   336us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:26.111882 (+ 77333us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:26.111911 (+    29us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:26.124998 (+ 13087us) log.cc:844] Serialized 52 byte log entry
0709 12:12:26.566999 (+442001us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:26.567049 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:26.567139 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:27.362234 (+795095us) raft_consensus.cc:1685] finished
0709 12:12:27.362263 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:27.362453 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24858,"raft.queue_time_us":24988,"raft.run_cpu_time_us":202,"raft.run_wall_time_us":13120,"thread_start_us":387177,"threads_started":1,"wal-append.queue_time_us":983472}
I20260709 12:12:27.257674 18737 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:27.401366 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3043) took 1832 ms. Trace:
W20260709 12:12:27.354812 18783 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 12:12:27.219547 18528 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:27.169700 18763 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:27.275364 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:27.357270 18781 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:27.370281 12599 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:27.362874 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2941) took 1365 ms. Trace:
W20260709 12:12:27.368093 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 379659us
W20260709 12:12:27.368434 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 182216us
I20260709 12:12:27.373073 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:27.383252 13338 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:27.319610 18680 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:12:27.406648 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 47345.2us
I20260709 12:12:27.401937 12676 rpcz_store.cc:276] 0709 12:12:25.541884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:26.046476 (+504592us) service_pool.cc:224] Handling call
0709 12:12:26.047070 (+   594us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:26.104063 (+ 56993us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:12:26.527593 (+423530us) spinlock_profiling.cc:243] Waited 7.15 ms on lock 0x7b4800060858. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:12:26.559791 (+ 32198us) log.cc:844] Serialized 14 byte log entry
0709 12:12:27.097060 (+537269us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:27.169470 (+ 72410us) log.cc:844] Serialized 26 byte log entry
0709 12:12:27.169695 (+   225us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:27.169732 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:27.169805 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:27.362589 (+192784us) raft_consensus.cc:1685] finished
0709 12:12:27.373667 (+ 11078us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:27.373832 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":38263,"prepare.queue_time_us":582640,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":57068,"raft.run_cpu_time_us":14453,"raft.run_wall_time_us":450444,"spinlock_wait_cycles":16454016,"thread_start_us":880679,"threads_started":3,"wal-append.queue_time_us":558618}
I20260709 12:12:27.211050 18617 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18.
I20260709 12:12:27.365667 18640 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:27.424660 12597 rpcz_store.cc:276] 0709 12:12:25.997927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:26.285167 (+287240us) service_pool.cc:224] Handling call
0709 12:12:26.285629 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:26.286098 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:26.286177 (+    79us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:26.288773 (+  2596us) log.cc:844] Serialized 26 byte log entry
0709 12:12:26.670074 (+381301us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:26.670993 (+   919us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:26.671173 (+   180us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:27.362666 (+691493us) raft_consensus.cc:1685] finished
0709 12:12:27.362686 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:27.362814 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":86385,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":7811,"thread_start_us":315341,"threads_started":1,"wal-append.queue_time_us":688168}
I20260709 12:12:27.425105 13335 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:27.431829 18640 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:27.353091 18597 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:27.435508 18597 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:27.436810 18597 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:27.367455 18679 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:27.210148 18760 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:27.440006 18760 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:27.274724 18756 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:27.367189 18767 raft_consensus.cc:697] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:27.351553 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:27.404114 18246 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:27.443374 18246 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:27.444312 18246 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:27.443902 18767 consensus_queue.cc:237] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:27.445643 18246 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:27.450470 13335 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:12:27.417752 18769 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:27.481931 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 54890.1us
I20260709 12:12:27.448212 18781 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:27.503145 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:27.504571 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:27.507107 13337 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:12:27.509882 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:27.510591 13336 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:27.468787 18640 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:27.489742 18597 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:27.481742 18694 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:27.527725 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 313750us
I20260709 12:12:27.503247 18680 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:27.518064 18739 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:27.579751 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:27.582088 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:27.583069 12675 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 10 FOLLOWER]: Advancing to term 11
I20260709 12:12:27.584878 12677 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:27.535787 18582 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:27.591650 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:27.573559 18720 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.040s	user 0.003s	sys 0.004s
I20260709 12:12:27.511492 18781 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:27.595201 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:27.595947 12679 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:27.600554 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:27.604175 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:27.605041 12677 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:27.604074 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:27.608204 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 153751us
I20260709 12:12:27.597054 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:27.609373 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:27.609301 12678 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:27.611119 12673 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:12:27.631268 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:27.642765 18794 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.196s	user 0.001s	sys 0.000s
I20260709 12:12:27.575289 18677 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:27.652294 12427 catalog_manager.cc:5697] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 12 to 15, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 15 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:12:27.134433 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:27.653702 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:27.667695 18706 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:27.668780 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2685) took 1936 ms. Trace:
I20260709 12:12:27.682710 12674 rpcz_store.cc:276] 0709 12:12:25.732965 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:26.158517 (+425552us) service_pool.cc:224] Handling call
0709 12:12:26.191380 (+ 32863us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:27.251354 (+1059974us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:27.668464 (+417110us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b54003bd154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:27.668706 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2379814272}
I20260709 12:12:27.670466 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2675) took 1116 ms. Trace:
I20260709 12:12:27.683707 12524 rpcz_store.cc:276] 0709 12:12:26.553864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:26.553993 (+   129us) service_pool.cc:224] Handling call
0709 12:12:27.670215 (+1116222us) spinlock_profiling.cc:243] Waited 179 ms on lock 0x7b540007ce54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:27.670368 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":411972864}
I20260709 12:12:27.588454 18739 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:27.671518 18769 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:27.675002 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:27.691115 18739 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:27.678350 12569 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:27.701555 12679 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:12:27.703086 18587 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:27.703179 18677 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:27.696987 18680 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:27.688493 18785 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.182s	user 0.006s	sys 0.000s
I20260709 12:12:27.704881 18763 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:27.658819 18676 consensus_queue.cc:1059] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.037s
W20260709 12:12:27.706184 18499 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.879s	user 0.014s	sys 0.003s
I20260709 12:12:27.677938 12675 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 11.
W20260709 12:12:27.714316 18728 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.052s	user 0.003s	sys 0.000s
I20260709 12:12:27.723716 12677 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:27.691579 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:12:27.735771 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 44272.1us
W20260709 12:12:27.709841 18793 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.201s	user 0.000s	sys 0.004s
W20260709 12:12:27.750253 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:27.758458 18499 raft_consensus.cc:3055] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Stepping down as leader of term 12
W20260709 12:12:27.776124 18789 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:12:27.770610 18499 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:27.799783 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2684) took 2087 ms. Trace:
I20260709 12:12:27.800382 12680 rpcz_store.cc:276] 0709 12:12:25.712771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:26.152215 (+439444us) service_pool.cc:224] Handling call
0709 12:12:26.152652 (+   437us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:26.621404 (+468752us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:26.621461 (+    57us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:26.654884 (+ 33423us) log.cc:844] Serialized 52 byte log entry
0709 12:12:27.039985 (+385101us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:27.040023 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:27.040098 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:27.799469 (+759371us) raft_consensus.cc:1685] finished
0709 12:12:27.799498 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:27.799702 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23386,"raft.queue_time_us":76051,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":35714,"thread_start_us":779972,"threads_started":2,"wal-append.queue_time_us":909366}
I20260709 12:12:27.802815 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:27.803898 12680 raft_consensus.cc:2410] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 17 because replica has last-logged OpId of term: 15 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 11.
I20260709 12:12:27.769884 18807 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:27.820113 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:27.821192 12674 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 16: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:27.812990 18786 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:27.811990 18807 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:27.823657 18759 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.676s	user 0.013s	sys 0.000s
I20260709 12:12:27.843225 18786 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:27.851639 18799 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
W20260709 12:12:27.859279 18809 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:12:27.845888 18807 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:27.865619 18570 consensus_meta.cc:287] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.680s	user 0.011s	sys 0.005s
W20260709 12:12:27.874312 18792 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 12:12:27.851186 18759 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:27.882872 18570 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:27.890636 18742 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:27.891987 12496 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:27.892588 18729 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:27.893690 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:27.901517 13337 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:27.894348 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:27.892036 18645 raft_consensus.cc:3055] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:27.903882 18430 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:27.904176 18645 raft_consensus.cc:740] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:27.904621 18580 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:12:27.905642 18580 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:27.900848 12496 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:27.903122 18729 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:27.907004 18810 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:12:27.909013 18729 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:27.909194 18745 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:27.903518 18766 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:27.903442 18784 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260709 12:12:27.910516 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 349834us
I20260709 12:12:27.911015 18781 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:27.906080 18580 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:27.908793 18742 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:27.898633 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 547172us
W20260709 12:12:27.929716 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 251497us
I20260709 12:12:27.930299 12572 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:27.905187 18645 consensus_queue.cc:260] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:27.917388 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:27.916200 12494 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:27.890772 18586 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:27.930327 12496 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:27.914232 18805 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:12:27.936036 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2940) took 1938 ms. Trace:
I20260709 12:12:27.936743 12602 rpcz_store.cc:276] 0709 12:12:25.997658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:26.246527 (+248869us) service_pool.cc:224] Handling call
0709 12:12:26.342381 (+ 95854us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:26.709312 (+366931us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:12:26.962121 (+252809us) log.cc:844] Serialized 14 byte log entry
0709 12:12:27.896195 (+934074us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b180036d480. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:12:27.896338 (+   143us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:27.901806 (+  5468us) log.cc:844] Serialized 26 byte log entry
0709 12:12:27.902018 (+   212us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:27.902077 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:27.902248 (+   171us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:27.916692 (+ 14444us) raft_consensus.cc:1685] finished
0709 12:12:27.935780 (+ 19088us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:27.935959 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":103069,"prepare.queue_time_us":457709,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":1869,"raft.queue_time_us":851568,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":6315,"spinlock_wait_cycles":24734080,"thread_start_us":783732,"threads_started":3,"wal-append.queue_time_us":352487}
I20260709 12:12:27.943073 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:27.945324 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:27.948467 12597 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:12:27.949041 12494 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:27.927625 18582 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.006s
I20260709 12:12:27.951391 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:27.957535 12604 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:12:27.956530 12606 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:27.877676 18737 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:27.955574 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:27.961452 12605 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:12:27.934006 18640 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:27.969149 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:27.940876 18694 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:27.931648 18706 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.033s
I20260709 12:12:27.956656 18729 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Leader election won for term 11
I20260709 12:12:27.963737 18726 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:27.972952 18523 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:12:27.959649 18430 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:27.985018 12599 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 14. (index mismatch)
I20260709 12:12:27.987412 12572 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:27.960515 18803 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:27.987418 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:27.990631 18815 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:12:27.991323 18703 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:27.992862 18742 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:27.941216 18586 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.025683 18796 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:12:28.027449 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 63974.4us
I20260709 12:12:28.023126 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:28.028676 12601 raft_consensus.cc:2376] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16. Re-sending same reply.
I20260709 12:12:28.027804 18586 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:27.965794 18640 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:28.030054 18640 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:28.027828 12572 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:28.031716 18796 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:28.032099 18796 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:28.032437 12572 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:28.032436 18796 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:27.999125 18784 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:27.998843 18729 raft_consensus.cc:697] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:28.049635 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 119396us
I20260709 12:12:27.979724 18694 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:28.056011 18570 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:28.057296 18739 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:28.058249 18791 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:12:28.058755 18813 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:12:28.059123 18772 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:27.978353 18745 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.058s
I20260709 12:12:28.063014 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:28.091290 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:28.034453 18528 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:28.022019 18580 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.072279 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:28.096922 12602 raft_consensus.cc:2376] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17. Re-sending same reply.
I20260709 12:12:28.035648 18523 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.046s
I20260709 12:12:28.082052 12604 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:12:28.041869 18703 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:28.022410 18767 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 12.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.049s
I20260709 12:12:28.022854 18430 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:28.093973 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 61585.6us
I20260709 12:12:28.095446 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:28.100571 12643 leader_election.cc:336] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.555s (SENT)
I20260709 12:12:28.083911 12675 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:51308
I20260709 12:12:28.104964 12675 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:28.105373 12675 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:28.106954 18737 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:28.108600 18430 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:28.101773 18703 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:28.109438 18775 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:12:28.109498 18703 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:28.114300 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:28.114977 12680 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:28.116519 18796 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.118268 18703 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.119402 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:28.121129 12674 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
W20260709 12:12:28.122282 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 59421.4us
I20260709 12:12:28.123369 18680 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
W20260709 12:12:28.123687 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:28.123827 18680 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:28.124185 18680 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:28.126909 18586 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:28.127409 18586 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.129452 18586 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:28.091495 18772 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:28.142710 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:28.155462 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:28.155881 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:28.156988 12525 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:12:28.093672 18791 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:28.162380 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:28.163690 12525 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 16: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:28.165759 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:28.159696 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:28.167963 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:28.168622 12530 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:28.168841 12522 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 14. Ops: []
I20260709 12:12:28.166503 12525 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:28.101578 18528 raft_consensus.cc:697] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:28.172510 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:28.160454 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:28.093111 18752 consensus_queue.cc:1059] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:12:28.099761 18746 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 12:12:28.184598 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:28.154939 18772 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:28.194594 12531 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:28.193253 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:28.197533 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:28.183146 12680 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:28.203699 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:28.203882 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:28.200863 12646 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:28.205837 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:28.164746 18759 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:28.219302 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:28.216166 12601 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:28.196913 12675 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.217182 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:28.217410 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:28.305362 12602 raft_consensus.cc:2410] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:12:28.218032 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:28.305752 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:28.219110 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 11 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:28.306572 12605 raft_consensus.cc:2376] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15. Re-sending same reply.
I20260709 12:12:28.306972 12604 raft_consensus.cc:2393] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 11: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:28.218371 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:28.307592 12680 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:12:28.307744 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:28.308324 12604 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:12:28.308812 12680 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:28.220297 12531 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:28.309163 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:28.197898 18476 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:12:28.223831 18570 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:28.235808 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 2951) took 1356 ms. Trace:
I20260709 12:12:28.239351 18740 raft_consensus.cc:3055] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:28.242479 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:28.266644 18819 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.085s
I20260709 12:12:28.246395 18832 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:28.250691 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:28.229403 18749 consensus_meta.cc:287] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.021s	user 0.013s	sys 0.005s
I20260709 12:12:28.259454 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:28.227325 18739 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.041s
I20260709 12:12:28.274107 18766 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:28.277308 18758 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:28.284092 18828 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:28.284932 18820 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:28.285396 18592 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:28.285578 18821 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:28.287662 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:28.251675 18694 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:28.229450 18587 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.049s
I20260709 12:12:28.184281 18729 consensus_queue.cc:237] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 9.6, Last appended by leader: 6, Current term: 11, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.290731 18703 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:28.230747 18781 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.051s
I20260709 12:12:28.304349 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:28.326097 12604 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 17: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:28.326375 12680 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:12:28.326273 12646 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:28.420284 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 94144.8us
W20260709 12:12:28.340076 18597 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.798s	user 0.008s	sys 0.007s
I20260709 12:12:28.327620 12679 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:28.310141 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:28.430783 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:28.306583 12606 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:28.329310 12674 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:28.328430 12598 rpcz_store.cc:276] 0709 12:12:26.879601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:27.038901 (+159300us) service_pool.cc:224] Handling call
0709 12:12:28.235501 (+1196600us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b540023fd54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:28.235717 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":419931904}
I20260709 12:12:28.326646 12599 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 17: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:28.328266 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:28.325598 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:28.432324 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:28.305658 18797 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:28.310817 18826 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.147s	user 0.000s	sys 0.001s
I20260709 12:12:28.309994 18756 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:28.407918 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:28.322135 18796 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:28.397993 18246 consensus_meta.cc:287] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.952s	user 0.017s	sys 0.000s
W20260709 12:12:28.421451 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:28.327193 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:28.393798 18766 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:28.475502 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:28.475615 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:28.476073 12605 raft_consensus.cc:2376] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15. Re-sending same reply.
I20260709 12:12:28.476565 12599 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:28.477200 12605 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:28.423780 12680 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:12:28.479174 12605 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: [12.6-12.6]
I20260709 12:12:28.328619 18726 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:28.431455 12679 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:28.331710 12572 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:28.433121 12606 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 13: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:28.327039 18740 raft_consensus.cc:740] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:28.434688 12597 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:28.434736 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:28.495126 12605 raft_consensus.cc:3055] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:28.437776 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:28.495935 12602 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:28.496446 12604 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:28.440141 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:28.302492 18528 consensus_queue.cc:237] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.473349 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:28.436812 18570 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:28.497336 12531 raft_consensus.cc:3055] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:28.497900 18570 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:28.482160 18597 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:28.485395 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2942) took 2487 ms. Trace:
I20260709 12:12:28.395324 18807 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:28.490993 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:28.492820 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 187968us
I20260709 12:12:28.511576 12598 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:28.398623 18476 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:28.496845 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 192585us
I20260709 12:12:28.458473 18797 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.497860 12531 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:28.499701 12599 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:12:28.394016 18749 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.504276 12600 rpcz_store.cc:276] 0709 12:12:25.998226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:26.348860 (+350634us) service_pool.cc:224] Handling call
0709 12:12:26.349293 (+   433us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:26.974826 (+625533us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:28.485092 (+1510266us) spinlock_profiling.cc:243] Waited 609 ms on lock 0x7b54002715d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:28.485323 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1401425664}
I20260709 12:12:28.530220 18788 raft_consensus.cc:3055] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:28.530666 18788 raft_consensus.cc:740] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:28.531555 18788 consensus_queue.cc:260] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 13.6, Last appended by leader: 6, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.532874 18788 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:28.490777 12672 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:28.536687 13330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 774) took 1720 ms. Trace:
I20260709 12:12:28.506060 12605 raft_consensus.cc:740] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:28.395466 18832 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.537788 12605 consensus_queue.cc:260] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.538977 12605 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:28.539093 18476 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:28.542510 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2704) took 2185 ms. Trace:
I20260709 12:12:28.524642 12531 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.542912 12603 rpcz_store.cc:276] 0709 12:12:26.356905 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:26.649364 (+292459us) service_pool.cc:224] Handling call
0709 12:12:26.935954 (+286590us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:27.039300 (+103346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:28.541975 (+1502675us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b5400374950. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:28.542354 (+   379us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":627672320}
I20260709 12:12:28.543365 12531 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:28.398382 18758 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:28.527396 18836 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:12:28.528252 18786 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:28.523496 18740 consensus_queue.cc:260] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:28.544196 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 630031us
I20260709 12:12:28.544443 12677 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:28.527014 18221 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:28.549811 12671 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:28.499286 18796 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:28.550886 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 674) took 1898 ms. Trace:
I20260709 12:12:28.565896 12526 rpcz_store.cc:276] 0709 12:12:26.652516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:26.661789 (+  9273us) service_pool.cc:224] Handling call
0709 12:12:26.662098 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:26.770483 (+108385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:28.550572 (+1780089us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b5400015c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:28.550805 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":241282432}
I20260709 12:12:28.537063 13330 rpcz_store.cc:276] 0709 12:12:26.825637 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:26.834155 (+  8518us) service_pool.cc:224] Handling call
0709 12:12:26.834540 (+   385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:26.834794 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:12:27.173362 (+338568us) log.cc:844] Serialized 14 byte log entry
0709 12:12:27.414021 (+240659us) log.cc:844] Serialized 14 byte log entry
0709 12:12:27.415035 (+  1014us) log.cc:844] Serialized 14 byte log entry
0709 12:12:28.536114 (+1121079us) spinlock_profiling.cc:243] Waited 4.29 ms on lock 0x7b1800251440. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:12:28.536256 (+   142us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:28.536309 (+    53us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:28.536355 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:28.536425 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:28.536637 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2752,"prepare.queue_time_us":537934,"prepare.run_wall_time_us":85,"raft.queue_time_us":3724,"raft.run_wall_time_us":14683,"spinlock_wait_cycles":140094976,"thread_start_us":543519,"threads_started":2,"wal-append.queue_time_us":458772}
I20260709 12:12:28.304144 18791 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:28.495908 18756 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.555455 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:28.472340 18680 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.569082 12605 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:12:28.402333 18828 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.573153 12531 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:28.521996 18746 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:28.497083 18640 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:28.588084 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:28.554047 18523 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:28.551913 18694 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:28.586972 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:28.589713 12643 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:28.595084 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 1.14499e+06us
I20260709 12:12:28.595800 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:28.595960 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:28.597770 12643 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:28.601025 12522 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:12:28.600373 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:28.602496 12530 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 14: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:28.603479 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:28.583669 18796 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:28.589339 18740 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:28.563645 18597 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:28.609664 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:28.610937 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:28.615272 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:28.613076 12676 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 5. Preceding OpId from leader: term: 14 index: 6. (index mismatch)
I20260709 12:12:28.616564 12526 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:28.617935 12529 raft_consensus.cc:2393] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 16: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:28.619282 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:28.620496 12643 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:28.569964 18303 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
I20260709 12:12:28.632964 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:28.636029 18840 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:28.636821 18840 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:28.636929 12643 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:28.638378 18743 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:28.640462 18828 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:28.640646 12531 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: [14.9-14.9]
W20260709 12:12:28.641503 18844 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.126s	user 0.001s	sys 0.000s
I20260709 12:12:28.642534 12529 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:12:28.631399 18597 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.641386 12526 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:28.644830 18597 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:28.646587 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:28.647544 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:28.648490 18840 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:28.650485 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:28.651300 18840 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:28.651700 18840 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:28.652026 18840 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:28.599654 18807 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:28.607317 18796 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:28.612262 18781 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:28.594056 18746 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:28.661731 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 96984.1us
I20260709 12:12:28.598649 18523 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.664090 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:28.591619 12679 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:28.633318 12676 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 17 index: 8. (index mismatch)
I20260709 12:12:28.686440 12676 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:28.633845 18739 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:28.601318 18694 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:28.634020 18246 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260709 12:12:28.655323 18819 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:28.702186 12643 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:28.702932 12679 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:12:28.587493 18726 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:28.733386 18523 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:28.735157 18680 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:28.744065 18694 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:28.725832 18739 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:12:28.746147 18841 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
I20260709 12:12:28.713383 18781 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:12:28.750814 18797 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:28.751761 18797 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:28.752552 18847 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:28.753064 18847 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:28.753410 18847 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:28.713912 18819 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:28.757032 18819 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:28.617687 18570 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.666191 18597 consensus_queue.cc:1059] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
W20260709 12:12:28.779261 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 198205us
I20260709 12:12:28.751789 18781 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:28.789656 13336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38482 (request call id 614) took 1025 ms. Trace:
I20260709 12:12:28.779659 18597 consensus_queue.cc:1243] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 13.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:12:28.820638 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:28.805995 13336 rpcz_store.cc:276] 0709 12:12:27.764212 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:27.764358 (+   146us) service_pool.cc:224] Handling call
0709 12:12:27.771242 (+  6884us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:27.771460 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:12:27.830061 (+ 58601us) log.cc:844] Serialized 14 byte log entry
0709 12:12:28.788889 (+958828us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:28.788966 (+    77us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:28.789292 (+   326us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:28.789390 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:28.789571 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":713550,"prepare.queue_time_us":85745,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":60,"raft.queue_time_us":958471,"thread_start_us":290185,"threads_started":2}
I20260709 12:12:28.812176 18781 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:28.832263 18781 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:28.746037 18739 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.846494 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:28.849823 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:28.854846 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:28.863168 18854 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:28.863770 18782 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.500s	user 0.004s	sys 0.001s
I20260709 12:12:28.864473 18854 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:28.869856 18847 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.871837 18749 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:28.882691 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 677) took 1945 ms. Trace:
I20260709 12:12:28.883113 12527 rpcz_store.cc:276] 0709 12:12:26.937107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:26.937293 (+   186us) service_pool.cc:224] Handling call
0709 12:12:26.937677 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:26.974821 (+ 37144us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:26.974858 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:26.975517 (+   659us) log.cc:844] Serialized 26 byte log entry
0709 12:12:27.098993 (+123476us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:12:27.099130 (+   137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:27.099334 (+   204us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:28.862908 (+1763574us) spinlock_profiling.cc:243] Waited 2.41 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:28.882356 (+ 19448us) raft_consensus.cc:1685] finished
0709 12:12:28.882384 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:28.882630 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17944,"raft.queue_time_us":19700,"raft.run_cpu_time_us":228,"raft.run_wall_time_us":15063,"spinlock_wait_cycles":5538688,"thread_start_us":63362,"threads_started":1,"wal-append.queue_time_us":284978}
I20260709 12:12:28.885141 18821 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:28.849419 18640 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:28.895527 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 221363us
W20260709 12:12:28.875957 18837 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.349s	user 0.000s	sys 0.000s
I20260709 12:12:28.873215 18847 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:28.863835 18796 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:28.896586 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:28.947772 12676 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:28.854652 18756 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:28.876353 18854 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:28.856806 18858 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:28.902007 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 199985us
I20260709 12:12:28.961402 12643 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:28.905995 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:28.960057 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:28.949728 18821 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:28.989907 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:28.994503 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:28.979628 18592 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.571s	user 0.005s	sys 0.008s
I20260709 12:12:28.965747 18781 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:28.986429 18740 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:28.994282 18786 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:29.018919 18746 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:28.971143 18858 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:28.996330 18766 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:29.028789 18847 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:28.997151 18821 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:29.051589 18850 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.020s
I20260709 12:12:29.037637 18740 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:29.087306 18586 raft_consensus.cc:3055] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:29.087954 18586 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:29.088905 18586 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:29.090710 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:29.091887 18838 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:29.095818 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:29.066907 18695 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:29.104113 18838 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:29.071103 18862 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:29.066256 18847 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:29.121544 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:29.121133 12496 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:29.130623 18796 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:29.146881 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:29.147583 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:29.150324 12496 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:29.107564 18592 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:29.155467 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:29.146982 18856 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:12:29.151894 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:29.159538 12526 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:29.163220 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:29.164089 12527 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 16: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:12:29.166289 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:29.167214 18570 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:29.167933 18570 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:29.168443 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:29.169497 12672 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 16: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:29.169847 18839 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:29.170537 18839 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:29.171499 18839 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:29.171988 18570 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:29.172895 18570 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:29.172996 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:29.173395 18570 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:29.176954 18645 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:29.177899 18570 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:29.178920 18570 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:29.179363 18570 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:29.182614 18666 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:29.183276 18666 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:29.183775 18570 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:29.185568 18666 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:29.185847 18570 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:29.186569 18570 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:29.192488 18570 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:29.193324 18861 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
I20260709 12:12:29.202811 18511 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:29.156224 12522 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:29.144184 18586 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:29.170001 18743 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:29.206902 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:29.218222 12676 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:29.211524 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:29.212339 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:29.158109 12524 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:12:29.213503 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:29.223989 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:29.224920 12527 raft_consensus.cc:2410] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 19 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:12:29.219643 12603 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
W20260709 12:12:29.225646 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 73822.1us
I20260709 12:12:29.229826 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:29.151510 18617 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:29.230829 12603 raft_consensus.cc:2410] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
W20260709 12:12:29.158402 18860 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.197s	user 0.002s	sys 0.000s
I20260709 12:12:29.236078 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:29.238173 12528 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 17: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:29.231966 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:29.231896 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:29.244705 12676 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:29.245509 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:29.236078 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:29.246414 12672 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:29.248185 12496 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:29.249426 12603 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 14: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:29.251165 12496 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:29.251966 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:29.253248 12599 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 13: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:29.312009 12605 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 7. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:12:29.238395 18743 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:29.317359 18866 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.172s	user 0.000s	sys 0.000s
I20260709 12:12:29.255897 18723 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260709 12:12:29.318419 18743 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:29.319535 18723 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:29.320021 18723 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:29.320987 18680 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:12:29.240769 18753 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.081s
I20260709 12:12:29.236778 18476 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.086s
I20260709 12:12:29.322583 12680 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 12. (term mismatch)
I20260709 12:12:29.322976 12680 pending_rounds.cc:85] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 11
I20260709 12:12:29.323199 12680 pending_rounds.cc:107] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 15.12
I20260709 12:12:29.239506 18834 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:29.323443 12680 raft_consensus.cc:2889] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:29.323694 18858 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:29.324257 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:29.325104 18740 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:29.325608 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:29.335626 12673 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:29.329620 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:29.330866 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:29.332010 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:29.339430 12600 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:29.320171 12599 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:29.328260 12678 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:12:29.345163 12606 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:29.348743 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:29.353281 12672 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:12:29.254530 18756 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:12:29.354164 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:29.355033 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:29.357848 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:29.359051 12672 raft_consensus.cc:2393] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 16: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:29.320997 18723 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:29.247354 18752 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.110s
I20260709 12:12:29.323727 18834 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:29.346792 18868 log.cc:927] Time spent T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:12:29.369732 12600 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:29.369513 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:29.400403 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:29.373057 12606 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 6. Preceding OpId from leader: term: 14 index: 6. (term mismatch)
I20260709 12:12:29.401206 12672 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 16: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:29.401288 12606 pending_rounds.cc:85] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 5
I20260709 12:12:29.401744 12606 pending_rounds.cc:107] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 13.6
I20260709 12:12:29.402153 12606 raft_consensus.cc:2889] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:29.373148 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:29.370471 18756 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:29.438468 18756 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 14.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:29.442469 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:29.383762 18499 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:29.443804 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:29.444700 12606 raft_consensus.cc:2410] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:12:29.444530 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:29.445619 18850 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:29.361110 18772 consensus_meta.cc:287] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.153s	user 0.019s	sys 0.004s
I20260709 12:12:29.399497 18834 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:29.446084 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:29.446538 18772 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:29.446117 18850 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:29.448755 18772 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:29.254635 18640 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.106s
W20260709 12:12:29.383948 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:29.451543 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:29.452735 12646 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:29.453657 12646 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:29.384804 18821 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:29.453962 13337 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:29.454447 18821 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:29.455034 18821 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:29.457242 18821 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:29.270643 18303 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.097s
I20260709 12:12:29.386607 18847 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:29.393115 18819 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.636s	user 0.004s	sys 0.008s
I20260709 12:12:29.460724 13334 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 9. Preceding OpId from leader: term: 13 index: 9. (term mismatch)
I20260709 12:12:29.397264 18867 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:29.461287 13334 pending_rounds.cc:85] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 8
I20260709 12:12:29.398312 18666 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:29.461359 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:29.402168 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:29.463220 13332 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 16 index: 12. (index mismatch)
I20260709 12:12:29.458406 18250 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:29.385294 18766 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:29.460529 18819 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:29.393888 18840 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.741s	user 0.012s	sys 0.004s
I20260709 12:12:29.486363 18840 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:29.487461 18303 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:29.488103 18303 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:29.484728 18666 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:29.489429 18840 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:29.398907 18858 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:29.490314 18666 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:29.490733 18858 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:29.491739 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3067) took 1416 ms. Trace:
I20260709 12:12:29.492138 12674 rpcz_store.cc:276] 0709 12:12:28.075238 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:28.121784 (+ 46546us) service_pool.cc:224] Handling call
0709 12:12:28.122100 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:28.434491 (+312391us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:29.491443 (+1056952us) spinlock_profiling.cc:243] Waited 207 ms on lock 0x7b5400013bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:29.491670 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":475760000}
I20260709 12:12:29.493057 18858 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:29.494495 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:29.495139 12606 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:29.466598 12646 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:29.474434 13335 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:29.466455 13331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 662) took 1984 ms. Trace:
W20260709 12:12:29.497794 12494 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.132:45801 timed out after 2.133s (SENT) [suppressed 24 similar messages]
I20260709 12:12:29.509799 12606 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:29.461704 13334 pending_rounds.cc:107] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 12.9
I20260709 12:12:29.483619 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2696) took 1300 ms. Trace:
I20260709 12:12:29.462091 18676 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:29.485466 12528 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:29.511415 12530 rpcz_store.cc:276] 0709 12:12:28.183470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:28.418657 (+235187us) service_pool.cc:224] Handling call
0709 12:12:29.483408 (+1064751us) spinlock_profiling.cc:243] Waited 105 ms on lock 0x7b54003c0854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:29.483556 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":240798080}
I20260709 12:12:29.499207 18828 raft_consensus.cc:3055] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:12:29.499990 18769 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 15 current term: 15 is higher.
I20260709 12:12:29.502313 13331 rpcz_store.cc:276] 0709 12:12:27.481876 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:27.482203 (+   327us) service_pool.cc:224] Handling call
0709 12:12:27.506823 (+ 24620us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:28.784184 (+1277361us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:28.784226 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:28.810170 (+ 25944us) log.cc:844] Serialized 26 byte log entry
0709 12:12:29.067931 (+257761us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:29.072292 (+  4361us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:29.072395 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:29.466222 (+393827us) raft_consensus.cc:1685] finished
0709 12:12:29.466243 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:29.466388 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1036764,"raft.queue_time_us":1273928,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":28128,"thread_start_us":478116,"threads_started":2,"wal-append.queue_time_us":326071}
I20260709 12:12:29.486390 18723 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:29.420884 12676 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:29.506603 18250 raft_consensus.cc:697] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:29.509145 12674 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:12:29.526268 12674 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: [13.6-13.6]
I20260709 12:12:29.512856 13334 raft_consensus.cc:2889] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:29.527405 12494 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:29.531901 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:29.533774 12494 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:29.535166 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:29.539947 12531 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:29.540118 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:29.535857 12679 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 14: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:29.466014 18850 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:29.543520 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:29.533063 18842 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.052s	user 0.002s	sys 0.000s
I20260709 12:12:29.519708 18869 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:29.555166 12603 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:12:29.555393 18869 raft_consensus.cc:697] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:29.555882 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 3081) took 1154 ms. Trace:
I20260709 12:12:29.496943 18834 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:29.556259 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:29.556465 18869 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:29.557451 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:29.558159 18834 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:29.517361 18828 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:29.559369 18828 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:29.560225 18832 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:29.560396 18828 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:29.542659 18753 raft_consensus.cc:2764] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:12:29.549933 18723 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:29.531399 18587 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.029s
I20260709 12:12:29.553648 18250 consensus_queue.cc:237] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:29.537088 18476 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 12:12:29.556278 12678 rpcz_store.cc:276] 0709 12:12:28.401186 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:28.407598 (+  6412us) service_pool.cc:224] Handling call
0709 12:12:29.555704 (+1148106us) spinlock_profiling.cc:243] Waited 860 ms on lock 0x7b5400013e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:29.555823 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1978221440}
I20260709 12:12:29.569499 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:29.558010 12603 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:29.570178 12606 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:29.532145 18772 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:29.564354 12599 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
W20260709 12:12:29.538530 13302 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.023s	sys 0.034s
W20260709 12:12:29.573694 13302 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.611s	user 0.023s	sys 0.035s
W20260709 12:12:29.566556 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 445096us
W20260709 12:12:29.567619 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:29.461719 18867 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:29.562681 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:29.607857 12600 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:29.566356 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:29.609601 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:29.575079 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:29.575662 13337 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
W20260709 12:12:29.575855 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 669879us
I20260709 12:12:29.610956 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:29.614223 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:29.614954 12674 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
I20260709 12:12:29.615459 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:29.615777 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:29.616098 12678 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
I20260709 12:12:29.617000 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:29.606447 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:29.534952 18858 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:12:29.561523 18303 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:29.623129 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:29.626662 13302 connection.cc:256] RPC call timeout handler was delayed by -1.1738958609998917s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:12:29.616425 12674 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:12:29.655661 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:29.656519 13331 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:29.656760 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:29.658560 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:29.659503 13334 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:29.646037 12600 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:12:29.662209 12600 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:12:29.619688 12599 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 14: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:29.665107 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:29.668565 13337 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
W20260709 12:12:29.670356 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 60843.5us
I20260709 12:12:29.670514 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:29.671456 13330 raft_consensus.cc:2410] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 17 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:12:29.673573 12494 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:29.674041 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:29.677206 13337 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 10 index: 10. Preceding OpId from leader: term: 13 index: 11. (index mismatch)
I20260709 12:12:29.678828 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:29.679714 13334 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
W20260709 12:12:29.689196 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 66199.3us
I20260709 12:12:29.689599 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:29.691020 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:29.645234 18640 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:29.620252 18851 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:29.696496 13331 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:29.651907 18772 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:29.692727 18781 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:29.711194 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:29.713902 12643 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:29.715166 18676 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:29.715672 18676 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:29.716281 18676 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:29.716425 12643 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:29.717868 18821 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:29.725996 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:29.727355 18723 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:12:29.727816 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52800us
I20260709 12:12:29.735514 12606 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:29.749676 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:29.750638 13330 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:29.716358 18511 raft_consensus.cc:3055] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:29.755139 12494 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:29.699903 18640 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:12:29.743170 18862 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.527s	user 0.012s	sys 0.009s
W20260709 12:12:29.769179 18862 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.013s	sys 0.009s
I20260709 12:12:29.767454 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:29.769651 18862 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:29.770179 18772 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:29.770922 18862 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:29.729951 18781 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:29.769114 18476 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:29.773324 18680 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:12:29.777602 12569 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:29.770967 18772 raft_consensus.cc:697] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:29.779536 18772 consensus_queue.cc:237] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:29.746672 18840 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:12:29.781212 18760 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.011s	sys 0.019s
I20260709 12:12:29.788138 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:29.789414 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:29.745762 18858 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:29.782716 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:29.790549 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:29.796105 12646 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:29.792301 12528 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:12:29.790028 12527 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:29.803989 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:29.794828 18760 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.605s	user 0.012s	sys 0.019s
I20260709 12:12:29.805503 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:29.775750 18640 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:29.763093 18511 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:29.785174 18851 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:12:29.799345 12671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 3053) took 2209 ms (client timeout 2115 ms). Trace:
W20260709 12:12:29.815583 12671 rpcz_store.cc:269] 0709 12:12:27.578682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:27.578802 (+   120us) service_pool.cc:224] Handling call
0709 12:12:29.787488 (+2208686us) spinlock_profiling.cc:243] Waited 888 ms on lock 0x7b540016eed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:29.787641 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2043075840}
I20260709 12:12:29.747211 18834 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.048s
I20260709 12:12:29.757532 18753 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:12:29.749410 18592 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:12:29.810245 18679 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:29.784384 18781 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:29.756117 18769 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.057s
I20260709 12:12:29.789489 18840 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:29.798869 18221 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:29.765720 18752 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.043s
I20260709 12:12:29.763121 18726 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 12:12:29.814546 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:29.791535 12603 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:29.834808 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:29.828958 18679 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:29.810781 18858 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 17 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:29.797993 18680 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:12:29.839071 13330 raft_consensus.cc:2376] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21. Re-sending same reply.
I20260709 12:12:29.807250 18587 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.025s
I20260709 12:12:29.829651 18760 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:29.831980 18840 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:29.853422 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:29.809125 18745 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260709 12:12:29.768705 18676 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:29.839777 18221 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:29.844956 18679 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:29.874367 12496 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.600s	user 0.012s	sys 0.020s
I20260709 12:12:29.827122 18851 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:29.874866 12496 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.012s	sys 0.020s
W20260709 12:12:29.881585 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 630438us
I20260709 12:12:29.865576 18726 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:29.865193 18303 raft_consensus.cc:3055] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:29.885818 12496 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:29.798800 18617 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:29.887740 12677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 3073) took 1810 ms (client timeout 1744 ms). Trace:
W20260709 12:12:29.899273 12677 rpcz_store.cc:269] 0709 12:12:28.077257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:28.092356 (+ 15099us) service_pool.cc:224] Handling call
0709 12:12:29.887504 (+1795148us) spinlock_profiling.cc:243] Waited 440 ms on lock 0x7b54003d45d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:29.887663 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1012069632}
I20260709 12:12:29.871119 18760 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:29.892344 12605 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.003s	sys 0.016s
I20260709 12:12:29.867501 18786 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:29.874037 18221 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:29.889648 18726 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:29.902685 12605 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.530s	user 0.003s	sys 0.016s
I20260709 12:12:29.876111 18821 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:12:29.874579 18839 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.047s	sys 0.055s
W20260709 12:12:29.948858 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:29.891999 12598 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.561s	user 0.003s	sys 0.021s
W20260709 12:12:29.935153 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:29.968827 12598 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.003s	sys 0.022s
I20260709 12:12:29.896497 18303 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:29.895787 18511 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:30.004595 18760 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:30.005470 18511 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:30.009101 18303 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:30.010659 18303 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:29.977670 18680 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:30.014469 18726 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:30.015439 18680 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:29.944259 18839 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.048s	sys 0.056s
I20260709 12:12:29.980494 18796 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:30.023350 18796 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:30.025645 18796 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:30.015900 18726 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:29.913642 18756 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:30.043605 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:30.045694 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:30.051170 13304 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:30.001992 18867 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:29.914453 18786 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:30.029497 18723 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:30.036657 18756 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:30.035425 18821 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:30.038403 18680 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:30.057791 18692 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:12:30.089143 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:30.079311 18528 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.547s	user 0.011s	sys 0.041s
I20260709 12:12:29.943276 18858 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:30.145190 13337 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:12:30.088006 18756 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:30.143152 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:30.153673 12602 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.017s	sys 0.019s
W20260709 12:12:30.131691 18528 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.599s	user 0.012s	sys 0.041s
I20260709 12:12:30.090929 18587 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:30.130653 18692 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:30.154245 12602 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.506s	user 0.018s	sys 0.019s
I20260709 12:12:30.143185 18879 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:30.090574 18821 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:30.153410 18877 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:12:30.208896 18885 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
W20260709 12:12:30.210727 18881 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.267s	user 0.001s	sys 0.000s
I20260709 12:12:30.212332 18740 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:30.218869 18876 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.158s	user 0.000s	sys 0.000s
W20260709 12:12:30.219930 18887 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
I20260709 12:12:30.230017 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:30.144810 18645 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:12:30.199241 18692 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:30.078915 18617 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:30.142180 18680 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 6, Time since last communication: 0.063s
I20260709 12:12:30.146852 18723 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 12:12:30.147660 18807 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.077s
W20260709 12:12:30.230451 18828 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.670s	user 0.012s	sys 0.002s
I20260709 12:12:30.224506 18752 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:30.228189 18879 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:30.199232 18587 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:30.246170 18740 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:30.237720 18676 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:30.305322 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:30.288079 18891 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:12:30.365425 18786 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:30.368984 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:30.290055 18221 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:30.373253 18221 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:30.293860 18679 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:30.376152 18679 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:30.381403 12606 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
W20260709 12:12:30.388391 18781 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.542s	user 0.009s	sys 0.001s
I20260709 12:12:30.389297 18781 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:30.396332 18781 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:30.399892 12675 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:30.400914 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:51308 (request call id 3064) took 2333 ms. Trace:
W20260709 12:12:30.406919 18892 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:12:30.408017 18499 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:30.411600 18821 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:30.412452 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2698) took 1244 ms. Trace:
I20260709 12:12:30.413034 12680 rpcz_store.cc:276] 0709 12:12:29.167702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:29.174017 (+  6315us) service_pool.cc:224] Handling call
0709 12:12:29.174317 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:29.323990 (+149673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:30.411980 (+1087990us) spinlock_profiling.cc:243] Waited 148 ms on lock 0x7b540036d154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:30.412357 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":340140928}
I20260709 12:12:30.419157 18580 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:30.423480 12427 catalog_manager.cc:5697] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:12:30.426702 12424 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58980 (request call id 184) took 49917 ms (client timeout 15000 ms). Trace:
I20260709 12:12:30.425498 12427 catalog_manager.cc:5697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 10 to 15, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 15 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:12:30.429911 12427 catalog_manager.cc:5697] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 12 to 13, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 13 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:12:30.431470 12427 catalog_manager.cc:5697] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 6 to 10, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 10 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:12:30.432722 18640 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.608s	user 0.007s	sys 0.002s
W20260709 12:12:30.427268 12424 rpcz_store.cc:269] 0709 12:11:40.509402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:40.512344 (+  2942us) service_pool.cc:224] Handling call
0709 12:12:30.426372 (+49914028us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:21.582376 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:21.631964 (+ 49588us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:21.637145 (+  5181us) write_op.cc:435] Acquired schema lock
0709 12:12:21.639985 (+  2840us) tablet.cc:662] Decoding operations
0709 12:12:21.832298 (+192313us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:21.832426 (+   128us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:21.832462 (+    36us) tablet.cc:685] Acquiring locks for 19 operations
0709 12:12:21.857411 (+ 24949us) tablet.cc:701] Row locks acquired
0709 12:12:21.865686 (+  8275us) write_op.cc:260] PREPARE: finished
0709 12:12:21.900830 (+ 35144us) write_op.cc:270] Start()
0709 12:12:21.931412 (+ 30582us) write_op.cc:276] Timestamp: P: 1783599141876936 usec, L: 0
0709 12:12:21.938386 (+  6974us) op_driver.cc:348] REPLICATION: starting
0709 12:12:21.939264 (+   878us) log.cc:844] Serialized 7521 byte log entry
0709 12:12:22.306435 (+367171us) spinlock_profiling.cc:243] Waited 56 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:22.942843 (+636408us) op_driver.cc:464] REPLICATION: finished
0709 12:12:23.150567 (+207724us) write_op.cc:301] APPLY: starting
0709 12:12:23.150760 (+   193us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:23.665071 (+514311us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:23.718948 (+ 53877us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:24.533888 (+814940us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:24.576674 (+ 42786us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=38,key_file_lookups=38,delta_file_lookups=38,mrs_lookups=0
0709 12:12:24.576749 (+    75us) write_op.cc:312] APPLY: finished
0709 12:12:24.577354 (+   605us) log.cc:844] Serialized 172 byte log entry
0709 12:12:24.862593 (+285239us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:24.916454 (+ 53861us) write_op.cc:454] Released schema lock
0709 12:12:25.009864 (+ 93410us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:25.200219 (+190355us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:25.200276 (+    57us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":8102,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":195829,"apply.run_cpu_time_us":58619,"apply.run_wall_time_us":2054335,"cfile_cache_hit":78,"cfile_cache_hit_bytes":83433,"num_ops":19,"prepare.queue_time_us":98113,"prepare.run_cpu_time_us":34607,"prepare.run_wall_time_us":434980,"raft.queue_time_us":244163,"raft.run_cpu_time_us":56350,"raft.run_wall_time_us":180938,"replication_time_us":988784,"spinlock_wait_cycles":128733312,"thread_start_us":679404,"threads_started":5,"wal-append.queue_time_us":497211,"wal-append.run_cpu_time_us":35497,"wal-append.run_wall_time_us":2886915}]]}
I20260709 12:12:30.433152 18640 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:30.401377 12675 rpcz_store.cc:276] 0709 12:12:28.073716 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:28.078043 (+  4327us) service_pool.cc:224] Handling call
0709 12:12:30.399155 (+2321112us) spinlock_profiling.cc:243] Waited 45.9 ms on lock 0x7b4c00333000. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1486ceca20 00007f1486cfda29 00007f1486cfe13f 00007f1486cc22ab 00007f1486ceff3c 00007f1486cf03fe 00007f1486f7d88b 00007f148897c495 00007f14889cd30f 00007f1488a17a55 00007f148d9e7cd0
0709 12:12:30.400766 (+  1611us) inbound_call.cc:177] Queueing success response
Metrics: {"dns_us":258,"spinlock_wait_cycles":105621760}
W20260709 12:12:30.438172 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:30.433689 12427 catalog_manager.cc:5697] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:30.462316 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:12:30.462741 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:30.464744 13334 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:30.252790 18851 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:30.482944 13334 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:30.359444 18752 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:30.461651 18676 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:30.514003 12571 connection.cc:256] RPC call timeout handler was delayed by -1.1083254540001235s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15280000000000002s for a response
W20260709 12:12:30.474517 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 321049us
I20260709 12:12:30.479704 12427 catalog_manager.cc:5697] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 12 to 16, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 16 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:30.523850 12427 catalog_manager.cc:5697] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 10 to 13, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 13 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:12:30.524829 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:30.470388 18250 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:30.550173 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 12 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:30.488704 18796 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:30.545569 18894 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:12:30.485459 18221 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:30.485606 18786 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:30.555187 13337 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:12:30.470248 18855 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:30.559688 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:30.577294 18901 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:30.545634 18879 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:30.546741 18723 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:30.545660 18740 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:12:30.623193 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:30.575568 12571 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:30.602952 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2967) took 1289 ms. Trace:
W20260709 12:12:30.628695 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 53222.3us
I20260709 12:12:30.599246 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:30.547513 18676 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:30.566464 18250 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:30.635682 12606 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 18: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:30.628989 12605 rpcz_store.cc:276] 0709 12:12:29.313256 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:29.314207 (+   951us) service_pool.cc:224] Handling call
0709 12:12:29.357967 (+ 43760us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:29.359394 (+  1427us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:12:29.916579 (+557185us) log.cc:844] Serialized 14 byte log entry
0709 12:12:30.593503 (+676924us) log.cc:844] Serialized 14 byte log entry
0709 12:12:30.593750 (+   247us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:30.593810 (+    60us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:30.593889 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:30.602675 (+  8786us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:30.602864 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13613,"prepare.queue_time_us":535802,"prepare.run_cpu_time_us":98,"prepare.run_wall_time_us":3535,"raft.queue_time_us":25506,"raft.run_cpu_time_us":63,"raft.run_wall_time_us":63,"thread_start_us":1205211,"threads_started":3,"wal-append.queue_time_us":574133}
W20260709 12:12:30.631788 18908 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:12:30.632894 18587 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:30.576838 18796 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:30.575143 18786 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:30.592310 18855 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:30.657678 12672 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 11. Preceding OpId from leader: term: 13 index: 11. (term mismatch)
I20260709 12:12:30.623193 18723 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:30.643359 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:30.661114 12606 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:30.629678 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:30.658195 12672 pending_rounds.cc:85] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 10
I20260709 12:12:30.662750 12672 pending_rounds.cc:107] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 12.11
I20260709 12:12:30.663132 12672 raft_consensus.cc:2889] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:30.622514 18770 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 6, Time since last communication: 0.025s
I20260709 12:12:30.668910 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2969) took 1341 ms. Trace:
W20260709 12:12:30.655128 18511 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.626s	user 0.004s	sys 0.010s
I20260709 12:12:30.666833 12571 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:30.573350 18221 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:30.697192 12572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.002s	sys 0.039s
W20260709 12:12:30.697624 12572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.511s	user 0.002s	sys 0.039s
I20260709 12:12:30.699170 18221 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:30.670331 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:30.700839 12606 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:30.673411 18912 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:12:30.672689 12598 rpcz_store.cc:276] 0709 12:12:29.327401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:29.330259 (+  2858us) service_pool.cc:224] Handling call
0709 12:12:29.330650 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:29.330875 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:12:29.969368 (+638493us) log.cc:844] Serialized 14 byte log entry
0709 12:12:30.668428 (+699060us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:30.668492 (+    64us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:30.668568 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:30.668676 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:30.668832 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":563488,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":58005,"raft.run_cpu_time_us":66,"raft.run_wall_time_us":65,"thread_start_us":1335204,"threads_started":3,"wal-append.queue_time_us":314840}
I20260709 12:12:30.670698 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:30.625277 18752 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:30.672369 18770 consensus_queue.cc:1243] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 15.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6
I20260709 12:12:30.595670 18901 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:30.712395 18879 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:12:30.712754 18910 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:12:30.714100 18676 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:30.714987 13337 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:30.695627 18740 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:30.715785 18855 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:30.717104 18250 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:30.718719 18903 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
W20260709 12:12:30.719578 18911 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:12:30.721638 18914 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:30.882628 18914 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:30.883456 18914 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:30.723248 12604 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 17 index: 8. (term mismatch)
I20260709 12:12:30.886387 12604 pending_rounds.cc:85] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 7
I20260709 12:12:30.886771 12604 pending_rounds.cc:107] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 16.8
I20260709 12:12:30.887079 12604 raft_consensus.cc:2889] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:30.736776 18869 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:30.887884 18766 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:30.888286 18869 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:30.891484 18869 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:30.738322 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:30.892912 12671 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:30.739049 18769 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:30.740304 18828 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:12:30.895090 18828 consensus_queue.cc:1243] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:12:30.740931 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:30.897910 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:30.898756 12675 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:30.899302 18786 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:30.741802 12680 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:30.905193 18796 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:30.743486 12598 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:30.745474 18745 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:30.745925 18677 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:30.745934 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:30.846587 18586 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:30.746747 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:30.748026 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:30.748626 12676 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:30.749004 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:30.749636 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:30.922436 12677 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:30.749972 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:30.923672 12527 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 19: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:30.751024 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:30.752008 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:30.753686 18592 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:30.755129 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:30.931416 12679 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:30.756276 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:30.757782 12674 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:30.761425 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:30.770874 12599 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 17. Current term is 18. Ops: [17.14-17.14]
W20260709 12:12:30.775897 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:30.691922 18499 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:30.826599 13337 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:30.931442 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:30.714885 18758 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:30.828168 12602 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.007s	sys 0.016s
W20260709 12:12:30.943770 12602 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.008s	sys 0.016s
I20260709 12:12:30.856604 18580 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:30.944640 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2973) took 1616 ms. Trace:
W20260709 12:12:30.836310 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 169543us
I20260709 12:12:30.946305 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:30.944999 12602 rpcz_store.cc:276] 0709 12:12:29.328700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:29.333181 (+  4481us) service_pool.cc:224] Handling call
0709 12:12:29.333526 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:29.333787 (+   261us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:12:29.648104 (+314317us) log.cc:844] Serialized 14 byte log entry
0709 12:12:30.944285 (+1296181us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:30.944349 (+    64us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:30.944407 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:30.944476 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:30.944583 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":560997,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":53767,"raft.run_cpu_time_us":69,"raft.run_wall_time_us":69,"thread_start_us":1586331,"threads_started":3,"wal-append.queue_time_us":721675}
I20260709 12:12:30.830862 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:30.821424 18858 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:30.848565 18640 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:30.920802 18745 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:30.950538 18745 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:30.851311 18740 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:30.949453 18858 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:30.952198 12522 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:12:30.814688 18879 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:30.861932 18919 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.166s	user 0.000s	sys 0.000s
I20260709 12:12:30.866464 18221 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:30.888617 18766 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:30.737450 18679 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:30.734584 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:30.971688 13335 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:30.891835 18723 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:30.746301 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:30.930894 12524 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 14: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:12:30.916867 18840 consensus_meta.cc:287] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.044s	user 0.014s	sys 0.001s
I20260709 12:12:30.975724 12531 raft_consensus.cc:2410] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:12:30.976181 13305 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:30.943658 12677 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:30.977880 12678 raft_consensus.cc:2376] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17. Re-sending same reply.
I20260709 12:12:30.839078 12606 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:30.978900 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:30.920763 18726 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.865s	user 0.006s	sys 0.009s
I20260709 12:12:30.813129 18920 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:30.954620 12571 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:30.944943 12675 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:30.961277 12679 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:30.929816 18677 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:30.994670 12571 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:30.997454 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:30.999341 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:31.004566 12679 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 14: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:31.004971 12677 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:30.996819 13339 pending_rounds.cc:85] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 12
I20260709 12:12:31.007007 13339 pending_rounds.cc:107] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:12:31.006672 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:31.007306 13339 raft_consensus.cc:2889] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:30.939613 13337 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 17.14, Last appended by leader: 14, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:31.009311 13337 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:31.010756 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:31.012706 12675 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:30.966310 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:30.981093 13305 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:30.962448 18828 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:30.962687 18773 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:30.965790 18879 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:31.030865 12524 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:30.965682 18929 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:31.019826 12671 raft_consensus.cc:2393] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 16: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:31.033061 12675 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:12:30.952275 18499 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:31.031418 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:31.065922 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:31.069339 12531 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
I20260709 12:12:31.001173 18855 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:30.965564 18580 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:31.035192 18511 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:31.028134 18828 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:31.017382 18928 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:31.093205 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:12:31.101329 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:30.981197 18221 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:31.109052 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:30.994377 18679 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:30.994611 18840 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:31.019423 18920 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:31.059631 13335 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
W20260709 12:12:31.102715 18756 consensus_meta.cc:287] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.911s	user 0.008s	sys 0.006s
I20260709 12:12:31.089370 18929 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:31.099941 18914 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:31.107946 18922 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:31.143281 12679 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 9 index: 6. Preceding OpId from leader: term: 11 index: 7. (index mismatch)
I20260709 12:12:31.135005 18862 raft_consensus.cc:2764] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:12:31.147222 12599 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:31.061801 18773 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:31.170383 12643 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.013s	sys 0.030s
W20260709 12:12:31.170902 12643 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.553s	user 0.013s	sys 0.030s
I20260709 12:12:31.123783 18901 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:31.171315 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 553529us
I20260709 12:12:31.147518 18819 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:31.140163 18756 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:31.081583 13337 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 17 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:12:31.169677 18929 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:31.218850 12522 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:12:31.220701 18819 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:31.220920 18743 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:31.221493 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:31.221403 18819 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:31.221756 18743 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:31.222893 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:31.223747 18786 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:31.224109 18819 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:31.224269 18786 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:31.224721 18819 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:31.255983 12677 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:31.170629 12572 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:31.260540 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 90061.1us
I20260709 12:12:31.261832 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:31.267572 12572 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:31.148056 18917 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.082s
I20260709 12:12:31.252822 18743 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:31.202404 18915 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.013s	sys 0.028s
W20260709 12:12:31.309664 18930 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.178s	user 0.000s	sys 0.000s
I20260709 12:12:31.310241 18920 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:31.311129 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:31.311374 18499 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:31.184187 18828 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:31.312151 18677 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:31.218843 18838 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.094s
I20260709 12:12:31.313676 18773 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:31.315645 18756 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:31.316213 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:31.317178 18786 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:31.363212 18917 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:31.317456 18858 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:31.318408 18680 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:31.364068 18917 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:31.364552 18917 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:31.364517 18680 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:31.361829 18580 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:31.367192 18680 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:31.319437 18221 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:31.320683 18931 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
I20260709 12:12:31.321632 18586 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:31.369751 18586 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:31.344089 18819 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:31.347218 18740 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:31.343052 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 75671.2us
I20260709 12:12:31.347721 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:31.382043 13335 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:31.211161 18914 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:31.210850 18855 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:31.348567 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:31.313864 18877 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:31.361874 18828 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:31.298354 18901 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:31.356505 12677 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22.
I20260709 12:12:31.363607 18679 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:31.382265 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:31.365721 18917 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:31.297355 18753 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.000s	sys 0.036s
W20260709 12:12:31.392316 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 467485us
W20260709 12:12:31.388509 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 355350us
I20260709 12:12:31.395334 18877 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:31.387260 12494 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:31.410144 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:31.386814 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2993) took 1249 ms. Trace:
I20260709 12:12:31.390954 18528 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:31.371696 18920 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:31.396586 18915 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:31.475390 12605 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:31.596802 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:31.597534 12605 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:31.598397 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:31.471288 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 89194.3us
I20260709 12:12:31.600494 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:31.475557 12603 rpcz_store.cc:276] 0709 12:12:30.137328 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:30.148379 (+ 11051us) service_pool.cc:224] Handling call
0709 12:12:30.190092 (+ 41713us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:30.552940 (+362848us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:30.553015 (+    75us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:30.554306 (+  1291us) log.cc:844] Serialized 52 byte log entry
0709 12:12:31.043991 (+489685us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:31.385760 (+341769us) log.cc:844] Serialized 14 byte log entry
0709 12:12:31.386252 (+   492us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:31.386436 (+   184us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:31.386498 (+    62us) raft_consensus.cc:1685] finished
0709 12:12:31.386517 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:31.386734 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17134,"prepare.queue_time_us":38137,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":70,"raft.queue_time_us":59671,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":8873,"thread_start_us":1071436,"threads_started":3,"wal-append.queue_time_us":630347}
I20260709 12:12:31.472982 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:31.475224 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 307232us
I20260709 12:12:31.484467 12569 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:31.582301 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 195106us
W20260709 12:12:31.392832 18921 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.538s	user 0.010s	sys 0.010s
I20260709 12:12:31.391834 18914 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:31.606352 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:31.607314 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:31.599568 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:31.471755 18221 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:31.396067 18855 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:31.587291 18935 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
W20260709 12:12:31.631527 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 158600us
I20260709 12:12:31.631587 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2996) took 1493 ms. Trace:
I20260709 12:12:31.633589 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:31.634524 12603 raft_consensus.cc:2410] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 17 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:12:31.635567 13302 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:31.635854 13338 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 15 index: 11. (index mismatch)
W20260709 12:12:31.631587 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 147203us
I20260709 12:12:31.633221 12601 rpcz_store.cc:276] 0709 12:12:30.138266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:30.376924 (+238658us) service_pool.cc:224] Handling call
0709 12:12:30.377300 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:30.377745 (+   445us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:12:30.378212 (+   467us) log.cc:844] Serialized 14 byte log entry
0709 12:12:30.781568 (+403356us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:30.811017 (+ 29449us) log.cc:844] Serialized 26 byte log entry
0709 12:12:30.820972 (+  9955us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:30.821029 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:30.821121 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:31.631322 (+810201us) raft_consensus.cc:1685] finished
0709 12:12:31.631356 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:31.631513 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17861,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":52,"raft.queue_time_us":83103,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":3364,"thread_start_us":402939,"threads_started":1,"wal-append.queue_time_us":686466}
W20260709 12:12:31.632534 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:31.592295 18511 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:31.594084 18753 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:31.595880 18840 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:31.662297 13302 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:31.608538 18921 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.011s	sys 0.011s
I20260709 12:12:31.396082 18929 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:31.608338 18894 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.132s	user 0.001s	sys 0.000s
I20260709 12:12:31.697134 18949 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:31.708005 18949 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:31.674042 18933 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.296s	user 0.000s	sys 0.001s
I20260709 12:12:31.697357 18221 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:31.711059 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2992) took 1574 ms. Trace:
I20260709 12:12:31.698072 18511 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:31.711521 12597 rpcz_store.cc:276] 0709 12:12:30.136987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:30.139720 (+  2733us) service_pool.cc:224] Handling call
0709 12:12:30.144286 (+  4566us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:30.497407 (+353121us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:12:30.497450 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:30.501538 (+  4088us) log.cc:844] Serialized 52 byte log entry
0709 12:12:31.063092 (+561554us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:31.063136 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:31.063206 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:31.710771 (+647565us) raft_consensus.cc:1685] finished
0709 12:12:31.710793 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:31.710974 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":94813,"spinlock_wait_cycles":4111104,"thread_start_us":818377,"threads_started":2,"wal-append.queue_time_us":763450}
I20260709 12:12:31.578285 18901 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.139s
I20260709 12:12:31.673388 18592 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:31.673818 18873 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:31.678539 18766 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:31.595278 18819 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:31.663601 18753 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:12:31.681857 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:31.829789 12494 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:31.830907 12494 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:31.609122 18941 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:31.831797 18941 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:31.610601 18858 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:31.598320 18917 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:31.663843 18920 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:31.588428 18528 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:31.714071 12597 raft_consensus.cc:1217] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Deduplicated request from leader. Original: 11.6->[13.7-19.10]   Dedup: 13.7->[16.8-19.10]
W20260709 12:12:31.716475 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:31.706434 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:31.704756 12529 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.703s	user 0.022s	sys 0.000s
I20260709 12:12:31.613423 12605 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:31.629302 18944 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:31.642001 18915 raft_consensus.cc:697] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:31.826910 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:31.709276 18949 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:31.699743 18914 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:31.846800 18773 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:12:31.839107 12671 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.681s	user 0.019s	sys 0.025s
I20260709 12:12:31.840657 18941 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:31.848186 12529 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:31.846904 18756 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:31.851897 12531 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
I20260709 12:12:31.835202 18753 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:31.696978 18855 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:31.838512 18745 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:31.846858 12597 pending_rounds.cc:85] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 7
I20260709 12:12:31.874843 12597 pending_rounds.cc:107] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 12:12:31.875267 12597 raft_consensus.cc:2889] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:31.871435 13305 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:12:31.859231 12671 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.701s	user 0.019s	sys 0.025s
I20260709 12:12:31.842381 18873 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:31.865594 18499 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:31.887205 18499 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:31.888826 18873 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:31.889467 18873 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:31.889532 18499 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:31.890146 18873 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:31.890511 18745 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:31.847275 18858 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:31.842815 18920 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:31.894685 18920 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:31.848943 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:31.896555 12605 raft_consensus.cc:2410] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 22 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:12:31.804677 18592 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:31.899677 18592 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:31.892968 18745 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:31.872295 18756 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:31.907894 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:31.915377 13335 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:31.917480 12494 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:31.858067 18773 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:31.933893 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3093) took 1125 ms. Trace:
I20260709 12:12:31.945055 12675 rpcz_store.cc:276] 0709 12:12:30.808325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:31.063966 (+255641us) service_pool.cc:224] Handling call
0709 12:12:31.078968 (+ 15002us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:31.079205 (+   237us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:12:31.391757 (+312552us) log.cc:844] Serialized 14 byte log entry
0709 12:12:31.933099 (+541342us) log.cc:844] Serialized 14 byte log entry
0709 12:12:31.933424 (+   325us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:31.933489 (+    65us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:31.933552 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:31.933645 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:31.933812 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":755270,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":83,"raft.queue_time_us":49555,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"thread_start_us":728130,"threads_started":2}
I20260709 12:12:31.953172 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:31.862165 18511 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:31.859568 18915 consensus_queue.cc:237] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:31.949774 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:31.963581 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3095) took 1154 ms. Trace:
I20260709 12:12:31.924340 18756 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:31.960821 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:31.856719 18944 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:31.965155 12673 rpcz_store.cc:276] 0709 12:12:30.808894 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:31.184094 (+375200us) service_pool.cc:224] Handling call
0709 12:12:31.184484 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:31.184735 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:12:31.480320 (+295585us) log.cc:844] Serialized 14 byte log entry
0709 12:12:31.963223 (+482903us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:31.963282 (+    59us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:31.963333 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:31.963394 (+    61us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:31.963529 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3176,"prepare.queue_time_us":675930,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":4674,"thread_start_us":744197,"threads_started":2}
I20260709 12:12:31.967875 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:31.968726 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 268822us
I20260709 12:12:31.969795 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:31.959937 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 359534us
I20260709 12:12:31.944355 18773 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:31.968716 12673 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:31.860116 18955 consensus_queue.cc:1059] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 13.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.106s
I20260709 12:12:31.892984 18858 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:31.973790 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:12:31.993830 13331 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38472 (request call id 668) took 2277 ms (client timeout 1979 ms). Trace:
W20260709 12:12:31.994314 13331 rpcz_store.cc:269] 0709 12:12:29.710784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:29.710913 (+   129us) service_pool.cc:224] Handling call
0709 12:12:31.988161 (+2277248us) spinlock_profiling.cc:243] Waited 546 ms on lock 0x7b54001261d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:31.988377 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1256185728}
W20260709 12:12:31.959599 18951 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.070s	user 0.001s	sys 0.000s
I20260709 12:12:31.979672 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:31.982651 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:31.940065 18745 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:31.963861 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:31.962224 18499 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:31.989876 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:32.015358 12673 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:31.964293 18840 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:31.996181 12524 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:12:32.013440 12675 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:32.015491 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:32.019114 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:32.019657 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:32.020642 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:31.991355 18740 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:32.024425 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:32.026302 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:32.027033 12524 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:32.027307 12675 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 18: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:32.012743 18745 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:32.016036 18921 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260709 12:12:32.031138 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:32.034013 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:32.038872 12675 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 19: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:32.034148 12572 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:32.039362 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:32.040025 12673 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
W20260709 12:12:32.026386 18828 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.623s	user 0.010s	sys 0.001s
I20260709 12:12:32.033469 18745 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:32.038096 18580 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:12:32.047816 18877 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:32.049077 13337 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:32.051657 18948 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:32.052158 18745 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:32.018734 18499 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:32.025424 18914 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:32.056907 12675 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: [16.12-16.12]
I20260709 12:12:32.056530 18873 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:32.059204 18873 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:32.060602 18586 consensus_meta.cc:287] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.690s	user 0.012s	sys 0.000s
I20260709 12:12:32.061331 18586 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:32.062523 18499 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:32.062532 18914 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:32.064971 12572 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:32.065853 18921 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:32.070544 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3098) took 1260 ms. Trace:
W20260709 12:12:32.070756 12598 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.597s	user 0.046s	sys 0.082s
I20260709 12:12:32.071147 12675 rpcz_store.cc:276] 0709 12:12:30.809757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:32.056138 (+1246381us) service_pool.cc:224] Handling call
0709 12:12:32.056577 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:32.070108 (+ 13531us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:32.070392 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:32.073057 12675 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:12:32.073762 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3099) took 1263 ms. Trace:
I20260709 12:12:32.074385 12528 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
W20260709 12:12:32.038256 18692 consensus_meta.cc:287] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.681s	user 0.020s	sys 0.001s
I20260709 12:12:32.023383 18840 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:32.042986 18879 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.968s	user 0.012s	sys 0.013s
I20260709 12:12:32.075731 12675 rpcz_store.cc:276] 0709 12:12:30.810027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:32.072446 (+1262419us) service_pool.cc:224] Handling call
0709 12:12:32.072824 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:32.073499 (+   675us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:32.073710 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:32.077692 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:32.029584 18766 consensus_queue.cc:1059] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.047s
I20260709 12:12:32.029868 18740 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:32.135852 12675 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:32.144320 12524 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:32.134330 12673 raft_consensus.cc:2410] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 14 because replica has last-logged OpId of term: 12 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:12:32.106840 18745 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:32.153618 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3100) took 1321 ms. Trace:
I20260709 12:12:32.154274 12675 rpcz_store.cc:276] 0709 12:12:30.832601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:32.131344 (+1298743us) service_pool.cc:224] Handling call
0709 12:12:32.131672 (+   328us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:32.151916 (+ 20244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:32.153540 (+  1624us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:32.156143 12675 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:12:32.156898 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3101) took 1323 ms. Trace:
I20260709 12:12:32.157271 12675 rpcz_store.cc:276] 0709 12:12:30.833046 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:32.155442 (+1322396us) service_pool.cc:224] Handling call
0709 12:12:32.155857 (+   415us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:32.156601 (+   744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:32.156837 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:32.153013 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:32.158119 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:32.165195 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:32.075415 18921 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.077s
I20260709 12:12:32.185415 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:32.186374 12524 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:32.188416 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3094) took 1379 ms. Trace:
W20260709 12:12:32.191983 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 274633us
I20260709 12:12:32.189635 12671 rpcz_store.cc:276] 0709 12:12:30.808617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:31.075430 (+266813us) service_pool.cc:224] Handling call
0709 12:12:31.104945 (+ 29515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:31.105256 (+   311us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:12:31.877458 (+772202us) log.cc:844] Serialized 14 byte log entry
0709 12:12:32.187714 (+310256us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:32.187782 (+    68us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:12:32.187844 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:32.187918 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:32.188338 (+   420us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32482,"prepare.queue_time_us":746940,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":2150,"thread_start_us":1026884,"threads_started":2}
I20260709 12:12:32.145272 18692 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:32.163203 18877 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:32.146162 18879 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:32.150566 18920 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:32.214030 12570 service_pool.cc:136] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:51782 dropped due to backpressure: service queue is full with 50 items
I20260709 12:12:32.217494 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:32.218463 13335 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:32.227902 13330 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:32.228276 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:32.228937 12671 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:32.230166 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:32.226573 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:32.224736 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:32.196148 18586 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:32.194407 18838 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:32.237715 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:32.239071 12526 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 17: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:32.198863 18752 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:12:32.220908 18959 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.236s	user 0.001s	sys 0.000s
I20260709 12:12:32.244649 12646 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:32.246682 12524 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
W20260709 12:12:32.247109 18960 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.227s	user 0.000s	sys 0.001s
I20260709 12:12:32.219990 18819 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.040s
I20260709 12:12:32.254196 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3097) took 1444 ms. Trace:
I20260709 12:12:32.248451 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:32.256494 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:32.256803 13305 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:32.257758 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:32.258625 12524 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:12:32.260601 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:32.264778 18877 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:32.265465 18877 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:32.266400 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:32.266628 18877 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:32.254662 12677 rpcz_store.cc:276] 0709 12:12:30.809466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:31.401392 (+591926us) service_pool.cc:224] Handling call
0709 12:12:31.401798 (+   406us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:12:31.475392 (+ 73594us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:31.475433 (+    41us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:12:31.581988 (+106555us) log.cc:844] Serialized 78 byte log entry
0709 12:12:31.991916 (+409928us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:31.991960 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:31.992045 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:32.253850 (+261805us) raft_consensus.cc:1685] finished
0709 12:12:32.253873 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:32.254077 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11575,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":105720,"thread_start_us":327537,"threads_started":1,"wal-append.queue_time_us":551073}
I20260709 12:12:32.269369 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:32.270047 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:32.271700 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:32.274662 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:32.276415 12646 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:32.290253 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:32.290361 12527 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:32.290216 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:32.292661 12677 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:32.293918 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:32.278192 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:32.291146 12671 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:32.242661 18944 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:32.253624 18819 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:32.302989 13335 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:32.298092 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:32.250329 18586 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:32.246876 18966 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:12:32.383309 18971 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
W20260709 12:12:32.386729 18961 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.151s	user 0.000s	sys 0.000s
I20260709 12:12:32.386597 18692 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:32.387578 18499 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:32.388723 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:32.248018 18752 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:32.263271 18745 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:32.391577 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3096) took 1582 ms. Trace:
I20260709 12:12:32.393275 12496 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:32.519270 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 126097us
I20260709 12:12:32.520166 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:32.395179 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:32.396282 12677 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:12:32.537114 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3106) took 1702 ms. Trace:
I20260709 12:12:32.537617 12677 rpcz_store.cc:276] 0709 12:12:30.834597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:32.391083 (+1556486us) service_pool.cc:224] Handling call
0709 12:12:32.391587 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:32.536692 (+145105us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:32.537029 (+   337us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:32.539881 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3111) took 1522 ms. Trace:
I20260709 12:12:32.540361 12677 rpcz_store.cc:276] 0709 12:12:31.017315 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:32.538709 (+1521394us) service_pool.cc:224] Handling call
0709 12:12:32.539177 (+   468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:32.539429 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:32.539469 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:32.539540 (+    71us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:32.539597 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:32.539669 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:32.539810 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:32.398108 12671 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:32.412073 18985 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:32.245079 18838 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:32.416539 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:32.390988 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:32.567528 18740 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:32.629640 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2705) took 3420 ms. Trace:
I20260709 12:12:32.572048 18729 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:12:32.573124 18970 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.182s	user 0.000s	sys 0.000s
W20260709 12:12:32.578787 18753 consensus_meta.cc:287] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.685s	user 0.009s	sys 0.005s
I20260709 12:12:32.517405 12679 rpcz_store.cc:276] 0709 12:12:30.809192 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:31.328008 (+518816us) service_pool.cc:224] Handling call
0709 12:12:31.328419 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:31.337645 (+  9226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:31.337708 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:31.340445 (+  2737us) log.cc:844] Serialized 26 byte log entry
0709 12:12:31.737999 (+397554us) raft_consensus.cc:1645] Marking committed up to 5
0709 12:12:31.738046 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:31.738151 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:32.391341 (+653190us) raft_consensus.cc:1685] finished
0709 12:12:32.391361 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:32.391517 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1496,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":94,"thread_start_us":397138,"threads_started":1,"wal-append.queue_time_us":602925}
I20260709 12:12:32.542904 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3110) took 1574 ms. Trace:
I20260709 12:12:32.631843 12672 rpcz_store.cc:276] 0709 12:12:29.209649 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:29.459036 (+249387us) service_pool.cc:224] Handling call
0709 12:12:29.459366 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:30.663744 (+1204378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:32.628794 (+1965050us) spinlock_profiling.cc:243] Waited 1.2 s on lock 0x7b54000924d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:32.629536 (+   742us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2750449920}
I20260709 12:12:32.655069 12671 rpcz_store.cc:276] 0709 12:12:30.968759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:32.397346 (+1428587us) service_pool.cc:224] Handling call
0709 12:12:32.397695 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:32.542541 (+144846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:32.542832 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:32.396239 18966 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:32.570863 18974 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.269s	user 0.000s	sys 0.001s
I20260709 12:12:32.631100 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3112) took 1487 ms. Trace:
I20260709 12:12:32.412150 18987 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.105s
I20260709 12:12:32.656071 12679 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
W20260709 12:12:32.659942 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 115205us
W20260709 12:12:32.636025 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 115942us
I20260709 12:12:32.689018 12496 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:32.297703 18756 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:32.671741 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:32.653093 18985 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:12:32.674355 18978 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
I20260709 12:12:32.678460 12677 rpcz_store.cc:276] 0709 12:12:31.061648 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:32.542069 (+1480421us) service_pool.cc:224] Handling call
0709 12:12:32.547859 (+  5790us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:32.548040 (+   181us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:32.548078 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:32.548138 (+    60us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:32.548185 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:32.548257 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:32.548394 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:32.486068 18499 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:32.667409 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3000) took 2132 ms. Trace:
I20260709 12:12:32.684207 12671 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:32.690744 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3113) took 1601 ms. Trace:
I20260709 12:12:32.652693 18740 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:32.675719 18966 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:12:32.628243 18745 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:32.814332 12672 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
W20260709 12:12:32.732049 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 342143us
I20260709 12:12:32.214793 12570 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:51782 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 638)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 637)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2742)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2762)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2772)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2748)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2755)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 634)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2744)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2750)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2763)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2757)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2769)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2756)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3023)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2761)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 639)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2767)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2773)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2753)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2758)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2749)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3022)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2768)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2747)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2754)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2760)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2745)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3009)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3012)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3013)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3014)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3018)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 633)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 635)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3019)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3020)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3021)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2739)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2740)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2741)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2752)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2765)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2770)
I20260709 12:12:32.781914 12677 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:32.816041 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2728) took 1176 ms. Trace:
I20260709 12:12:32.816334 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2725) took 1182 ms. Trace:
I20260709 12:12:32.817651 12672 rpcz_store.cc:276] 0709 12:12:31.634029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:32.682063 (+1048034us) service_pool.cc:224] Handling call
0709 12:12:32.805120 (+123057us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:32.815957 (+ 10837us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:32.816256 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:32.816533 12677 rpcz_store.cc:276] 0709 12:12:31.639526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:32.780889 (+1141363us) service_pool.cc:224] Handling call
0709 12:12:32.781399 (+   510us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:32.815633 (+ 34234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:32.815960 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:32.775059 18499 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:32.787532 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 62027.4us
I20260709 12:12:32.632977 18840 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:32.681963 18753 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:32.853550 18990 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:32.861837 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2712) took 3377 ms. Trace:
I20260709 12:12:32.862481 12523 rpcz_store.cc:276] 0709 12:12:29.483954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:29.484078 (+   124us) service_pool.cc:224] Handling call
0709 12:12:29.534060 (+ 49982us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:29.534626 (+   566us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:12:29.534656 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:29.538947 (+  4291us) log.cc:844] Serialized 26 byte log entry
0709 12:12:30.349124 (+810177us) spinlock_profiling.cc:243] Waited 4.56 ms on lock 0x7b180041ff80. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:12:30.830159 (+481035us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:30.830218 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:30.830292 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:32.860689 (+2030397us) spinlock_profiling.cc:243] Waited 1.77 ms on lock 0x7b240011b4a8. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14857b99c4 00007f14857b7acb 00007f1485972dc9 00007f1485972a55 00007f1485971fb0 00007f14885e8488 00007f14885e8be8 00007f14885c0398 00007f14885bb76a 00007f14885babb6
0709 12:12:32.861482 (+   793us) raft_consensus.cc:1685] finished
0709 12:12:32.861512 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:32.861747 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":137,"raft.queue_time_us":848,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":6251,"spinlock_wait_cycles":14567680,"thread_start_us":480733,"threads_started":1,"wal-append.queue_time_us":1526753}
I20260709 12:12:32.866338 18901 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:32.867201 18901 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:32.865811 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:32.870846 12523 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:32.871873 12570 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:51782 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 638)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 637)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2742)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2762)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2772)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2748)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2755)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 634)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2744)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2750)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2763)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2757)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2769)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2756)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3023)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2761)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 639)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2767)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2773)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2753)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2758)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2749)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3022)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2768)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2747)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2754)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2760)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2745)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3009)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3012)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3013)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3014)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3018)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 633)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 635)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3019)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3020)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3021)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2739)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2740)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2741)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2752)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2765)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2770)
I20260709 12:12:32.407254 18586 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:32.788640 12679 rpcz_store.cc:276] 0709 12:12:31.084353 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:32.655090 (+1570737us) service_pool.cc:224] Handling call
0709 12:12:32.655594 (+   504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:32.685706 (+ 30112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:32.686066 (+   360us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:32.842660 18920 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:32.892655 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:32.661307 18828 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:32.845319 18980 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:32.687217 12494 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:32.790791 12530 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:32.791568 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2727) took 1152 ms. Trace:
I20260709 12:12:32.902377 12671 rpcz_store.cc:276] 0709 12:12:31.639185 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:32.683360 (+1044175us) service_pool.cc:224] Handling call
0709 12:12:32.683919 (+   559us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:32.790774 (+106855us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:32.791378 (+   604us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:32.903436 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2972) took 3575 ms. Trace:
I20260709 12:12:32.903712 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:32.690359 18729 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:32.903872 12604 rpcz_store.cc:276] 0709 12:12:29.328377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:29.332519 (+  4142us) service_pool.cc:224] Handling call
0709 12:12:29.332918 (+   399us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:30.887694 (+1554776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:32.903134 (+2015440us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54001617d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:32.903350 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3196551552}
I20260709 12:12:32.704527 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:32.721839 12599 rpcz_store.cc:276] 0709 12:12:30.535376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:30.936899 (+401523us) service_pool.cc:224] Handling call
0709 12:12:30.937191 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:31.156783 (+219592us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:32.666955 (+1510172us) spinlock_profiling.cc:243] Waited 210 ms on lock 0x7b54002706d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:32.667301 (+   346us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":482125696}
I20260709 12:12:32.893518 12679 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
I20260709 12:12:32.906512 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:32.876969 18944 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:32.838388 18476 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:32.912129 18476 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:32.912276 18729 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:32.914718 18838 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:32.915303 18944 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:32.916056 18476 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:32.916396 12643 leader_election.cc:336] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.113s (SENT)
I20260709 12:12:32.916920 12643 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:32.917168 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:32.917969 12523 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:32.723904 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:32.842999 18879 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:32.807504 18740 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:32.685274 18971 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
W20260709 12:12:32.910055 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 222964us
I20260709 12:12:32.928669 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:32.929579 12671 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:32.910511 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 206091us
I20260709 12:12:32.904062 18980 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:32.926097 12531 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:32.854476 18990 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:32.917589 18944 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:32.919601 18972 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:12:32.955629 12643 leader_election.cc:336] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.278s (SENT)
I20260709 12:12:32.920759 18983 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:32.868055 18985 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:32.942909 18980 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:32.977535 12646 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:32.976835 18773 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.962s	user 0.009s	sys 0.003s
I20260709 12:12:32.966854 18752 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:32.948170 18828 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.038s
W20260709 12:12:32.993131 12571 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.562s	user 0.003s	sys 0.012s
W20260709 12:12:33.002651 12571 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.572s	user 0.004s	sys 0.013s
W20260709 12:12:33.003170 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 572817us
I20260709 12:12:32.893052 12570 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:51782 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 638)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 637)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2742)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2762)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2772)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2748)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2755)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 634)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2744)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2750)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2763)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2757)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2769)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2756)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3023)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2761)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 639)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2767)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2773)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2753)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2758)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2749)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3022)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2768)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2747)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2754)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2760)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2745)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3009)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3012)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3013)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3014)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3018)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 633)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 635)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3019)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3020)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3021)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2739)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2740)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2741)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2752)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2765)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2770)
I20260709 12:12:32.977922 18983 raft_consensus.cc:697] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:32.988536 18984 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.189s	user 0.002s	sys 0.000s
I20260709 12:12:32.970829 18989 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:33.009100 12671 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:12:33.011571 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:33.012778 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:33.014071 12671 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 17: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:12:33.015358 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 134920us
I20260709 12:12:32.978716 18972 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 14.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:12:33.002156 18901 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:32.996215 18756 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:33.025013 13304 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:33.025552 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:33.027088 13335 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:33.029062 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:33.030315 12671 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:33.009438 18944 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:33.015955 18920 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:33.021347 18877 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:33.001863 18773 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:33.036787 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:33.029749 18745 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:33.030393 18753 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:33.052947 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3103) took 2219 ms. Trace:
I20260709 12:12:33.031592 18993 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:33.040318 18989 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:33.079291 18773 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:33.081171 19000 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:33.041965 18920 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:33.071218 18986 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.129s	user 0.000s	sys 0.001s
W20260709 12:12:33.072185 18979 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.154s	user 0.000s	sys 0.002s
I20260709 12:12:33.060575 18740 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:33.064602 18993 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:33.092597 18993 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:33.041392 18944 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:33.066694 18983 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:33.094515 18862 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:33.096771 18862 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:33.099416 18862 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:33.132362 18967 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.899s	user 0.000s	sys 0.012s
W20260709 12:12:33.143589 18936 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.928s	user 0.020s	sys 0.032s
I20260709 12:12:33.144986 18990 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:33.075280 18989 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:33.148334 18989 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:33.087970 19000 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:33.151055 18920 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:33.151996 18944 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:33.155575 18993 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:33.157778 19000 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:33.157810 18993 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:33.199637 12523 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: []
W20260709 12:12:33.200742 12643 leader_election.cc:336] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.570s (SENT)
I20260709 12:12:33.201229 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2733) took 1228 ms. Trace:
I20260709 12:12:33.201617 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:33.203861 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 2995) took 3065 ms. Trace:
W20260709 12:12:33.206614 12643 leader_election.cc:336] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.807s (SENT)
I20260709 12:12:33.207183 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3008) took 1867 ms. Trace:
I20260709 12:12:33.207435 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:33.204474 12600 rpcz_store.cc:276] 0709 12:12:30.137942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:30.149622 (+ 11680us) service_pool.cc:224] Handling call
0709 12:12:30.149930 (+   308us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:30.568121 (+418191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:30.568179 (+    58us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:30.571196 (+  3017us) log.cc:844] Serialized 26 byte log entry
0709 12:12:31.099684 (+528488us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:31.099750 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:31.099863 (+   113us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:33.142940 (+2043077us) spinlock_profiling.cc:243] Waited 3.12 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:33.202847 (+ 59907us) raft_consensus.cc:1685] finished
0709 12:12:33.202930 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:33.203529 (+   599us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10005,"raft.queue_time_us":98700,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":131,"spinlock_wait_cycles":7185664,"thread_start_us":839733,"threads_started":2,"wal-append.queue_time_us":551881}
I20260709 12:12:33.199455 12524 raft_consensus.cc:2368] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18.
I20260709 12:12:33.207679 12597 rpcz_store.cc:276] 0709 12:12:31.339452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:31.713424 (+373972us) service_pool.cc:224] Handling call
0709 12:12:31.713829 (+   405us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 12:12:31.902256 (+188427us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:12:31.902885 (+   629us) log.cc:844] Serialized 14 byte log entry
0709 12:12:32.191565 (+288680us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:12:32.193374 (+  1809us) log.cc:844] Serialized 78 byte log entry
0709 12:12:32.193530 (+   156us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:32.194744 (+  1214us) log.cc:844] Serialized 14 byte log entry
0709 12:12:32.197374 (+  2630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:32.197488 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:33.206392 (+1008904us) raft_consensus.cc:1685] finished
0709 12:12:33.206451 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:33.206928 (+   477us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13849,"prepare.queue_time_us":1293,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":100,"raft.queue_time_us":38545,"raft.run_cpu_time_us":424,"raft.run_wall_time_us":16194,"thread_start_us":271314,"threads_started":1,"wal-append.queue_time_us":322558}
I20260709 12:12:33.201769 12523 rpcz_store.cc:276] 0709 12:12:31.972502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.197715 (+1225213us) service_pool.cc:224] Handling call
0709 12:12:33.199023 (+  1308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:33.200763 (+  1740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:33.201116 (+   353us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.219897 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:33.058838 12675 rpcz_store.cc:276] 0709 12:12:30.833671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:32.158094 (+1324423us) service_pool.cc:224] Handling call
0709 12:12:32.159581 (+  1487us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:32.181140 (+ 21559us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:32.181190 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:32.183002 (+  1812us) log.cc:844] Serialized 26 byte log entry
0709 12:12:32.297650 (+114648us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:32.297778 (+   128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:32.297935 (+   157us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:33.052522 (+754587us) raft_consensus.cc:1685] finished
0709 12:12:33.052567 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:33.052836 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2000,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":18874,"thread_start_us":64535,"threads_started":1,"wal-append.queue_time_us":279390}
I20260709 12:12:33.227284 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3104) took 2393 ms. Trace:
I20260709 12:12:33.095458 18586 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:33.114356 18862 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:33.240660 18877 raft_consensus.cc:697] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:33.243770 18877 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:33.133330 18966 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:33.247790 12675 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:33.228057 12673 rpcz_store.cc:276] 0709 12:12:30.833942 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:32.158651 (+1324709us) service_pool.cc:224] Handling call
0709 12:12:32.159069 (+   418us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:32.172542 (+ 13473us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:32.172594 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:32.173273 (+   679us) log.cc:844] Serialized 26 byte log entry
0709 12:12:32.475014 (+301741us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:32.478405 (+  3391us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:32.478538 (+   133us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:33.226463 (+747925us) raft_consensus.cc:1685] finished
0709 12:12:33.226518 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:33.226915 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9556,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":2745,"thread_start_us":276920,"threads_started":1,"wal-append.queue_time_us":661200}
I20260709 12:12:33.205363 12570 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:51782 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2742)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2762)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2772)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2748)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2755)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 634)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2744)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2750)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2763)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2757)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2769)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2756)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3023)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2761)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 639)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2767)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2773)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2753)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2758)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2749)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3022)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3027)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2768)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2747)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2754)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2760)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2745)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3009)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3012)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3013)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3014)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3018)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 633)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 635)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3019)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3020)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3021)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2739)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2740)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2741)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2752)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2765)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3028)
I20260709 12:12:33.270735 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3118) took 1099 ms. Trace:
I20260709 12:12:33.277547 12675 rpcz_store.cc:276] 0709 12:12:32.169535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.228178 (+1058643us) service_pool.cc:224] Handling call
0709 12:12:33.229209 (+  1031us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:33.258620 (+ 29411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:33.268960 (+ 10340us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:33.284960 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 259925us
W20260709 12:12:33.289319 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:33.289778 12570 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51782 dropped due to backpressure: service queue is full with 50 items [suppressed 4 similar messages]
I20260709 12:12:33.295436 18966 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:33.299361 12570 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51782 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2772)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2748)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2755)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 634)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2744)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2750)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2763)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2757)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2769)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2756)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3023)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2761)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 639)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2767)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2773)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2753)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2758)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3030)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2749)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3022)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3027)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2768)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3032)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2747)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2754)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2760)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2745)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3009)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3012)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3013)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3014)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3018)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 633)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 635)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3019)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3020)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3021)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2739)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2740)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2741)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2752)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2765)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3028)
I20260709 12:12:33.311721 18989 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:33.162083 18980 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:33.315650 18980 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:33.310530 18993 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:33.320652 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:33.322667 12643 leader_election.cc:336] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 3.280s (SENT)
I20260709 12:12:33.323521 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:33.324740 18993 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:33.324906 12673 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:33.326491 13305 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:33.327292 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 630) took 2808 ms. Trace:
I20260709 12:12:33.328926 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3006) took 1995 ms. Trace:
I20260709 12:12:33.329281 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3120) took 1158 ms. Trace:
I20260709 12:12:33.330090 18966 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:33.332861 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:33.339010 12604 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 637) took 1362 ms (client timeout 1592 ms). Trace:
I20260709 12:12:33.333838 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2736) took 1497 ms. Trace:
W20260709 12:12:33.339588 12604 rpcz_store.cc:269] 0709 12:12:31.976333 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.332038 (+1355705us) service_pool.cc:224] Handling call
0709 12:12:33.338892 (+  6854us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.335105 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3004) took 2002 ms. Trace:
I20260709 12:12:33.325076 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:33.340773 12606 rpcz_store.cc:276] 0709 12:12:31.333031 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:31.333351 (+   320us) service_pool.cc:224] Handling call
0709 12:12:31.333668 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:31.471859 (+138191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:31.471895 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:31.471942 (+    47us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:31.471997 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:33.334350 (+1862353us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b5400310bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:33.334477 (+   127us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:33.334820 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":317294336}
I20260709 12:12:33.342473 12646 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:33.342990 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:33.343444 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:33.344121 12606 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:12:33.344581 18973 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:33.345064 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2772) took 1153 ms. Trace:
I20260709 12:12:33.345240 18973 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:33.345798 18973 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:33.347604 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:33.352339 18993 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:32.238302 12572 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51792 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 638)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 637)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2742)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2762)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2772)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2748)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2755)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 634)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2744)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2750)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2763)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2757)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2769)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2756)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2743)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3023)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2761)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 639)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2767)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2773)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2753)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2758)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2749)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3022)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2768)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2747)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2754)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2760)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2745)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3009)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3012)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3013)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3014)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3018)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 633)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 635)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3019)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3020)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3021)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2739)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2740)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2741)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2752)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2765)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2770)
I20260709 12:12:33.345551 12606 rpcz_store.cc:276] 0709 12:12:32.191436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.342560 (+1151124us) service_pool.cc:224] Handling call
0709 12:12:33.344961 (+  2401us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.352909 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2766) took 1205 ms. Trace:
I20260709 12:12:33.354338 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:33.363634 12604 rpcz_store.cc:276] 0709 12:12:32.147395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.347160 (+1199765us) service_pool.cc:224] Handling call
0709 12:12:33.352846 (+  5686us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.355418 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:33.349761 18499 consensus_meta.cc:287] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.507s	user 0.008s	sys 0.017s
I20260709 12:12:33.329746 12675 rpcz_store.cc:276] 0709 12:12:32.170453 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.307282 (+1136829us) service_pool.cc:224] Handling call
0709 12:12:33.328184 (+ 20902us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:33.328859 (+   675us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:33.328894 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:33.328973 (+    79us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:33.329039 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:33.329109 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:33.329219 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.351322 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3007) took 2017 ms. Trace:
I20260709 12:12:33.344839 12531 raft_consensus.cc:2393] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 15: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:33.325960 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3119) took 1156 ms. Trace:
I20260709 12:12:33.329504 12602 rpcz_store.cc:276] 0709 12:12:31.333756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:31.335612 (+  1856us) service_pool.cc:224] Handling call
0709 12:12:31.335923 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:31.710400 (+374477us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:31.710433 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:31.710476 (+    43us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:31.710507 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:33.328197 (+1617690us) spinlock_profiling.cc:243] Waited 374 ms on lock 0x7b54001946d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:33.328281 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:33.328658 (+   377us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":860697472}
I20260709 12:12:33.366004 12673 rpcz_store.cc:276] 0709 12:12:32.169913 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.304964 (+1135051us) service_pool.cc:224] Handling call
0709 12:12:33.324242 (+ 19278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:33.325543 (+  1301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:33.325841 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.360121 12680 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:33.352828 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2737) took 1330 ms. Trace:
W20260709 12:12:33.310262 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 358918us
I20260709 12:12:33.373373 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:33.372172 12524 rpcz_store.cc:276] 0709 12:12:32.022871 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:32.388401 (+365530us) service_pool.cc:224] Handling call
0709 12:12:33.352578 (+964177us) spinlock_profiling.cc:243] Waited 762 ms on lock 0x7b5400408654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:33.352774 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1753218816}
I20260709 12:12:33.374832 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2755) took 1243 ms. Trace:
I20260709 12:12:33.375393 12602 rpcz_store.cc:276] 0709 12:12:32.131387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.372790 (+1241403us) service_pool.cc:224] Handling call
0709 12:12:33.374754 (+  1964us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.375778 12676 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:12:33.376658 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3121) took 1205 ms. Trace:
I20260709 12:12:33.377156 12676 rpcz_store.cc:276] 0709 12:12:32.170770 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.358013 (+1187243us) service_pool.cc:224] Handling call
0709 12:12:33.373345 (+ 15332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:33.376291 (+  2946us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:33.376600 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.357856 12671 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:12:33.378607 18949 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:33.379374 12597 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:12:33.380039 12597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2762) took 1244 ms (client timeout 1592 ms). Trace:
I20260709 12:12:33.380118 13330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 795) took 2614 ms. Trace:
I20260709 12:12:33.379264 18944 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.002s
W20260709 12:12:33.380484 12597 rpcz_store.cc:269] 0709 12:12:32.135368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.354998 (+1219630us) service_pool.cc:224] Handling call
0709 12:12:33.379938 (+ 24940us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.381877 18944 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:33.376652 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:33.382886 12673 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:12:33.382803 18944 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:33.385035 18920 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:33.382019 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:33.376821 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:33.356806 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:33.359045 12646 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:33.392925 12600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2742) took 1277 ms (client timeout 1570 ms). Trace:
W20260709 12:12:33.393528 12600 rpcz_store.cc:269] 0709 12:12:32.112540 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.353899 (+1241359us) service_pool.cc:224] Handling call
0709 12:12:33.389692 (+ 35793us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.380688 13330 rpcz_store.cc:276] 0709 12:12:30.765149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:30.765335 (+   186us) service_pool.cc:224] Handling call
0709 12:12:30.765773 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:32.231852 (+1466079us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:33.379465 (+1147613us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b540061e9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:33.379808 (+   343us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3362130304}
I20260709 12:12:33.386009 18920 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:33.406754 12602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 634) took 1669 ms (client timeout 2181 ms). Trace:
W20260709 12:12:33.407461 12602 rpcz_store.cc:269] 0709 12:12:31.736940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.376374 (+1639434us) service_pool.cc:224] Handling call
0709 12:12:33.406655 (+ 30281us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.413331 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:33.413136 18920 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:33.427570 18942 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.835s	user 0.006s	sys 0.026s
I20260709 12:12:33.427765 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:33.429571 12597 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:33.417302 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:33.418583 12674 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:12:33.431209 18901 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:33.431746 18901 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:33.432127 18901 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:33.434334 18858 raft_consensus.cc:3055] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:33.434819 18858 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:33.435067 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:33.436352 18858 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:33.440419 18914 raft_consensus.cc:3055] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:33.328183 12598 rpcz_store.cc:276] 0709 12:12:30.519028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:30.920010 (+400982us) service_pool.cc:224] Handling call
0709 12:12:30.944823 (+ 24813us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:31.125231 (+180408us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:12:31.409713 (+284482us) log.cc:844] Serialized 14 byte log entry
0709 12:12:32.071670 (+661957us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:32.071748 (+    78us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:12:32.071812 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:33.315806 (+1243994us) spinlock_profiling.cc:243] Waited 180 ms on lock 0x7b54001167d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:33.315885 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:33.327026 (+ 11141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":60920,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":4300,"raft.queue_time_us":101,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":1642,"spinlock_wait_cycles":414137088,"thread_start_us":849270,"threads_started":2,"wal-append.queue_time_us":723136}
I20260709 12:12:33.440963 18914 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:33.442047 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:33.443078 12598 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:33.443248 18922 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:33.444156 18922 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:33.447301 19004 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:33.449105 19004 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:12:33.449205 18937 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 2.099s	user 0.000s	sys 0.015s
I20260709 12:12:33.388295 18499 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:33.451028 19004 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:12:33.451478 18945 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.770s	user 0.004s	sys 0.006s
I20260709 12:12:33.453027 18499 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:33.419606 13331 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:12:33.420111 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:33.385092 18944 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:33.455749 18981 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:33.456216 18944 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:33.456437 18981 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:33.421885 18878 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:33.458606 18878 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:33.460134 18968 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.234s	user 0.000s	sys 0.005s
I20260709 12:12:33.422477 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:33.422596 13339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 796) took 2485 ms. Trace:
I20260709 12:12:33.463107 13339 rpcz_store.cc:276] 0709 12:12:30.937495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:30.937728 (+   233us) service_pool.cc:224] Handling call
0709 12:12:30.938157 (+   429us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:31.008091 (+ 69934us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:31.008123 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:31.017642 (+  9519us) log.cc:844] Serialized 52 byte log entry
0709 12:12:31.476680 (+459038us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:31.476727 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:33.421795 (+1945068us) spinlock_profiling.cc:243] Waited 58.4 ms on lock 0x7b54004439d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:33.421889 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:33.421965 (+    76us) raft_consensus.cc:1685] finished
0709 12:12:33.421977 (+    12us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:33.422364 (+   387us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9683,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":14773,"spinlock_wait_cycles":134326016,"thread_start_us":306694,"threads_started":1,"wal-append.queue_time_us":786426,"wal-append.run_cpu_time_us":6445,"wal-append.run_wall_time_us":1205576}
I20260709 12:12:33.423570 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:33.465920 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2750) took 1349 ms. Trace:
I20260709 12:12:33.466377 12600 rpcz_store.cc:276] 0709 12:12:32.116794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.423095 (+1306301us) service_pool.cc:224] Handling call
0709 12:12:33.465819 (+ 42724us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.467556 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:33.468461 12600 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:33.469075 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2769) took 1307 ms. Trace:
I20260709 12:12:33.469448 12600 rpcz_store.cc:276] 0709 12:12:32.161367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.467115 (+1305748us) service_pool.cc:224] Handling call
0709 12:12:33.468997 (+  1882us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.365712 12603 rpcz_store.cc:276] 0709 12:12:31.334054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:31.686967 (+352913us) service_pool.cc:224] Handling call
0709 12:12:31.687502 (+   535us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:31.994848 (+307346us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:12:31.995234 (+   386us) log.cc:844] Serialized 14 byte log entry
0709 12:12:32.197781 (+202547us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:32.197858 (+    77us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:32.197918 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:33.350710 (+1152792us) spinlock_profiling.cc:243] Waited 307 ms on lock 0x7b54002ffb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:33.350796 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:33.351079 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6247,"prepare.queue_time_us":16964,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":68,"raft.queue_time_us":6375,"spinlock_wait_cycles":715694848,"thread_start_us":195800,"threads_started":1,"wal-append.queue_time_us":248934}
I20260709 12:12:33.452584 19004 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:33.471349 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:33.471591 19004 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:33.472440 12603 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 14: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:33.473096 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2756) took 1341 ms. Trace:
W20260709 12:12:33.454388 18946 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.857s	user 0.009s	sys 0.006s
I20260709 12:12:33.473518 12603 rpcz_store.cc:276] 0709 12:12:32.131667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.470831 (+1339164us) service_pool.cc:224] Handling call
0709 12:12:33.473021 (+  2190us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:33.472353 12522 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 688) took 2441 ms (client timeout 2146 ms). Trace:
I20260709 12:12:33.474200 12528 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:33.454807 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:33.425771 13334 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 627) took 1815 ms (client timeout 2005 ms). Trace:
W20260709 12:12:33.455654 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 69454.2us
I20260709 12:12:33.461760 18878 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:33.418889 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:12:33.475775 12522 rpcz_store.cc:269] 0709 12:12:31.030525 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:31.031147 (+   622us) service_pool.cc:224] Handling call
0709 12:12:33.471863 (+2440716us) spinlock_profiling.cc:243] Waited 125 ms on lock 0x7b54001068d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:33.472116 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":287873920}
W20260709 12:12:33.477394 13334 rpcz_store.cc:269] 0709 12:12:31.609936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.412446 (+1802510us) service_pool.cc:224] Handling call
0709 12:12:33.425591 (+ 13145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.454283 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:33.484575 12606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2748) took 1368 ms (client timeout 1760 ms). Trace:
W20260709 12:12:33.485162 12606 rpcz_store.cc:269] 0709 12:12:32.116215 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.391696 (+1275481us) service_pool.cc:224] Handling call
0709 12:12:33.484493 (+ 92797us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:33.430415 18943 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.957s	user 0.002s	sys 0.015s
I20260709 12:12:33.493144 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:33.339803 12672 rpcz_store.cc:276] 0709 12:12:31.837687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:32.828338 (+990651us) service_pool.cc:224] Handling call
0709 12:12:32.835339 (+  7001us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:32.836073 (+   734us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:12:32.970765 (+134692us) log.cc:844] Serialized 14 byte log entry
0709 12:12:33.330909 (+360144us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:33.331025 (+   116us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:33.333155 (+  2130us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:33.333288 (+   133us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:33.333560 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51846,"prepare.queue_time_us":246852,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"raft.queue_time_us":514,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":245,"thread_start_us":448917,"threads_started":2}
W20260709 12:12:33.496156 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 137477us
I20260709 12:12:33.504312 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:33.505424 18499 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:33.506001 18499 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:33.506484 18499 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:33.506629 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:33.508149 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:33.509317 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:33.510234 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3023) took 1587 ms. Trace:
I20260709 12:12:33.510553 12600 rpcz_store.cc:276] 0709 12:12:31.923337 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.493717 (+1570380us) service_pool.cc:224] Handling call
0709 12:12:33.510163 (+ 16446us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.511713 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:33.505440 13339 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:33.512472 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2771) took 1327 ms. Trace:
I20260709 12:12:33.514492 12600 rpcz_store.cc:276] 0709 12:12:32.184674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.511341 (+1326667us) service_pool.cc:224] Handling call
0709 12:12:33.512418 (+  1077us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.457985 18981 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:33.515736 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:33.518500 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:33.487529 18993 consensus_queue.cc:1243] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 16.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:12:33.516759 18981 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:33.524284 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:33.527582 12522 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:33.537806 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:33.534806 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:33.540949 12531 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
I20260709 12:12:33.530458 18528 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:33.562546 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:33.564508 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2737) took 1654 ms. Trace:
W20260709 12:12:33.567974 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54260.5us
I20260709 12:12:33.499828 18914 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:33.580524 12522 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:33.582109 12603 raft_consensus.cc:2410] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
I20260709 12:12:33.582266 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 639) took 1592 ms. Trace:
I20260709 12:12:33.584247 12600 rpcz_store.cc:276] 0709 12:12:31.989995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.525211 (+1535216us) service_pool.cc:224] Handling call
0709 12:12:33.582190 (+ 56979us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.586450 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2743) took 1473 ms. Trace:
I20260709 12:12:33.586973 12603 rpcz_store.cc:276] 0709 12:12:32.112867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.475832 (+1362965us) service_pool.cc:224] Handling call
0709 12:12:33.586365 (+110533us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.587625 12674 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:12:33.589102 12597 raft_consensus.cc:2410] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:12:33.593300 12602 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:12:33.593283 12597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2744) took 1476 ms (client timeout 1807 ms). Trace:
I20260709 12:12:33.567564 12677 rpcz_store.cc:276] 0709 12:12:31.909538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:32.830065 (+920527us) service_pool.cc:224] Handling call
0709 12:12:32.830551 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:33.026654 (+196103us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:33.026686 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:33.027215 (+   529us) log.cc:844] Serialized 26 byte log entry
0709 12:12:33.339093 (+311878us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:33.339940 (+   847us) log.cc:844] Serialized 14 byte log entry
0709 12:12:33.340703 (+   763us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:33.340843 (+   140us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:33.560396 (+219553us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b180033f3a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:12:33.563834 (+  3438us) raft_consensus.cc:1685] finished
0709 12:12:33.563997 (+   163us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:33.564232 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24359,"prepare.queue_time_us":208,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":60,"raft.queue_time_us":77440,"raft.run_cpu_time_us":364,"raft.run_wall_time_us":7710,"spinlock_wait_cycles":61848832,"thread_start_us":469526,"threads_started":2,"wal-append.queue_time_us":276553}
W20260709 12:12:33.600167 12602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2763) took 1464 ms (client timeout 1899 ms). Trace:
W20260709 12:12:33.600832 12602 rpcz_store.cc:269] 0709 12:12:32.135682 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.480718 (+1345036us) service_pool.cc:224] Handling call
0709 12:12:33.600070 (+119352us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:33.594107 12597 rpcz_store.cc:269] 0709 12:12:32.113165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.381535 (+1268370us) service_pool.cc:224] Handling call
0709 12:12:33.589456 (+207921us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.602548 18769 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:33.603003 12606 raft_consensus.cc:2410] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:12:33.614504 13336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 801) took 1971 ms. Trace:
I20260709 12:12:33.615039 13336 rpcz_store.cc:276] 0709 12:12:31.643316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.613063 (+1969747us) service_pool.cc:224] Handling call
0709 12:12:33.613539 (+   476us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:33.613892 (+   353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:33.613921 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:33.613986 (+    65us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:33.614043 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:33.614099 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:33.614266 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.594659 18796 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:33.593760 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:33.630985 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:33.609709 12522 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:12:33.635366 13339 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:33.635037 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:33.645476 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:33.651284 18740 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.029s
W20260709 12:12:33.655524 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 80737.2us
I20260709 12:12:33.657684 18878 consensus_queue.cc:1059] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 13.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.004s
I20260709 12:12:33.632539 18796 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:33.669945 12646 leader_election.cc:400] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18.
I20260709 12:12:33.670334 12646 leader_election.cc:403] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:33.671901 12524 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 12:12:33.665587 18796 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:33.674003 12603 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 14: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:33.674705 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2773) took 1483 ms. Trace:
I20260709 12:12:33.675087 12603 rpcz_store.cc:276] 0709 12:12:32.191773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.591104 (+1399331us) service_pool.cc:224] Handling call
0709 12:12:33.674634 (+ 83530us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.667279 12494 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:33.676456 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2722) took 1726 ms. Trace:
I20260709 12:12:33.672322 12524 pending_rounds.cc:85] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 12
I20260709 12:12:33.682641 12527 rpcz_store.cc:276] 0709 12:12:31.949755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:31.952259 (+  2504us) service_pool.cc:224] Handling call
0709 12:12:31.952641 (+   382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:32.291547 (+338906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:33.675991 (+1384444us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b54000eef54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:33.676268 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":766187904}
I20260709 12:12:33.686328 12527 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:33.669395 18914 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:33.676183 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:33.682902 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 5 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:33.682875 12524 pending_rounds.cc:107] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 12:12:33.697485 12524 raft_consensus.cc:2889] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:33.697635 12600 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
W20260709 12:12:33.698382 12600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2767) took 1549 ms (client timeout 2061 ms). Trace:
W20260709 12:12:33.698714 12600 rpcz_store.cc:269] 0709 12:12:32.148730 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.585086 (+1436356us) service_pool.cc:224] Handling call
0709 12:12:33.698320 (+113234us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:33.700562 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 202275us
W20260709 12:12:33.706467 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 109657us
I20260709 12:12:33.707358 12646 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:33.708824 12646 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:33.710268 18877 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:33.710779 18877 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:33.711140 18877 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:33.715268 19002 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:12:33.717217 18922 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:33.718394 12603 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:33.718611 18990 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:33.724951 18990 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:33.725597 18990 raft_consensus.cc:697] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:33.676672 18920 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:33.722615 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:33.728242 12597 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:33.728731 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2758) took 1596 ms. Trace:
I20260709 12:12:33.729344 12597 rpcz_store.cc:276] 0709 12:12:32.132202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.602251 (+1470049us) service_pool.cc:224] Handling call
0709 12:12:33.728667 (+126416us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.734175 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:12:33.747263 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 116270us
W20260709 12:12:33.754716 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:33.775075 12602 raft_consensus.cc:2410] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 14 because replica has last-logged OpId of term: 14 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 5.
W20260709 12:12:33.779579 12602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2753) took 1662 ms (client timeout 2113 ms). Trace:
W20260709 12:12:33.780670 12602 rpcz_store.cc:269] 0709 12:12:32.117608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.634388 (+1516780us) service_pool.cc:224] Handling call
0709 12:12:33.779492 (+145104us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.778810 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:33.744488 18920 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:33.788269 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:33.793855 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:33.797127 18949 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:33.797791 18528 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:33.800823 18949 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:12:33.802284 12678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 3092) took 2941 ms (client timeout 1549 ms). Trace:
W20260709 12:12:33.802817 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58192.6us
W20260709 12:12:33.802995 12678 rpcz_store.cc:269] 0709 12:12:30.808007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:30.830487 (+ 22480us) service_pool.cc:224] Handling call
0709 12:12:33.746884 (+2916397us) spinlock_profiling.cc:243] Waited 30.2 ms on lock 0x7b54002c4a54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:33.749381 (+  2497us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":69351680}
I20260709 12:12:33.809533 18862 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:12:33.810272 18862 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 23 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:33.811496 18862 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:33.798194 12679 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:12:33.818547 18753 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:33.819623 18499 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:33.823243 18677 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:12:33.824632 18677 raft_consensus.cc:3055] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:33.825062 18677 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:33.826915 18677 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:33.829036 18677 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:33.744838 18990 consensus_queue.cc:237] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:33.831938 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:33.839071 12529 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2666) took 3087 ms (client timeout 2022 ms). Trace:
W20260709 12:12:33.839623 12529 rpcz_store.cc:269] 0709 12:12:30.751650 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:30.751766 (+   116us) service_pool.cc:224] Handling call
0709 12:12:33.837999 (+3086233us) spinlock_profiling.cc:243] Waited 69 ms on lock 0x7b54003f8c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:33.838838 (+   839us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":158652288}
W20260709 12:12:33.840742 12599 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 638) took 1855 ms (client timeout 1522 ms). Trace:
W20260709 12:12:33.841197 12599 rpcz_store.cc:269] 0709 12:12:31.985483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.840300 (+1854817us) service_pool.cc:211] Skipping call since client already timed out
0709 12:12:33.840508 (+   208us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:12:33.842592 18992 raft_consensus.cc:3055] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:33.843276 18992 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:33.843629 18726 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:33.844555 18726 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:33.845080 18992 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:33.847445 18944 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:33.848197 18944 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:33.848692 18944 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:33.853273 18855 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:12:33.776839 18901 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:33.857026 18901 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:33.860147 18796 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:33.864182 18922 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:33.867801 18796 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:33.868999 12600 raft_consensus.cc:2393] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:33.868870 18723 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:33.879050 12598 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:33.880515 12603 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
W20260709 12:12:33.908600 12598 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2757) took 1750 ms (client timeout 1916 ms). Trace:
W20260709 12:12:33.909339 12598 rpcz_store.cc:269] 0709 12:12:32.131924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.441540 (+1309616us) service_pool.cc:224] Handling call
0709 12:12:33.882145 (+440605us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.902743 12597 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:33.816461 12572 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:51782 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3027)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2778)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3039)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3033)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2768)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3032)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3038)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2747)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2754)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3037)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2779)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2760)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 640)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2787)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 641)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2780)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2786)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 642)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2785)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2776)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3034)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2774)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2782)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2775)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2745)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2783)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2788)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2781)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2777)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2784)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3009)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3012)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3013)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3014)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3018)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 633)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 635)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3019)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3020)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3021)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2739)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2740)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2741)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2751)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2752)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2764)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2765)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2770)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3028)
W20260709 12:12:33.930269 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:33.934350 18723 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:33.936525 19015 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.598s	user 0.003s	sys 0.000s
I20260709 12:12:33.939508 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:33.940469 12601 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:33.940960 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2749) took 1824 ms. Trace:
I20260709 12:12:33.947690 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:33.948604 12605 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:33.949036 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2768) took 1797 ms. Trace:
I20260709 12:12:33.949604 18920 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:33.951062 18723 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:33.949344 12605 rpcz_store.cc:276] 0709 12:12:32.152064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.947161 (+1795097us) service_pool.cc:224] Handling call
0709 12:12:33.948975 (+  1814us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:33.951782 12606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2761) took 1816 ms (client timeout 2030 ms). Trace:
W20260709 12:12:33.952195 12606 rpcz_store.cc:269] 0709 12:12:32.135021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.488861 (+1353840us) service_pool.cc:224] Handling call
0709 12:12:33.951582 (+462721us) spinlock_profiling.cc:243] Waited 83.3 ms on lock 0x7b5400271ad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:33.951724 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":191698816}
I20260709 12:12:33.952518 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:33.953214 12605 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:33.953291 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:33.941615 12600 rpcz_store.cc:276] 0709 12:12:32.116503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.722212 (+1605709us) service_pool.cc:224] Handling call
0709 12:12:33.940646 (+218434us) spinlock_profiling.cc:243] Waited 138 ms on lock 0x7b54001617d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:33.940881 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":316486912}
W20260709 12:12:33.956008 13302 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.522s (SENT)
I20260709 12:12:33.956570 13302 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:33.960253 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:33.965451 13333 raft_consensus.cc:2410] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:12:33.965107 18877 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:33.965237 19004 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:33.956593 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:33.972070 13337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 798) took 2656 ms. Trace:
I20260709 12:12:33.969102 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:33.994601 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:33.998750 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 201641us
I20260709 12:12:33.971827 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:33.999465 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 243252us
W20260709 12:12:33.969714 12526 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2662) took 3221 ms (client timeout 1979 ms). Trace:
I20260709 12:12:34.001824 13337 rpcz_store.cc:276] 0709 12:12:31.315992 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:31.319824 (+  3832us) service_pool.cc:224] Handling call
0709 12:12:31.320270 (+   446us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:32.049486 (+729216us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:33.971387 (+1921901us) spinlock_profiling.cc:243] Waited 729 ms on lock 0x7b540055c8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:33.971850 (+   463us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1675608320}
I20260709 12:12:33.970661 12525 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:34.015966 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2754) took 1885 ms. Trace:
I20260709 12:12:34.016786 12603 rpcz_store.cc:276] 0709 12:12:32.117870 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.993743 (+1875873us) service_pool.cc:224] Handling call
0709 12:12:34.003145 (+  9402us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:33.968099 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3115) took 1807 ms. Trace:
I20260709 12:12:34.020556 12679 rpcz_store.cc:276] 0709 12:12:32.160690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:32.907331 (+746641us) service_pool.cc:224] Handling call
0709 12:12:33.764220 (+856889us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:33.818598 (+ 54378us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:33.967850 (+149252us) spinlock_profiling.cc:243] Waited 842 ms on lock 0x7b54002c4a50. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:33.968032 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1936204672}
W20260709 12:12:34.028861 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:34.015697 12526 rpcz_store.cc:269] 0709 12:12:30.747669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:30.747797 (+   128us) service_pool.cc:224] Handling call
0709 12:12:33.969242 (+3221445us) spinlock_profiling.cc:243] Waited 1.32 s on lock 0x7b54003cb9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:33.969515 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3024554240}
I20260709 12:12:33.973734 18772 raft_consensus.cc:3055] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:33.966189 18499 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:34.033905 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3022) took 2183 ms. Trace:
I20260709 12:12:34.035943 12602 rpcz_store.cc:276] 0709 12:12:31.850485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.792005 (+1941520us) service_pool.cc:224] Handling call
0709 12:12:34.033817 (+241812us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:34.037210 12598 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:33.966697 18726 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:34.048015 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2724) took 2097 ms. Trace:
I20260709 12:12:34.048498 12530 rpcz_store.cc:276] 0709 12:12:31.950538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:31.953689 (+  3151us) service_pool.cc:224] Handling call
0709 12:12:31.967725 (+ 14036us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:32.902722 (+934997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.047490 (+1144768us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b54003c1754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:34.047828 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1892275968}
I20260709 12:12:34.050410 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:34.051424 12602 raft_consensus.cc:2410] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 15 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:12:34.053047 18993 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:34.040021 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:34.034698 18772 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:34.053771 18993 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:34.054565 18772 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:34.056026 18772 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:34.056367 18993 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:34.057188 18973 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.710s	user 0.012s	sys 0.003s
I20260709 12:12:34.046564 18753 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:34.058055 18987 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:34.052371 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.064010 18987 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:34.064651 12603 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:34.067241 12678 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:12:34.069725 18987 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:34.057209 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:34.076725 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:34.086246 12601 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:34.088768 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.092397 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:34.093741 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 290059us
I20260709 12:12:34.057685 18973 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:34.103152 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.058456 18753 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:34.108107 12678 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:34.120476 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.082046 18858 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Advancing to term 14
W20260709 12:12:34.126442 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 134445us
I20260709 12:12:34.083011 18914 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:34.132562 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: true dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:34.133883 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2743) took 1728 ms. Trace:
W20260709 12:12:34.138157 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:34.140307 12604 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 17: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:34.140592 12599 raft_consensus.cc:2368] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17.
I20260709 12:12:34.146447 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.150776 12601 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:12:34.151894 12602 raft_consensus.cc:2410] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:12:34.152472 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2760) took 2017 ms. Trace:
I20260709 12:12:34.152786 12602 rpcz_store.cc:276] 0709 12:12:32.134619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.052041 (+1917422us) service_pool.cc:224] Handling call
0709 12:12:34.152410 (+100369us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:34.137593 12523 rpcz_store.cc:276] 0709 12:12:32.395092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.225756 (+830664us) service_pool.cc:224] Handling call
0709 12:12:33.302821 (+ 77065us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:33.312607 (+  9786us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:12:33.572804 (+260197us) log.cc:844] Serialized 14 byte log entry
0709 12:12:34.004058 (+431254us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:34.038954 (+ 34896us) log.cc:844] Serialized 26 byte log entry
0709 12:12:34.039193 (+   239us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:34.039245 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.039359 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:34.117012 (+ 77653us) raft_consensus.cc:1685] finished
0709 12:12:34.117048 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:34.123615 (+  6567us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3135,"prepare.queue_time_us":360394,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":68,"raft.queue_time_us":6239,"raft.run_cpu_time_us":451,"raft.run_wall_time_us":7210,"thread_start_us":664143,"threads_started":2,"wal-append.queue_time_us":493376}
W20260709 12:12:34.159695 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 333092us
I20260709 12:12:34.164053 12606 raft_consensus.cc:2410] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:12:34.136569 18914 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:34.167689 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.165524 12598 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:34.173178 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2745) took 1777 ms. Trace:
I20260709 12:12:34.173365 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3027) took 1166 ms. Trace:
I20260709 12:12:34.174388 12598 rpcz_store.cc:276] 0709 12:12:33.007467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.971212 (+963745us) service_pool.cc:224] Handling call
0709 12:12:34.173306 (+202094us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:34.183393 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2730) took 2220 ms. Trace:
I20260709 12:12:34.179589 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.189900 12603 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:34.191133 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:34.191727 12603 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:34.199419 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:34.183871 12528 rpcz_store.cc:276] 0709 12:12:31.962859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:32.914879 (+952020us) service_pool.cc:224] Handling call
0709 12:12:33.473701 (+558822us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:33.486635 (+ 12934us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.183172 (+696537us) spinlock_profiling.cc:243] Waited 558 ms on lock 0x7b54001068d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:34.183326 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1284120320}
W20260709 12:12:34.206108 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:34.205724 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:34.184734 18769 consensus_meta.cc:287] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.581s	user 0.011s	sys 0.002s
I20260709 12:12:34.212155 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.213730 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.216220 12603 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:34.217839 13335 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:34.218704 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:34.214093 12606 raft_consensus.cc:2410] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:12:34.210443 12677 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:12:34.158667 18944 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:34.173652 12531 rpcz_store.cc:276] 0709 12:12:32.395696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.575136 (+1179440us) service_pool.cc:224] Handling call
0709 12:12:33.654180 (+ 79044us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:33.695270 (+ 41090us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:12:34.000337 (+305067us) log.cc:844] Serialized 14 byte log entry
0709 12:12:34.170939 (+170602us) log.cc:844] Serialized 14 byte log entry
0709 12:12:34.171424 (+   485us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:34.171569 (+   145us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:34.171714 (+   145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.172977 (+  1263us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:34.173116 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5802,"prepare.queue_time_us":27318,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":104,"raft.queue_time_us":8695,"raft.run_cpu_time_us":230,"raft.run_wall_time_us":232,"thread_start_us":393847,"threads_started":2}
I20260709 12:12:34.212708 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.227847 12604 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 16: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:34.221724 18769 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:34.229631 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:34.229995 12600 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:34.232355 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:34.234102 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.236122 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:34.237056 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:34.237510 12606 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 14: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:34.240831 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:34.241565 12676 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:34.241154 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:34.242336 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.237097 12674 raft_consensus.cc:2376] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16. Re-sending same reply.
W20260709 12:12:34.257175 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:34.244163 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:34.258728 12680 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:34.258919 18914 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:34.259886 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2747) took 2145 ms. Trace:
I20260709 12:12:34.260411 18914 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:34.247714 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.260310 12600 rpcz_store.cc:276] 0709 12:12:32.115949 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:33.956244 (+1840295us) service_pool.cc:224] Handling call
0709 12:12:34.259735 (+303491us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b5400231254. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:34.259827 (+    92us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":522535680}
I20260709 12:12:34.262804 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.263605 12600 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 12.
I20260709 12:12:34.264674 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:34.251183 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.256402 12606 raft_consensus.cc:2410] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 16 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:12:34.266737 12598 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:34.267709 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 689) took 3236 ms. Trace:
I20260709 12:12:34.268110 12525 rpcz_store.cc:276] 0709 12:12:31.030815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:31.030982 (+   167us) service_pool.cc:224] Handling call
0709 12:12:33.970287 (+2939305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:34.038382 (+ 68095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.267473 (+229091us) spinlock_profiling.cc:243] Waited 2.94 s on lock 0x7b54003cb9d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:34.267624 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6758675840}
I20260709 12:12:34.267534 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.269659 12606 raft_consensus.cc:2368] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18.
I20260709 12:12:34.271250 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.272686 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.273506 12605 raft_consensus.cc:2410] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:12:34.267722 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.274663 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:34.275393 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:34.275951 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 156345us
I20260709 12:12:34.261288 12601 raft_consensus.cc:2376] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14. Re-sending same reply.
I20260709 12:12:34.276736 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:34.277451 12671 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:34.279402 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:34.282356 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.275573 12677 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:34.283102 12601 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:34.284162 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.284947 12601 raft_consensus.cc:2393] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:34.285728 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.287487 18992 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:34.308812 12671 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:34.306106 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.212803 19027 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.162s
I20260709 12:12:34.213862 19002 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18.
I20260709 12:12:34.293366 18987 consensus_queue.cc:1059] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.006s
I20260709 12:12:34.327732 12600 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:34.299389 18877 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:34.293565 18499 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:12:34.293605 18990 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
I20260709 12:12:34.359582 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.293763 18993 consensus_queue.cc:1059] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.007s
I20260709 12:12:34.290774 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:34.296229 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:34.296087 18819 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 12:12:34.297727 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:34.300280 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.298671 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:34.294797 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:34.365135 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:34.365796 12523 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:12:34.305276 12601 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:34.312837 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.316284 12604 raft_consensus.cc:2410] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 18 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 9.
W20260709 12:12:34.320228 19034 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:12:34.320667 12678 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:34.326573 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:34.364203 12677 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:34.341322 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:34.341933 12671 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: [16.9-16.9]
W20260709 12:12:34.371320 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 93512.6us
I20260709 12:12:34.346032 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:34.301167 18973 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:34.276160 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2745) took 2162 ms. Trace:
I20260709 12:12:34.367507 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:34.373832 12527 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: [15.11-15.11]
I20260709 12:12:34.370419 12606 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 14: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:34.339237 12524 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:34.372033 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.372663 12603 raft_consensus.cc:2376] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19. Re-sending same reply.
I20260709 12:12:34.364800 12602 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:34.370373 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:34.374197 12522 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:34.369427 12605 raft_consensus.cc:2410] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 16 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
W20260709 12:12:34.374431 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 158901us
I20260709 12:12:34.373610 12598 rpcz_store.cc:276] 0709 12:12:32.113441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.267407 (+2153966us) service_pool.cc:224] Handling call
0709 12:12:34.276094 (+  8687us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:34.375051 12528 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:12:34.380990 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.384398 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:34.385212 12601 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:34.389899 12676 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:34.391098 12530 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:34.390228 12531 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:34.392402 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:34.392798 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 99933.2us
I20260709 12:12:34.393074 12606 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:34.387176 12599 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:34.396593 13337 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
I20260709 12:12:34.397450 12526 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:34.397421 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.398360 12602 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:34.398612 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.400487 12597 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:34.400720 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.401918 12600 raft_consensus.cc:2376] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18. Re-sending same reply.
I20260709 12:12:34.402424 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.404070 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.404376 12529 raft_consensus.cc:2410] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 15 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
I20260709 12:12:34.406840 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.402449 12494 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:34.370764 19004 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 12:12:34.413694 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:34.415740 12605 raft_consensus.cc:2410] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 19 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
W20260709 12:12:34.443691 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 43983.4us
I20260709 12:12:34.427079 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:34.442762 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 11 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:34.454037 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49555.2us
I20260709 12:12:34.449769 18877 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 12:12:34.456918 12597 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 11. (term mismatch)
I20260709 12:12:34.457342 12597 pending_rounds.cc:85] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 10
I20260709 12:12:34.457561 12597 pending_rounds.cc:107] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:12:34.457787 12597 raft_consensus.cc:2889] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:34.459920 12598 raft_consensus.cc:2410] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 12 because replica has last-logged OpId of term: 11 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 9 index: 6.
I20260709 12:12:34.460033 12604 raft_consensus.cc:2410] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 17 because replica has last-logged OpId of term: 14 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
W20260709 12:12:34.463280 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 80313us
I20260709 12:12:34.467056 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3009) took 3118 ms. Trace:
I20260709 12:12:34.468199 12597 rpcz_store.cc:276] 0709 12:12:31.339836 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.415402 (+3075566us) service_pool.cc:224] Handling call
0709 12:12:34.456536 (+ 41134us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:34.458323 (+  1787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.458519 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:34.463127 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:34.471494 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:34.477806 12527 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:34.487159 12599 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:34.488406 12600 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:12:34.479657 12496 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:34.487892 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3014) took 3143 ms. Trace:
I20260709 12:12:34.491331 12599 rpcz_store.cc:276] 0709 12:12:31.344717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.435267 (+3090550us) service_pool.cc:224] Handling call
0709 12:12:34.478937 (+ 43670us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:34.487616 (+  8679us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.487831 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:34.496119 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3012) took 3147 ms. Trace:
I20260709 12:12:34.496626 12600 rpcz_store.cc:276] 0709 12:12:31.341587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.412546 (+3070959us) service_pool.cc:224] Handling call
0709 12:12:34.475041 (+ 62495us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:34.488814 (+ 13773us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.488999 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:34.499050 12604 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:34.473317 18772 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:34.477671 18987 consensus_queue.cc:1243] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 19.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:12:34.506174 12605 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:12:34.504979 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 635) took 2767 ms. Trace:
I20260709 12:12:34.483270 18944 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:34.508507 12604 rpcz_store.cc:276] 0709 12:12:31.737205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.467860 (+2730655us) service_pool.cc:224] Handling call
0709 12:12:34.495408 (+ 27548us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:34.504643 (+  9235us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.504892 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:34.508550 12598 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:34.518889 12597 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:34.519585 13302 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:34.519860 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3019) took 2670 ms. Trace:
I20260709 12:12:34.508100 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3013) took 3166 ms. Trace:
I20260709 12:12:34.520633 12605 rpcz_store.cc:276] 0709 12:12:31.341879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.452579 (+3110700us) service_pool.cc:224] Handling call
0709 12:12:34.493227 (+ 40648us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:34.506777 (+ 13550us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.508025 (+  1248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:34.521308 12522 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:34.520385 12597 rpcz_store.cc:276] 0709 12:12:31.849471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.477616 (+2628145us) service_pool.cc:224] Handling call
0709 12:12:34.508246 (+ 30630us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:34.519509 (+ 11263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.519769 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:34.524187 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 633) took 2787 ms. Trace:
I20260709 12:12:34.524741 12598 rpcz_store.cc:276] 0709 12:12:31.736641 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.477267 (+2740626us) service_pool.cc:224] Handling call
0709 12:12:34.508330 (+ 31063us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:34.523795 (+ 15465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.524109 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:34.448930 18862 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:12:34.499730 18819 raft_consensus.cc:3055] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:34.535455 12600 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:34.536211 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3021) took 2686 ms. Trace:
I20260709 12:12:34.536525 12600 rpcz_store.cc:276] 0709 12:12:31.850107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.497425 (+2647318us) service_pool.cc:224] Handling call
0709 12:12:34.535215 (+ 37790us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:34.535976 (+   761us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.536159 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:34.533188 12599 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:34.539835 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3020) took 2690 ms. Trace:
I20260709 12:12:34.540338 12599 rpcz_store.cc:276] 0709 12:12:31.849796 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.492474 (+2642678us) service_pool.cc:224] Handling call
0709 12:12:34.527163 (+ 34689us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:34.539506 (+ 12343us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.539761 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:34.511190 18772 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:34.510071 18723 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
W20260709 12:12:34.533921 19039 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.134s	user 0.001s	sys 0.001s
I20260709 12:12:34.557440 19050 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.003s
I20260709 12:12:34.563366 12599 raft_consensus.cc:1217] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Deduplicated request from leader. Original: 10.10->[12.11-12.11]   Dedup: 12.11->[]
I20260709 12:12:34.564204 12599 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 13. Ops: [12.11-12.11]
I20260709 12:12:34.564986 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2759) took 2430 ms. Trace:
I20260709 12:12:34.565380 12599 rpcz_store.cc:276] 0709 12:12:32.134318 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.541316 (+2406998us) service_pool.cc:224] Handling call
0709 12:12:34.563176 (+ 21860us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:34.564583 (+  1407us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.564878 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:34.566679 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:34.577715 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:34.568384 12605 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 12. Current term is 14. Ops: []
I20260709 12:12:34.579121 13337 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:12:34.534991 18819 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:12:34.566782 19038 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:12:34.576608 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:34.588773 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2739) took 2477 ms. Trace:
I20260709 12:12:34.589401 12598 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: [18.9-18.9]
I20260709 12:12:34.589180 12605 rpcz_store.cc:276] 0709 12:12:32.111631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.534309 (+2422678us) service_pool.cc:224] Handling call
0709 12:12:34.568165 (+ 33856us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:34.588436 (+ 20271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.588686 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:34.591437 12604 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:12:34.587441 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:34.595216 12601 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:34.597831 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2741) took 2485 ms. Trace:
I20260709 12:12:34.598624 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2751) took 2481 ms. Trace:
I20260709 12:12:34.598352 12598 rpcz_store.cc:276] 0709 12:12:32.112251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.542337 (+2430086us) service_pool.cc:224] Handling call
0709 12:12:34.589184 (+ 46847us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:34.589879 (+   695us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.597759 (+  7880us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:34.599334 12604 rpcz_store.cc:276] 0709 12:12:32.117071 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.533487 (+2416416us) service_pool.cc:224] Handling call
0709 12:12:34.553772 (+ 20285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:34.591810 (+ 38038us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.598533 (+  6723us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:34.606094 19040 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
W20260709 12:12:34.608165 19041 log.cc:927] Time spent T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
W20260709 12:12:34.610559 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 144708us
I20260709 12:12:34.611528 18990 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:34.612409 12496 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:34.612751 18990 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:34.614970 12600 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:34.615655 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2752) took 2498 ms. Trace:
I20260709 12:12:34.615948 19047 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:34.616586 19047 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:34.613745 19014 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:34.618323 18692 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:34.618672 19014 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:34.618888 18692 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:34.619323 19014 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:34.620299 18692 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:34.621562 18692 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:34.581545 18877 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:34.624481 18772 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:34.627022 18819 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:34.615976 12600 rpcz_store.cc:276] 0709 12:12:32.117338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.546272 (+2428934us) service_pool.cc:224] Handling call
0709 12:12:34.614591 (+ 68319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:34.615392 (+   801us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.615595 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:34.628738 18819 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:34.629776 12600 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:34.613531 18990 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:34.634557 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 139467us
I20260709 12:12:34.634835 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:34.635736 18677 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
W20260709 12:12:34.635859 12494 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:51782 dropped due to backpressure: service queue is full with 50 items [suppressed 42 similar messages]
I20260709 12:12:34.636199 18677 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:34.634948 12599 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:12:34.637837 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2764) took 2501 ms. Trace:
I20260709 12:12:34.638343 12599 rpcz_store.cc:276] 0709 12:12:32.135984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.587894 (+2451910us) service_pool.cc:224] Handling call
0709 12:12:34.634571 (+ 46677us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:34.637478 (+  2907us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.637764 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:34.613574 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:34.636121 13302 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:34.642644 12604 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:12:34.646420 12604 pending_rounds.cc:85] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 8
I20260709 12:12:34.647038 12604 pending_rounds.cc:107] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 15.9
W20260709 12:12:34.647428 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 60064.8us
I20260709 12:12:34.636754 18677 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:34.647766 12604 raft_consensus.cc:2889] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:34.642711 12678 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:34.654026 12530 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:34.656855 18877 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:34.648944 19014 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:34.670751 18877 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:34.671312 18877 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:34.672824 19014 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:34.673242 19002 raft_consensus.cc:3055] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:34.673738 19002 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:34.668246 18677 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:34.674809 18973 raft_consensus.cc:3055] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:34.675141 19031 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:34.675354 18973 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:34.675714 19031 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:34.676168 19031 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:34.676879 18973 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:34.662456 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:34.678997 13335 raft_consensus.cc:2410] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:12:34.679870 18677 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:34.681223 12676 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:12:34.682799 12527 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:34.683537 18303 raft_consensus.cc:2764] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:12:34.685729 12643 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:34.686686 19045 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:34.687923 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:34.689265 12531 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:34.690344 12529 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:34.692447 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:34.696979 12603 raft_consensus.cc:2410] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 7.
W20260709 12:12:34.697587 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48279.7us
I20260709 12:12:34.698583 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:34.698936 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:34.699162 12601 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:34.698601 18990 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:34.705261 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:34.668308 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:34.706197 12523 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
I20260709 12:12:34.707000 12646 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:34.648649 18858 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.514s	user 0.012s	sys 0.001s
I20260709 12:12:34.726994 12606 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:34.744737 12526 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:34.759992 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2770) took 2582 ms. Trace:
I20260709 12:12:34.762867 12601 rpcz_store.cc:276] 0709 12:12:32.165832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.637920 (+2472088us) service_pool.cc:224] Handling call
0709 12:12:34.698988 (+ 61068us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:34.747808 (+ 48820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.748069 (+   261us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:34.768872 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 61993us
I20260709 12:12:34.770440 12496 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:34.771699 12598 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:12:34.772706 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3028) took 1760 ms. Trace:
I20260709 12:12:34.773195 12598 rpcz_store.cc:276] 0709 12:12:33.012449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.638108 (+1625659us) service_pool.cc:224] Handling call
0709 12:12:34.762273 (+124165us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:34.772302 (+ 10029us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:34.772635 (+   333us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:34.778878 12646 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:34.783721 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:34.807349 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:34.743515 19045 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:34.756457 19050 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:34.810130 12646 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:34.825958 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:34.806787 18990 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:34.829269 18990 raft_consensus.cc:697] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:34.810139 19052 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:34.832238 19052 raft_consensus.cc:697] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:12:34.833441 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 71923.3us
I20260709 12:12:34.833216 19052 consensus_queue.cc:237] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:34.833624 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:34.834805 18858 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:34.835419 18858 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:34.835561 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:34.836445 18858 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:34.836884 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:34.825716 18819 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:34.838937 18819 raft_consensus.cc:697] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:34.839710 18987 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:12:34.839998 18819 consensus_queue.cc:237] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:34.834568 18914 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:34.842196 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:34.842527 19002 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:34.843523 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:34.843657 19002 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:34.844769 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:34.844900 19032 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:12:34.846113 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:34.846240 18973 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:34.846875 12531 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 14: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:34.847441 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:34.848455 19032 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:12:34.850332 18677 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:34.852283 12523 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:12:34.843828 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:34.856251 12526 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 16. Ops: [14.9-14.9]
I20260709 12:12:34.853984 18990 consensus_queue.cc:237] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 12.6, Last appended by leader: 6, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:34.861311 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:34.781440 18303 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:34.849973 12529 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 12:12:34.867298 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:34.881381 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 195742us
I20260709 12:12:34.890735 19014 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:34.904758 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:34.897445 19050 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.013s
W20260709 12:12:34.911196 18992 consensus_meta.cc:287] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.589s	user 0.000s	sys 0.010s
I20260709 12:12:34.897683 18677 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:34.914466 18677 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:34.915652 18753 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.781s	user 0.006s	sys 0.006s
I20260709 12:12:34.916159 18753 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:34.916673 18677 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:34.918406 12524 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:12:34.921180 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:34.920919 18877 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:34.927511 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:34.930503 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:34.926329 19050 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.009s
I20260709 12:12:34.891249 19045 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:34.938256 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:34.941466 13337 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:12:34.904142 19047 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:34.951632 12527 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
W20260709 12:12:34.954731 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 318882us
W20260709 12:12:34.972518 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:34.974046 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:34.950500 19004 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:34.953270 19045 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:34.929179 18303 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.037s
W20260709 12:12:34.863700 18993 consensus_queue.cc:1175] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" responder_term: 19 status { last_received { term: 19 index: 11 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:12:34.953006 19050 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:12:34.999073 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:35.000208 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:35.009083 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:35.009743 18692 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:34.953240 18992 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:35.019718 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:35.021096 19050 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:35.031325 19050 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:35.032550 19054 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.084s	user 0.000s	sys 0.001s
I20260709 12:12:35.033104 19050 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:35.042923 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:35.052757 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:35.053682 12604 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
W20260709 12:12:35.063863 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 63806.6us
I20260709 12:12:35.037204 19053 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:35.069813 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:35.073096 12643 leader_election.cc:400] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17.
I20260709 12:12:35.074375 12643 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:35.076130 18769 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:35.076786 18769 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:35.078984 18769 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:35.028203 19004 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:35.086254 19004 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:35.087674 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:35.088413 12527 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
I20260709 12:12:35.089632 18753 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:35.100137 12604 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 14: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:35.026077 18692 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:35.114725 18753 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:35.117336 18769 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:35.118041 18769 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:35.119936 18987 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:35.121394 18769 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:35.124650 18877 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:35.104744 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 57988us
I20260709 12:12:35.104964 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:35.047570 18992 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:35.155236 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 80949.6us
W20260709 12:12:35.161829 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:35.162508 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 511114us
I20260709 12:12:35.156730 18769 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:35.165030 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:35.165196 18769 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:35.165266 18753 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:12:35.168118 18692 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:35.137941 18944 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:35.170192 12643 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:35.156157 18877 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:35.191004 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:35.168712 18692 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:35.155995 19004 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:35.201346 12571 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:35.202860 13335 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 18. Current term is 19. Ops: []
W20260709 12:12:35.197916 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 319120us
I20260709 12:12:35.207196 18753 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:35.215418 18753 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:35.221839 12678 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:35.223871 12673 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:35.225345 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:35.226608 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:35.227435 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:35.230566 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:35.233687 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:35.236013 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:35.239480 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:35.240698 12678 raft_consensus.cc:2410] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:12:35.243116 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:35.252707 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:35.257185 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:35.258881 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:35.200842 19004 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:35.281863 12671 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 16 index: 11. (term mismatch)
I20260709 12:12:35.282626 12671 pending_rounds.cc:85] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 10
I20260709 12:12:35.289112 12671 pending_rounds.cc:107] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 12:12:35.265949 18973 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:35.198606 18944 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:35.289529 12671 raft_consensus.cc:2889] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:35.280840 19004 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:35.271719 19066 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:35.310787 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:35.253646 18877 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:35.255117 18692 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:35.283041 12572 connection.cc:256] RPC call timeout handler was delayed by -2.4193763230000513s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19790000000000002s for a response
I20260709 12:12:35.318415 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:35.319111 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 149730us
W20260709 12:12:35.322739 12572 connection.cc:256] RPC call timeout handler was delayed by -2.3806763229999888s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2022s for a response
I20260709 12:12:35.321000 12677 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:35.331120 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:35.334591 12678 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
W20260709 12:12:35.342046 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 185773us
I20260709 12:12:35.349856 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:35.357731 12572 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:35.359851 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 201942us
W20260709 12:12:35.345544 19031 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.669s	user 0.012s	sys 0.003s
I20260709 12:12:35.340699 18858 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:35.387986 18944 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:35.315272 19066 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:35.403173 12673 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:35.338469 18772 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.052s
W20260709 12:12:35.412084 19064 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
W20260709 12:12:35.414662 19057 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:12:35.424188 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:35.424906 12673 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:35.428140 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3018) took 4011 ms. Trace:
I20260709 12:12:35.432890 13338 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 23 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23. Ops: []
I20260709 12:12:35.397156 18769 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
W20260709 12:12:35.440580 19059 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
W20260709 12:12:35.451683 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:35.452989 19062 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.101s	user 0.000s	sys 0.001s
I20260709 12:12:35.428016 19070 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.049s
W20260709 12:12:35.433737 19065 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:12:35.438680 18753 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:35.457700 18753 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:35.466152 18692 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:35.466351 18753 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:35.468015 18753 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:35.390762 19031 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:35.472446 12675 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:35.432972 12602 rpcz_store.cc:276] 0709 12:12:31.395612 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.455502 (+3059890us) service_pool.cc:224] Handling call
0709 12:12:34.496779 (+ 41277us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:34.504369 (+  7590us) raft_consensus.cc:1534] Early marking committed up to index 6
0709 12:12:34.944010 (+439641us) log.cc:844] Serialized 14 byte log entry
0709 12:12:35.396602 (+452592us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:35.396662 (+    60us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:35.406497 (+  9835us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:35.406926 (+   429us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:35.407216 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":662067,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"thread_start_us":735735,"threads_started":3}
W20260709 12:12:35.484687 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 140986us
I20260709 12:12:35.487969 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:35.491046 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:35.493495 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 135892us
I20260709 12:12:35.398450 18858 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:35.458808 18992 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:35.486531 19071 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:12:35.509557 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:35.511314 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:35.512305 12673 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:35.515395 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2740) took 3403 ms. Trace:
W20260709 12:12:35.515750 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:35.514173 13338 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
W20260709 12:12:35.523731 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:35.511087 19026 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.010s	sys 0.029s
I20260709 12:12:35.523345 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:12:35.537575 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 215090us
I20260709 12:12:35.524257 18769 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:35.534363 18992 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:35.541393 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:35.559440 12529 raft_consensus.cc:2410] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 18 because replica has last-logged OpId of term: 16 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 11.
I20260709 12:12:35.550450 12571 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:35.561757 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 75976.8us
I20260709 12:12:35.564138 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:35.573863 12675 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:12:35.575006 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:35.545779 19063 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.197s	user 0.001s	sys 0.001s
I20260709 12:12:35.523869 19074 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:35.585237 12522 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
W20260709 12:12:35.580631 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 65105.7us
W20260709 12:12:35.540849 19026 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.010s	sys 0.030s
I20260709 12:12:35.587496 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:35.588860 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2765) took 3452 ms. Trace:
I20260709 12:12:35.595863 12527 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 16 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:12:35.553097 18858 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:35.587821 12494 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:35.606061 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 94056.4us
I20260709 12:12:35.611153 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:35.613720 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:35.593837 12605 rpcz_store.cc:276] 0709 12:12:32.136503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.591632 (+2455129us) service_pool.cc:224] Handling call
0709 12:12:34.666625 (+ 74993us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:34.767854 (+101229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:34.767911 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:34.824247 (+ 56336us) log.cc:844] Serialized 26 byte log entry
0709 12:12:35.263407 (+439160us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:35.263471 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:35.263580 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:35.583885 (+320305us) raft_consensus.cc:1685] finished
0709 12:12:35.583909 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:35.588768 (+  4859us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":75700,"raft.queue_time_us":167288,"raft.run_cpu_time_us":82,"raft.run_wall_time_us":83,"thread_start_us":338336,"threads_started":1,"wal-append.queue_time_us":428139}
I20260709 12:12:35.599793 18877 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:35.629427 18972 raft_consensus.cc:2764] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:12:35.629698 19087 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:12:35.630159 19089 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
W20260709 12:12:35.631536 19084 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:12:35.649814 18617 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:35.617462 19067 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:35.609555 18992 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:12:35.649187 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 85115.7us
W20260709 12:12:35.652072 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 88680.3us
I20260709 12:12:35.666149 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:35.671819 19094 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:12:35.653965 18987 consensus_meta.cc:287] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.533s	user 0.014s	sys 0.000s
I20260709 12:12:35.644860 19087 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:35.681270 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 69428.4us
I20260709 12:12:35.686450 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:35.687587 12674 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:35.690428 19094 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:12:35.690937 19081 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
W20260709 12:12:35.704993 19050 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.671s	user 0.008s	sys 0.005s
W20260709 12:12:35.705708 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 92224.4us
I20260709 12:12:35.706583 12494 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:35.707813 12680 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:12:35.667539 18914 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:35.709760 19031 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:35.710721 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2764) took 1026 ms. Trace:
I20260709 12:12:35.710929 12523 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:12:35.711118 12525 rpcz_store.cc:276] 0709 12:12:34.684098 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.686014 (+  1916us) service_pool.cc:224] Handling call
0709 12:12:34.686734 (+   720us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:34.687272 (+   538us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:34.687303 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:34.687913 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 12:12:35.023488 (+335575us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:35.023551 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:35.023735 (+   184us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:35.710495 (+686760us) raft_consensus.cc:1685] finished
0709 12:12:35.710516 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:35.710647 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":114,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":122,"thread_start_us":335130,"threads_started":1,"wal-append.queue_time_us":510308}
I20260709 12:12:35.636021 18862 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 12:12:35.678617 19066 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:35.676657 19074 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.039s
I20260709 12:12:35.676973 18617 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:35.717192 18617 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:35.676393 19069 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
I20260709 12:12:35.684127 18987 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:35.723815 19004 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:35.686465 19076 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:12:35.725615 19087 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:35.734705 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:35.735674 12527 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:35.752324 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:35.753986 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 47488.9us
I20260709 12:12:35.755597 18723 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:35.756201 18723 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:35.756417 19094 raft_consensus.cc:3055] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:35.756583 18723 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:35.756824 19094 raft_consensus.cc:740] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:35.757686 19094 consensus_queue.cc:260] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:35.758992 19094 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:35.755676 12494 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:35.760857 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 78479.6us
W20260709 12:12:35.763622 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:35.762807 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:35.767120 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 88825.1us
I20260709 12:12:35.768702 12680 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 18 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:12:35.770314 12572 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:35.813971 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:35.818413 12680 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:35.818490 12527 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:12:35.760798 18769 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:35.820991 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:12:35.826706 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46320.6us
W20260709 12:12:35.830282 19079 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:12:35.821316 19074 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:35.778980 18303 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:35.865281 12680 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:12:35.812820 18944 raft_consensus.cc:2764] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:12:35.827152 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:35.902281 12572 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:12:35.907442 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 92247us
I20260709 12:12:35.515867 12597 rpcz_store.cc:276] 0709 12:12:32.111962 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.533417 (+2421455us) service_pool.cc:224] Handling call
0709 12:12:34.590105 (+ 56688us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:34.668551 (+ 78446us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:34.668616 (+    65us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:34.669365 (+   749us) log.cc:844] Serialized 26 byte log entry
0709 12:12:34.970347 (+300982us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:35.282782 (+312435us) log.cc:844] Serialized 14 byte log entry
0709 12:12:35.456003 (+173221us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b18003dfde0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:12:35.501985 (+ 45982us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:35.502073 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:35.502140 (+    67us) raft_consensus.cc:1685] finished
0709 12:12:35.515031 (+ 12891us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:35.515323 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14997,"prepare.queue_time_us":295028,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":6193,"raft.queue_time_us":250873,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":5547,"spinlock_wait_cycles":6104960,"thread_start_us":612254,"threads_started":3,"wal-append.queue_time_us":613435}
I20260709 12:12:35.821939 19004 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:35.912595 12494 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:35.914738 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:35.900913 19027 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:35.905126 18862 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:35.920495 12496 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:35.920807 13302 leader_election.cc:336] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.592s (SENT)
I20260709 12:12:35.922961 12494 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:35.924016 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:35.913096 19070 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:35.922924 19027 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:35.925267 18862 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:35.944252 19027 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:35.922420 18753 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:35.947052 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:35.948516 18592 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:35.949487 18592 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:35.955648 18592 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:35.956233 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2682) took 1255 ms. Trace:
I20260709 12:12:35.957000 18592 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:35.957824 19100 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:12:35.958837 12680 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:35.959151 18677 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:12:35.960314 12680 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:12:35.960247 18677 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:12:35.967869 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49090.3us
I20260709 12:12:35.973316 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:35.914570 18303 raft_consensus.cc:697] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:35.915943 19099 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:35.930401 13302 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:35.931770 12496 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:35.970258 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 48311.6us
I20260709 12:12:35.948608 12527 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 18: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:35.966466 19014 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:35.925495 19066 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 12:12:35.956656 12523 rpcz_store.cc:276] 0709 12:12:34.700932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:34.853237 (+152305us) service_pool.cc:224] Handling call
0709 12:12:35.710650 (+857413us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:35.711443 (+   793us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:35.956011 (+244568us) spinlock_profiling.cc:243] Waited 857 ms on lock 0x7b54003cc3d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:35.956167 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1970979712}
I20260709 12:12:35.925099 18944 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.018s
I20260709 12:12:35.978456 19032 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:35.978693 19031 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:12:35.983430 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 51762.8us
I20260709 12:12:35.974054 18753 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:35.997618 18944 consensus_queue.cc:1243] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 23.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:12:35.979851 19070 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:35.998399 18944 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:12:35.986218 19032 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:35.982860 18723 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:35.991817 19066 consensus_queue.cc:1243] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:12:35.991546 19004 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:35.986970 18617 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:36.013686 18914 raft_consensus.cc:3055] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:36.014601 18617 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:36.014804 18914 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:36.015103 18617 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:36.016088 18914 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:36.017513 18914 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:35.998591 12496 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:36.019816 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:35.998433 18753 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.022706 19102 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:12:36.023854 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 256034us
I20260709 12:12:36.024662 12571 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:36.026598 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:36.004369 18303 consensus_queue.cc:237] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.026607 19066 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:36.028458 19066 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:36.025761 19102 raft_consensus.cc:3055] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:36.029909 19102 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:36.026007 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:36.030936 19074 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:36.022877 19014 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:36.032212 19014 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:36.029109 19066 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:36.032564 19014 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:36.031423 12571 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:36.034696 19074 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:36.035219 19074 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:36.035120 19031 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:36.036506 19108 raft_consensus.cc:3055] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:36.036718 12571 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:36.036916 19108 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:36.037891 19108 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.039271 19108 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:36.031701 19102 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.043848 19087 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:35.990051 18862 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:35.999303 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 75367.5us
I20260709 12:12:35.986063 12569 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:36.055330 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 72879.1us
W20260709 12:12:36.003644 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 82863.9us
I20260709 12:12:36.000290 18944 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:36.007462 18723 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:36.006774 19070 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.002203 19032 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:36.008229 18677 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:35.991019 18769 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.058578 19031 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:36.065037 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:36.100814 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:36.043861 19102 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:36.022673 18592 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:36.062379 19067 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:36.079276 18677 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:12:36.081182 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:36.064669 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:36.082381 18723 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260709 12:12:36.082624 19096 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:36.090435 18944 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:36.091670 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:36.091506 12572 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:36.067579 18862 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:36.101245 19031 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:36.117514 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:36.108935 18592 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:36.119508 18862 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:36.120203 18944 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:12:36.106009 19094 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:12:36.142514 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 110283us
I20260709 12:12:36.130765 19031 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:36.114658 19093 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:36.130836 19096 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:36.158993 19095 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.486s	user 0.007s	sys 0.000s
I20260709 12:12:36.161624 12676 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:36.099403 19074 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.114693 19067 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.154454 19096 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:36.204010 18592 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.211440 19032 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:36.212817 12525 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:36.214805 19032 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:36.215672 19114 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:36.220184 18944 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:36.230188 18862 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:36.236614 19087 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:36.165370 18753 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.043s
W20260709 12:12:36.242877 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 576786us
I20260709 12:12:36.244004 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:36.247115 18992 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:12:36.248597 13302 leader_election.cc:336] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.856s (SENT)
I20260709 12:12:36.249221 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:36.247840 19052 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:36.265298 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:36.266942 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:36.265491 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 525537us
I20260709 12:12:36.275822 19032 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.002s
I20260709 12:12:36.277622 19085 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:36.278503 18753 raft_consensus.cc:3055] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:36.278995 18753 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:36.279891 18753 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.282671 12494 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:36.270733 19094 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:36.281229 18753 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:36.297128 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 205753us
W20260709 12:12:36.311054 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 62501.7us
W20260709 12:12:36.320091 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 203483us
I20260709 12:12:36.323124 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:36.275298 19014 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.327662 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:36.328583 13338 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:36.332115 12572 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:36.291216 19115 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:12:36.335741 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 68907.9us
I20260709 12:12:36.343417 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:36.336807 19093 raft_consensus.cc:697] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:36.340243 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:36.341150 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:36.346180 12602 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:36.353336 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:36.291404 19032 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.038s
W20260709 12:12:36.358819 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:36.362295 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:36.365820 12572 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:36.368546 12604 raft_consensus.cc:2410] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 22 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:12:36.374503 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:36.338990 18973 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:12:36.378634 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:36.379283 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:36.385069 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 113194us
I20260709 12:12:36.356309 19070 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:36.401118 12600 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: [17.10-18.11]
W20260709 12:12:36.403148 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:36.404062 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:36.372211 18858 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:36.369923 19099 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:12:36.409626 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 66296us
I20260709 12:12:36.410562 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:36.416069 12603 raft_consensus.cc:2410] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 23 because replica has last-logged OpId of term: 22 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 15.
I20260709 12:12:36.353292 19002 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.037s
I20260709 12:12:36.434937 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:36.436995 19116 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:12:36.438819 12599 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 12:12:36.439082 19113 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.248s	user 0.003s	sys 0.001s
I20260709 12:12:36.439899 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:36.440699 12599 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 14.
I20260709 12:12:36.441624 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:36.406895 19070 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.413151 19114 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:12:36.443970 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:36.444428 19070 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:36.444574 12678 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:36.445185 19067 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:36.409883 19120 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:36.446350 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:36.371446 18914 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.043s
I20260709 12:12:36.449044 12604 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: [17.8-17.8]
I20260709 12:12:36.449905 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:36.452389 12601 raft_consensus.cc:2410] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 19 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 14.
I20260709 12:12:36.455257 12598 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:36.455900 12599 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:12:36.475991 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:36.430478 18973 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:36.480422 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 65125.9us
I20260709 12:12:36.498765 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:36.500015 12679 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:12:36.500597 12598 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 15. (term mismatch)
I20260709 12:12:36.482438 12673 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23. Ops: []
I20260709 12:12:36.501083 12598 pending_rounds.cc:85] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 14
I20260709 12:12:36.501406 12598 pending_rounds.cc:107] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 18.15
I20260709 12:12:36.501664 12598 raft_consensus.cc:2889] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:36.502539 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:36.504969 12599 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:36.508905 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:36.407534 19094 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.523711 12603 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 16. (term mismatch)
I20260709 12:12:36.548072 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:36.548425 12603 pending_rounds.cc:85] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 15
I20260709 12:12:36.474750 19067 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:36.536775 19066 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.504s	user 0.010s	sys 0.003s
I20260709 12:12:36.472451 19032 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.064s
I20260709 12:12:36.496800 19114 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:12:36.479465 18992 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:36.546492 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:36.548765 12603 pending_rounds.cc:107] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 22.16
I20260709 12:12:36.492053 12678 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
W20260709 12:12:36.549234 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 163325us
I20260709 12:12:36.547358 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:36.526240 12600 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:36.552434 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:36.557250 12531 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:12:36.569610 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:36.426244 18858 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:36.571435 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:36.567054 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:36.561576 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:36.545764 12527 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:36.494984 19053 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:36.502583 18914 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:36.540064 18617 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.524s	user 0.013s	sys 0.000s
I20260709 12:12:36.570345 12678 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:36.605621 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:36.606415 12678 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:36.568831 12603 raft_consensus.cc:2889] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:36.579556 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:36.580260 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:36.578266 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:36.562296 19120 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:36.508644 18723 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 12:12:36.613732 12529 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:36.614648 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:36.615367 12531 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 12 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:12:36.617544 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:36.470285 19093 consensus_queue.cc:237] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.646579 12674 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:12:36.513077 18973 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:36.595064 19128 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:36.594678 19099 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:36.648728 13337 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 12:12:36.649669 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 151055us
I20260709 12:12:36.653537 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:36.558044 19115 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.059s
I20260709 12:12:36.655385 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:36.656337 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 103091us
I20260709 12:12:36.610729 19121 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:36.657804 13302 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:36.667003 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:36.666157 12676 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:36.564834 19067 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:36.566689 19066 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.667598 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:36.668143 12604 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:36.679734 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:36.681130 12603 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 9. (term mismatch)
I20260709 12:12:36.682466 13333 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 15. (term mismatch)
I20260709 12:12:36.681695 12603 pending_rounds.cc:85] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 8
I20260709 12:12:36.691747 12603 pending_rounds.cc:107] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:12:36.683712 13333 pending_rounds.cc:85] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 14
I20260709 12:12:36.692034 12603 raft_consensus.cc:2889] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:36.692274 13333 pending_rounds.cc:107] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 16.15
I20260709 12:12:36.692739 13333 raft_consensus.cc:2889] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:36.655238 19047 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:36.654436 19130 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:36.708468 12601 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:36.611083 18617 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.654374 19128 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:36.710741 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 125435us
W20260709 12:12:36.682096 19105 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
I20260709 12:12:36.678972 18862 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:36.621606 18914 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.690618 19014 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:36.758404 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:36.755062 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:36.693975 19074 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:36.778206 12603 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:36.742726 19108 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.683s	user 0.014s	sys 0.002s
I20260709 12:12:36.783377 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:36.795984 12676 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:36.615689 19053 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.751010 18862 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:36.820513 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 167072us
I20260709 12:12:36.738924 19026 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:36.721786 19128 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:36.821499 12598 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 12:12:36.795323 19089 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:12:36.670969 19099 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.656239 19120 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.830330 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:36.831851 18858 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:36.832576 19135 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:36.718858 19047 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.693869 19070 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.088s
I20260709 12:12:36.797185 19002 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:36.678681 19121 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:36.833592 19131 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.149s	user 0.001s	sys 0.000s
W20260709 12:12:36.872412 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 204899us
W20260709 12:12:36.847371 19126 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.291s	user 0.001s	sys 0.000s
I20260709 12:12:36.850502 19108 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:36.827553 18862 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:36.835877 19026 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:36.890398 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:36.910977 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:36.895166 12601 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
I20260709 12:12:36.811600 19074 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:36.897075 12676 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:36.915313 12676 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:12:36.737002 19130 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:36.891426 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:36.907851 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 196264us
I20260709 12:12:36.911691 13302 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:36.912599 18858 consensus_queue.cc:1059] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:12:36.919746 19142 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:36.937216 12599 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:36.948117 12603 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:36.951782 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:36.952903 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:36.978832 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:36.913846 19026 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:36.988852 12599 raft_consensus.cc:2410] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 19 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:12:36.995855 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3166) took 1258 ms. Trace:
W20260709 12:12:36.984866 13338 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.650s	user 0.006s	sys 0.010s
I20260709 12:12:37.015556 13338 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:37.016940 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:37.017653 12603 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
W20260709 12:12:37.027343 12602 consensus_meta.cc:287] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.671s	user 0.014s	sys 0.000s
I20260709 12:12:37.027808 12602 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:36.914672 19108 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:37.050648 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:37.052310 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:37.012157 12680 rpcz_store.cc:276] 0709 12:12:35.737314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:35.961277 (+223963us) service_pool.cc:224] Handling call
0709 12:12:35.961604 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:36.316445 (+354841us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:36.316496 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:36.321489 (+  4993us) log.cc:844] Serialized 26 byte log entry
0709 12:12:36.441771 (+120282us) spinlock_profiling.cc:243] Waited 7.96 ms on lock 0x7b4800060858. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:12:36.450712 (+  8941us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:36.450863 (+   151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:36.451071 (+   208us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:36.985873 (+534802us) raft_consensus.cc:1685] finished
0709 12:12:36.985895 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:36.995769 (+  9874us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":349161,"raft.queue_time_us":359653,"raft.run_cpu_time_us":2333,"raft.run_wall_time_us":117512,"spinlock_wait_cycles":18297600,"thread_start_us":79636,"threads_started":1,"wal-append.queue_time_us":242271}
I20260709 12:12:37.046407 12494 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:36.897137 19135 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:37.095019 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:37.096447 12602 raft_consensus.cc:2376] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18. Re-sending same reply.
I20260709 12:12:37.107355 12603 raft_consensus.cc:2376] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16. Re-sending same reply.
I20260709 12:12:37.041868 18973 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:37.038079 18723 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 12:12:37.023927 18972 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 12:12:37.147818 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:37.096619 19066 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:37.148736 13338 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 19: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:37.038506 19085 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.079s
I20260709 12:12:37.106814 19067 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:37.162295 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:37.160584 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:37.109620 18617 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:36.996129 19142 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:37.152573 18914 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:37.142897 19145 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:37.169246 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:12:37.166163 19031 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.002s	user 0.009s	sys 0.011s
W20260709 12:12:37.184633 19122 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.470s	user 0.001s	sys 0.001s
I20260709 12:12:37.193724 18941 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:37.195902 19053 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:37.196971 19099 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:37.198298 19135 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:37.199601 19047 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:37.202857 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:37.258060 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:37.204878 19052 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.939s	user 0.009s	sys 0.002s
I20260709 12:12:37.205992 19120 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:37.206660 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:37.209954 19130 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:37.215364 19133 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.394s	user 0.002s	sys 0.000s
I20260709 12:12:37.215258 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:37.184770 19144 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.189s
I20260709 12:12:37.186050 19103 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 15.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.147s
I20260709 12:12:37.273195 12603 raft_consensus.cc:2368] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18.
I20260709 12:12:37.274576 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:37.275803 12603 raft_consensus.cc:2410] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 17 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:12:37.280220 12598 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:12:37.261849 19053 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.285972 19052 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:37.166628 19070 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:37.234054 19067 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.234890 19158 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:37.238518 19150 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:37.239604 19121 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:37.242486 19156 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.099s	user 0.002s	sys 0.000s
W20260709 12:12:37.244614 19163 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:12:37.244982 19143 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.245366 18723 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:37.247033 18973 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:37.250437 18941 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:37.252390 18972 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:37.229527 19031 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:37.252851 19074 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:37.296051 12598 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:37.270617 19135 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.278857 18993 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:37.224795 12604 consensus_meta.cc:287] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.534s	user 0.005s	sys 0.005s
I20260709 12:12:37.381114 12604 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:37.383083 12604 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
I20260709 12:12:37.315593 19070 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:37.327751 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 497468us
W20260709 12:12:37.323705 19045 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.000s	sys 0.013s
I20260709 12:12:37.333415 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3044) took 1017 ms. Trace:
W20260709 12:12:37.359344 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:37.310806 19053 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:37.322911 19158 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:37.394295 19158 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:37.395603 19158 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:37.390673 18993 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:37.390424 19045 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.001s	sys 0.014s
I20260709 12:12:37.399806 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3155) took 1733 ms. Trace:
W20260709 12:12:37.400791 19096 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.140s	user 0.007s	sys 0.007s
I20260709 12:12:37.400475 18941 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:37.401341 19096 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:37.402557 12606 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:37.400820 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3154) took 1746 ms. Trace:
I20260709 12:12:37.404259 18993 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:37.386863 18753 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.074s	user 0.005s	sys 0.010s
I20260709 12:12:37.403647 12677 rpcz_store.cc:276] 0709 12:12:35.654584 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:35.654734 (+   150us) service_pool.cc:224] Handling call
0709 12:12:35.672307 (+ 17573us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:36.246833 (+574526us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:36.246882 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:36.247827 (+   945us) log.cc:844] Serialized 52 byte log entry
0709 12:12:36.497476 (+249649us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:36.521111 (+ 23635us) log.cc:844] Serialized 14 byte log entry
0709 12:12:36.567527 (+ 46416us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:36.567620 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:37.400580 (+832960us) raft_consensus.cc:1685] finished
0709 12:12:37.400605 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:37.400757 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":653886,"prepare.queue_time_us":24162,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":625634,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":30366,"spinlock_wait_cycles":7552,"thread_start_us":248950,"threads_started":1,"wal-append.queue_time_us":430912}
W20260709 12:12:37.360347 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 371083us
I20260709 12:12:37.336218 18973 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:37.408998 18973 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:37.410944 19142 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:37.372216 19135 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:37.333143 18723 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:37.321121 19067 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:37.412681 19027 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:12:37.413395 19135 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:37.413797 19027 raft_consensus.cc:3055] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:37.423166 19027 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:37.424330 19027 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:37.425804 19027 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:37.450878 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 49198.5us
I20260709 12:12:37.329494 19103 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.044s
I20260709 12:12:37.360143 19108 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:37.382687 12494 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:37.384099 19119 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.390692 19070 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:37.454511 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:37.462549 19103 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:37.462982 19103 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:37.463366 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:37.463907 19103 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:37.464628 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:37.465242 19103 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:37.466846 12679 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:12:37.467923 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:37.400286 12671 rpcz_store.cc:276] 0709 12:12:35.666250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:35.666802 (+   552us) service_pool.cc:224] Handling call
0709 12:12:35.679129 (+ 12327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:36.263954 (+584825us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:12:36.269663 (+  5709us) log.cc:844] Serialized 14 byte log entry
0709 12:12:36.522952 (+253289us) spinlock_profiling.cc:243] Waited 972 us on lock 0x7b18000278e0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a 00007f1488a2aa2a
0709 12:12:36.961568 (+438616us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:36.982349 (+ 20781us) log.cc:844] Serialized 26 byte log entry
0709 12:12:36.982497 (+   148us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:36.982537 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:36.982602 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:37.241627 (+259025us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:37.399561 (+157934us) raft_consensus.cc:1685] finished
0709 12:12:37.399593 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:37.399740 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":574952,"prepare.queue_time_us":1532,"prepare.run_cpu_time_us":76,"prepare.run_wall_time_us":7793,"raft.queue_time_us":866987,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":189,"spinlock_wait_cycles":41426176,"thread_start_us":438208,"threads_started":2,"wal-append.queue_time_us":744584}
I20260709 12:12:37.468765 12679 raft_consensus.cc:2410] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 18 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:12:37.469506 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:37.407411 19031 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:37.414258 18723 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:37.414331 19067 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:37.379793 12603 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
W20260709 12:12:37.414686 19161 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.091s	user 0.000s	sys 0.000s
I20260709 12:12:37.415468 19096 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:37.419835 19053 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:37.330588 19150 raft_consensus.cc:697] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:37.420915 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3159) took 1685 ms. Trace:
I20260709 12:12:37.373471 19129 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.421427 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:37.383649 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:37.427171 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:37.428503 19171 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:12:37.429780 18617 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.342447 19143 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:37.430665 19168 raft_consensus.cc:3055] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:37.431252 12530 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:37.431258 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:37.432644 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:37.433290 13333 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:37.378345 19146 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.056s
I20260709 12:12:37.390527 12605 rpcz_store.cc:276] 0709 12:12:36.315389 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:36.315574 (+   185us) service_pool.cc:224] Handling call
0709 12:12:36.326681 (+ 11107us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:36.375125 (+ 48444us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:12:36.645713 (+270588us) log.cc:844] Serialized 14 byte log entry
0709 12:12:36.962027 (+316314us) log.cc:844] Serialized 14 byte log entry
0709 12:12:37.238189 (+276162us) spinlock_profiling.cc:243] Waited 37.3 ms on lock 0x7b18001913e0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:12:37.323051 (+ 84862us) log.cc:844] Serialized 14 byte log entry
0709 12:12:37.325368 (+  2317us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:37.325458 (+    90us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:37.325545 (+    87us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:37.325653 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:37.332907 (+  7254us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11738,"prepare.queue_time_us":324669,"prepare.run_cpu_time_us":2911,"prepare.run_wall_time_us":52909,"raft.queue_time_us":441927,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":27732,"spinlock_wait_cycles":111304320,"thread_start_us":816151,"threads_started":3,"wal-append.queue_time_us":447722}
I20260709 12:12:37.435335 19151 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:12:37.438253 19128 consensus_meta.cc:287] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.604s	user 0.011s	sys 0.003s
I20260709 12:12:37.439010 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:37.441816 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:37.443347 12572 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:37.331902 19004 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.137s	user 0.014s	sys 0.012s
I20260709 12:12:37.470034 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:37.456491 19178 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.457078 19045 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260709 12:12:37.457661 19002 consensus_meta.cc:287] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.562s	user 0.008s	sys 0.003s
I20260709 12:12:37.516433 12605 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: [14.8-14.8]
I20260709 12:12:37.458187 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:37.457978 19135 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:37.459024 19164 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:37.507997 19128 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:37.535432 19164 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:37.537259 19128 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:37.459832 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:37.473047 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 90463.4us
I20260709 12:12:37.539772 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:37.541172 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:37.474838 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3043) took 1160 ms. Trace:
I20260709 12:12:37.542284 12597 rpcz_store.cc:276] 0709 12:12:36.314476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:36.314661 (+   185us) service_pool.cc:224] Handling call
0709 12:12:36.319368 (+  4707us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:36.319590 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:12:36.415532 (+ 95942us) log.cc:844] Serialized 14 byte log entry
0709 12:12:37.474078 (+1058546us) spinlock_profiling.cc:243] Waited 37 ms on lock 0x7b18000bffe0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a
0709 12:12:37.474336 (+   258us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:37.474405 (+    69us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:37.474509 (+   104us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:37.474600 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:37.474762 (+   162us) inbound_call.cc:177] Queueing success response
0709 12:12:37.531434 (+ 56672us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
Metrics: {"mutex_wait_us":29070,"prepare.queue_time_us":279861,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":47,"raft.queue_time_us":1558,"raft.run_cpu_time_us":107,"raft.run_wall_time_us":108,"spinlock_wait_cycles":769274880,"thread_start_us":216727,"threads_started":2,"wal-append.queue_time_us":266641}
I20260709 12:12:37.545719 18723 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:37.546506 18723 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:37.546939 18972 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:37.547513 18972 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:37.549086 18723 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:37.499657 19004 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.305s	user 0.016s	sys 0.012s
I20260709 12:12:37.494153 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:37.471051 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:37.470504 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:37.534574 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:37.557742 12523 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:37.498845 12672 rpcz_store.cc:276] 0709 12:12:35.735266 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:35.737799 (+  2533us) service_pool.cc:224] Handling call
0709 12:12:35.738240 (+   441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:36.265239 (+526999us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:36.265269 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:36.277664 (+ 12395us) log.cc:844] Serialized 26 byte log entry
0709 12:12:36.895230 (+617566us) spinlock_profiling.cc:243] Waited 3.87 ms on lock 0x7b18000277c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:12:36.936023 (+ 40793us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:36.936135 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:36.936280 (+   145us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:37.243672 (+307392us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:37.420660 (+176988us) raft_consensus.cc:1685] finished
0709 12:12:37.420687 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:37.420846 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":577161,"raft.queue_time_us":537490,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":95,"spinlock_wait_cycles":56672256,"thread_start_us":40189,"threads_started":1,"wal-append.queue_time_us":735467}
I20260709 12:12:37.487540 19182 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:37.505957 19146 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:37.494429 19119 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:37.569880 12523 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
W20260709 12:12:37.480228 18862 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.518s	user 0.012s	sys 0.000s
I20260709 12:12:37.575778 18862 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:37.576398 12672 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:37.503419 19108 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
W20260709 12:12:37.506727 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 119633us
I20260709 12:12:37.578339 19182 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:37.579083 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:37.469749 12527 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:12:37.579887 12679 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:37.581339 12679 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:12:37.582449 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:37.465786 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:37.583333 12673 raft_consensus.cc:2410] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 19 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:12:37.583652 19119 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:37.498953 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 115385us
W20260709 12:12:37.587638 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:37.549861 18972 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:37.520788 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:37.518231 19053 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:37.518333 19178 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:37.522324 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:37.522802 19135 raft_consensus.cc:3055] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:37.523921 19176 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 15.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:37.529927 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:37.516376 19151 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:37.453135 19142 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.078s
I20260709 12:12:37.635238 19142 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:37.635681 19142 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:37.516785 19145 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:37.636096 19031 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:12:37.507649 18303 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:37.637104 18303 raft_consensus.cc:697] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:37.637992 18303 consensus_queue.cc:237] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:37.638823 19151 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:37.549783 18723 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 12:12:37.641547 19144 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:12:37.532226 19138 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.681s	user 0.003s	sys 0.001s
I20260709 12:12:37.517516 19168 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:37.534377 19002 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:37.643496 19168 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:37.644858 19168 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:37.556911 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:37.561460 12494 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:12:37.646905 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 85553.1us
I20260709 12:12:37.581538 19182 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:37.470983 19066 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.571197 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:37.648658 19066 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:37.573912 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:37.650189 12605 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:37.574074 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:37.515450 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:37.591017 12531 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:12:37.651646 12605 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:37.579454 12643 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:37.592314 12527 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:37.589262 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:37.652784 19108 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:37.653189 12605 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:12:37.653357 19108 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:37.653793 18592 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:37.635681 19070 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:37.654470 19108 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:37.624173 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3168) took 1142 ms. Trace:
I20260709 12:12:37.591578 12646 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:37.537990 19164 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:37.636483 19145 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:37.533051 19129 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:37.499879 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 56626.6us
I20260709 12:12:37.534389 19150 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:37.570819 19114 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.658527 12605 raft_consensus.cc:3055] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:37.659127 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:37.656738 12678 rpcz_store.cc:276] 0709 12:12:36.481361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:36.650289 (+168928us) service_pool.cc:224] Handling call
0709 12:12:36.650633 (+   344us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:36.960636 (+310003us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:36.960685 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:36.961363 (+   678us) log.cc:844] Serialized 26 byte log entry
0709 12:12:37.144384 (+183021us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:37.144432 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:37.425988 (+281556us) spinlock_profiling.cc:243] Waited 4.03 ms on lock 0x7b4800060858. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:12:37.623541 (+197553us) spinlock_profiling.cc:243] Waited 39.9 ms on lock 0x7b54003d45d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:37.623608 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:37.623688 (+    80us) raft_consensus.cc:1685] finished
0709 12:12:37.623703 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:37.624075 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":958,"raft.queue_time_us":469848,"raft.run_cpu_time_us":2386,"raft.run_wall_time_us":261437,"spinlock_wait_cycles":101125760,"thread_start_us":383861,"threads_started":2,"wal-append.queue_time_us":329595}
I20260709 12:12:37.570526 19146 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:37.664741 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:37.652182 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 136867us
I20260709 12:12:37.509266 18617 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:37.493582 19026 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:37.624441 18753 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.672397 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:37.673381 12605 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:37.623836 19053 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:37.624235 19135 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:37.682812 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:37.683251 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:37.695266 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:37.696702 12571 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:37.684677 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:12:37.688320 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 96808.1us
I20260709 12:12:37.671386 19066 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:37.689170 19074 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:37.718532 12525 raft_consensus.cc:2410] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:12:37.692109 19093 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:37.692962 19178 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:37.672940 19180 raft_consensus.cc:3055] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:37.693349 19026 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:37.657228 18972 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:12:37.666515 19145 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:37.701727 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:37.670678 18862 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:37.707396 12571 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:37.702523 19169 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:12:37.714779 19096 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:37.703642 19118 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:37.693696 19158 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:37.730546 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:37.797242 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 217903us
W20260709 12:12:37.799710 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:37.720388 19180 raft_consensus.cc:740] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:37.695691 19045 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:37.699265 19002 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:37.693977 18753 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:37.694969 19053 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:37.815907 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 108643us
I20260709 12:12:37.816205 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:37.816571 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:37.724557 18914 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:37.724639 18769 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:37.682576 19114 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:37.842613 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:37.706503 19115 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.099s
I20260709 12:12:37.798108 19118 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:37.847647 12571 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:37.803972 19070 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:37.816875 19158 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:37.801399 19093 consensus_queue.cc:1243] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260709 12:12:37.865450 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 217309us
I20260709 12:12:37.909832 12494 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:37.818439 19045 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:37.847471 19047 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:37.720297 19067 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.103s
W20260709 12:12:37.905978 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 90004us
W20260709 12:12:37.906630 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 59186.2us
I20260709 12:12:37.915776 12597 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:37.851622 18769 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:37.924059 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:37.904503 19118 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:37.930542 12526 raft_consensus.cc:2410] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:12:37.930411 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:37.941640 18592 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:37.834575 19146 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:37.916160 19045 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:37.736330 19135 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:37.965603 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:37.927052 19096 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:37.941984 19179 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:37.190706 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:37.865442 19142 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:37.976150 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:37.914008 19074 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
W20260709 12:12:37.973615 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 63895.9us
W20260709 12:12:37.980639 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50308.3us
I20260709 12:12:37.943051 19164 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:37.963867 19157 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:38.006012 12601 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 651) took 1568 ms (client timeout 1944 ms). Trace:
W20260709 12:12:38.027581 12601 rpcz_store.cc:269] 0709 12:12:36.437741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:36.468949 (+ 31208us) service_pool.cc:224] Handling call
0709 12:12:38.005716 (+1536767us) spinlock_profiling.cc:243] Waited 181 ms on lock 0x7b54002e59d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:38.005879 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":416907008}
W20260709 12:12:37.995386 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:37.988971 19067 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:37.904618 19180 consensus_queue.cc:260] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:37.998749 18617 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:38.000145 19185 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.125s	user 0.000s	sys 0.001s
W20260709 12:12:38.000909 19184 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 12:12:38.001808 19125 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.002s	user 0.003s	sys 0.005s
I20260709 12:12:38.004987 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3049) took 1651 ms. Trace:
I20260709 12:12:38.006278 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3064) took 1335 ms. Trace:
I20260709 12:12:38.055403 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:38.055513 12598 rpcz_store.cc:276] 0709 12:12:36.670372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:37.368998 (+698626us) service_pool.cc:224] Handling call
0709 12:12:37.369398 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:37.387297 (+ 17899us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:37.387354 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:37.419776 (+ 32422us) log.cc:844] Serialized 26 byte log entry
0709 12:12:37.795923 (+376147us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:37.795983 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:37.796078 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:38.006022 (+209944us) raft_consensus.cc:1685] finished
0709 12:12:38.006045 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:38.006212 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65279,"raft.queue_time_us":15363,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":41174,"thread_start_us":375460,"threads_started":1,"wal-append.queue_time_us":485376}
I20260709 12:12:38.008140 12600 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: [17.14-18.15]
I20260709 12:12:37.986330 19119 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:38.014775 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3060) took 1411 ms. Trace:
I20260709 12:12:38.063983 12603 rpcz_store.cc:276] 0709 12:12:36.603752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:37.276534 (+672782us) service_pool.cc:224] Handling call
0709 12:12:37.276946 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:37.498742 (+221796us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:38.014445 (+515703us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54001946d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:38.014693 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":235844864}
I20260709 12:12:38.018318 19129 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:37.975276 19093 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 12:12:38.023020 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:38.024554 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:38.036808 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 103358us
I20260709 12:12:37.983920 18941 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:38.071326 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:38.048326 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:38.053108 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:38.055436 12606 rpcz_store.cc:276] 0709 12:12:36.353666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:36.353976 (+   310us) service_pool.cc:224] Handling call
0709 12:12:36.371194 (+ 17218us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:37.402938 (+1031744us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:38.004565 (+601627us) spinlock_profiling.cc:243] Waited 1.03 s on lock 0x7b5400231c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:38.004874 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2371450112}
I20260709 12:12:38.073992 12527 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:38.075331 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:38.073237 12529 raft_consensus.cc:2410] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:12:38.045727 19164 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.051282 19031 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.061619 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:38.081461 12494 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:38.081969 12598 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:38.055403 19142 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:38.075620 12527 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:12:38.086769 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:38.087586 12601 raft_consensus.cc:2376] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19. Re-sending same reply.
I20260709 12:12:38.045161 19135 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:38.042501 19146 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:38.046269 18914 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:38.067159 19180 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:38.091194 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:38.104643 12606 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:38.063460 19066 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.106709 13302 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:38.077054 12603 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:38.107916 19085 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:38.108616 19085 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:38.109633 19085 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:38.072889 18941 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:38.120610 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 16 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:38.121903 12643 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:38.050889 19067 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:38.123720 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:38.080353 19119 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:38.125326 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 458917us
I20260709 12:12:38.056442 19157 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:38.121282 19066 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:38.136225 19047 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:38.135512 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:38.141242 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:38.055100 18592 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:38.119709 19194 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:38.140761 19047 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:38.123044 18941 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:38.157675 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:38.158492 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:38.092407 19164 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:38.092602 19031 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:38.163883 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:38.137388 19158 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:38.100116 19195 consensus_queue.cc:1059] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.064s
I20260709 12:12:38.161320 19047 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:38.184913 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 63123.7us
I20260709 12:12:38.154918 19194 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:38.133986 18914 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:38.138168 19119 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:38.193423 12606 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15.
I20260709 12:12:38.186214 12598 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
I20260709 12:12:38.136679 19200 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.056s
I20260709 12:12:38.198148 13302 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:38.185778 18772 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:38.214478 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:38.193877 19050 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.362s	user 0.018s	sys 0.013s
I20260709 12:12:38.231937 18723 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.003s
I20260709 12:12:38.236398 18769 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:38.184187 19118 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:38.241051 19118 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:38.223574 19189 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.188s	user 0.001s	sys 0.000s
I20260709 12:12:38.246263 18914 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:38.247376 12602 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:12:38.251586 19157 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:38.247653 19164 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:38.191489 19194 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:38.178586 18617 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
I20260709 12:12:38.260927 12606 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: []
W20260709 12:12:38.262681 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 76773.2us
W20260709 12:12:38.230712 19050 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.399s	user 0.018s	sys 0.013s
I20260709 12:12:38.269387 12643 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:38.259275 19205 raft_consensus.cc:3055] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:38.277201 12606 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:38.263212 19026 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader election won for term 15
W20260709 12:12:38.274931 19197 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.207s	user 0.001s	sys 0.001s
W20260709 12:12:38.285339 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 125935us
W20260709 12:12:38.288313 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 73219.1us
W20260709 12:12:38.289623 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 106080us
W20260709 12:12:38.278612 19198 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.089s	user 0.000s	sys 0.001s
I20260709 12:12:38.276047 19164 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:38.278416 19182 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:38.275864 19205 raft_consensus.cc:740] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:38.276239 18772 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:38.302575 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:38.309226 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:38.329599 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 128472us
I20260709 12:12:38.294346 19026 raft_consensus.cc:697] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 15 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:12:38.319419 19190 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.217s	user 0.000s	sys 0.002s
I20260709 12:12:38.335031 19050 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:12:38.348608 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 53147.5us
I20260709 12:12:38.352183 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:38.337105 19145 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.547s	user 0.014s	sys 0.000s
I20260709 12:12:38.288187 12572 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:38.341285 19199 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
W20260709 12:12:38.366569 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 78449.8us
I20260709 12:12:38.367923 12572 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:38.343349 19066 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:38.387485 19202 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
W20260709 12:12:38.365796 19193 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.219s	user 0.000s	sys 0.001s
W20260709 12:12:38.391248 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:38.391700 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:38.371236 19164 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:38.400830 12572 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:38.400046 12643 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:38.408529 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:12:38.393203 19191 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.213s	user 0.001s	sys 0.001s
I20260709 12:12:38.392093 19031 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:38.393950 19212 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:38.364544 19145 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:38.398452 19207 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:12:38.453533 19198 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:12:38.409423 19209 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:12:38.437417 19050 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:38.411052 18914 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:38.389712 19205 consensus_queue.cc:260] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:38.411351 18772 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 12.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:38.472493 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 72540us
I20260709 12:12:38.520895 13333 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
W20260709 12:12:38.519644 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:38.509225 19053 consensus_meta.cc:287] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.651s	user 0.012s	sys 0.000s
W20260709 12:12:38.515566 19216 consensus_queue.cc:1175] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "193a40da6e364ead8ab561e14c3cc8d1" responder_term: 14 status { last_received { term: 14 index: 8 } last_committed_idx: 7 last_received_current_leader { term: 14 index: 8 } }
I20260709 12:12:38.523566 19119 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:38.498369 19208 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.143s	user 0.000s	sys 0.001s
W20260709 12:12:38.497637 19206 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.154s	user 0.001s	sys 0.000s
W20260709 12:12:38.498721 19204 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
W20260709 12:12:38.511600 19045 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.528s	user 0.008s	sys 0.005s
W20260709 12:12:38.489977 19193 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:12:38.538781 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:38.543967 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:38.502028 19168 consensus_meta.cc:287] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.857s	user 0.012s	sys 0.000s
I20260709 12:12:38.483907 19212 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:38.512036 19067 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:38.437611 19026 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 13.13, Last appended by leader: 13, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:38.527714 19182 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:38.555608 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:38.557308 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 342895us
I20260709 12:12:38.559233 12530 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:38.568755 12528 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 17: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:12:38.568928 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:38.456480 19066 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.104s
I20260709 12:12:38.546139 19053 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:38.562883 19212 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:38.577919 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:38.599179 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:38.561091 19045 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:38.613350 19143 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:38.620692 19205 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:38.623822 19150 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:38.624279 19150 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:38.625236 19150 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:38.626746 19209 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.627084 19145 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:38.627519 19209 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:38.630930 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:38.631745 12601 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:38.634442 19182 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:38.636255 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:38.636952 13334 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:38.638566 12569 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:38.639431 19208 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.070s	user 0.000s	sys 0.002s
I20260709 12:12:38.639557 18914 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:38.640275 18914 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:38.640679 18914 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:38.640806 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3071) took 1952 ms. Trace:
I20260709 12:12:38.641281 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3077) took 1393 ms. Trace:
I20260709 12:12:38.641741 12597 rpcz_store.cc:276] 0709 12:12:37.247602 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:37.925102 (+677500us) service_pool.cc:224] Handling call
0709 12:12:37.929706 (+  4604us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:37.946698 (+ 16992us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:12:37.983814 (+ 37116us) log.cc:844] Serialized 14 byte log entry
0709 12:12:38.322056 (+338242us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:38.335984 (+ 13928us) log.cc:844] Serialized 26 byte log entry
0709 12:12:38.336164 (+   180us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:12:38.336377 (+   213us) log.cc:844] Serialized 14 byte log entry
0709 12:12:38.336609 (+   232us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:38.336682 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:38.639171 (+302489us) spinlock_profiling.cc:243] Waited 2.36 ms on lock 0x7b240033f7b8. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14857b99c4 00007f14857b7acb 00007f1485972dc9 00007f1485972a55 00007f148597209d 00007f14885a410e 00007f14885bb88c 00007f14885babb6 00007f14885ba7d8 00007f14885c4221
0709 12:12:38.641041 (+  1870us) raft_consensus.cc:1685] finished
0709 12:12:38.641065 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:38.641212 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36113,"prepare.queue_time_us":37121,"raft.queue_time_us":98898,"raft.run_cpu_time_us":461,"raft.run_wall_time_us":15468,"spinlock_wait_cycles":14698112,"thread_start_us":261360,"threads_started":2,"wal-append.queue_time_us":250811}
I20260709 12:12:38.645112 19053 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:38.648329 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:38.649062 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:38.649890 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:38.650332 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2705) took 1115 ms. Trace:
I20260709 12:12:38.650928 12526 rpcz_store.cc:276] 0709 12:12:37.518791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:37.522014 (+  3223us) service_pool.cc:224] Handling call
0709 12:12:38.633846 (+1111832us) spinlock_profiling.cc:243] Waited 333 ms on lock 0x7b5400052b54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:38.634024 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":764824832}
I20260709 12:12:38.656437 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:38.657533 19045 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:38.659333 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 306948us
I20260709 12:12:38.660575 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:38.661633 12601 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:12:38.663044 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:38.652880 19150 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:38.641299 12599 rpcz_store.cc:276] 0709 12:12:36.688017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:37.586112 (+898095us) service_pool.cc:224] Handling call
0709 12:12:37.586620 (+   508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:37.587239 (+   619us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:12:37.587613 (+   374us) log.cc:844] Serialized 14 byte log entry
0709 12:12:37.913906 (+326293us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:37.922878 (+  8972us) log.cc:844] Serialized 26 byte log entry
0709 12:12:37.923041 (+   163us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:37.923076 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:37.923150 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:38.640558 (+717408us) raft_consensus.cc:1685] finished
0709 12:12:38.640581 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:38.640740 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45888,"prepare.queue_time_us":305,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":18536,"raft.run_cpu_time_us":323,"raft.run_wall_time_us":14058,"thread_start_us":325673,"threads_started":1,"wal-append.queue_time_us":492826}
W20260709 12:12:38.655203 19180 consensus_meta.cc:287] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.532s	user 0.009s	sys 0.005s
I20260709 12:12:38.670414 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:38.666440 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:38.674681 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:38.653995 19143 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.679822 12523 raft_consensus.cc:1217] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Deduplicated request from leader. Original: 12.7->[13.8-15.9]   Dedup: 13.8->[15.9-15.9]
I20260709 12:12:38.664472 19218 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.680696 12523 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: [13.8-15.9]
I20260709 12:12:38.681092 19218 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:38.683804 19218 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:38.684988 18914 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:38.688977 19142 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.586s	user 0.007s	sys 0.003s
I20260709 12:12:38.660205 19209 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:38.692716 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:38.694054 19135 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:38.695113 19180 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:38.695834 19135 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:38.697486 19143 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:38.700184 19143 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:38.700517 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:38.702773 19066 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:38.703711 19066 consensus_queue.cc:1243] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260709 12:12:38.706712 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:38.707508 12599 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:12:38.707723 19047 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.513s	user 0.011s	sys 0.000s
I20260709 12:12:38.708181 19047 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:38.716602 12572 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:12:38.716900 19146 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.617s	user 0.011s	sys 0.001s
I20260709 12:12:38.718632 19142 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
W20260709 12:12:38.720448 18941 consensus_meta.cc:287] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.546s	user 0.010s	sys 0.000s
I20260709 12:12:38.721119 18941 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:38.720373 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3078) took 1472 ms. Trace:
I20260709 12:12:38.723809 12602 rpcz_store.cc:276] 0709 12:12:37.247997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:38.012584 (+764587us) service_pool.cc:224] Handling call
0709 12:12:38.013047 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:38.248260 (+235213us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:38.720076 (+471816us) spinlock_profiling.cc:243] Waited 234 ms on lock 0x7b5400375d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:38.720302 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":538005376}
W20260709 12:12:38.737511 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:38.738989 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:38.748183 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:38.749049 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:38.749864 13339 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:38.722460 19194 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:38.754909 19151 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:38.755764 19114 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:38.756728 19194 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:38.758553 12527 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17. Ops: []
W20260709 12:12:38.747884 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 53289.4us
I20260709 12:12:38.775858 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:38.777506 12643 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:38.732194 18914 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:38.782241 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:38.801030 13339 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:12:38.786479 19045 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:38.796754 19180 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
W20260709 12:12:38.811371 19188 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.840s	user 0.003s	sys 0.000s
W20260709 12:12:38.811491 19220 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.424s	user 0.000s	sys 0.002s
I20260709 12:12:38.797605 19047 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:38.812985 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3179) took 2083 ms. Trace:
I20260709 12:12:38.813367 12676 rpcz_store.cc:276] 0709 12:12:36.729659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:36.916134 (+186475us) service_pool.cc:224] Handling call
0709 12:12:36.916441 (+   307us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:37.376591 (+460150us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:37.376623 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:37.377162 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 12:12:37.776688 (+399526us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:37.776732 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:37.798790 (+ 22058us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:38.722097 (+923307us) spinlock_profiling.cc:243] Waited 3.99 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:38.812671 (+ 90574us) raft_consensus.cc:1685] finished
0709 12:12:38.812693 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:38.812924 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":292240,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":110,"spinlock_wait_cycles":9170688,"thread_start_us":811084,"threads_started":2,"wal-append.queue_time_us":530212}
I20260709 12:12:38.817214 19141 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:38.819674 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3067) took 2148 ms. Trace:
I20260709 12:12:38.825369 18753 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:38.826915 19233 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
I20260709 12:12:38.817902 19141 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:38.820547 12604 rpcz_store.cc:276] 0709 12:12:36.671340 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:37.393448 (+722108us) service_pool.cc:224] Handling call
0709 12:12:37.590395 (+196947us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:37.661820 (+ 71425us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:37.661856 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:37.680199 (+ 18343us) log.cc:844] Serialized 26 byte log entry
0709 12:12:37.847121 (+166922us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:37.847174 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:37.847243 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:37.901452 (+ 54209us) spinlock_profiling.cc:243] Waited 8.68 ms on lock 0x7b480024c258. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:12:38.486376 (+584924us) raft_consensus.cc:1685] finished
0709 12:12:38.486404 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:38.819219 (+332815us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b54002f1a50. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:38.819557 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3960,"raft.run_cpu_time_us":3374,"raft.run_wall_time_us":271722,"spinlock_wait_cycles":471573504,"thread_start_us":152612,"threads_started":1,"wal-append.queue_time_us":371810}
I20260709 12:12:38.803342 12569 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:38.834137 19169 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:38.834904 19169 raft_consensus.cc:697] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:38.798257 19118 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:38.843847 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:38.800807 19194 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:38.861716 19108 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.748s	user 0.042s	sys 0.060s
I20260709 12:12:38.865861 19194 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:38.868824 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:38.876718 13333 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:38.874584 19130 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:38.879504 19130 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:12:38.873955 19108 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.042s	sys 0.060s
I20260709 12:12:38.895310 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 659) took 1392 ms. Trace:
I20260709 12:12:38.898175 12603 rpcz_store.cc:276] 0709 12:12:37.502879 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:38.108079 (+605200us) service_pool.cc:224] Handling call
0709 12:12:38.108458 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:38.347765 (+239307us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:38.347804 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:38.355734 (+  7930us) log.cc:844] Serialized 26 byte log entry
0709 12:12:38.763056 (+407322us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:12:38.796913 (+ 33857us) log.cc:844] Serialized 14 byte log entry
0709 12:12:38.797171 (+   258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:38.797256 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:38.813638 (+ 16382us) spinlock_profiling.cc:243] Waited 9.99 ms on lock 0x7b18004478e0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:12:38.885176 (+ 71538us) raft_consensus.cc:1685] finished
0709 12:12:38.895110 (+  9934us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:38.895253 (+   143us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":34541,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":53,"raft.queue_time_us":113208,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":10754,"spinlock_wait_cycles":30957184,"thread_start_us":610384,"threads_started":2,"wal-append.queue_time_us":431986}
W20260709 12:12:38.903785 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:38.905155 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:38.905706 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:38.905958 12603 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:38.906337 12602 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:38.909207 12601 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:12:38.911526 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:38.914309 19093 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:38.914932 19093 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:38.915323 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2787) took 1476 ms. Trace:
I20260709 12:12:38.915551 12531 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:38.915688 12525 rpcz_store.cc:276] 0709 12:12:37.438451 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:37.438722 (+   271us) service_pool.cc:224] Handling call
0709 12:12:38.915099 (+1476377us) spinlock_profiling.cc:243] Waited 220 ms on lock 0x7b54003c0d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:38.915267 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":505229696}
I20260709 12:12:38.917039 19093 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:38.836330 19169 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:38.918334 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:38.918917 19211 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.572s	user 0.003s	sys 0.000s
I20260709 12:12:38.920212 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2797) took 1352 ms. Trace:
I20260709 12:12:38.917618 19118 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:38.921252 19118 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:38.923370 19118 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:38.831037 19141 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:38.924921 19141 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:38.925585 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:38.927060 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:38.928516 18914 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:12:38.930820 19192 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.778s	user 0.002s	sys 0.001s
I20260709 12:12:38.932565 13338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 822) took 1504 ms. Trace:
I20260709 12:12:38.933472 12605 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:38.934661 12605 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:38.932925 13338 rpcz_store.cc:276] 0709 12:12:37.428476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:37.431616 (+  3140us) service_pool.cc:224] Handling call
0709 12:12:37.431933 (+   317us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:37.542111 (+110178us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:37.542172 (+    61us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:37.542671 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 12:12:37.963986 (+421315us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:37.964051 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:38.754595 (+790544us) spinlock_profiling.cc:243] Waited 110 ms on lock 0x7b540046a5d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:38.754649 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:38.930224 (+175575us) spinlock_profiling.cc:243] Waited 3.39 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:38.932322 (+  2098us) raft_consensus.cc:1685] finished
0709 12:12:38.932343 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:38.932480 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3026,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":140,"spinlock_wait_cycles":260127360,"thread_start_us":420818,"threads_started":1,"wal-append.queue_time_us":546899}
I20260709 12:12:38.938202 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:38.920545 12524 rpcz_store.cc:276] 0709 12:12:37.567653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:37.628956 (+ 61303us) service_pool.cc:224] Handling call
0709 12:12:37.629472 (+   516us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:37.946100 (+316628us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:37.946168 (+    68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:37.965519 (+ 19351us) log.cc:844] Serialized 26 byte log entry
0709 12:12:38.289181 (+323662us) raft_consensus.cc:1645] Marking committed up to 6
0709 12:12:38.289225 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:38.289313 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:38.918370 (+629057us) spinlock_profiling.cc:243] Waited 3.19 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:38.919910 (+  1540us) raft_consensus.cc:1685] finished
0709 12:12:38.919931 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:38.920147 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4313,"raft.queue_time_us":6811,"raft.run_cpu_time_us":85,"raft.run_wall_time_us":85,"spinlock_wait_cycles":7331584,"thread_start_us":459363,"threads_started":2,"wal-append.queue_time_us":182556}
I20260709 12:12:38.940382 12603 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: [17.8-17.8]
I20260709 12:12:38.941282 12606 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:38.882079 18941 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:38.944335 12571 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:38.948148 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:38.944679 12671 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:38.950766 12671 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: []
W20260709 12:12:38.954703 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 78667.5us
I20260709 12:12:38.946944 19096 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:38.956725 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:38.957290 12601 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:38.960394 12676 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 10. (term mismatch)
I20260709 12:12:38.960800 12676 pending_rounds.cc:85] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 9
I20260709 12:12:38.961012 12676 pending_rounds.cc:107] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 16.10
I20260709 12:12:38.961233 12676 raft_consensus.cc:2889] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:38.962471 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:38.964407 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:38.964608 19194 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:38.964707 12676 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:38.965037 12677 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:38.946286 12602 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:38.966642 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:38.967257 12676 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:38.970896 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:38.973376 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:38.987807 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:38.996656 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:38.997946 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:38.998777 12679 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:12:38.998669 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:38.999811 12603 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:39.000172 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.003013 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.004385 12678 raft_consensus.cc:2410] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 20 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:12:39.010403 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:39.011420 12604 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
W20260709 12:12:39.013480 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 69282.6us
I20260709 12:12:39.014566 12571 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:39.016801 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.021188 12671 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:38.939776 19153 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:39.049698 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.050880 12671 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 18: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:38.898572 19212 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:39.052032 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:39.042688 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:39.080035 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:38.898079 19032 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.099s
I20260709 12:12:39.045883 12601 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:39.038770 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.008169 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:38.980362 19114 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:39.084020 12679 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:39.084364 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:39.064210 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52531.7us
I20260709 12:12:39.077411 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:39.040699 12571 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:39.085393 12604 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:39.088941 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.089395 12603 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
W20260709 12:12:39.091959 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 51427.4us
I20260709 12:12:39.092852 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:39.094349 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:39.095156 12676 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:38.880522 19130 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:39.024149 19096 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:39.008435 18993 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:39.108225 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:39.054402 19240 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
W20260709 12:12:39.151996 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 66461.6us
I20260709 12:12:39.174211 12530 raft_consensus.cc:2410] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 19 because replica has last-logged OpId of term: 17 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 12:12:39.152753 12676 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:39.164937 18862 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:39.122278 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:38.993513 19200 consensus_queue.cc:1059] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 15.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.100s
I20260709 12:12:39.045624 19093 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:12:39.184473 19237 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 12:12:39.185778 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3080) took 1646 ms. Trace:
I20260709 12:12:39.068318 19153 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:39.163643 12678 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:39.190616 12598 rpcz_store.cc:276] 0709 12:12:37.539799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:38.198944 (+659145us) service_pool.cc:224] Handling call
0709 12:12:38.199782 (+   838us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:38.230415 (+ 30633us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:12:38.529558 (+299143us) log.cc:844] Serialized 14 byte log entry
0709 12:12:38.723995 (+194437us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:38.762994 (+ 38999us) log.cc:844] Serialized 52 byte log entry
0709 12:12:38.772669 (+  9675us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:38.797732 (+ 25063us) log.cc:844] Serialized 14 byte log entry
0709 12:12:39.184962 (+387230us) spinlock_profiling.cc:243] Waited 1.03 ms on lock 0x7b1800292780. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f14885d6d70 00007f14885ca293 00007f14885bfcea 00007f14885c023a 00007f1488a2aa2a 00007f1488a3a0bf 00007f1488a3a038
0709 12:12:39.185283 (+   321us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:39.185380 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:39.185533 (+   153us) raft_consensus.cc:1685] finished
0709 12:12:39.185553 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:39.185708 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54759,"prepare.queue_time_us":364148,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":989,"raft.queue_time_us":23536,"raft.run_cpu_time_us":341,"raft.run_wall_time_us":25407,"spinlock_wait_cycles":3938432,"thread_start_us":434603,"threads_started":2,"wal-append.queue_time_us":244272}
I20260709 12:12:39.216447 12677 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:39.167176 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:39.171341 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 215727us
I20260709 12:12:39.034590 18723 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.117s
I20260709 12:12:39.022445 19218 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 14.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
I20260709 12:12:39.106588 19169 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:12:39.176955 12672 raft_consensus.cc:2410] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 19 because replica has last-logged OpId of term: 17 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 7.
I20260709 12:12:39.174897 12494 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:39.187275 19135 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:39.160804 19047 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:39.189350 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.231549 12676 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:39.150625 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.155186 19114 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:39.191754 19074 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:39.192559 18862 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:39.199057 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:39.163323 13335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 817) took 2552 ms. Trace:
W20260709 12:12:39.163674 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:39.217739 12604 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:39.219306 12643 leader_election.cc:400] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18.
I20260709 12:12:39.223444 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3068) took 2551 ms. Trace:
I20260709 12:12:39.152799 19070 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.082s
W20260709 12:12:39.161208 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 68443.3us
I20260709 12:12:39.259022 12600 rpcz_store.cc:276] 0709 12:12:36.671632 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:37.407000 (+735368us) service_pool.cc:224] Handling call
0709 12:12:38.007725 (+600725us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:38.058533 (+ 50808us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:39.223184 (+1164651us) spinlock_profiling.cc:243] Waited 600 ms on lock 0x7b54002e59d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:39.223364 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1380365184}
I20260709 12:12:39.256791 12679 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:39.261246 12605 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
W20260709 12:12:39.265368 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46279.3us
I20260709 12:12:39.266597 12571 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:39.266768 12643 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:39.267779 12679 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:12:39.268764 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3074) took 2091 ms. Trace:
I20260709 12:12:39.257208 13335 rpcz_store.cc:276] 0709 12:12:36.610476 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:36.616537 (+  6061us) service_pool.cc:224] Handling call
0709 12:12:36.654784 (+ 38247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:36.672419 (+ 17635us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:12:36.687576 (+ 15157us) log.cc:844] Serialized 14 byte log entry
0709 12:12:37.656218 (+968642us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b18003f7240. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a 00007f1488a2aa2a
0709 12:12:38.185462 (+529244us) log.cc:844] Serialized 14 byte log entry
0709 12:12:38.186008 (+   546us) log.cc:844] Serialized 14 byte log entry
0709 12:12:38.903112 (+717104us) spinlock_profiling.cc:243] Waited 1.36 ms on lock 0x7b18003f7240. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:12:39.162952 (+259840us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:39.163016 (+    64us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:39.163060 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:39.163138 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:39.163261 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":523801,"prepare.run_cpu_time_us":231,"prepare.run_wall_time_us":11370,"raft.queue_time_us":24818,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":16854,"spinlock_wait_cycles":35360896,"thread_start_us":788484,"threads_started":3,"wal-append.queue_time_us":1419056}
I20260709 12:12:39.181144 18993 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:39.273067 19053 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:39.274984 12672 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:12:39.281210 12605 rpcz_store.cc:276] 0709 12:12:37.177720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:37.653999 (+476279us) service_pool.cc:224] Handling call
0709 12:12:37.658379 (+  4380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:38.933083 (+1274704us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b54005125a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f148899e06f 00007f1488a16f60 00007f1488a187de 00007f1488a22b49 00007f1488a23416 00007f1488a1fd8f 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62
0709 12:12:39.264976 (+331893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:39.268684 (+  3708us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":496617216}
I20260709 12:12:39.281605 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:39.285959 19246 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:12:39.291815 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.299929 19108 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:39.301452 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:39.304025 12571 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:39.302086 19169 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
W20260709 12:12:39.285887 19150 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.614s	user 0.013s	sys 0.000s
I20260709 12:12:39.308452 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.266674 19099 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:39.304649 19213 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:12:39.262110 19114 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:39.314496 12571 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:39.316509 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.316609 12524 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
W20260709 12:12:39.318809 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 144015us
I20260709 12:12:39.324210 12678 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 14.
I20260709 12:12:39.325445 12571 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:39.317437 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.326802 12679 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 14.
I20260709 12:12:39.325127 19108 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:39.316308 19096 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:39.316177 19213 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:39.329314 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.331439 19146 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:12:39.332821 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:39.325434 19222 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:39.324296 18772 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:39.336584 19212 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:39.381795 13335 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:12:39.338533 19153 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:39.345327 18993 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:39.346400 13304 leader_election.cc:336] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.902s (SENT)
W20260709 12:12:39.347568 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:39.348150 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 177669us
W20260709 12:12:39.417243 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 84505.1us
W20260709 12:12:39.417531 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 71178.9us
W20260709 12:12:39.418449 13304 leader_election.cc:336] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.779s (SENT)
I20260709 12:12:39.418848 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:12:39.419510 13304 leader_election.cc:336] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.874s (SENT)
I20260709 12:12:39.419744 13339 raft_consensus.cc:2410] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 20 because replica has last-logged OpId of term: 19 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:12:39.420667 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:39.350351 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:39.318250 19099 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:39.305071 19142 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.042s
I20260709 12:12:39.421350 12524 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:39.328639 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:39.347987 19141 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:39.317878 18914 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.038s
I20260709 12:12:39.426994 19151 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:39.427526 12530 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:39.428018 19146 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:12:39.423750 18993 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:39.429209 18993 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:39.392369 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:39.393657 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 14 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:39.394078 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:39.370481 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.395514 19143 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:12:39.395632 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:39.432554 13336 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:39.370199 19053 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:39.406929 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.411656 18941 raft_consensus.cc:3055] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:39.412531 19195 raft_consensus.cc:3055] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:39.429561 19032 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:39.394495 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:39.431145 12678 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
I20260709 12:12:39.427006 12528 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 12:12:39.446508 12528 pending_rounds.cc:85] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 10
I20260709 12:12:39.435498 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:39.446810 12528 pending_rounds.cc:107] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:12:39.447096 13339 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:39.433338 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:39.376932 19222 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:39.389360 18772 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:39.378028 19213 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:12:39.443450 12496 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:39.448341 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.447149 12528 raft_consensus.cc:2889] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:39.449689 12678 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
I20260709 12:12:39.451156 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.451680 12496 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:39.446172 12646 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:39.422792 19153 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:39.455945 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.455256 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:39.459599 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 131066us
I20260709 12:12:39.471246 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:39.471539 12571 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:39.435487 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:39.466708 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.468664 13339 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
W20260709 12:12:39.469604 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:39.474346 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:39.475257 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:39.459916 18303 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.017s
I20260709 12:12:39.472805 19194 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:39.455806 19222 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:39.478785 18993 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:39.479488 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.442370 19195 raft_consensus.cc:740] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:39.480226 12678 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:39.481539 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:39.482208 12678 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:39.457728 19234 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:12:39.482686 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:39.499281 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:39.438408 18941 raft_consensus.cc:740] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:39.539974 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.540982 12672 raft_consensus.cc:2410] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:12:39.545625 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.546416 12679 raft_consensus.cc:2368] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 18. Current term is 19.
W20260709 12:12:39.551507 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52544us
I20260709 12:12:39.555406 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:39.573002 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:39.492208 19214 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:39.562947 12528 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:39.445844 19226 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.095s
W20260709 12:12:39.577093 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 81546.3us
I20260709 12:12:39.578858 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:39.578903 12601 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:12:39.585388 12676 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:39.538322 19234 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:39.593657 19234 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:39.533519 18914 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:39.599534 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.600471 12679 raft_consensus.cc:2368] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 18. Current term is 19.
I20260709 12:12:39.601421 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.602885 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:39.603489 12679 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:39.575878 19142 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:39.607053 12605 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:12:39.458626 19182 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.123s
I20260709 12:12:39.608302 19182 consensus_queue.cc:1243] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 19.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:12:39.576967 19119 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:39.610579 18941 consensus_queue.cc:260] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:39.576622 18772 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:39.612896 12643 connection.cc:256] RPC call timeout handler was delayed by -2.9349724820001484s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17170000000000002s for a response
I20260709 12:12:39.613623 18772 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:39.445454 19053 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:39.584113 19141 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:39.619231 19141 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:39.456341 19151 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.093s
I20260709 12:12:39.534732 19026 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:39.581697 12600 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:12:39.615126 18993 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:39.648123 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:39.474858 19153 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:39.671625 12643 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:39.663905 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:39.691419 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:39.585378 19045 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:39.585106 19180 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:39.622570 19251 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:39.623725 19070 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:39.672498 19074 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:39.751780 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:39.610088 19119 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:39.730010 12678 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15.
I20260709 12:12:39.739780 12676 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:39.589637 19164 consensus_queue.cc:1059] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.121s
I20260709 12:12:39.691792 18941 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:39.713089 19142 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:39.711161 19045 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:39.651136 19032 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.100s
I20260709 12:12:39.765252 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:39.767565 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 307433us
W20260709 12:12:39.768450 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 305577us
I20260709 12:12:39.774000 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.773916 12496 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:39.781072 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:39.767531 19256 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:39.790521 13304 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.120s (SENT) [suppressed 18 similar messages]
I20260709 12:12:39.787781 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:39.795099 12496 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:39.804618 12679 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:12:39.806815 12679 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 14 index: 11. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:12:39.814827 12679 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:39.810721 18303 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:39.771989 19141 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:39.773531 19045 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:39.761824 19070 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:39.795131 12494 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:39.772327 19250 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:39.762065 19074 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:39.894515 19074 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:39.813926 12496 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:39.822746 18303 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:39.896170 18303 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:39.898422 19263 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:12:39.834614 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 338112us
I20260709 12:12:39.819072 18972 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election won for term 17
W20260709 12:12:39.922727 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 108912us
I20260709 12:12:39.784132 19053 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:39.834774 18993 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:39.830309 19194 raft_consensus.cc:697] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:39.889565 19257 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
W20260709 12:12:39.889842 12427 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47524 (request call id 100) took 24735 ms (client timeout 15000 ms). Trace:
I20260709 12:12:39.921731 12494 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:39.891186 19255 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.240s	user 0.000s	sys 0.000s
I20260709 12:12:39.760623 19251 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:39.921360 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 249860us
I20260709 12:12:39.945003 12643 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:39.816847 19256 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:39.887761 18723 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:39.914183 19222 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:39.954922 12677 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:12:39.956035 19222 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:39.893987 19234 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:39.950886 19251 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:39.953049 19256 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:39.958323 19256 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:39.958803 19234 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:39.953612 12678 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:39.959466 18723 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:39.959916 18723 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:39.961443 12678 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 13 index: 13. Preceding OpId from leader: term: 15 index: 14. (index mismatch)
I20260709 12:12:39.963405 19153 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:39.964141 12678 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: [16.13-16.13]
I20260709 12:12:39.966384 19265 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:39.967131 19265 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:39.967612 19265 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:39.975369 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:39.976357 12528 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:39.976841 13304 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:39.979336 19182 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:39.980026 19182 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:39.981112 19218 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:39.981642 19218 raft_consensus.cc:697] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:39.949790 19253 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:12:39.939790 19074 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:39.956127 12677 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:39.987826 12677 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:39.984478 13304 leader_election.cc:400] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 18. Current term is 19.
I20260709 12:12:39.989796 13304 leader_election.cc:403] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:39.988955 12677 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:39.991457 12677 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:39.992429 19026 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 18. Current term is 19.
I20260709 12:12:39.992780 19047 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:39.993359 19047 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:39.994465 19047 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:39.996714 13304 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:39.999022 19142 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:39.999634 19142 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.001700 18723 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.002662 19142 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:40.003501 12571 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:40.004158 18723 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:40.005144 12571 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:40.007675 19169 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:40.008138 19108 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:40.008427 19169 raft_consensus.cc:697] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:40.010071 19169 consensus_queue.cc:237] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.011298 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:40.013537 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:40.014765 18914 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:39.999171 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:40.018062 19026 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:40.022082 19182 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:40.038331 12676 raft_consensus.cc:2376] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15. Re-sending same reply.
I20260709 12:12:40.053762 12677 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:12:40.051283 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:40.069166 12676 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:40.070757 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:40.085745 19258 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:12:40.046873 18914 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.105046 18914 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:40.024367 19085 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:12:40.110316 19194 consensus_queue.cc:237] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.112229 19226 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:40.112787 19226 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:39.986552 19146 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 12:12:40.113770 19226 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.115716 19226 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:40.044672 19212 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:40.118439 19212 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.120622 19212 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:40.123008 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:40.123842 12522 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:40.124217 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:40.125186 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:40.125818 12527 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:12:40.126995 12525 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:40.128058 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:40.128700 12523 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:40.130009 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3189) took 1178 ms. Trace:
I20260709 12:12:40.130416 12675 rpcz_store.cc:276] 0709 12:12:38.951492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:38.951607 (+   115us) service_pool.cc:224] Handling call
0709 12:12:38.951911 (+   304us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:38.989875 (+ 37964us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:38.989938 (+    63us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:39.005890 (+ 15952us) log.cc:844] Serialized 26 byte log entry
0709 12:12:39.460624 (+454734us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:39.460671 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:39.460738 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:40.129744 (+669006us) raft_consensus.cc:1685] finished
0709 12:12:40.129768 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:40.129918 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37505,"raft.queue_time_us":57288,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":18382,"thread_start_us":366184,"threads_started":1,"wal-append.queue_time_us":734173}
I20260709 12:12:40.138234 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 9 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:40.139487 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:40.140671 12597 raft_consensus.cc:2410] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 20 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:12:40.141600 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:40.142848 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:40.143610 12598 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 12:12:40.143908 12603 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 15: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:40.144009 12598 pending_rounds.cc:85] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 8
I20260709 12:12:40.144595 12598 pending_rounds.cc:107] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:12:40.143951 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:40.145217 12523 raft_consensus.cc:3055] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:40.144999 12598 raft_consensus.cc:2889] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:40.145710 12523 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:40.067255 19180 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:40.148285 12601 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:12:40.149724 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:40.152881 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:40.146756 12523 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:40.160337 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:40.188865 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:40.045851 19074 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:40.227788 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:40.203861 12523 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:40.204698 19262 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.121s	user 0.000s	sys 0.000s
I20260709 12:12:40.154908 19212 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:40.208971 12605 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 14 index: 7. (index mismatch)
I20260709 12:12:40.124037 19266 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.144s
I20260709 12:12:40.217928 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3195) took 1216 ms. Trace:
I20260709 12:12:40.242375 18303 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.043004 19218 consensus_queue.cc:237] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.083946 19142 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:40.221040 12599 raft_consensus.cc:2410] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 17 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:12:40.222644 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:40.225623 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 301999us
I20260709 12:12:40.158008 19119 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:40.190438 19180 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:40.228418 19213 consensus_meta.cc:287] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.705s	user 0.006s	sys 0.006s
W20260709 12:12:40.241647 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:40.244611 12524 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:12:40.291024 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:39.936410 12427 rpcz_store.cc:269] 0709 12:12:15.037366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:15.038307 (+   941us) service_pool.cc:224] Handling call
0709 12:12:39.768826 (+24730519us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:31.700804 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:31.701558 (+   754us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:31.701617 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:12:31.701639 (+    22us) tablet.cc:662] Decoding operations
0709 12:12:31.703107 (+  1468us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:31.703218 (+   111us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:31.703251 (+    33us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:12:31.708551 (+  5300us) tablet.cc:701] Row locks acquired
0709 12:12:31.708600 (+    49us) write_op.cc:260] PREPARE: finished
0709 12:12:31.708795 (+   195us) write_op.cc:270] Start()
0709 12:12:31.708967 (+   172us) write_op.cc:276] Timestamp: P: 1783599151708722 usec, L: 0
0709 12:12:31.708997 (+    30us) op_driver.cc:348] REPLICATION: starting
0709 12:12:31.709768 (+   771us) log.cc:844] Serialized 5196 byte log entry
0709 12:12:33.130535 (+1420767us) spinlock_profiling.cc:243] Waited 4.59 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:33.291111 (+160576us) op_driver.cc:464] REPLICATION: finished
0709 12:12:33.646279 (+355168us) write_op.cc:301] APPLY: starting
0709 12:12:33.651578 (+  5299us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:34.339517 (+687939us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:34.339581 (+    64us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:35.089306 (+749725us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:35.089717 (+   411us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0709 12:12:35.089781 (+    64us) write_op.cc:312] APPLY: finished
0709 12:12:35.114854 (+ 25073us) log.cc:844] Serialized 121 byte log entry
0709 12:12:35.317304 (+202450us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:35.353816 (+ 36512us) write_op.cc:454] Released schema lock
0709 12:12:35.391191 (+ 37375us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:35.445391 (+ 54200us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:35.445455 (+    64us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4401205,"reported_tablets":21,"child_traces":[["op",{"apply.queue_time_us":297447,"apply.run_cpu_time_us":42916,"apply.run_wall_time_us":1846602,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58389,"num_ops":13,"prepare.queue_time_us":916496,"prepare.run_cpu_time_us":23469,"prepare.run_wall_time_us":314269,"raft.queue_time_us":8246,"raft.run_cpu_time_us":18656,"raft.run_wall_time_us":228884,"replication_time_us":1581785,"spinlock_wait_cycles":10559360,"thread_start_us":1058425,"threads_started":5,"wal-append.queue_time_us":760297,"wal-append.run_cpu_time_us":19721,"wal-append.run_wall_time_us":3144253}]]}
I20260709 12:12:40.248030 12643 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:40.249151 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:40.251421 12522 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
W20260709 12:12:40.231974 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 310358us
I20260709 12:12:40.281926 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:40.242782 12671 rpcz_store.cc:276] 0709 12:12:39.001198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:39.099182 (+ 97984us) service_pool.cc:224] Handling call
0709 12:12:39.099968 (+   786us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:39.438272 (+338304us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:39.438327 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:39.458269 (+ 19942us) log.cc:844] Serialized 26 byte log entry
0709 12:12:39.827144 (+368875us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:39.827186 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:39.827279 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:40.217613 (+390334us) raft_consensus.cc:1685] finished
0709 12:12:40.217659 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:40.217847 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51404,"raft.queue_time_us":55023,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":11495,"thread_start_us":555479,"threads_started":2,"wal-append.queue_time_us":615825}
I20260709 12:12:40.293251 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:40.283514 12496 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:40.304535 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:40.319595 12598 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
W20260709 12:12:40.319658 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 71780.8us
I20260709 12:12:40.319790 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:40.314530 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 660) took 2275 ms. Trace:
I20260709 12:12:40.320783 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:40.302642 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:40.227095 19182 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:12:40.273103 18772 consensus_meta.cc:287] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.635s	user 0.012s	sys 0.000s
I20260709 12:12:40.311612 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:40.320648 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:40.305521 13339 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:40.314692 12599 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 8. (term mismatch)
I20260709 12:12:40.311676 12600 raft_consensus.cc:2410] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 17 because replica has last-logged OpId of term: 16 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:12:40.325376 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:40.271778 19266 consensus_queue.cc:1243] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:12:40.340255 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:40.304886 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:40.300151 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:40.336794 12599 pending_rounds.cc:85] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 7
I20260709 12:12:40.336750 12671 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:40.344058 12599 pending_rounds.cc:107] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 14.8
I20260709 12:12:40.344434 12599 raft_consensus.cc:2889] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:40.346170 12671 pending_rounds.cc:85] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 14
I20260709 12:12:40.321488 12606 rpcz_store.cc:276] 0709 12:12:38.039656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:38.310707 (+271051us) service_pool.cc:224] Handling call
0709 12:12:38.342393 (+ 31686us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:38.941778 (+599385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:40.314156 (+1372378us) spinlock_profiling.cc:243] Waited 599 ms on lock 0x7b54001167d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:40.314416 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1376575872}
I20260709 12:12:40.347376 12671 pending_rounds.cc:107] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 19.15
I20260709 12:12:40.347829 12671 raft_consensus.cc:2889] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:40.282953 19142 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:40.335500 19103 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:40.318650 19070 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:40.360355 19103 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:40.360828 19103 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:40.306771 19205 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:40.238646 19212 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.364838 19070 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260709 12:12:40.365489 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:40.366230 12604 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:40.374028 12605 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:40.378444 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:40.378584 12604 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:40.379694 12599 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:40.382627 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:40.364674 19205 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:40.304864 18862 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:40.387449 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3181) took 1444 ms. Trace:
W20260709 12:12:40.350287 19164 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.028s	sys 0.024s
I20260709 12:12:40.394086 12523 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 12:12:40.336990 19119 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:40.464972 12523 pending_rounds.cc:85] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 13
I20260709 12:12:40.465394 12523 pending_rounds.cc:107] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 12:12:40.403635 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:40.465734 12523 raft_consensus.cc:2889] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:40.369870 19180 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:40.359084 18993 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:40.354239 18972 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:40.475760 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 650) took 1204 ms. Trace:
I20260709 12:12:40.306255 19216 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.089s
I20260709 12:12:40.358834 19142 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:40.330857 19213 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:40.441422 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:40.309443 19195 consensus_queue.cc:260] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:40.442274 12674 rpcz_store.cc:276] 0709 12:12:38.943455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:38.955741 (+ 12286us) service_pool.cc:224] Handling call
0709 12:12:38.956172 (+   431us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:39.002196 (+ 46024us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:39.002249 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:39.398903 (+396654us) spinlock_profiling.cc:243] Waited 3.61 ms on lock 0x7b2c00037868. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:12:39.400101 (+  1198us) log.cc:844] Serialized 52 byte log entry
0709 12:12:39.767779 (+367678us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:39.767825 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:39.767894 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:40.357352 (+589458us) raft_consensus.cc:1685] finished
0709 12:12:40.357388 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:40.387341 (+ 29953us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36044,"raft.queue_time_us":67357,"raft.run_cpu_time_us":190,"raft.run_wall_time_us":18240,"spinlock_wait_cycles":8312576,"thread_start_us":367132,"threads_started":1,"wal-append.queue_time_us":585945}
I20260709 12:12:40.473886 12572 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:40.481073 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:40.387105 19273 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:40.351784 18941 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.078s
I20260709 12:12:40.472157 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:40.420148 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:40.488093 12672 rpcz_store.cc:276] 0709 12:12:39.270945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:39.785152 (+514207us) service_pool.cc:224] Handling call
0709 12:12:39.823511 (+ 38359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:39.823801 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:12:40.150755 (+326954us) log.cc:844] Serialized 14 byte log entry
0709 12:12:40.475379 (+324624us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:40.475433 (+    54us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:12:40.475481 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:40.475565 (+    84us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:40.475695 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":547355,"prepare.run_cpu_time_us":160,"prepare.run_wall_time_us":159,"raft.queue_time_us":3964,"thread_start_us":580453,"threads_started":2}
I20260709 12:12:40.489508 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:40.338328 19182 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:40.482542 12674 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
I20260709 12:12:40.459100 12605 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:40.433980 19263 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:40.464815 19164 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.028s	sys 0.024s
W20260709 12:12:40.485785 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 269583us
I20260709 12:12:40.497025 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:40.464447 19276 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:40.508603 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:40.476063 19250 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:40.509379 12606 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:40.511073 12602 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 24: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:40.479465 19216 consensus_queue.cc:1243] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 15.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:12:40.513283 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:40.512560 19135 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:40.515722 19135 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:40.512023 12523 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:40.519758 12523 raft_consensus.cc:606] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:40.523739 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:40.526501 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:40.530016 13338 raft_consensus.cc:2410] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 13.
I20260709 12:12:40.524794 12604 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:40.535944 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:40.532968 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 239816us
I20260709 12:12:40.536789 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:40.536706 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:40.537539 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:40.510337 19266 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:40.536712 12606 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:40.467585 19269 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.092s
I20260709 12:12:40.517982 19135 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:12:40.510357 19143 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:40.506217 19273 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:40.477361 19119 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.512176 19251 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:40.513346 19212 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:40.512357 19053 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:40.556699 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:40.557382 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 240185us
I20260709 12:12:40.536793 19169 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:40.546991 19207 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:40.593897 19287 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:40.594098 19288 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:40.576435 19053 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.594702 19287 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:40.594844 19288 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.596669 19053 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:40.597710 19287 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:40.597548 19288 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:40.513720 19276 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.558463 19212 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:40.558137 19074 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:40.621497 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 157672us
I20260709 12:12:40.623046 12606 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:40.555627 19251 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.555884 19143 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:40.593343 19169 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:12:40.627359 19226 consensus_meta.cc:287] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.511s	user 0.013s	sys 0.002s
W20260709 12:12:40.638326 19277 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.127s	user 0.001s	sys 0.000s
I20260709 12:12:40.641113 19182 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:40.649398 12494 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:40.661571 12680 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:40.591198 19263 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.664340 19279 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:40.672413 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:40.636518 19273 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:40.671142 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2805) took 1084 ms. Trace:
I20260709 12:12:40.681398 12677 rpcz_store.cc:276] 0709 12:12:39.586996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:40.057234 (+470238us) service_pool.cc:224] Handling call
0709 12:12:40.057638 (+   404us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:40.058072 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:40.058149 (+    77us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:40.058744 (+   595us) log.cc:844] Serialized 26 byte log entry
0709 12:12:40.438935 (+380191us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:40.486665 (+ 47730us) log.cc:844] Serialized 14 byte log entry
0709 12:12:40.486924 (+   259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:40.487011 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:40.670913 (+183902us) raft_consensus.cc:1685] finished
0709 12:12:40.670935 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:40.671082 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":63021,"prepare.queue_time_us":60503,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":230,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":28092,"spinlock_wait_cycles":14308736,"thread_start_us":305181,"threads_started":1,"wal-append.queue_time_us":480561}
I20260709 12:12:40.669308 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2806) took 1039 ms. Trace:
I20260709 12:12:40.685322 12676 rpcz_store.cc:276] 0709 12:12:39.629574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:40.082149 (+452575us) service_pool.cc:224] Handling call
0709 12:12:40.082626 (+   477us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:40.122995 (+ 40369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:40.123028 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:40.132975 (+  9947us) log.cc:844] Serialized 52 byte log entry
0709 12:12:40.510357 (+377382us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:40.510407 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:40.510484 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:40.669060 (+158576us) raft_consensus.cc:1685] finished
0709 12:12:40.669084 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:40.669242 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":88703,"raft.queue_time_us":40538,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":18572,"thread_start_us":295779,"threads_started":1,"wal-append.queue_time_us":397968}
I20260709 12:12:40.689967 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:40.636315 19283 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:40.575992 18972 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.678557 12605 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:40.665692 19298 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:40.694507 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:40.676975 19259 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
W20260709 12:12:40.673072 19265 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.705s	user 0.011s	sys 0.000s
I20260709 12:12:40.610407 19207 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:40.698652 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 143642us
I20260709 12:12:40.701759 12671 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
W20260709 12:12:40.702415 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:40.704666 12604 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:12:40.663918 19213 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:40.718712 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:40.720204 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:40.720877 12605 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:40.721699 12494 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:40.660558 19278 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.042s
I20260709 12:12:40.726055 19218 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:40.729197 19218 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:40.732152 12531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:40.732966 12531 raft_consensus.cc:606] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:40.725731 19093 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:40.734601 19093 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:40.735023 19093 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:40.736486 19099 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:40.738477 19294 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.013s
I20260709 12:12:40.740659 19218 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:40.743146 19251 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:40.744947 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:40.745463 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:40.745574 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:40.719020 19265 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.667425 19182 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 12:12:40.710878 18941 consensus_queue.cc:1243] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 17.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:12:40.747653 13337 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
W20260709 12:12:40.723991 19045 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.838s	user 0.009s	sys 0.005s
I20260709 12:12:40.741566 19103 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.763396 13335 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:12:40.768774 12529 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 17. Current term is 18. Ops: []
W20260709 12:12:40.761936 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 288170us
I20260709 12:12:40.773974 12569 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:40.779024 12572 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:40.760191 19169 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:12:40.712284 19283 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:40.747253 19293 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:12:40.767176 19045 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.795717 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:40.765605 19194 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:12:40.812354 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:40.816115 12572 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:40.759979 19234 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:12:40.773350 19213 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:40.826184 12676 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 11.
I20260709 12:12:40.826520 19213 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.827219 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:40.827937 12676 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 18: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:40.772488 19212 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:40.828667 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:40.828991 19212 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.829800 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:40.831171 19212 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:40.828667 19213 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:40.832612 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:40.780571 19287 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:12:40.838693 19169 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:40.839247 19169 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:40.839640 19169 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:40.843267 19296 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 12:12:40.843181 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:40.844089 12603 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
I20260709 12:12:40.845062 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:40.847354 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49581.5us
I20260709 12:12:40.848832 12680 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: []
W20260709 12:12:40.850978 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 46869.2us
I20260709 12:12:40.852807 19311 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:40.853161 19074 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:12:40.854753 19074 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:40.855155 19074 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:40.856030 19074 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:40.857045 19074 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:40.859004 19108 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:40.858750 19045 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:40.859604 19108 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:40.863113 19213 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:12:40.863760 19214 consensus_meta.cc:287] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.275s	user 0.015s	sys 0.000s
I20260709 12:12:40.864316 19214 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:40.863817 19213 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:40.865411 18303 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:40.867578 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:40.869145 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:40.869395 19213 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:40.847517 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:40.871299 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:40.872025 19283 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:40.872339 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:40.873660 19216 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:40.874073 19216 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:40.874475 19216 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:40.876025 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:40.875437 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:40.870529 12675 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:12:40.880967 12676 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 12.
W20260709 12:12:40.875524 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:40.881196 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:40.900213 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 84325.9us
I20260709 12:12:40.905843 12646 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:40.927246 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:40.929061 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:40.929809 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 94694.6us
I20260709 12:12:40.936594 12601 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:40.960672 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:40.963721 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:40.966773 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:40.874936 19287 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:40.968515 12643 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:40.969230 12525 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:40.984658 12527 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:40.876137 19207 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:40.931489 19194 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:40.979343 19074 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
W20260709 12:12:40.968323 19196 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.243s	user 0.006s	sys 0.023s
W20260709 12:12:41.013332 19196 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.299s	user 0.006s	sys 0.024s
I20260709 12:12:40.959733 18993 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:12:41.023486 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 55080.3us
I20260709 12:12:41.025472 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:41.026482 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:40.968648 19266 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:41.022419 12672 pending_rounds.cc:85] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 12
I20260709 12:12:41.031343 12672 pending_rounds.cc:107] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:12:41.031683 12672 raft_consensus.cc:2889] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:40.984593 19213 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:41.029824 12522 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:40.980926 19287 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:12:41.028715 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 49081.2us
I20260709 12:12:40.980279 19276 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:41.036214 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:40.981393 19251 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.050s
I20260709 12:12:41.026582 18993 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:41.041440 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:41.033614 19164 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:41.022048 19194 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:41.038159 19273 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.018s
I20260709 12:12:41.043069 19256 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:41.031445 19093 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:41.050161 18993 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:41.079769 19308 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
I20260709 12:12:41.064606 19275 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:41.091495 12676 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:41.042976 19213 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:41.069370 19142 consensus_meta.cc:287] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.573s	user 0.006s	sys 0.011s
I20260709 12:12:41.097782 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:41.099296 19265 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:41.106822 19195 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:41.109365 19070 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:12:41.110746 19047 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:41.101755 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:41.094318 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:41.169157 12604 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:41.170192 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:41.171160 12604 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:41.172236 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:41.139019 12606 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:12:41.152484 19326 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:12:41.156677 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:41.177501 12601 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:41.157557 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2792) took 2166 ms. Trace:
I20260709 12:12:41.163724 12603 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:41.178299 12673 rpcz_store.cc:276] 0709 12:12:38.991557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:39.001216 (+  9659us) service_pool.cc:224] Handling call
0709 12:12:39.001605 (+   389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:39.280259 (+278654us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:39.280309 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:39.280371 (+    62us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:39.280421 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.157279 (+1876858us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b54000924d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:41.157333 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:41.157472 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":640228224}
I20260709 12:12:41.181226 19093 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:41.184103 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2804) took 1601 ms. Trace:
I20260709 12:12:41.184634 12678 rpcz_store.cc:276] 0709 12:12:39.582935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:39.965265 (+382330us) service_pool.cc:224] Handling call
0709 12:12:39.966029 (+   764us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:39.967377 (+  1348us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:12:40.286696 (+319319us) log.cc:844] Serialized 14 byte log entry
0709 12:12:40.722208 (+435512us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:40.722804 (+   596us) log.cc:844] Serialized 26 byte log entry
0709 12:12:40.722997 (+   193us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:40.724724 (+  1727us) log.cc:844] Serialized 14 byte log entry
0709 12:12:40.725040 (+   316us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:40.725202 (+   162us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:40.844298 (+119096us) spinlock_profiling.cc:243] Waited 23.1 ms on lock 0x7b4800060858. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:12:41.183874 (+339576us) raft_consensus.cc:1685] finished
0709 12:12:41.183901 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:41.184033 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23390,"prepare.queue_time_us":405621,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":94,"raft.queue_time_us":330,"raft.run_cpu_time_us":2475,"raft.run_wall_time_us":148389,"spinlock_wait_cycles":53016960,"thread_start_us":646171,"threads_started":2,"wal-append.queue_time_us":697656}
I20260709 12:12:41.188362 19143 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:41.189884 12601 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:12:41.190237 19194 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:41.191926 12601 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
W20260709 12:12:41.192888 19282 log.cc:927] Time spent T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.662s	user 0.000s	sys 0.001s
I20260709 12:12:41.194041 12601 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:12:41.197360 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2832) took 1056 ms. Trace:
I20260709 12:12:41.197778 12602 rpcz_store.cc:276] 0709 12:12:40.141172 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:40.141301 (+   129us) service_pool.cc:224] Handling call
0709 12:12:41.197145 (+1055844us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b54002e68d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:41.197289 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":847294208}
I20260709 12:12:41.077222 19266 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 15, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:41.199270 12606 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:12:41.200989 12601 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:41.174439 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:41.173840 12494 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:41.168502 12599 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:41.202277 12604 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 15: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:41.203121 12599 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:41.138048 19141 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:41.207162 12494 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:41.218145 12599 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20. Ops: []
W20260709 12:12:41.229197 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 187914us
I20260709 12:12:41.230320 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:41.165491 19142 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:41.248523 19142 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:41.228785 19299 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.057s	user 0.000s	sys 0.001s
I20260709 12:12:41.225786 19070 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:41.249735 12598 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 20: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:41.255523 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2820) took 1432 ms. Trace:
I20260709 12:12:41.266407 12528 rpcz_store.cc:276] 0709 12:12:39.822970 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:39.823159 (+   189us) service_pool.cc:224] Handling call
0709 12:12:39.886838 (+ 63679us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:39.976914 (+ 90076us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.255246 (+1278332us) spinlock_profiling.cc:243] Waited 89.2 ms on lock 0x7b54003caad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:41.255452 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":205114112}
I20260709 12:12:41.267899 12597 pending_rounds.cc:85] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 8
I20260709 12:12:41.268301 12597 pending_rounds.cc:107] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 15.9
I20260709 12:12:41.268649 12597 raft_consensus.cc:2889] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:41.224617 19141 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:41.232208 18723 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
W20260709 12:12:41.272596 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:41.276480 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:41.280121 19045 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:41.281179 19045 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:41.283528 19169 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:41.287750 19169 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:41.257992 19093 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:41.291782 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2744) took 1156 ms. Trace:
I20260709 12:12:41.292325 12529 rpcz_store.cc:276] 0709 12:12:40.135747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:40.135998 (+   251us) service_pool.cc:224] Handling call
0709 12:12:40.136371 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:40.769925 (+633554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.291473 (+521548us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b540042a9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:41.291705 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1453924736}
I20260709 12:12:41.306463 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:41.313931 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52260.8us
I20260709 12:12:41.311018 19070 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:41.318113 19070 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:41.299350 19300 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:41.311298 19251 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.011s
I20260709 12:12:41.322113 19251 consensus_queue.cc:1243] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:12:41.322290 19141 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:41.324397 19313 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:12:41.298286 19213 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:41.308856 19093 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:41.335791 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2821) took 1508 ms. Trace:
I20260709 12:12:41.336510 12496 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:41.336257 12524 rpcz_store.cc:276] 0709 12:12:39.827361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:39.827726 (+   365us) service_pool.cc:224] Handling call
0709 12:12:39.926516 (+ 98790us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:40.319029 (+392513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.335469 (+1016440us) spinlock_profiling.cc:243] Waited 318 ms on lock 0x7b54003f7354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:41.335719 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":730666368}
I20260709 12:12:41.306308 19047 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:41.309691 19169 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:41.317016 19221 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.506s	user 0.003s	sys 0.011s
W20260709 12:12:41.359530 19221 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.003s	sys 0.011s
I20260709 12:12:41.371524 12605 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24. Ops: []
W20260709 12:12:41.370918 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 64552.3us
I20260709 12:12:41.374684 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:41.307183 19216 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:41.311236 19287 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:41.403435 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3118) took 1074 ms. Trace:
I20260709 12:12:41.352175 19032 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:12:41.373255 18753 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.126s
I20260709 12:12:41.525899 12523 raft_consensus.cc:2368] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16.
I20260709 12:12:41.447041 19045 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:41.478523 18972 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:41.546402 12599 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
I20260709 12:12:41.546480 18972 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:41.547688 19287 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:41.548977 18972 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:41.531634 19150 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:41.550196 19150 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:41.559927 18993 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:41.569355 19207 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:41.570050 19207 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:41.572439 19207 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:41.528623 19263 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:41.581302 19263 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:41.589735 19332 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:41.596041 19216 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:41.599555 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:41.593189 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3108) took 1293 ms. Trace:
I20260709 12:12:41.600752 12598 rpcz_store.cc:276] 0709 12:12:40.300202 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:40.320529 (+ 20327us) service_pool.cc:224] Handling call
0709 12:12:41.592935 (+1272406us) spinlock_profiling.cc:243] Waited 927 ms on lock 0x7b54002f2454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:41.593108 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2132126080}
I20260709 12:12:41.602170 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:41.602787 12598 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:41.379407 19169 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:41.532647 19319 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:12:41.399502 19276 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.145s
I20260709 12:12:41.603762 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:41.590837 12646 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:41.643422 12598 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:41.608395 12494 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:41.578853 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3113) took 1251 ms. Trace:
W20260709 12:12:41.643836 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 53772.2us
I20260709 12:12:41.644407 12603 rpcz_store.cc:276] 0709 12:12:40.326972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:40.845896 (+518924us) service_pool.cc:224] Handling call
0709 12:12:41.163458 (+317562us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:41.179222 (+ 15764us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.578604 (+399382us) spinlock_profiling.cc:243] Waited 316 ms on lock 0x7b540022fbd0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:41.578775 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":726292864}
I20260709 12:12:41.645069 12598 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 15. Current term is 16. Ops: [15.8-15.8]
W20260709 12:12:41.478935 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 104393us
I20260709 12:12:41.646394 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:41.646727 12598 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:41.617312 13333 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:12:41.639966 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:41.648136 12530 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:41.499233 19194 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:41.642386 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:41.658730 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:41.641129 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:41.660051 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:41.650960 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:41.643337 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:41.660419 12524 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:41.669915 12526 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:12:41.679000 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:41.479998 19047 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:41.677927 12531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:41.682211 12531 raft_consensus.cc:606] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:41.683328 12530 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:41.591071 19108 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:41.617552 19251 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:41.617833 19102 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:41.662161 19273 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:41.705456 19273 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:41.590993 19332 consensus_queue.cc:1059] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.101s
I20260709 12:12:41.706759 19332 consensus_queue.cc:1243] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 16.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:12:41.706579 19273 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:41.676049 19252 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.032s
I20260709 12:12:41.708881 19252 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:41.709295 19252 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:41.710234 19252 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:41.710691 19250 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:41.711587 19252 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:41.711496 19250 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:41.678777 19115 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:41.714673 19115 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:41.720966 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3130) took 1060 ms. Trace:
I20260709 12:12:41.679420 19093 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:41.721480 12597 rpcz_store.cc:276] 0709 12:12:40.660758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.257949 (+597191us) service_pool.cc:224] Handling call
0709 12:12:41.262709 (+  4760us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:41.269368 (+  6659us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:41.269400 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:41.270278 (+   878us) log.cc:844] Serialized 52 byte log entry
0709 12:12:41.626518 (+356240us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:41.626569 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.626659 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:41.720659 (+ 94000us) raft_consensus.cc:1685] finished
0709 12:12:41.720683 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:41.720874 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":406,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":105,"thread_start_us":355784,"threads_started":1,"wal-append.queue_time_us":448451}
I20260709 12:12:41.722754 19205 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:41.723552 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:41.724185 12597 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:41.693114 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:41.668192 19135 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.031s
I20260709 12:12:41.702407 19070 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:12:41.702615 19164 consensus_meta.cc:287] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.643s	user 0.011s	sys 0.003s
I20260709 12:12:41.702467 19263 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:41.753808 19053 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:41.689543 19047 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:41.671562 19194 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:41.775422 19194 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:41.780251 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:41.781157 13330 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:41.782446 19070 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:41.784653 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:41.784894 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:41.785320 19151 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:41.785578 12674 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:41.785744 19151 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:41.786741 19151 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:41.787782 19151 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:41.791565 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:41.792213 12674 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:41.705075 19169 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:41.728680 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:41.724129 19205 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:41.732717 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:41.734447 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:41.745288 19250 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:41.751472 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:41.754326 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:41.755316 12572 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:12:41.811729 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 56506.7us
I20260709 12:12:41.812608 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:41.756995 12529 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:41.648626 19142 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.076s
I20260709 12:12:41.689109 19045 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:41.764501 19053 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:41.767764 19047 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:41.768877 13302 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:41.694016 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 85736us
I20260709 12:12:41.794052 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:41.796027 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:41.795029 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 679) took 1317 ms. Trace:
I20260709 12:12:41.797305 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3212) took 2487 ms. Trace:
I20260709 12:12:41.823292 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:41.797021 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 13 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:41.785105 19093 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:41.798915 19338 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.085s	user 0.002s	sys 0.000s
I20260709 12:12:41.800458 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:41.830513 12671 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:41.830696 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:41.831986 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:41.801342 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:41.759227 19164 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:12:41.834682 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 65980.4us
I20260709 12:12:41.836478 13302 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:41.804422 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:41.817024 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:41.755800 19332 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:41.821528 12673 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:41.843830 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:41.823658 12679 rpcz_store.cc:276] 0709 12:12:39.309635 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:39.821933 (+512298us) service_pool.cc:224] Handling call
0709 12:12:41.294344 (+1472411us) spinlock_profiling.cc:243] Waited 70.7 ms on lock 0x7b58001f2c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:12:41.294584 (+   240us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:41.300553 (+  5969us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:41.300586 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:41.301074 (+   488us) log.cc:844] Serialized 26 byte log entry
0709 12:12:41.610370 (+309296us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:41.610408 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.610485 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:41.760042 (+149557us) raft_consensus.cc:1685] finished
0709 12:12:41.760074 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:41.797242 (+ 37168us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":89642,"raft.queue_time_us":10922,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":5414,"spinlock_wait_cycles":162701696,"thread_start_us":308946,"threads_started":1,"wal-append.queue_time_us":399606}
I20260709 12:12:41.821528 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:41.822629 12676 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:12:41.822623 12675 rpcz_store.cc:276] 0709 12:12:40.477401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:40.883844 (+406443us) service_pool.cc:224] Handling call
0709 12:12:40.884217 (+   373us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:40.928407 (+ 44190us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:12:40.967040 (+ 38633us) log.cc:844] Serialized 14 byte log entry
0709 12:12:41.411121 (+444081us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:41.523001 (+111880us) log.cc:844] Serialized 52 byte log entry
0709 12:12:41.545091 (+ 22090us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:12:41.548196 (+  3105us) log.cc:844] Serialized 14 byte log entry
0709 12:12:41.550520 (+  2324us) log.cc:844] Serialized 14 byte log entry
0709 12:12:41.551557 (+  1037us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.551755 (+   198us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:41.719639 (+167884us) spinlock_profiling.cc:243] Waited 10.5 ms on lock 0x7b1800351940. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f
0709 12:12:41.720512 (+   873us) spinlock_profiling.cc:243] Waited 37.6 ms on lock 0x7b4800060858. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:12:41.794742 (+ 74230us) raft_consensus.cc:1685] finished
0709 12:12:41.794771 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:41.794958 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":54832,"prepare.queue_time_us":47700,"prepare.run_cpu_time_us":153,"prepare.run_wall_time_us":1675,"raft.queue_time_us":69099,"raft.run_cpu_time_us":3618,"raft.run_wall_time_us":841326,"spinlock_wait_cycles":111833216,"thread_start_us":373039,"threads_started":1,"wal-append.queue_time_us":768293}
I20260709 12:12:41.821749 12494 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:41.842778 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:41.835932 12569 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:41.799621 12571 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:41.844779 12529 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:41.896675 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 97214.8us
I20260709 12:12:41.898155 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:41.847612 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:41.818620 19194 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:41.857885 19245 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:41.858318 19214 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:12:41.858937 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:41.861274 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2816) took 1121 ms. Trace:
W20260709 12:12:41.880728 19342 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.153s	user 0.001s	sys 0.000s
I20260709 12:12:41.849954 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:41.832463 19341 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.127s	user 0.000s	sys 0.001s
I20260709 12:12:41.865722 12673 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:41.878834 12674 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:41.919517 12677 rpcz_store.cc:276] 0709 12:12:40.740290 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.098895 (+358605us) service_pool.cc:224] Handling call
0709 12:12:41.099222 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:41.099474 (+   252us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:41.099502 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:41.099978 (+   476us) log.cc:844] Serialized 26 byte log entry
0709 12:12:41.312523 (+212545us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:12:41.312936 (+   413us) log.cc:844] Serialized 14 byte log entry
0709 12:12:41.343340 (+ 30404us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.343471 (+   131us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:41.861015 (+517544us) raft_consensus.cc:1685] finished
0709 12:12:41.861037 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:41.861207 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10978,"prepare.queue_time_us":223,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":29988,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":26713,"thread_start_us":211972,"threads_started":1,"wal-append.queue_time_us":518684}
I20260709 12:12:41.877504 19047 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:41.841459 19169 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:41.878057 19142 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:41.851734 19311 raft_consensus.cc:3055] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:41.874048 12603 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7b825dff40dc4290a25d71534b598330"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:32832
I20260709 12:12:41.892071 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:41.886034 12530 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:12:41.843956 19332 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:41.915017 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:41.859936 19307 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:41.916514 12597 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
W20260709 12:12:41.922370 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 72320us
W20260709 12:12:41.886478 19221 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.033s	sys 0.049s
I20260709 12:12:41.924862 12571 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:41.879317 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:41.904085 19026 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:41.950659 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:41.966269 19026 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:41.968633 19026 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:41.970381 19195 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:12:41.971043 13330 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:12:41.971525 19195 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 12.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260709 12:12:41.972368 19195 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
W20260709 12:12:41.974676 19334 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.242s	user 0.000s	sys 0.001s
I20260709 12:12:41.893941 18993 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:41.975387 19130 raft_consensus.cc:3055] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Stepping down as leader of term 15
W20260709 12:12:41.976128 19343 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:12:41.842602 19093 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:41.978423 19093 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:41.961704 12531 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:41.975998 19130 raft_consensus.cc:740] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:41.979662 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2826) took 1104 ms. Trace:
I20260709 12:12:41.980275 12527 rpcz_store.cc:276] 0709 12:12:40.875669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:40.876881 (+  1212us) service_pool.cc:224] Handling call
0709 12:12:40.877161 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:41.014043 (+136882us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.979321 (+965278us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b54003cc3d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:41.979579 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":235486080}
I20260709 12:12:41.980894 19130 consensus_queue.cc:260] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:41.982627 19130 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:41.983144 19221 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.034s	sys 0.050s
I20260709 12:12:41.983583 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2832) took 1044 ms. Trace:
I20260709 12:12:41.974567 19216 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:41.984607 19216 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:41.985540 19216 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:41.983985 12523 rpcz_store.cc:276] 0709 12:12:40.938935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:40.939164 (+   229us) service_pool.cc:224] Handling call
0709 12:12:41.983364 (+1044200us) spinlock_profiling.cc:243] Waited 564 ms on lock 0x7b54003c0854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:41.983513 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1297144064}
I20260709 12:12:41.986773 19103 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:41.989801 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 138442us
I20260709 12:12:41.996304 12572 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:41.997264 19214 raft_consensus.cc:2764] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:12:41.986821 19216 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:42.002597 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:42.003490 19102 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:42.004154 19102 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:42.008603 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3117) took 1680 ms. Trace:
W20260709 12:12:42.010876 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 86143.4us
I20260709 12:12:42.011233 12674 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:42.009233 12602 rpcz_store.cc:276] 0709 12:12:40.328270 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.198743 (+870473us) service_pool.cc:224] Handling call
0709 12:12:41.199112 (+   369us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:12:41.199553 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:41.199582 (+    29us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:12:41.200326 (+   744us) log.cc:844] Serialized 78 byte log entry
0709 12:12:41.329443 (+129117us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:41.370585 (+ 41142us) log.cc:844] Serialized 14 byte log entry
0709 12:12:41.779385 (+408800us) spinlock_profiling.cc:243] Waited 1.21 ms on lock 0x7b1800485320. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf 00007f1488a3a038
0709 12:12:41.779604 (+   219us) log.cc:844] Serialized 14 byte log entry
0709 12:12:41.779848 (+   244us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.779925 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:42.008246 (+228321us) raft_consensus.cc:1685] finished
0709 12:12:42.008282 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:42.008516 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":324847,"prepare.queue_time_us":459774,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":3749,"raft.queue_time_us":14856,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":196,"spinlock_wait_cycles":52511872,"thread_start_us":128676,"threads_started":1,"wal-append.queue_time_us":328951}
I20260709 12:12:42.011773 12571 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:42.012626 19214 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:42.013028 19214 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:42.013446 19214 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:42.015235 12602 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 17 index: 13. (term mismatch)
I20260709 12:12:42.015806 12602 pending_rounds.cc:85] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 12
I20260709 12:12:42.016141 12602 pending_rounds.cc:107] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 16.13
I20260709 12:12:42.016471 12602 raft_consensus.cc:2889] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:42.018368 19093 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:42.018888 19093 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:42.019332 19093 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:42.020274 19115 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:42.024686 19287 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:41.471869 12606 rpcz_store.cc:276] 0709 12:12:40.328618 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.200162 (+871544us) service_pool.cc:224] Handling call
0709 12:12:41.200824 (+   662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:41.201040 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:12:41.221106 (+ 20066us) log.cc:844] Serialized 14 byte log entry
0709 12:12:41.403057 (+181951us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:41.403115 (+    58us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:41.403161 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.403239 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:41.403373 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2713,"prepare.queue_time_us":20954,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":456,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":42817,"spinlock_wait_cycles":44288,"thread_start_us":181092,"threads_started":1}
I20260709 12:12:42.029599 12572 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:42.032166 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:42.032519 13336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 706) took 1289 ms. Trace:
I20260709 12:12:41.914028 19194 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:42.036473 19194 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:42.037381 19213 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:12:42.039031 19213 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:42.039745 19213 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:41.951236 12676 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
W20260709 12:12:42.042573 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:42.051327 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3127) took 1403 ms. Trace:
I20260709 12:12:42.051790 12599 rpcz_store.cc:276] 0709 12:12:40.647535 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.219061 (+571526us) service_pool.cc:224] Handling call
0709 12:12:41.219325 (+   264us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:41.546941 (+327616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:42.051099 (+504158us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b5400375854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:42.051269 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":751710464}
I20260709 12:12:42.055245 19119 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:42.056738 19053 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:42.063117 19332 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:42.064509 19332 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:41.918370 19141 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Leader pre-election won for term 12
W20260709 12:12:41.982911 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:42.069528 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:42.072263 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 45158us
I20260709 12:12:42.073439 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:42.079694 13305 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:42.081362 12606 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:12:41.928709 18592 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:42.088125 12678 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:41.919297 19245 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:42.032953 13336 rpcz_store.cc:276] 0709 12:12:40.743141 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:40.743315 (+   174us) service_pool.cc:224] Handling call
0709 12:12:41.201931 (+458616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:41.202208 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:12:41.222625 (+ 20417us) log.cc:844] Serialized 14 byte log entry
0709 12:12:41.296818 (+ 74193us) spinlock_profiling.cc:243] Waited 2.06 ms on lock 0x7b18001b0ca0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a 00007f1488a2aa2a
0709 12:12:41.446750 (+149932us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:41.446815 (+    65us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:12:41.446863 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.446939 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:42.032326 (+585387us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b54004e84d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:42.032458 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":20263,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":59,"raft.queue_time_us":65240,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":26548,"spinlock_wait_cycles":1058517120,"thread_start_us":117389,"threads_started":1,"wal-append.queue_time_us":553675}
I20260709 12:12:41.928507 19207 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:12:42.122424 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:42.083561 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:42.123682 12676 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:41.955140 19142 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:42.103008 12529 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:41.892309 19283 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.073s
W20260709 12:12:42.126060 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52708.2us
I20260709 12:12:42.135143 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:41.953696 19169 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:42.129431 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:42.129987 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:42.139106 12678 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:12:42.120054 12523 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 18 index: 15. (term mismatch)
I20260709 12:12:42.139783 12523 pending_rounds.cc:85] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 14
I20260709 12:12:42.140357 12523 pending_rounds.cc:107] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 17.15
I20260709 12:12:42.140700 12523 raft_consensus.cc:2889] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:42.130541 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3124) took 1716 ms. Trace:
I20260709 12:12:41.961683 19311 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:42.097666 12531 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:42.085521 19141 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:42.119380 18592 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:42.128830 19245 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:42.142439 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:42.141647 12601 rpcz_store.cc:276] 0709 12:12:40.414323 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.210315 (+795992us) service_pool.cc:224] Handling call
0709 12:12:41.210686 (+   371us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:41.210941 (+   255us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:41.210972 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:41.211510 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 12:12:41.352365 (+140855us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:41.352428 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:41.352535 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:42.130159 (+777624us) raft_consensus.cc:1685] finished
0709 12:12:42.130195 (+    36us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:42.130448 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":139735,"threads_started":1,"wal-append.queue_time_us":437513}
I20260709 12:12:42.088660 18862 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:42.163602 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:42.135063 19142 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:42.134073 18723 raft_consensus.cc:2764] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:12:42.141306 19070 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:42.145977 19169 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:42.175302 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:42.153435 19213 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:12:42.169554 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 523374us
I20260709 12:12:42.181478 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:42.136204 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:42.156333 19141 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:41.959769 19047 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:42.160179 18592 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:42.186455 19304 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:42.194645 12674 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: [18.13-18.13]
I20260709 12:12:42.196986 19093 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:42.168942 18862 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:42.198271 12425 catalog_manager.cc:5697] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 8 to 10, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 10 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:12:42.183674 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2843) took 1823 ms. Trace:
I20260709 12:12:42.199415 18862 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:42.201954 19045 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:42.201012 12425 catalog_manager.cc:5697] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 7 to 8, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 8 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:42.199680 12604 rpcz_store.cc:276] 0709 12:12:40.385877 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.206167 (+820290us) service_pool.cc:224] Handling call
0709 12:12:41.206494 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:41.218851 (+ 12357us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:41.218905 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:41.234896 (+ 15991us) log.cc:844] Serialized 26 byte log entry
0709 12:12:41.398178 (+163282us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:41.782227 (+384049us) log.cc:844] Serialized 14 byte log entry
0709 12:12:42.183154 (+400927us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:42.183356 (+   202us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:42.183409 (+    53us) raft_consensus.cc:1685] finished
0709 12:12:42.183426 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:42.183605 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":383584,"prepare.queue_time_us":384511,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":3392,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":13710,"thread_start_us":536899,"threads_started":2,"wal-append.queue_time_us":533227}
I20260709 12:12:42.210426 12643 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:42.218721 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:42.183105 19213 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:42.224574 12674 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 15. Current term is 16. Ops: [12.7-15.8]
I20260709 12:12:42.224891 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:42.183326 19070 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:42.226204 12676 raft_consensus.cc:2393] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 19: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:12:42.226801 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 84608us
I20260709 12:12:42.248677 12678 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:12:42.248886 19130 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:42.225139 19213 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:42.222973 19119 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:42.250943 12643 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:42.222632 19045 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:42.229609 19207 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:42.262221 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:42.261757 12425 catalog_manager.cc:5697] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 4 to 7. New cstate: current_term: 7 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:12:42.266649 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:42.205011 19311 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:42.317438 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50912.1us
I20260709 12:12:42.318393 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:42.261121 19045 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:42.265015 19207 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:42.325587 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:42.326992 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:42.345930 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:42.327708 19355 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:42.334556 18592 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:42.338207 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:42.348546 12674 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:12:42.348827 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:42.301509 19250 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:42.348992 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:42.352656 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:42.353364 12671 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:42.356290 12671 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:12:42.260222 19119 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:42.364850 12680 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:42.350478 19207 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:42.347692 19251 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:42.395812 12671 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 17 index: 13. (index mismatch)
W20260709 12:12:42.401712 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:42.407444 19047 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:42.411561 19093 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:42.354374 19130 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:42.351320 19250 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:42.417959 12674 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:42.437734 19250 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:42.455060 19360 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.098s	user 0.000s	sys 0.000s
I20260709 12:12:42.398397 19135 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:42.276851 19214 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:42.467396 12602 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:42.437114 19216 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:42.468727 19216 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:42.474555 19130 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:42.475694 19130 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:42.496433 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 464311us
I20260709 12:12:42.473675 19045 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:42.448688 19047 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:42.397096 19102 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:42.505405 19355 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:42.520148 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3132) took 1662 ms. Trace:
W20260709 12:12:42.521050 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 287602us
I20260709 12:12:42.515338 12572 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:42.520895 12425 catalog_manager.cc:5697] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 6 to 8. New cstate: current_term: 8 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } }
W20260709 12:12:42.522724 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:42.521768 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:42.527922 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:42.528203 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:42.520578 12605 rpcz_store.cc:276] 0709 12:12:40.857867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.372391 (+514524us) service_pool.cc:224] Handling call
0709 12:12:41.409059 (+ 36668us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:41.409354 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:12:41.782390 (+373036us) log.cc:844] Serialized 14 byte log entry
0709 12:12:42.519788 (+737398us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:42.519846 (+    58us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:42.519891 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:42.519960 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:42.520086 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":334932,"prepare.queue_time_us":335804,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":75202,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":106,"thread_start_us":736819,"threads_started":2,"wal-append.queue_time_us":586418}
W20260709 12:12:42.537150 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:42.508924 19047 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:42.518299 19214 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:12:42.512768 18592 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:42.545367 19047 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:42.555778 19119 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:42.555929 19355 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:42.545177 18862 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:42.512135 19093 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.035s
I20260709 12:12:42.536726 19216 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:42.566627 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 217719us
I20260709 12:12:42.568392 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:42.573269 13338 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:42.562150 19053 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:42.585757 12643 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:42.572989 19214 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:12:42.585283 19273 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:12:42.581172 19119 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:42.582448 19370 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:12:42.609174 19332 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.544s	user 0.007s	sys 0.008s
I20260709 12:12:42.616255 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:42.608409 19251 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:42.619570 12605 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:42.617635 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:42.608273 19373 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:42.626721 19245 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:42.628086 19142 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:42.616235 19252 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.904s	user 0.000s	sys 0.017s
W20260709 12:12:42.630898 19365 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 12:12:42.631542 19361 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.100s	user 0.000s	sys 0.001s
I20260709 12:12:42.635089 12601 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:42.639150 19226 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:42.610307 19214 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:42.613035 19370 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:42.632879 19358 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.154s	user 0.000s	sys 0.002s
W20260709 12:12:42.648186 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 120166us
I20260709 12:12:42.652768 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:42.653343 19151 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:12:42.663370 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:42.679929 12599 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 17: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:12:42.664963 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:42.680600 12643 connection.cc:256] RPC call timeout handler was delayed by -1.0632555510001112s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17500000000000002s for a response
I20260709 12:12:42.680738 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:42.681486 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:42.641943 19373 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:42.674727 12605 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:12:42.633227 19216 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:12:42.683614 12571 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:42.683887 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:42.677940 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:42.686617 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:42.694043 12601 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:42.676306 19102 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:42.682482 19252 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:42.689637 19151 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:12:42.689350 19150 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:42.679795 19370 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:42.668053 19332 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:42.642539 19251 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:42.700302 19226 raft_consensus.cc:697] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:42.704444 12598 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:42.737900 12601 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:42.680519 19214 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:42.691409 19373 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:42.722836 19151 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:42.729689 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 48300.7us
I20260709 12:12:42.720922 12604 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:42.753952 19355 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:42.727751 19252 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:42.743436 18862 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:42.732944 19332 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:42.786216 19375 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:12:42.786700 19252 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:42.787668 19151 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:42.788681 19214 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:42.789372 19151 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:12:42.791435 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:42.780033 19366 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.178s	user 0.001s	sys 0.000s
I20260709 12:12:42.795586 19142 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:42.761090 19207 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:42.806773 19207 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:42.811527 19307 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:12:42.812134 19307 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:42.814344 19355 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:42.814384 19252 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:42.816005 19252 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:42.818522 19252 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:42.775799 19130 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:42.823040 19130 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:42.796106 19332 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:42.802881 19251 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:42.799821 19226 consensus_queue.cc:237] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:42.814451 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:42.831809 12680 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15.
W20260709 12:12:42.800734 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 97674.8us
I20260709 12:12:42.851466 12528 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:12:42.857770 12678 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:42.859350 12494 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:42.862910 19265 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:42.851152 19130 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:42.863370 19265 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:42.863821 19265 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:42.865531 19151 consensus_queue.cc:1059] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:12:42.836583 19245 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:42.890455 18772 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:12:42.893322 19372 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
W20260709 12:12:42.900650 19169 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.713s	user 0.010s	sys 0.003s
W20260709 12:12:42.842337 19141 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.638s	user 0.011s	sys 0.002s
I20260709 12:12:42.813321 19307 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:42.901422 19169 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:42.815220 19355 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:42.920570 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:42.923931 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:42.911427 19141 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:42.929549 19298 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:42.930409 19298 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:42.930859 19298 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:42.932047 19141 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:42.918079 19307 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:12:42.934720 19354 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.961s	user 0.008s	sys 0.000s
I20260709 12:12:42.935534 12603 raft_consensus.cc:606] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:42.936002 12603 raft_consensus.cc:614] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:42.938856 19169 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:42.943397 12597 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 11. Current term is 12. Ops: []
I20260709 12:12:42.946085 19369 raft_consensus.cc:3055] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:42.946743 19369 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:42.949306 19135 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:42.949867 19135 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:42.922952 18862 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:42.964857 18862 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:42.981433 19369 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:42.982892 19369 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:42.987808 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:42.988154 13305 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:42.989749 19385 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:42.991492 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:42.992712 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:42.997983 12528 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:43.003171 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:43.008364 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2839) took 2718 ms. Trace:
I20260709 12:12:43.009262 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:43.008991 12600 rpcz_store.cc:276] 0709 12:12:40.289596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:40.350162 (+ 60566us) service_pool.cc:224] Handling call
0709 12:12:40.350554 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:40.351043 (+   489us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:12:41.112831 (+761788us) spinlock_profiling.cc:243] Waited 2.57 ms on lock 0x7b18003c8c20. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf 00007f1488a3a038
0709 12:12:41.335593 (+222762us) log.cc:844] Serialized 14 byte log entry
0709 12:12:41.998205 (+662612us) spinlock_profiling.cc:243] Waited 7.71 ms on lock 0x7b480024c258. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:12:42.316902 (+318697us) log.cc:844] Serialized 14 byte log entry
0709 12:12:42.317098 (+   196us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:42.344443 (+ 27345us) log.cc:844] Serialized 26 byte log entry
0709 12:12:42.344633 (+   190us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:42.344869 (+   236us) log.cc:844] Serialized 14 byte log entry
0709 12:12:42.345054 (+   185us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:42.345129 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:42.934336 (+589207us) spinlock_profiling.cc:243] Waited 97.8 ms on lock 0x7b240033f7b8. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14857b99c4 00007f14857b7acb 00007f1485972dc9 00007f1485972a55 00007f148597209d 00007f14885a410e 00007f14885bb88c 00007f14885babb6 00007f14885ba7d8 00007f14885c4221
0709 12:12:43.008052 (+ 73716us) raft_consensus.cc:1685] finished
0709 12:12:43.008087 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:43.008275 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42272,"prepare.queue_time_us":780625,"prepare.run_cpu_time_us":172,"prepare.run_wall_time_us":191,"raft.queue_time_us":47373,"raft.run_cpu_time_us":2213,"raft.run_wall_time_us":164636,"spinlock_wait_cycles":254801408,"thread_start_us":1030872,"threads_started":3,"wal-append.queue_time_us":595818}
I20260709 12:12:42.924903 19250 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.017256 12646 leader_election.cc:400] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16.
I20260709 12:12:43.020656 13305 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:43.023900 12646 leader_election.cc:403] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:42.990340 19385 raft_consensus.cc:697] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:43.001199 19394 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.043398 18862 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:43.043653 19394 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.045634 19355 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:43.029414 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:43.034040 19370 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.035399 19373 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:43.027307 19143 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:43.040561 19311 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:43.001493 18993 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.042888 19194 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:43.028649 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:43.046422 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:43.046746 19281 raft_consensus.cc:3055] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Stepping down as leader of term 12
I20260709 12:12:43.049443 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:43.051347 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 52179.6us
I20260709 12:12:43.063104 12676 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
I20260709 12:12:43.063365 12646 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:43.051496 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:43.020190 18972 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.052763 12675 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:43.028903 19070 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:43.059827 13332 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:12:43.076678 19143 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:43.076501 18972 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.077226 19143 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:43.064646 13304 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:43.028048 19399 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.062760 19281 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:43.078871 18972 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:43.078675 19399 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:43.079025 19385 consensus_queue.cc:237] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:43.079257 19281 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.080278 19281 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:43.081151 19399 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:43.069022 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:43.070040 12606 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:43.082293 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:43.070480 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:43.083107 19373 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:12:43.060961 18993 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.078733 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:12:43.083804 19213 consensus_meta.cc:287] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.819s	user 0.008s	sys 0.017s
I20260709 12:12:43.058996 19047 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.084270 19213 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.085306 18993 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:43.050184 19394 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:43.078013 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:43.086439 19399 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:43.086936 19399 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:43.087319 19047 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:43.087430 19399 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:43.019570 19265 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.092384 19265 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:43.092883 19119 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:43.093490 19119 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:43.094398 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 412219us
I20260709 12:12:43.094936 19379 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:43.095240 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:43.095500 19379 raft_consensus.cc:697] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:43.096411 19379 consensus_queue.cc:237] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 14.6, Last appended by leader: 6, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.098609 19378 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:43.099198 19378 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:43.100190 19378 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.101977 12530 raft_consensus.cc:2368] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18.
I20260709 12:12:43.103338 13333 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
W20260709 12:12:43.104557 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:43.105222 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:43.055182 19130 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16.
I20260709 12:12:43.115507 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 758) took 1114 ms. Trace:
I20260709 12:12:43.116118 12523 rpcz_store.cc:276] 0709 12:12:42.001200 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:42.001536 (+   336us) service_pool.cc:224] Handling call
0709 12:12:42.002164 (+   628us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:42.141181 (+139017us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:43.115239 (+974058us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b54003cb9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:43.115442 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":270179200}
I20260709 12:12:43.121783 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:32832 (request call id 15) took 1331 ms. Trace:
I20260709 12:12:43.122098 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:43.123116 12531 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 20 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 12.
I20260709 12:12:43.122478 12603 rpcz_store.cc:276] 0709 12:12:41.783590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:41.862724 (+ 79134us) service_pool.cc:224] Handling call
0709 12:12:43.115174 (+1252450us) spinlock_profiling.cc:243] Waited 973 ms on lock 0x7b540007ba54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1a853 00007f148d9e860a 00007f1486f6e7d4 00007f1486f6e762 00007f1486f6e691 00007f1486f6e60c 00007f1486f6d8c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:43.115333 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2239028096}
I20260709 12:12:43.141414 12531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:43.142835 12531 raft_consensus.cc:606] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:43.143620 12531 raft_consensus.cc:614] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:43.135517 19142 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.153306 19142 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:43.149435 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:43.155181 12600 raft_consensus.cc:2368] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 16. Current term is 17.
I20260709 12:12:43.151291 19135 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.152277 19298 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:43.160025 12601 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24. Ops: []
W20260709 12:12:43.163832 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 636549us
I20260709 12:12:43.166077 12572 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:43.167254 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3142) took 1586 ms. Trace:
I20260709 12:12:43.167878 12602 rpcz_store.cc:276] 0709 12:12:41.581165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:42.017661 (+436496us) service_pool.cc:224] Handling call
0709 12:12:42.018086 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:42.468083 (+449997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:43.166926 (+698843us) spinlock_profiling.cc:243] Waited 442 ms on lock 0x7b5400079c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:43.167174 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1017365248}
I20260709 12:12:43.171416 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:43.172159 12602 raft_consensus.cc:2368] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 16. Current term is 17.
I20260709 12:12:43.173509 12600 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 18 index: 15. (index mismatch)
I20260709 12:12:43.179144 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:43.179394 13338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 847) took 1031 ms. Trace:
I20260709 12:12:43.179791 13338 rpcz_store.cc:276] 0709 12:12:42.148441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:42.148646 (+   205us) service_pool.cc:224] Handling call
0709 12:12:43.179224 (+1030578us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b54000d7854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:43.179335 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":939982208}
I20260709 12:12:43.138561 19213 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:43.186053 12572 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:43.190197 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:43.187470 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:43.195020 12598 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 22: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:43.198316 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:43.169770 19294 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:43.189786 19141 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:43.131422 19119 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 12.7, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:43.221441 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 158152us
I20260709 12:12:43.199491 19357 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:43.207401 19214 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:12:43.229229 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 83705.7us
I20260709 12:12:43.235395 12601 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:43.209690 19213 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:43.242033 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:43.243359 12602 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:43.244208 12601 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:43.245617 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:43.247278 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:43.248075 12601 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:43.249167 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:43.239518 19287 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:43.251119 19287 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.217706 19300 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.258272 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:43.259686 12602 raft_consensus.cc:2393] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:43.262871 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:43.260727 12601 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 17: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:43.266413 13302 leader_election.cc:400] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 16. Current term is 17.
I20260709 12:12:43.268043 19298 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:43.275251 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:43.238168 19141 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.277025 19141 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.278640 12598 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:43.280269 19141 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:43.224467 19093 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:43.282582 19093 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.295436 19045 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:43.296981 19399 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:43.299573 19135 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:43.302726 19395 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.145s	user 0.000s	sys 0.000s
I20260709 12:12:43.352628 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:43.310717 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:43.315768 19273 consensus_meta.cc:287] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.704s	user 0.009s	sys 0.005s
I20260709 12:12:43.316296 19300 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.365237 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:43.365928 12675 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:12:43.366326 12675 pending_rounds.cc:85] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 10
I20260709 12:12:43.366561 12675 pending_rounds.cc:107] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:12:43.366766 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:43.367391 12601 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:43.285619 19093 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:43.322969 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:43.380474 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:43.380681 12525 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:43.277017 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:43.293625 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:43.381842 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:43.382112 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:43.382561 12680 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:43.325578 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:43.343178 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 264515us
I20260709 12:12:43.383963 12671 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:43.384732 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:43.344585 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:43.385016 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 162160us
I20260709 12:12:43.232371 19250 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:43.367451 12678 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
W20260709 12:12:43.385931 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 118582us
I20260709 12:12:43.368078 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:43.368404 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:43.387882 12601 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:43.368746 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:43.366787 12675 raft_consensus.cc:2889] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:43.371361 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:43.390291 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 206471us
I20260709 12:12:43.374641 12598 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:43.325016 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:43.383476 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:43.393072 12680 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:12:43.319660 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:43.395130 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:43.387952 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:43.388679 12530 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 18: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:43.265508 19370 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:43.427793 13337 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:43.345446 19151 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.345647 19045 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:43.321151 19150 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:43.421281 19385 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:43.219708 19294 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:43.361304 18993 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
W20260709 12:12:43.424396 19401 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.140s	user 0.001s	sys 0.000s
I20260709 12:12:43.440815 12525 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:43.443490 12496 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:43.441591 12602 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:12:43.408875 19298 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 12:12:43.383096 19273 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:12:43.264137 19213 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:43.415903 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:43.479460 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:43.481333 12643 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:43.452924 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:43.452039 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:43.432952 19278 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.451941 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:43.468475 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:43.453547 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:43.485553 12678 raft_consensus.cc:2410] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:12:43.476060 19265 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:43.433661 19374 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:43.428983 12601 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:43.470691 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:43.404102 19263 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:12:43.471256 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 85276.6us
I20260709 12:12:43.458842 12529 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
W20260709 12:12:43.473418 19396 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.335s	user 0.000s	sys 0.001s
I20260709 12:12:43.473410 19164 consensus_queue.cc:1059] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 14.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:43.475330 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:43.424808 18723 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:43.432353 19032 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.477360 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:43.483767 12676 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:43.453156 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:43.486670 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:43.489022 12679 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24. Ops: []
I20260709 12:12:43.450852 19151 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.486511 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:43.489619 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:43.495018 12605 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:43.519209 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:43.519887 12646 leader_election.cc:400] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18.
I20260709 12:12:43.490684 19265 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:43.476590 19332 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:43.501132 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:43.508808 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:43.507383 19373 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:43.511806 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:43.522145 12646 leader_election.cc:403] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:43.477906 19205 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.038s
I20260709 12:12:43.496769 18993 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:43.520059 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:43.509853 12680 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:43.491148 19374 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:43.524765 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:43.533957 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:43.518074 19383 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.707s	user 0.005s	sys 0.022s
I20260709 12:12:43.522941 19390 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:43.527047 19265 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:43.527920 19251 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:43.490176 19278 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.510854 19263 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:43.539644 19386 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.540293 19374 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:43.538215 19164 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 17. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18.
I20260709 12:12:43.518047 18723 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:43.545994 19383 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.735s	user 0.005s	sys 0.023s
I20260709 12:12:43.551429 19390 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:43.521255 19032 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:43.526909 19332 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.571398 19390 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:43.577018 19143 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:43.578233 19213 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:43.578970 19263 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:43.579331 19294 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:43.579900 19300 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:43.580873 19332 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:43.568071 19386 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:43.584933 19307 consensus_meta.cc:287] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.651s	user 0.016s	sys 0.003s
I20260709 12:12:43.599527 19386 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:43.601970 19373 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:12:43.602669 19311 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.544s	user 0.005s	sys 0.007s
W20260709 12:12:43.476112 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:43.475485 12496 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:43.838578 19307 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:43.839555 19307 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:43.648187 12522 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:43.506108 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:43.840678 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 365148us
I20260709 12:12:43.842000 12524 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:43.650096 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:43.646802 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:43.846686 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:43.650738 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:43.848223 12527 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 15: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:43.677995 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:43.850239 18592 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:12:43.850800 12675 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:12:43.681150 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:43.682664 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:43.661871 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:43.693871 18723 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:43.696255 19151 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:43.530819 19045 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.697851 19032 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:43.701915 19213 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:43.587323 19278 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:43.724151 13305 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:43.733672 19130 raft_consensus.cc:3055] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:43.734902 19245 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:43.736567 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:43.739802 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:43.753147 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:43.754616 19207 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:43.756072 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:43.757020 19273 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.758141 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:43.760560 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:43.766927 19164 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:43.770794 19214 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.772922 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:43.777683 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:43.604306 19369 consensus_meta.cc:287] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.621s	user 0.014s	sys 0.001s
I20260709 12:12:43.795125 19135 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:43.839596 19386 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:43.873380 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:43.873090 19164 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:43.873947 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 193233us
I20260709 12:12:43.875681 19300 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:43.876379 19300 raft_consensus.cc:697] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:43.667017 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:43.685331 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:43.856706 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:43.857179 19373 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:43.879923 12525 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:43.880180 19307 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:43.879848 19373 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.880934 19135 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.881623 19287 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:43.882807 19389 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.883663 19135 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:43.883483 19389 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:43.860301 19407 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.286s	user 0.000s	sys 0.002s
W20260709 12:12:43.887248 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 163567us
I20260709 12:12:43.887662 19245 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:43.860605 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:43.890566 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:43.883510 19300 consensus_queue.cc:237] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 15.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:43.892297 12527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:43.867537 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:43.893177 12527 raft_consensus.cc:606] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:43.893635 12527 raft_consensus.cc:614] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:43.894707 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:43.895502 13338 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:43.870450 12602 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 17: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:43.896037 19135 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.870911 19213 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.870646 12673 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:43.896777 19135 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.897671 12597 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:43.870801 12672 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 17: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:43.871316 19214 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.899055 19213 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:43.899559 19135 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:43.901749 19214 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:43.871515 19207 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.872439 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:43.872607 12679 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:43.877341 19164 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:43.877676 19311 consensus_queue.cc:1059] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:43.871923 12676 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:12:43.871443 19273 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.864210 19130 raft_consensus.cc:740] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:43.896020 12603 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15. Ops: [12.6-14.7]
I20260709 12:12:43.904209 19373 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:43.905826 12524 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
W20260709 12:12:43.921854 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58689us
I20260709 12:12:43.922866 19130 consensus_queue.cc:260] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.924130 19130 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:43.930294 12597 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:43.940423 12571 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:43.920022 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:43.942827 12605 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:43.943356 19150 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.944339 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:43.944796 19389 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:43.944952 19373 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.945719 19373 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.946672 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:12:43.944080 19150 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.951422 19373 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:43.949741 19150 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:43.958500 12605 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:12:43.962601 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:43.963454 12605 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:12:43.965710 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:43.966599 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:43.967458 12603 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:43.958323 19378 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:43.972787 19378 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:43.974488 12602 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:12:43.980866 19207 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:43.983255 19385 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.533s	user 0.015s	sys 0.004s
I20260709 12:12:43.984113 19385 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:43.985473 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:43.987207 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:43.985806 19214 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:12:43.987592 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 62183.7us
I20260709 12:12:43.993134 19357 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.002861 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:43.969790 19273 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:44.006831 12571 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:44.013015 12673 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:44.017637 19142 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:44.021278 13333 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:44.015051 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:44.023741 13337 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:44.028688 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:44.032812 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46808.2us
I20260709 12:12:44.035956 19213 raft_consensus.cc:3055] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:44.036464 19213 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:44.048256 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:44.052068 13333 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 14: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:44.059427 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:44.065169 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:44.066200 19142 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:12:44.072142 12605 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: []
W20260709 12:12:44.074719 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 73060.3us
W20260709 12:12:44.075083 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 50375.7us
I20260709 12:12:44.075731 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:44.076059 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:44.078315 19389 raft_consensus.cc:2764] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:12:44.079002 18723 consensus_queue.cc:1059] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:44.080739 19389 consensus_queue.cc:1059] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:44.081061 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:44.082762 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:44.085328 12603 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:44.081650 18723 raft_consensus.cc:3055] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:44.077971 12494 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:44.022758 19357 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:44.043246 18993 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:44.092480 12524 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:12:44.098369 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:44.037496 19213 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:44.095676 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:44.045928 19265 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:44.091650 18723 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:44.091876 19378 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:44.110291 12597 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:44.081696 19273 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:44.112090 19373 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:44.114756 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:44.126237 19135 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:44.127065 19135 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:44.127435 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:44.127827 19135 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:12:44.128647 19390 consensus_meta.cc:287] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.548s	user 0.006s	sys 0.010s
I20260709 12:12:44.129290 19390 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.095942 18993 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.119469 18592 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:44.143649 12524 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:12:44.145985 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:44.148125 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:44.149524 12597 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:44.153736 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:44.165401 12524 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 14. Current term is 15. Ops: []
W20260709 12:12:44.166477 19374 consensus_meta.cc:287] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.597s	user 0.020s	sys 0.000s
I20260709 12:12:44.167090 19374 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:44.168290 12643 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:44.170002 12605 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:12:44.170459 19355 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:12:44.171567 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50277.9us
I20260709 12:12:44.171960 19399 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:44.171425 19273 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:44.174069 19426 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.238s	user 0.001s	sys 0.000s
W20260709 12:12:44.174659 19428 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.198s	user 0.001s	sys 0.000s
W20260709 12:12:44.174930 19411 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.516s	user 0.000s	sys 0.003s
I20260709 12:12:44.207546 12600 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 12. Current term is 13. Ops: []
I20260709 12:12:44.151571 18592 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:44.211510 19373 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.212041 19390 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:44.214807 12601 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 17 index: 11. (index mismatch)
I20260709 12:12:44.219231 18723 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:44.221165 18723 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:44.222522 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:44.230232 12525 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:12:44.230469 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:44.231176 12646 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:44.231742 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:44.220990 19151 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:44.233027 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:44.233460 19151 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:44.233901 19151 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:44.234517 12646 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:44.235131 19151 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:44.235808 19419 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:44.236711 19151 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:44.238736 19389 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:44.239435 19389 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:44.231616 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:44.240240 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:44.241385 12528 raft_consensus.cc:2393] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:44.240867 19389 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.241815 12525 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:12:44.158962 19045 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:44.242875 19045 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.244575 19205 raft_consensus.cc:3055] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:44.245205 19205 raft_consensus.cc:740] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:44.239768 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:44.246825 12597 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:44.255566 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:44.258889 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:44.217516 18592 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:44.267714 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:44.273010 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:44.275699 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 120373us
I20260709 12:12:44.276580 12572 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:44.277557 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:44.263010 19164 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:12:44.285982 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 181266us
I20260709 12:12:44.270542 19304 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:44.292186 12524 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:44.297871 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:44.271023 19213 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:44.246649 19150 consensus_queue.cc:1059] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.077s
I20260709 12:12:44.326481 12530 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:12:44.329674 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 254067us
I20260709 12:12:44.327119 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:44.295755 19311 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:44.334578 19164 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:44.336503 19164 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:44.336925 19164 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:44.337332 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:44.335978 19142 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:44.274533 19032 consensus_queue.cc:1059] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260709 12:12:44.338383 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:44.334288 12526 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 18. Current term is 19. Ops: []
W20260709 12:12:44.342411 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 79932.6us
I20260709 12:12:44.342845 12678 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:44.346771 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:44.355957 12528 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 16. Current term is 17. Ops: []
W20260709 12:12:44.406160 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 212676us
W20260709 12:12:44.408241 19434 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:12:44.340740 19142 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:44.408948 19142 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:44.397733 19304 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:44.413771 19304 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:44.414283 19304 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:44.426016 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:44.401479 19433 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.176s	user 0.000s	sys 0.004s
W20260709 12:12:44.433312 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 95024.3us
I20260709 12:12:44.434413 19378 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:44.435029 19378 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:44.435436 19378 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:44.437292 19151 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:44.346083 19357 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:44.401623 19311 raft_consensus.cc:697] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:44.444089 19311 consensus_queue.cc:237] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.446074 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3169) took 1207 ms. Trace:
I20260709 12:12:44.447260 19413 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.448020 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3165) took 1214 ms. Trace:
I20260709 12:12:44.447889 19413 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:44.322660 19205 consensus_queue.cc:260] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:44.451246 19205 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:44.446758 12599 rpcz_store.cc:276] 0709 12:12:43.237990 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:43.238329 (+   339us) service_pool.cc:224] Handling call
0709 12:12:43.238712 (+   383us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:43.242407 (+  3695us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:12:43.802976 (+560569us) log.cc:844] Serialized 14 byte log entry
0709 12:12:44.277737 (+474761us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:44.342479 (+ 64742us) log.cc:844] Serialized 52 byte log entry
0709 12:12:44.342646 (+   167us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:44.342681 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:44.342768 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:44.445583 (+102815us) raft_consensus.cc:1685] finished
0709 12:12:44.445612 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:44.445803 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":356481,"prepare.queue_time_us":536651,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":46,"raft.queue_time_us":129956,"raft.run_cpu_time_us":315,"raft.run_wall_time_us":95359,"thread_start_us":637647,"threads_started":2,"wal-append.queue_time_us":495811}
I20260709 12:12:44.452533 19278 raft_consensus.cc:3055] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Stepping down as leader of term 11
I20260709 12:12:44.452934 19278 raft_consensus.cc:740] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:44.453089 19413 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:44.457244 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:44.457954 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:44.460253 19135 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.460906 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:44.461541 12530 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:44.472657 12646 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:44.474720 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 74544.2us
W20260709 12:12:44.478109 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:44.475682 19431 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.153s	user 0.000s	sys 0.000s
I20260709 12:12:44.479310 19130 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:12:44.479984 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:44.484647 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:44.484298 19150 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:12:44.486279 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 79239.5us
I20260709 12:12:44.489284 18993 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:44.490123 19430 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.441s	user 0.002s	sys 0.001s
I20260709 12:12:44.490629 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:44.497370 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:44.499060 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:44.448544 12606 rpcz_store.cc:276] 0709 12:12:43.232983 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:43.233286 (+   303us) service_pool.cc:224] Handling call
0709 12:12:43.266691 (+ 33405us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:43.351549 (+ 84858us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:12:43.809252 (+457703us) log.cc:844] Serialized 14 byte log entry
0709 12:12:44.272923 (+463671us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:44.335347 (+ 62424us) log.cc:844] Serialized 26 byte log entry
0709 12:12:44.398282 (+ 62935us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:44.398329 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:44.403603 (+  5274us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:44.447537 (+ 43934us) raft_consensus.cc:1685] finished
0709 12:12:44.447568 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:44.447803 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":369622,"prepare.queue_time_us":449479,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":36,"raft.queue_time_us":237789,"raft.run_cpu_time_us":260,"raft.run_wall_time_us":30182,"thread_start_us":600286,"threads_started":2,"wal-append.queue_time_us":459966}
I20260709 12:12:44.399121 19265 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:44.508616 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:44.512164 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:44.511708 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:44.517271 12605 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:44.514032 13333 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:12:44.523958 12604 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:44.453927 19278 consensus_queue.cc:260] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 11, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.526186 19278 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:44.504599 19182 consensus_queue.cc:1059] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.038s
I20260709 12:12:44.539469 19135 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:44.541067 19357 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.500066 19385 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 12:12:44.542331 19032 raft_consensus.cc:3055] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:44.542735 19032 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:44.543821 19032 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:44.558192 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 57527.1us
I20260709 12:12:44.559149 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:44.560487 12571 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:44.564821 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:44.541718 19357 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:44.572118 19357 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:44.572962 19357 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:44.574661 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:44.574805 19304 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.577005 19413 raft_consensus.cc:3055] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:44.578168 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:44.579125 12604 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:44.583716 12675 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
I20260709 12:12:44.577543 19413 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:12:44.588233 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 243328us
W20260709 12:12:44.606063 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:44.549533 18723 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:44.585388 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:44.573989 19357 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:12:44.622146 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 77103.1us
W20260709 12:12:44.634614 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 341638us
I20260709 12:12:44.612624 18723 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:44.619814 19357 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:44.569638 19378 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.658493 12646 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:44.589599 18592 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:44.648036 19300 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.011s	sys 0.007s
I20260709 12:12:44.663880 19135 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:44.667847 12531 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20. Ops: []
W20260709 12:12:44.668529 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:44.667896 19135 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:44.669237 19135 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:44.667920 12523 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:44.670063 19390 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:44.674264 19364 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:44.675051 19364 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:44.676254 19207 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:44.677322 19364 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:44.677111 19207 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:44.674304 13338 raft_consensus.cc:3055] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:44.678594 13338 raft_consensus.cc:740] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:44.657537 19357 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:44.666440 19300 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.639s	user 0.011s	sys 0.007s
I20260709 12:12:44.686707 12604 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:44.688349 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:44.682250 19032 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:44.680056 19364 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:44.696452 13338 consensus_queue.cc:260] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:44.698314 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:44.685277 19213 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:44.689816 12572 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:44.691263 19389 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:12:44.702224 19214 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 12:12:44.704346 18723 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.724176 18723 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:44.711318 12494 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:44.706549 13338 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:44.735528 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46141.7us
W20260709 12:12:44.733803 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 44067.4us
I20260709 12:12:44.684237 19413 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:44.719883 19207 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:44.709380 19378 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:44.741830 18993 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:44.767156 12494 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:44.746006 19304 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:44.744817 19142 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:44.763494 18993 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:44.783233 19142 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:44.783457 18993 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:44.771046 12494 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:44.784739 19273 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:44.785569 12494 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:44.786965 19373 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:44.744109 19164 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:44.789402 19164 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:44.796139 19045 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:44.763679 19130 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.798648 19130 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.800861 19130 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:44.808516 19304 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.809353 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:44.811033 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 68493.2us
I20260709 12:12:44.777011 12646 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:44.760738 19364 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:44.778637 18592 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:44.778873 19374 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:44.784973 19265 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:44.811631 19135 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.812837 12678 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:44.775380 19357 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.713465 19205 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:12:44.813838 19384 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:12:44.829097 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 505381us
I20260709 12:12:44.776328 19278 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:44.759162 19213 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:44.758638 19390 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 12:12:44.813800 19150 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:12:44.841251 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:44.845027 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:44.845728 13337 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:12:44.846235 19213 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:44.846060 19443 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:44.848188 19205 raft_consensus.cc:3055] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:44.847895 19443 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.848598 19205 raft_consensus.cc:740] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:44.849534 19205 consensus_queue.cc:260] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.850919 19205 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:44.855366 19357 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:44.858489 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:44.859930 19374 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:12:44.860136 19032 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:44.860160 19443 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:44.861198 19045 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:44.861877 19045 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:44.862525 19355 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:44.863260 18592 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:44.864210 19355 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:44.864970 19439 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:44.866240 19439 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:44.834275 19278 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:44.866627 19439 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:44.866804 19278 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:44.867671 19439 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.869508 19439 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:44.809360 19304 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.871644 19364 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:44.872357 19304 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:44.865957 19273 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:44.847043 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:44.829602 19384 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:44.861598 19374 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.018s
I20260709 12:12:44.836150 19164 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:44.826225 19130 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:44.872506 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 95645.2us
I20260709 12:12:44.889575 19164 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:44.890946 12522 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:12:44.863071 19045 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.894023 13305 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:44.896201 19298 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:44.896793 19298 raft_consensus.cc:697] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:44.897187 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:44.897715 19298 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 15.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:44.878209 19438 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:44.899271 19438 raft_consensus.cc:697] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:44.900230 19438 consensus_queue.cc:237] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.7, Last appended by leader: 7, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.878866 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:44.902523 19278 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.834743 12569 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:44.889664 12646 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:44.906188 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 62350.8us
W20260709 12:12:44.907315 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 346943us
W20260709 12:12:44.909235 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 74567.1us
I20260709 12:12:44.842660 19150 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:44.878949 19182 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 17 current term: 17 is higher.
I20260709 12:12:44.883172 19135 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:44.876788 19413 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:44.879263 19384 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:44.966622 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:44.880285 19374 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:12:44.976838 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 55339.5us
I20260709 12:12:44.878774 19273 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:12:44.979166 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 120736us
I20260709 12:12:44.980679 12646 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:44.981694 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 51810.9us
I20260709 12:12:44.921415 19304 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:12:44.921216 19164 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:44.984519 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:44.930344 19150 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:44.883579 19130 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.990209 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:44.993852 12569 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:44.982605 19265 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:45.004067 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:45.004153 12529 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 8. (term mismatch)
I20260709 12:12:44.983861 19374 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:44.976563 18993 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:44.985314 19304 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:45.004691 12529 pending_rounds.cc:85] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 7
I20260709 12:12:45.012190 12522 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 12:12:44.985982 19273 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:45.013419 12529 pending_rounds.cc:107] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 15.8
I20260709 12:12:45.014597 19135 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:45.020808 12529 raft_consensus.cc:2889] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:45.021342 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:45.021054 19135 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:45.023605 19135 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:45.030738 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 101138us
I20260709 12:12:45.031876 12646 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:45.032754 13302 leader_election.cc:336] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.572s (SENT)
I20260709 12:12:44.985240 18592 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:12:45.015225 19445 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:45.035750 19445 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.036099 19273 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.037479 19273 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:45.039184 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:45.001081 19142 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:12:45.042538 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 76046.2us
I20260709 12:12:45.022047 12523 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:45.004817 19447 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:12:45.049619 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:45.049907 19135 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:45.051079 19135 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:45.051425 18993 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:45.051529 19135 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:45.052906 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:45.053905 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:45.038322 19455 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:45.054455 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:45.055424 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:45.054714 12530 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 21: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:45.056777 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:45.057650 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:45.051105 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52857.4us
I20260709 12:12:45.015069 19304 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:12:45.044091 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 257003us
I20260709 12:12:45.079141 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:45.080267 18993 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:12:45.081092 18993 consensus_queue.cc:1243] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:12:45.082049 19438 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:45.083214 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:45.083865 19438 raft_consensus.cc:3055] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:45.084290 19438 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:45.085273 18723 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:12:45.094077 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:45.086423 19438 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.097219 19438 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:45.049681 19447 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:45.057368 19045 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:45.110836 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:45.111114 12529 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
W20260709 12:12:45.112803 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:45.114022 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:45.068029 19445 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:45.119552 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:45.121747 12571 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:45.098001 19130 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:45.126657 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:45.127998 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:45.119699 13337 pending_rounds.cc:85] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 11
I20260709 12:12:45.134390 13337 pending_rounds.cc:107] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 16.12
I20260709 12:12:45.134769 13337 raft_consensus.cc:2889] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:45.126842 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:45.140106 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:45.141690 12530 raft_consensus.cc:2368] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21.
I20260709 12:12:45.108661 19447 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:12:45.141848 12524 raft_consensus.cc:2393] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 13: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:45.102376 18723 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:12:45.108438 19045 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:45.144459 19278 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:45.146957 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:45.150486 19265 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.154219 19364 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:12:45.126667 19357 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:45.156211 19357 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:45.158295 19045 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:45.159148 19357 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:45.161713 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:45.162660 13338 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 14: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:45.160269 19355 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:45.159056 19135 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.146855 19461 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:45.164470 19461 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:45.102396 19456 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 12:12:45.166646 19461 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:45.169550 19045 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.171664 19135 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:45.172014 19045 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:45.178064 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:45.179265 12529 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 16: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:45.180578 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:45.181447 12524 raft_consensus.cc:2393] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:45.182444 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:45.183621 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:45.184487 12524 raft_consensus.cc:2410] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 17 because replica has last-logged OpId of term: 12 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 11 index: 6.
W20260709 12:12:45.202999 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 218543us
W20260709 12:12:45.205965 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:45.206550 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 98299.9us
W20260709 12:12:45.211975 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 104652us
W20260709 12:12:45.206255 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 60617.1us
W20260709 12:12:45.214669 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 51882.5us
I20260709 12:12:45.213932 12572 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:45.218837 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:45.203291 19445 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader election lost for term 12. Reason: could not achieve majority
I20260709 12:12:45.221302 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:45.222745 19364 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:45.223251 19364 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:45.223371 19311 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:45.223815 19364 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:45.223984 19386 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:45.224814 19386 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:45.226891 12643 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:45.228402 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:45.229585 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:45.133728 19374 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.231007 19130 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:45.231825 19130 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:45.232491 19130 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:45.232947 19130 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:45.223272 18993 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.237187 18993 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.239367 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:45.239629 18993 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:45.214327 19213 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.022s
I20260709 12:12:45.225713 19386 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:45.241410 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:45.242674 12605 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: [16.9-17.10]
I20260709 12:12:45.241308 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:45.247896 19374 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:45.249665 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:45.255007 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:45.253479 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:45.263536 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:45.265755 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:45.265877 12522 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:45.267753 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:45.264437 12529 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 13 index: 9. (index mismatch)
I20260709 12:12:45.233985 19130 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.275847 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 12 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:45.274170 12599 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 21: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:45.265358 19445 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:12:45.275823 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:45.256198 19357 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:45.276870 12601 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:45.277161 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:45.290088 12605 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:45.295984 19304 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:45.308202 12603 raft_consensus.cc:2393] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 12: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:45.308861 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:45.312511 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:45.306443 12494 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:45.269769 19384 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.290051 19385 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.299249 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:45.324035 12598 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:12:45.266332 19444 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
W20260709 12:12:45.335441 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 82089.3us
I20260709 12:12:45.344223 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:45.312348 19311 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:45.312467 19439 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.298851 19207 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:12:45.349938 12605 raft_consensus.cc:2368] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21.
I20260709 12:12:45.317144 19373 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:45.359205 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52854.5us
I20260709 12:12:45.359923 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:45.360626 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:45.365097 19459 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.108s	user 0.000s	sys 0.001s
I20260709 12:12:45.366765 19151 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:45.367358 19151 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:45.367837 19151 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:45.374923 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:45.385129 19384 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:45.388723 19213 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:45.391779 19357 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:45.392416 19357 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.358538 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:12:45.394569 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 138078us
I20260709 12:12:45.394938 19443 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 17.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:12:45.396348 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:45.399279 12523 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:12:45.399410 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:45.400971 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:45.401381 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:45.401575 12676 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:45.402160 12673 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 19: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:45.402189 19373 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.402829 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:45.402915 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:45.403533 12679 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: [14.8-16.9]
I20260709 12:12:45.347354 19273 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:45.404259 19273 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:45.404698 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 11 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:45.404929 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:45.343003 18993 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:45.406509 18993 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:45.408124 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47625.6us
I20260709 12:12:45.408466 18993 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:45.409394 19386 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:12:45.410197 19466 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:12:45.403610 12672 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 18: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:45.402454 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:45.410794 19142 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:45.359576 19439 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.416812 19213 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.417280 19439 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:45.418361 19311 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.420790 19311 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:45.421166 19364 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.404771 19373 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:45.423267 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:45.421556 19182 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:12:45.409400 12571 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:45.429271 12604 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:45.417376 19213 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:45.446585 19213 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:45.447091 19389 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:45.447770 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:45.448029 19151 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.444195 12602 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:45.442334 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:45.344660 19455 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:45.451658 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:45.454137 12678 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:45.457005 12600 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:12:45.464834 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:45.465189 19213 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
I20260709 12:12:45.465572 12673 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:45.465730 19304 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:45.466414 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:45.470150 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:45.472360 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:45.472743 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:12:45.473382 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:45.436592 19182 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:45.478947 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:45.478919 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:45.516597 13302 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.720s (SENT) [suppressed 2 similar messages]
I20260709 12:12:45.518285 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:45.518718 12672 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 12:12:45.519200 12672 pending_rounds.cc:85] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 10
I20260709 12:12:45.519505 12672 pending_rounds.cc:107] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:12:45.519804 12672 raft_consensus.cc:2889] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:45.522439 12600 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: [16.9-17.10]
I20260709 12:12:45.523423 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:45.524402 12676 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:45.524865 19451 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 13.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:12:45.530903 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:45.532012 12677 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:12:45.348896 19385 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:45.534103 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 124797us
I20260709 12:12:45.537487 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:45.533139 13336 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:45.538182 12600 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:12:45.546967 12571 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:45.440876 19448 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.074s
W20260709 12:12:45.488762 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:45.516572 19182 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:12:45.592837 13338 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:45.476225 19413 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.068s
I20260709 12:12:45.557001 12674 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:12:45.543536 19273 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:45.618621 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:45.608083 12603 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: [18.12-18.12]
I20260709 12:12:45.619299 12602 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:45.620179 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:45.621016 12523 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:45.622499 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:45.593851 18592 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.628669 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:45.629559 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:45.641477 12599 raft_consensus.cc:2368] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19.
I20260709 12:12:45.624759 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:45.631810 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 95307.1us
I20260709 12:12:45.643373 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:45.602578 19304 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:45.649088 12522 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:45.642163 12529 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:45.652489 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:45.654693 12676 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:12:45.601270 19357 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:45.657502 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:45.606279 19438 consensus_meta.cc:287] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.506s	user 0.011s	sys 0.008s
I20260709 12:12:45.547063 19447 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:45.662874 12646 leader_election.cc:400] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21.
I20260709 12:12:45.666334 12529 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20. Ops: [17.13-19.14]
W20260709 12:12:45.667167 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 76266.9us
I20260709 12:12:45.725037 12646 leader_election.cc:403] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
W20260709 12:12:45.723227 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 51562.2us
I20260709 12:12:45.725660 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:12:45.668293 19470 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.064s	user 0.000s	sys 0.000s
W20260709 12:12:45.669068 19463 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:12:45.669181 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:45.671123 19150 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.658s	user 0.009s	sys 0.001s
I20260709 12:12:45.671866 19357 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:45.554808 19455 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:45.688673 19468 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:12:45.648752 19273 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:12:45.705688 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 6 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:45.742360 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 110073us
I20260709 12:12:45.743427 12602 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
W20260709 12:12:45.733112 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 293952us
W20260709 12:12:45.748373 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 79291.1us
I20260709 12:12:45.750316 19384 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:45.750975 19384 raft_consensus.cc:697] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:45.751823 19311 consensus_queue.cc:1059] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 13.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:45.752113 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:45.751940 19384 consensus_queue.cc:237] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.753165 19311 raft_consensus.cc:3055] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:45.753545 19311 raft_consensus.cc:740] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:45.754894 12601 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 18. Current term is 19. Ops: [18.15-18.15]
I20260709 12:12:45.755842 19389 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:45.755982 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:45.756381 19389 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:45.757419 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:45.758522 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:45.758867 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:45.760563 12679 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 14 index: 6. Preceding OpId from leader: term: 16 index: 7. (index mismatch)
I20260709 12:12:45.761755 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:45.763633 19455 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:45.764161 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:45.764892 12605 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:45.765810 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:45.766930 19385 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:45.768121 12605 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:45.768767 19461 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21.
I20260709 12:12:45.771566 19472 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:45.772030 19472 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:45.772352 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:45.774509 19447 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:45.774673 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:45.737478 19150 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.779194 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:45.779955 12676 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
W20260709 12:12:45.785501 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:45.740175 19357 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:45.729804 19213 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:45.656646 18592 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:45.739224 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:45.740108 12673 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
W20260709 12:12:45.788641 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 44612.9us
I20260709 12:12:45.788906 12496 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:45.804023 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:45.804819 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:45.805788 12522 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:12:45.792052 12678 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 16 index: 8. (index mismatch)
I20260709 12:12:45.793720 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:45.788800 19465 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.359s	user 0.000s	sys 0.002s
I20260709 12:12:45.795342 12531 raft_consensus.cc:2393] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 18: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:45.745796 19364 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:45.734653 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 218095us
I20260709 12:12:45.799980 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:45.807864 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:45.794404 12601 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 25: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:45.802522 12646 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:45.809829 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:45.817070 12676 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
W20260709 12:12:45.808642 19473 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.128s	user 0.001s	sys 0.000s
I20260709 12:12:45.788192 19278 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:45.821287 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:45.821715 19278 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:45.822469 19453 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:45.823913 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:45.825212 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:45.825592 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:45.826297 12530 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:45.826702 12597 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 14.
I20260709 12:12:45.826890 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:45.797116 19357 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:45.819604 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:45.823915 19472 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:45.824084 19311 consensus_queue.cc:260] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:45.834868 12496 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:45.860877 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:45.862336 19461 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:12:45.863903 19461 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:45.864684 19461 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:45.835212 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:45.865731 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:45.835642 19482 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:45.829633 19130 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 11 FOLLOWER]: Advancing to term 12
I20260709 12:12:45.866400 12597 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 13.
I20260709 12:12:45.868798 12571 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:45.869608 19364 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:45.870325 19355 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:45.871030 12571 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:45.878422 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 75987.9us
I20260709 12:12:45.838508 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:45.837405 19182 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:45.843282 19278 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.881174 19364 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:45.881843 19278 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:45.881578 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:45.883473 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:45.884433 12680 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:12:45.884912 19364 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:12:45.885473 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:45.840776 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:45.846869 19205 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:45.886761 19205 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:45.887981 19182 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:45.887694 19205 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.853044 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:45.888972 12675 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:45.887989 12680 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:12:45.854206 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:45.858693 12600 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:45.879403 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:45.890597 12526 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:45.891067 12676 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:45.890952 19364 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260709 12:12:45.892357 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:45.890522 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:45.893570 12571 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:45.894095 13333 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 25: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:45.894729 12571 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:45.895886 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:45.898479 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:45.888972 19205 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:45.845698 12522 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:12:45.914740 12646 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:45.919015 19455 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.919767 19455 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.921844 19390 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:45.922465 19390 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:45.922868 19390 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:45.923183 19455 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:45.931493 19373 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:45.932586 12643 leader_election.cc:400] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21.
I20260709 12:12:45.943095 19373 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:45.943888 19373 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.946866 19373 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:45.943095 12643 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:45.949654 19373 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:45.950228 12678 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 21: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:45.951310 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:45.952523 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:45.953140 12678 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:12:45.828536 19273 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority
I20260709 12:12:45.954334 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:45.954890 19389 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:45.955772 19273 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:45.955734 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:45.956462 19273 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:45.957238 19389 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:45.957526 19455 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:12:45.957903 19413 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:45.829702 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:45.959864 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 125100us
I20260709 12:12:45.960628 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:45.814531 19142 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:45.828773 19374 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.567s	user 0.008s	sys 0.003s
I20260709 12:12:45.957238 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:45.975003 12675 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:12:45.948873 12526 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
W20260709 12:12:46.006484 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49344.9us
W20260709 12:12:46.011785 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 195225us
I20260709 12:12:46.014498 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:46.014977 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:46.015169 13337 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:12:46.015774 12675 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:45.809641 19213 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:45.958438 19413 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:45.795136 12421 catalog_manager.cc:5697] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:45.966845 19482 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:46.022663 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:46.006038 19142 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.006136 19472 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:45.957587 19273 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.006276 19045 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:45.958755 19385 consensus_queue.cc:1175] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" responder_term: 18 status { last_received { term: 18 index: 13 } last_committed_idx: 13 last_received_current_leader { term: 18 index: 13 } }
I20260709 12:12:46.030725 19456 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.030922 19142 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:46.031728 19438 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:46.034291 19475 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
W20260709 12:12:46.034735 19474 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.219s	user 0.001s	sys 0.000s
I20260709 12:12:46.035106 19355 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:46.008738 19389 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:46.042418 12526 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20. Ops: []
W20260709 12:12:46.072469 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 58884.8us
I20260709 12:12:46.020326 19455 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:46.063213 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:46.082521 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:46.066903 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 62266.7us
W20260709 12:12:46.068225 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 172405us
I20260709 12:12:46.083539 12678 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:12:46.075592 12643 leader_election.cc:400] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19.
I20260709 12:12:46.075644 19495 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:12:46.086102 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:46.087285 12643 leader_election.cc:403] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Cancelling election due to peer responding with higher term
W20260709 12:12:46.079727 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49355.3us
I20260709 12:12:46.088656 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:46.036331 19413 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:46.091392 12528 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:46.023106 19374 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.062s
I20260709 12:12:46.095953 12496 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:46.073709 19142 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:12:46.090481 19386 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.681s	user 0.011s	sys 0.000s
I20260709 12:12:46.098476 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:46.107446 19213 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:46.110961 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:46.111339 12671 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 20 because replica has last-logged OpId of term: 19 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
I20260709 12:12:46.111160 12523 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:46.086647 19447 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:46.111654 12526 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 14.
I20260709 12:12:46.082664 19389 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:46.073755 19045 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:46.073385 19390 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.118230 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:46.126205 19487 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.107s	user 0.001s	sys 0.000s
I20260709 12:12:46.085477 19456 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:46.114050 19032 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:12:46.132697 19477 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.318s	user 0.000s	sys 0.001s
I20260709 12:12:46.133103 19488 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 12:12:46.114274 19389 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:46.135828 19386 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:46.135906 19182 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:46.139509 19151 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:46.140638 19373 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:46.140745 19304 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:46.117871 19475 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election lost for term 18. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19.
I20260709 12:12:46.114044 19447 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:46.142745 19501 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:46.142980 19485 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:12:46.198997 19447 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:46.225567 19413 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:46.226155 19413 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:46.169520 19384 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.065s
I20260709 12:12:46.222991 18592 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:46.227308 19413 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:46.228453 19482 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:46.228724 19413 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:46.233628 19447 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:46.236156 19447 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:46.143774 19130 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260709 12:12:46.239120 12523 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:46.116492 19213 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:46.242408 19494 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.242936 19213 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.244784 19456 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:46.168040 13337 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
W20260709 12:12:46.170562 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:46.181197 12529 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 10. (term mismatch)
I20260709 12:12:46.247844 12529 pending_rounds.cc:85] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 9
I20260709 12:12:46.248178 12529 pending_rounds.cc:107] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 17.10
I20260709 12:12:46.248473 12529 raft_consensus.cc:2889] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:46.189421 19142 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:46.250345 19142 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:46.250792 19142 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:12:46.251551 19477 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:12:46.253362 19390 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:46.132814 19045 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:12:46.198684 19501 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:46.254797 19501 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:46.257628 19453 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:46.258945 19475 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.259630 19475 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.263311 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2801) took 1210 ms. Trace:
I20260709 12:12:46.263876 12531 rpcz_store.cc:276] 0709 12:12:45.052488 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:45.052634 (+   146us) service_pool.cc:224] Handling call
0709 12:12:46.262998 (+1210364us) spinlock_profiling.cc:243] Waited 727 ms on lock 0x7b54002107d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:46.263194 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1672760576}
I20260709 12:12:46.269039 12527 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:12:46.272442 12646 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:46.275981 19373 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:46.277063 19373 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:46.278501 19503 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:46.279350 19503 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:46.281203 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:46.280763 19503 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.281891 12524 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:12:46.282474 19045 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:46.282914 19045 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:46.283944 19045 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.284751 19494 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.285497 19045 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:46.287786 19494 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:46.294040 19461 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:46.295593 19500 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:46.296088 19500 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:46.299060 12646 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:46.315217 12421 catalog_manager.cc:5697] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 15 to 17, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 17 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:46.321611 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:46.233321 19357 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.325994 12606 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:46.326721 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:46.384275 12523 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:12:46.384981 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 86043.1us
W20260709 12:12:46.334255 19467 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.705s	user 0.001s	sys 0.001s
I20260709 12:12:46.335291 19150 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:46.335574 19464 log.cc:927] Time spent T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.994s	user 0.004s	sys 0.000s
I20260709 12:12:46.336593 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:46.340710 19480 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.571s	user 0.001s	sys 0.000s
I20260709 12:12:46.343794 19389 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.348804 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2881) took 1203 ms. Trace:
I20260709 12:12:46.346774 19311 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:46.350205 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:46.350668 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:46.350961 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:46.194208 19386 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:46.190390 19364 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:46.354588 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:46.297233 19500 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:46.370931 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 295717us
I20260709 12:12:46.392053 19513 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:46.376488 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:46.377461 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:46.379316 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:46.379758 12604 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 11 index: 6. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
I20260709 12:12:46.382401 12600 raft_consensus.cc:2393] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 18: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:46.324644 12601 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 7. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:46.367726 19213 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:46.328629 19444 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:46.398918 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 310311us
I20260709 12:12:46.402227 12603 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:46.388392 19461 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:46.407907 12602 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:46.408144 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 777) took 2127 ms. Trace:
W20260709 12:12:46.408640 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 310208us
W20260709 12:12:46.423259 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:46.456718 12496 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:46.457154 13302 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:46.423574 12597 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:46.424522 12527 rpcz_store.cc:276] 0709 12:12:45.145015 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:45.145157 (+   142us) service_pool.cc:224] Handling call
0709 12:12:46.268678 (+1123521us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:46.269572 (+   894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:46.348472 (+ 78900us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54002107d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:46.348683 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2525455744}
I20260709 12:12:46.397225 19475 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:46.371512 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 286535us
I20260709 12:12:46.459903 19461 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:46.461234 19389 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:46.461647 19509 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:46.489243 19461 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:46.489483 19472 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:46.463146 19032 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.489846 19509 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:12:46.490291 19509 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:46.490242 19032 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.491544 19509 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.492786 19032 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:46.492830 19509 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:46.464237 19265 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.494488 19265 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.496804 19265 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:46.464651 19453 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.497591 19453 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.499836 19453 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:46.465338 19491 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:12:46.456002 12525 rpcz_store.cc:276] 0709 12:12:44.290784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:44.333241 (+ 42457us) service_pool.cc:224] Handling call
0709 12:12:44.398177 (+ 64936us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:44.686491 (+288314us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:12:44.986608 (+300117us) log.cc:844] Serialized 14 byte log entry
0709 12:12:45.234710 (+248102us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b180048cf40. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a 00007f1488a2aa2a
0709 12:12:45.441419 (+206709us) log.cc:844] Serialized 14 byte log entry
0709 12:12:45.441655 (+   236us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:45.444651 (+  2996us) log.cc:844] Serialized 26 byte log entry
0709 12:12:45.444785 (+   134us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:12:45.444812 (+    27us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:45.444873 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:46.333474 (+888601us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:46.407850 (+ 74376us) raft_consensus.cc:1685] finished
0709 12:12:46.407883 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:46.408059 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":48132,"prepare.queue_time_us":394900,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":5510,"raft.queue_time_us":110052,"raft.run_cpu_time_us":354,"raft.run_wall_time_us":15490,"spinlock_wait_cycles":74928384,"thread_start_us":631814,"threads_started":3,"wal-append.queue_time_us":475587}
I20260709 12:12:46.475132 19444 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.500620 19491 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:46.476792 12425 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59006 (request call id 176) took 81794 ms (client timeout 15000 ms). Trace:
I20260709 12:12:46.462721 19451 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.501601 19491 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.502554 19451 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.505546 19451 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:46.507256 19444 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:46.488692 12572 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:46.508585 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 51494.8us
I20260709 12:12:46.509807 13302 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:46.509807 12572 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:46.511134 19298 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:46.511580 19298 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:46.512018 19298 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:46.512642 19500 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:46.512693 19510 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:12:46.513659 19510 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:46.511968 19364 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:46.514683 19364 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:46.515187 19364 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:46.514936 19510 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:46.511678 12572 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:46.522832 19207 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:46.523494 19207 raft_consensus.cc:697] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:46.524583 19207 consensus_queue.cc:237] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.490185 19472 raft_consensus.cc:697] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 14 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:46.532913 19357 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:46.533829 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:46.535111 12601 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 14: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:46.532971 19472 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 13.11, Last appended by leader: 11, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:46.537271 19298 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:46.541157 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:46.406858 12421 catalog_manager.cc:5697] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 10 to 16, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 16 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:12:46.543172 19298 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:46.541877 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:46.545499 13333 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:46.550096 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:12:46.550423 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 464382us
W20260709 12:12:46.550947 12569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.507s	user 0.034s	sys 0.051s
I20260709 12:12:46.551136 12571 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:46.552416 12571 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:46.559460 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 115931us
I20260709 12:12:46.561326 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:46.562059 12643 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:46.567138 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:46.572192 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:46.574780 12524 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 9. (term mismatch)
I20260709 12:12:46.575348 12524 pending_rounds.cc:85] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 8
I20260709 12:12:46.575634 12524 pending_rounds.cc:107] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 16.9
I20260709 12:12:46.575925 12524 raft_consensus.cc:2889] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:46.577701 12672 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:12:46.578775 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:46.579214 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:46.579694 12524 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:46.582683 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 14 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:46.584676 12680 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:12:46.587271 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:46.404497 12494 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:12:46.590529 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:46.595774 12523 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:12:46.599655 12571 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:46.602949 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:46.605680 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:46.606436 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:46.604862 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:46.541224 19142 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.614692 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:46.615721 12526 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:46.563449 19453 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:46.627089 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:46.454903 19513 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:46.633368 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 242674us
I20260709 12:12:46.603781 19304 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:12:46.629489 12572 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:46.621609 19390 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.657979 12673 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 7. Preceding OpId from leader: term: 15 index: 8. (index mismatch)
W20260709 12:12:46.502311 12425 rpcz_store.cc:269] 0709 12:11:24.682679 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:24.683404 (+   725us) service_pool.cc:224] Handling call
0709 12:12:46.476542 (+81793138us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:42.759891 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:42.797032 (+ 37141us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:42.797091 (+    59us) write_op.cc:435] Acquired schema lock
0709 12:12:42.797110 (+    19us) tablet.cc:662] Decoding operations
0709 12:12:42.797825 (+   715us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:42.797939 (+   114us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:42.797975 (+    36us) tablet.cc:685] Acquiring locks for 4 operations
0709 12:12:42.798515 (+   540us) tablet.cc:701] Row locks acquired
0709 12:12:42.798559 (+    44us) write_op.cc:260] PREPARE: finished
0709 12:12:42.798735 (+   176us) write_op.cc:270] Start()
0709 12:12:42.798889 (+   154us) write_op.cc:276] Timestamp: P: 1783599162798668 usec, L: 0
0709 12:12:42.798911 (+    22us) op_driver.cc:348] REPLICATION: starting
0709 12:12:42.810303 (+ 11392us) log.cc:844] Serialized 1702 byte log entry
0709 12:12:43.285460 (+475157us) spinlock_profiling.cc:243] Waited 40.4 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14859160ac 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b
0709 12:12:43.607853 (+322393us) op_driver.cc:464] REPLICATION: finished
0709 12:12:44.031957 (+424104us) write_op.cc:301] APPLY: starting
0709 12:12:44.068315 (+ 36358us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:44.460598 (+392283us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:44.460658 (+    60us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:44.668581 (+207923us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:44.668931 (+   350us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=8,key_file_lookups=8,delta_file_lookups=8,mrs_lookups=0
0709 12:12:44.669009 (+    78us) write_op.cc:312] APPLY: finished
0709 12:12:44.669438 (+   429us) log.cc:844] Serialized 49 byte log entry
0709 12:12:44.789269 (+119831us) spinlock_profiling.cc:243] Waited 8.14 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14859160ac 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b
0709 12:12:45.045239 (+255970us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:45.086435 (+ 41196us) write_op.cc:454] Released schema lock
0709 12:12:45.086513 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:45.087160 (+   647us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:45.093100 (+  5940us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1412133,"reported_tablets":53,"spinlock_wait_cycles":18176,"child_traces":[["op",{"apply.queue_time_us":350979,"apply.run_cpu_time_us":59137,"apply.run_wall_time_us":1087503,"cfile_cache_hit":18,"cfile_cache_hit_bytes":20823,"num_ops":4,"prepare.queue_time_us":94414,"prepare.run_cpu_time_us":33412,"prepare.run_wall_time_us":842660,"raft.queue_time_us":31292,"raft.run_cpu_time_us":15630,"raft.run_wall_time_us":250199,"replication_time_us":808712,"spinlock_wait_cycles":111689472,"thread_start_us":1378130,"threads_started":5,"wal-append.queue_time_us":1142368,"wal-append.run_cpu_time_us":20269,"wal-append.run_wall_time_us":2077133}]]}
I20260709 12:12:46.637553 12494 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:46.669629 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 70080.2us
I20260709 12:12:46.670435 12571 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:46.675686 12526 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15.
I20260709 12:12:46.676589 12522 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:46.632422 19357 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:12:46.641883 19491 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:46.544330 12421 catalog_manager.cc:5697] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 14 to 18, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 18 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:46.688784 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:46.615793 19461 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:46.693804 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:46.697165 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:46.697279 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 67911.4us
I20260709 12:12:46.697651 12671 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:46.679025 19032 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:46.607075 19386 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.640621 19374 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:12:46.702346 12672 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:46.679111 19355 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:46.679782 19373 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:46.741678 12526 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:46.742601 12526 raft_consensus.cc:606] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:46.641199 19453 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.745249 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:46.701318 19491 raft_consensus.cc:697] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:46.722502 19438 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260709 12:12:46.756178 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:46.723821 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:46.699677 19150 consensus_queue.cc:1059] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:12:46.727602 19506 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:46.731161 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:46.732293 19142 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:46.733088 12602 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:46.737941 19304 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:46.725523 19207 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:46.699870 18723 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260709 12:12:46.718387 19443 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:46.736392 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:46.794440 12605 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:46.734257 12601 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:12:46.737442 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:46.738251 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:46.796785 12604 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
W20260709 12:12:46.766330 19521 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.072s	user 0.000s	sys 0.001s
I20260709 12:12:46.749511 12494 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:46.750025 19451 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:46.682502 19390 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:46.718297 13337 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:46.736600 19496 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.100s
I20260709 12:12:46.757771 12673 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
I20260709 12:12:46.733560 19505 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:46.738492 19032 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:46.789108 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:46.823174 12671 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:46.793352 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:46.824455 12606 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 12.
W20260709 12:12:46.748958 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:46.747967 19515 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:46.761083 19438 consensus_queue.cc:1243] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 11.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260709 12:12:46.766196 19506 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:46.798947 19304 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.787573 19514 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:46.817585 19451 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.827636 19486 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:46.823688 19505 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:46.845305 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:46.842439 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 161779us
I20260709 12:12:46.849911 12599 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:12:46.827965 19515 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:46.815846 19512 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.022s
W20260709 12:12:46.852412 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 182051us
I20260709 12:12:46.833878 19486 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:46.855252 12526 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:46.856956 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:46.822220 19443 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.024s
W20260709 12:12:46.859730 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 224209us
I20260709 12:12:46.860600 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:46.830893 19506 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:46.863816 12597 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:46.854537 12605 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:12:46.859184 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:46.883677 12646 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:46.886559 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:46.887148 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:46.880494 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:46.888914 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:46.889770 12605 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:46.880836 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:46.863763 12606 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20. Ops: [17.8-19.9]
I20260709 12:12:46.884783 12601 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:46.891240 12599 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
W20260709 12:12:46.880529 19517 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
W20260709 12:12:46.900038 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 150626us
I20260709 12:12:46.853179 19505 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:46.858139 19515 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:46.837028 19514 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:46.882781 19486 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:46.901618 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:46.893695 19495 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:12:46.896649 19513 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:47.012027 19514 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:47.028228 19486 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.032183 19515 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:46.844084 19491 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:47.066462 19522 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.318s	user 0.001s	sys 0.000s
I20260709 12:12:47.068784 19273 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.069705 19273 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:47.076416 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 192821us
W20260709 12:12:46.996366 19518 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.281s	user 0.000s	sys 0.001s
I20260709 12:12:47.079844 19508 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:47.080451 19508 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:47.081714 19527 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:47.082288 19130 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:47.082944 19130 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:47.083458 19390 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:47.084040 19390 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:47.084396 19390 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:46.983968 19530 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:12:46.997537 19032 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.8, Last appended by leader: 8, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.086396 19130 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:47.087443 19487 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:12:47.088168 19495 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:47.089733 19495 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:47.090957 19530 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:47.091475 19530 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.859122 12421 catalog_manager.cc:5697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 20 to 23, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 23 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:47.093592 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:47.095515 12601 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:12:47.098707 19364 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.583s	user 0.009s	sys 0.003s
I20260709 12:12:47.099308 19364 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:46.982743 19461 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
W20260709 12:12:46.985383 19045 consensus_meta.cc:287] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.609s	user 0.014s	sys 0.000s
I20260709 12:12:47.084744 19390 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:47.081387 19508 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.201931 19515 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:47.143594 12524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3a68955866d543408acca9e959ad1b26"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
W20260709 12:12:47.160444 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 273953us
I20260709 12:12:47.236788 12524 raft_consensus.cc:606] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:47.237353 12524 raft_consensus.cc:614] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:47.201128 19509 raft_consensus.cc:3055] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:47.199025 19130 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:47.160578 19386 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:47.205204 19364 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:47.206274 12602 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:47.201489 19530 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:47.207556 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 133080us
I20260709 12:12:47.209631 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:47.200543 19475 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:47.200138 19273 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:47.215394 12679 raft_consensus.cc:2410] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 20 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
W20260709 12:12:47.173406 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 271865us
I20260709 12:12:47.093971 12421 catalog_manager.cc:5697] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 15 to 17, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 17 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:47.201401 19453 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:47.239547 19501 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.984s	user 0.013s	sys 0.003s
I20260709 12:12:47.230496 12597 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:12:47.237143 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:47.200704 19045 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:12:47.228211 19311 consensus_meta.cc:287] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.785s	user 0.015s	sys 0.000s
W20260709 12:12:47.223232 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 410194us
I20260709 12:12:47.262138 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:47.263075 12678 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 14.
I20260709 12:12:47.273705 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:47.243968 19509 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:47.280179 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:47.281159 12678 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 25: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:47.277545 12598 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 14.
I20260709 12:12:47.249560 19461 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:47.251941 19451 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:47.296171 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:47.297183 12676 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:12:47.260879 19304 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:47.312006 12678 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 15 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:12:47.279589 19273 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:47.318796 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:47.252640 19130 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:47.291975 19525 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.195s	user 0.000s	sys 0.004s
I20260709 12:12:47.322948 12528 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:47.335660 12528 raft_consensus.cc:606] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:47.326598 12524 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 20: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:47.286731 19364 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:47.338289 19364 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:47.339782 12496 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:47.340857 12496 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:47.341679 12524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:47.342346 19539 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:12:47.342551 12524 raft_consensus.cc:606] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:47.342731 12421 catalog_manager.cc:5697] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:12:47.341925 12496 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:47.283002 19357 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.325407 12646 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:47.351217 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 448835us
I20260709 12:12:47.263154 19475 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.263046 19501 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.272053 19045 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:47.362397 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:47.364195 12677 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:47.366196 12496 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:47.372149 12524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:47.375834 12525 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: [13.9-13.9]
I20260709 12:12:47.295147 19461 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:47.373019 12524 raft_consensus.cc:606] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:47.358749 19532 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.383879 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:47.383744 19500 consensus_meta.cc:287] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.870s	user 0.007s	sys 0.007s
I20260709 12:12:47.356480 19357 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:47.400718 18993 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.401309 18993 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.402182 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:47.403168 12528 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:47.354033 19526 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:47.388051 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:47.417055 12676 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 20.
I20260709 12:12:47.358875 19453 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:47.390794 12604 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
W20260709 12:12:47.398864 19506 consensus_meta.cc:287] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.502s	user 0.010s	sys 0.007s
I20260709 12:12:47.424779 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:47.384572 19543 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260709 12:12:47.427111 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:47.399632 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:47.428010 12531 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:47.354609 19509 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.430910 19509 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:47.358024 19389 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:47.413679 19532 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.435226 19389 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:47.435727 19389 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:47.437141 19532 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:47.418565 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:47.438926 12526 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
W20260709 12:12:47.419909 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:47.399338 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:47.440181 19500 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:47.440379 18993 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:47.440788 12674 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:47.443207 19475 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:47.443485 19364 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:47.440812 19500 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.447149 19500 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:47.452791 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:47.469069 12531 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:12:47.469513 19413 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.479189 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:47.482182 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:47.484297 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:47.485152 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:47.404520 19536 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:47.489934 13304 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:47.491115 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:47.488622 12646 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:47.416776 19526 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:47.499737 12646 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:47.425290 19453 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:47.504516 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:47.470703 19543 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:47.499570 19536 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:47.507606 19385 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:47.533046 19385 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:47.533623 19385 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:47.506951 19389 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.540342 19389 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:47.541195 12528 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:12:47.542794 19540 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.155s	user 0.001s	sys 0.000s
I20260709 12:12:47.542394 12421 catalog_manager.cc:5697] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 13 to 15. New cstate: current_term: 15 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:12:47.543936 19538 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:12:47.544332 12421 catalog_manager.cc:5697] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:12:47.470039 19413 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:47.548182 19413 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:47.505980 18993 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:12:47.550221 19536 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:47.546090 12675 raft_consensus.cc:2410] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:12:47.509673 19374 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:12:47.511530 19455 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:47.552830 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:47.510281 19453 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:47.508823 19526 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:47.474570 19506 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.512476 19542 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:47.513947 19543 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 12 FOLLOWER]: Advancing to term 13
W20260709 12:12:47.563910 19505 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.567s	user 0.009s	sys 0.003s
W20260709 12:12:47.571445 12598 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 685) took 1820 ms (client timeout 1735 ms). Trace:
I20260709 12:12:47.577137 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:47.577401 19501 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:47.577162 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:47.525063 19298 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:12:47.578631 19130 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:47.577915 12523 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:47.562903 19455 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:47.579589 19455 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:47.540377 19559 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:47.581941 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:47.582588 12678 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:47.582739 19390 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:47.585070 19390 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:47.577404 19045 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:47.566995 19374 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:47.563455 19472 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:47.574635 12598 rpcz_store.cc:269] 0709 12:12:45.751057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:45.758265 (+  7208us) service_pool.cc:224] Handling call
0709 12:12:47.571252 (+1812987us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54002e54d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:47.571353 (+   101us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3491703040}
I20260709 12:12:47.586688 12672 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:47.591789 12678 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:47.593906 12600 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 18: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:47.595461 12677 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15.
I20260709 12:12:47.597805 12646 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:47.573127 19461 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:47.600020 19472 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:47.607802 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:47.609477 19558 raft_consensus.cc:993] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: : Instructing follower 193a40da6e364ead8ab561e14c3cc8d1 to start an election
I20260709 12:12:47.610088 19558 raft_consensus.cc:1081] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Signalling peer 193a40da6e364ead8ab561e14c3cc8d1 to start an election
I20260709 12:12:47.610235 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:47.611119 13334 raft_consensus.cc:2393] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 19: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:47.612114 12496 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:47.582365 19559 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:47.618670 19559 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:47.602829 19506 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:47.573604 19542 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.621443 19542 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:47.594101 19505 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:47.622351 19541 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:47.622879 19541 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:47.622638 19374 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:47.623317 19541 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:47.624497 19505 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:47.607379 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:47.631531 19461 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:47.631736 12496 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:47.632324 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:47.600934 19501 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.609776 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:47.632961 12531 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
I20260709 12:12:47.637108 19265 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:12:47.637873 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:47.641004 12496 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:47.642037 18723 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:47.642609 18723 raft_consensus.cc:697] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:47.643759 19558 raft_consensus.cc:993] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201: : Instructing follower 193a40da6e364ead8ab561e14c3cc8d1 to start an election
I20260709 12:12:47.643478 18723 consensus_queue.cc:237] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.644400 19558 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.644599 19032 raft_consensus.cc:1081] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Signalling peer 193a40da6e364ead8ab561e14c3cc8d1 to start an election
I20260709 12:12:47.644887 19558 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.647285 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:47.647650 19558 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:47.649550 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:47.650327 13338 raft_consensus.cc:2368] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15.
W20260709 12:12:47.651620 19560 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.074s	user 0.003s	sys 0.000s
I20260709 12:12:47.652925 12643 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:47.653644 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:47.654385 12643 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:47.610037 19472 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:47.666568 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:47.667681 12677 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:47.632370 19461 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:47.663266 12569 leader_election.cc:400] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15.
I20260709 12:12:47.675899 12569 leader_election.cc:403] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Cancelling election due to peer responding with higher term
I20260709 12:12:47.679908 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 3296) took 1113 ms. Trace:
I20260709 12:12:47.659621 19045 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:47.680419 12679 rpcz_store.cc:276] 0709 12:12:46.562079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:46.581514 (+ 19435us) service_pool.cc:224] Handling call
0709 12:12:47.669567 (+1088053us) spinlock_profiling.cc:243] Waited 513 ms on lock 0x7b54000aafd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:47.675084 (+  5517us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1179486080}
I20260709 12:12:47.664072 19481 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:12:47.710213 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 47100.6us
I20260709 12:12:47.654676 19501 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:47.708796 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 72812.9us
I20260709 12:12:47.677055 19461 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:47.715492 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:12:47.701133 12674 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:51308
I20260709 12:12:47.716480 12674 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:47.716840 12674 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:47.717689 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:47.718694 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:47.704824 19481 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:47.567836 12421 catalog_manager.cc:5697] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 13 to 21, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 21 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:12:47.709354 19032 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.663291 12672 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:12:47.717758 19142 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:47.786360 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:47.787036 13334 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
I20260709 12:12:47.788651 12569 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:47.789646 19542 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:47.790078 19542 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:47.790434 19542 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:47.732383 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:47.801101 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:47.733810 19447 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:12:47.734743 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 368369us
I20260709 12:12:47.803716 12494 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:47.735669 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:47.735307 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:47.736721 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:47.740007 12599 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:47.740950 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:47.742830 19541 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.744109 12605 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 14. Current term is 15. Ops: []
I20260709 12:12:47.809219 19486 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:47.809688 19541 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:47.818832 19541 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:12:47.819633 19273 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:47.754186 19355 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.759039 19526 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.760006 19501 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:47.765318 19536 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.771591 12674 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.773581 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:47.828662 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:47.775182 19357 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:47.776971 19503 raft_consensus.cc:3055] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:47.779407 12421 catalog_manager.cc:5697] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 10 to 13. New cstate: current_term: 13 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:12:47.724680 19481 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:47.784704 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3218) took 1049 ms. Trace:
I20260709 12:12:47.784467 19453 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.727689 12677 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
I20260709 12:12:47.803617 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:12:47.847532 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 44256.3us
I20260709 12:12:47.737709 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 14 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:47.746289 19130 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:47.810774 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2822) took 1256 ms. Trace:
I20260709 12:12:47.858693 12525 rpcz_store.cc:276] 0709 12:12:46.554765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:46.554932 (+   167us) service_pool.cc:224] Handling call
0709 12:12:46.555393 (+   461us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:47.378833 (+823440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:47.810483 (+431650us) spinlock_profiling.cc:243] Waited 820 ms on lock 0x7b54003cb4d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:47.810711 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1886057216}
I20260709 12:12:47.810388 19447 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:47.815634 19438 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:47.816493 12494 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:47.718452 19532 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.817476 19482 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:47.782100 19385 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:47.841159 12671 raft_consensus.cc:2368] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20.
I20260709 12:12:47.822710 19355 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:47.865926 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49525.5us
I20260709 12:12:47.843994 12674 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:47.845175 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:47.867724 12680 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:47.795522 19032 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.845477 12600 rpcz_store.cc:276] 0709 12:12:46.735380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:46.737207 (+  1827us) service_pool.cc:224] Handling call
0709 12:12:47.784479 (+1047272us) spinlock_profiling.cc:243] Waited 797 ms on lock 0x7b54002e63d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:47.784640 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1833801472}
I20260709 12:12:47.859849 13302 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:47.862275 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:47.862267 12571 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:47.869946 12677 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:47.837462 19503 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:47.846226 18993 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:47.866617 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 362211us
I20260709 12:12:47.877287 12571 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:47.877761 12600 pending_rounds.cc:85] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 5
I20260709 12:12:47.871042 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:47.825853 19543 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.878094 12600 pending_rounds.cc:107] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 11.6
I20260709 12:12:47.884766 12600 raft_consensus.cc:2889] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:47.885720 13302 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:47.864001 19505 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:12:47.887035 13338 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:47.864071 19527 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:47.890412 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:47.885545 13304 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:47.893069 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:47.871157 19541 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:47.910495 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:47.911064 19541 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:47.911578 19541 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:47.911823 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:47.915186 12525 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 17. Current term is 18. Ops: [17.9-17.9]
I20260709 12:12:47.915386 19142 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election lost for term 14. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15.
I20260709 12:12:47.887001 18993 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:47.916638 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:47.892562 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:47.918421 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:47.918723 12605 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:47.895653 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:47.919508 12522 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:47.919625 12675 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 18: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:47.835630 19455 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:47.920804 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:47.921486 12523 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:12:47.895637 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:47.922945 12601 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:47.899041 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:47.899187 19032 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:47.911597 12671 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 21: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:47.898228 13330 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 15. (index mismatch)
I20260709 12:12:47.864670 19413 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.024s
W20260709 12:12:47.899809 19554 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.322s	user 0.001s	sys 0.000s
I20260709 12:12:47.903211 19453 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:47.904345 19472 consensus_queue.cc:1059] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:47.886796 19357 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:47.941824 12525 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:12:47.941934 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:47.943130 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:47.906726 19527 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:47.907657 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:47.909322 19447 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:47.932566 18993 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:47.901386 19385 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:47.928431 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:47.929334 12599 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 18: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:47.962483 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:47.931429 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:47.964120 12529 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:12:47.921973 12496 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:47.932747 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:47.886916 19514 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:12:47.888152 19506 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:47.906337 19355 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:47.939170 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:47.969484 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 44010.1us
I20260709 12:12:47.970556 12522 raft_consensus.cc:2368] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15.
I20260709 12:12:47.970826 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:47.971673 12599 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:47.971858 12494 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:47.867852 19532 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:47.961367 19508 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:47.973336 19508 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:47.973831 19508 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:47.974429 12672 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
I20260709 12:12:47.975190 19532 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:47.916702 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:47.983757 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:47.955456 12524 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
W20260709 12:12:47.987164 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 101482us
I20260709 12:12:47.987709 12602 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15.
I20260709 12:12:47.953249 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:47.898255 19558 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:47.941563 19142 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:47.998121 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:47.999233 12677 raft_consensus.cc:2368] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15.
I20260709 12:12:47.967801 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:47.968094 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46159.8us
I20260709 12:12:47.968961 19455 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:47.972642 19453 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:12:47.957825 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50267.9us
I20260709 12:12:48.000375 12671 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:48.002051 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 12 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:47.972507 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:48.000939 12496 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:47.953197 19265 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.011368 12496 leader_election.cc:400] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20.
I20260709 12:12:48.014415 12496 leader_election.cc:403] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:48.012796 12643 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:47.973116 19298 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:48.017637 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:47.969494 19555 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:48.019176 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:47.944345 19557 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.072s
I20260709 12:12:47.996404 19506 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.036541 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:47.999783 19514 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:48.038069 12526 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:12:48.021407 12675 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:12:48.043529 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: true dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:48.044529 12524 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:48.039928 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: true dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:48.040112 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:48.040553 12674 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:51308
I20260709 12:12:48.039177 12528 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:48.046311 12599 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:48.046376 12674 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:12:48.046851 12674 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:48.048760 19514 raft_consensus.cc:697] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 15 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:48.050088 19514 consensus_queue.cc:237] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 13.9, Last appended by leader: 9, Current term: 15, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:48.056066 12674 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:48.053112 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:48.052030 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:48.009802 19355 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.059352 12528 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:48.010178 19558 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:48.009181 19453 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:48.045218 12524 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:48.065284 12678 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: []
W20260709 12:12:48.065708 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 46203us
I20260709 12:12:48.019675 19444 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:48.082681 12572 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:48.021641 19501 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:12:48.008057 19142 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:48.021175 19357 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.089555 12674 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:48.078102 12605 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:48.021296 19045 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:12:48.084609 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 199171us
I20260709 12:12:48.087919 19445 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:12:48.091127 12494 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:48.094727 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:48.095310 12676 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:12:48.022066 18993 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.058s
I20260709 12:12:48.090385 12524 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:48.022632 19265 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:48.099059 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:48.102490 12674 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 17 index: 9. (index mismatch)
I20260709 12:12:48.103960 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:48.200014 12601 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:48.081604 19453 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:48.223701 12524 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:48.232489 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:48.235822 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:48.240013 12601 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:48.060847 19506 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:48.244793 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 153749us
I20260709 12:12:48.083184 19444 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:48.246862 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:48.253780 12672 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
I20260709 12:12:48.071674 19355 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:48.251173 12525 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:48.093835 19491 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:47.979553 12421 catalog_manager.cc:5697] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 9 to 15, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 15 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:12:48.089217 19472 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:48.084358 19555 raft_consensus.cc:697] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:48.085381 19447 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.167s
I20260709 12:12:48.102885 19536 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:48.265517 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:48.267123 12605 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:48.239303 19032 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:48.098837 19494 consensus_queue.cc:1059] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.157s
I20260709 12:12:48.080128 19559 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:48.260095 19552 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.147s	user 0.001s	sys 0.000s
I20260709 12:12:48.200863 19045 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:48.101533 19357 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:48.256219 19444 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:48.302531 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:48.265040 19482 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:48.266237 19512 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.317385 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:48.318986 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:48.333925 19512 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:48.334940 12673 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
I20260709 12:12:48.335248 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:48.336133 12679 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:48.336748 12673 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 13. Current term is 14. Ops: []
I20260709 12:12:48.337086 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:48.337785 13335 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 13.
I20260709 12:12:48.340569 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:48.340416 12421 catalog_manager.cc:5697] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 11 to 13. New cstate: current_term: 13 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:12:48.341658 12602 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:12:48.352962 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:48.289629 19566 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:48.353690 19566 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:48.354826 19566 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:48.356963 19566 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:48.302759 19555 consensus_queue.cc:237] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 12.8, Last appended by leader: 8, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:48.290354 19558 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:48.324432 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:48.362941 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:48.363317 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:48.364009 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:48.364722 12602 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:48.365114 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:48.365720 12598 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:12:48.325203 19481 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:48.326299 19482 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:48.366938 12602 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 7. Preceding OpId from leader: term: 20 index: 9. (index mismatch)
I20260709 12:12:48.366998 19482 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:48.367612 19265 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:48.303457 19500 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:48.370941 19500 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:48.373327 19500 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:48.306809 19543 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:48.374696 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:48.375579 12601 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:12:48.259812 19524 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 17.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.318s
I20260709 12:12:48.377650 13336 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:48.384454 19481 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:48.333004 19506 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:48.348321 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:48.406737 12606 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:12:48.320803 19569 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.313s	user 0.001s	sys 0.000s
I20260709 12:12:48.308087 19576 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.410887 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:48.392299 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:48.393602 13333 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:12:48.372062 12599 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:48.416407 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:48.417207 12523 raft_consensus.cc:3055] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:48.417584 12523 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:48.420758 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:48.421777 12672 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:48.423385 12599 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:48.428637 12598 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:12:48.403728 12679 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
I20260709 12:12:48.405285 12528 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:12:48.407548 12671 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:48.432955 12673 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:48.248008 19542 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:48.402940 12680 consensus_meta.cc:287] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.533s	user 0.012s	sys 0.000s
I20260709 12:12:48.434070 12680 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:48.439275 12528 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:12:48.443034 12678 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:48.457899 12671 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
W20260709 12:12:48.467444 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:48.275527 19472 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:48.456722 12523 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:48.516901 12523 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:48.336817 19512 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:48.414415 19500 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.519973 19527 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:48.520231 19541 consensus_meta.cc:287] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.608s	user 0.014s	sys 0.000s
I20260709 12:12:48.447829 19576 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:48.292148 19491 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:48.523068 19472 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:48.523639 12599 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:48.533005 19542 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:48.507316 19524 raft_consensus.cc:3055] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:48.578146 19524 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:48.570696 19500 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:48.578706 19142 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:48.538578 12597 raft_consensus.cc:2368] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 14. Current term is 15.
I20260709 12:12:48.540907 12646 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:48.541487 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:48.523138 12602 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:12:48.518671 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:12:48.581337 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 62735.4us
I20260709 12:12:48.402535 19032 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:48.546967 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 322774us
I20260709 12:12:48.563221 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: true dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:48.585572 13336 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:48.564491 12679 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:48.427599 19390 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:48.571564 12528 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 12:12:48.577482 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:48.545404 12524 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:12:48.586896 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:48.588989 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:48.589381 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:48.583443 12677 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.004s	sys 0.019s
W20260709 12:12:48.594903 12677 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.005s	sys 0.020s
W20260709 12:12:48.589601 19514 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.007s	sys 0.020s
I20260709 12:12:48.602665 12523 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
W20260709 12:12:48.610355 12675 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.521s	user 0.012s	sys 0.016s
W20260709 12:12:48.619887 12675 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.013s	sys 0.017s
W20260709 12:12:48.646698 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:48.622655 13304 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:48.569751 19541 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:48.709838 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 87264.5us
W20260709 12:12:48.613479 19514 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.556s	user 0.007s	sys 0.020s
I20260709 12:12:48.514248 19503 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:48.615882 19461 consensus_meta.cc:287] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.886s	user 0.011s	sys 0.000s
I20260709 12:12:48.599567 19390 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:48.611375 19455 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:48.727089 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 14 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:48.726357 19481 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:48.579099 19524 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:48.742257 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:48.750734 12529 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:48.724254 19581 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:48.733197 19481 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:48.754383 19581 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:48.754952 19581 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:48.779107 13336 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
W20260709 12:12:48.739077 19574 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.173s	user 0.000s	sys 0.000s
I20260709 12:12:48.748420 19390 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:48.786595 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2897) took 1065 ms. Trace:
I20260709 12:12:48.742673 19032 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:48.745137 19491 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:48.793649 12678 rpcz_store.cc:276] 0709 12:12:47.721474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.074306 (+352832us) service_pool.cc:224] Handling call
0709 12:12:48.442598 (+368292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:48.443775 (+  1177us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:48.786224 (+342449us) spinlock_profiling.cc:243] Waited 368 ms on lock 0x7b5400013e50. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:48.786491 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":845912448}
I20260709 12:12:48.755797 19576 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:48.732007 19579 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:12:48.788889 19541 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:48.788786 19357 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:48.791496 19508 consensus_meta.cc:287] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.817s	user 0.011s	sys 0.003s
I20260709 12:12:48.747650 19555 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.741035 19559 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:48.730666 19273 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.058s
W20260709 12:12:48.808763 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 294817us
W20260709 12:12:48.810736 12676 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.559s	user 0.025s	sys 0.056s
I20260709 12:12:48.732228 19461 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:48.794353 19578 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.834573 19491 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:12:48.828395 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 244596us
I20260709 12:12:48.822834 19355 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:48.876044 19461 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:48.876199 19355 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:48.876681 19461 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.876839 19355 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:48.877166 19461 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:48.877770 19453 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.625s	user 0.006s	sys 0.003s
I20260709 12:12:48.755744 19481 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:48.835772 13338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 762) took 1050 ms. Trace:
I20260709 12:12:48.805178 19579 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:48.843932 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:48.843972 13302 leader_election.cc:336] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.735s (SENT)
I20260709 12:12:48.808424 19576 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:12:48.867763 12676 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.025s	sys 0.056s
I20260709 12:12:48.829761 19032 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:12:48.873546 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:48.884337 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:48.830147 19508 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:48.880834 13338 rpcz_store.cc:276] 0709 12:12:47.785864 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:47.795423 (+  9559us) service_pool.cc:224] Handling call
0709 12:12:47.795774 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:47.888285 (+ 92511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:48.835542 (+947257us) spinlock_profiling.cc:243] Waited 91 ms on lock 0x7b54004e84d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:48.835716 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":209385728}
I20260709 12:12:48.885717 13302 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:48.899353 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:48.908466 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:48.909332 12530 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
W20260709 12:12:48.912434 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:48.914184 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 70266.9us
I20260709 12:12:48.917860 12606 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:12:48.920365 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:48.833807 19555 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:48.847774 19583 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 12:12:49.023380 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:49.027192 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:48.891934 19579 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:49.020212 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 127868us
I20260709 12:12:49.072780 12646 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:48.881570 19491 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:48.873806 19578 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:48.920382 19580 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260709 12:12:49.028971 19482 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.885259 19453 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:49.044486 19385 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:49.021245 19582 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:48.910991 19032 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:49.106714 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:49.070075 19596 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:48.918618 19558 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.180s
I20260709 12:12:49.120394 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:49.121482 12528 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:12:49.112332 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:49.125324 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2895) took 1400 ms. Trace:
I20260709 12:12:49.133143 12522 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:49.126108 12675 rpcz_store.cc:276] 0709 12:12:47.714914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.050365 (+335451us) service_pool.cc:224] Handling call
0709 12:12:48.050737 (+   372us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:48.050971 (+   234us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:12:48.626530 (+575559us) log.cc:844] Serialized 14 byte log entry
0709 12:12:49.114803 (+488273us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:49.114897 (+    94us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:49.114947 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:49.115013 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:49.115148 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8801,"prepare.queue_time_us":701466,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":52,"raft.queue_time_us":115933,"raft.run_cpu_time_us":253,"raft.run_wall_time_us":26371,"thread_start_us":1025308,"threads_started":3,"wal-append.queue_time_us":490045}
W20260709 12:12:49.127573 19444 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.729s	user 0.011s	sys 0.003s
W20260709 12:12:49.156610 19501 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.059s	user 0.006s	sys 0.007s
I20260709 12:12:49.157478 19501 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20.
I20260709 12:12:49.112314 19385 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:49.160374 19385 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:49.162139 19590 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.255s	user 0.000s	sys 0.000s
I20260709 12:12:49.163345 13334 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:49.164319 13334 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 867) took 1279 ms. Trace:
I20260709 12:12:49.164757 13334 rpcz_store.cc:276] 0709 12:12:47.884881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:47.885038 (+   157us) service_pool.cc:224] Handling call
0709 12:12:49.162757 (+1277719us) spinlock_profiling.cc:243] Waited 25.9 ms on lock 0x7b5800163298. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:12:49.163070 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:49.163951 (+   881us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:49.164214 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59628288}
I20260709 12:12:49.168491 19570 raft_consensus.cc:3055] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:49.168985 19570 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:49.171542 19578 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:49.174005 19558 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:49.174630 19558 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:49.176904 19558 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:49.179245 19453 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:49.117003 19582 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:49.195379 19355 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:49.193995 12526 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
W20260709 12:12:49.199714 13333 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 667) took 1593 ms (client timeout 1809 ms). Trace:
W20260709 12:12:49.200214 13333 rpcz_store.cc:269] 0709 12:12:47.605953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:47.609504 (+  3551us) service_pool.cc:224] Handling call
0709 12:12:49.199493 (+1589989us) spinlock_profiling.cc:243] Waited 760 ms on lock 0x7b54001b49d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:49.199617 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1748020992}
W20260709 12:12:49.202992 12597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 698) took 1465 ms (client timeout 1735 ms). Trace:
W20260709 12:12:49.203509 12597 rpcz_store.cc:269] 0709 12:12:47.737368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:47.737490 (+   122us) service_pool.cc:224] Handling call
0709 12:12:49.202760 (+1465270us) spinlock_profiling.cc:243] Waited 688 ms on lock 0x7b54002e54d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:49.202929 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1582300928}
I20260709 12:12:49.207242 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2904) took 1447 ms. Trace:
I20260709 12:12:49.131067 19596 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:49.207638 12674 rpcz_store.cc:276] 0709 12:12:47.759448 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.201834 (+442386us) service_pool.cc:224] Handling call
0709 12:12:48.202175 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:48.410273 (+208098us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:12:48.777952 (+367679us) log.cc:844] Serialized 14 byte log entry
0709 12:12:48.882907 (+104955us) log.cc:844] Serialized 14 byte log entry
0709 12:12:48.883249 (+   342us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:48.883292 (+    43us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:48.883340 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:49.164838 (+281498us) spinlock_profiling.cc:243] Waited 7.27 ms on lock 0x7b1800406300. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:12:49.206917 (+ 42079us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b54001e2154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:49.206973 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:49.207163 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":345389,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":64,"raft.queue_time_us":368158,"raft.run_cpu_time_us":412,"raft.run_wall_time_us":39516,"spinlock_wait_cycles":494691840,"thread_start_us":374609,"threads_started":2,"wal-append.queue_time_us":326021}
I20260709 12:12:49.183215 19444 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:49.211609 19444 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:49.212670 19304 consensus_queue.cc:1059] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:49.214182 19304 raft_consensus.cc:3055] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:49.207011 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:49.247771 12527 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 17: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:49.252305 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:49.253129 13333 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 13 FOLLOWER]: Advancing to term 15
I20260709 12:12:49.214643 19304 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:49.113775 19482 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:49.319391 12528 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 20.
I20260709 12:12:49.320367 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:49.321221 13334 raft_consensus.cc:2410] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 20 because replica has last-logged OpId of term: 18 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 12:12:49.311275 19576 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:12:49.330983 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 222154us
I20260709 12:12:49.331741 12527 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
W20260709 12:12:49.333501 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 400713us
I20260709 12:12:49.252403 19385 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:49.308342 19566 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:12:49.338162 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:49.341506 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 949258us
I20260709 12:12:49.315948 19461 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:49.370762 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:49.319257 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:49.375087 12524 raft_consensus.cc:2368] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 20. Current term is 21.
I20260709 12:12:49.404052 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:49.411918 12606 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:12:49.412590 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:49.377820 19150 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:49.424897 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:49.511649 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3224) took 1635 ms. Trace:
I20260709 12:12:49.512312 12597 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:49.425386 19615 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:49.325194 19364 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.079s
W20260709 12:12:49.424029 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 1.23128e+06us
I20260709 12:12:49.372928 19527 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:12:49.427500 13305 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:49.429797 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 59132us
I20260709 12:12:49.436496 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:49.521320 12602 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:12:49.521509 12571 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:49.442162 19526 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:49.435948 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:49.526932 12599 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:49.341243 19445 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:49.426555 19357 raft_consensus.cc:3055] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:49.422417 19529 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:49.373201 19385 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:49.427609 19508 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:49.382449 19566 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:49.409065 19304 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 17.9, Last appended by leader: 9, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:49.474591 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:49.545930 19617 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:49.546926 19617 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:49.547710 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:49.549165 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:49.475735 19625 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:12:49.476091 19390 consensus_meta.cc:287] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.674s	user 0.013s	sys 0.002s
I20260709 12:12:49.550642 19390 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:49.550880 19625 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:49.552537 19624 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:49.553323 19390 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:49.476861 19555 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:49.553414 19624 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:49.554687 12672 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 20: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:12:49.487255 19605 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
W20260709 12:12:49.488296 19608 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:12:49.483690 19491 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:49.489477 19455 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:49.484041 19596 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:49.490213 19505 consensus_queue.cc:1059] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:12:49.490895 19032 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:49.557466 19455 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:49.456274 19581 consensus_meta.cc:287] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.701s	user 0.018s	sys 0.000s
I20260709 12:12:49.558411 19505 raft_consensus.cc:3055] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:12:49.516990 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:49.169816 19570 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:49.558979 19505 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:49.518299 12572 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:49.559548 19570 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:49.560587 12572 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:49.525072 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 97692.4us
I20260709 12:12:49.562067 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:49.512207 12600 rpcz_store.cc:276] 0709 12:12:47.876322 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:47.877197 (+   875us) service_pool.cc:224] Handling call
0709 12:12:47.877537 (+   340us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:12:47.888931 (+ 11394us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:47.888961 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:12:47.889565 (+   604us) log.cc:844] Serialized 78 byte log entry
0709 12:12:48.062272 (+172707us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:48.091934 (+ 29662us) log.cc:844] Serialized 14 byte log entry
0709 12:12:49.210953 (+1119019us) spinlock_profiling.cc:243] Waited 9.52 ms on lock 0x7b1800499540. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:12:49.375695 (+164742us) log.cc:844] Serialized 14 byte log entry
0709 12:12:49.493538 (+117843us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:49.493806 (+   268us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:49.511352 (+ 17546us) raft_consensus.cc:1685] finished
0709 12:12:49.511379 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:49.511576 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47498,"prepare.queue_time_us":31634,"prepare.run_cpu_time_us":2083,"prepare.run_wall_time_us":25968,"raft.queue_time_us":1203390,"raft.run_cpu_time_us":686,"raft.run_wall_time_us":33525,"spinlock_wait_cycles":21888896,"thread_start_us":334039,"threads_started":3,"wal-append.queue_time_us":377659}
I20260709 12:12:49.562705 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:49.563969 13305 leader_election.cc:400] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 20. Current term is 21.
I20260709 12:12:49.564239 13305 leader_election.cc:403] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:12:49.505124 19582 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:49.565265 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:49.565699 19582 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:49.568174 19582 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:49.565986 19626 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:49.569595 19626 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 20 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:49.566161 19481 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:12:49.566303 19580 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:49.517834 19542 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:49.573879 19542 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:49.558297 19581 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:49.575799 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:49.576370 13336 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:12:49.537847 19610 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:49.577325 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:49.577530 19610 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:49.578166 19581 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:49.579782 19626 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:49.581441 19566 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:49.530296 19445 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:49.583207 19445 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:49.511978 19150 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:49.585026 19150 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:49.586090 19150 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:49.587534 19364 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:49.587997 19364 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:49.588388 19364 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:49.590832 19529 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:49.592695 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2925) took 1581 ms. Trace:
I20260709 12:12:49.593189 19529 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:49.528878 19482 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:49.533596 19559 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:49.525763 19615 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:49.596603 19455 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:49.594635 19529 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:12:49.603374 19627 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:12:49.537109 19357 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:49.598807 19455 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:49.604988 12525 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2928) took 1587 ms (client timeout 1702 ms). Trace:
I20260709 12:12:49.605432 19357 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 13.9, Last appended by leader: 9, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:49.605919 12525 rpcz_store.cc:269] 0709 12:12:48.017230 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.017350 (+   120us) service_pool.cc:224] Handling call
0709 12:12:49.604651 (+1587301us) spinlock_profiling.cc:243] Waited 233 ms on lock 0x7b540038ead4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:49.604911 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":535232256}
I20260709 12:12:49.606593 19385 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:49.606868 19357 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:49.593082 12522 rpcz_store.cc:276] 0709 12:12:48.011749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.011909 (+   160us) service_pool.cc:224] Handling call
0709 12:12:48.012208 (+   299us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:49.133709 (+1121501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:49.592450 (+458741us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540007ce54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:49.592634 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2573632768}
I20260709 12:12:49.610265 19445 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:49.612301 19610 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:49.612608 19532 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:49.605032 19627 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:49.614530 19532 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:49.615113 19532 raft_consensus.cc:697] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:49.616075 19532 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:49.618765 19589 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.759s	user 0.005s	sys 0.000s
W20260709 12:12:49.619100 12605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3233) took 1736 ms (client timeout 1809 ms). Trace:
W20260709 12:12:49.619704 12605 rpcz_store.cc:269] 0709 12:12:47.882927 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.099995 (+217068us) service_pool.cc:224] Handling call
0709 12:12:49.618843 (+1518848us) spinlock_profiling.cc:243] Waited 27 ms on lock 0x7b54002e6dd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:49.619014 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":62118784}
I20260709 12:12:49.596956 19615 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:49.623562 19539 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:49.624284 19539 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:49.626336 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 553638us
I20260709 12:12:49.627592 12646 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:49.631316 13334 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: [19.15-19.15]
I20260709 12:12:49.634526 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:49.630985 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2903) took 1874 ms. Trace:
I20260709 12:12:49.652798 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:49.653920 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:49.650889 12603 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:49.656713 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:49.657366 12527 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:12:49.659809 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:49.537496 12597 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:12:49.660776 12527 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
I20260709 12:12:49.650679 12676 rpcz_store.cc:276] 0709 12:12:47.756528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.203860 (+447332us) service_pool.cc:224] Handling call
0709 12:12:48.232335 (+ 28475us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:48.233000 (+   665us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:12:48.892127 (+659127us) log.cc:844] Serialized 14 byte log entry
0709 12:12:49.630632 (+738505us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:49.630684 (+    52us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:49.630729 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:49.630791 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:49.630927 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":503723,"prepare.queue_time_us":519706,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":504361,"raft.run_cpu_time_us":78,"raft.run_wall_time_us":79,"thread_start_us":874508,"threads_started":2,"wal-append.queue_time_us":440397}
I20260709 12:12:49.664131 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:49.630306 19595 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:49.665314 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:49.632627 19597 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:49.668982 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:49.630048 18993 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:49.700394 19619 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:49.700718 19629 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:12:49.701200 19619 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:49.668000 19482 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:12:49.702874 19482 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:49.703274 19482 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:49.703936 19619 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:49.673979 19597 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:49.682191 12527 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
I20260709 12:12:49.674909 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:49.634799 19617 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:49.557152 19624 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:49.710021 12603 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
W20260709 12:12:49.725492 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:49.725257 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 203794us
I20260709 12:12:49.714195 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:49.734484 13305 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:49.728680 13333 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:49.668464 19595 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:49.735319 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 60498.3us
I20260709 12:12:49.666175 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:49.739431 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 177424us
I20260709 12:12:49.740784 12597 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:49.742084 12572 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:49.736681 12571 leader_election.cc:400] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15.
I20260709 12:12:49.743608 13305 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:49.628309 19576 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:49.745532 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:49.754765 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:49.754573 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:49.730255 19570 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:49.751812 19150 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:49.757507 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:49.721367 18993 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:49.756420 12494 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:49.762814 12671 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:49.763195 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:49.771513 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:49.772585 12671 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:49.773913 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:49.773666 12494 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:49.711205 19443 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.047s
I20260709 12:12:49.738914 19615 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:12:49.805722 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:49.744665 19444 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:49.778502 12571 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:49.833560 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:49.829964 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:49.758473 19385 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:49.803437 12671 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 20 index: 9. (term mismatch)
I20260709 12:12:49.746418 19595 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:49.829772 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:49.845716 12572 leader_election.cc:400] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15.
I20260709 12:12:49.846356 12679 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:49.834208 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 55805.1us
I20260709 12:12:49.764241 19390 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:49.852538 12597 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:49.758226 19559 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:49.811970 19629 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:12:49.873028 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 243951us
W20260709 12:12:49.881451 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:49.877364 12646 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:49.855041 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:49.845489 19385 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:49.810854 19510 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 16.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.085s
I20260709 12:12:49.768280 19570 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:49.860236 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 86667.3us
I20260709 12:12:49.836848 12671 pending_rounds.cc:85] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 8
I20260709 12:12:49.764814 19562 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:49.768887 19461 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:49.707470 19505 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:49.830184 19615 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:49.866504 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:49.759372 19619 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:49.811098 19539 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:49.811259 19495 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:49.832953 19582 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:49.865696 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3231) took 1983 ms. Trace:
I20260709 12:12:49.833272 19515 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 12:12:49.857647 12527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:49.897392 12597 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:49.929131 12604 rpcz_store.cc:276] 0709 12:12:47.882437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:47.883490 (+  1053us) service_pool.cc:224] Handling call
0709 12:12:47.918495 (+ 35005us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:47.977682 (+ 59187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:47.977729 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:47.977782 (+    53us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:47.977827 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:49.865338 (+1887511us) spinlock_profiling.cc:243] Waited 58.9 ms on lock 0x7b5400375354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:49.865430 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:49.865615 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135489920}
I20260709 12:12:49.934891 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:49.920866 19579 consensus_meta.cc:287] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.821s	user 0.012s	sys 0.006s
I20260709 12:12:49.861728 19597 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:49.886705 19629 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:49.867192 19559 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:49.899339 19526 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:49.907316 12679 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:49.904075 19633 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:49.899088 19385 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:49.898031 19628 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Leader election won for term 13
I20260709 12:12:49.913192 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3237) took 1985 ms. Trace:
I20260709 12:12:49.835232 19625 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:12:49.907261 12671 pending_rounds.cc:107] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 19.9
I20260709 12:12:50.001137 12671 raft_consensus.cc:2889] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:49.920045 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.001926 12530 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:49.927866 19619 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.003077 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:49.999643 12601 rpcz_store.cc:276] 0709 12:12:47.927592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.376414 (+448822us) service_pool.cc:224] Handling call
0709 12:12:48.376896 (+   482us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:49.340311 (+963415us) raft_consensus.cc:1534] Early marking committed up to index 7
0709 12:12:49.340378 (+    67us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:49.403805 (+ 63427us) log.cc:844] Serialized 52 byte log entry
0709 12:12:49.720278 (+316473us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:12:49.720321 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:49.720389 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:49.912940 (+192551us) raft_consensus.cc:1685] finished
0709 12:12:49.912962 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:49.913120 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":962826,"raft.queue_time_us":963727,"spinlock_wait_cycles":14311680,"thread_start_us":267025,"threads_started":1,"wal-append.queue_time_us":470101}
I20260709 12:12:49.863577 19624 raft_consensus.cc:2764] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election decision vote started in defunct term 18: won
I20260709 12:12:49.934949 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:49.857527 19390 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:49.858506 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.012822 12603 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:49.915071 19562 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:49.946161 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:49.947731 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.015669 12531 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:49.921696 19615 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:12:49.891323 19273 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.042s
I20260709 12:12:49.993727 12604 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
W20260709 12:12:49.994613 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 315222us
I20260709 12:12:49.957398 19629 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:49.996728 19526 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:49.997471 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:50.028396 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:49.998153 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 364228us
I20260709 12:12:49.906708 12494 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:49.933131 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.029798 12597 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:50.007510 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:49.925427 19461 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:49.992910 19510 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:49.991071 19559 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:50.034936 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 128313us
I20260709 12:12:50.036706 12646 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:50.029306 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:50.037921 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:50.027086 12601 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:12:50.031383 12675 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:49.958985 19505 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:49.896133 19444 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:50.033020 12671 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
W20260709 12:12:50.034859 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 186806us
I20260709 12:12:50.043185 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:50.043900 12671 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:12:50.045581 19604 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:50.046314 19606 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:50.046031 19604 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:50.049626 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:50.030593 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 183355us
I20260709 12:12:49.952298 19632 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.088s
I20260709 12:12:49.928869 18993 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.016669 19628 raft_consensus.cc:697] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:50.022740 19562 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:12:50.057853 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:50.058717 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:49.937008 19635 consensus_queue.cc:1059] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.101s
I20260709 12:12:50.053702 19604 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:50.060570 12677 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 12 FOLLOWER]: Advancing to term 13
I20260709 12:12:50.065582 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:50.061825 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:50.011797 19495 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.040s
I20260709 12:12:49.953238 19579 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:50.038849 19510 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:50.029987 19526 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:50.060001 19455 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:50.057070 19606 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.041489 19032 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:50.077845 12677 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13.
I20260709 12:12:50.079964 12597 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:12:50.054441 19273 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.080945 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:50.069304 19142 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:50.047777 19582 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:50.081869 12601 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 14. Current term is 15. Ops: [14.12-14.12]
I20260709 12:12:50.102656 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 15 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:50.065968 19566 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.054749 19491 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 12:12:50.056569 19515 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.028s
I20260709 12:12:50.122356 19619 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:50.111330 19624 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.112951 19273 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.124198 19624 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:50.126464 19273 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:50.104354 19032 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.127718 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:50.129794 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:50.130147 19582 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:50.130897 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.131127 19582 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.131911 12603 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
W20260709 12:12:50.124114 19364 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.535s	user 0.008s	sys 0.003s
I20260709 12:12:50.139750 19364 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.142503 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:50.144146 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:50.111044 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:50.145752 12530 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:50.081434 19596 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.154908 19579 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:50.155043 19596 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.157322 19596 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:50.114478 19142 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.159472 19570 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:50.160863 19142 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:50.152268 19566 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:50.161744 19045 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:50.163897 19566 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:50.165217 19357 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.557s	user 0.008s	sys 0.004s
I20260709 12:12:50.166329 19595 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.116590 19628 consensus_queue.cc:237] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 13, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:50.120378 19580 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.548s	user 0.013s	sys 0.000s
I20260709 12:12:50.121273 12525 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:50.181272 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.110338 12604 raft_consensus.cc:2376] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15. Re-sending same reply.
I20260709 12:12:50.182088 12600 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 14 index: 9. (term mismatch)
I20260709 12:12:50.182529 12600 pending_rounds.cc:85] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 8
I20260709 12:12:50.138654 19444 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.144902 12674 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:12:50.185366 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:50.131920 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:50.115715 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:50.170230 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.186993 12522 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:50.101130 19455 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.174438 19559 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:50.196251 19390 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:50.175148 19461 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:50.176311 19482 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.199354 19482 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:50.182787 12600 pending_rounds.cc:107] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 13.9
I20260709 12:12:50.201531 12600 raft_consensus.cc:2889] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:50.178002 19595 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:50.188639 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 72965.1us
I20260709 12:12:50.192696 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:50.193439 19585 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 1.108s	user 0.002s	sys 0.001s
I20260709 12:12:50.208214 19559 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:50.208705 19500 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:50.189666 19045 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:50.194783 19455 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:50.200333 12525 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 22: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:50.186077 12571 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:50.204044 12674 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 19
I20260709 12:12:50.213243 19503 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:50.108067 19606 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.214493 19505 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:50.215200 19505 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:50.215579 19505 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:50.216066 19606 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:50.218086 19385 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:50.220794 19385 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:50.190038 12671 raft_consensus.cc:2393] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:50.207669 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.221418 12598 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:12:50.224581 12676 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:50.225699 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:50.237694 12604 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:50.238061 12522 raft_consensus.cc:2368] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19.
I20260709 12:12:50.226225 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.204401 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:50.188315 19580 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 20. Current term is 21.
I20260709 12:12:50.228911 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 802) took 1311 ms. Trace:
I20260709 12:12:50.239957 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:50.235391 12526 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2931) took 2192 ms (client timeout 1936 ms). Trace:
I20260709 12:12:50.240967 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:50.247639 12600 raft_consensus.cc:2393] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 15: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:50.247542 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:50.251143 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:50.239102 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:50.232610 12674 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
W20260709 12:12:50.238483 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 183755us
W20260709 12:12:50.247399 12526 rpcz_store.cc:269] 0709 12:12:48.043000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.052812 (+  9812us) service_pool.cc:224] Handling call
0709 12:12:50.235176 (+2182364us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54003c0354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:50.235288 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2528611200}
I20260709 12:12:50.239094 12523 rpcz_store.cc:276] 0709 12:12:48.917825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.917978 (+   153us) service_pool.cc:224] Handling call
0709 12:12:48.918335 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:49.114634 (+196299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:49.114664 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:49.114708 (+    44us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:12:49.114739 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:50.228633 (+1113894us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b5400105ed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:50.228690 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:50.228845 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":450933632}
I20260709 12:12:50.255566 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.255728 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.256201 12526 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:50.256922 12523 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 17: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:50.259449 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.256193 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:50.271451 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:50.272250 12526 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:50.197647 19604 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.254328 19130 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:50.275367 12496 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:50.254405 19632 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.235217 19526 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.260888 19606 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:50.274080 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:50.275559 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.179342 19629 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.261240 19045 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:12:50.254567 19142 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:50.242236 19357 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.289657 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.303246 19482 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:50.285120 19595 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.022s
I20260709 12:12:50.304344 19610 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:12:50.305195 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.257581 19505 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.307718 19614 raft_consensus.cc:3055] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:50.291989 12671 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:50.343896 19357 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.346191 19482 raft_consensus.cc:697] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:50.347003 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:50.347649 12671 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:50.347354 19482 consensus_queue.cc:237] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.348155 19357 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:50.348563 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:50.349687 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:50.308902 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.350958 19505 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:50.270291 19500 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.067s
I20260709 12:12:50.289786 12643 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:50.314600 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.353914 12527 raft_consensus.cc:606] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:12:50.313550 12672 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 727) took 1725 ms (client timeout 1715 ms). Trace:
I20260709 12:12:50.293066 19606 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:50.339090 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:50.311707 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 723) took 2077 ms. Trace:
I20260709 12:12:50.291100 19632 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.351667 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.274776 19578 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 7, Time since last communication: 0.071s
I20260709 12:12:50.313530 19390 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:12:50.350986 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.354568 12527 raft_consensus.cc:614] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:50.317674 19527 raft_consensus.cc:3055] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:12:50.354797 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 65118.9us
I20260709 12:12:50.337059 19045 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:50.354662 12672 rpcz_store.cc:269] 0709 12:12:48.588597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.588723 (+   126us) service_pool.cc:224] Handling call
0709 12:12:50.313296 (+1724573us) spinlock_profiling.cc:243] Waited 961 ms on lock 0x7b540015d6d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:50.313473 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2209523072}
I20260709 12:12:50.346218 19614 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:50.369740 12526 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:50.358700 19606 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:50.357101 12680 rpcz_store.cc:276] 0709 12:12:48.233815 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.435052 (+201237us) service_pool.cc:224] Handling call
0709 12:12:48.435419 (+   367us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:48.823335 (+387916us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:48.823404 (+    69us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:48.833519 (+ 10115us) log.cc:844] Serialized 26 byte log entry
0709 12:12:48.927647 (+ 94128us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:12:49.069295 (+141648us) log.cc:844] Serialized 14 byte log entry
0709 12:12:49.609605 (+540310us) spinlock_profiling.cc:243] Waited 4.52 ms on lock 0x7b18004191a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:12:49.759761 (+150156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:50.311321 (+551560us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b54000c26d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:50.311380 (+    59us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:50.311445 (+    65us) raft_consensus.cc:1685] finished
0709 12:12:50.311461 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:50.311640 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23433,"prepare.queue_time_us":141407,"prepare.run_cpu_time_us":62,"prepare.run_wall_time_us":64,"raft.queue_time_us":844144,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":197,"spinlock_wait_cycles":369264384,"thread_start_us":441409,"threads_started":3,"wal-append.queue_time_us":287592}
I20260709 12:12:50.363672 12604 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:12:50.355845 19130 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:50.358415 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:50.378821 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.378993 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:50.342511 19142 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.382719 12680 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
I20260709 12:12:50.385532 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:50.362987 19443 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:50.386837 12680 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:12:50.382099 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:50.368217 19045 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:50.364861 19527 raft_consensus.cc:740] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:50.377198 19445 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.766s	user 0.013s	sys 0.000s
I20260709 12:12:50.389737 12526 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:50.392237 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.377935 19357 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.397735 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.398795 12522 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 20: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:50.395849 12604 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:50.397020 12601 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:50.408718 12494 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:50.408887 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:50.409890 12671 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:50.413030 12680 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 14 index: 9. (index mismatch)
I20260709 12:12:50.413256 12526 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:12:50.413441 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 14 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.388949 19647 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:50.414115 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.414422 12604 raft_consensus.cc:2410] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 16 because replica has last-logged OpId of term: 15 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 7.
W20260709 12:12:50.416618 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 92390us
I20260709 12:12:50.418263 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.419571 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:50.424397 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:50.425588 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:50.425401 12604 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
W20260709 12:12:50.425402 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 140955us
I20260709 12:12:50.427347 12496 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:50.429823 13304 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:50.402810 19443 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:50.436236 12680 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:12:50.433526 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:50.438004 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: true dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:50.439356 12496 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:50.439677 12604 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:50.443257 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:50.402542 19357 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.410943 19130 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:50.419001 19647 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:50.445065 19617 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.007s
I20260709 12:12:50.452205 19443 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 17.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.453332 19617 raft_consensus.cc:2764] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:12:50.438501 19500 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:50.457643 19142 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:50.458221 19591 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.507s	user 0.003s	sys 0.001s
I20260709 12:12:50.447054 19514 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:12:50.462263 19130 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:50.463037 19514 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:50.463701 19514 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:12:50.464874 19130 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:50.466105 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:50.468676 19629 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:50.469501 19629 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:50.469960 19629 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:50.470567 19629 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:50.429785 19649 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:50.472213 19647 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:50.472491 19649 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.427793 19648 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.062s
I20260709 12:12:50.434897 19578 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:50.434614 19581 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:12:50.475828 19578 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:50.476387 19578 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:50.477005 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:50.477677 12598 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:50.442413 19527 consensus_queue.cc:260] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.455768 19357 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:50.438512 19580 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:50.457855 19500 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:50.447355 19536 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.030s
I20260709 12:12:50.454053 19614 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.455411 19632 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:50.505283 19651 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:12:50.478632 19527 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:50.504967 19580 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:50.505162 19130 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.515266 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:50.479537 12604 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:12:50.523011 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.523972 12604 raft_consensus.cc:2368] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19.
I20260709 12:12:50.525657 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:50.526510 12604 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:50.518530 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 79238us
I20260709 12:12:50.537776 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:50.544579 12604 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:50.562364 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.563212 12604 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:50.564139 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.564517 12604 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:12:50.565027 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.565681 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:50.566799 12604 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:50.567793 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.568595 12526 raft_consensus.cc:2410] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 19 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 8.
I20260709 12:12:50.569028 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2907) took 2712 ms. Trace:
I20260709 12:12:50.569955 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:50.569592 12673 rpcz_store.cc:276] 0709 12:12:47.850634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.376446 (+525812us) service_pool.cc:224] Handling call
0709 12:12:48.432769 (+ 56323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:48.446142 (+ 13373us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:50.562568 (+2116426us) spinlock_profiling.cc:243] Waited 56 ms on lock 0x7b54000c3ad0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:50.562685 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":128725888}
I20260709 12:12:50.520139 19580 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:50.562271 19636 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:50.514753 19615 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:50.580492 12496 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:50.577049 19614 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:50.570616 19652 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:50.591811 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.577447 19636 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:50.540331 19130 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:50.585322 12496 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:50.605199 12671 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 12:12:50.605726 12671 pending_rounds.cc:85] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 10
I20260709 12:12:50.605979 12671 pending_rounds.cc:107] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:12:50.606241 12671 raft_consensus.cc:2889] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:50.582587 19482 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:50.582818 19648 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.608011 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:50.590581 19444 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:50.612249 19500 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:50.608758 19581 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.609195 19355 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.595919 19649 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:50.616303 19659 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:50.621595 19582 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:50.621345 19355 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.643862 12598 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:12:50.618734 19648 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.628914 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.646752 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.647146 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.648005 12600 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
I20260709 12:12:50.645023 19582 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:50.648850 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.619251 19444 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:50.648250 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:50.650899 12600 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:12:50.653570 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.654409 12598 raft_consensus.cc:2376] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21. Re-sending same reply.
I20260709 12:12:50.655372 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.655453 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.656201 12600 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:50.656265 12680 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 15 index: 10. (index mismatch)
I20260709 12:12:50.656519 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.657232 12598 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:50.657262 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.658732 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.643122 19659 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:50.659349 12598 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:50.658869 12600 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:50.662834 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:50.655733 19562 consensus_meta.cc:287] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.581s	user 0.006s	sys 0.006s
I20260709 12:12:50.655117 19444 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:50.650539 19582 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:50.663352 12600 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:50.623440 19581 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:50.662665 19658 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:12:50.654084 19536 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:50.629602 19636 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.647096 19617 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:12:50.659484 19482 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:50.664649 19659 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:50.678714 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.665373 19562 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:50.677946 19656 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:12:50.687582 12524 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:12:50.684576 12680 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:12:50.694913 19364 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:50.674361 19514 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:50.708137 19514 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:50.676946 19536 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:50.710441 19536 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:50.710698 13304 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.956s (SENT) [suppressed 13 similar messages]
I20260709 12:12:50.711390 12522 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:50.712186 19045 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.713361 19604 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:50.715176 12672 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
I20260709 12:12:50.717898 12606 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 18: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:50.719024 12526 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 19. Current term is 20. Ops: [19.11-19.11]
W20260709 12:12:50.719700 12606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3256) took 1200 ms (client timeout 1540 ms). Trace:
W20260709 12:12:50.720089 12606 rpcz_store.cc:269] 0709 12:12:49.519401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.528936 (+  9535us) service_pool.cc:224] Handling call
0709 12:12:50.314413 (+785477us) spinlock_profiling.cc:243] Waited 82.6 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:12:50.719494 (+405081us) spinlock_profiling.cc:243] Waited 363 ms on lock 0x7b540022ecd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:50.719639 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1025161344}
I20260709 12:12:50.721390 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.724818 19524 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:50.724970 19647 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:50.702630 19615 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:50.734652 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.689395 19636 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:50.738103 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.722050 12606 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:50.740991 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.742221 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:50.743495 12600 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:50.745260 19647 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:50.745467 19648 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:50.746222 19444 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:50.748893 12572 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:50.750937 12605 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:12:50.751613 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3255) took 1232 ms. Trace:
I20260709 12:12:50.733505 19606 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:50.751955 12605 rpcz_store.cc:276] 0709 12:12:49.519125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.621386 (+102261us) service_pool.cc:224] Handling call
0709 12:12:50.750356 (+1128970us) spinlock_profiling.cc:243] Waited 13 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:12:50.750612 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:50.751347 (+   735us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:50.751555 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":29980416}
I20260709 12:12:50.743633 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.755051 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:50.757404 12600 raft_consensus.cc:2393] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 15: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:50.757265 12572 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:50.757515 12526 raft_consensus.cc:2410] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:12:50.759271 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.760622 12604 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:50.761004 12600 raft_consensus.cc:2393] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 13: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:50.762172 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:50.765374 12606 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:12:50.767377 12606 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:12:50.682070 19482 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:50.759893 19444 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:50.760072 19364 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:50.777328 19581 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:50.759472 19045 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:50.778257 19130 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:50.782642 12673 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:50.783643 19606 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:50.755491 19495 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:50.758353 19562 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:50.796516 19578 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.805454 12646 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:50.807133 19606 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:12:50.807528 12678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 730) took 1964 ms (client timeout 1956 ms). Trace:
W20260709 12:12:50.807955 12678 rpcz_store.cc:269] 0709 12:12:48.843207 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:48.843355 (+   148us) service_pool.cc:224] Handling call
0709 12:12:50.807281 (+1963926us) spinlock_profiling.cc:243] Waited 1.22 s on lock 0x7b54003d4fd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:50.807455 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2803935104}
I20260709 12:12:50.808996 12646 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:50.810068 19606 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:50.810851 19606 raft_consensus.cc:697] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:50.810923 12601 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: []
I20260709 12:12:50.814162 19606 consensus_queue.cc:237] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.816176 12597 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:50.819962 19032 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:50.806100 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 46711.1us
I20260709 12:12:50.822299 12646 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:50.824987 12646 leader_election.cc:400] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19.
I20260709 12:12:50.825364 12646 leader_election.cc:403] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:12:50.823596 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:50.830716 12646 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:50.769703 19632 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:50.769433 19357 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.793529 19629 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.834018 12646 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:50.829959 19142 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:50.847241 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:50.839771 12646 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:50.830951 19475 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:50.834801 19614 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 18. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19.
I20260709 12:12:50.857604 19495 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:50.836400 19130 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election won for term 18
W20260709 12:12:50.881644 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:50.866658 12528 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21. Ops: []
I20260709 12:12:50.890558 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 3350) took 1128 ms. Trace:
I20260709 12:12:50.891228 12677 rpcz_store.cc:276] 0709 12:12:49.762055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.762963 (+   908us) service_pool.cc:224] Handling call
0709 12:12:50.890292 (+1127329us) spinlock_profiling.cc:243] Waited 290 ms on lock 0x7b54002c5954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:50.890473 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":667147520}
I20260709 12:12:50.836223 19364 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:12:50.848835 19597 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:50.855463 19475 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:50.891479 19130 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:50.861725 19604 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:50.846517 19357 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:50.907725 19475 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:50.888823 19580 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:50.890656 19495 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.912675 19130 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:50.876006 19659 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.924374 12679 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:50.914523 19529 raft_consensus.cc:3055] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:50.934959 19580 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:50.935170 19659 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:50.930259 19529 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:50.936590 19529 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.939889 19529 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:50.918154 19597 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:50.943094 19597 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.955529 19495 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:50.958876 19629 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:50.961047 19357 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:50.930361 19508 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:50.962157 19508 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.922808 19604 raft_consensus.cc:697] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:50.964066 19604 consensus_queue.cc:237] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.967293 19482 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:50.923254 19578 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:50.968556 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:50.970285 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:50.970896 12672 raft_consensus.cc:2376] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18. Re-sending same reply.
I20260709 12:12:50.973169 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:50.988598 19582 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:50.991604 19582 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:50.991712 19624 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:50.992542 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 3356) took 1149 ms. Trace:
I20260709 12:12:50.992956 12675 rpcz_store.cc:276] 0709 12:12:49.843039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.843324 (+   285us) service_pool.cc:224] Handling call
0709 12:12:50.992370 (+1149046us) spinlock_profiling.cc:243] Waited 116 ms on lock 0x7b540012d754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:50.992478 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":267511040}
W20260709 12:12:50.993520 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:50.996112 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:50.997016 12678 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:50.997076 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:50.997972 12672 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 20: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:51.011834 12678 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:12:51.019702 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:51.026707 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 181550us
I20260709 12:12:50.977568 19475 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:51.029455 12494 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:51.031212 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2945) took 1086 ms. Trace:
I20260709 12:12:51.034579 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 3363) took 1140 ms. Trace:
I20260709 12:12:51.039083 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:51.057793 12674 rpcz_store.cc:276] 0709 12:12:49.894334 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.185001 (+290667us) service_pool.cc:224] Handling call
0709 12:12:51.034386 (+849385us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b54000933d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:51.034498 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41279360}
I20260709 12:12:51.059240 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:51.059854 12597 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:12:51.039842 19646 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.851s	user 0.003s	sys 0.002s
I20260709 12:12:51.044121 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3344) took 1284 ms. Trace:
I20260709 12:12:51.046572 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:51.049749 13302 connection.cc:256] RPC call timeout handler was delayed by -1.4106826469999305s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
I20260709 12:12:51.041653 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:51.061415 12676 rpcz_store.cc:276] 0709 12:12:49.760029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.760178 (+   149us) service_pool.cc:224] Handling call
0709 12:12:49.769663 (+  9485us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:50.224356 (+454693us) spinlock_profiling.cc:243] Waited 2.07 ms on lock 0x7b540028c920. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488997dc0 00007f1488a221c0 00007f1488a23231 00007f1488a1fd8f 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c
0709 12:12:50.239274 (+ 14918us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:51.042562 (+803288us) spinlock_profiling.cc:243] Waited 4.12 ms on lock 0x7b540028c920. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488997dc0 00007f1488a23a02 00007f1488a1fdd6 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:12:51.043855 (+  1293us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b54003d45d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:51.044056 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":784811264}
I20260709 12:12:51.052040 12643 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:51.055559 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3368) took 1037 ms. Trace:
I20260709 12:12:51.058465 12494 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:51.063242 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:51.057699 12524 rpcz_store.cc:276] 0709 12:12:49.945297 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.945462 (+   165us) service_pool.cc:224] Handling call
0709 12:12:49.945850 (+   388us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:50.696123 (+750273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:51.030909 (+334786us) spinlock_profiling.cc:243] Waited 741 ms on lock 0x7b5400407c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:51.031141 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1705235712}
I20260709 12:12:51.050365 19581 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:51.062636 12671 rpcz_store.cc:276] 0709 12:12:50.018756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.417988 (+399232us) service_pool.cc:224] Handling call
0709 12:12:50.418309 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:50.606664 (+188355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:51.055302 (+448638us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b54003d40d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:51.055494 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":428566784}
I20260709 12:12:51.069455 12643 leader_election.cc:400] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19.
I20260709 12:12:51.070372 12643 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:51.071745 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:51.028918 19636 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.063387 12569 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:51.077250 13333 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:51.075161 12601 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15.
I20260709 12:12:51.079085 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:51.077987 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:51.079360 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:51.087841 13334 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:51.089425 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:51.073483 19581 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:51.091087 19486 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:51.088016 19636 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:51.256867 19486 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:51.257409 19486 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:51.258497 19636 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:51.259013 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:51.259673 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:51.260298 12676 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:51.260339 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:51.078534 19438 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.262452 19438 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:51.262866 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 13 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:51.265115 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 15 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:51.269023 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:51.087330 19606 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:12:51.075749 19604 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:51.270188 19508 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:51.223189 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:51.071337 19659 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.224327 19510 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:51.275843 19510 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:51.275583 19659 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:51.225991 13335 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24. Ops: []
I20260709 12:12:51.226967 19150 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:51.227198 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:12:51.242847 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 153670us
I20260709 12:12:51.279029 19150 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:51.279632 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:51.280046 19150 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:51.281172 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:51.281266 19150 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:51.283776 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:51.284757 12679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 3360) took 1425 ms (client timeout 1540 ms). Trace:
W20260709 12:12:51.285637 12679 rpcz_store.cc:269] 0709 12:12:49.859907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.997160 (+137253us) service_pool.cc:224] Handling call
0709 12:12:51.284608 (+1287448us) spinlock_profiling.cc:243] Waited 896 ms on lock 0x7b5400091fd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:51.284702 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2060608768}
I20260709 12:12:51.286422 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:51.286756 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:51.287180 13336 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 21: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:51.288621 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:51.289621 13336 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:51.287286 12599 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:51.291812 12676 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
W20260709 12:12:51.285656 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:51.292984 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:51.227826 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:51.228185 19445 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.229426 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48476 (request call id 72) took 1372 ms. Trace:
I20260709 12:12:51.294298 12527 rpcz_store.cc:276] 0709 12:12:49.857293 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:49.857486 (+   193us) service_pool.cc:224] Handling call
0709 12:12:51.229200 (+1371714us) spinlock_profiling.cc:243] Waited 421 ms on lock 0x7b540038ead4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1a853 00007f148d9e860a 00007f1486f6e7d4 00007f1486f6e762 00007f1486f6e691 00007f1486f6e60c 00007f1486f6d8c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:51.229322 (+   122us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":969208192}
I20260709 12:12:51.231781 19624 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.073787 19618 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:12:51.087694 19494 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:51.233513 19475 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:51.227845 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:51.234066 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:51.234068 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:51.314779 13332 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 20: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:51.314774 13330 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 18: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:51.234622 13333 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: []
I20260709 12:12:51.235319 19597 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:51.236271 19542 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:51.236333 19614 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:51.237901 19526 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:51.239269 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:51.240141 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:51.240239 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:51.241575 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:51.242832 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:51.296425 19604 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:51.331314 12527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3af899b7fab042d0ac433430f162d106"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:51.331948 12527 raft_consensus.cc:606] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:51.323166 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:51.332718 13335 raft_consensus.cc:2393] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 15: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:12:51.332789 13302 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 15 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.613s (SENT)
I20260709 12:12:51.248317 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:51.254798 19130 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:51.061650 19482 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:12:51.087059 19461 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:51.292052 12597 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
I20260709 12:12:51.301002 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:51.302151 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:51.336737 12643 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:51.307452 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:51.337596 12601 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:12:51.310073 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:51.324898 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:51.339401 13333 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 26: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:12:51.326017 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 93720.3us
I20260709 12:12:51.327716 12672 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:51.244189 12680 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:51.329793 12599 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:12:51.341735 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:51.326817 12675 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:12:51.308167 19618 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:51.299953 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:51.313043 19606 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:51.350041 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:51.353551 12676 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:51.319777 19390 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:51.317016 19597 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:51.370522 19597 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:51.382310 13332 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:12:51.384492 12678 raft_consensus.cc:2368] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22.
I20260709 12:12:51.384737 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:51.356171 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:51.356423 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:51.385381 12601 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:51.385843 12602 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:51.356742 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:51.357564 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:51.386791 12597 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:51.389968 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:51.391028 12527 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:51.390688 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:51.358781 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:51.391768 12524 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:51.358891 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:51.306265 19364 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:12:51.322355 19503 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:51.322481 19614 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:51.299342 19445 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:51.334651 19443 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.361670 19482 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:51.364859 19659 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:51.358517 19304 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:51.365814 19577 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:51.366544 19461 raft_consensus.cc:697] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:51.366560 19539 consensus_queue.cc:1059] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 16.9, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:51.367695 12677 raft_consensus.cc:2368] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18.
I20260709 12:12:51.420289 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:51.367820 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:51.421948 12680 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:12:51.368084 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:51.306457 19624 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:51.352347 19618 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:51.427663 19625 consensus_queue.cc:1059] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:51.428737 19627 raft_consensus.cc:3055] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:51.429167 19627 raft_consensus.cc:740] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:51.403693 19524 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:51.430075 19627 consensus_queue.cc:260] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:51.430961 19524 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:51.431371 19529 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 12:12:51.432363 19529 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:12:51.432727 19529 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:51.393847 12526 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:12:51.434602 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:51.435232 19539 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:12:51.435672 19539 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:51.435911 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:51.436568 19539 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:51.437139 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:51.311669 19486 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:51.424446 19580 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.336861 19604 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:51.437837 19539 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:51.396859 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:51.397069 12571 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:51.401402 19581 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:51.403653 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:51.404129 12525 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:12:51.405377 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:51.406633 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:51.454963 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58015.9us
W20260709 12:12:51.367839 19680 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:12:51.365103 19562 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:51.334695 19617 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.069s
I20260709 12:12:51.360173 19526 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election won for term 15
W20260709 12:12:51.414048 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:51.424906 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:51.408921 19503 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:51.439333 12526 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:51.408414 19614 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:51.433604 19529 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:51.453684 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:51.352255 19508 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:12:51.455605 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 94366.7us
I20260709 12:12:51.455837 12571 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:51.454794 12597 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:51.486274 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:51.487179 12597 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:51.465780 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:51.466297 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:51.461696 12676 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
W20260709 12:12:51.469123 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 142209us
I20260709 12:12:51.469223 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2846) took 1188 ms. Trace:
I20260709 12:12:51.490993 12523 rpcz_store.cc:276] 0709 12:12:50.280446 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.362382 (+ 81936us) service_pool.cc:224] Handling call
0709 12:12:51.464472 (+1102090us) spinlock_profiling.cc:243] Waited 934 ms on lock 0x7b54003f8c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:51.469150 (+  4678us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2148757760}
I20260709 12:12:51.396697 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:51.491820 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:51.481107 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:51.498234 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:51.458874 19506 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:51.527019 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:51.412847 19649 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 12:12:51.516932 12571 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:51.519277 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:51.531586 12672 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 19: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:51.468546 19526 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:51.533715 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 169067us
I20260709 12:12:51.421260 19443 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:51.495474 19486 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:51.438845 19580 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:51.541946 12680 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:51.537561 12601 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:51.533761 12602 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:51.453711 19364 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.073s
I20260709 12:12:51.548071 12525 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:51.544689 12597 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:51.545501 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2951) took 1590 ms. Trace:
I20260709 12:12:51.528671 19494 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:51.526748 19495 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:12:51.531718 19542 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.556702 19486 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:51.535022 19482 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:51.558626 19636 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:12:51.560171 19659 raft_consensus.cc:3055] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:51.577747 19659 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:51.550168 12530 rpcz_store.cc:276] 0709 12:12:49.955434 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.147746 (+192312us) service_pool.cc:224] Handling call
0709 12:12:50.148165 (+   419us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:50.148595 (+   430us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:50.148637 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:50.978492 (+829855us) spinlock_profiling.cc:243] Waited 11.8 ms on lock 0x7b180020c680. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a
0709 12:12:50.979045 (+   553us) spinlock_profiling.cc:243] Waited 3.28 ms on lock 0x7b2c00058028. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:12:50.979715 (+   670us) log.cc:844] Serialized 26 byte log entry
0709 12:12:51.030949 (+ 51234us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:51.030987 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:51.031067 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:51.367232 (+336165us) spinlock_profiling.cc:243] Waited 23.8 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:51.545266 (+178034us) raft_consensus.cc:1685] finished
0709 12:12:51.545291 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:51.545438 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3112,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":89,"spinlock_wait_cycles":89368960,"thread_start_us":50345,"threads_started":1,"wal-append.queue_time_us":111176}
I20260709 12:12:51.565702 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:51.581700 12675 raft_consensus.cc:2410] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:12:51.539211 19526 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:51.542915 19475 raft_consensus.cc:2764] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:12:51.534199 19606 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:12:51.551817 19506 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:51.547300 19503 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:51.555773 19529 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:12:51.609483 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:51.575176 19636 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:51.574707 19495 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:51.611399 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:51.611399 12675 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:12:51.615424 12674 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 18. Ops: []
I20260709 12:12:51.587041 19500 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election lost for term 13. Reason: could not achieve majority
I20260709 12:12:51.578752 19659 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:51.591228 19562 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.634719 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:51.640326 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:51.580570 19150 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:12:51.693660 12597 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:12:51.595629 19506 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:51.633865 19679 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.394s	user 0.004s	sys 0.001s
I20260709 12:12:51.576344 19542 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:51.660310 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 326277us
I20260709 12:12:51.555210 19461 consensus_queue.cc:237] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:51.694783 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 177994us
I20260709 12:12:51.617269 19364 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:12:51.612993 19597 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:51.695533 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 176349us
I20260709 12:12:51.761124 12524 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:51.743078 12680 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 19. Current term is 20. Ops: [19.11-19.11]
W20260709 12:12:51.705516 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 176020us
I20260709 12:12:51.746539 12676 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:51.762931 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:51.761427 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:51.765064 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:12:51.723466 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:51.743018 13302 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:51.700166 19506 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:51.769344 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:51.784801 12571 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:51.786687 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:51.787501 12530 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:51.777935 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:51.718008 19685 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:12:51.774911 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:51.804374 12673 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 17: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:51.805387 12523 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:12:51.810276 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:51.809547 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:51.812808 12603 raft_consensus.cc:2376] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20. Re-sending same reply.
I20260709 12:12:51.812824 12597 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 15: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:51.747120 19624 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:51.705293 19445 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:51.740648 19130 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:51.761312 19503 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:51.806741 13302 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:51.816120 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 253271us
W20260709 12:12:51.814625 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 183425us
I20260709 12:12:51.741364 19527 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:12:51.814561 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:51.724455 19696 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:12:51.699038 19562 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:51.819424 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:51.826300 12680 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 20: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:51.786969 19508 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:51.823776 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:51.824051 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:51.824355 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:51.822903 19130 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.002s
I20260709 12:12:51.719935 19510 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:51.822829 12597 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:12:51.841372 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:51.842059 12597 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:51.815546 19642 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:51.828452 12601 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:51.706375 19514 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.110s
I20260709 12:12:51.820195 19527 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:51.831697 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:51.855111 12680 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:12:51.702415 19500 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:51.818472 19364 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:51.827191 19696 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:51.828617 12676 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:12:51.775404 19694 raft_consensus.cc:2764] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:12:51.835531 12598 raft_consensus.cc:2410] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 18 because replica has last-logged OpId of term: 17 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
W20260709 12:12:51.834179 19701 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:12:51.720649 19695 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.097s
I20260709 12:12:51.803812 19580 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:51.837926 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:51.838925 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:51.839720 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:51.843086 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:51.827453 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:51.844583 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:51.815958 19699 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:51.825554 19624 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:51.761492 19495 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:51.802819 19610 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:12:51.709442 19636 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:51.839572 19130 consensus_queue.cc:1243] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 18.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:12:51.836915 19443 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:51.835489 19693 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:51.838441 19508 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:51.915210 19508 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:51.876368 19357 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:51.915973 19364 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:51.916424 19357 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:51.917322 19364 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:12:51.918819 19627 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:51.872148 19642 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:51.877089 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:51.876843 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 70147.8us
I20260709 12:12:51.880892 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:51.922356 12605 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:51.881290 12603 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:51.925385 12603 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:51.878455 12597 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:51.883306 12496 leader_election.cc:400] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18.
I20260709 12:12:51.878108 12525 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:12:51.927721 12496 leader_election.cc:403] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Cancelling election due to peer responding with higher term
I20260709 12:12:51.886876 19696 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:51.875735 19697 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:51.889469 19615 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:12:51.890858 19514 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:12:51.826470 19503 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:51.856518 19500 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:51.894649 19527 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:51.896857 19524 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:51.899241 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:51.901778 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:51.902486 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:51.903414 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 76275.5us
I20260709 12:12:51.946250 12678 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 14.
I20260709 12:12:51.946197 12606 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:51.903585 12530 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:51.851008 19604 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:51.857002 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 322135us
I20260709 12:12:51.849802 19510 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:51.879225 19699 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:51.952580 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:51.954540 12525 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:12:51.933072 19694 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:12:51.929867 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 117667us
W20260709 12:12:51.931895 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48724.5us
I20260709 12:12:51.906296 19150 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:51.932984 12597 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:12:51.967267 12597 pending_rounds.cc:85] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 10
W20260709 12:12:51.934140 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 107429us
W20260709 12:12:51.936231 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:51.933576 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 109561us
I20260709 12:12:51.909286 19445 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:51.967700 12597 pending_rounds.cc:107] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:12:51.987948 12597 raft_consensus.cc:2889] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:51.943606 12605 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:51.988711 12602 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:51.971256 12530 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 15. Current term is 16. Ops: []
W20260709 12:12:51.979357 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:51.985292 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:51.947479 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:51.919497 19693 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:51.950598 12680 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
I20260709 12:12:52.011054 12677 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:51.958693 12494 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:51.922104 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:51.944339 19500 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:51.956236 19527 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:51.956236 19699 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:51.948789 19610 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election lost for term 17. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 17. Current term is 18.
I20260709 12:12:51.948149 19542 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:52.004380 12678 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:12:51.987016 12646 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:51.977103 19614 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:51.932337 19642 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:51.977185 19700 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:12:52.026014 12646 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:51.986424 19696 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:12:52.037353 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 78767.1us
W20260709 12:12:52.041137 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 202097us
I20260709 12:12:52.045389 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:52.052346 12678 raft_consensus.cc:2410] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 26 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:12:52.053481 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:51.978706 19510 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:51.976689 19503 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:52.058344 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 2969) took 1643 ms. Trace:
I20260709 12:12:52.054843 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:52.013211 19597 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:51.967670 19705 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:52.013103 19693 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:52.066418 12671 raft_consensus.cc:2393] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 21: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:12:52.067312 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 145160us
I20260709 12:12:51.945526 19697 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.075644 12601 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:51.978864 19629 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.074s
I20260709 12:12:52.079974 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:52.080897 12606 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:12:52.043900 19475 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:52.058944 12600 rpcz_store.cc:276] 0709 12:12:50.415211 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.769772 (+354561us) service_pool.cc:224] Handling call
0709 12:12:50.831542 (+ 61770us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:50.858407 (+ 26865us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:50.858725 (+   318us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:50.860967 (+  2242us) log.cc:844] Serialized 26 byte log entry
0709 12:12:51.088274 (+227307us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:51.131431 (+ 43157us) log.cc:844] Serialized 14 byte log entry
0709 12:12:51.222628 (+ 91197us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:51.240164 (+ 17536us) spinlock_profiling.cc:243] Waited 5.65 ms on lock 0x7b480024c258. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:12:52.057819 (+817655us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b540007a154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:52.057903 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:52.057972 (+    69us) raft_consensus.cc:1685] finished
0709 12:12:52.057993 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:52.058235 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3758,"prepare.queue_time_us":133111,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":4047,"raft.run_cpu_time_us":2683,"raft.run_wall_time_us":383085,"spinlock_wait_cycles":61521024,"thread_start_us":226771,"threads_started":1,"wal-append.queue_time_us":463024}
I20260709 12:12:52.036242 19150 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:51.971338 19524 consensus_queue.cc:1059] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 12:12:52.052686 19698 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.052s
I20260709 12:12:52.084240 19461 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:12:52.063253 19045 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:52.023905 19130 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
I20260709 12:12:52.051649 19582 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:52.072726 19705 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:12:52.077989 19539 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.621s	user 0.009s	sys 0.003s
I20260709 12:12:52.102658 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:52.118563 12604 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
I20260709 12:12:52.100395 19508 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.121237 19045 raft_consensus.cc:697] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:52.122679 19045 consensus_queue.cc:237] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.126857 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:52.127993 12527 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:12:52.078271 19541 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:12:52.090056 19714 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.137s	user 0.000s	sys 0.000s
I20260709 12:12:52.131430 12526 raft_consensus.cc:606] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:52.129295 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:52.068992 12494 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:52.084988 19707 raft_consensus.cc:3055] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:52.085839 19475 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:52.156428 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:52.153919 12526 raft_consensus.cc:614] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:52.187350 19503 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:12:52.188270 19130 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:52.187384 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:52.192112 19461 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:52.192644 19461 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.193105 19461 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:12:52.093786 19618 consensus_meta.cc:287] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.665s	user 0.004s	sys 0.008s
W20260709 12:12:52.109866 19529 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.501s	user 0.010s	sys 0.001s
I20260709 12:12:52.198589 19618 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:12:52.159680 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 90761.1us
I20260709 12:12:52.135397 12602 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:52.101128 19514 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:12:52.202775 19514 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 21 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:52.203091 12494 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:52.203809 19514 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:52.204499 12494 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:52.205540 19482 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:52.090530 19604 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:52.206362 19539 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:52.206103 19482 raft_consensus.cc:697] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:52.207278 19604 consensus_queue.cc:1059] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:52.207962 19482 consensus_queue.cc:237] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 16, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:52.208649 19604 raft_consensus.cc:3055] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:52.209092 19604 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:52.207305 19539 raft_consensus.cc:697] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:52.211440 19539 consensus_queue.cc:237] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.215320 12494 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:52.209964 19604 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.217585 19604 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:52.221352 19443 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:52.222039 19443 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.222535 19443 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:52.201820 19618 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:52.232326 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:52.234324 19618 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:52.237582 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:52.105039 19524 raft_consensus.cc:3055] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:52.247673 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:52.102697 19642 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:52.104260 19614 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:52.249477 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:52.121937 19508 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:52.116081 19705 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:52.256407 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:52.257099 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 294385us
I20260709 12:12:52.141412 19582 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:52.160269 19707 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:52.182921 19629 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:52.182708 19597 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:52.249640 19130 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:12:52.103822 19526 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.251116 19524 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:52.104023 19150 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.8, Last appended by leader: 8, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.293048 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:52.337159 19705 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.268040 12494 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:52.296681 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:52.247496 19475 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:52.293354 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:52.292699 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:52.110213 19710 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.174s
I20260709 12:12:52.316087 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:52.316182 19582 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.316787 19597 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:52.318922 12675 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 14. Current term is 15. Ops: [14.9-14.9]
I20260709 12:12:52.319330 19642 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:52.266853 19541 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:52.320784 19697 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:52.321736 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:52.271577 19578 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:52.289121 19614 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.034s
I20260709 12:12:52.331327 19524 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.302690 19615 consensus_queue.cc:1059] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 12:12:52.333912 13334 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:12:52.334956 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:52.339953 12531 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:52.406543 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2972) took 1017 ms. Trace:
I20260709 12:12:52.407054 12531 rpcz_store.cc:276] 0709 12:12:51.389601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.389716 (+   115us) service_pool.cc:224] Handling call
0709 12:12:52.339556 (+949840us) spinlock_profiling.cc:243] Waited 1.74 ms on lock 0x7b5800002a98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbe9c 00007f14858fba5b 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:12:52.339765 (+   209us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:52.406192 (+ 66427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:52.406462 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4011648}
I20260709 12:12:52.344942 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:52.349400 19445 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:12:52.417043 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:52.417783 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:52.297987 19508 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:12:52.294055 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:52.349574 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:12:52.373427 12598 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2946) took 2141 ms (client timeout 2780 ms). Trace:
W20260709 12:12:52.426007 12598 rpcz_store.cc:269] 0709 12:12:50.231853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:50.670617 (+438764us) service_pool.cc:224] Handling call
0709 12:12:52.373021 (+1702404us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b54001176d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:52.373323 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2653113216}
W20260709 12:12:52.376889 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 80270.3us
I20260709 12:12:52.381008 19720 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.390952 19495 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.383598 19705 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:52.344625 19130 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:52.347905 19506 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.534s	user 0.011s	sys 0.000s
I20260709 12:12:52.392097 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:52.338795 19707 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:52.350728 19698 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:52.480822 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 146427us
I20260709 12:12:52.484140 13332 raft_consensus.cc:2376] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15. Re-sending same reply.
I20260709 12:12:52.411437 19697 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.471257 19482 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:12:52.480139 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 212165us
I20260709 12:12:52.498401 13302 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:52.433221 19723 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.501502 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:52.389740 19642 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.483525 19524 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:52.500427 19725 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:52.514459 19725 consensus_queue.cc:1059] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 13.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:52.515714 19725 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:52.516060 19725 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:12:52.516427 19725 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:52.516925 19610 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:12:52.519974 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3384) took 1230 ms. Trace:
I20260709 12:12:52.520526 12677 rpcz_store.cc:276] 0709 12:12:51.289795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.613900 (+324105us) service_pool.cc:224] Handling call
0709 12:12:51.614314 (+   414us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:52.011562 (+397248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:52.519641 (+508079us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b54003be554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:52.519891 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":911811968}
I20260709 12:12:52.495864 19130 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:52.523993 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:52.524643 19130 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:52.527777 19721 raft_consensus.cc:3055] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:52.528441 19721 raft_consensus.cc:740] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:52.529107 19732 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:52.530478 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:52.534317 12527 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
I20260709 12:12:52.500640 19629 raft_consensus.cc:697] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:52.557667 19629 consensus_queue.cc:237] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.559824 19443 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.562732 19355 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:52.563349 19461 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.529537 19721 consensus_queue.cc:260] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.10, Last appended by leader: 10, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:52.566025 19721 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:52.566256 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48476 (request call id 74) took 1145 ms. Trace:
I20260709 12:12:52.566409 19461 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:52.569849 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:52.570725 19715 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.299s	user 0.001s	sys 0.000s
W20260709 12:12:52.572669 19730 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:12:52.573832 12677 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 17: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:52.574978 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:52.576485 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:52.578454 12677 raft_consensus.cc:2393] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 17: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:52.578279 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:52.566835 12526 rpcz_store.cc:276] 0709 12:12:51.421069 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.438113 (+ 17044us) service_pool.cc:224] Handling call
0709 12:12:52.566053 (+1127940us) spinlock_profiling.cc:243] Waited 661 ms on lock 0x7b5400052b54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1a853 00007f148d9e860a 00007f1486f6e7d4 00007f1486f6e762 00007f1486f6e691 00007f1486f6e60c 00007f1486f6d8c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:52.566192 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1520167296}
W20260709 12:12:52.578720 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:52.601383 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:52.579342 19443 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:52.484195 19722 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:12:52.487208 19495 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:52.592775 19527 consensus_meta.cc:287] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.566s	user 0.008s	sys 0.004s
I20260709 12:12:52.484385 19720 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.535959 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 820) took 1135 ms. Trace:
W20260709 12:12:52.607906 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:52.623970 12603 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:52.487380 19506 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.604385 19482 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:52.629091 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:52.600855 19732 raft_consensus.cc:3055] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Stepping down as leader of term 16
W20260709 12:12:52.633293 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:52.604682 19693 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.520s	user 0.011s	sys 0.001s
I20260709 12:12:52.534102 19707 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:52.602761 19614 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.634737 12606 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:12:52.648914 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:52.507447 19697 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.651971 12598 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:12:52.652419 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:52.653188 12606 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:52.611488 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:52.494419 19698 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:52.653847 12600 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 18 index: 9. (term mismatch)
I20260709 12:12:52.654425 12600 pending_rounds.cc:85] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 8
I20260709 12:12:52.654713 12600 pending_rounds.cc:107] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 17.9
I20260709 12:12:52.655040 12600 raft_consensus.cc:2889] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:52.647017 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:52.656684 12599 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:52.660290 12604 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:52.632733 19625 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:52.683771 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 13 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:52.713244 12677 raft_consensus.cc:2376] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 13. Re-sending same reply.
I20260709 12:12:52.645881 19693 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.685043 19494 raft_consensus.cc:3055] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:52.686622 12597 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:12:52.687271 13335 raft_consensus.cc:2393] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:52.689656 19698 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:52.690318 19697 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:52.694196 19443 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:52.698506 19699 consensus_meta.cc:287] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.660s	user 0.004s	sys 0.007s
W20260709 12:12:52.692448 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:52.704632 19500 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.673s	user 0.011s	sys 0.001s
I20260709 12:12:52.710987 19642 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:52.659863 19614 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.510746 19723 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:52.636514 19732 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:52.714497 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:52.742303 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:52.744277 12599 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:52.645043 12527 rpcz_store.cc:276] 0709 12:12:51.400497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.126449 (+725952us) service_pool.cc:224] Handling call
0709 12:12:52.535863 (+409414us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:52.504065 19597 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.720520 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3380) took 1431 ms. Trace:
I20260709 12:12:52.754055 12679 rpcz_store.cc:276] 0709 12:12:51.288633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.565048 (+276415us) service_pool.cc:224] Handling call
0709 12:12:51.580482 (+ 15434us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:51.580776 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:12:51.597014 (+ 16238us) log.cc:844] Serialized 14 byte log entry
0709 12:12:52.713177 (+1116163us) spinlock_profiling.cc:243] Waited 36.9 ms on lock 0x7b180044f260. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a
0709 12:12:52.719664 (+  6487us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:52.719747 (+    83us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:12:52.719840 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:52.719923 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:52.720431 (+   508us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17208,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":6342,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":5776,"spinlock_wait_cycles":84912000,"thread_start_us":61082,"threads_started":1,"wal-append.queue_time_us":102484}
I20260709 12:12:52.756942 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:52.683073 19364 consensus_meta.cc:287] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.765s	user 0.015s	sys 0.001s
I20260709 12:12:52.731216 12606 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:12:52.696457 19495 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:52.760751 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:52.762087 12679 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:12:52.764178 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:52.766862 12679 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:52.770464 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:52.774080 12604 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:12:52.774855 12606 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:52.772567 12677 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: []
I20260709 12:12:52.718227 19494 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:52.779124 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3393) took 1481 ms. Trace:
I20260709 12:12:52.770293 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:52.782593 12677 rpcz_store.cc:276] 0709 12:12:51.297964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.771842 (+1473878us) service_pool.cc:224] Handling call
0709 12:12:52.772287 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:52.778764 (+  6477us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:52.779049 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:52.785029 12677 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: []
I20260709 12:12:52.723595 19625 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:52.795634 19720 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:52.796304 19625 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:52.740505 19699 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:52.733158 19390 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.740407 19500 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:52.799057 13330 raft_consensus.cc:2368] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16.
I20260709 12:12:52.800073 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:52.802103 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3391) took 1505 ms. Trace:
I20260709 12:12:52.802497 12679 rpcz_store.cc:276] 0709 12:12:51.296878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.766154 (+1469276us) service_pool.cc:224] Handling call
0709 12:12:52.766609 (+   455us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:52.801836 (+ 35227us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:52.802050 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:52.800467 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2929) took 1467 ms. Trace:
I20260709 12:12:52.807933 12677 rpcz_store.cc:276] 0709 12:12:51.333508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.784330 (+1450822us) service_pool.cc:224] Handling call
0709 12:12:52.784747 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:52.785634 (+   887us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:52.800403 (+ 14769us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:52.809870 12604 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:12:52.803714 19390 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.812299 12677 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 13 index: 8. (index mismatch)
I20260709 12:12:52.813060 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3396) took 1466 ms. Trace:
I20260709 12:12:52.813275 19390 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:52.813699 19725 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:52.814289 12677 rpcz_store.cc:276] 0709 12:12:51.346554 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.809601 (+1463047us) service_pool.cc:224] Handling call
0709 12:12:52.809918 (+   317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:52.812765 (+  2847us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:52.812959 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:52.815945 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 314345us
I20260709 12:12:52.818817 19614 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:52.820173 19732 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.739686 19443 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:52.823083 12569 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:52.839149 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2923) took 1575 ms. Trace:
I20260709 12:12:52.858664 12671 rpcz_store.cc:276] 0709 12:12:51.264102 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.264868 (+   766us) service_pool.cc:224] Handling call
0709 12:12:52.838859 (+1573991us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b54000aafd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:52.839061 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1840845696}
I20260709 12:12:52.859997 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:52.840045 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:52.843458 12571 leader_election.cc:400] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22.
I20260709 12:12:52.863449 12571 leader_election.cc:403] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:12:52.843869 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:52.841305 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:52.797991 19438 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:52.869561 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:52.870471 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:52.871591 12671 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 16: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:12:52.877080 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:52.876941 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:52.878633 12677 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: [20.9-20.9]
I20260709 12:12:52.878206 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:52.880735 12494 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:52.880772 12671 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 22: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:12:52.825275 19692 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.084s	user 0.001s	sys 0.002s
I20260709 12:12:52.882153 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:52.879688 12606 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:12:52.898383 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:52.894634 12679 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 13.
W20260709 12:12:52.895263 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:52.834584 19495 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.824882 19443 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:52.834436 19500 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:52.835498 19524 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:52.875694 19614 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:52.839938 19699 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:52.831148 19486 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:52.923900 19486 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.831416 19494 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:52.924372 19486 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:52.908656 19697 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:52.924286 19693 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:52.920485 19482 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:52.836411 19541 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:52.830950 19506 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:52.834782 19577 raft_consensus.cc:3055] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:52.833477 19720 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.825126 19625 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:12:52.902062 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:52.899518 12671 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:52.882220 19732 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:52.869132 19725 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:52.872260 19364 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22.
I20260709 12:12:52.918023 12571 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:52.879809 19390 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:52.926461 19697 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:52.928022 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:52.902529 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:52.900578 19582 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:52.918259 19737 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:12:52.928148 19629 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.011s
I20260709 12:12:52.934780 19581 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.939618 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:52.910372 19495 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:52.950222 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:52.950700 12679 raft_consensus.cc:2393] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 17: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:52.899024 19705 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.051s
I20260709 12:12:52.901782 19604 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 12:12:52.967285 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 2971) took 1578 ms. Trace:
I20260709 12:12:52.935042 19577 raft_consensus.cc:740] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:52.967296 12571 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:52.977841 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2977) took 1154 ms. Trace:
I20260709 12:12:52.979533 12600 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 15. Current term is 16. Ops: []
W20260709 12:12:52.979920 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 99263.3us
I20260709 12:12:52.980083 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:52.972482 12528 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 20.
I20260709 12:12:52.978313 12602 rpcz_store.cc:276] 0709 12:12:51.823063 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.823793 (+   730us) service_pool.cc:224] Handling call
0709 12:12:52.977603 (+1153810us) spinlock_profiling.cc:243] Waited 152 ms on lock 0x7b54001171d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:52.977760 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":349689088}
I20260709 12:12:52.984659 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:52.985333 12606 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
I20260709 12:12:52.954632 19597 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:52.953792 19582 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:52.953843 19625 raft_consensus.cc:697] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:52.966384 19529 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260709 12:12:53.004494 12602 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:52.928086 19524 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:53.005371 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 185214us
I20260709 12:12:53.003135 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:52.973012 19475 consensus_meta.cc:287] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.586s	user 0.007s	sys 0.005s
I20260709 12:12:52.946674 19720 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:52.983325 19578 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.579s	user 0.008s	sys 0.007s
I20260709 12:12:53.024165 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:52.971632 12529 rpcz_store.cc:276] 0709 12:12:51.389243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.389413 (+   170us) service_pool.cc:224] Handling call
0709 12:12:51.411071 (+ 21658us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:51.411300 (+   229us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:51.411324 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:51.421126 (+  9802us) log.cc:844] Serialized 26 byte log entry
0709 12:12:51.757722 (+336596us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:51.757786 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:51.758196 (+   410us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:52.824357 (+1066161us) spinlock_profiling.cc:243] Waited 6.77 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:52.954218 (+129861us) raft_consensus.cc:1685] finished
0709 12:12:52.954245 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:52.967184 (+ 12939us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":15579392,"thread_start_us":242896,"threads_started":1,"wal-append.queue_time_us":281881}
I20260709 12:12:53.022487 13302 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:53.033695 12671 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:52.984289 19506 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:53.011731 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:53.029712 12494 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:52.994673 12643 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:12:53.048409 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:52.987447 19614 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.051502 12605 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:12:53.051702 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:53.053041 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 58519.3us
I20260709 12:12:53.034746 12606 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:53.011078 19529 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.055198 12602 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:12:53.010236 19582 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:53.063741 12679 raft_consensus.cc:2410] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:12:52.971040 19581 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:53.071830 12671 raft_consensus.cc:2393] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 20: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:53.071376 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:53.075335 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:53.076399 12671 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:53.034868 19578 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:53.053498 19725 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:53.056699 19642 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:53.054226 19506 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.081157 12571 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:53.092010 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:53.083102 19443 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:12:53.061836 19390 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:12:53.064872 19529 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:53.007479 19610 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.075s
I20260709 12:12:53.015478 19597 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.099916 19506 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:53.035127 19737 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.043s
I20260709 12:12:53.066763 19482 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:53.194092 19482 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:53.195281 19482 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.034894 19693 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 16.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 12:12:53.033396 19475 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.091308 19736 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.089s
I20260709 12:12:53.199361 19693 raft_consensus.cc:3055] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:53.199765 12678 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:12:53.201092 19723 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:53.201277 19475 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:53.123579 19725 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 17 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:12:53.203333 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:53.202817 19725 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:53.094441 12494 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:12:53.205284 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 110945us
I20260709 12:12:53.207039 19524 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:53.207235 12494 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:53.083774 19304 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.208623 19304 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:53.199791 19693 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:53.212631 19304 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:53.065289 19614 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.095288 19364 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.213510 19693 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.214550 19364 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:53.215214 19743 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.288s	user 0.001s	sys 0.001s
I20260709 12:12:53.215173 19693 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:53.181900 19740 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 12:12:53.182461 19495 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:53.184494 19697 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:53.185374 19707 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.538s	user 0.016s	sys 0.002s
I20260709 12:12:53.073503 19738 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 12:12:53.125281 19527 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:12:53.190037 19747 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.124s	user 0.000s	sys 0.000s
I20260709 12:12:53.070604 19577 consensus_queue.cc:260] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.191942 19581 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:53.193099 19597 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:53.198153 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:53.124009 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:53.273291 12671 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 16
I20260709 12:12:53.216986 19364 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:53.217255 19736 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:12:53.217798 19705 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:53.218488 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:53.116351 19438 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 12:12:53.223260 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:53.282793 12679 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 15.
I20260709 12:12:53.283895 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:53.095039 19625 consensus_queue.cc:237] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:53.179394 19709 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:53.225736 19150 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.230736 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 2981) took 1402 ms. Trace:
I20260709 12:12:53.299401 12601 rpcz_store.cc:276] 0709 12:12:51.828752 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.872193 (+ 43441us) service_pool.cc:224] Handling call
0709 12:12:53.230555 (+1358362us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b540022ecd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:53.230664 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":298202368}
I20260709 12:12:53.235457 19733 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:53.237058 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2922) took 1991 ms. Trace:
I20260709 12:12:53.248085 13305 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:53.249150 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:53.249469 13302 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:53.300606 12674 rpcz_store.cc:276] 0709 12:12:51.245298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.263842 (+ 18544us) service_pool.cc:224] Handling call
0709 12:12:51.615160 (+351318us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:51.627964 (+ 12804us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:53.236769 (+1608805us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b5400013e50. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:53.236967 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":807244160}
W20260709 12:12:53.302042 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52668.5us
I20260709 12:12:53.258059 19443 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:53.267258 19527 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.284960 12643 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:53.301285 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 53310.3us
I20260709 12:12:53.255012 19604 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:12:53.282013 19705 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.276238 19614 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:53.314035 12671 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:12:53.331992 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:53.310657 19527 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:53.303020 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:53.311518 19580 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:53.319864 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:53.336287 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:53.336673 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 96991.8us
I20260709 12:12:53.308118 19707 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:53.321395 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:53.338970 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:53.354599 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:53.356700 12531 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
I20260709 12:12:53.357872 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:53.357101 12525 raft_consensus.cc:2376] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24. Re-sending same reply.
I20260709 12:12:53.341760 12599 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:12:53.344394 12527 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:53.308722 19736 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:53.359987 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:53.360694 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 75878.1us
I20260709 12:12:53.366925 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:53.369556 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:53.370790 12674 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 16: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:53.299013 19150 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.373396 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 744) took 1566 ms. Trace:
I20260709 12:12:53.373930 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:53.373837 12674 rpcz_store.cc:276] 0709 12:12:51.807509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:53.372614 (+1565105us) service_pool.cc:224] Handling call
0709 12:12:53.372873 (+   259us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:53.373064 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:53.373107 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:53.373152 (+    45us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:12:53.373196 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:53.373248 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:53.373342 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:53.375862 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:53.376204 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:53.378372 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:53.383005 12522 pending_rounds.cc:85] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 12
I20260709 12:12:53.383584 12522 pending_rounds.cc:107] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 18.13
I20260709 12:12:53.384097 12522 raft_consensus.cc:2889] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:53.385555 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:53.386365 12671 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:53.331053 19705 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:53.391686 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:53.385900 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:53.309406 19709 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.395196 12530 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:12:53.395637 19709 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:53.396255 12679 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:53.345680 19604 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:53.379895 19597 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:12:53.372677 19720 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:53.422793 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:53.423638 19736 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.433375 12679 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:53.390743 19524 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:53.313666 19733 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.404201 19580 raft_consensus.cc:697] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:12:53.444648 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 58811.3us
I20260709 12:12:53.444949 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:53.405735 19697 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:53.452569 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:53.372058 19737 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.058s
I20260709 12:12:53.435921 19725 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:53.390475 19614 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:12:53.457481 12530 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
W20260709 12:12:53.454682 19486 consensus_meta.cc:287] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.530s	user 0.009s	sys 0.008s
I20260709 12:12:53.465384 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:53.444291 19495 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:53.446110 19524 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.450783 19597 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:12:53.474637 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:53.463972 19725 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:53.383906 19443 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.477628 12531 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:53.471800 19763 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:53.485430 19763 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:53.485675 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:53.479207 19756 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.486517 19756 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.487047 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:53.488797 19756 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:53.488888 12530 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:53.488282 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:53.474207 19768 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:53.490149 19768 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.490578 19768 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:53.490732 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:53.492254 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:53.492900 12601 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:53.493839 19506 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.472999 19495 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.468103 19582 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.461374 19697 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.485857 19763 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:53.483247 19733 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:53.500660 12602 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 12:12:53.561388 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50382us
I20260709 12:12:53.510391 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:53.469664 19524 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:53.474359 19759 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.203s	user 0.001s	sys 0.001s
I20260709 12:12:53.565680 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 13 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:53.568260 12601 raft_consensus.cc:2410] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 18 because replica has last-logged OpId of term: 14 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 7.
I20260709 12:12:53.566200 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:53.564682 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:53.562958 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:53.496294 19506 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:53.576661 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:53.583758 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:53.584502 12604 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:53.604013 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:53.605335 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:53.472052 19486 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:53.523900 19582 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:53.609642 12604 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 17: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:53.500869 19597 raft_consensus.cc:697] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:53.613845 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:53.615491 12601 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:53.619262 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:53.612955 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:53.586378 19604 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:53.622900 19604 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.623517 19604 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:53.626171 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:53.628031 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:53.628098 19304 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.629009 19720 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:53.628762 19304 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.629772 19720 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:53.630188 19720 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.630524 19720 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:53.632164 19304 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:53.635324 19604 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:53.637902 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:53.656950 12526 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 20. Current term is 21. Ops: []
I20260709 12:12:53.498014 19495 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:53.643783 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:53.643945 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:53.645808 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:53.658768 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:53.603775 19738 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:53.569235 19693 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:53.609184 19739 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.645712 19506 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.651439 19709 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.615285 19529 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.673246 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:53.498423 19580 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 14.12, Last appended by leader: 12, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:53.675153 12523 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:12:53.660851 19614 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.660952 19738 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.662946 19771 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.665720 19150 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:53.692752 12672 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21. Ops: []
I20260709 12:12:53.700112 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:53.705526 12676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 3394) took 2394 ms (client timeout 2116 ms). Trace:
W20260709 12:12:53.710095 12676 rpcz_store.cc:269] 0709 12:12:51.330171 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.491562 (+161391us) service_pool.cc:224] Handling call
0709 12:12:53.692402 (+2200840us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b54003bd654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:53.692541 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":583790336}
I20260709 12:12:53.711638 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:53.712736 12523 raft_consensus.cc:2368] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21.
I20260709 12:12:53.690658 19738 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:53.720228 12673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2927) took 2454 ms (client timeout 2005 ms). Trace:
W20260709 12:12:53.725270 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 46868.3us
I20260709 12:12:53.685531 19724 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:53.725612 12673 rpcz_store.cc:269] 0709 12:12:51.266036 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.347085 (+ 81049us) service_pool.cc:224] Handling call
0709 12:12:53.719941 (+2372856us) spinlock_profiling.cc:243] Waited 337 ms on lock 0x7b54000ac8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:53.720130 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":774485376}
I20260709 12:12:53.668109 19693 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:53.672386 19506 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.721335 12671 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:53.731122 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:53.730719 13305 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:53.733711 13305 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:53.679968 19709 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:53.691423 19614 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.745776 12646 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:53.671103 19739 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:53.701531 19771 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.748862 12646 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:53.738976 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:53.742988 19773 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:12:53.738880 19699 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:12:53.756790 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:53.758436 19781 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:53.741724 19304 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.762574 19304 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:53.765234 19642 consensus_meta.cc:287] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.670s	user 0.008s	sys 0.004s
I20260709 12:12:53.765707 19642 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:53.766465 19486 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:53.771849 19614 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:53.730626 19724 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.775815 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:53.776576 12522 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:53.777489 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:53.777684 19709 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:53.778286 19624 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:53.794656 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 45870.6us
I20260709 12:12:53.870337 12646 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:53.871770 12646 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:53.782573 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:53.778698 19724 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:53.782027 19739 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:53.779762 19604 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:53.733755 19720 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.783777 19771 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:53.714488 19597 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:53.876842 19624 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.892971 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:53.894409 19697 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:53.801910 19693 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:53.894934 19624 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:53.798801 19614 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:12:53.799389 19500 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:53.896361 19500 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:53.896055 19705 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.897368 19500 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.779683 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:53.806692 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 67789.8us
I20260709 12:12:53.801851 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:53.755173 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:53.899082 12529 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:53.751701 19725 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:53.879637 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:53.880352 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:53.880704 12646 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:53.885315 19763 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.886487 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:53.898949 13305 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:53.870317 13332 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:53.878454 19779 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:12:53.900635 19500 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:53.881779 19781 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:12:53.900990 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3388) took 2610 ms. Trace:
I20260709 12:12:53.899593 12673 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:53.877337 19707 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:12:53.910499 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 50161.3us
I20260709 12:12:53.903945 12601 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:53.917534 12569 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:53.923722 13331 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:12:53.924986 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:53.915539 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:53.927134 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:53.929646 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:53.924602 12675 rpcz_store.cc:276] 0709 12:12:51.290890 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.627059 (+336169us) service_pool.cc:224] Handling call
0709 12:12:51.627552 (+   493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:52.378855 (+751303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:53.900717 (+1521862us) spinlock_profiling.cc:243] Waited 691 ms on lock 0x7b54003d4ad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:53.900929 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1589364608}
I20260709 12:12:53.925223 12527 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:12:53.930086 13334 raft_consensus.cc:2410] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:12:53.915284 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:53.930691 12569 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:53.931990 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:53.882051 19709 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:53.933626 12569 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:53.934965 12646 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:53.942469 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:53.942292 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:53.948103 12597 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
W20260709 12:12:53.948547 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:53.950474 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:53.951186 12673 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 17: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:53.924733 19614 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:12:53.925356 19779 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:53.938920 19624 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:53.954321 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:53.922464 19725 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.938760 19697 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:53.925822 19604 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.034s
I20260709 12:12:53.925424 19739 consensus_queue.cc:1059] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 12:12:53.938903 19699 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:53.944662 19724 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:12:53.973701 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:53.983547 12673 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
I20260709 12:12:53.901609 19780 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.065s
W20260709 12:12:53.955152 19527 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.593s	user 0.012s	sys 0.000s
I20260709 12:12:53.943902 19625 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:53.977288 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:53.952901 19781 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:12:53.961390 19705 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:53.930500 19768 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:53.961687 19779 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:53.991957 19781 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:53.992789 19780 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:12:53.988672 19625 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:53.977743 19725 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:53.999265 19781 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:54.019894 19763 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:54.008359 19780 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:54.023049 19725 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:54.024367 19738 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:54.027017 19738 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:54.030584 19776 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.295s	user 0.000s	sys 0.003s
W20260709 12:12:54.103982 12421 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47524 (request call id 101) took 17947 ms (client timeout 15000 ms). Trace:
I20260709 12:12:54.117945 12522 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:54.120191 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.123297 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.125546 12525 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:12:54.127866 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.128985 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.133674 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.135933 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.137686 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.132885 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.139487 12526 raft_consensus.cc:2393] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 21: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:54.140906 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.142295 12526 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 20.
I20260709 12:12:54.144632 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.144078 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.197851 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:54.146256 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.201493 13333 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:54.125803 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.203516 12527 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:12:54.204735 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:54.205660 13339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 691) took 1227 ms. Trace:
I20260709 12:12:54.129642 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:54.150667 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 215655us
I20260709 12:12:54.152361 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:54.105079 12421 rpcz_store.cc:269] 0709 12:12:36.156694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:36.158404 (+  1710us) service_pool.cc:224] Handling call
0709 12:12:54.103405 (+17945001us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:50.082585 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:50.198187 (+115602us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:50.198255 (+    68us) write_op.cc:435] Acquired schema lock
0709 12:12:50.198278 (+    23us) tablet.cc:662] Decoding operations
0709 12:12:50.199705 (+  1427us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:50.199830 (+   125us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:50.199868 (+    38us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:12:50.201667 (+  1799us) tablet.cc:701] Row locks acquired
0709 12:12:50.201723 (+    56us) write_op.cc:260] PREPARE: finished
0709 12:12:50.201918 (+   195us) write_op.cc:270] Start()
0709 12:12:50.202199 (+   281us) write_op.cc:276] Timestamp: P: 1783599170201841 usec, L: 0
0709 12:12:50.202224 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:12:50.334193 (+131969us) log.cc:844] Serialized 5196 byte log entry
0709 12:12:50.771267 (+437074us) op_driver.cc:464] REPLICATION: finished
0709 12:12:50.916199 (+144932us) write_op.cc:301] APPLY: starting
0709 12:12:50.922428 (+  6229us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:50.983866 (+ 61438us) tablet.cc:1370] finished BulkCheckPresence
0709 12:12:50.984005 (+   139us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:12:51.375300 (+391295us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:12:51.375634 (+   334us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0709 12:12:51.375688 (+    54us) write_op.cc:312] APPLY: finished
0709 12:12:51.376075 (+   387us) log.cc:844] Serialized 121 byte log entry
0709 12:12:51.376267 (+   192us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:51.376449 (+   182us) write_op.cc:454] Released schema lock
0709 12:12:51.376508 (+    59us) write_op.cc:341] FINISH: Updating metrics
0709 12:12:51.377028 (+   520us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:12:51.377061 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1066029,"reported_tablets":21,"child_traces":[["op",{"apply.queue_time_us":132507,"apply.run_cpu_time_us":26808,"apply.run_wall_time_us":463393,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58389,"num_ops":13,"prepare.queue_time_us":467948,"prepare.run_cpu_time_us":19225,"prepare.run_wall_time_us":369749,"raft.queue_time_us":93753,"raft.run_cpu_time_us":15628,"raft.run_wall_time_us":198938,"replication_time_us":545588,"thread_start_us":517496,"threads_started":4,"wal-append.queue_time_us":229643,"wal-append.run_cpu_time_us":9351,"wal-append.run_wall_time_us":1809940}]]}
I20260709 12:12:54.200210 12526 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
W20260709 12:12:54.157930 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:54.162225 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:54.163277 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:54.206074 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:54.216802 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:54.163931 12528 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 819) took 2763 ms (client timeout 1788 ms). Trace:
I20260709 12:12:54.163857 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3397) took 2812 ms. Trace:
I20260709 12:12:54.167766 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:54.169866 12678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 3408) took 2139 ms (client timeout 1706 ms). Trace:
I20260709 12:12:54.214627 12680 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 20: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:54.207677 12525 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21. Ops: []
I20260709 12:12:54.174625 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2943) took 1298 ms. Trace:
I20260709 12:12:54.179246 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:54.181339 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2926) took 2915 ms. Trace:
I20260709 12:12:54.184787 12598 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:54.187248 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.216831 12527 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 17: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:54.187703 12604 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:12:54.188639 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3399) took 2224 ms. Trace:
W20260709 12:12:54.189849 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:54.189970 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:54.195827 12603 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:12:54.217588 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.173048 12602 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:54.213842 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:54.212364 12526 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 12:12:54.218794 12528 rpcz_store.cc:269] 0709 12:12:51.400193 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.839713 (+1439520us) service_pool.cc:224] Handling call
0709 12:12:54.163453 (+1323740us) spinlock_profiling.cc:243] Waited 66.5 ms on lock 0x7b5400015754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:54.163718 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":152887552}
I20260709 12:12:54.206189 13339 rpcz_store.cc:276] 0709 12:12:52.977878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.978022 (+   144us) service_pool.cc:224] Handling call
0709 12:12:54.205131 (+1227109us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b54004781d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:54.205414 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":791650176}
W20260709 12:12:54.219594 12678 rpcz_store.cc:269] 0709 12:12:52.030725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.054580 (+ 23855us) service_pool.cc:224] Handling call
0709 12:12:54.169430 (+2114850us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b54003be054. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:54.169660 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2616924032}
I20260709 12:12:54.220872 12679 rpcz_store.cc:276] 0709 12:12:52.876079 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:53.283594 (+407515us) service_pool.cc:224] Handling call
0709 12:12:54.174196 (+890602us) spinlock_profiling.cc:243] Waited 112 ms on lock 0x7b54000131d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:54.174419 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":256913152}
I20260709 12:12:54.208413 12646 leader_election.cc:400] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21.
I20260709 12:12:54.258611 12646 leader_election.cc:403] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:12:54.222689 12671 rpcz_store.cc:276] 0709 12:12:51.963621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:53.422862 (+1459241us) service_pool.cc:224] Handling call
0709 12:12:53.721050 (+298188us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:53.731687 (+ 10637us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:54.188170 (+456483us) spinlock_profiling.cc:243] Waited 274 ms on lock 0x7b54000ac8d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:54.188457 (+   287us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":629833088}
I20260709 12:12:54.218915 12677 rpcz_store.cc:276] 0709 12:12:51.351775 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.815324 (+1463549us) service_pool.cc:224] Handling call
0709 12:12:52.864272 (+ 48948us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:52.879136 (+ 14864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:54.163366 (+1284230us) spinlock_profiling.cc:243] Waited 48.5 ms on lock 0x7b54000aafd0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:54.163641 (+   275us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":111577344}
W20260709 12:12:54.250412 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 95697.3us
I20260709 12:12:54.221745 12672 rpcz_store.cc:276] 0709 12:12:51.265768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.542662 (+276894us) service_pool.cc:224] Handling call
0709 12:12:53.692536 (+2149874us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:53.696647 (+  4111us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:54.180806 (+484159us) spinlock_profiling.cc:243] Waited 2.15 s on lock 0x7b54003bd650. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:54.181126 (+   320us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4935006592}
I20260709 12:12:54.260282 12599 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:12:54.265795 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:54.266434 12531 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:54.266028 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:54.011075 19625 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:54.277582 19495 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:53.990968 19527 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:54.278730 19495 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:54.281076 19527 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:54.282559 12680 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:54.282797 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:54.283929 12598 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:53.991536 19524 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:54.290948 12605 raft_consensus.cc:2368] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19.
I20260709 12:12:54.292351 19524 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:54.293735 12674 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:54.012208 19707 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 24 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:54.296600 19792 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:54.296926 19707 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:54.297783 19699 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.298789 12673 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:54.298748 19699 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:54.207301 19150 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:54.296933 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 118179us
I20260709 12:12:54.303964 19150 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:54.304836 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:54.269208 19720 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:54.307648 12604 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:54.297597 19791 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:54.310377 19791 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:54.310549 19697 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:54.310895 19791 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:54.313449 19736 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:54.316970 19364 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:54.317852 19364 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:54.318586 19705 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:54.269335 19357 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:54.320984 19357 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.321745 19357 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:54.324465 19357 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:54.325048 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:54.326030 12671 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:12:54.291420 12422 catalog_manager.cc:5697] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 8 to 14. New cstate: current_term: 14 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:54.327441 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:54.320343 19705 raft_consensus.cc:3055] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:54.367170 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:54.383963 12599 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:54.369095 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:54.358918 19495 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:54.368642 19357 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:54.397696 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.392222 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.360054 19705 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:54.395561 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:54.404532 12679 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:54.405797 19771 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.406515 19771 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:54.409147 19771 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:54.418258 12526 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:12:54.418958 19514 raft_consensus.cc:3055] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:54.419265 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:54.420353 12672 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 19: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:54.421521 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:54.422353 19357 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:54.419427 19514 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:54.427553 19514 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:54.429020 19514 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:54.428611 12422 catalog_manager.cc:5697] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 9 to 12, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 12 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
W20260709 12:12:54.434396 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:54.435858 19364 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:54.436663 12522 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:54.436798 19150 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:54.444363 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:54.447669 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 239591us
I20260709 12:12:54.448776 12646 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:54.450557 19524 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:12:54.451012 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:54.297595 19792 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:54.429126 19781 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:54.433849 12597 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:12:54.464792 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:54.467373 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:54.468564 12522 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 17: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:54.472709 12598 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:12:54.460386 19150 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:54.485029 12673 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
W20260709 12:12:54.491958 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46306.1us
I20260709 12:12:54.504161 12422 catalog_manager.cc:5697] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 9 to 12, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 12 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:12:54.506330 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.506583 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:54.487185 12671 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:54.514214 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:54.514384 12599 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:12:54.386482 12525 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:12:54.518455 12597 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:54.491257 19150 raft_consensus.cc:697] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:54.527711 19150 consensus_queue.cc:237] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:54.531999 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.532982 12522 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:12:54.533634 19659 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:54.538983 19705 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:54.540566 19705 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:54.534036 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.549515 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:54.550209 12599 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:12:54.545437 12525 raft_consensus.cc:606] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:54.555387 12525 raft_consensus.cc:614] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:12:54.487082 19527 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.566102 12679 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
W20260709 12:12:54.562623 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:54.559752 12672 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:54.568563 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:54.532090 19636 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:54.577808 19527 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:54.579295 12598 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:54.579535 19697 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:54.580348 19527 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:54.580678 19781 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:54.581465 19781 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:54.582680 19781 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:54.583400 12496 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:54.584721 19693 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:12:54.585445 19693 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:54.585604 19781 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:54.580685 12604 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:54.587206 19693 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:54.590183 19618 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.591018 19618 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:54.591907 19795 consensus_queue.cc:1059] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 12.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:54.594910 19491 raft_consensus.cc:3055] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:12:54.595419 19491 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:54.596491 19491 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:54.593641 19618 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:54.599531 19699 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:54.600978 12571 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:54.601172 19792 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:54.602427 19357 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:54.567790 19720 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21.
W20260709 12:12:54.604260 13330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 683) took 2311 ms (client timeout 2051 ms). Trace:
W20260709 12:12:54.605001 13330 rpcz_store.cc:269] 0709 12:12:52.292305 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:52.292428 (+   123us) service_pool.cc:224] Handling call
0709 12:12:54.603788 (+2311360us) spinlock_profiling.cc:243] Waited 424 ms on lock 0x7b540063ecd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:54.604045 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":974062848}
I20260709 12:12:54.606403 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 736) took 1232 ms. Trace:
I20260709 12:12:54.608045 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:54.610050 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:54.611229 19527 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:54.612121 19627 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:12:54.612349 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:54.612861 19500 consensus_meta.cc:287] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.685s	user 0.013s	sys 0.000s
I20260709 12:12:54.612872 19527 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:54.613703 12646 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:54.613149 12603 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
W20260709 12:12:54.615181 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 61407.7us
I20260709 12:12:54.616427 19304 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:54.615645 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:54.617408 12603 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:12:54.618633 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:54.621608 12603 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: []
I20260709 12:12:54.602370 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:54.623112 19699 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:54.623545 19699 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:54.619443 19527 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 16.11, Last appended by leader: 11, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:54.623970 19699 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:54.612573 19627 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:54.606820 12600 rpcz_store.cc:276] 0709 12:12:53.373644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:53.932873 (+559229us) service_pool.cc:224] Handling call
0709 12:12:53.956146 (+ 23273us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:53.957460 (+  1314us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:12:53.958561 (+  1101us) log.cc:844] Serialized 14 byte log entry
0709 12:12:54.294991 (+336430us) log.cc:844] Serialized 14 byte log entry
0709 12:12:54.295283 (+   292us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:54.296290 (+  1007us) log.cc:844] Serialized 26 byte log entry
0709 12:12:54.296483 (+   193us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:12:54.296519 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:54.296617 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:54.595548 (+298931us) spinlock_profiling.cc:243] Waited 3.66 ms on lock 0x7b180030e980. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:12:54.602822 (+  7274us) spinlock_profiling.cc:243] Waited 29.1 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:12:54.605964 (+  3142us) raft_consensus.cc:1685] finished
0709 12:12:54.605986 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:54.606193 (+   207us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":302389,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":72,"raft.queue_time_us":4545,"raft.run_cpu_time_us":453,"raft.run_wall_time_us":9924,"spinlock_wait_cycles":75387136,"thread_start_us":138289,"threads_started":1,"wal-append.queue_time_us":587191}
I20260709 12:12:54.616456 12646 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:12:54.593954 19745 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.499s	user 0.001s	sys 0.005s
I20260709 12:12:54.644418 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.634048 19627 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:54.660029 12522 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:12:54.673422 12673 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 18 index: 9. (index mismatch)
I20260709 12:12:54.675105 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.597985 19791 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:54.682282 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3402) took 2711 ms. Trace:
I20260709 12:12:54.655471 19720 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:12:54.683859 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:54.640312 19526 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:54.685935 12673 rpcz_store.cc:276] 0709 12:12:51.964469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:54.568653 (+2604184us) service_pool.cc:224] Handling call
0709 12:12:54.652242 (+ 83589us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:54.673872 (+ 21630us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:54.675915 (+  2043us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:54.688087 12672 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:12:54.668511 19781 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:54.691799 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3410) took 2644 ms. Trace:
I20260709 12:12:54.694578 12672 rpcz_store.cc:276] 0709 12:12:52.046972 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:54.570358 (+2523386us) service_pool.cc:224] Handling call
0709 12:12:54.669334 (+ 98976us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:54.691428 (+ 22094us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:54.691723 (+   295us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:54.687801 19618 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:54.698422 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:54.698410 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 43874.2us
W20260709 12:12:54.701610 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:54.704633 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:54.716307 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:54.735339 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:54.732769 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:54.725371 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 109083us
I20260709 12:12:54.742208 12603 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:12:54.718583 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:54.706058 13302 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:54.747179 12597 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:54.749046 12598 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:54.748301 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:54.706410 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:54.717885 19791 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:54.744232 19500 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:54.756924 19618 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:54.757974 12646 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:54.759846 12673 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:12:54.760883 12672 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 17: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:12:54.764935 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 59052.5us
I20260709 12:12:54.767781 19514 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:54.768296 19514 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:54.775269 12600 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:12:54.768754 19514 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:54.780270 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:54.760718 19500 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:54.718544 19581 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 12:12:54.784273 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:54.790249 19624 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:54.798983 19696 consensus_queue.cc:1059] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.021s
W20260709 12:12:54.799976 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:54.799463 19624 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:54.801688 19707 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:54.802239 19707 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:54.802639 19707 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:12:54.802762 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:54.804386 19806 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:54.804857 19806 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:54.805233 19806 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:54.806468 19779 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.742s	user 0.003s	sys 0.013s
I20260709 12:12:54.807061 19779 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:54.808511 13334 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:12:54.810057 19781 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260709 12:12:54.811388 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:54.811980 19781 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 8, Time since last communication: 0.000s
W20260709 12:12:54.814327 19807 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 12:12:54.814672 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:54.815706 13335 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:12:54.817011 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:54.817606 13330 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:12:54.817698 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.818437 12527 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:54.819109 13338 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:54.790619 19500 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:54.820016 12598 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:12:54.826609 12422 catalog_manager.cc:5697] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 7 to 13, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 13 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:54.827883 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3398) took 2864 ms. Trace:
I20260709 12:12:54.828307 12674 rpcz_store.cc:276] 0709 12:12:51.963286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:53.383922 (+1420636us) service_pool.cc:224] Handling call
0709 12:12:53.384274 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:54.294354 (+910080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:54.827565 (+533211us) spinlock_profiling.cc:243] Waited 909 ms on lock 0x7b54002c5454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:54.827807 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2090369408}
I20260709 12:12:54.828791 12422 catalog_manager.cc:5697] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 9 to 14. New cstate: current_term: 14 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:54.831504 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.833552 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:54.836097 19627 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:54.836649 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 16 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:54.837344 12678 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:54.840867 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:54.841392 12598 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:54.841630 12602 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:54.843421 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:54.843523 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:54.843973 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:54.844347 12677 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:54.844919 12673 raft_consensus.cc:2410] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 20 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:12:54.846326 12673 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:12:54.846942 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3412) took 2732 ms. Trace:
I20260709 12:12:54.847283 12673 rpcz_store.cc:276] 0709 12:12:52.114896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:54.845559 (+2730663us) service_pool.cc:224] Handling call
0709 12:12:54.845891 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:54.846678 (+   787us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:54.846864 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:54.848825 12673 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 15. Current term is 16. Ops: [15.10-15.10]
I20260709 12:12:54.849505 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 749) took 2409 ms. Trace:
I20260709 12:12:54.829717 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:54.875063 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:54.851106 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3332) took 1405 ms. Trace:
I20260709 12:12:54.905810 12674 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:12:54.849841 12673 rpcz_store.cc:276] 0709 12:12:52.440064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:54.848106 (+2408042us) service_pool.cc:224] Handling call
0709 12:12:54.848535 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:54.849244 (+   709us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:54.849459 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:54.849743 19696 raft_consensus.cc:3055] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:54.956552 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:54.924733 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:54.915969 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 72646.1us
I20260709 12:12:54.845875 12677 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:54.867295 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:54.764124 19720 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.060s
I20260709 12:12:54.939920 12530 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
W20260709 12:12:54.972481 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:54.974030 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:54.916628 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:54.978479 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 67673.2us
W20260709 12:12:54.863507 19767 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.062s	user 0.001s	sys 0.001s
I20260709 12:12:54.864389 19527 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:54.939611 19562 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:54.980497 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3411) took 2922 ms. Trace:
I20260709 12:12:54.982831 12598 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:12:54.987711 12677 rpcz_store.cc:276] 0709 12:12:52.055614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:54.845237 (+2789623us) service_pool.cc:224] Handling call
0709 12:12:54.845604 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:54.977287 (+131683us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:54.977538 (+   251us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:54.983875 13331 raft_consensus.cc:2410] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 17 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:12:54.824220 19779 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:54.813862 19781 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.102s
I20260709 12:12:54.856885 19808 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.866428 19625 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:54.907562 12601 rpcz_store.cc:276] 0709 12:12:53.378006 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:53.939544 (+561538us) service_pool.cc:224] Handling call
0709 12:12:53.957756 (+ 18212us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:54.452933 (+495177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:54.460747 (+  7814us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:54.559013 (+ 98266us) log.cc:844] Serialized 26 byte log entry
0709 12:12:54.749703 (+190690us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:54.749738 (+    35us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:54.749812 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:54.774385 (+ 24573us) raft_consensus.cc:1685] finished
0709 12:12:54.774409 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:54.782897 (+  8488us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":21929,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":4061,"thread_start_us":561153,"threads_started":2,"wal-append.queue_time_us":179993}
I20260709 12:12:54.984570 12530 raft_consensus.cc:606] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:55.032591 12572 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:54.844467 19443 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 17.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.269s
I20260709 12:12:54.869519 19529 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:55.031013 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:54.944363 19494 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:55.048557 13334 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38482 (request call id 698) took 1085 ms. Trace:
W20260709 12:12:55.003072 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 283577us
I20260709 12:12:55.121086 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:55.012522 12604 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:54.978361 19150 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:55.131177 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3323) took 1891 ms. Trace:
I20260709 12:12:55.005116 12600 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
W20260709 12:12:55.104681 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.007719 12494 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:54.855784 19526 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:12:55.101142 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.042567 12569 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:55.049887 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3407) took 3036 ms. Trace:
W20260709 12:12:55.018412 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 265511us
I20260709 12:12:55.118799 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:54.998070 12676 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:12:54.856367 19724 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:12:54.998888 12605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3319) took 1782 ms (client timeout 1574 ms). Trace:
I20260709 12:12:55.010845 19764 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.042789 19763 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 16.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.117s
I20260709 12:12:55.031930 19697 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:55.117133 19527 raft_consensus.cc:697] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:55.024616 19506 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:55.051987 19524 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:12:55.003537 19577 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:55.047796 19491 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:12:55.160104 19642 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:54.917917 19696 raft_consensus.cc:740] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:55.151113 13334 rpcz_store.cc:276] 0709 12:12:53.968433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:53.968710 (+   277us) service_pool.cc:224] Handling call
0709 12:12:53.969348 (+   638us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:54.808938 (+839590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.048177 (+239239us) spinlock_profiling.cc:243] Waited 839 ms on lock 0x7b54001b49d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:55.048467 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1929020288}
I20260709 12:12:54.990625 12524 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:12:55.186354 12678 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
W20260709 12:12:54.993202 19749 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.897s	user 0.000s	sys 0.007s
I20260709 12:12:55.187462 12679 rpcz_store.cc:276] 0709 12:12:52.013982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:54.576153 (+2562171us) service_pool.cc:224] Handling call
0709 12:12:54.657662 (+ 81509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:54.698581 (+ 40919us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:12:54.706501 (+  7920us) log.cc:844] Serialized 14 byte log entry
0709 12:12:54.786581 (+ 80080us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:54.850530 (+ 63949us) log.cc:844] Serialized 26 byte log entry
0709 12:12:54.850731 (+   201us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:54.850780 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:54.850878 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:55.049587 (+198709us) raft_consensus.cc:1685] finished
0709 12:12:55.049618 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:55.049781 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1105,"prepare.queue_time_us":27914,"prepare.run_cpu_time_us":82,"prepare.run_wall_time_us":7633,"raft.queue_time_us":15386,"raft.run_cpu_time_us":348,"raft.run_wall_time_us":17518,"thread_start_us":73369,"threads_started":1,"wal-append.queue_time_us":340315}
I20260709 12:12:55.248090 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:55.156581 19150 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:54.990430 19032 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:55.012660 19624 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:54.994910 19768 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:55.005947 19723 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:12:55.267772 19723 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:12:55.269429 12676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2948) took 1766 ms (client timeout 1596 ms). Trace:
W20260709 12:12:55.161975 19699 consensus_meta.cc:287] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.537s	user 0.013s	sys 0.000s
W20260709 12:12:55.182137 12605 rpcz_store.cc:269] 0709 12:12:53.216943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:53.218188 (+  1245us) service_pool.cc:224] Handling call
0709 12:12:54.998620 (+1780432us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b540022ecd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:54.998786 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2324440448}
I20260709 12:12:55.001748 19762 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:12:55.001187 19618 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:55.011323 19627 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:55.251811 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 209349us
I20260709 12:12:55.271320 19699 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:55.273417 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3335) took 1059 ms. Trace:
I20260709 12:12:55.276085 12569 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:55.257539 19445 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:55.276969 19445 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:55.277598 19445 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:55.278328 19791 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:55.278848 19791 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:55.279233 19791 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:55.280844 19506 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.258383 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:55.259164 19443 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:55.282594 19443 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:55.283079 19443 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:55.263949 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:55.284677 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:55.265702 19032 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:55.286214 19769 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:54.942997 19806 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:55.286906 19769 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:55.287801 19539 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:55.268959 19723 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:55.289459 19806 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:55.290478 19624 consensus_queue.cc:1059] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.013s
I20260709 12:12:55.290943 19763 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:55.291601 19763 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:55.291997 19624 raft_consensus.cc:3055] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:12:55.292413 19624 raft_consensus.cc:740] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:55.292759 19529 raft_consensus.cc:3055] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:55.293355 19529 raft_consensus.cc:740] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:55.290196 19723 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:55.274484 19762 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:55.295646 19762 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:12:55.269994 12676 rpcz_store.cc:269] 0709 12:12:53.502523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:53.711297 (+208774us) service_pool.cc:224] Handling call
0709 12:12:55.269095 (+1557798us) spinlock_profiling.cc:243] Waited 1.29 s on lock 0x7b54000ac3d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:55.269285 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2956360960}
I20260709 12:12:55.293532 19624 consensus_queue.cc:260] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 15.11, Last appended by leader: 11, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:55.285003 12679 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:55.285781 12673 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:55.274075 12599 rpcz_store.cc:276] 0709 12:12:54.214390 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:54.559540 (+345150us) service_pool.cc:224] Handling call
0709 12:12:54.587793 (+ 28253us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:54.588747 (+   954us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:54.588786 (+    39us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:54.589488 (+   702us) log.cc:844] Serialized 52 byte log entry
0709 12:12:54.753077 (+163589us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:54.753118 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:54.753193 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:55.273058 (+519865us) raft_consensus.cc:1685] finished
0709 12:12:55.273092 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:55.273332 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10064,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":124,"thread_start_us":153992,"threads_started":1,"wal-append.queue_time_us":551765}
I20260709 12:12:55.281692 19506 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:55.301476 19529 consensus_queue.cc:260] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:55.039146 12422 catalog_manager.cc:5697] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 12 to 15. New cstate: current_term: 15 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:12:55.273957 19699 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:55.235749 19798 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:12:55.310667 19798 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:55.312116 19798 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.9, Last appended by leader: 9, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:55.317755 12524 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 2981) took 3555 ms (client timeout 2228 ms). Trace:
I20260709 12:12:55.318740 19032 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:55.318373 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 60092.3us
W20260709 12:12:55.318334 12524 rpcz_store.cc:269] 0709 12:12:51.762573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:51.762681 (+   108us) service_pool.cc:224] Handling call
0709 12:12:55.317417 (+3554736us) spinlock_profiling.cc:243] Waited 3.21 s on lock 0x7b540038e5d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:55.317635 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7388591872}
I20260709 12:12:55.307348 12422 catalog_manager.cc:5697] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 7 to 14. New cstate: current_term: 14 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:12:55.320418 19625 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:55.321301 19529 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:55.304425 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:55.323254 19810 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.521s	user 0.003s	sys 0.001s
W20260709 12:12:55.323735 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 291345us
I20260709 12:12:55.157049 12606 rpcz_store.cc:276] 0709 12:12:53.239622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:53.271252 (+ 31630us) service_pool.cc:224] Handling call
0709 12:12:53.271563 (+   311us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:53.271997 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:53.272059 (+    62us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:53.272767 (+   708us) log.cc:844] Serialized 52 byte log entry
0709 12:12:53.371685 (+ 98918us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:53.398805 (+ 27120us) log.cc:844] Serialized 14 byte log entry
0709 12:12:53.425240 (+ 26435us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:53.425349 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:54.025529 (+600180us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b180045a300. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485933522 00007f14859357c1
0709 12:12:54.027202 (+  1673us) spinlock_profiling.cc:243] Waited 5.35 ms on lock 0x7b180045a300. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485933522 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:12:54.799565 (+772363us) spinlock_profiling.cc:243] Waited 5.61 ms on lock 0x7b180045a300. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:12:55.130679 (+331114us) raft_consensus.cc:1685] finished
0709 12:12:55.130703 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:55.130944 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10306,"prepare.queue_time_us":35107,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":53,"raft.queue_time_us":43517,"raft.run_cpu_time_us":306,"raft.run_wall_time_us":8956,"spinlock_wait_cycles":29290880,"thread_start_us":52135,"threads_started":1,"wal-append.queue_time_us":139896}
W20260709 12:12:55.326663 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 319061us
W20260709 12:12:55.328035 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 88076.5us
I20260709 12:12:55.345971 13302 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:55.346086 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 128654us
I20260709 12:12:55.347060 12676 raft_consensus.cc:2410] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:12:55.269774 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3405) took 3304 ms. Trace:
I20260709 12:12:55.347671 12671 rpcz_store.cc:276] 0709 12:12:51.965263 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.240315 (+3275052us) service_pool.cc:224] Handling call
0709 12:12:55.269048 (+ 28733us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:55.269393 (+   345us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:55.269426 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:55.269478 (+    52us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:12:55.269534 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.269596 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:55.269715 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:55.348582 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:55.329977 12601 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 13 index: 11. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:12:55.245509 12677 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:55.350112 12680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s	user 0.048s	sys 0.088s
I20260709 12:12:55.240512 19524 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:55.325320 19150 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:55.327575 19624 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:12:55.350500 12680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s	user 0.048s	sys 0.088s
I20260709 12:12:55.282233 19582 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.027985 19500 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:54.996829 19780 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:55.365242 19733 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:12:55.370759 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:55.251204 19764 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:55.358556 19699 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election won for term 17
W20260709 12:12:55.380399 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.363093 19798 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:55.393288 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:55.250612 19808 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:55.377492 19705 consensus_meta.cc:287] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.836s	user 0.012s	sys 0.005s
I20260709 12:12:55.404595 12646 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:55.338508 19697 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:55.412987 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 64516.4us
W20260709 12:12:55.413278 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 51501.7us
W20260709 12:12:55.414839 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.428278 19769 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:55.395908 19699 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:55.452518 12494 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:55.468201 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:12:55.471194 12604 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:12:55.474200 19768 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.474884 19768 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:55.477393 19768 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:55.479280 12671 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:55.481230 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 14 candidate_status { last_received { term: 11 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:55.479281 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 101759us
I20260709 12:12:55.484268 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 748) took 3044 ms. Trace:
I20260709 12:12:55.486248 12678 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:55.487018 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 2933) took 2654 ms. Trace:
I20260709 12:12:55.487339 12678 rpcz_store.cc:276] 0709 12:12:52.832853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.478191 (+2645338us) service_pool.cc:224] Handling call
0709 12:12:55.478607 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:55.486707 (+  8100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.486930 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:55.479375 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:55.489667 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 750) took 3049 ms. Trace:
I20260709 12:12:55.490077 12671 rpcz_store.cc:276] 0709 12:12:52.447188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.377918 (+2930730us) service_pool.cc:224] Handling call
0709 12:12:55.434616 (+ 56698us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:55.489393 (+ 54777us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.489618 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:55.484690 12675 rpcz_store.cc:276] 0709 12:12:52.439689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:54.847155 (+2407466us) service_pool.cc:224] Handling call
0709 12:12:54.847641 (+   486us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:54.847897 (+   256us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:54.847935 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:54.848532 (+   597us) log.cc:844] Serialized 26 byte log entry
0709 12:12:55.328503 (+479971us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:12:55.328559 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.328668 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:55.480591 (+151923us) raft_consensus.cc:1685] finished
0709 12:12:55.480611 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:55.484194 (+  3583us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7040,"thread_start_us":479499,"threads_started":1,"wal-append.queue_time_us":618425}
I20260709 12:12:55.376688 19724 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.083s
I20260709 12:12:55.499652 12679 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:55.499872 13305 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:55.455760 19733 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:55.479686 19699 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:55.376716 19582 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:55.462450 19506 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:55.520715 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 130982us
W20260709 12:12:55.510421 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 93928.7us
I20260709 12:12:55.524361 12672 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:55.525295 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3418) took 2921 ms. Trace:
I20260709 12:12:55.525624 12672 rpcz_store.cc:276] 0709 12:12:52.603455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.455640 (+2852185us) service_pool.cc:224] Handling call
0709 12:12:55.494113 (+ 38473us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:55.525014 (+ 30901us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.525237 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:55.498467 19032 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:12:55.530225 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.495245 19769 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:12:55.510525 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3420) took 2637 ms. Trace:
W20260709 12:12:55.528780 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.529806 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3416) took 3004 ms. Trace:
I20260709 12:12:55.533810 12676 rpcz_store.cc:276] 0709 12:12:52.867218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.462910 (+2595692us) service_pool.cc:224] Handling call
0709 12:12:55.504160 (+ 41250us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:55.504301 (+   141us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:55.504316 (+    15us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:55.504344 (+    28us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:55.504363 (+    19us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.504396 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:55.504472 (+    76us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:55.534452 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 146709us
I20260709 12:12:55.533236 12674 rpcz_store.cc:276] 0709 12:12:52.525182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.439334 (+2914152us) service_pool.cc:224] Handling call
0709 12:12:55.501298 (+ 61964us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:55.529459 (+ 28161us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:55.529489 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:55.529541 (+    52us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:55.529577 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.529639 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:55.529750 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:55.494179 19763 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:55.502749 19801 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader election won for term 16
W20260709 12:12:55.538854 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 105963us
I20260709 12:12:55.540477 12673 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:12:55.402705 19696 consensus_queue.cc:260] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:55.520526 19486 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:55.545572 12675 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
W20260709 12:12:55.546744 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 116992us
I20260709 12:12:55.540904 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:55.520180 19689 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.549249 19689 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:55.550660 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:12:55.503477 19724 consensus_queue.cc:1243] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 21.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:12:55.556643 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 757) took 2299 ms. Trace:
I20260709 12:12:55.529181 19779 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:55.442468 19524 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:55.560516 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:55.566228 12672 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:55.567041 12524 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:55.546507 12673 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:12:55.558028 12494 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:55.557097 12675 rpcz_store.cc:276] 0709 12:12:53.257180 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.502281 (+2245101us) service_pool.cc:224] Handling call
0709 12:12:55.544716 (+ 42435us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:55.556328 (+ 11612us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.556576 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:55.532809 19705 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.580335 12678 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:55.581202 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 758) took 1854 ms. Trace:
I20260709 12:12:55.581429 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3423) took 1427 ms. Trace:
I20260709 12:12:55.552124 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:12:55.548153 19707 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.745s	user 0.013s	sys 0.001s
I20260709 12:12:55.570097 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3422) took 1417 ms. Trace:
I20260709 12:12:55.491920 19527 consensus_queue.cc:237] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:55.575623 12530 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:12:55.545532 19032 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:55.554687 19801 raft_consensus.cc:697] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:55.553282 19769 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:55.547629 19539 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.076s
I20260709 12:12:55.581794 12673 rpcz_store.cc:276] 0709 12:12:54.153846 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.541279 (+1387433us) service_pool.cc:224] Handling call
0709 12:12:55.541598 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:55.580852 (+ 39254us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.581370 (+   518us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:55.581589 12678 rpcz_store.cc:276] 0709 12:12:53.727280 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.521234 (+1793954us) service_pool.cc:224] Handling call
0709 12:12:55.561104 (+ 39870us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:55.580924 (+ 19820us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.581142 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:55.774017 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:55.669500 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:55.612004 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 69028.7us
W20260709 12:12:55.591607 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:55.789506 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.681354 12671 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 15. Current term is 16. Ops: []
W20260709 12:12:55.623888 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52185.5us
I20260709 12:12:55.793097 12678 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 15. Current term is 16. Ops: []
I20260709 12:12:55.790753 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 755) took 2570 ms. Trace:
I20260709 12:12:55.795061 12671 rpcz_store.cc:276] 0709 12:12:53.220163 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.501510 (+2281347us) service_pool.cc:224] Handling call
0709 12:12:55.681133 (+179623us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:55.790474 (+109341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.790694 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:55.719964 12679 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 14.
W20260709 12:12:55.679661 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:55.673813 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.754729 12646 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:55.552613 19689 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:55.575927 19818 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.096s	user 0.000s	sys 0.001s
I20260709 12:12:55.790478 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:12:55.798839 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 80152.9us
W20260709 12:12:55.799832 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 191068us
I20260709 12:12:55.786945 12672 rpcz_store.cc:276] 0709 12:12:54.153376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.533979 (+1380603us) service_pool.cc:224] Handling call
0709 12:12:55.534363 (+   384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:55.566740 (+ 32377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.570034 (+  3294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:55.675386 12675 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 12:12:55.623428 19724 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.638561 19500 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:55.694340 19524 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:55.568385 19486 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:55.608525 19798 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:55.696235 19720 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:55.646944 19697 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:55.755223 19764 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:55.808975 12676 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 13. Current term is 14. Ops: [13.8-13.8]
I20260709 12:12:55.818411 12571 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:55.804631 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 231733us
I20260709 12:12:55.817703 12675 pending_rounds.cc:85] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 10
I20260709 12:12:55.682238 19636 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:12:55.564214 19562 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.238s
I20260709 12:12:55.831053 12673 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:12:55.793586 19494 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:55.688665 19725 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.232s
I20260709 12:12:55.601187 19781 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.223s
I20260709 12:12:55.795779 19707 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:55.852717 12678 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 12:12:55.845918 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.614391 19445 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:55.692728 19443 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:55.835886 19798 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:55.844461 19720 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:55.853523 19494 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:55.844753 19636 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:55.851660 19764 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:55.854633 19562 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:12:55.855710 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:55.794601 19762 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:55.790299 19769 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:55.840194 19724 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:55.862313 19625 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:12:55.866720 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 76363.8us
I20260709 12:12:55.867931 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:55.843910 12675 pending_rounds.cc:107] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:12:55.900902 12675 raft_consensus.cc:2889] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:55.869002 19615 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:12:55.822152 19808 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:55.843389 12671 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 18 index: 13. (index mismatch)
I20260709 12:12:55.810876 19705 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:55.918496 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:55.919716 19705 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:55.919816 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:55.869987 19032 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:55.926811 19624 consensus_meta.cc:287] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.556s	user 0.008s	sys 0.004s
I20260709 12:12:55.870667 19526 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:55.877137 19798 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:55.927706 19526 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:55.928225 19526 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:55.615097 12422 catalog_manager.cc:5697] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 12 to 17, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 17 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
W20260709 12:12:55.931792 19529 consensus_meta.cc:287] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.587s	user 0.011s	sys 0.000s
I20260709 12:12:55.884939 19610 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 12:12:55.888069 12678 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: []
W20260709 12:12:55.878192 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 55629.7us
I20260709 12:12:55.890595 13334 raft_consensus.cc:2368] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21.
I20260709 12:12:55.670729 19733 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:55.871076 12679 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
W20260709 12:12:55.900951 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:55.886937 19577 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.554s	user 0.004s	sys 0.007s
I20260709 12:12:55.847445 19768 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 12:12:55.878997 19500 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:55.877792 12677 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.508s	user 0.010s	sys 0.003s
I20260709 12:12:55.926302 19486 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:55.916851 19494 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:12:55.932510 12422 catalog_manager.cc:5697] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 6 to 9, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 9 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:55.933808 12671 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 14 index: 12. Preceding OpId from leader: term: 16 index: 13. (index mismatch)
I20260709 12:12:55.921296 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:55.935081 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3425) took 1780 ms. Trace:
I20260709 12:12:55.984045 12524 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22.
W20260709 12:12:55.977936 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:55.983985 12673 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:55.978000 12677 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:12:55.985244 12674 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:55.984381 12675 rpcz_store.cc:276] 0709 12:12:54.154767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.580725 (+1425958us) service_pool.cc:224] Handling call
0709 12:12:55.625229 (+ 44504us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:55.934742 (+309513us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.935026 (+   284us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:55.986001 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 2959) took 1123 ms. Trace:
I20260709 12:12:55.986557 12677 rpcz_store.cc:276] 0709 12:12:54.862758 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.118433 (+255675us) service_pool.cc:224] Handling call
0709 12:12:55.985934 (+867501us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:55.933893 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:55.883383 19764 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:55.912034 19723 consensus_meta.cc:287] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.617s	user 0.011s	sys 0.003s
I20260709 12:12:55.989290 12571 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:55.950838 19514 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.153s	user 0.011s	sys 0.002s
W20260709 12:12:55.934033 19791 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.654s	user 0.012s	sys 0.001s
W20260709 12:12:55.949843 19491 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.696s	user 0.012s	sys 0.002s
W20260709 12:12:56.002594 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 331336us
W20260709 12:12:55.935608 19820 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.366s	user 0.001s	sys 0.000s
I20260709 12:12:55.900324 19625 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:55.955760 19445 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:55.978166 19826 raft_consensus.cc:3055] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:55.912199 19720 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:55.942433 19780 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:55.999007 12529 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:55.934163 19697 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:12:55.941059 19443 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:55.958526 19707 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:55.934420 19539 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:56.022452 19529 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:12:56.016680 19791 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:55.995061 19705 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.023034 19698 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.008s
I20260709 12:12:55.947978 19724 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:56.022485 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3424) took 1868 ms. Trace:
I20260709 12:12:56.066812 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:56.024434 12422 catalog_manager.cc:5697] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 8 to 16, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 16 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:55.951747 19762 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:55.934743 19582 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:55.911824 19801 consensus_queue.cc:237] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 13.10, Last appended by leader: 10, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:55.996241 19764 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:56.031942 19689 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.032816 19827 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:12:56.037037 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 103281us
I20260709 12:12:56.033847 19826 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:56.044703 19830 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.050935 13305 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:56.049542 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3413) took 3886 ms. Trace:
I20260709 12:12:56.029199 19494 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader election won for term 14
I20260709 12:12:55.999943 19808 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.050s
I20260709 12:12:55.991080 19500 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.033713 19625 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:56.103106 19494 raft_consensus.cc:697] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:56.074551 19443 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:56.094950 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 98454.5us
I20260709 12:12:56.049150 19828 consensus_queue.cc:1059] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 11.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260709 12:12:56.081437 12673 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
W20260709 12:12:56.056432 19699 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.527s	user 0.007s	sys 0.004s
I20260709 12:12:56.039593 19693 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.067539 12530 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:56.067212 12676 rpcz_store.cc:276] 0709 12:12:54.154283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.561495 (+1407212us) service_pool.cc:224] Handling call
0709 12:12:55.804685 (+243190us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:55.842744 (+ 38059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:56.022193 (+179449us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b54002c5950. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:56.022404 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":514708736}
I20260709 12:12:56.016139 19514 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:56.109829 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 59002.6us
W20260709 12:12:56.059862 12643 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.912s (SENT) [suppressed 23 similar messages]
W20260709 12:12:56.070502 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 210314us
I20260709 12:12:55.956561 19636 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.059540 19539 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:56.060066 19697 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:56.083418 19724 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.113510 19443 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.104820 19830 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:56.035617 19824 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
I20260709 12:12:56.118909 19539 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.121425 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:56.090610 19689 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.070153 19705 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.135491 19529 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:56.135632 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:56.121645 19762 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.016s
I20260709 12:12:56.136498 19529 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:56.122049 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:56.138688 12676 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:12:56.096488 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3426) took 1941 ms. Trace:
I20260709 12:12:56.139295 19828 raft_consensus.cc:3055] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Stepping down as leader of term 13
I20260709 12:12:56.139804 19828 raft_consensus.cc:740] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:56.140223 19529 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 16.8, Last appended by leader: 8, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.140898 19828 consensus_queue.cc:260] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 13.8, Last appended by leader: 8, Current term: 13, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.124351 19724 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:56.097178 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:56.094298 12529 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2884) took 1984 ms (client timeout 1961 ms). Trace:
W20260709 12:12:56.145917 12529 rpcz_store.cc:269] 0709 12:12:54.109574 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:54.151677 (+ 42103us) service_pool.cc:224] Handling call
0709 12:12:56.094012 (+1942335us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b54003c0854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:56.094198 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4117108608}
I20260709 12:12:56.124517 19698 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:12:56.148078 19698 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:56.126160 19693 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.152871 12531 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:12:56.126185 19699 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:56.157631 19150 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.767s	user 0.007s	sys 0.006s
I20260709 12:12:56.127121 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.064646 19032 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.082871 12680 rpcz_store.cc:276] 0709 12:12:52.163076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:54.847114 (+2684038us) service_pool.cc:224] Handling call
0709 12:12:54.847595 (+   481us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:54.848049 (+   454us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:54.848081 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:54.848691 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 12:12:55.363057 (+514366us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:55.363103 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:55.363178 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:56.049297 (+686119us) raft_consensus.cc:1685] finished
0709 12:12:56.049318 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:56.049477 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10470,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":127,"thread_start_us":513923,"threads_started":1,"wal-append.queue_time_us":656278}
I20260709 12:12:56.116705 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:56.170886 19032 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.171759 12671 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:12:56.062664 19833 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.047s
I20260709 12:12:56.118086 19514 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:56.175549 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:56.158093 19150 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.169930 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:56.181380 19798 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.182447 19527 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:56.183090 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:56.191129 12494 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:56.193524 19618 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:56.194291 19618 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.197443 19781 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.139453 12674 rpcz_store.cc:276] 0709 12:12:54.155243 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.696057 (+1540814us) service_pool.cc:224] Handling call
0709 12:12:55.985066 (+289009us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:55.988860 (+  3794us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:56.096209 (+107349us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b54001e2150. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:56.096410 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":429992704}
I20260709 12:12:56.122718 12523 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:12:56.092754 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:56.208508 12678 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:12:56.057099 19831 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.082726 19827 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:56.213140 12528 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 17: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:12:56.217654 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 104364us
W20260709 12:12:56.211042 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 323646us
W20260709 12:12:56.218746 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:56.194989 19618 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:12:56.213534 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 102740us
W20260709 12:12:56.082281 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 200522us
W20260709 12:12:56.263620 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 44456us
W20260709 12:12:56.264010 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 136058us
W20260709 12:12:56.266741 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 91219.7us
I20260709 12:12:56.131064 19710 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:12:56.221463 19836 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.090s	user 0.000s	sys 0.001s
I20260709 12:12:56.283084 12569 leader_election.cc:400] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21.
I20260709 12:12:56.295359 12569 leader_election.cc:403] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:12:56.183108 19527 raft_consensus.cc:697] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:56.214567 19696 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:56.128464 19791 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:56.149538 19698 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:56.291615 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:56.308951 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:56.207787 19539 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:56.221480 19150 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:56.221946 19808 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:56.314316 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:56.269955 19798 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:56.327734 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:56.333458 19724 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:56.123922 19697 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.335264 19689 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:56.336655 19699 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:56.127967 19494 consensus_queue.cc:237] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 11.7, Last appended by leader: 7, Current term: 14, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.338399 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:56.343525 13334 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:12:56.215039 19781 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:56.289873 19710 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:56.352946 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.349189 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:56.303761 19762 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:56.338543 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:56.355080 13333 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 19: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:56.356863 12572 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:56.356940 19705 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:56.350759 19791 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.358886 19791 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.311878 19828 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 13 FOLLOWER]: Advancing to term 14
I20260709 12:12:56.363654 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.354537 13331 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:56.362972 12572 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:56.365792 12530 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
I20260709 12:12:56.360944 19791 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:56.283514 19445 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 12:12:56.317782 19779 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.338501 19833 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:12:56.258152 19831 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.391500 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.123442 12422 catalog_manager.cc:5697] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 9 to 16, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 16 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:56.338095 19693 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:56.317395 19615 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:12:56.397769 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 106886us
I20260709 12:12:56.351002 19032 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:56.400846 12572 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:56.300669 19829 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.091s
I20260709 12:12:56.414971 19779 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:56.420032 19831 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:56.375222 19705 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:12:56.322800 19514 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.072s
I20260709 12:12:56.449570 19500 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:56.450640 19845 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:56.455309 12422 catalog_manager.cc:5697] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 11 to 15, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 15 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:12:56.456712 19443 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:56.459143 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 70658.4us
I20260709 12:12:56.459985 12572 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:56.411998 12675 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:12:56.362949 19762 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:56.132336 19826 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.407891 19830 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:56.435277 13334 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:12:56.436223 13331 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:12:56.446300 19615 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:56.470049 19636 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:56.347115 19824 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.056s
I20260709 12:12:56.457607 12422 catalog_manager.cc:5697] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 11 to 15, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 15 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:12:56.496202 12422 catalog_manager.cc:5697] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:12:56.471239 19762 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:56.330848 19527 consensus_queue.cc:237] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 18.9, Last appended by leader: 9, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.485683 19826 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:56.507465 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:56.507691 12524 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:12:56.508512 12531 raft_consensus.cc:2410] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:12:56.509243 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:56.509781 12531 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:56.486404 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:56.346829 19808 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:56.403551 19791 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:12:56.509954 12524 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:56.510551 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:56.511451 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.535737 12494 leader_election.cc:400] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19.
W20260709 12:12:56.486225 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 426358us
I20260709 12:12:56.522146 12676 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:12:56.540949 12529 raft_consensus.cc:2410] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
I20260709 12:12:56.542935 12569 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:56.545123 12527 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:12:56.472157 19834 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:56.514142 19768 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.005s	sys 0.015s
I20260709 12:12:56.552455 12569 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:56.554248 12522 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:12:56.560508 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.534871 19791 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.458524 19625 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.584116 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.584777 12529 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
W20260709 12:12:56.561748 19524 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.728s	user 0.011s	sys 0.002s
I20260709 12:12:56.567096 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:56.558149 19529 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:56.556758 19697 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:56.571244 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:56.571587 19844 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:12:56.574285 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49984.1us
I20260709 12:12:56.576301 12646 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:56.523663 19842 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:12:56.554512 19768 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.559s	user 0.006s	sys 0.015s
I20260709 12:12:56.540773 19500 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:56.579553 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:56.581430 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 121527us
I20260709 12:12:56.583173 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.492681 19693 consensus_queue.cc:1059] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.063s
I20260709 12:12:56.590025 13331 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 12:12:56.569480 12523 raft_consensus.cc:2410] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 22 because replica has last-logged OpId of term: 20 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
I20260709 12:12:56.579808 12571 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:56.603585 12672 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 14. (term mismatch)
W20260709 12:12:56.548456 19769 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.014s	sys 0.027s
I20260709 12:12:56.615326 12524 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:12:56.584493 19791 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:12:56.583811 19830 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.641903 12572 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:56.472000 19827 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.555719 19834 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:56.552598 19710 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.13, Last appended by leader: 13, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:56.583539 19443 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 12:12:56.595865 19529 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.659677 19618 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.635282 19705 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:12:56.635269 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.639295 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.629016 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:56.638186 12672 pending_rounds.cc:85] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 13
I20260709 12:12:56.614301 19833 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:12:56.653616 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 73877.7us
W20260709 12:12:56.688794 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 112605us
I20260709 12:12:56.689571 12646 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:56.654570 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.656054 19840 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
W20260709 12:12:56.659847 19526 consensus_meta.cc:287] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.731s	user 0.004s	sys 0.009s
I20260709 12:12:56.660377 19529 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:56.619007 19851 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:12:56.702279 19624 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:12:56.704170 12679 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 17. Current term is 18. Ops: [17.13-17.13]
I20260709 12:12:56.661827 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:56.662305 19779 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:56.588053 19831 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.054s
W20260709 12:12:56.709702 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:56.664446 19764 consensus_meta.cc:287] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.553s	user 0.011s	sys 0.000s
I20260709 12:12:56.666258 19524 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:12:56.667707 19625 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:56.654526 19769 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.629s	user 0.014s	sys 0.027s
I20260709 12:12:56.642670 19500 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:56.671595 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 43573.5us
I20260709 12:12:56.672513 19697 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:56.738137 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.674470 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:56.676096 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.679126 19808 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:56.686112 12524 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 20.
I20260709 12:12:56.686091 12522 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:12:56.686476 12672 pending_rounds.cc:107] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 12:12:56.743151 12672 raft_consensus.cc:2889] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:56.691970 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:56.744290 12673 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 22: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:56.695020 12680 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:56.744612 13305 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:56.697235 19833 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 21 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:56.645825 19798 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.075s
W20260709 12:12:56.685321 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48820.6us
I20260709 12:12:56.706640 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:56.751792 12523 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:56.692233 19725 raft_consensus.cc:3055] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:12:56.724886 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:56.725210 12530 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:56.669296 19781 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:56.673549 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:56.753635 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.702847 19624 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:56.739300 12678 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:56.653223 19830 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:56.684851 19842 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:56.747136 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:56.725497 19827 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:56.750393 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:56.754940 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.680290 19689 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.055s
I20260709 12:12:56.756855 12531 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:56.695456 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:56.758301 12606 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:56.782717 12529 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:56.758808 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.726564 19764 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.760763 13335 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:56.749903 19801 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:56.764362 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:56.764565 12605 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:12:56.768136 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.768625 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:56.782341 12671 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:56.786521 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:12:56.770342 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:56.774987 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 85481us
I20260709 12:12:56.733045 19780 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:56.777439 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:56.780790 13334 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:56.781618 12524 raft_consensus.cc:2368] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21.
I20260709 12:12:56.781551 12679 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:12:56.795645 19853 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:56.796342 19853 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:56.755226 19725 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:56.761878 19833 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:56.798694 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:56.799419 12680 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:12:56.799655 19808 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.800436 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:56.800272 19808 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.789436 12646 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:56.732205 19850 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:56.813329 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:56.813326 12675 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
I20260709 12:12:56.815261 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.818877 12598 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:12:56.797377 19853 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:56.829478 12528 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 17: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:56.724326 19526 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:56.820283 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:56.776598 19625 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.838279 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.782063 19842 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.737700 19762 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.849152 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:56.846951 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:56.839352 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:56.787719 19801 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:56.842577 12597 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:56.786372 19781 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:56.859676 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:56.874027 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:56.862340 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.859562 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:56.878031 12605 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:56.861924 12602 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:12:56.871337 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:56.861652 12679 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:12:56.877584 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:56.841638 19610 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.884312 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:56.871940 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.882761 12599 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:12:56.888965 12674 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 18: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:56.889837 12600 raft_consensus.cc:2410] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 22 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:12:56.894847 12680 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 11. (term mismatch)
I20260709 12:12:56.895622 12523 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
I20260709 12:12:56.895885 12680 pending_rounds.cc:85] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 10
I20260709 12:12:56.897346 12680 pending_rounds.cc:107] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 12:12:56.882757 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:56.818295 19780 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:56.899312 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.847878 19769 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.885479 12598 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:12:56.950073 12678 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
I20260709 12:12:56.902690 12680 raft_consensus.cc:2889] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:56.932819 12526 raft_consensus.cc:2410] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 21 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:12:56.931520 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:56.933014 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:56.861083 19801 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:56.859066 19527 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:56.932282 12528 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 22: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:56.847667 19842 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:56.935948 13304 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:56.932147 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:56.985406 12675 raft_consensus.cc:2393] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 22: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:56.985615 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:56.986320 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49820.6us
I20260709 12:12:56.984863 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:56.993661 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:56.992666 12674 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:12:56.902531 19851 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:56.997782 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 61950.4us
I20260709 12:12:56.996783 12599 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
W20260709 12:12:57.016819 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 227478us
I20260709 12:12:57.015429 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:57.019593 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:57.020246 12676 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:57.025151 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:57.026053 12523 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 19: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:57.027604 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:57.028513 12678 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:12:57.029644 12678 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: [14.8-18.9]
I20260709 12:12:57.032696 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:57.033349 12674 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:57.034238 12528 raft_consensus.cc:2368] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21.
I20260709 12:12:57.035408 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.036545 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.037590 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.038686 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:57.039880 12528 raft_consensus.cc:2410] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 19 because replica has last-logged OpId of term: 18 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:12:57.041678 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:57.043730 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.044914 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.045545 12528 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:57.047225 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:56.887521 19781 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:56.882261 19625 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:56.904776 19831 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.119s
I20260709 12:12:57.051687 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.060127 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:56.986640 19768 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:57.015885 19829 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:57.055263 12601 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:56.993647 19859 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:57.063505 19862 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 12:12:57.059165 12525 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:57.058377 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:57.054662 19854 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.242s	user 0.001s	sys 0.000s
I20260709 12:12:57.047683 19839 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:56.989482 19826 consensus_queue.cc:1059] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.066s
I20260709 12:12:56.989259 19693 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.069s
I20260709 12:12:57.102977 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:57.105607 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:56.951840 19610 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.116786 12675 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 19: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:12:57.115476 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 65641.9us
I20260709 12:12:56.989964 19769 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.022504 19527 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.113252 12601 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:12:57.110617 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:57.067538 19618 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:57.131258 12599 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
W20260709 12:12:57.131368 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:57.140591 12599 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:12:57.141396 19443 raft_consensus.cc:3055] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:12:57.141848 19443 raft_consensus.cc:740] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:57.103832 19636 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 16.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.044s
I20260709 12:12:57.143244 19443 consensus_queue.cc:260] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:57.145282 19443 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:57.146171 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:12:57.148063 19860 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.152s	user 0.000s	sys 0.001s
W20260709 12:12:57.148692 19858 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
I20260709 12:12:57.149315 13336 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:12:57.156950 12605 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:57.165099 19500 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:57.165905 19808 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:57.180657 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:57.134577 19857 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.298s	user 0.002s	sys 0.000s
I20260709 12:12:57.178959 19764 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:57.129017 19868 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.210278 13332 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:12:57.219445 12599 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
W20260709 12:12:57.181767 19862 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.067s	user 0.001s	sys 0.000s
I20260709 12:12:57.094883 19859 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.181381 19864 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:57.200343 19830 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:57.112030 19839 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:57.261495 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 122566us
I20260709 12:12:57.230305 19762 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:57.265247 12646 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:57.270434 19878 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 12:12:57.273327 19494 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.274262 19494 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.177506 19851 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:57.288399 19526 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:57.292063 19859 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:57.295251 19850 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:12:57.295755 19850 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.296128 19850 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:57.268108 19762 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:57.296789 19762 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.299254 19762 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:57.296016 12646 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:57.232962 19868 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:57.302868 19724 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:12:57.303208 19781 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:57.303498 19724 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:57.304658 19868 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:57.305531 19780 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:57.306360 19769 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:57.306906 19834 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
W20260709 12:12:57.308301 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:57.311687 19527 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:57.304670 19724 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.316414 19839 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:57.264441 19873 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.302203 12646 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:57.327212 12529 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
W20260709 12:12:57.295321 19866 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 12:12:57.352320 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:57.359790 12678 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 20: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:57.360841 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:57.358705 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:57.369715 13336 raft_consensus.cc:2393] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:57.370216 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:57.330361 19834 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:57.356710 19828 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.968s	user 0.003s	sys 0.011s
I20260709 12:12:57.373225 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:57.348623 19696 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.378310 12678 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
I20260709 12:12:57.379456 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:57.380275 13330 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:57.383031 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:57.383906 12678 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:57.385411 12599 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:57.383709 13304 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:57.363547 19526 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:57.257838 19864 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.392293 12678 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: [15.10-16.11]
I20260709 12:12:57.377470 19875 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:57.399837 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 97810.1us
I20260709 12:12:57.380517 19494 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:57.402822 12646 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:57.391764 19780 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:57.334532 19873 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:57.410964 19791 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.757s	user 0.014s	sys 0.000s
I20260709 12:12:57.411585 19791 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:57.414494 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.415398 12529 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
W20260709 12:12:57.416281 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:57.384786 19834 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.420087 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.421314 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.385578 19696 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.427739 19696 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:57.428102 19801 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:57.429176 19826 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:57.429766 13339 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 21: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:57.430946 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.431298 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:57.431962 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.431893 12529 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:12:57.433298 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:12:57.435431 19870 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 12:12:57.435422 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3017) took 1029 ms. Trace:
I20260709 12:12:57.436470 12530 rpcz_store.cc:276] 0709 12:12:56.406070 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.406332 (+   262us) service_pool.cc:224] Handling call
0709 12:12:57.435165 (+1028833us) spinlock_profiling.cc:243] Waited 213 ms on lock 0x7b54001072d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:57.435343 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":489615104}
I20260709 12:12:57.438223 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:57.439623 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:57.439287 19725 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:57.440944 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:57.441468 19615 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:57.441854 12598 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:57.442945 19615 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:12:57.456979 19769 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.445672 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3438) took 2454 ms. Trace:
I20260709 12:12:57.457705 19769 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.457751 12672 rpcz_store.cc:276] 0709 12:12:54.990878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:55.818305 (+827427us) service_pool.cc:224] Handling call
0709 12:12:55.818743 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:56.743722 (+924979us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:57.445288 (+701566us) spinlock_profiling.cc:243] Waited 784 ms on lock 0x7b54002c4a54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:57.445584 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1803907840}
I20260709 12:12:57.456136 13302 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:57.459868 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:57.448249 19720 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:12:57.460779 19720 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:12:57.413574 19529 consensus_meta.cc:287] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.714s	user 0.011s	sys 0.003s
I20260709 12:12:57.449890 12529 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:12:57.451417 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:57.429847 19826 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:12:57.453373 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:57.457080 19615 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.468456 19615 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:57.468693 19445 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:57.469215 19884 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.469442 19445 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.469830 19884 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:57.472050 19445 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:57.472518 19884 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:57.406073 19875 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.474825 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:57.461838 19720 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:57.465444 12599 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:12:57.475665 12673 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 20: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:57.476718 19882 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.476630 19875 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:57.485493 12523 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:57.504995 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.505512 12523 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:12:57.506296 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.476925 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:57.478003 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:57.477674 19887 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:12:57.483898 19877 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:12:57.483501 19871 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.123s	user 0.001s	sys 0.000s
I20260709 12:12:57.467461 19491 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.017s
I20260709 12:12:57.487888 12571 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:57.482201 19808 consensus_queue.cc:1059] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 18.9, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:12:57.446362 19780 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.423278 19526 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:57.502213 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:57.396265 19872 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.250s
I20260709 12:12:57.509006 12673 raft_consensus.cc:2410] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 22 because replica has last-logged OpId of term: 18 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 12.
W20260709 12:12:57.510015 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 47890.8us
I20260709 12:12:57.487555 19529 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:57.529590 19827 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.726s	user 0.010s	sys 0.003s
I20260709 12:12:57.552599 19877 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:57.589020 12674 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
W20260709 12:12:57.557931 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 70128.5us
I20260709 12:12:57.589618 12674 pending_rounds.cc:85] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 10
I20260709 12:12:57.589903 12674 pending_rounds.cc:107] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 17.11
I20260709 12:12:57.590158 12674 raft_consensus.cc:2889] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:57.590492 12674 pending_rounds.cc:107] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 12:12:57.590763 12674 raft_consensus.cc:2889] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:57.590911 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:57.476227 19720 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:57.592315 19886 consensus_queue.cc:1059] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:57.554358 19873 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:57.593436 19874 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:57.532243 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:57.566692 19864 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:57.594648 12530 raft_consensus.cc:2393] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 21: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:57.594861 19886 raft_consensus.cc:3055] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:12:57.551776 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:57.596762 12600 raft_consensus.cc:2410] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 22 because replica has last-logged OpId of term: 21 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 14.
I20260709 12:12:57.499516 19840 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.095s
I20260709 12:12:57.598448 12494 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:57.558255 19762 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:12:57.555012 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:57.600374 19762 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:57.600955 19762 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:57.564208 19834 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:57.552404 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:57.584582 19710 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:57.560413 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:57.603220 12602 raft_consensus.cc:2393] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:57.604403 19529 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:57.604578 19827 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:57.605118 19827 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.586699 19808 consensus_queue.cc:1243] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 19.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 8
I20260709 12:12:57.605558 19827 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:57.606591 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.607151 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:57.608481 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.609071 12529 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:12:57.595755 19886 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:57.565371 19624 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.610540 19886 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.610894 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:57.611407 19624 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:57.611208 12646 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:57.612258 19886 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:57.613039 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:57.613572 19524 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:12:57.614182 19524 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.614248 19874 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.614704 19524 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:12:57.614773 19874 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:57.617542 19769 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:57.618423 19791 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:57.618439 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:57.619082 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:57.611627 12671 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:57.601738 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:57.621018 12604 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:57.587611 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.622095 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:57.622440 12524 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:57.622830 12604 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:12:57.607276 12530 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:12:57.624204 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:57.563761 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.613685 12597 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 14.
W20260709 12:12:57.565333 19842 consensus_meta.cc:287] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.542s	user 0.008s	sys 0.004s
I20260709 12:12:57.573446 19850 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:57.537573 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:57.629401 12602 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:57.629438 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:57.525995 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.632475 12598 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:57.633823 12524 raft_consensus.cc:3055] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:57.634927 13302 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:57.635164 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:57.639568 12524 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:57.625818 12494 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:57.637042 12572 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:57.635878 12529 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 9. (term mismatch)
I20260709 12:12:57.641487 13330 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:57.642254 19875 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.642199 19842 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.642997 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:57.643394 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:57.640579 12529 pending_rounds.cc:85] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 8
I20260709 12:12:57.644366 12529 pending_rounds.cc:107] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 18.9
I20260709 12:12:57.645880 12603 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:57.646195 12529 raft_consensus.cc:2889] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:57.640594 12605 raft_consensus.cc:2410] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 22 because replica has last-logged OpId of term: 20 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 8.
I20260709 12:12:57.641718 12526 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 13 index: 8. Preceding OpId from leader: term: 14 index: 8. (term mismatch)
I20260709 12:12:57.648135 12526 pending_rounds.cc:85] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 7
I20260709 12:12:57.647997 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:57.648600 12526 pending_rounds.cc:107] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 13.8
I20260709 12:12:57.649022 12526 raft_consensus.cc:2889] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:12:57.647598 12530 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:12:57.576501 19801 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:57.652133 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.687980 13305 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:12:57.533645 19494 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:57.664800 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:57.619103 19150 raft_consensus.cc:3055] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:57.647271 19491 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:12:57.646785 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 263164us
I20260709 12:12:57.667863 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.631660 19526 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.686386 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:57.648757 12525 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:12:57.700181 19526 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:57.689275 12672 raft_consensus.cc:2368] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22.
I20260709 12:12:57.690972 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:57.701670 12602 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
W20260709 12:12:57.691237 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 54532.1us
I20260709 12:12:57.691912 12598 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:57.663611 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:57.692967 12523 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:12:57.697544 13335 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:57.646298 19791 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:57.692972 12524 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.9, Last appended by leader: 9, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.706714 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 15 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:57.707599 12524 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:57.708830 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.704697 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:57.670377 19443 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:57.525166 19882 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:57.791553 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54473us
I20260709 12:12:57.791673 12572 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:57.793387 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 155538us
I20260709 12:12:57.795384 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:57.669235 19831 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.797281 12671 raft_consensus.cc:2393] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:57.686944 19873 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:57.643015 19875 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:57.801477 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:57.797870 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 93331.9us
I20260709 12:12:57.693310 19150 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:57.698182 19491 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.791183 19840 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:57.708464 19872 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.807852 12494 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:57.801177 19615 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:57.864459 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:57.883208 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:57.795913 19833 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:57.883766 12674 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
W20260709 12:12:57.891145 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:57.790632 19443 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:12:57.853195 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 45426.9us
I20260709 12:12:57.854916 19896 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:12:57.899314 19896 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.900069 19896 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:12:57.899212 19443 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:57.902112 19875 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:57.855598 19872 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.905400 12524 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 18 index: 9. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:12:57.857672 19850 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:57.905715 19872 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:57.906651 19850 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.907186 19850 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:57.790692 19696 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.035s
I20260709 12:12:57.910158 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:57.861366 13339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 895) took 1093 ms. Trace:
I20260709 12:12:57.804678 19831 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.862920 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:57.928776 12671 raft_consensus.cc:2410] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 22 because replica has last-logged OpId of term: 19 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 10.
I20260709 12:12:57.864782 13332 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
I20260709 12:12:57.865094 19780 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:57.795861 19801 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:57.803628 19032 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:57.873216 19615 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:57.875317 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 770) took 1222 ms. Trace:
I20260709 12:12:57.876407 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:57.876927 13334 raft_consensus.cc:2410] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:12:57.878669 19769 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Stepping down as leader of term 18
W20260709 12:12:57.879478 19895 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.206s	user 0.001s	sys 0.000s
I20260709 12:12:57.880012 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:57.961561 12597 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:12:57.802194 19636 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.963778 12598 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
W20260709 12:12:57.889057 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:57.964458 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:57.891619 12602 raft_consensus.cc:2410] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:12:57.851483 19840 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.894593 19150 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.969641 19874 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:57.970356 19840 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:57.895141 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.970845 12571 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:57.971865 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:57.972347 19874 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:57.972488 12525 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:57.972630 12571 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:57.973138 19874 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:12:57.973611 19874 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:57.802419 19791 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:57.792681 19494 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:57.898497 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 106890us
I20260709 12:12:57.912153 13339 rpcz_store.cc:276] 0709 12:12:56.767997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.768293 (+   296us) service_pool.cc:224] Handling call
0709 12:12:57.860814 (+1092521us) spinlock_profiling.cc:243] Waited 643 ms on lock 0x7b54006419d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:57.861280 (+   466us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1479450368}
I20260709 12:12:57.892642 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:57.935818 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:57.936867 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:57.823803 19491 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:12:57.969704 19801 raft_consensus.cc:697] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:57.939853 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:57.893683 19851 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:12:57.945382 19808 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 19 current term: 19 is higher.
I20260709 12:12:57.946215 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:57.951545 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:57.952041 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:58.049153 12674 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:57.948431 12677 rpcz_store.cc:276] 0709 12:12:56.653007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.676461 (+ 23454us) service_pool.cc:224] Handling call
0709 12:12:56.677230 (+   769us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:56.783905 (+106675us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:56.783938 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:56.828785 (+ 44847us) log.cc:844] Serialized 26 byte log entry
0709 12:12:57.221728 (+392943us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:12:57.381612 (+159884us) log.cc:844] Serialized 14 byte log entry
0709 12:12:57.381883 (+   271us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:57.874912 (+493029us) spinlock_profiling.cc:243] Waited 78.1 ms on lock 0x7b54002c5e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:57.874964 (+    52us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:57.875013 (+    49us) raft_consensus.cc:1685] finished
0709 12:12:57.875107 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:57.875257 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15649,"prepare.queue_time_us":43275,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":55,"raft.queue_time_us":14227,"raft.run_cpu_time_us":365,"raft.run_wall_time_us":39048,"spinlock_wait_cycles":179617536,"thread_start_us":440852,"threads_started":2,"wal-append.queue_time_us":607590}
I20260709 12:12:57.890163 19833 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.047935 12671 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:12:57.887593 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:57.967747 19864 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:57.947928 19032 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:57.955837 19769 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:57.969990 19877 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:57.970359 13338 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:12:57.991852 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:58.056317 12673 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:12:57.982362 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:58.040197 12572 leader_election.cc:400] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21.
I20260709 12:12:58.057440 12597 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:12:57.990224 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:57.985296 12531 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 12:12:58.058310 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:58.058427 12672 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:57.973990 19874 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:58.055831 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:58.059473 12531 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:12:57.975107 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:58.061287 12602 raft_consensus.cc:2410] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 20 because replica has last-logged OpId of term: 19 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:12:58.061592 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:58.002353 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:57.986726 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3360) took 1183 ms. Trace:
W20260709 12:12:58.047356 19900 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:12:57.987267 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3442) took 1222 ms. Trace:
I20260709 12:12:58.046175 12571 leader_election.cc:400] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22.
I20260709 12:12:57.891862 19842 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:58.054102 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:12:57.987855 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:58.056583 19898 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.062s	user 0.002s	sys 0.000s
I20260709 12:12:58.065690 12675 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:12:58.043789 12524 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:12:57.974617 19875 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:58.064445 12603 rpcz_store.cc:276] 0709 12:12:56.803311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.818057 (+ 14746us) service_pool.cc:224] Handling call
0709 12:12:56.842093 (+ 24036us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:57.652712 (+810619us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:57.986450 (+333738us) spinlock_profiling.cc:243] Waited 803 ms on lock 0x7b5400271ad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:57.986661 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1847938944}
I20260709 12:12:57.975204 19625 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:58.079792 12571 leader_election.cc:403] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:12:58.079736 12680 rpcz_store.cc:276] 0709 12:12:56.764657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.801413 (+ 36756us) service_pool.cc:224] Handling call
0709 12:12:56.801736 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:56.990392 (+188656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:57.987036 (+996644us) spinlock_profiling.cc:243] Waited 77.4 ms on lock 0x7b54000f00d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:57.987208 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":177922944}
I20260709 12:12:57.994426 19850 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:12:58.160531 19762 consensus_meta.cc:287] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.559s	user 0.012s	sys 0.006s
I20260709 12:12:58.161036 19762 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:58.162710 19526 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:58.163736 19762 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
W20260709 12:12:58.086755 19845 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.612s	user 0.015s	sys 0.008s
W20260709 12:12:58.063148 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 181186us
I20260709 12:12:58.171561 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:58.172444 12671 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 18: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:58.107210 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3376) took 1043 ms. Trace:
I20260709 12:12:58.123046 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:58.122195 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:58.175612 12602 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 20.
I20260709 12:12:58.125589 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:58.055896 19833 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:58.055361 19032 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:58.177480 12601 rpcz_store.cc:276] 0709 12:12:57.063611 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.130867 (+ 67256us) service_pool.cc:224] Handling call
0709 12:12:57.180132 (+ 49265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:57.382862 (+202730us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:57.382905 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:57.391756 (+  8851us) log.cc:844] Serialized 26 byte log entry
0709 12:12:57.510838 (+119082us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:57.510883 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:57.510951 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:58.106956 (+596005us) raft_consensus.cc:1685] finished
0709 12:12:58.106979 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:58.107149 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20426,"raft.queue_time_us":101506,"raft.run_cpu_time_us":122,"raft.run_wall_time_us":122,"thread_start_us":260254,"threads_started":2,"wal-append.queue_time_us":101931}
I20260709 12:12:58.129629 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:58.181099 12597 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:58.101872 12525 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:12:58.134889 19539 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:58.135762 12522 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:12:58.136129 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:58.138283 19791 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:58.139820 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:58.142567 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:58.146582 19801 consensus_queue.cc:237] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:58.147667 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:58.149219 12531 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:58.151652 19494 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:58.151763 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:12:58.153654 19524 consensus_meta.cc:287] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.538s	user 0.011s	sys 0.010s
I20260709 12:12:58.154249 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:58.061429 19864 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:58.145758 19840 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:58.155658 19769 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:58.156625 19624 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.545s	user 0.008s	sys 0.003s
I20260709 12:12:58.156903 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:58.267925 12673 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
W20260709 12:12:58.157471 12679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 3446) took 1368 ms (client timeout 1562 ms). Trace:
W20260709 12:12:58.268798 12679 rpcz_store.cc:269] 0709 12:12:56.788658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:56.813067 (+ 24409us) service_pool.cc:224] Handling call
0709 12:12:58.157247 (+1344180us) spinlock_profiling.cc:243] Waited 46.9 ms on lock 0x7b54002c4f54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:58.157393 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":107776256}
I20260709 12:12:58.184834 19845 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21.
I20260709 12:12:58.158425 19842 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:12:58.154721 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 114702us
I20260709 12:12:58.166289 12605 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:12:58.167169 12675 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:12:58.275260 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3467) took 1007 ms. Trace:
I20260709 12:12:58.275661 12675 rpcz_store.cc:276] 0709 12:12:57.268148 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.166694 (+898546us) service_pool.cc:224] Handling call
0709 12:12:58.166974 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:58.274969 (+107995us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.275197 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.134449 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:58.278620 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3471) took 1008 ms. Trace:
I20260709 12:12:58.279138 12679 rpcz_store.cc:276] 0709 12:12:57.319190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.277485 (+958295us) service_pool.cc:224] Handling call
0709 12:12:58.277895 (+   410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:58.278181 (+   286us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:58.278223 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:58.278287 (+    64us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:12:58.278335 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.278415 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:58.278549 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.280810 12523 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 21: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:58.282636 19526 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:58.283144 12675 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 13 index: 7. Preceding OpId from leader: term: 19 index: 10. (index mismatch)
I20260709 12:12:58.255157 19791 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.028s
I20260709 12:12:58.284603 12599 raft_consensus.cc:2410] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 22 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:12:58.284729 12675 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 11 index: 7. Preceding OpId from leader: term: 14 index: 8. (index mismatch)
I20260709 12:12:58.287026 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:58.265152 19723 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.293277 12679 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 13 index: 10. Preceding OpId from leader: term: 16 index: 11. (index mismatch)
I20260709 12:12:58.187705 12674 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22.
I20260709 12:12:58.297003 12674 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
I20260709 12:12:58.272356 19909 raft_consensus.cc:3055] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:58.126621 19875 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.201391 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:12:58.255915 19524 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:58.203763 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:57.967787 19636 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:58.213536 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:58.266026 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:58.339545 12601 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:12:58.107035 19827 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:58.180248 19610 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:58.157851 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 116688us
I20260709 12:12:58.290311 19769 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:58.271967 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:58.132233 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:58.289850 12677 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: [21.13-21.13]
W20260709 12:12:58.287842 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 115032us
I20260709 12:12:58.347540 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:57.987577 19831 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:58.187484 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:58.298870 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:58.350592 12606 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:58.350782 12598 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:58.351184 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 753) took 1001 ms. Trace:
I20260709 12:12:58.274989 19696 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:58.197839 12672 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:12:58.302110 12673 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:12:58.201983 12678 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
W20260709 12:12:58.315678 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:58.316943 19875 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:58.323820 19725 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:12:58.326926 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:58.357291 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:58.357532 12679 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 19 index: 9. (index mismatch)
I20260709 12:12:58.348536 12597 raft_consensus.cc:2410] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 10.
W20260709 12:12:58.354825 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 462275us
W20260709 12:12:58.355000 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 80024.4us
I20260709 12:12:58.359769 12572 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:58.361172 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:58.361881 12672 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:12:58.363783 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3468) took 1095 ms. Trace:
I20260709 12:12:58.364207 12678 rpcz_store.cc:276] 0709 12:12:57.268806 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.182027 (+913221us) service_pool.cc:224] Handling call
0709 12:12:58.201815 (+ 19788us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:58.363513 (+161698us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.363726 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.364815 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:58.365532 12677 raft_consensus.cc:2368] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
I20260709 12:12:58.301909 19909 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:58.355890 12528 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:58.345212 19845 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:58.300804 19723 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:58.351485 12598 rpcz_store.cc:276] 0709 12:12:57.349947 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.186972 (+837025us) service_pool.cc:224] Handling call
0709 12:12:58.351119 (+164147us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.367525 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:58.368772 12601 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
I20260709 12:12:58.373338 12672 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:12:58.375221 12496 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:58.375841 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:58.376511 12496 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:58.376799 12606 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:58.378633 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3381) took 1053 ms. Trace:
I20260709 12:12:58.379034 12601 rpcz_store.cc:276] 0709 12:12:57.325724 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.350970 (+1025246us) service_pool.cc:224] Handling call
0709 12:12:58.368556 (+ 17586us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:58.378370 (+  9814us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.378579 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.381579 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:58.383785 12494 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:58.383862 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:58.384649 12673 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
W20260709 12:12:58.385068 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:58.353314 19696 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.382583 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:58.389618 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:58.390658 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3382) took 1013 ms. Trace:
I20260709 12:12:58.391240 12598 rpcz_store.cc:276] 0709 12:12:57.374076 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.372276 (+998200us) service_pool.cc:224] Handling call
0709 12:12:58.386571 (+ 14295us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:58.386811 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:58.386883 (+    72us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:58.386945 (+    62us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:12:58.386998 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.387086 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:58.387238 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.397745 12572 leader_election.cc:400] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21.
I20260709 12:12:58.349686 19539 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
W20260709 12:12:58.400105 19905 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:12:58.402691 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3459) took 1406 ms. Trace:
I20260709 12:12:58.403105 12676 rpcz_store.cc:276] 0709 12:12:56.995948 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.066542 (+ 70594us) service_pool.cc:224] Handling call
0709 12:12:57.101736 (+ 35194us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:57.102158 (+   422us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:12:57.686509 (+584351us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b180046f420. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf
0709 12:12:57.686684 (+   175us) log.cc:844] Serialized 14 byte log entry
0709 12:12:58.068663 (+381979us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:58.169661 (+100998us) log.cc:844] Serialized 26 byte log entry
0709 12:12:58.169876 (+   215us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:12:58.173621 (+  3745us) log.cc:844] Serialized 14 byte log entry
0709 12:12:58.173938 (+   317us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.174045 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:58.402442 (+228397us) raft_consensus.cc:1685] finished
0709 12:12:58.402466 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:58.402628 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":220,"prepare.queue_time_us":157775,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":110,"raft.queue_time_us":129452,"raft.run_cpu_time_us":597,"raft.run_wall_time_us":68359,"spinlock_wait_cycles":4218752,"thread_start_us":363256,"threads_started":2,"wal-append.queue_time_us":551857}
I20260709 12:12:58.374848 19624 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:58.380853 19845 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:12:58.408991 12603 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:12:58.411639 12528 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:58.395761 12571 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:58.379457 19733 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:12:58.418486 19723 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:58.455660 19909 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:58.540012 19791 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:12:58.540231 19909 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:58.540558 19791 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:58.541183 19791 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:12:58.420938 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:58.407382 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:58.455036 19864 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:58.543093 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 144007us
I20260709 12:12:58.543862 19864 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:58.544574 12572 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:58.444635 19150 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:58.545611 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:58.547024 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3385) took 1149 ms. Trace:
I20260709 12:12:58.547541 12603 rpcz_store.cc:276] 0709 12:12:57.398039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.405361 (+1007322us) service_pool.cc:224] Handling call
0709 12:12:58.406023 (+   662us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:58.532571 (+126548us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.546936 (+ 14365us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.442099 19851 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:58.548852 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:58.454543 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3465) took 1226 ms. Trace:
I20260709 12:12:58.471401 19636 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:58.433005 12599 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 16 index: 15. (index mismatch)
I20260709 12:12:58.551568 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3022) took 1146 ms. Trace:
I20260709 12:12:58.550418 12680 rpcz_store.cc:276] 0709 12:12:57.228048 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.136770 (+908722us) service_pool.cc:224] Handling call
0709 12:12:58.137479 (+   709us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:58.138303 (+   824us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:58.138345 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:58.138907 (+   562us) log.cc:844] Serialized 26 byte log entry
0709 12:12:58.273184 (+134277us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:58.273250 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.273355 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:58.454284 (+180929us) raft_consensus.cc:1685] finished
0709 12:12:58.454306 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:58.454472 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2277,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":96,"thread_start_us":133807,"threads_started":1,"wal-append.queue_time_us":193274}
I20260709 12:12:58.551949 12599 rpcz_store.cc:276] 0709 12:12:57.405119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.432209 (+1027090us) service_pool.cc:224] Handling call
0709 12:12:58.432622 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:58.551284 (+118662us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.551510 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.396885 19912 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:58.434677 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:58.553329 12525 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:58.436100 19827 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:58.437510 19610 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:58.392594 19696 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:58.545135 19624 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:58.554533 19524 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:12:58.406769 19853 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:58.556341 19697 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:58.556567 19853 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.557031 19853 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:58.556969 19697 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:58.456758 19896 consensus_meta.cc:287] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.556s	user 0.019s	sys 0.000s
I20260709 12:12:58.558545 19896 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:58.559278 13333 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:12:58.559908 19494 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:58.560915 12673 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:12:58.561071 19913 consensus_queue.cc:1059] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 18.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:12:58.562702 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3047) took 1079 ms. Trace:
I20260709 12:12:58.563081 12605 rpcz_store.cc:276] 0709 12:12:57.483625 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.664698 (+181073us) service_pool.cc:224] Handling call
0709 12:12:58.562508 (+897810us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b5400079c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:58.562627 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1070392576}
I20260709 12:12:58.564539 12605 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20. Ops: [17.11-19.12]
I20260709 12:12:58.564805 19032 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:58.565196 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3027) took 1096 ms. Trace:
I20260709 12:12:58.568073 19697 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:58.411289 19917 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 8, Time since last communication: 0.057s
I20260709 12:12:58.565546 12605 rpcz_store.cc:276] 0709 12:12:57.468459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.563846 (+1095387us) service_pool.cc:224] Handling call
0709 12:12:58.564249 (+   403us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:58.564894 (+   645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.565142 (+   248us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.569192 19875 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:58.456507 13304 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:12:58.570569 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 114141us
I20260709 12:12:58.571313 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:58.523013 12604 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
W20260709 12:12:58.531414 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 154984us
I20260709 12:12:58.572760 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:58.572824 12496 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:58.531936 12601 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
W20260709 12:12:58.539168 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 143524us
I20260709 12:12:58.574019 12598 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:12:58.409699 19762 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:58.560685 19494 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:12:58.457223 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:58.574982 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3037) took 1103 ms. Trace:
I20260709 12:12:58.577759 12598 rpcz_store.cc:276] 0709 12:12:57.471776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.573508 (+1101732us) service_pool.cc:224] Handling call
0709 12:12:58.573839 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:58.574641 (+   802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.574905 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:12:58.578455 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:58.579527 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3038) took 1107 ms. Trace:
I20260709 12:12:58.579964 12598 rpcz_store.cc:276] 0709 12:12:57.472068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.578586 (+1106518us) service_pool.cc:224] Handling call
0709 12:12:58.578901 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:58.579217 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:58.579247 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:58.579288 (+    41us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:58.579318 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.579369 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:58.579472 (+   103us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.549822 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 757) took 1143 ms. Trace:
I20260709 12:12:58.582803 12603 rpcz_store.cc:276] 0709 12:12:57.406573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.548734 (+1142161us) service_pool.cc:224] Handling call
0709 12:12:58.549179 (+   445us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:58.549365 (+   186us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:58.549403 (+    38us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:58.549446 (+    43us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:12:58.549478 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.549530 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:58.549665 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.383093 19867 raft_consensus.cc:3055] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:58.576246 12572 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:58.573271 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:58.574687 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:58.574041 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 756) took 1178 ms. Trace:
I20260709 12:12:58.587795 12601 rpcz_store.cc:276] 0709 12:12:57.395692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.397130 (+1001438us) service_pool.cc:224] Handling call
0709 12:12:58.531659 (+134529us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:58.573754 (+ 42095us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.573981 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:12:58.590014 12605 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:12:58.589800 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:58.594542 12571 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:58.542222 19689 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:58.560617 19896 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:58.596555 12606 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
I20260709 12:12:58.598937 12602 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 18: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:58.602555 12572 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:12:58.586866 19443 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.685s	user 0.009s	sys 0.005s
I20260709 12:12:58.552672 19491 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:12:58.585719 19725 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:12:58.628549 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:58.591964 19845 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:12:58.586149 19867 raft_consensus.cc:740] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:58.633107 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:58.598745 19624 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:12:58.637495 19624 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.638089 19624 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:58.632472 19882 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:58.640844 19882 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:12:58.624594 19494 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.017s
I20260709 12:12:58.626458 19762 raft_consensus.cc:2764] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election decision vote started in defunct term 14: won
I20260709 12:12:58.634657 19851 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:12:58.645467 19874 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.525s	user 0.010s	sys 0.001s
I20260709 12:12:58.643167 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3031) took 1173 ms. Trace:
I20260709 12:12:58.647060 19853 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:58.647603 12605 rpcz_store.cc:276] 0709 12:12:57.469910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.569935 (+1100025us) service_pool.cc:224] Handling call
0709 12:12:58.570344 (+   409us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:58.590587 (+ 20243us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.642622 (+ 52035us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b5400375354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:58.642980 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":44589440}
I20260709 12:12:58.650787 19791 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:58.653272 19791 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:58.586503 19824 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.046s
I20260709 12:12:58.657045 19842 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22.
I20260709 12:12:58.607223 19831 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:58.658196 19831 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:58.659972 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:58.627034 19840 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:58.630414 19894 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.031s
I20260709 12:12:58.605908 19636 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:58.662925 19912 raft_consensus.cc:3055] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:12:58.663481 19912 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:58.664680 19912 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.13, Last appended by leader: 13, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:58.665598 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 751) took 1330 ms. Trace:
I20260709 12:12:58.601212 19689 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.666340 19689 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:58.666054 12604 rpcz_store.cc:276] 0709 12:12:57.335480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.623853 (+288373us) service_pool.cc:224] Handling call
0709 12:12:58.665372 (+1041519us) spinlock_profiling.cc:243] Waited 663 ms on lock 0x7b540022f6d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:58.665526 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1525125376}
I20260709 12:12:58.594242 19908 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:12:58.636487 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 63757.8us
I20260709 12:12:58.545943 19833 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:58.636811 19032 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:58.622886 19610 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:58.637059 19798 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:12:58.632711 19864 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:58.598614 19827 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.668150 12601 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:58.668318 12603 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:12:58.668519 12572 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:58.695147 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:58.696308 12572 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:58.585177 19524 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
W20260709 12:12:58.678092 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:58.699829 12530 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 23: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:58.697976 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:58.696381 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 335741us
I20260709 12:12:58.649885 19853 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:58.604102 19697 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.072s
I20260709 12:12:58.669438 19842 raft_consensus.cc:2764] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:12:58.660519 19831 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:58.669610 19725 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:58.711282 12496 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:58.712069 12646 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:58.692663 19798 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.714856 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:58.708478 13305 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:58.669842 19845 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:58.631417 19443 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:58.762929 19845 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:58.763895 19696 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:58.765002 19443 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:58.712826 13339 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 17: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:58.669610 19908 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:58.646068 19874 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:58.721750 19901 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.005s
W20260709 12:12:58.766620 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 58269.4us
I20260709 12:12:58.711192 19824 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.016s
I20260709 12:12:58.727486 13330 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:12:58.709019 19494 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:58.708751 19791 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.020s
I20260709 12:12:58.755681 19851 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:58.704638 19762 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:58.773823 19624 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:58.730137 19725 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:58.776355 19624 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:58.731390 19831 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:58.703619 19864 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:12:58.722972 13333 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 897) took 1121 ms. Trace:
I20260709 12:12:58.778883 19831 raft_consensus.cc:697] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:58.714831 19909 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.017s
I20260709 12:12:58.780290 19831 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:58.732867 19723 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.784548 19723 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:58.703473 19840 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:12:58.724162 19875 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:58.724210 19610 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:58.724587 19867 consensus_queue.cc:260] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:58.724900 19842 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:58.724663 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:58.722416 19882 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.015s
I20260709 12:12:58.738063 19826 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:58.738768 19693 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:58.740753 19914 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:58.694269 19827 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:58.741523 19798 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:12:58.741777 19917 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:58.744436 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3469) took 1475 ms. Trace:
I20260709 12:12:58.745934 12496 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:58.778838 19696 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:58.749799 19833 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:58.752274 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:58.752964 19853 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:12:58.753950 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3384) took 1368 ms. Trace:
I20260709 12:12:58.755681 19636 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:12:58.762027 12643 leader_election.cc:336] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.621s (SENT)
W20260709 12:12:58.823758 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 61787.9us
W20260709 12:12:58.825320 12643 leader_election.cc:336] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.912s (SENT)
I20260709 12:12:58.766899 19779 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:58.779290 13333 rpcz_store.cc:276] 0709 12:12:57.601573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.634783 (+ 33210us) service_pool.cc:224] Handling call
0709 12:12:58.722776 (+1087993us) spinlock_profiling.cc:243] Waited 919 ms on lock 0x7b540055ee54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:58.722912 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2114145536}
I20260709 12:12:58.706087 19896 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:58.767453 12526 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:12:58.726289 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 13 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:58.825824 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:58.789462 12569 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:12:58.829002 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 417427us
I20260709 12:12:58.795302 13336 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 17: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:58.800498 12600 raft_consensus.cc:2368] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19.
W20260709 12:12:58.804703 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58857.9us
W20260709 12:12:58.831210 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:58.747570 19491 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:12:58.833107 13333 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17. Ops: [16.15-16.15]
I20260709 12:12:58.802701 12671 rpcz_store.cc:276] 0709 12:12:57.269314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.182485 (+913171us) service_pool.cc:224] Handling call
0709 12:12:58.182857 (+   372us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:58.184144 (+  1287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:58.184176 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:58.199160 (+ 14984us) log.cc:844] Serialized 52 byte log entry
0709 12:12:58.387193 (+188033us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:58.743583 (+356390us) log.cc:844] Serialized 14 byte log entry
0709 12:12:58.743923 (+   340us) log.cc:844] Serialized 14 byte log entry
0709 12:12:58.744098 (+   175us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.744168 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:58.744214 (+    46us) raft_consensus.cc:1685] finished
0709 12:12:58.744225 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:58.744378 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":356821,"prepare.queue_time_us":359000,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":51,"raft.queue_time_us":8416,"raft.run_cpu_time_us":297,"raft.run_wall_time_us":7805,"thread_start_us":91080,"threads_started":1,"wal-append.queue_time_us":169889}
W20260709 12:12:58.834515 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 45191.8us
I20260709 12:12:58.803898 19914 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:58.800179 19693 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.817471 12602 rpcz_store.cc:276] 0709 12:12:57.385900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.185848 (+799948us) service_pool.cc:224] Handling call
0709 12:12:58.753648 (+567800us) spinlock_profiling.cc:243] Waited 282 ms on lock 0x7b54002ff654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:58.753857 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":649429760}
I20260709 12:12:58.831807 12494 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:58.810925 19867 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:58.850235 12569 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:58.811084 19826 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:58.861569 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:58.862016 12677 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:12:58.813362 19909 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:58.865550 12680 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:12:58.870833 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:58.871685 12672 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:58.882905 12676 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:12:58.883265 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 51592us
I20260709 12:12:58.886035 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:58.886998 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:58.803042 19842 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:58.828898 19896 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.889117 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:58.892603 12675 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
W20260709 12:12:58.906808 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:58.848361 19875 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:58.924952 12494 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:58.869031 19762 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:58.915853 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:58.843844 19693 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:58.868475 19524 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:12:58.868021 19909 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:58.929667 12674 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 23: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:58.929968 12524 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
W20260709 12:12:58.935395 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 229630us
I20260709 12:12:58.937496 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:58.940634 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:58.943539 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:58.946071 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:58.945536 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 120265us
I20260709 12:12:58.946920 12528 raft_consensus.cc:2393] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 19: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:12:58.947070 12643 leader_election.cc:336] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.995s (SENT)
I20260709 12:12:58.947801 12643 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:58.919896 19896 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:58.822724 19908 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:58.865001 19624 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.079s
I20260709 12:12:58.938076 19705 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:58.934935 19909 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:12:58.889739 19828 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.060s
I20260709 12:12:58.929128 19835 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:58.954025 19769 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:12:58.983860 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:58.957127 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 362644us
I20260709 12:12:58.992235 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:59.008628 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:59.009815 12529 raft_consensus.cc:2368] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
I20260709 12:12:59.012794 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:59.013794 12529 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:12:59.016080 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:59.016727 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:59.026026 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:12:59.019716 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 307736us
I20260709 12:12:59.027323 12646 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:59.019824 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:59.029060 12529 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:12:59.031023 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:59.016553 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3044) took 1534 ms. Trace:
I20260709 12:12:59.015550 12672 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
W20260709 12:12:59.045648 12600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3039) took 1573 ms (client timeout 1912 ms). Trace:
W20260709 12:12:59.047130 12600 rpcz_store.cc:269] 0709 12:12:57.472354 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.369043 (+896689us) service_pool.cc:224] Handling call
0709 12:12:59.045389 (+676346us) spinlock_profiling.cc:243] Waited 418 ms on lock 0x7b5400374e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:12:59.045573 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":961365760}
I20260709 12:12:58.917559 19864 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:59.064932 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:59.070026 12646 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:59.070569 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:12:59.072947 12523 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:59.073262 12522 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 18: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:12:58.990432 19835 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:59.019981 19830 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:12:59.015439 19925 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:12:59.085820 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 69835.5us
I20260709 12:12:59.093122 12571 leader_election.cc:400] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
I20260709 12:12:59.093573 12571 leader_election.cc:403] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:12:59.066713 19526 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:12:59.063844 19636 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.015s
I20260709 12:12:59.041780 19882 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:59.021524 19443 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 12:12:59.076719 19874 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:59.116957 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:59.086709 19835 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:12:59.086264 19917 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:12:59.089432 19769 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:12:59.112857 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 188070us
I20260709 12:12:59.095690 19526 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:59.131219 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:59.089673 19723 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:59.116279 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:12:59.103317 19924 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.110s	user 0.001s	sys 0.000s
W20260709 12:12:59.117880 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 130889us
I20260709 12:12:59.114606 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:12:59.143543 13336 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:59.044409 19845 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.055s
I20260709 12:12:59.095237 19791 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:12:59.118414 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.119243 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:59.123677 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:59.152133 12606 raft_consensus.cc:2410] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 18 because replica has last-logged OpId of term: 17 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:12:59.099015 19762 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:59.113766 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:59.129151 12494 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:12:59.131219 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:59.132104 12602 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:59.109987 19827 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:59.138219 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 17 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:59.146082 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:59.161505 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:59.118703 19928 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:59.116523 19930 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:12:59.160593 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46921.5us
I20260709 12:12:59.124879 19917 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:59.157186 12523 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:12:59.159032 12603 raft_consensus.cc:2410] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 15 because replica has last-logged OpId of term: 14 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 12 index: 8.
I20260709 12:12:59.162228 12604 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:12:59.165768 12605 raft_consensus.cc:2393] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 22: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:59.096155 19830 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:59.171931 12572 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:59.172063 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:12:59.135659 19526 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:12:58.981278 19491 consensus_queue.cc:1175] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "193a40da6e364ead8ab561e14c3cc8d1" responder_term: 22 status { last_received { term: 20 index: 12 } last_committed_idx: 12 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: [21.13-21.13]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:12:59.150507 19769 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:59.184851 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:59.181691 19942 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:12:59.159751 19762 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:59.169996 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.167764 19624 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:12:59.114076 19882 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:12:59.179601 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50545.6us
I20260709 12:12:59.180317 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:59.180521 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:59.142810 19625 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:12:59.180969 12680 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:12:59.198750 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:59.199559 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:59.183100 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 67389.7us
I20260709 12:12:59.150594 19791 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:59.166692 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:59.207831 12672 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:12:59.209287 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:59.209873 12672 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:59.210845 12606 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:12:59.118927 19851 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 12:12:59.216099 19851 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:59.174028 19842 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:59.194695 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:59.195901 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:59.217737 12678 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 17: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:12:59.218168 19636 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:12:59.218854 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:59.220130 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:59.220680 12678 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:12:59.179430 19926 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
I20260709 12:12:59.223398 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 752) took 1874 ms. Trace:
I20260709 12:12:59.220374 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:59.223776 12597 rpcz_store.cc:276] 0709 12:12:57.349592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.358695 (+1009103us) service_pool.cc:224] Handling call
0709 12:12:58.376956 (+ 18261us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:12:58.394074 (+ 17118us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:58.394110 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:12:58.406930 (+ 12820us) log.cc:844] Serialized 52 byte log entry
0709 12:12:58.789701 (+382771us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:58.789748 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:58.789822 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:59.223134 (+433312us) raft_consensus.cc:1685] finished
0709 12:12:59.223163 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:59.223331 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4140,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":5231,"thread_start_us":368010,"threads_started":1,"wal-append.queue_time_us":518705}
I20260709 12:12:59.226310 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:59.227828 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:59.229856 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:59.227875 12597 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
W20260709 12:12:59.235230 19933 log.cc:927] Time spent T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:12:59.235525 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:59.356125 12603 raft_consensus.cc:2410] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:12:59.357316 13336 raft_consensus.cc:2410] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 19 because replica has last-logged OpId of term: 17 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 12.
I20260709 12:12:59.240589 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:12:59.248036 19932 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:12:59.194258 19624 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:12:59.250176 19725 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:12:59.252888 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:59.254316 19801 consensus_queue.cc:1059] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:59.255867 19830 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:59.256454 19912 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:12:59.256722 19723 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:12:59.257488 19925 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:12:59.259393 19610 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:12:59.260546 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3046) took 1693 ms. Trace:
I20260709 12:12:59.260911 19914 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:59.437146 12526 rpcz_store.cc:276] 0709 12:12:57.567735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:57.694356 (+126621us) service_pool.cc:224] Handling call
0709 12:12:57.796213 (+101857us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:58.846826 (+1050613us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:59.260258 (+413432us) spinlock_profiling.cc:243] Waited 970 ms on lock 0x7b540042a9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:59.260449 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2230661504}
I20260709 12:12:59.263681 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:59.236585 19791 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:12:59.348788 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 119010us
I20260709 12:12:59.443593 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:59.260375 19850 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:12:59.349097 19917 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:12:59.355078 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.178876 19827 consensus_queue.cc:1059] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:12:59.349459 19769 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:59.181504 19874 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.108s
I20260709 12:12:59.238945 12678 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:12:59.357415 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:59.366534 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:59.453442 12606 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:12:59.353981 12600 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:12:59.370148 12680 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:12:59.378531 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.454057 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:12:59.386973 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:59.174568 19928 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:59.349476 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:59.186065 19779 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.152s
I20260709 12:12:59.397768 12672 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:12:59.400913 12604 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:12:59.206741 19942 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:59.343598 19636 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:12:59.359269 19923 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.371s	user 0.000s	sys 0.001s
I20260709 12:12:59.405261 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:12:59.387329 19936 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.283s	user 0.000s	sys 0.005s
W20260709 12:12:59.364807 19689 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.698s	user 0.017s	sys 0.003s
W20260709 12:12:59.367944 19934 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.184s	user 0.002s	sys 0.003s
W20260709 12:12:59.391341 19937 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.196s	user 0.002s	sys 0.002s
W20260709 12:12:59.426955 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:59.403338 19935 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.300s	user 0.001s	sys 0.004s
W20260709 12:12:59.406625 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 458737us
W20260709 12:12:59.474822 12494 connection.cc:256] RPC call timeout handler was delayed by -1.12813763500003s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1562s for a response
I20260709 12:12:59.402586 19624 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:59.203989 12646 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:59.550990 19944 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:12:59.395529 19894 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.044s
I20260709 12:12:59.553669 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3026) took 2104 ms. Trace:
I20260709 12:12:59.549882 19945 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.005s
I20260709 12:12:59.554268 12599 rpcz_store.cc:276] 0709 12:12:57.450100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.552758 (+1102658us) service_pool.cc:224] Handling call
0709 12:12:58.553160 (+   402us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:58.598958 (+ 45798us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:12:58.740920 (+141962us) spinlock_profiling.cc:243] Waited 20.1 ms on lock 0x7b18004e5a40. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf 00007f1488a3a038
0709 12:12:58.928673 (+187753us) log.cc:844] Serialized 14 byte log entry
0709 12:12:59.128913 (+200240us) log.cc:844] Serialized 14 byte log entry
0709 12:12:59.129339 (+   426us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:59.132785 (+  3446us) log.cc:844] Serialized 26 byte log entry
0709 12:12:59.133021 (+   236us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:12:59.133055 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:59.133171 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:59.258767 (+125596us) spinlock_profiling.cc:243] Waited 2.81 ms on lock 0x7b18004e5a40. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:12:59.553369 (+294602us) raft_consensus.cc:1685] finished
0709 12:12:59.553402 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:59.553594 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50169,"prepare.queue_time_us":145717,"prepare.run_cpu_time_us":79,"prepare.run_wall_time_us":81,"raft.queue_time_us":48745,"raft.run_cpu_time_us":370,"raft.run_wall_time_us":15983,"spinlock_wait_cycles":52621952,"thread_start_us":320385,"threads_started":2,"wal-append.queue_time_us":250873}
I20260709 12:12:59.436849 19947 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:59.433827 19949 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:12:59.483196 12680 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:12:59.440250 19901 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:59.437294 19725 raft_consensus.cc:697] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 21 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:12:59.521526 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:59.522078 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:59.610728 12674 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:12:59.448855 19539 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:12:59.544821 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:59.434309 12571 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:59.457561 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:59.525182 19494 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:59.561980 19624 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:59.562837 19762 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:59.468147 19524 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:59.545387 12496 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:59.565673 19957 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:12:59.469777 19636 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:12:59.586434 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 138551us
I20260709 12:12:59.483963 19882 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:12:59.562067 19798 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.751s	user 0.011s	sys 0.000s
W20260709 12:12:59.622782 19923 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:12:59.613129 12604 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 21: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:12:59.651618 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 106332us
I20260709 12:12:59.544993 19917 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:59.543941 19618 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:59.573082 19827 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:12:59.627893 12494 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:59.627345 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:12:59.623196 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 188979us
I20260709 12:12:59.673949 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:59.675606 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:59.633042 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:59.545756 19689 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:59.610746 12606 raft_consensus.cc:2393] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:12:59.622367 19948 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.225s	user 0.000s	sys 0.001s
I20260709 12:12:59.468690 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:59.663825 12496 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:12:59.612247 19539 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:59.665295 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:59.682744 12571 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:59.685091 12571 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:12:59.685256 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:59.686147 12604 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:12:59.677336 12599 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:12:59.672953 12603 raft_consensus.cc:2376] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21. Re-sending same reply.
I20260709 12:12:59.652035 19636 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:59.664883 19914 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:12:59.683728 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.634212 19845 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:12:59.688977 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 17 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:59.688158 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:59.697636 12603 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:59.697777 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:12:59.686432 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:12:59.699005 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 71260.2us
I20260709 12:12:59.700076 12494 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:59.648595 19524 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:59.672755 19801 consensus_queue.cc:1243] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:12:59.674083 19618 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:59.695899 12528 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 17. Current term is 18. Ops: []
I20260709 12:12:59.690937 19874 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:12:59.671782 19827 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:59.702646 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:59.684639 19610 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:12:59.704705 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:12:59.682624 19696 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:12:59.726918 12604 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 12:12:59.727394 12604 pending_rounds.cc:85] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 12
I20260709 12:12:59.727758 12604 pending_rounds.cc:107] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:12:59.728120 12604 raft_consensus.cc:2889] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:12:59.726720 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 522866us
I20260709 12:12:59.642546 19494 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:59.733816 12672 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
W20260709 12:12:59.708928 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 230842us
I20260709 12:12:59.667548 19798 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:59.648375 19526 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:59.691084 19882 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:59.687449 19956 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:12:59.703810 19954 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:12:59.724674 19827 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:12:59.710938 19762 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:12:59.701735 19914 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:59.722793 19965 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:12:59.729212 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:59.748607 19524 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:12:59.744709 19875 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.817s	user 0.013s	sys 0.001s
I20260709 12:12:59.725263 19963 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:12:59.767014 12646 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:12:59.767040 12572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.009s	sys 0.009s
W20260709 12:12:59.769465 12572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.009s	sys 0.009s
I20260709 12:12:59.757041 12603 raft_consensus.cc:2393] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 17: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:12:59.730324 19831 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:12:59.771647 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 599769us
W20260709 12:12:59.743050 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 67538.5us
I20260709 12:12:59.764271 12599 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:12:59.772907 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:59.773232 12572 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:59.760941 12671 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 16. Current term is 17. Ops: []
I20260709 12:12:59.764722 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:12:59.774389 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:59.775022 12572 leader_election.cc:400] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
I20260709 12:12:59.696193 19725 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:59.756160 19882 raft_consensus.cc:697] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:59.769449 19953 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.061s	user 0.000s	sys 0.001s
W20260709 12:12:59.847201 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:12:59.847807 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:12:59.790786 12572 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:12:59.764740 19696 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:12:59.622320 19901 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:59.771227 19875 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:12:59.611141 19947 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:59.797828 19762 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:59.862032 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 71333.2us
I20260709 12:12:59.779697 19956 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:12:59.779102 19909 consensus_meta.cc:287] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.785s	user 0.013s	sys 0.000s
W20260709 12:12:59.798266 19939 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
I20260709 12:12:59.897827 12572 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:12:59.785020 19524 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:12:59.667725 19942 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:59.836557 19972 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:12:59.813856 19779 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:12:59.919544 12572 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:12:59.866763 19610 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:12:59.873627 19971 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:12:59.902230 19624 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:12:59.932209 19963 raft_consensus.cc:697] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:12:59.897830 19928 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:12:59.874275 19967 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:12:59.902637 19874 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:12:59.955734 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 181445us
I20260709 12:12:59.955165 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:12:59.970242 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:59.032789 12598 rpcz_store.cc:276] 0709 12:12:57.482623 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.581272 (+1098649us) service_pool.cc:224] Handling call
0709 12:12:58.581919 (+   647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:58.582328 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:12:58.603824 (+ 21496us) log.cc:844] Serialized 14 byte log entry
0709 12:12:58.752994 (+149170us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b1800441ac0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a 00007f1488a2aa2a
0709 12:12:59.016167 (+263173us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:12:59.016228 (+    61us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:12:59.016286 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:59.016353 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:59.016497 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":101394,"prepare.queue_time_us":22239,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":25544,"spinlock_wait_cycles":43809024,"thread_start_us":237075,"threads_started":1}
I20260709 12:12:59.974928 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3487) took 1066 ms. Trace:
I20260709 12:12:59.975456 12677 rpcz_store.cc:276] 0709 12:12:58.908720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.908896 (+   176us) service_pool.cc:224] Handling call
0709 12:12:58.909271 (+   375us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:58.909686 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:12:58.909715 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:58.910236 (+   521us) log.cc:844] Serialized 26 byte log entry
0709 12:12:59.113259 (+203023us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:12:59.113335 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:59.113437 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:12:59.974633 (+861196us) raft_consensus.cc:1685] finished
0709 12:12:59.974656 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:12:59.974860 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":780,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":10287,"thread_start_us":174884,"threads_started":1,"wal-append.queue_time_us":525527}
I20260709 12:12:59.804713 19831 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:59.932950 19971 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:12:59.930799 19769 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:12:59.910179 19909 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:12:59.925141 19875 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:12:59.895304 19960 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.060s
I20260709 12:12:59.985212 12606 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
W20260709 12:13:00.015182 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 45147.9us
I20260709 12:13:00.019672 12643 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:12:59.911317 19914 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:12:59.936206 19689 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:00.010690 19971 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:00.025767 19975 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:13:00.026543 19950 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.291s	user 0.001s	sys 0.000s
I20260709 12:12:59.970898 19967 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:00.029173 12674 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:12:59.954547 19972 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:12:59.923185 19980 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.121s
I20260709 12:12:59.933674 19779 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:00.030032 19896 consensus_meta.cc:287] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.044s	user 0.008s	sys 0.007s
I20260709 12:13:00.024143 19957 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:13:00.071038 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 51464.2us
W20260709 12:13:00.071274 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 371279us
I20260709 12:13:00.071908 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:00.075600 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3486) took 1208 ms. Trace:
I20260709 12:12:59.953476 19882 consensus_queue.cc:237] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 16.7, Last appended by leader: 7, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.026535 19610 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.018328 19762 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.9, Last appended by leader: 9, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.076215 12676 rpcz_store.cc:276] 0709 12:12:58.866893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.883835 (+ 16942us) service_pool.cc:224] Handling call
0709 12:12:58.884162 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:58.884693 (+   531us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:12:59.225243 (+340550us) spinlock_profiling.cc:243] Waited 22.5 ms on lock 0x7b18003e7a60. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf
0709 12:12:59.225391 (+   148us) log.cc:844] Serialized 14 byte log entry
0709 12:12:59.816351 (+590960us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:59.817067 (+   716us) log.cc:844] Serialized 26 byte log entry
0709 12:12:59.817269 (+   202us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:12:59.817317 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:59.817419 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:00.065661 (+248242us) raft_consensus.cc:1685] finished
0709 12:13:00.075273 (+  9612us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.075511 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51174,"prepare.queue_time_us":104022,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":62,"raft.queue_time_us":176325,"raft.run_cpu_time_us":317,"raft.run_wall_time_us":32882,"spinlock_wait_cycles":51812224,"thread_start_us":692511,"threads_started":3,"wal-append.queue_time_us":426320}
I20260709 12:13:00.127990 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 862) took 1259 ms. Trace:
I20260709 12:13:00.148696 12522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:13:00.125957 19901 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:00.148732 19791 consensus_meta.cc:287] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.694s	user 0.012s	sys 0.000s
I20260709 12:13:00.157580 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3064) took 1034 ms. Trace:
I20260709 12:13:00.112349 19874 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 16.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:00.181602 12603 rpcz_store.cc:276] 0709 12:12:59.123107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.704359 (+581252us) service_pool.cc:224] Handling call
0709 12:13:00.157266 (+452907us) spinlock_profiling.cc:243] Waited 32 ms on lock 0x7b54002e54d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:00.157485 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":73506688}
I20260709 12:13:00.183846 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:00.184235 19952 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.627s	user 0.002s	sys 0.001s
W20260709 12:13:00.109018 19940 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.921s	user 0.009s	sys 0.006s
W20260709 12:13:00.107620 19867 consensus_meta.cc:287] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.239s	user 0.014s	sys 0.000s
I20260709 12:13:00.154032 19697 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.163408 19908 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:00.186867 19697 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.189270 19697 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:00.160519 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 786) took 1013 ms. Trace:
I20260709 12:13:00.136097 19779 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:00.062482 19618 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:00.051520 19967 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:00.190577 12671 rpcz_store.cc:276] 0709 12:12:59.146851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.146981 (+   130us) service_pool.cc:224] Handling call
0709 12:12:59.147538 (+   557us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:59.798451 (+650913us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.160057 (+361606us) spinlock_profiling.cc:243] Waited 612 ms on lock 0x7b54000c26d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:00.160413 (+   356us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1407680256}
I20260709 12:13:00.102837 19964 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority
I20260709 12:13:00.153772 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:00.149585 19982 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.194933 19625 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:00.195577 19625 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:00.195509 19982 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.180294 12522 raft_consensus.cc:606] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:00.197175 12522 raft_consensus.cc:614] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:00.063009 19798 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:00.130530 19835 consensus_meta.cc:287] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.997s	user 0.015s	sys 0.000s
I20260709 12:13:00.154274 19975 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.198678 19835 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.201161 19835 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:00.124696 19930 consensus_meta.cc:287] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.942s	user 0.011s	sys 0.000s
I20260709 12:13:00.203116 19930 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
I20260709 12:13:00.181581 19791 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:00.153820 19853 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:00.204972 19853 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.205282 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:00.179554 12525 rpcz_store.cc:276] 0709 12:12:58.868952 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.869203 (+   251us) service_pool.cc:224] Handling call
0709 12:12:58.918875 (+ 49672us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:58.935699 (+ 16824us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:12:58.935736 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:58.936267 (+   531us) log.cc:844] Serialized 26 byte log entry
0709 12:12:59.174761 (+238494us) spinlock_profiling.cc:243] Waited 23.5 ms on lock 0x7b1800357460. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:12:59.400234 (+225473us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:12:59.400386 (+   152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:12:59.400463 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:00.127487 (+727024us) raft_consensus.cc:1685] finished
0709 12:13:00.127518 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.127796 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39651,"raft.queue_time_us":32651,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":5076,"spinlock_wait_cycles":54097792,"thread_start_us":225237,"threads_started":1,"wal-append.queue_time_us":668649}
I20260709 12:13:00.130508 19636 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.212450 19636 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:00.213665 12671 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:00.083478 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:00.215956 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 132626us
I20260709 12:13:00.217664 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:00.217978 19982 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:00.204228 19689 raft_consensus.cc:3055] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:00.228513 19689 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:00.229909 19689 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:00.231900 12522 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:13:00.199097 19975 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.234318 12605 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:00.234617 19975 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:00.236289 19964 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:00.237118 19964 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:00.237408 12605 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: [18.10-18.10]
I20260709 12:13:00.237496 19964 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:00.240677 19524 raft_consensus.cc:3055] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:00.063309 19972 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.173615 19827 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.283962 19833 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.287886 12527 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 12:13:00.338709 12527 pending_rounds.cc:85] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 12
I20260709 12:13:00.302524 12606 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:13:00.282110 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:00.341020 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:00.341131 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:00.297377 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3388) took 1034 ms. Trace:
I20260709 12:13:00.343858 12604 rpcz_store.cc:276] 0709 12:12:59.262655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.738874 (+476219us) service_pool.cc:224] Handling call
0709 12:12:59.739405 (+   531us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:59.927514 (+188109us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:59.927562 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:59.969891 (+ 42329us) log.cc:844] Serialized 26 byte log entry
0709 12:13:00.296889 (+326998us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:13:00.296952 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.297053 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:00.297115 (+    62us) raft_consensus.cc:1685] finished
0709 12:13:00.297135 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.297292 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":10497,"raft.run_cpu_time_us":171,"raft.run_wall_time_us":5252,"thread_start_us":456485,"threads_started":2,"wal-append.queue_time_us":269292}
I20260709 12:13:00.320045 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:00.319850 12603 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:00.351649 12603 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:00.221442 19791 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:00.139544 19909 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:00.070547 19875 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:00.140750 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:00.070176 19526 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:00.086092 19693 consensus_meta.cc:287] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.140s	user 0.010s	sys 0.006s
I20260709 12:13:00.095989 19960 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:00.285336 19824 raft_consensus.cc:3055] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:13:00.081657 19963 consensus_queue.cc:237] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:00.224416 19968 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.480s	user 0.000s	sys 0.001s
I20260709 12:13:00.122866 19831 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:00.105216 19938 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.926s	user 0.000s	sys 0.005s
I20260709 12:13:00.108453 19947 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:00.073019 19896 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.232599 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3049) took 1126 ms. Trace:
I20260709 12:13:00.488209 12600 rpcz_store.cc:276] 0709 12:12:59.105903 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.106184 (+   281us) service_pool.cc:224] Handling call
0709 12:13:00.232392 (+1126208us) spinlock_profiling.cc:243] Waited 136 ms on lock 0x7b54002715d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:00.232527 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":312631040}
I20260709 12:13:00.244051 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:00.246502 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2944) took 1261 ms. Trace:
I20260709 12:13:00.238067 19964 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:00.253952 12530 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: []
W20260709 12:13:00.261181 19917 consensus_meta.cc:287] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.594s	user 0.011s	sys 0.001s
I20260709 12:13:00.261919 19850 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.180368 12676 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:13:00.495908 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:00.272006 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 54435.5us
I20260709 12:13:00.275444 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:00.499792 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:00.277076 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:00.277117 19539 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.278226 19949 raft_consensus.cc:3055] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:13:00.279901 12601 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:13:00.095556 19494 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:00.127225 19912 consensus_meta.cc:287] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.694s	user 0.009s	sys 0.004s
W20260709 12:13:00.151575 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.432359 19875 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:00.316532 12599 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:00.519332 12600 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 16. Current term is 17. Ops: [16.11-16.11]
I20260709 12:13:00.328953 13302 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:00.334914 12672 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:13:00.339000 12527 pending_rounds.cc:107] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:13:00.327622 12597 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:00.450486 19693 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.520834 12527 raft_consensus.cc:2889] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:13:00.333033 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.521628 13334 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 901) took 1157 ms. Trace:
I20260709 12:13:00.522194 13334 rpcz_store.cc:276] 0709 12:12:59.364137 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.364341 (+   204us) service_pool.cc:224] Handling call
0709 12:13:00.518904 (+1154563us) spinlock_profiling.cc:243] Waited 974 ms on lock 0x7b54006419d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:00.521557 (+  2653us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2239234944}
W20260709 12:13:00.361689 12646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.000s	sys 0.013s
W20260709 12:13:00.523738 12646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.000s	sys 0.014s
I20260709 12:13:00.347581 12675 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:13:00.527113 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:00.317310 19524 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:00.363144 19874 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:00.530877 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3070) took 1364 ms. Trace:
I20260709 12:13:00.391077 19971 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.531344 12597 rpcz_store.cc:276] 0709 12:12:59.166594 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.238780 (+ 72186us) service_pool.cc:224] Handling call
0709 12:13:00.530696 (+1291916us) spinlock_profiling.cc:243] Waited 761 ms on lock 0x7b5400079c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:00.530804 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1750420608}
W20260709 12:13:00.395226 12678 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.009s	sys 0.004s
W20260709 12:13:00.532253 12678 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.777s	user 0.010s	sys 0.004s
I20260709 12:13:00.532929 12597 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:13:00.533057 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3005) took 1355 ms. Trace:
W20260709 12:13:00.362759 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 288779us
I20260709 12:13:00.402030 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:00.406527 12679 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:00.420800 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:00.536332 12526 raft_consensus.cc:2410] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 10.
I20260709 12:13:00.338340 19833 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:00.294384 19951 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.642s	user 0.002s	sys 0.001s
W20260709 12:13:00.313534 19845 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.611s	user 0.012s	sys 0.000s
I20260709 12:13:00.441390 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:00.177126 12572 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:00.288269 19723 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 12:13:00.407930 19864 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.845s	user 0.003s	sys 0.015s
I20260709 12:13:00.304397 19975 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:00.574687 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:00.371371 19914 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.075s
I20260709 12:13:00.408193 19827 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:00.481170 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:00.490289 12528 rpcz_store.cc:276] 0709 12:12:58.985261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.985527 (+   266us) service_pool.cc:224] Handling call
0709 12:12:58.985853 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:12:59.712052 (+726199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.246203 (+534151us) spinlock_profiling.cc:243] Waited 710 ms on lock 0x7b5400210cd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:00.246425 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1632542080}
I20260709 12:13:00.324463 19625 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.219599 19853 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:00.498430 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:00.449494 19960 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:00.501700 12643 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:00.450212 19824 raft_consensus.cc:740] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:00.506868 12523 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:13:00.507794 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3052) took 1400 ms. Trace:
I20260709 12:13:00.659961 12601 rpcz_store.cc:276] 0709 12:12:59.106941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.117069 (+ 10128us) service_pool.cc:224] Handling call
0709 12:13:00.279472 (+1162403us) spinlock_profiling.cc:243] Waited 1.18 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fbeee 000000000036eb39 00007f148d96f1e0 00007f148da380c4 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:13:00.279653 (+   181us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:00.507436 (+227783us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.507727 (+   291us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2703616}
I20260709 12:13:00.660732 12674 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 19. Current term is 20. Ops: [19.9-19.9]
I20260709 12:13:00.509246 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:00.652155 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:13:00.514165 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:00.122188 19826 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.235s	user 0.013s	sys 0.002s
I20260709 12:13:00.515112 12529 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 18: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:00.321482 19972 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:00.534485 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 205627us
I20260709 12:13:00.533445 12678 rpcz_store.cc:276] 0709 12:12:59.177579 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.452217 (+274638us) service_pool.cc:224] Handling call
0709 12:12:59.452548 (+   331us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:59.715571 (+263023us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:59.715607 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:59.717262 (+  1655us) log.cc:844] Serialized 26 byte log entry
0709 12:13:00.532656 (+815394us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:00.532699 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.532768 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:00.532837 (+    69us) raft_consensus.cc:1685] finished
0709 12:13:00.532850 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.532977 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18671,"raft.queue_time_us":202526,"raft.run_cpu_time_us":216,"raft.run_wall_time_us":15585,"thread_start_us":964040,"threads_started":2,"wal-append.queue_time_us":379426}
I20260709 12:13:00.546289 12525 raft_consensus.cc:2393] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 21: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:00.538002 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:00.669330 12672 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:13:00.544426 12675 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:13:00.556684 12494 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:13:00.563318 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 386321us
I20260709 12:13:00.567188 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2947) took 1572 ms. Trace:
I20260709 12:13:00.506654 19947 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:00.561946 12671 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:13:00.508181 19949 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:00.363238 19930 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.676355 13302 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:00.676942 12531 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 20 index: 17. (index mismatch)
I20260709 12:13:00.613300 19980 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.040s
I20260709 12:13:00.679354 12675 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23. Ops: []
W20260709 12:13:00.556319 19929 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.308s	user 0.013s	sys 0.010s
I20260709 12:13:00.541893 19539 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:00.722915 19625 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:00.679136 19970 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:00.724154 19970 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:00.724592 19970 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:00.726423 19833 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:00.659150 19762 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:00.711614 19929 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.463s	user 0.014s	sys 0.011s
I20260709 12:13:00.727360 19762 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.729085 12675 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:13:00.729735 19762 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:00.733301 20013 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:13:00.659405 19960 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:00.739675 19693 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:00.569600 19864 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.007s	user 0.004s	sys 0.015s
W20260709 12:13:00.746935 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 190384us
I20260709 12:13:00.529560 19526 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:00.505028 19967 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.613896 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:00.568853 19845 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:13:00.569285 19723 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:00.507555 19850 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:00.615733 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:00.576771 19494 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:13:00.596534 12496 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:00.617697 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:00.617834 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:00.558807 19962 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:13:00.568480 19636 consensus_queue.cc:1059] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 17.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.049s
W20260709 12:13:00.663930 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 162377us
I20260709 12:13:00.785928 12601 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:13:00.559942 19874 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.810771 12530 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:00.670610 12527 rpcz_store.cc:276] 0709 12:12:58.993985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.994155 (+   170us) service_pool.cc:224] Handling call
0709 12:12:59.044049 (+ 49894us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.521472 (+1477423us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.554626 (+ 33154us) spinlock_profiling.cc:243] Waited 1.24 s on lock 0x7b54003cbed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:00.566001 (+ 11375us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2859715968}
I20260709 12:13:00.820372 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:00.821391 12599 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:00.672947 12679 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:13:00.593223 19975 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.671795 19697 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:00.680877 19801 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:00.837263 19975 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:00.681217 19949 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 16.15, Last appended by leader: 15, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:00.838605 19949 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:00.682448 19990 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 17.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.082s
I20260709 12:13:00.680934 19930 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.844313 19930 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:00.683998 20015 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:13:00.686636 19971 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:00.688884 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:00.690527 19830 consensus_queue.cc:1059] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 19.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:13:00.848096 19830 raft_consensus.cc:3055] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:00.848541 19830 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:00.698659 19955 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.124s
I20260709 12:13:00.700270 19524 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:00.850641 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.850692 19524 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:00.849604 19830 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:00.852313 19830 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:00.702170 19875 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:00.650285 19977 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.854565 19977 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:00.704296 19824 consensus_queue.cc:260] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:00.856083 19824 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:00.677767 19840 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.858332 19494 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.858496 19840 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:00.858922 19494 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:00.859648 19779 consensus_queue.cc:1175] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" responder_term: 17 status { last_received { term: 15 index: 11 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 16. Current term is 17. Ops: [16.11-16.11]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:13:00.705869 19896 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:00.707010 19998 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:13:00.861188 19494 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:00.862061 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3013) took 1295 ms. Trace:
I20260709 12:13:00.862442 12673 rpcz_store.cc:276] 0709 12:12:59.566732 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.567040 (+   308us) service_pool.cc:224] Handling call
0709 12:12:59.609822 (+ 42782us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:59.927175 (+317353us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:12:59.927224 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:12:59.933472 (+  6248us) log.cc:844] Serialized 26 byte log entry
0709 12:13:00.179301 (+245829us) spinlock_profiling.cc:243] Waited 3.01 ms on lock 0x7b18003286c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:13:00.447171 (+267870us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:00.447217 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.538419 (+ 91202us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b54000131d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:00.538472 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:00.861841 (+323369us) raft_consensus.cc:1685] finished
0709 12:13:00.861862 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.861993 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13240,"raft.queue_time_us":54878,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":6987,"spinlock_wait_cycles":705695744,"thread_start_us":267648,"threads_started":1,"wal-append.queue_time_us":553620}
I20260709 12:13:00.865145 19990 raft_consensus.cc:3055] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Stepping down as leader of term 17
I20260709 12:13:00.693832 19961 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:00.865764 19990 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:00.867743 19990 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 17.11, Last appended by leader: 11, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.869527 19990 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:00.711491 12569 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:00.754272 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:00.870651 13334 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
W20260709 12:13:00.870945 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 159554us
I20260709 12:13:00.758496 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:00.871732 12606 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:00.766995 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2948) took 1762 ms. Trace:
I20260709 12:13:00.874461 19967 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:00.786935 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:00.875371 19975 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:13:00.876099 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 89270.4us
I20260709 12:13:00.876709 19971 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:00.877275 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:00.787458 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:00.879315 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:00.879957 12601 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:00.788805 12602 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:13:00.882403 19971 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:13:00.886695 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.780514 12571 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:00.806155 12526 raft_consensus.cc:2410] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:13:00.806933 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:00.808280 12600 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: [19.11-19.11]
I20260709 12:13:00.809620 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:00.890272 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:00.830740 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:00.833531 19962 raft_consensus.cc:697] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 17 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:00.834460 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:00.765066 19526 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:00.776479 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:00.892570 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:00.895255 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 114881us
I20260709 12:13:00.896308 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:00.898145 12606 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:13:00.900040 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:00.874328 12531 rpcz_store.cc:276] 0709 12:12:59.004659 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.020051 (+ 15392us) service_pool.cc:224] Handling call
0709 12:12:59.042927 (+ 22876us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.677490 (+1634563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.766632 (+ 89142us) spinlock_profiling.cc:243] Waited 1.63 s on lock 0x7b54003cb9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:00.766900 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3749012736}
I20260709 12:13:00.904114 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:00.901737 12529 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2954) took 1874 ms (client timeout 2088 ms). Trace:
I20260709 12:13:00.774758 19618 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:13:00.904776 12529 rpcz_store.cc:269] 0709 12:12:59.027014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.030661 (+  3647us) service_pool.cc:224] Handling call
0709 12:13:00.901569 (+1870908us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54003ca5d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:00.901686 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3407481728}
I20260709 12:13:00.905473 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:00.765048 19491 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.902407 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:00.906663 12606 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:13:00.908413 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:00.909875 12523 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:13:00.910517 13334 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:00.910951 12523 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3af899b7fab042d0ac433430f162d106"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:13:00.911594 12523 raft_consensus.cc:606] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:00.911886 12523 raft_consensus.cc:614] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:13:00.915096 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.907560 19930 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:00.916538 19930 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:00.916590 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:00.916134 19958 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.153s
I20260709 12:13:00.917826 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:00.918776 19958 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:00.917191 19930 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:00.920612 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:00.882929 19971 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:00.921869 12674 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:13:00.927064 12494 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:00.928349 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:00.935214 12494 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:00.946599 12529 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:13:00.952677 12599 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
I20260709 12:13:00.953956 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:00.954674 12603 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:00.919450 19958 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:13:00.956364 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 64121.6us
I20260709 12:13:01.004462 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:00.919215 19850 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:01.005054 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 70060us
I20260709 12:13:00.907898 19875 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:13:00.961506 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:00.784525 19833 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.137s
I20260709 12:13:00.957140 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3399) took 1574 ms. Trace:
I20260709 12:13:00.981396 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:00.922024 19618 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:00.939348 19971 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:00.939242 19693 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.938982 19798 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.943207 19982 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:13:00.945016 19914 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:13:00.993474 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 397017us
I20260709 12:13:00.949623 19697 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.053s
I20260709 12:13:00.955853 19947 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:13:00.955662 19972 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:00.962648 19526 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:01.066960 12523 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:00.960529 19867 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.068794 19867 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:01.015343 12605 rpcz_store.cc:276] 0709 12:12:59.383210 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.270620 (+887410us) service_pool.cc:224] Handling call
0709 12:13:00.271128 (+   508us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:00.272018 (+   890us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:13:00.407808 (+135790us) log.cc:844] Serialized 14 byte log entry
0709 12:13:00.646325 (+238517us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:00.675740 (+ 29415us) log.cc:844] Serialized 26 byte log entry
0709 12:13:00.675888 (+   148us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:13:00.675922 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:00.675988 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:00.956911 (+280923us) raft_consensus.cc:1685] finished
0709 12:13:00.956935 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:00.957080 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47477,"prepare.queue_time_us":63397,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":39,"raft.queue_time_us":52816,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":9260,"thread_start_us":251891,"threads_started":2,"wal-append.queue_time_us":351065}
I20260709 12:13:01.008092 19917 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:01.019393 12531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:13:01.012120 19875 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:01.074676 12531 raft_consensus.cc:606] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:01.075104 12531 raft_consensus.cc:614] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:00.926860 19491 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:01.014765 12673 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
W20260709 12:13:01.027364 12524 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.039s	sys 0.070s
I20260709 12:13:01.034178 12529 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:01.001909 13334 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
W20260709 12:13:01.077755 12524 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.039s	sys 0.070s
W20260709 12:13:01.034344 12643 connection.cc:256] RPC call timeout handler was delayed by -1.3860103619999791s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22040000000000004s for a response
I20260709 12:13:01.062618 12525 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:00.956089 19494 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 15, Time since last communication: 0.054s
I20260709 12:13:01.064651 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:01.082528 12679 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:00.939405 19845 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 16.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.052s
I20260709 12:13:01.020814 19539 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:13:01.076314 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.020676 19618 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:01.018487 19983 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:01.083854 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:01.073908 19984 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
W20260709 12:13:01.099041 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:01.103006 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.106280 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:01.030784 19827 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 12:13:01.117516 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:01.045269 19798 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:01.045619 19693 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:01.135699 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 15 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:01.089325 19636 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:01.071333 19960 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:01.123584 12674 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 20: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:01.079226 19875 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:01.044335 19980 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 12:13:01.072942 19972 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:01.176126 12643 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.645s (SENT) [suppressed 17 similar messages]
W20260709 12:13:01.188566 12602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3404) took 1803 ms (client timeout 1825 ms). Trace:
W20260709 12:13:01.189198 12602 rpcz_store.cc:269] 0709 12:12:59.385218 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.468325 (+ 83107us) service_pool.cc:224] Handling call
0709 12:13:01.188307 (+1719982us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540023f854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:01.188486 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2548118400}
I20260709 12:13:01.136533 12673 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
W20260709 12:13:01.192407 12598 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.013s	sys 0.035s
I20260709 12:13:01.020098 19962 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 15.14, Last appended by leader: 14, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:01.090569 19909 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.056s
W20260709 12:13:01.195262 12598 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.747s	user 0.013s	sys 0.035s
I20260709 12:13:01.198587 12679 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:01.199102 12673 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:13:01.203025 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3396) took 1846 ms. Trace:
I20260709 12:13:01.203682 12598 rpcz_store.cc:276] 0709 12:12:59.357038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.237674 (+880636us) service_pool.cc:224] Handling call
0709 12:13:00.237977 (+   303us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.243416 (+  5439us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:13:00.377948 (+134532us) log.cc:844] Serialized 14 byte log entry
0709 12:13:01.202592 (+824644us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:01.202668 (+    76us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:01.202742 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:01.202814 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:01.202947 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6658,"prepare.queue_time_us":91730,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"thread_start_us":842877,"threads_started":2}
I20260709 12:13:01.206586 12677 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:01.122836 12601 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:01.210727 12531 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:13:01.203428 19494 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:01.248886 19494 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:01.250836 19494 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.9, Last appended by leader: 9, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:01.252948 19494 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:13:01.201332 19624 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.009s	sys 0.040s
I20260709 12:13:01.182935 19636 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:01.217401 19842 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.012s	sys 0.039s
I20260709 12:13:01.212023 19874 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:01.217523 19985 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s	user 0.007s	sys 0.037s
W20260709 12:13:01.275224 13302 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.513s	user 0.005s	sys 0.026s
W20260709 12:13:01.275682 12572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.018s	sys 0.042s
I20260709 12:13:01.278849 12674 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 17 index: 8. Preceding OpId from leader: term: 22 index: 10. (index mismatch)
W20260709 12:13:01.312191 12572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.601s	user 0.018s	sys 0.042s
W20260709 12:13:01.312547 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 641137us
I20260709 12:13:01.144448 19958 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:01.217180 19963 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:01.282408 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.335341 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:01.322082 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:01.325397 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:01.337450 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:01.337713 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:01.338052 12598 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 22: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:01.328743 12602 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17. Ops: [16.15-16.15]
I20260709 12:13:01.338402 12606 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:13:01.331260 12604 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: [17.11-17.11]
I20260709 12:13:01.331274 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:01.336018 12571 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:01.301257 19842 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.012s	sys 0.042s
I20260709 12:13:01.341602 12571 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:01.346647 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:01.348177 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:01.349303 12599 raft_consensus.cc:2410] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
W20260709 12:13:01.311964 19985 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.008s	sys 0.039s
W20260709 12:13:01.281708 19624 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.828s	user 0.010s	sys 0.040s
I20260709 12:13:01.333956 12674 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 16 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:13:01.346724 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:01.331641 13302 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.005s	sys 0.026s
W20260709 12:13:01.359908 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 597986us
I20260709 12:13:01.343708 19975 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.347667 19840 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:01.349088 19845 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:13:01.372615 13304 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.642s	user 0.000s	sys 0.021s
W20260709 12:13:01.373180 13304 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.000s	sys 0.021s
W20260709 12:13:01.373571 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 643418us
W20260709 12:13:01.369187 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 325012us
W20260709 12:13:01.410244 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:01.418933 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.360704 19867 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:01.420763 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:01.365144 19842 raft_consensus.cc:2764] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:13:01.420858 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:01.423458 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:01.421970 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 377321us
I20260709 12:13:01.366019 19624 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:01.373709 19524 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.522s	user 0.009s	sys 0.005s
I20260709 12:13:01.343043 19963 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:01.429353 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3017) took 1208 ms. Trace:
I20260709 12:13:01.444617 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:01.389778 19491 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:01.390196 19845 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:01.444597 12680 rpcz_store.cc:276] 0709 12:13:00.220387 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.666427 (+446040us) service_pool.cc:224] Handling call
0709 12:13:00.666857 (+   430us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:13:00.681073 (+ 14216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:00.681112 (+    39us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:13:00.681856 (+   744us) log.cc:844] Serialized 78 byte log entry
0709 12:13:00.980464 (+298608us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:01.374734 (+394270us) log.cc:844] Serialized 14 byte log entry
0709 12:13:01.422855 (+ 48121us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:01.422953 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:01.422997 (+    44us) raft_consensus.cc:1685] finished
0709 12:13:01.423008 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:01.428660 (+  5652us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16663,"prepare.queue_time_us":90589,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":16021,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":2819,"thread_start_us":614440,"threads_started":2,"wal-append.queue_time_us":469746}
I20260709 12:13:01.440551 12605 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:13:01.462507 12671 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
W20260709 12:13:01.472172 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:01.444191 19964 consensus_meta.cc:287] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.923s	user 0.014s	sys 0.000s
W20260709 12:13:01.519110 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:01.532276 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:01.487736 19927 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.742s	user 0.013s	sys 0.000s
I20260709 12:13:01.437546 19917 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:01.430411 20033 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:01.373422 19975 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:01.577826 20022 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.160s	user 0.000s	sys 0.000s
I20260709 12:13:01.578332 19851 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:01.550004 19693 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:01.579324 20034 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 12:13:01.639242 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3074) took 1099 ms. Trace:
I20260709 12:13:01.639839 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3502) took 1443 ms. Trace:
W20260709 12:13:01.608012 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 431892us
I20260709 12:13:01.572733 20028 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.036s
I20260709 12:13:01.582934 20033 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:01.573508 19917 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:01.504577 19845 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:13:01.584260 19830 consensus_meta.cc:287] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.732s	user 0.011s	sys 0.004s
I20260709 12:13:01.488036 19824 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:01.676429 20035 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.400s	user 0.001s	sys 0.000s
W20260709 12:13:01.548462 19927 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.835s	user 0.014s	sys 0.000s
W20260709 12:13:01.527256 19949 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.688s	user 0.015s	sys 0.000s
I20260709 12:13:01.527451 19964 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:01.595667 19798 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:01.596902 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:01.604895 20036 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.600260 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:01.696324 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:01.603190 12496 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:13:01.718943 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 115851us
W20260709 12:13:01.604081 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 140617us
W20260709 12:13:01.547246 20023 log.cc:927] Time spent T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 12:13:01.461714 19624 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:01.631628 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.635654 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:01.640412 12677 rpcz_store.cc:276] 0709 12:13:00.196431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.198050 (+  1619us) service_pool.cc:224] Handling call
0709 12:13:00.198421 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:01.207050 (+1008629us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:01.639493 (+432443us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54000f00d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:01.639757 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2317940608}
W20260709 12:13:01.647177 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.512100 19842 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.097s
I20260709 12:13:01.639711 12523 rpcz_store.cc:276] 0709 12:13:00.539433 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.944748 (+405315us) service_pool.cc:224] Handling call
0709 12:13:00.945081 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:01.067598 (+122517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:01.638956 (+571358us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b540042a9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:01.639178 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":279635840}
I20260709 12:13:01.676707 19960 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:01.678699 12606 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
W20260709 12:13:01.627173 19970 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.902s	user 0.014s	sys 0.000s
I20260709 12:13:01.711861 13332 raft_consensus.cc:2410] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 18 because replica has last-logged OpId of term: 15 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 13 index: 9.
I20260709 12:13:01.745764 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:01.598970 13335 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:01.758155 12677 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:01.759255 12676 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 18: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:01.760898 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:01.761758 12678 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:01.769884 12680 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:01.774008 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:01.775311 12680 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
W20260709 12:13:01.729488 20022 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
W20260709 12:13:01.724632 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:01.727942 20027 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
I20260709 12:13:01.779227 20028 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:13:01.685708 19985 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.082s
I20260709 12:13:01.638615 19851 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:01.710227 19947 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:01.709911 19983 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:13:01.792639 12597 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.007s	sys 0.032s
I20260709 12:13:01.683478 19850 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.072s
I20260709 12:13:01.740952 19636 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:01.684405 19824 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:01.769827 19840 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:13:01.741169 19830 consensus_queue.cc:1059] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.064s
W20260709 12:13:01.607558 19697 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.015s	sys 0.047s
W20260709 12:13:01.842941 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 401999us
I20260709 12:13:01.733543 20036 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:01.817256 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:01.723655 12494 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:01.811365 19798 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:01.753484 19827 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:01.756613 19949 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:01.828095 12597 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.008s	sys 0.032s
I20260709 12:13:01.732225 19491 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.069s
I20260709 12:13:01.837278 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:01.780375 19964 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:01.823508 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:01.843250 19697 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.756s	user 0.016s	sys 0.048s
W20260709 12:13:01.907045 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.914319 13335 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:13:01.911289 12676 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:13:01.769493 19970 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:01.958107 19798 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:01.983778 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:01.854908 19693 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:01.908506 20045 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.180s	user 0.002s	sys 0.000s
W20260709 12:13:02.002737 12495 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.515s	user 0.002s	sys 0.024s
I20260709 12:13:01.852049 19840 raft_consensus.cc:697] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:02.028398 12495 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.540s	user 0.003s	sys 0.024s
W20260709 12:13:01.971666 19990 consensus_meta.cc:287] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.102s	user 0.009s	sys 0.009s
I20260709 12:13:01.852349 20041 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:01.843403 19947 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:02.063133 12675 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:02.064466 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2955) took 3027 ms. Trace:
W20260709 12:13:02.026719 19791 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.951s	user 0.016s	sys 0.012s
I20260709 12:13:01.836313 19983 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:01.994537 19964 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.085817 12524 rpcz_store.cc:276] 0709 12:12:59.027283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:59.048822 (+ 21539us) service_pool.cc:224] Handling call
0709 12:12:59.082547 (+ 33725us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:12:59.083019 (+   472us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:13:00.234619 (+1151600us) spinlock_profiling.cc:243] Waited 6.41 ms on lock 0x7b18004a91a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf 00007f1488a3a038
0709 12:13:00.339813 (+105194us) log.cc:844] Serialized 14 byte log entry
0709 12:13:01.087690 (+747877us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:01.088206 (+   516us) log.cc:844] Serialized 26 byte log entry
0709 12:13:01.088397 (+   191us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:01.088444 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:01.088552 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:01.991502 (+902950us) raft_consensus.cc:1685] finished
0709 12:13:02.054840 (+ 63338us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:02.055055 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50526,"prepare.queue_time_us":1155309,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":56017,"raft.run_cpu_time_us":381,"raft.run_wall_time_us":15964,"spinlock_wait_cycles":14731648,"thread_start_us":736023,"threads_started":2,"wal-append.queue_time_us":965604}
W20260709 12:13:02.056954 19874 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.010s	sys 0.009s
W20260709 12:13:02.097262 13304 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.607s	user 0.016s	sys 0.026s
W20260709 12:13:02.098587 13304 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.016s	sys 0.026s
W20260709 12:13:02.064832 20030 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.560s	user 0.001s	sys 0.001s
I20260709 12:13:01.972277 19949 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:02.056653 19697 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:01.969574 19827 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:02.064765 19896 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:02.085302 19842 raft_consensus.cc:3055] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:13:02.089483 19791 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.024s	user 0.017s	sys 0.012s
W20260709 12:13:02.111428 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 623808us
I20260709 12:13:02.094851 20042 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.139096 19791 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:02.139931 19791 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:02.108896 19850 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.142627 19850 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:02.145599 19850 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:02.103502 12678 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
W20260709 12:13:02.097708 19874 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.010s	sys 0.009s
I20260709 12:13:02.147706 20039 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:13:02.115980 19896 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:02.149853 19824 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:02.150312 19840 consensus_queue.cc:237] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 14.9, Last appended by leader: 9, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:01.851763 20033 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:02.151877 20043 log.cc:927] Time spent T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.306s	user 0.000s	sys 0.000s
I20260709 12:13:02.153003 20036 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:02.155129 19947 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:02.156580 19970 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:02.157543 19624 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:02.159644 19949 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:02.161728 19983 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:02.162514 20051 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:02.164124 19827 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:02.126374 19842 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:02.109807 19963 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:02.170665 19971 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.098s	user 0.014s	sys 0.000s
I20260709 12:13:02.111507 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:02.182302 19972 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:02.183076 19851 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:02.103574 19975 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:02.213121 19894 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:13:02.091688 20041 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:02.062615 19693 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:02.275856 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2960) took 1606 ms. Trace:
I20260709 12:13:02.193316 19853 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.116973 19697 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:02.113229 19977 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:02.218952 13305 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.581s	user 0.024s	sys 0.033s
I20260709 12:13:02.152807 20044 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.061s
W20260709 12:13:02.209117 19967 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.028s	sys 0.045s
W20260709 12:13:02.212034 19618 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.085s	user 0.016s	sys 0.000s
W20260709 12:13:02.218217 19930 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.268s	user 0.009s	sys 0.007s
I20260709 12:13:02.290668 19914 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:02.223883 19833 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.037s	sys 0.039s
I20260709 12:13:02.104363 19964 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:02.246011 20051 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:02.274559 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:02.201709 20042 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:02.276477 12528 rpcz_store.cc:276] 0709 12:13:00.669685 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.059264 (+389579us) service_pool.cc:224] Handling call
0709 12:13:01.059773 (+   509us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:01.087660 (+ 27887us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:01.087690 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:01.090674 (+  2984us) log.cc:844] Serialized 26 byte log entry
0709 12:13:01.437340 (+346666us) raft_consensus.cc:1645] Marking committed up to 8
0709 12:13:01.437397 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:01.632283 (+194886us) spinlock_profiling.cc:243] Waited 14.6 ms on lock 0x7b54003cc8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:01.632418 (+   135us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:02.266488 (+634070us) raft_consensus.cc:1685] finished
0709 12:13:02.266513 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:02.275740 (+  9227us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12594,"raft.queue_time_us":52271,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":7981,"spinlock_wait_cycles":33653376,"thread_start_us":346305,"threads_started":1,"wal-append.queue_time_us":702216}
I20260709 12:13:02.339447 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:02.234889 19947 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:02.292805 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:02.341792 12523 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 14.
W20260709 12:13:02.302371 13305 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.664s	user 0.024s	sys 0.033s
W20260709 12:13:02.343197 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 705103us
W20260709 12:13:02.344240 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:02.344034 13305 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:02.304910 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:02.270659 19894 consensus_queue.cc:1243] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 21.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:13:02.311120 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:02.311008 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:02.275403 20039 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:02.290233 19491 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:02.321038 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:02.302065 19853 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:02.310940 19967 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.800s	user 0.029s	sys 0.045s
W20260709 12:13:02.268143 19962 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.012s	sys 0.027s
I20260709 12:13:02.249032 19971 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:02.237641 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:02.303788 19791 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:02.362287 12571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.007s	sys 0.004s
W20260709 12:13:02.332463 19833 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.037s	sys 0.039s
I20260709 12:13:02.307657 12531 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:13:02.347476 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:02.322623 19875 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.121s	user 0.016s	sys 0.001s
I20260709 12:13:02.349860 12530 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:13:02.349716 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:02.415704 12680 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 21: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:02.416018 12528 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:13:02.249897 19896 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:02.353698 12646 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:02.301867 19972 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:02.356328 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 632722us
I20260709 12:13:02.422082 12494 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:02.356868 12524 raft_consensus.cc:2410] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 13.
W20260709 12:13:02.336762 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:02.363307 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 125801us
I20260709 12:13:02.428706 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:13:02.365271 12571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.523s	user 0.008s	sys 0.005s
I20260709 12:13:02.316694 19930 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:02.347119 19798 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:02.358462 19962 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.763s	user 0.012s	sys 0.028s
I20260709 12:13:02.413918 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3407) took 1664 ms. Trace:
I20260709 12:13:02.316532 19618 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:02.391382 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:02.334488 19914 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:02.311401 19842 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:02.365001 19494 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.057s	user 0.016s	sys 0.000s
W20260709 12:13:02.485610 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 357973us
W20260709 12:13:02.419718 12527 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.515s	user 0.006s	sys 0.010s
I20260709 12:13:02.487629 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 874) took 2222 ms. Trace:
I20260709 12:13:02.357640 19491 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:02.430379 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:02.350865 19947 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:02.435070 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:02.595913 12530 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:02.588490 13304 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:02.424362 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:02.427482 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:02.369318 19827 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:13:02.361922 20039 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:02.437769 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:02.420053 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 66452.9us
I20260709 12:13:02.689138 12646 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:02.691036 12605 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:13:02.416541 20063 raft_consensus.cc:3055] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:13:02.490336 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 60058.6us
I20260709 12:13:02.597574 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:02.440303 19830 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.024s	sys 0.044s
I20260709 12:13:02.603976 12604 raft_consensus.cc:2410] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 20 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:13:02.488010 12597 rpcz_store.cc:276] 0709 12:13:00.749872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.750334 (+   462us) service_pool.cc:224] Handling call
0709 12:13:00.784673 (+ 34339us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:01.193276 (+408603us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:01.193343 (+    67us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:01.207588 (+ 14245us) log.cc:844] Serialized 52 byte log entry
0709 12:13:01.908100 (+700512us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:02.363328 (+455228us) log.cc:844] Serialized 14 byte log entry
0709 12:13:02.366306 (+  2978us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:02.366596 (+   290us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:02.366788 (+   192us) raft_consensus.cc:1685] finished
0709 12:13:02.366812 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:02.413812 (+ 47000us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9060,"raft.queue_time_us":47866,"raft.run_cpu_time_us":368,"raft.run_wall_time_us":93339,"thread_start_us":1393251,"threads_started":3,"wal-append.queue_time_us":754003}
I20260709 12:13:02.720649 12523 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:13:02.491034 19990 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:02.436928 12572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.593s	user 0.025s	sys 0.030s
W20260709 12:13:02.723062 12572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.879s	user 0.025s	sys 0.030s
W20260709 12:13:02.723546 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 879536us
I20260709 12:13:02.724571 12572 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:02.641585 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:02.642197 20052 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.329s	user 0.000s	sys 0.001s
W20260709 12:13:02.643566 20067 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.288s	user 0.000s	sys 0.001s
I20260709 12:13:02.643464 20051 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:02.644613 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 220318us
I20260709 12:13:02.755973 12605 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
W20260709 12:13:02.645843 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 218427us
I20260709 12:13:02.597949 19982 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.049s
I20260709 12:13:02.647763 20070 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:02.602800 19985 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:02.656118 19853 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:02.676621 20041 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:02.831086 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 233596us
I20260709 12:13:02.437547 19824 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.677292 12676 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:02.678215 19693 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:02.679911 20064 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:13:02.716988 19494 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:02.683947 13336 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 17 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 16. Current term is 17. Ops: [16.15-16.15]
I20260709 12:13:02.488059 12527 rpcz_store.cc:276] 0709 12:13:00.265466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.939564 (+674098us) service_pool.cc:224] Handling call
0709 12:13:00.946429 (+  6865us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:00.946714 (+   285us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:13:01.332522 (+385808us) log.cc:844] Serialized 14 byte log entry
0709 12:13:01.847123 (+514601us) log.cc:844] Serialized 14 byte log entry
0709 12:13:02.487109 (+639986us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:02.487200 (+    91us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:02.487279 (+    79us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:02.487388 (+   109us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:02.487553 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":121914,"prepare.queue_time_us":46392,"prepare.run_cpu_time_us":9505,"prepare.run_wall_time_us":100928,"raft.queue_time_us":615590,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":26445,"thread_start_us":1399972,"threads_started":3,"wal-append.queue_time_us":770807}
W20260709 12:13:02.693864 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:02.482971 20057 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.755973 19963 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:02.486413 19798 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:13:02.437636 19983 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:02.851567 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:02.437862 19970 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Leader pre-election won for term 15
I20260709 12:13:02.420289 19875 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:02.482072 19524 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:02.916596 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:02.725883 12530 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
W20260709 12:13:02.598210 20050 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.359s	user 0.002s	sys 0.000s
I20260709 12:13:02.684067 19842 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:02.684576 19833 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:13:02.640439 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 294655us
W20260709 12:13:02.824777 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:02.825165 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 100765us
W20260709 12:13:02.721289 19830 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.814s	user 0.024s	sys 0.044s
I20260709 12:13:02.435281 19972 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:02.719909 20076 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:02.482295 19949 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.303s
I20260709 12:13:02.714421 20063 raft_consensus.cc:740] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:02.711656 19967 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:02.841042 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:02.917289 20052 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
I20260709 12:13:02.438292 19896 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:02.856331 19982 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:02.823815 20070 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:13:02.872900 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:02.834376 20071 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.162s
I20260709 12:13:02.912781 19964 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:02.957006 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:03.027695 12530 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 18: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:13:02.915273 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:02.836508 19850 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.827335 19851 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.827068 19985 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:02.828840 19624 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:02.843102 19894 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:02.803242 19990 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:02.847982 20053 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:02.990880 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3513) took 1245 ms. Trace:
I20260709 12:13:02.996421 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:03.050333 12678 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
I20260709 12:13:03.051481 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:03.049523 12674 rpcz_store.cc:276] 0709 12:13:01.744986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.745137 (+   151us) service_pool.cc:224] Handling call
0709 12:13:01.745548 (+   411us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:01.746270 (+   722us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:01.746300 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:01.746994 (+   694us) log.cc:844] Serialized 26 byte log entry
0709 12:13:02.269092 (+522098us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:02.269148 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:02.269246 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:02.990520 (+721274us) raft_consensus.cc:1685] finished
0709 12:13:02.990551 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:02.990786 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4709,"raft.run_cpu_time_us":176,"raft.run_wall_time_us":20586,"thread_start_us":437940,"threads_started":1,"wal-append.queue_time_us":516199}
I20260709 12:13:03.052392 12678 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 22: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:02.846666 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:02.957556 19970 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:03.006539 20042 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:02.849134 19963 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:03.009963 12572 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:03.036594 19964 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:03.010540 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3034) took 1735 ms. Trace:
I20260709 12:13:02.912935 19491 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:02.804693 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:03.049939 20061 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.456s	user 0.000s	sys 0.001s
W20260709 12:13:03.025205 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 266145us
I20260709 12:13:02.849831 19824 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:03.042227 19851 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:03.072435 19824 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:02.991665 19697 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:02.851447 20057 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:03.075325 19964 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:03.076304 20057 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:03.076934 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.040058 19850 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:03.077796 12523 raft_consensus.cc:2393] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 21: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:03.061357 19624 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:03.038548 20075 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.046s
I20260709 12:13:03.078796 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.079319 19850 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:03.080022 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.009344 20066 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.080812 20066 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:03.082437 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3510) took 1338 ms. Trace:
W20260709 12:13:03.083076 19917 consensus_meta.cc:287] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.393s	user 0.010s	sys 0.005s
I20260709 12:13:03.083113 20066 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:03.029384 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.083855 19624 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.058202 20053 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:03.084578 19624 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:03.039747 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.085775 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:03.086376 20053 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:03.086792 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3083) took 1695 ms. Trace:
I20260709 12:13:03.087054 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:03.087402 12606 rpcz_store.cc:276] 0709 12:13:01.390987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.419515 (+ 28528us) service_pool.cc:224] Handling call
0709 12:13:01.419860 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:01.743237 (+323377us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:03.086545 (+1343308us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b54002f1a54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:03.086732 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":594210176}
I20260709 12:13:03.088462 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:03.088573 19990 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:03.089375 20039 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:02.857738 20078 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.180s
I20260709 12:13:03.090272 19851 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:03.090902 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3035) took 1814 ms. Trace:
I20260709 12:13:03.091037 19964 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:02.849547 19494 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:03.092681 19624 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:03.091342 12679 rpcz_store.cc:276] 0709 12:13:01.276855 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.277056 (+   201us) service_pool.cc:224] Handling call
0709 12:13:01.311667 (+ 34611us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:01.312192 (+   525us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:13:01.359030 (+ 46838us) log.cc:844] Serialized 14 byte log entry
0709 12:13:01.847737 (+488707us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:01.920754 (+ 73017us) log.cc:844] Serialized 26 byte log entry
0709 12:13:01.920963 (+   209us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:13:01.965448 (+ 44485us) log.cc:844] Serialized 14 byte log entry
0709 12:13:01.984980 (+ 19532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:01.985099 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:03.090682 (+1105583us) raft_consensus.cc:1685] finished
0709 12:13:03.090704 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:03.090837 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18846,"prepare.queue_time_us":67683,"prepare.run_cpu_time_us":6128,"prepare.run_wall_time_us":53554,"raft.queue_time_us":32214,"raft.run_cpu_time_us":492,"raft.run_wall_time_us":75138,"thread_start_us":428088,"threads_started":1,"wal-append.queue_time_us":876551}
I20260709 12:13:02.965242 19524 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:03.093503 19494 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:03.024742 20070 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:03.019549 19982 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:03.066721 12494 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:03.022449 19949 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:03.023329 19833 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:03.053038 12527 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:13:03.053184 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:03.033574 19971 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:03.049669 20041 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.053s
I20260709 12:13:03.038452 19985 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:13:03.059055 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 49257.3us
W20260709 12:13:03.019837 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 133527us
I20260709 12:13:03.015254 19967 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:03.023837 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 263952us
I20260709 12:13:03.061405 12673 rpcz_store.cc:276] 0709 12:13:01.275061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.275216 (+   155us) service_pool.cc:224] Handling call
0709 12:13:01.275584 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:01.275838 (+   254us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:01.275870 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:01.282165 (+  6295us) log.cc:844] Serialized 26 byte log entry
0709 12:13:01.844966 (+562801us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:13:01.845016 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:01.845109 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:03.010044 (+1164935us) raft_consensus.cc:1685] finished
0709 12:13:03.010085 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:03.010399 (+   314us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":491962,"threads_started":1,"wal-append.queue_time_us":889180}
I20260709 12:13:03.059821 19963 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:03.057664 19970 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:13:03.062873 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:03.080155 19947 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:03.089567 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.084589 12530 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 21: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:03.095424 12523 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:03.139490 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:03.134274 12496 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:03.140170 12672 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:03.113410 19833 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:03.082894 12677 rpcz_store.cc:276] 0709 12:13:01.743687 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.749179 (+  5492us) service_pool.cc:224] Handling call
0709 12:13:01.749555 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:01.758772 (+  9217us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:03.082154 (+1323382us) spinlock_profiling.cc:243] Waited 8.28 ms on lock 0x7b54000c3fd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:03.082372 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":19052800}
I20260709 12:13:02.958798 19798 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:03.100145 12679 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:13:03.095176 12526 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:13:03.077905 19762 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:03.057049 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:03.113183 12674 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 22: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:03.113963 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.171058 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:03.111479 12494 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:03.172130 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 60740.2us
I20260709 12:13:03.173420 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:03.083911 20028 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.175877 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.176470 20028 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:03.119786 12572 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:03.179148 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 59458.5us
I20260709 12:13:03.179622 19967 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:03.180387 19967 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:03.121399 19875 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:03.091593 19723 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:03.182946 19524 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:03.112938 20079 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:03.183857 20079 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:03.184243 19723 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:03.186098 19798 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:03.123327 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.123621 19972 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:03.187841 12571 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:03.083756 19917 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:03.110602 19845 consensus_meta.cc:287] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.401s	user 0.010s	sys 0.007s
I20260709 12:13:03.190897 19845 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:03.192454 19917 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:03.131170 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:03.193027 19845 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:03.193627 12673 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 20: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:03.150895 19942 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:03.194402 12599 raft_consensus.cc:2368] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20.
I20260709 12:13:03.169952 19982 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:03.195495 12599 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3085) took 1775 ms (client timeout 1780 ms). Trace:
I20260709 12:13:03.054906 20063 consensus_queue.cc:260] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:03.195873 12599 rpcz_store.cc:269] 0709 12:13:01.420160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.420305 (+   145us) service_pool.cc:224] Handling call
0709 12:13:03.195290 (+1774985us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b5400230854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:03.195436 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4069088128}
I20260709 12:13:03.196522 20063 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:03.140388 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.197376 12523 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:13:03.198698 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.199476 12523 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:03.169775 19762 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:03.200698 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.201901 19762 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:03.097095 12676 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:03.149878 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:03.177384 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.108665 20078 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:03.155457 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.144698 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:03.113109 19949 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:03.061015 19930 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:03.184661 20079 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:03.133149 19697 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.193738 19845 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:13:03.207247 12529 raft_consensus.cc:2410] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 20 because replica has last-logged OpId of term: 18 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 9.
W20260709 12:13:03.240739 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.207815 12677 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:13:03.241797 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:03.208159 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.243117 12530 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:13:03.208771 12572 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:03.209107 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:03.244503 12527 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:03.175956 12674 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23. Ops: [21.11-22.12]
I20260709 12:13:03.211485 12494 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:13:03.218503 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 193923us
I20260709 12:13:03.140969 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:03.238771 12526 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:13:03.248644 12526 raft_consensus.cc:606] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:03.202077 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:03.251922 12643 leader_election.cc:336] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.820s (SENT)
W20260709 12:13:03.252485 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.252430 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:03.256131 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:03.256333 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 47705.4us
I20260709 12:13:03.257627 12530 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 20 index: 9. Preceding OpId from leader: term: 22 index: 10. (index mismatch)
I20260709 12:13:03.257496 12572 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:03.258461 12523 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 18 index: 8. (index mismatch)
I20260709 12:13:03.262455 12672 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
W20260709 12:13:03.258520 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 117628us
I20260709 12:13:03.250530 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:03.259850 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 48462.8us
I20260709 12:13:03.267606 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:03.268518 12598 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 20: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:03.269001 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:03.269644 12606 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:13:03.270758 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:03.273162 12672 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
I20260709 12:13:03.274230 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:03.274415 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:03.275313 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:03.275456 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:03.276082 12601 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:03.276194 12672 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:13:03.276527 12523 pending_rounds.cc:85] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 13
I20260709 12:13:03.277251 12523 pending_rounds.cc:107] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 12:13:03.277658 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:03.279036 12523 raft_consensus.cc:2889] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:03.254830 19826 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:13:03.195896 19982 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:03.282303 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:03.282517 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:03.283138 12643 leader_election.cc:336] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.190s (SENT)
I20260709 12:13:03.284360 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:03.296654 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:03.203373 19762 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:03.287819 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:03.292086 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:03.194445 19942 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:03.130422 20057 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.119s
I20260709 12:13:03.239857 20066 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:03.240476 19845 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:03.335085 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:03.304917 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:03.355294 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:03.361006 12600 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 20: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:03.281561 19524 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:03.289443 19624 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:13:03.375056 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.290030 19610 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:03.281724 19930 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:13:03.437247 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 82073.4us
W20260709 12:13:03.437500 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:03.443766 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 146573us
I20260709 12:13:03.444922 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:03.445863 12601 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:13:03.447197 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:03.299207 19929 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:03.299649 13304 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:03.290477 19840 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:13:03.452374 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.453429 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:03.448100 12599 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
W20260709 12:13:03.438611 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 151149us
I20260709 12:13:03.457651 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:03.335714 19972 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:03.289954 19917 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:03.451382 19840 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:03.470741 19840 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:03.360014 20066 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:03.473016 20066 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:03.443259 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:03.478787 12600 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:03.480075 19875 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:13:03.243965 20078 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:03.482309 12674 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:03.483049 12572 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:03.483994 20078 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:03.355772 19845 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:03.451756 19967 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:13:03.253364 19697 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:03.490145 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3509) took 3173 ms. Trace:
I20260709 12:13:03.490549 20057 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:13:03.491712 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48530.1us
W20260709 12:13:03.492063 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.300025 19942 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:03.442456 19610 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:03.493635 19625 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:03.494092 19625 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:03.494035 19610 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:03.465898 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.355954 20028 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:03.467999 20076 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:03.506369 20076 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:03.506855 20076 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:03.494799 19625 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:03.490614 12675 rpcz_store.cc:276] 0709 12:13:00.316725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.752199 (+435474us) service_pool.cc:224] Handling call
0709 12:13:00.827673 (+ 75474us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:02.084984 (+1257311us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:03.489845 (+1404861us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540036cc54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:03.490057 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2830542592}
I20260709 12:13:03.451997 20068 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:03.514547 20068 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:03.514801 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:03.515882 12675 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 22: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:03.516889 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:03.517510 12675 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:03.450822 19983 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.518636 19983 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:03.500620 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.498461 19967 raft_consensus.cc:697] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:03.523537 19949 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:03.505947 19840 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:03.515067 20068 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:13:03.449443 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 149859us
I20260709 12:13:03.470113 19917 raft_consensus.cc:697] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:03.499073 19972 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:03.526374 19917 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:03.523154 19967 consensus_queue.cc:237] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:03.531150 12527 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:03.532466 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:03.539893 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:03.541425 20028 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:03.541918 20028 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:03.542321 20028 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:03.497280 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:13:03.544979 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 47811.3us
I20260709 12:13:03.546568 12572 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:03.534022 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:03.491433 19697 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:13:03.540573 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.540377 19949 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.585670 12527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:13:03.590986 12527 raft_consensus.cc:606] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:03.591538 12527 raft_consensus.cc:614] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:13:03.616984 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.555115 19882 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:13:03.649799 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 61176us
W20260709 12:13:03.650772 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.584106 19842 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:03.677929 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.676075 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.608196 19624 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.692622 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.580165 19833 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:03.585176 20070 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:03.657510 19882 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:03.586936 19697 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
I20260709 12:13:03.706580 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.706552 12527 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
W20260709 12:13:03.699154 20063 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.502s	user 0.014s	sys 0.000s
I20260709 12:13:03.608997 19875 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.089s
I20260709 12:13:03.728576 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:13:03.716009 12525 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:03.654609 19949 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:03.585652 19985 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:03.672467 19971 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 12:13:03.715083 20058 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:13:03.720824 19970 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.573s	user 0.011s	sys 0.002s
W20260709 12:13:03.740403 12678 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s	user 0.009s	sys 0.012s
I20260709 12:13:03.716102 19882 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:03.697176 19624 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:03.748246 12678 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.009s	sys 0.013s
I20260709 12:13:03.751876 12672 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 21: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:13:03.740921 19977 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.826s	user 0.003s	sys 0.016s
I20260709 12:13:03.756551 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.757316 12525 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
W20260709 12:13:03.750665 19963 consensus_meta.cc:287] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.603s	user 0.011s	sys 0.001s
I20260709 12:13:03.754673 19539 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:03.775938 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.791174 12597 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:13:03.791337 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:03.798874 13336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38482 (request call id 716) took 2401 ms. Trace:
I20260709 12:13:03.799551 13336 rpcz_store.cc:276] 0709 12:13:01.397317 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.442787 (+ 45470us) service_pool.cc:224] Handling call
0709 12:13:01.473001 (+ 30214us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:02.841464 (+1368463us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:03.798414 (+956950us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400690354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:03.798745 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2500817152}
W20260709 12:13:03.803547 12604 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3084) took 2383 ms (client timeout 1529 ms). Trace:
W20260709 12:13:03.804064 12604 rpcz_store.cc:269] 0709 12:13:01.419795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.420543 (+   748us) service_pool.cc:224] Handling call
0709 12:13:03.803227 (+2382684us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b5400116cd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:03.803429 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2697672704}
W20260709 12:13:03.767222 19977 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.853s	user 0.004s	sys 0.016s
I20260709 12:13:03.763522 19983 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:03.811921 19723 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.627s	user 0.013s	sys 0.000s
I20260709 12:13:03.808970 19840 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:03.847020 20070 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:03.847982 20070 raft_consensus.cc:2764] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 20: won
I20260709 12:13:03.848040 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:03.781680 19625 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:03.778262 19963 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:03.849507 12604 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 20: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:03.851382 19625 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:03.819897 19985 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:03.853188 12675 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:03.853639 20031 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:03.820920 20097 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
I20260709 12:13:03.822942 20087 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:03.823307 12605 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 12. (term mismatch)
I20260709 12:13:03.861088 12605 pending_rounds.cc:85] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 11
I20260709 12:13:03.861363 12605 pending_rounds.cc:107] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 12:13:03.861677 12605 raft_consensus.cc:2889] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:13:03.860167 12606 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.520s	user 0.019s	sys 0.018s
I20260709 12:13:03.824169 19624 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:03.774976 19972 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:03.831463 19833 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:03.748588 19942 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:03.750514 19970 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:03.837291 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:03.838045 20068 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:03.839486 19762 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:03.840982 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.879956 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:03.881314 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:03.882210 12603 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:13:03.889010 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:03.860689 20031 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:03.892025 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:03.898198 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:03.963802 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:03.964874 13332 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:13:03.965363 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:03.967060 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 175828us
I20260709 12:13:03.781028 19539 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:03.856024 19949 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:03.865285 12606 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.019s	sys 0.018s
I20260709 12:13:03.875064 19624 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:03.987125 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:03.873885 19901 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:03.949036 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:03.945590 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:03.994692 12603 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:03.994956 12599 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 23: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:03.996112 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:03.996881 12599 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:13:03.872104 20058 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:03.955060 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:03.998703 12600 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:13:03.968252 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:03.866864 19845 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:03.866786 20087 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:03.999935 20100 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:04.000862 12598 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 15. Current term is 16. Ops: []
I20260709 12:13:03.880393 19779 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.096s
I20260709 12:13:04.001473 19845 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:03.865449 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 420732us
I20260709 12:13:04.001459 20087 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:04.004074 20070 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:13:03.997606 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.006110 12494 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:04.007663 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:04.008260 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:04.008461 12602 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:13:03.999190 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:04.010418 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:04.010421 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:04.011821 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.012013 12677 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:13:04.013131 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:04.013893 12677 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:03.997792 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:04.009166 12601 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:13:04.015355 13332 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: [19.15-20.16]
I20260709 12:13:03.967165 19963 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:03.945155 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:03.874840 20076 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:04.035514 12602 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 21: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:04.016628 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:03.886902 19762 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:04.039645 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:04.040546 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:04.041489 13333 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:13:04.022971 12673 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 20: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:04.029516 12494 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:04.038305 12601 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:13:04.051141 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3042) took 1224 ms. Trace:
I20260709 12:13:04.052287 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:04.052170 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:04.055625 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:04.058143 13335 raft_consensus.cc:2368] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21.
W20260709 12:13:04.017518 20057 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.526s	user 0.008s	sys 0.005s
I20260709 12:13:04.062561 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:04.064424 12677 raft_consensus.cc:2393] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 15: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:04.077525 12675 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:04.016296 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:04.078341 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:04.078596 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:04.079433 12677 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:04.065430 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:04.039113 20100 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:04.087348 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:04.088567 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:04.088970 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:04.080284 12673 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:04.091039 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:04.091907 12673 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:04.081180 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:04.081410 12571 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:04.095749 12643 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:04.040222 19917 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:04.111325 12599 raft_consensus.cc:2410] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 18 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:13:04.003721 20098 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.071s
I20260709 12:13:04.131027 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:04.141172 12599 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:13:04.142422 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:04.143162 12599 raft_consensus.cc:2376] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23. Re-sending same reply.
I20260709 12:13:04.144336 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:04.145337 12599 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:04.132501 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:04.055048 20081 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:03.998188 20058 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:04.136691 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 55357.2us
W20260709 12:13:04.153862 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 93802.8us
I20260709 12:13:04.152014 19697 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:13:04.153110 12571 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:04.042897 20070 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.093s
I20260709 12:13:03.990301 19624 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:04.165877 13334 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 17 index: 13. Preceding OpId from leader: term: 19 index: 13. (term mismatch)
I20260709 12:13:04.180382 13334 pending_rounds.cc:85] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 12
I20260709 12:13:04.180743 13334 pending_rounds.cc:107] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 17.13
I20260709 12:13:04.181061 13334 raft_consensus.cc:2889] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:04.169368 12496 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:04.157713 19697 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:04.211043 19697 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:13:04.215272 20102 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 12:13:04.216730 19963 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.217512 19963 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:04.085531 20100 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:04.229957 20100 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:04.247474 19875 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:04.248172 19875 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:04.248728 19875 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:13:04.254494 20079 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.972s	user 0.015s	sys 0.000s
I20260709 12:13:04.255268 20079 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:04.261519 20058 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:04.183863 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.184631 12599 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:04.300021 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:04.300916 12599 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:13:04.132298 19972 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:04.210422 20076 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:04.210093 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:04.167203 20081 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:04.226452 20094 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.146s	user 0.001s	sys 0.000s
W20260709 12:13:04.226888 20095 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.189s	user 0.000s	sys 0.000s
W20260709 12:13:04.227788 20089 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.240s	user 0.000s	sys 0.000s
I20260709 12:13:04.130234 20057 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:04.265583 19762 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:04.266693 19942 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:04.014995 19901 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:04.268057 20064 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:04.275329 19970 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:04.057691 19539 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:04.155197 19949 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:04.329924 19967 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.474550 19967 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:04.477162 19967 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:04.389576 19983 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
W20260709 12:13:04.479216 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:04.289359 20028 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.747s	user 0.014s	sys 0.000s
I20260709 12:13:04.301962 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:04.329375 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 233673us
I20260709 12:13:04.481083 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:04.481213 12643 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:13:04.317607 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 143850us
I20260709 12:13:04.482327 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:04.482940 12599 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:04.189248 12569 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:04.402518 12673 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:13:04.398329 12677 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:13:04.484443 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:04.485489 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:04.485774 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:04.409864 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.318506 19963 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:04.450261 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:04.462359 12603 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:13:04.448128 19949 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:04.479116 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3040) took 1735 ms. Trace:
I20260709 12:13:04.445107 12602 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:04.138156 19917 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:04.489624 12673 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 17: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:04.490090 12677 raft_consensus.cc:2410] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:13:04.500296 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:04.492015 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:04.488763 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 299635us
I20260709 12:13:04.501624 12603 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 21: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:04.504767 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:04.505505 12673 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 21
W20260709 12:13:04.509128 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 353889us
I20260709 12:13:04.494336 12678 rpcz_store.cc:276] 0709 12:13:02.743924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.172239 (+428315us) service_pool.cc:224] Handling call
0709 12:13:03.172621 (+   382us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:03.173026 (+   405us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:03.173057 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:03.173683 (+   626us) log.cc:844] Serialized 52 byte log entry
0709 12:13:03.751181 (+577498us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:03.977046 (+225865us) log.cc:844] Serialized 14 byte log entry
0709 12:13:03.978771 (+  1725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:03.978885 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:04.478838 (+499953us) raft_consensus.cc:1685] finished
0709 12:13:04.478861 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.479052 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":70949,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":33143,"raft.run_cpu_time_us":451,"raft.run_wall_time_us":32443,"thread_start_us":788936,"threads_started":2,"wal-append.queue_time_us":700254}
I20260709 12:13:04.445485 20064 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:04.505494 12599 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
W20260709 12:13:04.488976 19882 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.729s	user 0.014s	sys 0.000s
I20260709 12:13:04.512632 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:04.515136 12678 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:04.515939 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:04.517061 12603 raft_consensus.cc:2368] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21.
I20260709 12:13:04.518028 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 882) took 1402 ms. Trace:
I20260709 12:13:04.518575 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 13 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:04.519560 12603 raft_consensus.cc:2393] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 17: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:04.518451 12523 rpcz_store.cc:276] 0709 12:13:03.115644 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.275962 (+160318us) service_pool.cc:224] Handling call
0709 12:13:03.276278 (+   316us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:03.360259 (+ 83981us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:03.360347 (+    88us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:03.361209 (+   862us) log.cc:844] Serialized 52 byte log entry
0709 12:13:03.761014 (+399805us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:03.761059 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:03.761133 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:04.517792 (+756659us) raft_consensus.cc:1685] finished
0709 12:13:04.517814 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.517972 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":80040,"raft.queue_time_us":163598,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":39928,"thread_start_us":399388,"threads_started":1,"wal-append.queue_time_us":633264}
I20260709 12:13:04.520488 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:04.521355 12523 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:13:04.512797 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:04.502804 12571 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:04.510769 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3096) took 1488 ms. Trace:
I20260709 12:13:04.486905 19983 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:04.522374 12530 rpcz_store.cc:276] 0709 12:13:03.022722 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.258486 (+235764us) service_pool.cc:224] Handling call
0709 12:13:03.258813 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:03.259060 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:03.259100 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:03.259575 (+   475us) log.cc:844] Serialized 26 byte log entry
0709 12:13:03.705728 (+446153us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:13:03.705776 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:03.705854 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:04.510479 (+804625us) raft_consensus.cc:1685] finished
0709 12:13:04.510506 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.510692 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":270444,"threads_started":1,"wal-append.queue_time_us":557853}
I20260709 12:13:04.524587 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:04.524695 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:04.525588 12523 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:04.520490 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:04.494973 19942 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:04.527565 12603 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:13:04.501003 19949 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:04.531548 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:04.535682 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:04.536456 12530 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 20: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:04.537227 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:04.506433 19494 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:04.538483 19494 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:04.539032 19494 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:04.546880 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:04.537881 12530 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:04.547938 12678 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 20: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:04.505010 19963 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:13:04.534312 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 365050us
I20260709 12:13:04.546944 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3059) took 1301 ms. Trace:
I20260709 12:13:04.548588 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3061) took 1265 ms. Trace:
I20260709 12:13:04.554383 12674 rpcz_store.cc:276] 0709 12:13:03.245316 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.250240 (+  4924us) service_pool.cc:224] Handling call
0709 12:13:04.546719 (+1296479us) spinlock_profiling.cc:243] Waited 219 ms on lock 0x7b54002c5e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:04.546870 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":503080320}
I20260709 12:13:04.554826 12678 rpcz_store.cc:276] 0709 12:13:03.283286 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.546513 (+1263227us) service_pool.cc:224] Handling call
0709 12:13:04.548506 (+  1993us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:04.527943 12673 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:04.556327 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:04.557766 12674 raft_consensus.cc:2410] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:13:04.558000 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:04.558763 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:04.559844 12673 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:04.490057 20028 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:04.563460 12674 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
I20260709 12:13:04.529626 19983 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:04.531550 19942 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:04.565797 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3057) took 1340 ms. Trace:
I20260709 12:13:04.568295 12674 rpcz_store.cc:276] 0709 12:13:03.225208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.562969 (+1337761us) service_pool.cc:224] Handling call
0709 12:13:04.563274 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:04.565539 (+  2265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.565742 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:04.556329 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:04.553411 19963 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:04.577464 12601 raft_consensus.cc:2368] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21.
I20260709 12:13:04.569193 12530 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:13:04.575228 12523 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 20.
I20260709 12:13:04.584218 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:04.585553 12523 raft_consensus.cc:2368] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20.
W20260709 12:13:04.584532 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.529138 19882 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:04.589205 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:04.546703 20099 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:13:04.567898 19942 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:04.586004 19963 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:04.571066 20079 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:04.595546 12530 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 20.
I20260709 12:13:04.570823 20120 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:13:04.567169 20064 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:04.603041 12673 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
W20260709 12:13:04.606711 20115 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:13:04.648504 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:04.641795 19618 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:04.643092 13305 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:04.609066 19882 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:04.611734 19917 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:04.611874 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:04.612788 20111 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.082s	user 0.000s	sys 0.000s
I20260709 12:13:04.615020 20081 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:04.615674 12599 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:04.661978 20028 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:04.619076 20100 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:04.654661 19779 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:04.663324 19779 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 20 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:04.664419 19779 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:04.665012 20100 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:04.620385 12646 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:13:04.666258 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 45962.7us
I20260709 12:13:04.667315 12646 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:04.625375 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:04.626096 12600 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 21. Current term is 22. Ops: []
W20260709 12:13:04.626909 12680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 3517) took 2811 ms (client timeout 1876 ms). Trace:
W20260709 12:13:04.671974 12680 rpcz_store.cc:269] 0709 12:13:01.815301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:01.816889 (+  1588us) service_pool.cc:224] Handling call
0709 12:13:04.626540 (+2809651us) spinlock_profiling.cc:243] Waited 560 ms on lock 0x7b540008e154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:04.626789 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1287601536}
I20260709 12:13:04.628695 12606 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 17 index: 15. (index mismatch)
W20260709 12:13:04.631804 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:04.632972 20026 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 3.121s	user 0.016s	sys 0.028s
W20260709 12:13:04.638571 12524 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2967) took 2221 ms (client timeout 2090 ms). Trace:
I20260709 12:13:04.602361 20098 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:13:04.655311 12531 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 20: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:04.651852 13305 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:04.615751 19901 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:04.673985 12680 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 16. Current term is 17. Ops: []
W20260709 12:13:04.683609 12524 rpcz_store.cc:269] 0709 12:13:02.416829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.089312 (+672483us) service_pool.cc:224] Handling call
0709 12:13:04.638278 (+1548966us) spinlock_profiling.cc:243] Waited 541 ms on lock 0x7b5400054e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:04.638472 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1245237504}
I20260709 12:13:04.684799 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 802) took 1127 ms. Trace:
W20260709 12:13:04.685400 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 361901us
W20260709 12:13:04.686959 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 205827us
I20260709 12:13:04.684988 12523 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 21: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:04.685277 12680 rpcz_store.cc:276] 0709 12:13:03.557198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.673300 (+1116102us) service_pool.cc:224] Handling call
0709 12:13:04.673737 (+   437us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:04.684240 (+ 10503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.684723 (+   483us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:04.691604 12600 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
I20260709 12:13:04.676358 19827 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:04.676180 19970 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:04.696602 19827 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:04.689373 20064 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:04.691315 20078 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:04.708205 19827 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:04.711588 20078 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:04.730335 20032 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 3.266s	user 0.025s	sys 0.014s
I20260709 12:13:04.707571 19970 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:04.735773 20078 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:13:04.817279 20113 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.665s	user 0.008s	sys 0.011s
I20260709 12:13:04.789009 19970 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:04.826972 19827 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:04.839884 19972 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:04.841274 19960 raft_consensus.cc:3055] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:04.841332 19983 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:04.842010 19960 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:04.843303 19949 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:04.843190 19960 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.16, Last appended by leader: 16, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:04.844524 19983 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:04.844571 19960 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:04.845718 19949 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:04.841845 19963 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:04.850708 19963 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:04.852325 20031 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:04.862516 20092 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.705s	user 0.008s	sys 0.004s
W20260709 12:13:04.868940 20046 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 2.894s	user 0.034s	sys 0.092s
W20260709 12:13:04.903261 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 347161us
W20260709 12:13:04.905169 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:04.907032 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 374063us
W20260709 12:13:04.907356 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.908999 12496 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 15 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:04.911360 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:04.913709 12531 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:04.915110 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:04.916416 12524 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:04.917261 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2999) took 1212 ms. Trace:
I20260709 12:13:04.918872 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2991) took 1322 ms. Trace:
I20260709 12:13:04.919394 12523 rpcz_store.cc:276] 0709 12:13:03.596250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.593523 (+997273us) service_pool.cc:224] Handling call
0709 12:13:04.918621 (+325098us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:04.921149 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:04.917749 12524 rpcz_store.cc:276] 0709 12:13:03.704499 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.687286 (+982787us) service_pool.cc:224] Handling call
0709 12:13:04.917126 (+229840us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:04.927819 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:04.930734 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2979) took 1339 ms. Trace:
I20260709 12:13:04.932406 12524 rpcz_store.cc:276] 0709 12:13:03.591725 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.927311 (+1335586us) service_pool.cc:224] Handling call
0709 12:13:04.930635 (+  3324us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:04.926005 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:04.924086 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3109) took 1070 ms. Trace:
I20260709 12:13:04.940883 12531 rpcz_store.cc:276] 0709 12:13:03.853465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.690333 (+836868us) service_pool.cc:224] Handling call
0709 12:13:04.923971 (+233638us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:04.943188 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:04.945179 12531 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 18: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:04.948275 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2990) took 1352 ms. Trace:
I20260709 12:13:04.925241 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:04.953263 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:04.953959 12600 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:13:04.956840 12600 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:13:04.947042 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:04.958881 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:04.959862 12531 rpcz_store.cc:276] 0709 12:13:03.595934 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.942596 (+1346662us) service_pool.cc:224] Handling call
0709 12:13:04.948201 (+  5605us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:04.962814 12673 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 16 index: 15. Preceding OpId from leader: term: 17 index: 15. (term mismatch)
I20260709 12:13:04.967195 12673 pending_rounds.cc:85] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 14
I20260709 12:13:04.967329 12677 raft_consensus.cc:2393] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 20: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:04.967578 12673 pending_rounds.cc:107] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 16.15
I20260709 12:13:04.968209 12673 raft_consensus.cc:2889] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:04.969221 12680 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:04.969794 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 801) took 1413 ms. Trace:
I20260709 12:13:04.965101 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 798) took 1700 ms. Trace:
I20260709 12:13:04.970713 12673 rpcz_store.cc:276] 0709 12:13:03.556585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.671983 (+1115398us) service_pool.cc:224] Handling call
0709 12:13:04.692453 (+ 20470us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:04.969297 (+276844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.969713 (+   416us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:04.970791 12674 rpcz_store.cc:276] 0709 12:13:03.264542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.570815 (+1306273us) service_pool.cc:224] Handling call
0709 12:13:04.571224 (+   409us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:04.571524 (+   300us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:13:04.590087 (+ 18563us) log.cc:844] Serialized 14 byte log entry
0709 12:13:04.964218 (+374131us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:04.964319 (+   101us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:04.964442 (+   123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.964596 (+   154us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.964826 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":21444,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":13070,"thread_start_us":373260,"threads_started":1,"wal-append.queue_time_us":137330}
W20260709 12:13:04.966639 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.962708 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:04.975719 12673 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 15. Current term is 16. Ops: []
I20260709 12:13:04.976419 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:04.981038 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:04.977988 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:04.943939 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2982) took 1351 ms. Trace:
I20260709 12:13:04.984138 12523 rpcz_store.cc:276] 0709 12:13:03.592921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.925537 (+1332616us) service_pool.cc:224] Handling call
0709 12:13:04.940490 (+ 14953us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:04.987779 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:04.988348 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:04.989310 12523 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
I20260709 12:13:04.992462 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:04.986662 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:04.986704 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:04.993381 13336 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:04.994000 12600 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 20: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:04.995566 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:05.000526 13332 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:13:04.822263 19689 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:04.819861 19494 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:04.833313 19697 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.622s	user 0.009s	sys 0.005s
I20260709 12:13:05.006098 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:05.007459 19697 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:04.847833 19875 consensus_meta.cc:287] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.599s	user 0.012s	sys 0.004s
I20260709 12:13:05.013830 19875 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:05.016620 20060 raft_consensus.cc:3055] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:13:05.017607 20060 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:05.026181 20060 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:05.028882 19909 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:04.854001 19958 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:05.031631 20060 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:05.032303 19827 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:05.022693 19980 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.034757 19980 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.037375 19955 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:13:05.038663 19980 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:05.039038 19942 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.039570 19980 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:05.040238 19980 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:05.041431 12527 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2984) took 1447 ms (client timeout 1722 ms). Trace:
W20260709 12:13:05.062659 12527 rpcz_store.cc:269] 0709 12:13:03.593586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.682727 (+ 89141us) service_pool.cc:224] Handling call
0709 12:13:05.040881 (+1358154us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b54003f6e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:05.041174 (+   293us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":30215424}
I20260709 12:13:05.063719 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 797) took 1799 ms. Trace:
I20260709 12:13:05.063987 19762 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:05.065270 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3524) took 1038 ms. Trace:
I20260709 12:13:05.065891 12680 rpcz_store.cc:276] 0709 12:13:04.027124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.695173 (+668049us) service_pool.cc:224] Handling call
0709 12:13:04.695518 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:04.969945 (+274427us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.064884 (+ 94939us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b54003bc754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:05.065181 (+   297us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":627600256}
I20260709 12:13:05.067018 19625 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader election won for term 16
I20260709 12:13:05.029922 19909 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:05.047190 19494 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:05.067948 19625 raft_consensus.cc:697] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:05.048592 12674 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:05.049456 12606 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 18: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:04.964534 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:05.041803 19942 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:05.054931 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 552066us
I20260709 12:13:05.072606 12672 rpcz_store.cc:276] 0709 12:13:03.264295 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.277356 (+ 13061us) service_pool.cc:224] Handling call
0709 12:13:05.063036 (+1785680us) spinlock_profiling.cc:243] Waited 472 ms on lock 0x7b540016f8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:05.063398 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1086049664}
I20260709 12:13:05.074195 19942 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
W20260709 12:13:05.055397 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 552544us
I20260709 12:13:05.056512 20071 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.050163 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:05.057871 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.059165 20081 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:05.058713 20073 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:05.057780 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3427) took 1788 ms. Trace:
I20260709 12:13:05.061988 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3087) took 1183 ms. Trace:
I20260709 12:13:05.070641 19845 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.058097 20079 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:05.059891 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3063) took 1714 ms. Trace:
I20260709 12:13:04.985222 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:05.067420 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 884) took 1582 ms. Trace:
I20260709 12:13:05.091842 12527 rpcz_store.cc:276] 0709 12:13:03.484981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.064596 (+1579615us) service_pool.cc:224] Handling call
0709 12:13:05.065307 (+   711us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.065832 (+   525us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:05.066935 (+  1103us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:05.067037 (+   102us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:05.067081 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.067181 (+   100us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:05.067333 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:05.097591 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 409602us
I20260709 12:13:05.100865 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:05.077086 12598 rpcz_store.cc:276] 0709 12:13:03.269452 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.269981 (+   529us) service_pool.cc:224] Handling call
0709 12:13:03.270422 (+   441us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:04.001262 (+730840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.057106 (+1055844us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b54002710d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:05.057528 (+   422us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1679342848}
I20260709 12:13:05.088016 12675 rpcz_store.cc:276] 0709 12:13:03.345505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.533728 (+188223us) service_pool.cc:224] Handling call
0709 12:13:05.059321 (+1525593us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b5400012cd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:05.059592 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":700835712}
I20260709 12:13:05.077018 19625 consensus_queue.cc:237] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.124357 19845 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:05.124981 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:05.090016 20073 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:05.130626 20073 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:05.133294 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3108) took 1310 ms. Trace:
I20260709 12:13:05.133801 12524 rpcz_store.cc:276] 0709 12:13:03.822862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.964035 (+1141173us) service_pool.cc:224] Handling call
0709 12:13:05.133208 (+169173us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:05.061482 20087 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:13:05.143234 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:13:05.145157 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 146019us
I20260709 12:13:05.149461 12600 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 16. Current term is 17. Ops: [16.10-16.10]
I20260709 12:13:05.102382 20135 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:13:05.153259 12602 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:05.077601 12604 rpcz_store.cc:276] 0709 12:13:03.878179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.884508 (+  6329us) service_pool.cc:224] Handling call
0709 12:13:03.885001 (+   493us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:03.885501 (+   500us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:03.885556 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:03.886787 (+  1231us) log.cc:844] Serialized 26 byte log entry
0709 12:13:04.398294 (+511507us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:04.491737 (+ 93443us) log.cc:844] Serialized 14 byte log entry
0709 12:13:05.054104 (+562367us) spinlock_profiling.cc:243] Waited 2.21 ms on lock 0x7b18004491a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:13:05.060984 (+  6880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.061152 (+   168us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:05.061279 (+   127us) raft_consensus.cc:1685] finished
0709 12:13:05.061459 (+   180us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:05.061715 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5336,"prepare.queue_time_us":95489,"prepare.run_cpu_time_us":101,"prepare.run_wall_time_us":3244,"raft.queue_time_us":572360,"raft.run_cpu_time_us":3066,"raft.run_wall_time_us":53390,"spinlock_wait_cycles":5081472,"thread_start_us":406864,"threads_started":1,"wal-append.queue_time_us":563707}
I20260709 12:13:05.160750 13336 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:13:05.167060 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:05.170291 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 69624us
I20260709 12:13:05.170801 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:05.171492 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:05.171926 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2997) took 1474 ms. Trace:
I20260709 12:13:05.172395 12527 rpcz_store.cc:276] 0709 12:13:03.697694 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.105848 (+1408154us) service_pool.cc:224] Handling call
0709 12:13:05.125490 (+ 19642us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.171448 (+ 45958us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:13:05.171488 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:05.171543 (+    55us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:13:05.171587 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.171681 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:05.171832 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45150}
I20260709 12:13:05.175699 12522 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:13:05.176566 19840 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.177094 19840 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:05.178469 19845 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:05.180336 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 55659us
I20260709 12:13:05.181618 12572 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:05.182420 19962 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.147487 20071 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.183073 19962 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:05.184583 13333 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:05.184746 20081 raft_consensus.cc:697] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:05.185513 20123 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:05.186007 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:05.186879 12598 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:05.186362 20123 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:05.161908 12496 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:05.190630 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:05.192714 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:05.193750 13338 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 18: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:05.198068 19845 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:05.198627 19845 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:05.199061 19845 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:05.199118 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 15 index: 11 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:05.200204 13330 raft_consensus.cc:2410] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 18 because replica has last-logged OpId of term: 16 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 11.
I20260709 12:13:05.203450 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:05.204594 19827 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:05.205317 19827 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.205724 19827 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:05.207516 20135 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.208451 20135 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.209628 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:05.215119 19914 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:05.217368 20044 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:05.221889 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 883) took 2095 ms. Trace:
I20260709 12:13:05.222481 12526 rpcz_store.cc:276] 0709 12:13:03.126598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.296228 (+169630us) service_pool.cc:224] Handling call
0709 12:13:03.296551 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:03.721415 (+424864us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:03.721483 (+    68us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:03.735751 (+ 14268us) log.cc:844] Serialized 26 byte log entry
0709 12:13:03.967939 (+232188us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:03.967982 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.221101 (+1253119us) spinlock_profiling.cc:243] Waited 3.37 ms on lock 0x7b5400015754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:05.221171 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:05.221262 (+    91us) raft_consensus.cc:1685] finished
0709 12:13:05.221288 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:05.221659 (+   371us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":71039,"raft.queue_time_us":153452,"raft.run_cpu_time_us":245,"raft.run_wall_time_us":31786,"spinlock_wait_cycles":7748352,"thread_start_us":586191,"threads_started":2,"wal-append.queue_time_us":394849}
I20260709 12:13:05.226306 12526 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:05.226926 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:05.227293 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 886) took 1234 ms. Trace:
I20260709 12:13:05.227712 12526 rpcz_store.cc:276] 0709 12:13:03.992818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.224535 (+1231717us) service_pool.cc:224] Handling call
0709 12:13:05.225098 (+   563us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.227061 (+  1963us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.227238 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:05.228397 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:05.229420 12604 raft_consensus.cc:2410] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:13:05.224238 12572 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:05.226480 20044 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.230897 20044 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.231899 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:05.193830 20120 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:05.233759 12598 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:13:05.234678 20044 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:05.234524 19901 consensus_queue.cc:1059] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:13:05.236469 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 129059us
I20260709 12:13:05.246762 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 800) took 1932 ms. Trace:
I20260709 12:13:05.248208 12672 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:13:05.248646 19882 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:05.227136 20028 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:05.250789 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:05.252089 12680 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:05.254540 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3088) took 1375 ms. Trace:
I20260709 12:13:05.255602 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 779) took 1303 ms. Trace:
I20260709 12:13:05.255927 19762 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:05.256789 19624 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:05.256166 12599 rpcz_store.cc:276] 0709 12:13:03.952629 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.522770 (+570141us) service_pool.cc:224] Handling call
0709 12:13:04.523093 (+   323us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:04.662145 (+139052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.255048 (+592903us) spinlock_profiling.cc:243] Waited 92.3 ms on lock 0x7b54002e59d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:05.255390 (+   342us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":212361088}
I20260709 12:13:05.259455 20068 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:05.259613 20078 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:05.261178 13332 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:05.262696 19958 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:05.263458 19958 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:05.263625 12676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3043) took 2430 ms (client timeout 3111 ms). Trace:
I20260709 12:13:05.263991 19958 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:13:05.264230 12676 rpcz_store.cc:269] 0709 12:13:02.833489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.851081 (+ 17592us) service_pool.cc:224] Handling call
0709 12:13:05.263149 (+2412068us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b54002c5454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:05.263371 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":560779008}
I20260709 12:13:05.262665 19901 raft_consensus.cc:3055] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:05.140362 20079 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:05.266568 19901 raft_consensus.cc:740] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:13:05.257862 12601 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3111) took 1212 ms (client timeout 1529 ms). Trace:
W20260709 12:13:05.271973 12601 rpcz_store.cc:269] 0709 12:13:04.045813 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.051825 (+  6012us) service_pool.cc:224] Handling call
0709 12:13:05.257374 (+1205549us) spinlock_profiling.cc:243] Waited 521 ms on lock 0x7b5400116cd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:05.257648 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1198399744}
W20260709 12:13:05.278426 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 75224.8us
W20260709 12:13:05.274276 12671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3031) took 4320 ms (client timeout 1774 ms). Trace:
W20260709 12:13:05.269289 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 527718us
I20260709 12:13:05.247381 12678 rpcz_store.cc:276] 0709 12:13:03.265074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.580302 (+1315228us) service_pool.cc:224] Handling call
0709 12:13:04.580834 (+   532us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:04.581430 (+   596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:04.581473 (+    43us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:04.600202 (+ 18729us) log.cc:844] Serialized 52 byte log entry
0709 12:13:05.149254 (+549052us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:05.157805 (+  8551us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.164162 (+  6357us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:05.164212 (+    50us) raft_consensus.cc:1685] finished
0709 12:13:05.164218 (+     6us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:05.196981 (+ 32763us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4391,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":1183,"thread_start_us":459499,"threads_started":1,"wal-append.queue_time_us":443680}
I20260709 12:13:05.256438 19955 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election lost for term 15. Reason: could not achieve majority
W20260709 12:13:05.285657 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 90911us
W20260709 12:13:05.282953 12671 rpcz_store.cc:269] 0709 12:13:00.953559 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:00.953692 (+   133us) service_pool.cc:224] Handling call
0709 12:13:05.268936 (+4315244us) spinlock_profiling.cc:243] Waited 464 ms on lock 0x7b540016f3d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:05.273924 (+  4988us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1067262336}
I20260709 12:13:05.242774 19909 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:05.255077 12597 rpcz_store.cc:276] 0709 12:13:03.878587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.878747 (+   160us) service_pool.cc:224] Handling call
0709 12:13:03.884216 (+  5469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:04.137504 (+253288us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:04.137569 (+    65us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:04.137647 (+    78us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:13:04.137695 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.253255 (+1115560us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b1800441940. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419
0709 12:13:05.253889 (+   634us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b540007b054. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:05.253938 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:05.254302 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65740,"raft.queue_time_us":55570,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":12969,"spinlock_wait_cycles":486250112}
I20260709 12:13:05.186321 20081 consensus_queue.cc:237] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.298405 20112 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:05.300340 20112 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:05.301784 20112 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.302681 19980 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:05.303869 12527 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:05.304904 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3000) took 1589 ms. Trace:
I20260709 12:13:05.305349 12527 rpcz_store.cc:276] 0709 12:13:03.715257 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.173636 (+1458379us) service_pool.cc:224] Handling call
0709 12:13:05.270663 (+ 97027us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.304543 (+ 33880us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.304796 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:05.298810 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.310012 12523 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 14. Current term is 15. Ops: []
I20260709 12:13:05.310911 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:05.294864 20135 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:05.268072 19901 consensus_queue.cc:260] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:05.325474 12602 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:05.324821 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:05.337652 20133 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:05.338318 20133 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.338817 20133 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:05.343398 19901 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:05.328842 19955 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.017s
I20260709 12:13:05.347718 12646 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:05.349269 20135 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:05.349996 20135 raft_consensus.cc:697] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:05.351580 20135 consensus_queue.cc:237] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.329941 20071 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:05.361050 19942 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:05.364028 19697 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:05.368801 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 137078us
I20260709 12:13:05.366842 19875 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:13:05.368316 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3535) took 1200 ms. Trace:
I20260709 12:13:05.372460 12680 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
W20260709 12:13:05.377493 19970 consensus_meta.cc:287] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.540s	user 0.017s	sys 0.003s
I20260709 12:13:05.378198 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 781) took 1330 ms. Trace:
I20260709 12:13:05.378075 19970 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.378590 12603 rpcz_store.cc:276] 0709 12:13:03.977261 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.533077 (+555816us) service_pool.cc:224] Handling call
0709 12:13:04.533483 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:04.533705 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:13:04.558658 (+ 24953us) log.cc:844] Serialized 14 byte log entry
0709 12:13:05.160283 (+601625us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:05.238228 (+ 77945us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:05.290424 (+ 52196us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.294264 (+  3840us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:05.306637 (+ 12373us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":102233,"prepare.queue_time_us":28538,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":40,"raft.queue_time_us":114053,"raft.run_cpu_time_us":213,"raft.run_wall_time_us":213,"thread_start_us":487744,"threads_started":2,"wal-append.queue_time_us":161259}
I20260709 12:13:05.380450 19970 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:05.333276 20123 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:05.386785 12676 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:05.388479 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:05.389276 12671 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:13:05.389222 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 95649.6us
I20260709 12:13:05.390479 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 3528) took 1362 ms. Trace:
I20260709 12:13:05.393684 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3423) took 2126 ms. Trace:
I20260709 12:13:05.394223 12605 rpcz_store.cc:276] 0709 12:13:03.267547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.270393 (+  2846us) service_pool.cc:224] Handling call
0709 12:13:03.299082 (+ 28689us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:03.862294 (+563212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.393144 (+1530850us) spinlock_profiling.cc:243] Waited 524 ms on lock 0x7b54001946d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:05.393472 (+   328us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1204581760}
I20260709 12:13:05.394721 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:05.396616 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:05.396910 20098 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:13:05.397532 20123 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:05.398944 19762 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:05.402040 20077 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:13:05.402591 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3068) took 1346 ms. Trace:
W20260709 12:13:05.403014 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.403206 12674 rpcz_store.cc:276] 0709 12:13:04.056628 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.973289 (+916661us) service_pool.cc:224] Handling call
0709 12:13:04.973818 (+   529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.070861 (+ 97043us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.402322 (+331461us) spinlock_profiling.cc:243] Waited 74.2 ms on lock 0x7b540012d754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:05.402530 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170725632}
I20260709 12:13:05.356381 20078 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:05.405424 19824 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 12:13:05.402529 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:05.391085 12677 rpcz_store.cc:276] 0709 12:13:04.028367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.499912 (+471545us) service_pool.cc:224] Handling call
0709 12:13:05.390034 (+890122us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b5400013e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:05.390366 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1070359936}
I20260709 12:13:05.420086 20119 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:05.421042 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:05.410478 19697 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.372066 12673 rpcz_store.cc:276] 0709 12:13:04.168466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.976713 (+808247us) service_pool.cc:224] Handling call
0709 12:13:04.977271 (+   558us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:04.977968 (+   697us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:04.977998 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:05.176836 (+198838us) log.cc:844] Serialized 52 byte log entry
0709 12:13:05.277605 (+100769us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:05.277655 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.277729 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:05.368024 (+ 90295us) raft_consensus.cc:1685] finished
0709 12:13:05.368048 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:05.368249 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":40517,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":311,"thread_start_us":100253,"threads_started":1,"wal-append.queue_time_us":157929}
I20260709 12:13:05.428730 20119 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:05.436416 20119 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.437775 20119 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:05.457443 12572 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:05.458624 12678 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 18: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:05.460016 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3090) took 3171 ms. Trace:
I20260709 12:13:05.460271 13335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 725) took 1571 ms. Trace:
I20260709 12:13:05.460669 13335 rpcz_store.cc:276] 0709 12:13:03.888543 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.888686 (+   143us) service_pool.cc:224] Handling call
0709 12:13:05.459810 (+1571124us) spinlock_profiling.cc:243] Waited 154 ms on lock 0x7b54004e84d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:05.460068 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":354280832}
I20260709 12:13:05.438623 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:13:05.462276 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 178288us
I20260709 12:13:05.462872 13336 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:05.463294 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:05.465618 12496 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:05.443210 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:05.447547 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:13:05.453632 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.443194 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:05.486215 13338 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:13:05.486393 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:05.460484 12529 rpcz_store.cc:276] 0709 12:13:02.288898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.281737 (+992839us) service_pool.cc:224] Handling call
0709 12:13:03.282053 (+   316us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:03.295727 (+ 13674us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:03.295759 (+    32us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:03.296335 (+   576us) log.cc:844] Serialized 52 byte log entry
0709 12:13:05.225766 (+1929431us) spinlock_profiling.cc:243] Waited 7.47 ms on lock 0x7b18004c71a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:13:05.342311 (+116545us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:05.342431 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.342588 (+   157us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:05.425910 (+ 83322us) raft_consensus.cc:1685] finished
0709 12:13:05.425937 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:05.459781 (+ 33844us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13262,"raft.queue_time_us":70595,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":82043,"spinlock_wait_cycles":17188736,"thread_start_us":81964,"threads_started":1,"wal-append.queue_time_us":2052827}
I20260709 12:13:05.466411 19697 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.471024 19982 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:05.489522 12605 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
I20260709 12:13:04.051777 12679 rpcz_store.cc:276] 0709 12:13:02.827157 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.172567 (+345410us) service_pool.cc:224] Handling call
0709 12:13:03.172845 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:03.173041 (+   196us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:13:03.650010 (+476969us) log.cc:844] Serialized 14 byte log entry
0709 12:13:04.050513 (+400503us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:04.050610 (+    97us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:04.050690 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:04.050809 (+   119us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:04.051043 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3080,"prepare.queue_time_us":564854,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":2057,"raft.queue_time_us":3265,"thread_start_us":860511,"threads_started":2}
I20260709 12:13:05.478834 19762 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:05.493028 19697 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:05.496012 13333 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38472 (request call id 817) took 1419 ms. Trace:
I20260709 12:13:05.496462 13333 rpcz_store.cc:276] 0709 12:13:04.076811 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.077986 (+  1175us) service_pool.cc:224] Handling call
0709 12:13:05.495764 (+1417778us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540061f654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:05.495931 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2541614848}
W20260709 12:13:05.498401 12522 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3083) took 3219 ms (client timeout 3110 ms). Trace:
W20260709 12:13:05.498814 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 64214us
W20260709 12:13:05.498920 12522 rpcz_store.cc:269] 0709 12:13:02.278693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:02.278834 (+   141us) service_pool.cc:224] Handling call
0709 12:13:05.498073 (+3219239us) spinlock_profiling.cc:243] Waited 2.83 s on lock 0x7b54001068d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:05.498279 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6499124480}
I20260709 12:13:05.500021 12646 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:05.500147 19960 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:05.500978 19970 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:05.500782 19960 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:05.501398 19970 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.501833 19970 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:05.479137 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:05.480444 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:05.513585 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48091.1us
I20260709 12:13:05.502512 19942 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:13:05.515532 12671 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
W20260709 12:13:05.522810 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 126255us
I20260709 12:13:05.523751 12494 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:05.535001 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54826us
I20260709 12:13:05.535254 12529 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 16. Current term is 17. Ops: []
I20260709 12:13:05.515471 20060 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:13:05.514991 20133 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:05.540442 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54245.4us
I20260709 12:13:05.542111 20133 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:05.542975 20099 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:13:05.544737 19942 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:13:05.545123 20073 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:05.547801 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:05.549599 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:05.550565 20036 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:05.551348 20036 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:05.552191 19982 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:05.552863 20036 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:05.553471 20119 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:05.551322 20100 consensus_queue.cc:1059] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:13:05.553933 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:05.553979 20119 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:05.555130 20100 raft_consensus.cc:3055] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:13:05.555554 20100 raft_consensus.cc:740] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:05.555434 20119 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.556828 20100 consensus_queue.cc:260] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 15.8, Last appended by leader: 8, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.558255 20099 raft_consensus.cc:697] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:05.559538 20099 consensus_queue.cc:237] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:05.560562 20073 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:05.557314 20119 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:05.561527 12526 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:05.562393 19827 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:05.564893 19827 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:05.566897 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:05.568073 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:05.570910 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 97054us
I20260709 12:13:05.540472 12643 leader_election.cc:400] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21.
I20260709 12:13:05.573855 12643 leader_election.cc:403] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:13:05.571636 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 122526us
I20260709 12:13:05.507262 19845 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.534451 13336 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:05.494002 19958 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:05.569875 20141 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:13:05.584708 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 44485.6us
I20260709 12:13:05.587787 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:05.588735 12671 raft_consensus.cc:2368] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21.
I20260709 12:13:05.606016 13330 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 26: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:05.609649 12572 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:05.610354 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:05.587910 19762 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:05.617331 12494 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:05.610168 12680 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 12 index: 8. Preceding OpId from leader: term: 16 index: 10. (index mismatch)
I20260709 12:13:05.622532 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:05.589370 20100 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:05.587946 19697 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.036s
I20260709 12:13:05.596196 19827 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21.
I20260709 12:13:05.621099 19762 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.637384 19942 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:05.622535 19725 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.056s
I20260709 12:13:05.622567 19875 consensus_queue.cc:1059] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.013s
I20260709 12:13:05.633709 20073 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:13:05.670614 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 53356.9us
I20260709 12:13:05.655972 19610 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:05.663443 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:05.631475 20064 consensus_queue.cc:1059] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 14.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.095s
I20260709 12:13:05.664770 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:05.642050 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:05.653306 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 44063.4us
I20260709 12:13:05.712428 12680 raft_consensus.cc:2393] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 20: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:05.668309 12679 raft_consensus.cc:2368] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 21.
I20260709 12:13:05.654516 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:05.668798 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:05.667097 12674 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
W20260709 12:13:05.702546 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:05.722510 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49567.5us
I20260709 12:13:05.667575 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:05.712704 12603 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:13:05.726774 12672 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 24: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:05.734441 12671 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:05.736655 12571 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:05.659574 19875 raft_consensus.cc:3055] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Stepping down as leader of term 14
I20260709 12:13:05.670912 19982 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:05.648790 19762 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:05.659574 19779 raft_consensus.cc:3055] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:05.695147 20135 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:05.742466 19875 raft_consensus.cc:740] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:05.729508 19962 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:05.751664 20139 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.646304 19624 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:05.747766 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:05.655833 19958 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:05.669451 19960 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:05.781533 12646 leader_election.cc:400] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20.
I20260709 12:13:05.781961 12646 leader_election.cc:403] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:13:05.757233 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:05.761437 12672 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:13:05.751065 19693 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.761615 12494 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:05.763813 19842 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:05.760327 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:05.767220 19875 consensus_queue.cc:260] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 14.8, Last appended by leader: 8, Current term: 14, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.777483 12679 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:13:05.673897 19942 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:05.781399 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 185483us
I20260709 12:13:05.733495 20065 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:05.759780 12677 raft_consensus.cc:2410] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 19 because replica has last-logged OpId of term: 17 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 11.
I20260709 12:13:05.761065 20135 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:05.753523 19779 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:05.802542 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:05.803604 19779 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:05.805334 19779 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:05.765996 19901 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:05.786521 19693 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.789538 19842 raft_consensus.cc:697] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:05.808302 19842 consensus_queue.cc:237] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.790786 19875 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 14 FOLLOWER]: Advancing to term 15
I20260709 12:13:05.810698 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2993) took 2161 ms. Trace:
I20260709 12:13:05.811141 12531 rpcz_store.cc:276] 0709 12:13:03.649352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.984658 (+1335306us) service_pool.cc:224] Handling call
0709 12:13:05.810433 (+825775us) spinlock_profiling.cc:243] Waited 436 ms on lock 0x7b540034fb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:05.810602 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1002211200}
I20260709 12:13:05.812966 12531 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:13:05.791587 12674 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
W20260709 12:13:05.793936 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 144719us
W20260709 12:13:05.787906 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.789800 12675 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:05.762046 19982 raft_consensus.cc:697] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:05.798628 12680 raft_consensus.cc:2410] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 23 because replica has last-logged OpId of term: 22 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 9.
I20260709 12:13:05.800846 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 888) took 1148 ms. Trace:
I20260709 12:13:05.828989 12526 rpcz_store.cc:276] 0709 12:13:04.646580 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.228854 (+582274us) service_pool.cc:224] Handling call
0709 12:13:05.229270 (+   416us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.561940 (+332670us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.794921 (+232981us) spinlock_profiling.cc:243] Waited 332 ms on lock 0x7b540027f6d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:05.795111 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":763583744}
I20260709 12:13:05.808938 19539 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:05.809705 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:05.813849 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52339.7us
I20260709 12:13:05.814815 19960 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:05.792007 12679 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 20 index: 16. Preceding OpId from leader: term: 21 index: 16. (term mismatch)
I20260709 12:13:04.499246 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:13:05.814386 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 57852.3us
W20260709 12:13:05.801656 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 90931.2us
I20260709 12:13:05.766391 20073 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:05.844126 13336 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:05.832895 12679 pending_rounds.cc:85] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 15
I20260709 12:13:05.852878 12679 pending_rounds.cc:107] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 20.16
I20260709 12:13:05.853226 12679 raft_consensus.cc:2889] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:05.845809 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:05.854189 12529 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 26
W20260709 12:13:05.849418 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 217124us
I20260709 12:13:05.835353 12671 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:05.823712 20135 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:05.868214 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:05.872442 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:05.873457 12672 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:13:05.875002 19491 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:05.876768 19826 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:13:05.876961 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 67348us
W20260709 12:13:05.878286 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:05.879606 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:05.880282 20120 raft_consensus.cc:3055] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:13:05.880851 20120 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:05.881397 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:05.883510 20079 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:05.884598 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2983) took 2291 ms. Trace:
I20260709 12:13:05.885385 20127 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.889393 19840 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:05.878311 12496 leader_election.cc:400] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21.
I20260709 12:13:05.878232 19491 raft_consensus.cc:697] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:05.897739 12523 rpcz_store.cc:276] 0709 12:13:03.593246 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.067722 (+1474476us) service_pool.cc:224] Handling call
0709 12:13:05.068305 (+   583us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:05.312393 (+244088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.884258 (+571865us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b540038f9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:05.884532 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":552519424}
I20260709 12:13:05.915586 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:05.905157 12496 leader_election.cc:403] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:05.900760 20112 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:05.922261 20127 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.845593 19827 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20.
I20260709 12:13:05.929564 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:05.930820 12604 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 24: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:05.934458 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:05.935572 20127 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:05.937554 19625 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:05.941169 19624 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:13:05.942214 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:05.942826 19491 consensus_queue.cc:237] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.943527 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:05.943557 20073 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:05.949170 20141 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:05.949648 20141 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:13:05.950343 12643 leader_election.cc:336] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.529s (SENT)
I20260709 12:13:05.950917 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:05.950700 20141 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:05.952221 20141 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:05.830281 20139 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.953684 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:05.956753 20150 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.114s	user 0.002s	sys 0.001s
I20260709 12:13:05.968670 12598 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:05.902014 19958 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.068s
W20260709 12:13:05.958730 20154 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.122s	user 0.000s	sys 0.002s
I20260709 12:13:05.959190 19827 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:13:05.959887 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 140945us
I20260709 12:13:05.975626 12677 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 21: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:05.960569 19970 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:05.862088 20065 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:05.966272 20079 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:05.951819 19982 consensus_queue.cc:237] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:05.952353 12643 leader_election.cc:336] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.531s (SENT)
I20260709 12:13:05.995553 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:05.996146 12643 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:05.996469 12598 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:13:06.009269 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:05.921839 19845 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:05.970481 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:06.011433 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:06.015511 12678 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 22: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:06.007769 12571 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:06.017499 13330 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 21: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:13:06.024268 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52838.3us
W20260709 12:13:06.026752 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 148634us
I20260709 12:13:06.032413 12600 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:13:06.030529 19827 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:06.082261 19826 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:06.079566 12605 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:06.080799 20068 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:13:06.082897 19826 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:06.085158 12604 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 20: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:06.080996 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:06.081393 12597 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:06.084276 20068 raft_consensus.cc:3055] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:13:06.090637 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:06.100979 12571 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:13:06.107369 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.108238 12606 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:13:06.108513 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:06.093680 20068 raft_consensus.cc:740] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:06.038280 19901 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.052s
I20260709 12:13:06.043447 19624 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.053s
W20260709 12:13:06.117054 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 86482.6us
I20260709 12:13:06.117285 20068 consensus_queue.cc:260] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.8, Last appended by leader: 8, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:06.124744 20099 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.007s
W20260709 12:13:06.133343 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:06.134459 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 182158us
I20260709 12:13:06.122081 19697 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:06.114041 19618 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:13:06.078101 20079 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:06.145418 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3003) took 1231 ms. Trace:
I20260709 12:13:06.099105 19689 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.228s
W20260709 12:13:06.156643 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.127858 19845 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:13:06.132099 20068 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:06.164219 12571 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:06.177886 12524 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.573s	user 0.006s	sys 0.034s
W20260709 12:13:06.178385 12524 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.573s	user 0.006s	sys 0.034s
I20260709 12:13:06.134601 19958 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:06.132716 19693 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:06.154598 12527 rpcz_store.cc:276] 0709 12:13:04.914364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.368260 (+453896us) service_pool.cc:224] Handling call
0709 12:13:05.439360 (+ 71100us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:05.440071 (+   711us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:05.440108 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:05.440725 (+   617us) log.cc:844] Serialized 26 byte log entry
0709 12:13:05.583756 (+143031us) raft_consensus.cc:1645] Marking committed up to 7
0709 12:13:05.583830 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.583958 (+   128us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:05.958028 (+374070us) spinlock_profiling.cc:243] Waited 16.1 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:06.137910 (+179882us) raft_consensus.cc:1685] finished
0709 12:13:06.137933 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:06.145331 (+  7398us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3996,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":174,"spinlock_wait_cycles":36951424,"thread_start_us":142249,"threads_started":1,"wal-append.queue_time_us":297228}
I20260709 12:13:06.190585 12496 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:06.196537 12496 leader_election.cc:400] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 21.
I20260709 12:13:06.196913 12496 leader_election.cc:403] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
W20260709 12:13:06.202651 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 187323us
W20260709 12:13:06.210216 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.215463 12525 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:06.198346 20099 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:13:06.218102 13302 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.584s (SENT) [suppressed 27 similar messages]
I20260709 12:13:06.219892 12496 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:06.220849 19610 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:06.222347 20119 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:06.223317 20119 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:13:06.226805 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3001) took 1313 ms. Trace:
I20260709 12:13:06.227321 12528 rpcz_store.cc:276] 0709 12:13:04.913373 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.359382 (+446009us) service_pool.cc:224] Handling call
0709 12:13:05.359929 (+   547us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:05.385196 (+ 25267us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:05.385220 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:05.421932 (+ 36712us) log.cc:844] Serialized 26 byte log entry
0709 12:13:05.513286 (+ 91354us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:05.513347 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:05.513466 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:06.226284 (+712818us) raft_consensus.cc:1685] finished
0709 12:13:06.226308 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:06.226591 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21208,"raft.queue_time_us":26400,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":135,"thread_start_us":88082,"threads_started":1,"wal-append.queue_time_us":802050}
I20260709 12:13:06.234283 19826 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:06.235356 20135 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:06.236568 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:06.248899 20028 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:06.249639 20028 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:06.250222 20028 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:06.252089 20122 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.252727 20122 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:06.256214 19624 consensus_queue.cc:1059] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 19.10, Next index: 11, Last known committed idx: 8, Time since last communication: 0.001s
I20260709 12:13:06.258625 19875 raft_consensus.cc:3055] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:06.259371 19875 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:06.261077 12572 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:06.236670 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:06.263713 12496 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:06.238588 19618 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21.
I20260709 12:13:06.241798 20099 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:06.265446 19826 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:06.237682 19610 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 21.
I20260709 12:13:06.264569 19693 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:13:06.185305 19958 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:06.242938 20079 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:06.193503 19845 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:06.269923 12496 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:13:06.258906 20158 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.144s	user 0.000s	sys 0.001s
I20260709 12:13:06.276595 12496 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:06.273439 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 2998) took 2571 ms. Trace:
I20260709 12:13:06.272233 19970 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:06.278981 20079 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:06.280270 19826 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:06.278445 12524 rpcz_store.cc:276] 0709 12:13:03.701897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:05.134874 (+1432977us) service_pool.cc:224] Handling call
0709 12:13:05.178220 (+ 43346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:05.178853 (+   633us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:05.178882 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:05.380632 (+201750us) spinlock_profiling.cc:243] Waited 70.5 ms on lock 0x7b2c00058028. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:13:05.381263 (+   631us) log.cc:844] Serialized 26 byte log entry
0709 12:13:05.531647 (+150384us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:13:06.178835 (+647188us) log.cc:844] Serialized 14 byte log entry
0709 12:13:06.179209 (+   374us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:06.179295 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:06.267506 (+ 88211us) raft_consensus.cc:1685] finished
0709 12:13:06.270674 (+  3168us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:06.273367 (+  2693us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":542188,"prepare.run_cpu_time_us":116,"prepare.run_wall_time_us":9512,"raft.queue_time_us":560,"raft.run_cpu_time_us":282,"raft.run_wall_time_us":11043,"spinlock_wait_cycles":162069376,"thread_start_us":721616,"threads_started":2,"wal-append.queue_time_us":770002}
I20260709 12:13:06.316756 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:06.280248 19845 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:06.305439 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:06.282886 19960 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:06.323563 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:06.321763 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:06.293745 19842 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:13:06.328105 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:06.296725 19625 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:06.327514 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:06.287140 20099 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 12:13:06.280117 19912 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.172s
I20260709 12:13:06.286612 19610 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.043s
I20260709 12:13:06.309496 20079 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:06.344647 12674 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:06.342515 12673 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:06.325263 19942 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:06.361860 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:06.369418 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46762.7us
I20260709 12:13:06.358074 19958 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:06.383950 19958 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:06.356957 20079 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:06.386844 19960 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:06.409722 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:06.410260 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 54285.2us
I20260709 12:13:06.410688 12524 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:13:06.411803 12523 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: [20.16-20.16]
I20260709 12:13:06.413538 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:06.417466 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:06.422510 19539 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.565s	user 0.008s	sys 0.004s
I20260709 12:13:06.422823 19826 raft_consensus.cc:3055] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:13:06.423286 19826 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:06.424870 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:06.427304 20144 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:13:06.426571 12530 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:06.443918 12679 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 13 index: 9. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:13:06.444839 19845 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:06.459092 12678 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 17 index: 9. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:13:06.447678 20135 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:06.462586 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:06.463511 12526 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 23: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:06.464649 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:06.450517 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:06.449299 20028 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:06.456308 20079 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:06.470108 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:06.471385 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:06.453841 19827 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:06.472700 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:06.436187 20081 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.466030 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:06.424309 19826 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:06.481105 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:06.481665 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.496816 12526 raft_consensus.cc:2410] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 19 because replica has last-logged OpId of term: 18 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 10.
I20260709 12:13:06.486407 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:06.489161 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:06.507459 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:06.479265 20100 consensus_meta.cc:287] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.846s	user 0.007s	sys 0.006s
I20260709 12:13:06.508345 12600 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
W20260709 12:13:06.516551 12529 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.623s	user 0.013s	sys 0.000s
I20260709 12:13:06.517180 12529 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26.
I20260709 12:13:06.519275 12528 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:13:06.512858 12604 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:06.520226 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:06.521102 12602 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:06.524266 12522 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:06.519168 20064 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:06.522336 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:06.519749 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:06.534405 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:06.511345 19539 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:06.537866 12606 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
W20260709 12:13:06.534518 19762 consensus_meta.cc:287] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.755s	user 0.011s	sys 0.005s
I20260709 12:13:06.499697 20081 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:06.542541 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.533941 19826 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:06.540396 12601 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
W20260709 12:13:06.554845 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 390733us
I20260709 12:13:06.558584 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:06.545954 12496 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:06.538980 19539 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:06.580109 19539 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:06.598824 19851 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:06.593820 20099 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.005s
I20260709 12:13:06.574613 20139 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:06.599582 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:06.557143 19762 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:06.606930 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:06.612205 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:06.553869 20120 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:06.614636 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 396664us
I20260709 12:13:06.601583 19851 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:06.624645 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:06.631403 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 481805us
I20260709 12:13:06.632203 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:06.638271 19697 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:06.638547 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:06.639425 12603 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 23: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:06.639637 19697 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 19.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:06.640666 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:06.641300 12600 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:13:06.641878 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.643935 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:06.645366 19958 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:13:06.648397 12603 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 21. Current term is 22. Ops: [21.15-21.15]
W20260709 12:13:06.654484 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.653970 12600 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26.
I20260709 12:13:06.601423 19982 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:13:06.666141 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:06.674494 12678 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:06.629976 19779 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.031s
I20260709 12:13:06.657513 20079 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:06.685086 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:06.689633 12528 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:06.672529 19827 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:06.696971 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:06.696492 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:06.684029 20028 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:06.686573 19798 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:06.686486 20100 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:06.719101 12643 leader_election.cc:336] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.062s (SENT)
I20260709 12:13:06.719628 12643 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:06.722594 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:06.724593 12529 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:06.706074 19827 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:06.730523 20100 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:06.731600 19827 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:06.732858 20100 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:06.743398 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:06.745951 19624 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:06.763898 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50397.2us
I20260709 12:13:06.773092 12526 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:13:06.781584 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:06.779285 12524 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:06.784435 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:06.785458 12524 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 19: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:06.787147 12527 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:06.787791 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:06.769683 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:06.751902 12523 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:06.715036 19798 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:06.788441 12674 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:13:06.765609 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:13:06.796494 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.797158 12678 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:13:06.798535 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:06.808773 20139 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:06.746858 19624 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:06.810555 20079 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:06.811676 20079 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:06.812139 19624 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:06.764243 19960 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:13:06.750624 20171 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 12:13:06.814879 20193 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:13:06.815384 20192 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.815553 20193 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:06.815973 20192 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:06.816733 20193 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:06.775827 20065 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:06.819553 20128 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.820246 20128 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:06.747953 19762 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:06.808557 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:06.821585 12603 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
I20260709 12:13:06.822311 19798 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:06.746858 20175 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.823964 20175 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:06.747220 12572 connection.cc:256] RPC call timeout handler was delayed by -1.1969732390000445s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1807s for a response
I20260709 12:13:06.826686 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:06.827427 12524 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
I20260709 12:13:06.828521 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:06.809443 20139 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:06.829142 12524 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:06.746277 20068 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:06.746105 20141 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:06.838464 19960 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:06.839059 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:06.844342 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 17 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:06.845639 12605 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:06.843325 13305 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:06.746400 19845 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:13:06.850876 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.853628 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:06.772277 20122 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:06.856539 12496 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:06.857384 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:06.858328 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:06.831893 20079 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:06.857653 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 15 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:06.866977 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:06.869659 12605 raft_consensus.cc:2393] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:06.864804 12496 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:06.863891 20133 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:06.875921 12496 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:06.876644 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:06.878616 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:06.879518 12677 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:06.854465 20065 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:06.883383 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:06.877663 12603 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 20.
I20260709 12:13:06.890570 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:06.892422 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:06.873860 19610 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:06.909219 12523 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 20.
I20260709 12:13:06.867729 20192 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:06.873559 20175 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:06.857774 20141 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:06.910530 12603 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 22: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:06.855289 20068 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:06.877825 20100 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:06.914276 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 148735us
I20260709 12:13:06.885155 20128 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:06.923378 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:06.923749 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:06.883605 20065 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:06.938150 12675 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 23: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:06.926064 13305 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:06.890595 20122 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:06.866433 19845 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:06.892004 20079 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election won for term 19
W20260709 12:13:06.944267 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50025.4us
I20260709 12:13:06.942401 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:06.955366 12605 raft_consensus.cc:2410] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:13:06.956651 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 13 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:06.957262 12600 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:06.885263 19539 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:06.883033 20133 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:06.966665 12524 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:06.967440 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:06.962778 12496 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:06.957780 12603 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:06.920855 20192 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:06.978891 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:06.944463 20065 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:06.945446 20122 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:06.974735 20028 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:06.990065 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:06.996865 12526 raft_consensus.cc:2368] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
I20260709 12:13:06.998716 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:06.937803 20100 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:07.001117 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:06.967346 20079 raft_consensus.cc:697] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:07.000463 12530 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2987) took 3405 ms (client timeout 2642 ms). Trace:
I20260709 12:13:06.981689 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:07.011598 12523 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:07.015340 12524 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 26: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:13:07.015864 12530 rpcz_store.cc:269] 0709 12:13:03.594884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:04.648053 (+1053169us) service_pool.cc:224] Handling call
0709 12:13:06.984986 (+2336933us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b54003f7354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:07.000331 (+ 15345us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4082739968}
I20260709 12:13:07.005273 13339 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:07.010988 12605 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:07.023268 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:07.024395 12643 leader_election.cc:336] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.204s (SENT)
I20260709 12:13:07.032877 12643 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:07.026437 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 100448us
I20260709 12:13:07.034160 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:07.003049 12600 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
W20260709 12:13:07.026026 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 63358.4us
W20260709 12:13:07.038781 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 120040us
I20260709 12:13:07.037544 12603 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:07.013068 20122 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:07.048486 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:07.049638 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:07.053191 13339 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 21: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:07.054234 12646 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:07.056275 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:07.018729 20028 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:07.039194 20127 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:07.061749 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:07.016906 20192 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:07.076436 12603 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:07.065297 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:07.073508 12523 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:13:07.079984 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:07.080554 12524 raft_consensus.cc:2410] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 22 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
W20260709 12:13:07.081772 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 61537.8us
I20260709 12:13:07.078250 13334 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 19 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
I20260709 12:13:07.079468 12646 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:07.087260 20133 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:07.089263 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:07.089525 12599 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 15.
I20260709 12:13:07.090246 12597 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 22: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:07.090520 20206 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.091339 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 16 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:07.091279 20206 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:07.092572 19539 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:07.092895 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:07.120837 12605 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
W20260709 12:13:07.124727 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 59812.4us
I20260709 12:13:07.115747 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:07.126505 12646 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:07.128494 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 16 index: 7 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:07.129667 12603 raft_consensus.cc:2410] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 19 because replica has last-logged OpId of term: 18 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 7.
I20260709 12:13:07.051338 20099 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:07.138867 20099 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:07.139950 20099 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:07.056615 20196 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:07.132541 19625 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:07.132668 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:07.133092 20127 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:07.104595 20079 consensus_queue.cc:237] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:07.093997 20206 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:07.134111 20028 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:07.104079 19826 consensus_meta.cc:287] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.513s	user 0.007s	sys 0.004s
I20260709 12:13:07.093998 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:07.144038 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 107897us
I20260709 12:13:07.181496 12606 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:07.149668 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:07.109134 20100 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:07.182930 12599 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:13:07.094503 19845 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:07.183516 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:07.183080 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:07.189385 12522 raft_consensus.cc:2410] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 19 because replica has last-logged OpId of term: 18 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 17 index: 11.
I20260709 12:13:07.189563 12494 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:07.193392 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:07.176314 20133 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:07.194046 19960 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:07.194505 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:07.149581 20201 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:13:07.195454 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.195606 20201 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:07.128087 20170 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Leader election won for term 15
I20260709 12:13:07.175658 19625 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:07.199584 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:07.205463 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:07.205996 13339 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:07.199577 12530 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
W20260709 12:13:07.202703 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 76303.7us
I20260709 12:13:07.201082 13304 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:07.207293 12646 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:07.178582 20196 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:07.182590 20141 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:07.204739 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:07.137936 19798 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:07.218541 12525 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 2994) took 3551 ms (client timeout 1807 ms). Trace:
I20260709 12:13:07.224602 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:07.226796 12679 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:07.233425 12646 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:13:07.225909 12525 rpcz_store.cc:269] 0709 12:13:03.667088 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:03.836881 (+169793us) service_pool.cc:224] Handling call
0709 12:13:07.198331 (+3361450us) spinlock_profiling.cc:243] Waited 2.25 s on lock 0x7b5400054454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:07.218469 (+ 20138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5167364480}
I20260709 12:13:07.243732 12671 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:13:07.244992 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:07.245857 12606 raft_consensus.cc:2410] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:13:07.251331 12673 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 11. (term mismatch)
I20260709 12:13:07.252569 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:07.253002 12673 pending_rounds.cc:85] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 10
I20260709 12:13:07.253305 12679 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:07.253409 12673 pending_rounds.cc:107] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:13:07.251899 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:07.255337 12523 raft_consensus.cc:2368] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
I20260709 12:13:07.253933 12673 raft_consensus.cc:2889] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:07.257041 12496 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:07.258394 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:07.206662 20201 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:07.254382 13302 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:07.232869 20133 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:07.228891 20196 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:07.266413 12675 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 16.
W20260709 12:13:07.274020 12643 leader_election.cc:336] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 3.493s (SENT)
I20260709 12:13:07.276082 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:07.270854 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:07.269788 20141 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.252306 20193 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:13:07.282909 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 81897.2us
I20260709 12:13:07.249092 20170 raft_consensus.cc:697] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:07.284817 13339 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:13:07.263738 19960 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:07.254644 20210 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:13:07.291375 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 102102us
I20260709 12:13:07.249311 20127 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:07.285120 20100 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:07.288625 20141 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:07.299839 20141 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:07.291947 20206 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:07.292138 20128 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:07.317207 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:07.320768 12525 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:13:07.251226 19851 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:07.292884 20201 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 12:13:07.334281 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:07.345472 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:07.319619 19912 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.027s
I20260709 12:13:07.346380 20209 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:07.339665 19875 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.10, Last appended by leader: 10, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:07.347199 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:07.347357 19875 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:07.313854 19826 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.034s
I20260709 12:13:07.348881 12524 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:13:07.324707 20212 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:07.350733 20212 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:07.351167 12529 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 21: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:07.351764 20212 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:07.334466 20192 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:07.354624 20127 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:07.319170 20206 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:07.356050 20206 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:07.357191 12524 pending_rounds.cc:85] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 10
I20260709 12:13:07.357613 12524 pending_rounds.cc:107] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 19.11
I20260709 12:13:07.329137 20170 consensus_queue.cc:237] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 14.8, Last appended by leader: 8, Current term: 15, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:07.359757 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:07.360112 20139 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:07.357929 12524 raft_consensus.cc:2889] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:07.323881 19610 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:07.362565 20028 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:07.363170 12531 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:07.363221 20135 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:13:07.330965 19798 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:07.322527 20128 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:07.364846 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:07.366043 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:07.347085 20209 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:07.367126 12525 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
W20260709 12:13:07.373078 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 81804us
I20260709 12:13:07.377255 12572 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:07.338826 20068 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:07.369030 20128 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:07.399338 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:07.400444 12525 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:13:07.377786 20175 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:07.329430 20099 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:07.407343 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:07.423261 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 129984us
I20260709 12:13:07.428074 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:07.398152 20068 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.447711 12525 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:13:07.448269 12496 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:07.402727 20112 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:13:07.450241 20028 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:13:07.447778 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:07.398329 19845 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:13:07.458168 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.423221 20133 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:13:07.471266 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 226313us
I20260709 12:13:07.458097 19960 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:07.454227 20068 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:07.456826 20064 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:07.523965 20064 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:07.524474 20064 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:13:07.474462 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 79727.1us
I20260709 12:13:07.402469 20065 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:07.500551 20122 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:07.532462 20122 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:07.525714 20068 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:07.504652 12530 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:13:07.505061 20100 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:07.509112 20081 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:07.511775 12526 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
W20260709 12:13:07.512892 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 56439us
I20260709 12:13:07.454211 20139 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:13:07.555310 20139 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:07.555878 20139 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:07.556407 19958 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:07.556850 19958 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:07.520798 12646 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:07.521395 19960 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:07.528650 12572 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:07.529194 20065 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:07.529713 20133 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:07.502611 20209 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:07.533782 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 74369.5us
I20260709 12:13:07.546803 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:07.548792 20221 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.548954 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:07.549700 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:07.551846 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:07.552006 12679 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:07.537477 19779 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:07.554037 20099 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:07.555397 19625 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:07.594274 19826 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:07.595834 19625 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:07.575589 19845 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:07.520725 20219 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.599012 20219 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:07.606614 20219 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:07.608156 13305 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:07.609607 20193 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:07.610263 20193 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:07.562417 12523 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 19. Current term is 21. Ops: []
W20260709 12:13:07.571702 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 51027.5us
I20260709 12:13:07.588816 12676 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 19: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:07.562103 12525 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:07.589587 12673 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:13:07.609994 20076 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:13:07.593071 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46369.7us
I20260709 12:13:07.610709 20193 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:07.620508 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:07.621876 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:07.611912 12646 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:07.574944 20135 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:07.622839 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:07.618261 12494 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:07.614032 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:13:07.624454 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 355998us
I20260709 12:13:07.557847 19958 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:07.628630 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.632156 20076 raft_consensus.cc:3055] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:07.632643 20076 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:13:07.633427 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 106602us
I20260709 12:13:07.634912 12524 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:13:07.630215 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:07.636752 12572 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:07.636909 12676 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:13:07.637646 19845 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:07.638023 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:07.639488 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:07.640612 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:07.641336 12571 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:07.633762 20076 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:07.644522 20076 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:07.625342 20099 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.629558 19618 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:07.630002 20202 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:07.561134 20196 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:07.631150 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:07.631074 19958 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:07.648504 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:07.627749 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:07.667248 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:07.649550 12572 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:07.653926 20209 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:07.652704 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:07.655350 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 43522.1us
I20260709 12:13:07.677202 12646 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:13:07.655844 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.656390 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:07.656977 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:07.661783 12673 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:07.647356 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:07.650262 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:07.671677 12494 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:07.655750 19845 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:07.679764 12677 raft_consensus.cc:2368] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
W20260709 12:13:07.685490 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.613160 19779 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:07.619234 20221 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:07.699527 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:07.659008 19618 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:07.664994 20219 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:07.658746 20099 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:07.679678 20100 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:07.670068 19875 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:07.715862 12604 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22.
I20260709 12:13:07.649093 20064 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:07.715363 12572 leader_election.cc:400] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
I20260709 12:13:07.666064 20135 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:07.728299 12572 leader_election.cc:403] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:13:07.673892 20141 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:07.673725 20178 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:07.660166 20065 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.042s
I20260709 12:13:07.730345 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:07.732656 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:07.697161 20212 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:13:07.733461 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:07.734584 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:07.660806 19842 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 12:13:07.683375 20170 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:07.735366 12679 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:07.663743 20202 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:07.740746 20064 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:07.741974 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:07.728282 20100 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:07.777813 20100 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:07.730209 20217 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:13:07.743892 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:07.721903 20068 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.025s
I20260709 12:13:07.752815 19779 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:07.707198 20209 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:07.786561 20209 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:07.755869 20221 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:07.758203 20099 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:07.707516 20079 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:07.730794 19798 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:07.789877 19798 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:07.758884 20223 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:13:07.738548 12598 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:07.717235 19618 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:13:07.772280 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 64779.8us
I20260709 12:13:07.673499 20036 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 7, Time since last communication: 0.061s
I20260709 12:13:07.774747 20028 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:07.829123 12572 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:07.832151 12572 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:07.775306 20212 raft_consensus.cc:697] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:07.833993 12671 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 18 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:13:07.837152 12571 leader_election.cc:400] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
I20260709 12:13:07.837646 12571 leader_election.cc:403] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:13:07.838269 12572 leader_election.cc:400] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
I20260709 12:13:07.838968 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:07.735647 20141 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:07.840999 12671 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 16 index: 8. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:13:07.847718 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 900) took 1042 ms. Trace:
I20260709 12:13:07.848413 12525 rpcz_store.cc:276] 0709 12:13:06.804935 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:07.449933 (+644998us) service_pool.cc:224] Handling call
0709 12:13:07.516575 (+ 66642us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:07.612627 (+ 96052us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:07.847391 (+234764us) spinlock_profiling.cc:243] Waited 42.7 ms on lock 0x7b540027fbd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:07.847626 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":98123136}
I20260709 12:13:07.846000 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3120) took 1317 ms. Trace:
I20260709 12:13:07.851732 12522 rpcz_store.cc:276] 0709 12:13:06.530999 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:06.534082 (+  3083us) service_pool.cc:224] Handling call
0709 12:13:07.845732 (+1311650us) spinlock_profiling.cc:243] Waited 651 ms on lock 0x7b5400054e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:07.845923 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1496900992}
I20260709 12:13:07.858583 12600 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:13:07.863590 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:07.866246 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:07.839963 20127 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:07.878324 12677 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 16 index: 9. (index mismatch)
I20260709 12:13:07.716454 20220 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.123s
I20260709 12:13:07.866307 20178 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:13:07.889401 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 170775us
I20260709 12:13:07.890516 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:07.855026 20214 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.364s	user 0.004s	sys 0.000s
I20260709 12:13:07.885519 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:07.880756 12600 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:13:07.881836 20127 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:07.843654 19845 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:07.884436 12571 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:07.862294 19851 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:07.846912 20141 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:07.897033 20127 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:07.939000 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:07.939806 12600 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:07.910336 12601 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 22: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:07.911024 20212 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:07.912267 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:07.913048 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:07.931530 12425 catalog_manager.cc:5697] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 17 to 19. New cstate: current_term: 19 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:07.862962 19842 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:07.935346 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 51005.2us
W20260709 12:13:07.913826 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 80569.8us
I20260709 12:13:07.936663 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:07.948325 12598 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:07.945643 12604 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
W20260709 12:13:07.944497 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:07.829960 20206 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:07.829636 20170 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:07.955571 12673 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
W20260709 12:13:07.964030 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:07.966887 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 262645us
I20260709 12:13:07.953650 12496 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:07.983454 12600 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:07.934624 19845 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:13:07.964188 20228 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 12:13:07.939630 19539 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:08.028448 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:08.029441 12527 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:07.991704 12680 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:13:07.992938 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:08.036561 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 48767.6us
I20260709 12:13:07.970286 20219 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:07.974673 20175 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.065s
I20260709 12:13:08.016913 19798 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:08.021267 19610 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:08.020840 19845 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:08.071606 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.076465 12598 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
W20260709 12:13:08.081772 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 113199us
I20260709 12:13:08.082058 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:08.051636 20128 consensus_meta.cc:287] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.650s	user 0.009s	sys 0.005s
W20260709 12:13:08.091123 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:08.090965 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 143698us
I20260709 12:13:07.943387 20135 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:08.098173 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 150262us
I20260709 12:13:08.102909 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:08.104853 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:08.105302 12531 raft_consensus.cc:2368] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
I20260709 12:13:08.107903 12526 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:13:08.105667 12530 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:08.036386 20068 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.055s
I20260709 12:13:08.124698 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:08.124913 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:08.127004 12524 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 19 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:13:08.043066 19539 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:08.080824 19610 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:08.125743 12525 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:08.148813 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:08.092428 20065 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:08.135282 12527 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
I20260709 12:13:08.098712 20196 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:08.177784 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 95883.9us
I20260709 12:13:08.109931 20128 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:08.166019 19610 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:13:08.173019 20139 consensus_meta.cc:287] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.617s	user 0.013s	sys 0.005s
W20260709 12:13:08.181978 20193 consensus_meta.cc:287] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.562s	user 0.013s	sys 0.000s
I20260709 12:13:08.184706 12680 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:08.172881 20175 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:13:08.211944 12529 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: [21.11-21.11]
I20260709 12:13:08.218042 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:08.174661 20065 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:08.196316 20202 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:08.227855 20076 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.573s	user 0.010s	sys 0.002s
W20260709 12:13:08.243705 20230 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.308s	user 0.000s	sys 0.001s
I20260709 12:13:08.244464 20238 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:08.244324 19618 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:08.246661 20128 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:08.219393 19875 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:08.274595 19875 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:08.277274 20237 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.366s	user 0.003s	sys 0.000s
I20260709 12:13:08.171149 20219 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:08.199434 20139 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:08.280870 20139 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:08.258368 20242 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.075s	user 0.001s	sys 0.000s
I20260709 12:13:08.240442 20206 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:08.284087 19618 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:08.286679 20170 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:08.290762 20065 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:08.293768 20232 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.117s	user 0.001s	sys 0.000s
I20260709 12:13:08.255941 20244 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:08.272670 19539 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:08.294276 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 76375.6us
I20260709 12:13:08.306299 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:08.306718 12643 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:08.296471 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:08.308046 12522 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:13:08.301755 12530 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:13:08.214263 20193 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:08.317760 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:08.308857 13305 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:08.328662 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:08.294100 19958 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.621s	user 0.006s	sys 0.001s
I20260709 12:13:08.375480 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:08.366494 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:08.299193 20236 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:08.299055 20220 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:08.385700 12677 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:13:08.277308 19875 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:08.298362 19826 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:08.302760 20238 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:08.390758 13338 raft_consensus.cc:2368] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23.
I20260709 12:13:08.397004 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:08.394932 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 295990us
I20260709 12:13:08.397951 12679 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
W20260709 12:13:08.399977 12494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.680s	user 0.000s	sys 0.025s
W20260709 12:13:08.400609 12494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.000s	sys 0.025s
I20260709 12:13:08.365453 20028 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
I20260709 12:13:08.369782 19842 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
I20260709 12:13:08.403802 12522 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:13:08.383121 20076 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:13:08.409490 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 102869us
I20260709 12:13:08.413517 12524 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:13:08.383810 20178 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:08.394043 19826 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:08.438946 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:08.439283 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:08.402918 19958 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:08.300504 20244 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:08.440917 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:08.442373 12604 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:08.451273 12598 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:08.459349 12522 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
W20260709 12:13:08.438606 20100 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.660s	user 0.009s	sys 0.013s
I20260709 12:13:08.394346 20220 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:08.477444 12600 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 17 index: 10. Preceding OpId from leader: term: 19 index: 11. (index mismatch)
I20260709 12:13:08.437566 19842 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:08.442708 19958 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:08.482602 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:08.444743 20028 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:08.443986 20076 raft_consensus.cc:697] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 21 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:08.487268 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:08.488569 12679 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:13:08.491721 12606 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
W20260709 12:13:08.479300 20141 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.524s	user 0.012s	sys 0.001s
I20260709 12:13:08.495433 12522 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:13:08.494302 13339 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s	user 0.006s	sys 0.011s
I20260709 12:13:08.496806 13336 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 10. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:13:08.494182 12602 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
W20260709 12:13:08.494457 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 350016us
W20260709 12:13:08.507702 13339 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.007s	sys 0.011s
I20260709 12:13:08.405568 20238 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:08.513888 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:08.515743 20128 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:08.482419 20212 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:08.517941 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:08.518695 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.523238 12603 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:13:08.523137 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:08.524461 12494 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:08.524685 19689 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:08.511880 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:08.526260 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:08.527458 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:08.528901 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:08.529608 12599 raft_consensus.cc:2376] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21. Re-sending same reply.
I20260709 12:13:08.524573 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:08.489559 19958 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:08.493592 20249 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:08.473141 20100 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:08.549506 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:08.551371 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:08.552141 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:08.512054 19842 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:08.512118 20141 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:08.536314 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:08.575461 12601 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 22: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:08.511758 20254 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.059s
I20260709 12:13:08.579273 12425 catalog_manager.cc:5697] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 19 to 21. New cstate: current_term: 21 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:08.573554 20081 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:08.581110 20244 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:08.535806 20170 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:13:08.583037 20220 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:08.562700 12599 raft_consensus.cc:2376] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20. Re-sending same reply.
I20260709 12:13:08.564081 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:08.566344 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.513199 19618 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.049s
I20260709 12:13:08.573196 19851 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.020s
I20260709 12:13:08.586503 20238 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:08.587631 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 171376us
I20260709 12:13:08.588615 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:08.589748 20135 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:08.590425 20100 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:08.583850 20249 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:08.592077 12643 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:08.593441 20135 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:13:08.594099 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3021) took 1837 ms. Trace:
I20260709 12:13:08.594583 12529 rpcz_store.cc:276] 0709 12:13:06.756707 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:07.353504 (+596797us) service_pool.cc:224] Handling call
0709 12:13:07.353807 (+   303us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:08.215884 (+862077us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:08.593801 (+377917us) spinlock_profiling.cc:243] Waited 858 ms on lock 0x7b54001072d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:08.594040 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1972595456}
I20260709 12:13:08.558519 20076 consensus_queue.cc:237] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:08.600881 20127 consensus_meta.cc:287] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.640s	user 0.008s	sys 0.001s
I20260709 12:13:08.601064 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:08.603601 20141 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:08.604688 20238 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:13:08.608696 19610 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:08.573251 20193 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:08.614553 12531 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 24: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:08.579187 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:08.615638 12643 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:08.574031 12604 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:08.501268 20028 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:08.616227 12601 raft_consensus.cc:2368] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21.
W20260709 12:13:08.571393 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46996.8us
I20260709 12:13:08.588392 12603 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 22: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:08.623214 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:08.625015 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:08.642609 12600 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:13:08.629081 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 12 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:08.647059 12601 raft_consensus.cc:2393] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 17: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:08.647099 12599 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:08.647727 12604 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:08.619186 19689 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:08.643810 12425 catalog_manager.cc:5697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 15 to 21, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 21 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:08.660429 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:08.629552 19845 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.510s	user 0.009s	sys 0.002s
I20260709 12:13:08.630259 19610 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:08.625204 20249 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:08.621656 20081 raft_consensus.cc:697] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:08.715313 20081 consensus_queue.cc:237] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 19.10, Last appended by leader: 10, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:08.718431 19689 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:08.715155 19826 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:08.640426 20028 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:08.722028 19826 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:08.706444 20079 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.035s
I20260709 12:13:08.707243 19539 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:13:08.708992 12494 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:08.765012 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 56086.9us
I20260709 12:13:08.766782 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:08.708890 19958 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:08.635129 19842 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:08.710211 20251 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:08.777590 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:08.662537 20254 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:08.676380 20220 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:13:08.790302 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:08.619381 20127 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:08.668231 20196 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:08.645267 20135 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:13:08.732869 13304 leader_election.cc:336] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.584s (SENT)
W20260709 12:13:08.808521 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 75716.4us
I20260709 12:13:08.747756 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:08.754920 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 235100us
I20260709 12:13:08.789561 20209 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:08.790019 19610 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:08.819831 19610 raft_consensus.cc:697] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:08.820966 19610 consensus_queue.cc:237] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:08.763777 20193 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:08.823375 20193 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:08.828087 19958 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:08.812028 20209 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:08.830331 13338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 920) took 1116 ms. Trace:
I20260709 12:13:08.830649 20209 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:08.830756 13338 rpcz_store.cc:276] 0709 12:13:07.713622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:07.713788 (+   166us) service_pool.cc:224] Handling call
0709 12:13:08.830062 (+1116274us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b54006419d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:08.830259 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1486078208}
I20260709 12:13:08.771656 20238 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:08.813413 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:08.809484 12603 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:08.777150 20139 raft_consensus.cc:3055] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:08.812109 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:08.811081 12604 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:13:08.790853 19618 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.025s
I20260709 12:13:08.816159 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:08.817868 20127 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:08.784015 20079 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:08.811846 20196 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:08.816834 12671 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:08.833061 13336 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 21: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:08.806078 12599 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:13:08.700269 19845 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:08.841719 12601 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:08.843223 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:08.845350 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:08.803539 20239 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:08.825992 20193 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:08.855273 12494 leader_election.cc:400] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23.
I20260709 12:13:08.869372 12494 leader_election.cc:403] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:08.870455 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:08.873306 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:08.874070 12599 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:08.841367 20139 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:08.877770 12494 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:08.884959 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:08.890031 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:08.910408 20193 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:08.915617 12676 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:08.923379 12599 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
W20260709 12:13:08.919452 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 103437us
I20260709 12:13:08.928354 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:08.872623 20076 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.056s
I20260709 12:13:08.929977 13302 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:08.877205 20127 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:08.879843 20079 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:13:08.930651 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:08.933274 12599 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:08.847946 19842 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 12:13:08.879184 19826 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:08.849308 20238 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:08.908488 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.945999 20139 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:08.789846 20251 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:08.930996 20203 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:13:08.951031 12676 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:13:08.956311 12599 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:13:08.959401 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:08.987311 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:08.968468 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 108598us
W20260709 12:13:08.969758 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:08.914433 19539 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:13:08.915521 19845 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:08.947808 20127 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:08.800030 20254 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:08.872967 20239 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:08.950870 20268 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.174s	user 0.000s	sys 0.001s
I20260709 12:13:08.931463 19827 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:13:08.990682 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3090) took 1395 ms. Trace:
I20260709 12:13:08.988020 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:08.929574 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:08.995339 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:08.989269 12571 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:08.911566 20271 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:08.997167 12572 leader_election.cc:400] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
I20260709 12:13:09.021130 12572 leader_election.cc:403] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:09.025033 12680 rpcz_store.cc:276] 0709 12:13:07.595196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:07.656670 (+ 61474us) service_pool.cc:224] Handling call
0709 12:13:08.990489 (+1333819us) spinlock_profiling.cc:243] Waited 313 ms on lock 0x7b54002c5e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:08.990619 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":719507200}
I20260709 12:13:08.919020 20193 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:09.031780 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 102261us
I20260709 12:13:09.001844 20203 raft_consensus.cc:697] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:09.036875 20203 consensus_queue.cc:237] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 16.11, Last appended by leader: 11, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:09.042479 12604 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:09.003499 20139 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:13:09.045881 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 56699.1us
I20260709 12:13:08.975682 19826 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:09.000681 19539 raft_consensus.cc:697] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:09.055614 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:09.056349 12599 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:09.056643 12680 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:13:09.057752 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:09.059172 12676 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 18: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:09.005820 20262 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:09.058396 19845 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:09.079645 20239 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:09.040086 20219 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:09.079751 19845 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:09.008659 20127 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:09.082901 19539 consensus_queue.cc:237] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:09.086728 20275 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.128s	user 0.000s	sys 0.001s
I20260709 12:13:09.091914 19826 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:09.092500 20276 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.104s	user 0.000s	sys 0.000s
I20260709 12:13:09.092865 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:09.093343 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:09.094062 12598 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
I20260709 12:13:09.097898 19845 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:09.079049 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 82048.4us
I20260709 12:13:09.074561 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:09.099761 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:09.034933 20206 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:09.109671 12601 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:09.111462 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:09.114287 19610 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:09.109488 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:09.113044 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:09.053789 20266 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:13:09.122066 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:09.142373 13333 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:09.122710 20240 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:13:09.125586 20261 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.305s	user 0.001s	sys 0.000s
W20260709 12:13:09.156350 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.125193 20209 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:09.121132 13335 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:09.098757 19618 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:09.115919 19845 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:13:09.126820 20120 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:13:09.127696 20263 log.cc:927] Time spent T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.308s	user 0.002s	sys 0.000s
I20260709 12:13:09.128243 20193 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:09.129138 20270 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.303s	user 0.000s	sys 0.001s
W20260709 12:13:09.130461 20257 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 12:13:09.130401 20196 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:09.117264 12604 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:09.047734 20215 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:09.098896 20262 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:09.103404 20028 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
I20260709 12:13:09.155553 20240 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:09.192626 20240 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:09.194304 20262 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:09.175246 20127 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:09.196069 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3496) took 1297 ms. Trace:
I20260709 12:13:09.180444 12494 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:09.145262 20206 raft_consensus.cc:697] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 23 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:09.179606 12569 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:09.199355 20206 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:09.200661 20028 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:09.200871 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:09.201179 20028 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:09.201555 20028 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:09.196593 12597 rpcz_store.cc:276] 0709 12:13:07.898489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:07.898641 (+   152us) service_pool.cc:224] Handling call
0709 12:13:07.910827 (+ 12186us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:07.911256 (+   429us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:07.911307 (+    51us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:07.911944 (+   637us) log.cc:844] Serialized 52 byte log entry
0709 12:13:08.064694 (+152750us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:08.197345 (+132651us) log.cc:844] Serialized 14 byte log entry
0709 12:13:08.197770 (+   425us) log.cc:844] Serialized 14 byte log entry
0709 12:13:08.600638 (+402868us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b1800149ce0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:13:09.195541 (+594903us) spinlock_profiling.cc:243] Waited 3.76 ms on lock 0x7b1800149ce0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287
0709 12:13:09.195685 (+   144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:09.195764 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:09.195813 (+    49us) raft_consensus.cc:1685] finished
0709 12:13:09.195827 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:09.195998 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18077,"prepare.queue_time_us":294486,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":62,"raft.queue_time_us":402482,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":8225,"spinlock_wait_cycles":38740736,"thread_start_us":480968,"threads_started":3,"wal-append.queue_time_us":330600}
I20260709 12:13:09.178870 20266 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:09.184664 12603 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
I20260709 12:13:09.156900 19610 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:09.261960 20209 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:09.262331 20266 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:09.251679 20272 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:09.263836 20272 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:09.264806 20272 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:09.243103 19845 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:09.270818 19845 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:09.271920 20262 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:09.285641 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 252600us
I20260709 12:13:09.339370 12672 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:09.325969 12571 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:09.345677 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:09.347692 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 167369us
I20260709 12:13:09.348608 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3041) took 1231 ms. Trace:
I20260709 12:13:09.350548 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:09.354375 12677 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 23: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:09.349267 12527 rpcz_store.cc:276] 0709 12:13:08.117273 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:08.174332 (+ 57059us) service_pool.cc:224] Handling call
0709 12:13:08.175040 (+   708us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:08.175563 (+   523us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:08.176656 (+  1093us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:08.230359 (+ 53703us) log.cc:844] Serialized 52 byte log entry
0709 12:13:08.479827 (+249468us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:13:08.479865 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:08.479932 (+    67us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:09.124752 (+644820us) spinlock_profiling.cc:243] Waited 16.5 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:09.195451 (+ 70699us) raft_consensus.cc:1685] finished
0709 12:13:09.195482 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:09.348511 (+153029us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":916,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":2905,"spinlock_wait_cycles":38033280,"thread_start_us":165929,"threads_started":1,"wal-append.queue_time_us":410941}
I20260709 12:13:09.359243 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:09.360339 12672 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:13:09.366658 20262 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:09.373175 20254 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:09.348171 20079 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23.
I20260709 12:13:09.357120 20128 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:09.382624 20128 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:09.383071 20128 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:09.384057 19845 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:09.385099 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.391163 19610 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:09.393301 12606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3492) took 1527 ms (client timeout 1869 ms). Trace:
I20260709 12:13:09.398152 20219 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:09.398825 20219 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:09.402136 20251 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:09.405861 20219 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:09.408987 20064 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:09.411005 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:09.393771 12606 rpcz_store.cc:269] 0709 12:13:07.865823 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:07.865943 (+   120us) service_pool.cc:224] Handling call
0709 12:13:09.393104 (+1527161us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b5400375354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:09.393233 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1285521152}
I20260709 12:13:09.412521 12527 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:09.413226 12606 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:13:09.413705 12530 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:13:09.412585 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:09.428879 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:09.353484 20036 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:09.415349 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:09.436916 12527 raft_consensus.cc:2410] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 22 because replica has last-logged OpId of term: 19 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 8.
I20260709 12:13:09.416342 20264 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:09.440539 20264 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:09.442102 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:09.442837 12530 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:13:09.443153 20264 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:09.416780 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:09.443773 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:09.444578 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:09.445098 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:09.419170 12529 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:13:09.420529 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:09.423002 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:09.326999 20240 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:09.410025 20064 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:09.464156 13332 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:13:09.426632 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:09.435762 20036 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:09.455132 13305 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:09.439808 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:09.468271 12675 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:13:09.468444 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:09.445221 12526 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:09.424994 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:09.449923 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.451377 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:09.455696 20243 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:09.461786 20286 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:09.472287 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:09.500638 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:09.478585 12597 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 24: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:09.475128 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:09.507115 12606 raft_consensus.cc:2368] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22.
I20260709 12:13:09.516091 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:09.520243 12643 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:09.520359 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:09.521318 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:09.524863 12522 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:09.474447 20036 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:09.519775 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:09.528435 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:09.535216 13333 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:09.535374 12674 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:13:09.529485 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:09.538511 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:09.530722 12526 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
I20260709 12:13:09.564018 12526 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:13:09.500728 20203 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:09.437111 20028 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:09.568212 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:09.566427 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:09.571941 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:09.572616 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52452.5us
I20260709 12:13:09.585806 12675 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:09.574046 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:09.576668 12524 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:13:09.522748 20215 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:09.515206 19618 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:09.588004 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 239137us
I20260709 12:13:09.445907 20128 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:09.605414 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:09.578493 20203 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:09.659559 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:09.569801 20284 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:09.519762 20243 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:09.658767 20264 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:09.667379 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:09.669036 12677 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 23: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:09.694361 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:09.695982 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 97481.1us
I20260709 12:13:09.685936 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:09.699232 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:09.666481 20203 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:13:09.714306 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:09.687916 19827 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:13:09.696898 20120 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.538s	user 0.004s	sys 0.010s
I20260709 12:13:09.733125 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:09.738744 20128 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:09.740109 19618 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:09.702155 19625 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:09.743782 19625 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:09.748498 20295 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:13:09.750329 19875 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:09.751119 19875 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:09.754858 20064 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:09.755867 19982 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:09.756513 19982 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:09.757953 12528 raft_consensus.cc:2410] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 17 because replica has last-logged OpId of term: 16 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 9.
I20260709 12:13:09.763052 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 16 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:09.764966 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:09.769302 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:09.770114 12599 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 20
I20260709 12:13:09.773025 20068 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:09.769302 20283 log.cc:927] Time spent T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:13:09.777634 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:09.778899 12604 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 21: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:09.788362 20243 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:09.790298 12599 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:13:09.791224 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:09.792052 20028 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:09.791394 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:09.798717 12680 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 17. Current term is 18. Ops: [17.15-17.15]
I20260709 12:13:09.799746 12599 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]
I20260709 12:13:09.801380 12599 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:13:09.802527 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:09.804498 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 3038) took 1696 ms. Trace:
I20260709 12:13:09.804816 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3606) took 2150 ms. Trace:
I20260709 12:13:09.804935 12531 rpcz_store.cc:276] 0709 12:13:08.107583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:08.148371 (+ 40788us) service_pool.cc:224] Handling call
0709 12:13:09.804276 (+1655905us) spinlock_profiling.cc:243] Waited 457 ms on lock 0x7b54003f7354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:09.804418 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1051895040}
I20260709 12:13:09.805238 12671 rpcz_store.cc:276] 0709 12:13:07.654596 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:07.847758 (+193162us) service_pool.cc:224] Handling call
0709 12:13:07.848528 (+   770us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:08.851970 (+1003442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:09.804556 (+952586us) spinlock_profiling.cc:243] Waited 968 ms on lock 0x7b54002c5454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:09.804726 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2226370944}
I20260709 12:13:09.706703 19610 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:09.806751 19610 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:09.807858 19610 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:09.808939 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3508) took 1326 ms. Trace:
I20260709 12:13:09.809372 12600 rpcz_store.cc:276] 0709 12:13:08.482197 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:08.482355 (+   158us) service_pool.cc:224] Handling call
0709 12:13:09.808769 (+1326414us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b54002ff654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:09.808880 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":351362432}
I20260709 12:13:09.704187 20264 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:09.810324 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:09.810896 12600 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 15 index: 9. (index mismatch)
I20260709 12:13:09.811276 12494 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:09.812230 12494 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:09.812381 20120 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:09.812822 20240 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:09.813015 20120 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:09.813330 13339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 921) took 2086 ms. Trace:
I20260709 12:13:09.812249 20264 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:09.734351 20215 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:09.765836 12603 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:09.833513 12674 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:13:09.822279 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3613) took 1506 ms. Trace:
W20260709 12:13:09.824249 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 96823.2us
I20260709 12:13:09.818907 13331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 837) took 1008 ms. Trace:
I20260709 12:13:09.835242 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:09.834600 12673 rpcz_store.cc:276] 0709 12:13:08.315940 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:08.316207 (+   267us) service_pool.cc:224] Handling call
0709 12:13:08.372760 (+ 56553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:08.406164 (+ 33404us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:08.406210 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:08.406264 (+    54us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:08.406310 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:09.822012 (+1415702us) spinlock_profiling.cc:243] Waited 33.1 ms on lock 0x7b54000933d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:09.822066 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:09.822220 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76058880}
I20260709 12:13:09.838584 12599 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:09.832257 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:09.838963 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:09.839586 12674 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:13:09.837862 12675 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
I20260709 12:13:09.834075 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:09.840835 12679 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 22. Current term is 24. Ops: []
I20260709 12:13:09.842175 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:09.843189 12675 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:13:09.843170 12604 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:13:09.838279 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:09.826934 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:09.824801 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:09.919741 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:09.922067 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 17 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:09.921631 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:09.934021 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 14 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:09.835680 13331 rpcz_store.cc:276] 0709 12:13:08.810741 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:08.810880 (+   139us) service_pool.cc:224] Handling call
0709 12:13:08.829270 (+ 18390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:08.876726 (+ 47456us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:08.876762 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:08.876808 (+    46us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:13:08.876846 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:09.818677 (+941831us) spinlock_profiling.cc:243] Waited 47.2 ms on lock 0x7b54006b0154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:09.818725 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:09.818846 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":108612864}
I20260709 12:13:09.813740 13339 rpcz_store.cc:276] 0709 12:13:07.727279 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:07.727493 (+   214us) service_pool.cc:224] Handling call
0709 12:13:07.727817 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:07.732057 (+  4240us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:13:07.969792 (+237735us) log.cc:844] Serialized 14 byte log entry
0709 12:13:08.515136 (+545344us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:08.515197 (+    61us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:08.515259 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:09.129832 (+614573us) spinlock_profiling.cc:243] Waited 8.82 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:09.813050 (+683218us) spinlock_profiling.cc:243] Waited 3.94 ms on lock 0x7b540061e9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:09.813116 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:09.813240 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":312254,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":16188,"raft.queue_time_us":5803,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":12356,"spinlock_wait_cycles":29359616,"thread_start_us":767992,"threads_started":2,"wal-append.queue_time_us":972451}
I20260709 12:13:09.949955 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:09.950624 13332 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:09.955411 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:09.958326 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 133635us
I20260709 12:13:09.961824 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:09.963199 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 128039us
I20260709 12:13:09.963382 12600 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:09.978260 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:09.984318 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:09.969151 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:10.006598 12522 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:10.006598 13338 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:09.994055 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:09.988513 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:10.009334 12672 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 19: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:09.992864 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:10.010579 12671 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
W20260709 12:13:10.010402 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 46212.2us
I20260709 12:13:09.823199 19689 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.017107 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:10.017789 12672 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:10.013456 12494 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:09.968669 12643 leader_election.cc:400] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21.
W20260709 12:13:10.013125 12602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3489) took 2101 ms (client timeout 1550 ms). Trace:
I20260709 12:13:10.018904 12643 leader_election.cc:403] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:10.021098 13332 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:09.823884 19827 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:10.019032 12602 rpcz_store.cc:269] 0709 12:13:07.860482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:07.860600 (+   118us) service_pool.cc:224] Handling call
0709 12:13:09.961263 (+2100663us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b54002ffb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:09.961395 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1217766144}
I20260709 12:13:10.033949 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:09.814340 20202 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:10.029141 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:10.036935 12531 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:09.851960 20139 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:09.814525 20081 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:09.821831 20209 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:09.809001 20286 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:10.015810 19842 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.023s
W20260709 12:13:10.042024 20269 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.128s	user 0.008s	sys 0.000s
I20260709 12:13:09.964360 19875 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:10.039458 20128 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:10.039870 20178 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:13:10.040514 19845 consensus_queue.cc:1059] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 15.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260709 12:13:10.040832 20079 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:10.041606 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 73130.4us
I20260709 12:13:09.984701 20262 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:10.041903 20170 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:13:10.011516 20250 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.026497 20064 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:09.983474 20251 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:10.042619 12572 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:10.046562 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:09.971644 20218 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:10.055341 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:09.815146 20065 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.221s
I20260709 12:13:09.983954 20036 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:09.774057 20068 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:09.957849 20028 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.093s
I20260709 12:13:10.060518 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:10.113754 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:10.243759 12675 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:10.244300 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:10.246273 12529 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.020s	sys 0.045s
W20260709 12:13:10.246650 12529 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s	user 0.020s	sys 0.046s
I20260709 12:13:10.146910 12600 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22.
I20260709 12:13:10.046041 19625 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:10.056439 20081 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:10.266295 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:10.195221 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:09.814070 20120 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:10.055160 20202 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:10.284468 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:10.286778 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:10.208098 12598 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.677s	user 0.020s	sys 0.043s
I20260709 12:13:10.044790 20139 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:10.194060 20122 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.055s	sys 0.091s
I20260709 12:13:10.033358 19689 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:10.181569 20286 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 24 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:10.239372 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:10.301617 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 259089us
W20260709 12:13:10.187566 19826 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.033s	sys 0.032s
I20260709 12:13:10.047323 19827 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:10.261831 12675 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:13:10.196612 20178 consensus_queue.cc:1243] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:13:10.204170 20079 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:10.269129 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 74041.8us
I20260709 12:13:10.182140 20128 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:10.318550 20128 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:10.319213 20079 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:10.230751 20251 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:10.291285 12598 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.760s	user 0.020s	sys 0.044s
I20260709 12:13:10.191010 20209 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:10.302959 12572 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:10.242790 20254 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:10.310901 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:10.323884 12527 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:10.308036 20203 consensus_meta.cc:287] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.576s	user 0.006s	sys 0.004s
I20260709 12:13:10.233186 20064 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:10.305387 20122 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.056s	sys 0.092s
I20260709 12:13:10.285665 20215 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:10.330696 12526 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:10.283471 20081 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:13:10.311540 19826 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s	user 0.036s	sys 0.032s
I20260709 12:13:10.252815 20250 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:10.333156 12572 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:10.326738 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3514) took 1546 ms. Trace:
I20260709 12:13:10.363545 12598 rpcz_store.cc:276] 0709 12:13:08.780714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.094842 (+314128us) service_pool.cc:224] Handling call
0709 12:13:09.095177 (+   335us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:09.095562 (+   385us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:09.095595 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:09.096162 (+   567us) log.cc:844] Serialized 26 byte log entry
0709 12:13:09.479492 (+383330us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:10.322918 (+843426us) log.cc:844] Serialized 14 byte log entry
0709 12:13:10.326246 (+  3328us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.326371 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:10.326438 (+    67us) raft_consensus.cc:1685] finished
0709 12:13:10.326458 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:10.326643 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2862,"prepare.queue_time_us":61394,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":51,"raft.queue_time_us":10443,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":7535,"thread_start_us":1174747,"threads_started":2,"wal-append.queue_time_us":266649}
I20260709 12:13:10.242058 20218 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:10.347800 12600 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 15 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:13:10.322454 20251 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:10.371565 19827 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:10.372577 19827 consensus_queue.cc:1059] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:13:10.355187 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3107) took 1891 ms. Trace:
I20260709 12:13:10.373669 12678 rpcz_store.cc:276] 0709 12:13:08.463366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:08.463811 (+   445us) service_pool.cc:224] Handling call
0709 12:13:08.483034 (+ 19223us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:08.483297 (+   263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:08.483322 (+    25us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:08.483821 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 12:13:08.657297 (+173476us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:13:08.657542 (+   245us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:08.658295 (+   753us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:10.041423 (+1383128us) spinlock_profiling.cc:243] Waited 55.7 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:10.340300 (+298877us) raft_consensus.cc:1685] finished
0709 12:13:10.340337 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:10.355107 (+ 14770us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":128121088,"thread_start_us":138364,"threads_started":1,"wal-append.queue_time_us":340193}
I20260709 12:13:10.376574 20243 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:10.379271 20139 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:10.309831 20133 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.046s
I20260709 12:13:10.356813 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:10.383003 20133 consensus_queue.cc:1059] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:13:10.361620 12527 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
W20260709 12:13:10.345439 19539 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.004s	sys 0.038s
W20260709 12:13:10.384217 19539 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.570s	user 0.004s	sys 0.039s
I20260709 12:13:10.366276 12675 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:13:10.368394 12572 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:10.385375 20064 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:10.386051 20202 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:10.387022 20028 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:13:10.387419 12531 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:13:10.388917 12572 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:10.350999 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:10.346984 20203 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:10.390555 20243 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:10.391131 20243 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:10.391570 20243 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:10.392516 20203 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:10.383226 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:10.393424 20203 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:13:10.333688 20266 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.013s	sys 0.019s
I20260709 12:13:10.324620 20277 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.338788 20286 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:10.338616 19842 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:10.395008 13331 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:10.397670 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 809) took 1099 ms. Trace:
I20260709 12:13:10.402434 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 811) took 1074 ms. Trace:
I20260709 12:13:10.426577 12679 rpcz_store.cc:276] 0709 12:13:09.327867 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.745050 (+417183us) service_pool.cc:224] Handling call
0709 12:13:09.840666 (+ 95616us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:09.841271 (+   605us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.402166 (+560895us) spinlock_profiling.cc:243] Waited 95.2 ms on lock 0x7b540015d6d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:10.402355 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":219054464}
I20260709 12:13:10.416757 13338 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:10.419102 12672 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:10.394639 12572 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:10.426363 12680 rpcz_store.cc:276] 0709 12:13:09.298082 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.298257 (+   175us) service_pool.cc:224] Handling call
0709 12:13:09.798311 (+500054us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:09.799381 (+  1070us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.397422 (+598041us) spinlock_profiling.cc:243] Waited 473 ms on lock 0x7b54003bdb50. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:10.397595 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1088409856}
W20260709 12:13:10.398783 20303 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.659s	user 0.005s	sys 0.000s
W20260709 12:13:10.401593 20299 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.654s	user 0.005s	sys 0.006s
W20260709 12:13:10.395219 20266 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.578s	user 0.014s	sys 0.019s
W20260709 12:13:10.399951 20296 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.657s	user 0.001s	sys 0.015s
I20260709 12:13:10.450055 12523 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 15.
W20260709 12:13:10.426337 20249 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.476s	user 0.014s	sys 0.019s
W20260709 12:13:10.439989 20289 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.094s	user 0.001s	sys 0.000s
I20260709 12:13:10.389034 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:10.395206 20036 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:10.394105 20203 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:10.439779 20254 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:10.467037 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:10.474308 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:10.475154 12531 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:13:10.464432 20249 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.514s	user 0.014s	sys 0.019s
W20260709 12:13:10.479182 12677 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.530s	user 0.013s	sys 0.010s
W20260709 12:13:10.485304 12677 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.537s	user 0.013s	sys 0.010s
I20260709 12:13:10.401005 20206 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.066s
I20260709 12:13:10.461704 20215 raft_consensus.cc:3055] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:10.395128 20277 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:10.479651 13303 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.650s	user 0.009s	sys 0.014s
W20260709 12:13:10.495226 13303 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.009s	sys 0.015s
W20260709 12:13:10.476220 20264 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.016s	sys 0.032s
W20260709 12:13:10.514871 20264 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s	user 0.017s	sys 0.032s
I20260709 12:13:10.472680 19842 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.509915 20206 consensus_queue.cc:1243] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 15.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:13:10.545800 13331 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:13:10.525704 12531 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
W20260709 12:13:10.541965 12494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.020s	sys 0.019s
W20260709 12:13:10.551625 12494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.533s	user 0.021s	sys 0.019s
W20260709 12:13:10.551995 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 538645us
I20260709 12:13:10.554219 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:10.555761 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:10.556038 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:10.511322 20215 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:10.559091 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3518) took 1736 ms. Trace:
I20260709 12:13:10.559584 12601 rpcz_store.cc:276] 0709 12:13:08.822932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.127675 (+304743us) service_pool.cc:224] Handling call
0709 12:13:09.128014 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:09.154966 (+ 26952us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:13:09.155021 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:09.159579 (+  4558us) log.cc:844] Serialized 26 byte log entry
0709 12:13:09.566381 (+406802us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:09.566433 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:09.566511 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:10.399266 (+832755us) spinlock_profiling.cc:243] Waited 72.8 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:10.550670 (+151404us) raft_consensus.cc:1685] finished
0709 12:13:10.550693 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:10.559016 (+  8323us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4053,"raft.queue_time_us":11196,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":107,"spinlock_wait_cycles":167372544,"thread_start_us":339824,"threads_started":1,"wal-append.queue_time_us":540043}
I20260709 12:13:10.563158 12598 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 21: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:10.565896 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:10.570693 12600 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:13:10.544001 20068 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:10.550336 19618 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.014s	sys 0.018s
I20260709 12:13:10.576797 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:10.577682 12597 raft_consensus.cc:2393] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 23: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:10.549744 20238 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:10.563169 20122 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:10.545392 19842 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:10.582597 19618 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.014s	sys 0.018s
W20260709 12:13:10.615537 12676 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.019s	sys 0.009s
W20260709 12:13:10.616833 12676 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.662s	user 0.019s	sys 0.009s
I20260709 12:13:10.618369 12597 pending_rounds.cc:85] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 10
I20260709 12:13:10.620674 12597 pending_rounds.cc:107] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 15.11
I20260709 12:13:10.620986 12597 raft_consensus.cc:2889] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:10.581645 20064 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.030s
I20260709 12:13:10.616010 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 915) took 1192 ms. Trace:
I20260709 12:13:10.607470 19689 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:10.624557 12529 rpcz_store.cc:276] 0709 12:13:09.422061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.447303 (+ 25242us) service_pool.cc:224] Handling call
0709 12:13:09.447590 (+   287us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:09.447800 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:13:09.656224 (+208424us) log.cc:844] Serialized 14 byte log entry
0709 12:13:10.612978 (+956754us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:10.613038 (+    60us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:13:10.613097 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.614574 (+  1477us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:10.614776 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":331646,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":37,"raft.queue_time_us":74987,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":1151089,"threads_started":3,"wal-append.queue_time_us":802567}
W20260709 12:13:10.639335 13305 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.724s	user 0.017s	sys 0.005s
W20260709 12:13:10.641412 13305 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.726s	user 0.017s	sys 0.005s
W20260709 12:13:10.643748 12496 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.625s	user 0.001s	sys 0.021s
I20260709 12:13:10.585110 20203 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 20.17, Last appended by leader: 17, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:10.644372 12496 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.001s	sys 0.021s
I20260709 12:13:10.605414 20122 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:10.626296 20218 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:10.627524 20250 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:10.658066 20307 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.915s	user 0.005s	sys 0.007s
I20260709 12:13:10.658236 19982 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:10.660404 20122 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:10.610829 20215 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:10.656194 20310 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
I20260709 12:13:10.662038 20251 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:10.667121 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3519) took 1551 ms. Trace:
W20260709 12:13:10.666769 20313 log.cc:927] Time spent T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
W20260709 12:13:10.703392 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 385730us
W20260709 12:13:10.701913 20311 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.235s	user 0.006s	sys 0.000s
I20260709 12:13:10.699126 12605 rpcz_store.cc:276] 0709 12:13:09.115445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.793814 (+678369us) service_pool.cc:224] Handling call
0709 12:13:09.844768 (+ 50954us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:09.845221 (+   453us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:13:09.965132 (+119911us) log.cc:844] Serialized 14 byte log entry
0709 12:13:10.659785 (+694653us) spinlock_profiling.cc:243] Waited 6.89 ms on lock 0x7b1800341740. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f148c8f458f 00007f1485933571 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd
0709 12:13:10.665921 (+  6136us) log.cc:844] Serialized 14 byte log entry
0709 12:13:10.666464 (+   543us) log.cc:844] Serialized 14 byte log entry
0709 12:13:10.666640 (+   176us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:10.666679 (+    39us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:10.666736 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.666831 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:10.667064 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":761561,"prepare.queue_time_us":774275,"prepare.run_cpu_time_us":105,"prepare.run_wall_time_us":109,"raft.queue_time_us":63481,"raft.run_cpu_time_us":296,"raft.run_wall_time_us":6579,"spinlock_wait_cycles":15836032}
I20260709 12:13:10.666762 20277 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:10.672138 19842 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:10.681399 20209 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:10.702105 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50521.9us
I20260709 12:13:10.716682 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:10.724406 12643 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:10.716081 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:10.715866 20262 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.507s	user 0.009s	sys 0.005s
W20260709 12:13:10.730392 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 435106us
I20260709 12:13:10.730752 13334 raft_consensus.cc:2410] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:13:10.790194 12605 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 12:13:10.806603 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 90592.2us
I20260709 12:13:10.737251 20238 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:13:10.776473 20314 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.161s	user 0.000s	sys 0.001s
I20260709 12:13:10.811640 12496 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:10.815788 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 224305us
I20260709 12:13:10.713900 20081 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:10.821307 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 97026.3us
I20260709 12:13:10.825493 12643 leader_election.cc:400] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22.
I20260709 12:13:10.753510 20262 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
W20260709 12:13:10.810905 20311 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:13:10.826697 12599 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 17. Current term is 18. Ops: []
I20260709 12:13:10.846567 12572 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:10.847829 12606 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:10.848579 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3156) took 1397 ms. Trace:
I20260709 12:13:10.848965 12606 rpcz_store.cc:276] 0709 12:13:09.450883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.816818 (+365935us) service_pool.cc:224] Handling call
0709 12:13:10.847406 (+1030588us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:13:10.847625 (+   219us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:10.848326 (+   701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.848520 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4102016}
I20260709 12:13:10.850632 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:10.851969 12572 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:10.818511 20120 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:10.857146 12522 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
W20260709 12:13:10.859447 20170 consensus_meta.cc:287] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.648s	user 0.012s	sys 0.001s
I20260709 12:13:10.860049 20170 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21.
W20260709 12:13:10.860114 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48552.1us
I20260709 12:13:10.862319 20064 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:10.862748 20064 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:10.863133 20064 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:13:10.866828 20319 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.135s	user 0.001s	sys 0.000s
I20260709 12:13:10.867024 20251 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:10.779601 20243 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:10.871043 20243 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:10.856935 20262 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:13:10.873169 20128 consensus_meta.cc:287] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.554s	user 0.013s	sys 0.000s
I20260709 12:13:10.875392 20326 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.026s
I20260709 12:13:10.878608 20333 consensus_queue.cc:1059] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:13:10.877864 20081 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:10.861774 12496 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:10.884819 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:10.887030 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3132) took 1283 ms. Trace:
I20260709 12:13:10.880492 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3127) took 1472 ms. Trace:
I20260709 12:13:10.892308 12671 rpcz_store.cc:276] 0709 12:13:09.408196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.026847 (+618651us) service_pool.cc:224] Handling call
0709 12:13:10.027252 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:10.027543 (+   291us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:13:10.508438 (+480895us) log.cc:844] Serialized 14 byte log entry
0709 12:13:10.879934 (+371496us) log.cc:844] Serialized 14 byte log entry
0709 12:13:10.880099 (+   165us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:10.880154 (+    55us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:13:10.880216 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.880296 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:10.880426 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":622170,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":42,"raft.queue_time_us":69758,"raft.run_cpu_time_us":59,"raft.run_wall_time_us":60,"thread_start_us":833812,"threads_started":3}
I20260709 12:13:10.895123 12674 rpcz_store.cc:276] 0709 12:13:09.603531 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.257499 (+653968us) service_pool.cc:224] Handling call
0709 12:13:10.257819 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:10.258070 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:13:10.615145 (+357075us) log.cc:844] Serialized 14 byte log entry
0709 12:13:10.886555 (+271410us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:10.886638 (+    83us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:10.886707 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.886803 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:10.886956 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":27799,"prepare.queue_time_us":394104,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"thread_start_us":553966,"threads_started":2}
I20260709 12:13:10.935868 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:10.873649 20128 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:10.978475 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 154605us
I20260709 12:13:10.897078 20262 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:10.967643 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3124) took 1560 ms. Trace:
I20260709 12:13:10.984926 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:10.939114 20243 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:13:10.987397 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 125718us
I20260709 12:13:10.995697 12496 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:10.998212 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:10.988971 12676 rpcz_store.cc:276] 0709 12:13:09.407470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.789899 (+382429us) service_pool.cc:224] Handling call
0709 12:13:09.834266 (+ 44367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:09.834476 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:13:10.623073 (+788597us) log.cc:844] Serialized 14 byte log entry
0709 12:13:10.964212 (+341139us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:10.967294 (+  3082us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:10.967358 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.967429 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:10.967577 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":114677,"prepare.queue_time_us":793024,"prepare.run_cpu_time_us":68,"prepare.run_wall_time_us":4384,"raft.queue_time_us":91679,"raft.run_cpu_time_us":73,"raft.run_wall_time_us":73,"thread_start_us":1011882,"threads_started":3,"wal-append.queue_time_us":220086}
I20260709 12:13:11.006582 20250 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
I20260709 12:13:10.960978 20170 raft_consensus.cc:3055] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:13:11.023561 20170 raft_consensus.cc:740] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:11.024610 20170 consensus_queue.cc:260] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:11.025138 20128 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:10.999349 19982 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 12:13:10.985499 20265 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:11.033268 20265 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:11.033388 20243 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:11.034546 20243 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:11.034462 20265 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:11.035988 20265 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:11.039422 12529 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: []
W20260709 12:13:11.040827 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 45215.2us
I20260709 12:13:11.048908 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:11.049985 13334 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 21: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:13:11.051975 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:11.054276 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:11.054971 12531 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:11.059494 12604 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:11.064530 13331 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:11.064046 12425 catalog_manager.cc:5697] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 21 to 24, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 24 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:11.056734 19539 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:11.066111 12425 catalog_manager.cc:5697] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 13 to 16, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 16 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:11.066581 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:11.070466 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 170146us
I20260709 12:13:11.073985 12602 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 21: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:11.067428 12603 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
I20260709 12:13:11.069198 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:11.076716 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:11.077371 12606 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 15.
I20260709 12:13:11.078158 12603 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:13:11.080261 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3121) took 1753 ms. Trace:
I20260709 12:13:11.086396 12531 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
I20260709 12:13:11.088873 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:11.081077 13334 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 18: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:11.083467 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 821) took 1122 ms. Trace:
I20260709 12:13:11.090200 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:11.094902 12605 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:13:11.090591 12675 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:13:11.091333 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:11.091269 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:11.096652 12606 raft_consensus.cc:2368] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20.
I20260709 12:13:11.096663 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:11.098006 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:11.095592 13305 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:11.099740 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:11.106190 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:11.107718 12676 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:13:11.113047 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 3051) took 1277 ms. Trace:
I20260709 12:13:11.085666 12677 rpcz_store.cc:276] 0709 12:13:09.326812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.716924 (+390112us) service_pool.cc:224] Handling call
0709 12:13:09.717551 (+   627us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:09.717772 (+   221us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:13:10.486247 (+768475us) log.cc:844] Serialized 14 byte log entry
0709 12:13:10.819377 (+333130us) log.cc:844] Serialized 14 byte log entry
0709 12:13:11.062902 (+243525us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:11.062990 (+    88us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:11.079981 (+ 16991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:11.080057 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:11.080193 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64471,"prepare.queue_time_us":943429,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":87,"raft.queue_time_us":167289,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":41699,"thread_start_us":1038574,"threads_started":3,"wal-append.queue_time_us":477565}
I20260709 12:13:11.114537 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:11.094444 12599 rpcz_store.cc:276] 0709 12:13:09.961519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.970172 (+  8653us) service_pool.cc:224] Handling call
0709 12:13:09.970539 (+   367us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:10.842231 (+871692us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:11.083216 (+240985us) spinlock_profiling.cc:243] Waited 856 ms on lock 0x7b54001171d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:11.083401 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1968492288}
I20260709 12:13:11.118247 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:11.120289 12531 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:11.113517 12526 rpcz_store.cc:276] 0709 12:13:09.835633 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.921299 (+ 85666us) service_pool.cc:224] Handling call
0709 12:13:11.112885 (+1191586us) spinlock_profiling.cc:243] Waited 398 ms on lock 0x7b540027fbd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:11.112992 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":915175168}
I20260709 12:13:11.122802 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:11.124416 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:11.132527 12528 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 3013) took 4433 ms (client timeout 1725 ms). Trace:
I20260709 12:13:11.132596 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:11.133262 12528 rpcz_store.cc:269] 0709 12:13:06.679503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:06.696229 (+ 16726us) service_pool.cc:224] Handling call
0709 12:13:11.112494 (+4416265us) spinlock_profiling.cc:243] Waited 3.06 s on lock 0x7b54003cb4d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:11.112636 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":7034491264}
I20260709 12:13:11.080039 19539 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:11.149291 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 3049) took 1315 ms. Trace:
I20260709 12:13:11.150605 12527 rpcz_store.cc:276] 0709 12:13:09.833674 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.955092 (+121418us) service_pool.cc:224] Handling call
0709 12:13:11.149118 (+1194026us) spinlock_profiling.cc:243] Waited 367 ms on lock 0x7b5400052b54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:11.149230 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":844355456}
I20260709 12:13:11.154673 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:11.160190 12679 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 19: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:13:11.142254 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 62988.2us
I20260709 12:13:11.161350 12494 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:11.154670 20065 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:13:11.122988 20262 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.178507 20262 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:11.180260 20264 raft_consensus.cc:3055] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:13:11.180742 20264 raft_consensus.cc:740] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:11.182550 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:11.183290 12526 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:11.122885 20122 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.171712 20065 raft_consensus.cc:697] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:11.190968 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:11.191682 12599 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:11.195329 12531 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:11.106159 20326 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.062s
I20260709 12:13:11.106765 20203 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.066s
W20260709 12:13:11.204602 13304 leader_election.cc:336] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.501s (SENT)
I20260709 12:13:11.154541 20354 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:11.209600 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:11.211594 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:11.216603 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:11.218029 12531 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:11.220062 12531 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
W20260709 12:13:11.224131 13304 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.634s (SENT) [suppressed 17 similar messages]
I20260709 12:13:11.180989 20262 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:11.263901 12671 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:11.265408 12531 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 17 index: 12. (index mismatch)
I20260709 12:13:11.198508 19689 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:11.275305 12671 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:11.279500 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:11.281172 12599 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:11.291831 12526 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:11.392701 12679 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 19 index: 10. Preceding OpId from leader: term: 21 index: 11. (index mismatch)
W20260709 12:13:11.397045 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 154383us
I20260709 12:13:11.160487 19539 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:11.404428 12604 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:13:11.181669 20264 consensus_queue.cc:260] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:11.421108 12603 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:13:11.271456 20206 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:11.418025 12671 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:11.205566 20122 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:11.429322 12671 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:11.439388 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:11.425535 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 330068us
W20260709 12:13:11.440928 12646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.549s	user 0.083s	sys 0.176s
W20260709 12:13:11.441792 12646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.083s	sys 0.176s
I20260709 12:13:11.438913 20065 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:11.459100 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 650060us
W20260709 12:13:11.459834 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:11.460517 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:11.407629 19689 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:11.461356 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 386791us
I20260709 12:13:11.422377 20340 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:11.464003 12494 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:11.437044 20264 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:13:11.458020 20064 consensus_meta.cc:287] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.594s	user 0.014s	sys 0.000s
I20260709 12:13:11.273427 20329 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.187s
I20260709 12:13:11.479031 12671 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
W20260709 12:13:11.553925 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:11.583384 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 119435us
I20260709 12:13:11.585119 12494 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:11.488090 20354 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:11.490458 20139 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.174s
I20260709 12:13:11.483415 20293 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.170s
I20260709 12:13:11.669770 20206 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:11.673032 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 88002.7us
I20260709 12:13:11.465302 20349 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.163s
I20260709 12:13:11.680799 12494 leader_election.cc:400] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20.
I20260709 12:13:11.681267 12494 leader_election.cc:403] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Cancelling election due to peer responding with higher term
I20260709 12:13:11.487690 20064 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:11.660566 20342 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Leader pre-election won for term 16
I20260709 12:13:11.670743 20206 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:11.708266 19539 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:11.692180 19689 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:11.691728 20250 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:11.664434 20326 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.037s
W20260709 12:13:11.702262 20238 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.877s	user 0.010s	sys 0.002s
I20260709 12:13:11.698285 20342 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:11.752841 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 78950.2us
W20260709 12:13:11.725462 20317 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.546s	user 0.005s	sys 0.019s
I20260709 12:13:11.725345 20250 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:11.679450 20354 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:11.779867 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 313717us
W20260709 12:13:11.172055 20351 consensus_queue.cc:1175] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" responder_term: 19 status { last_received { term: 16 index: 10 } last_committed_idx: 10 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: [18.11-18.11]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:13:11.752750 20342 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:11.754251 20238 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election lost for term 17. Reason: could not achieve majority
I20260709 12:13:11.759200 20325 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:13:11.802031 12523 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 3054) took 1811 ms (client timeout 1872 ms). Trace:
W20260709 12:13:11.803817 12523 rpcz_store.cc:269] 0709 12:13:09.991105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.284171 (+293066us) service_pool.cc:224] Handling call
0709 12:13:11.801806 (+1517635us) spinlock_profiling.cc:243] Waited 165 ms on lock 0x7b540038f9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:11.801957 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":378951552}
W20260709 12:13:11.787634 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 309787us
I20260709 12:13:11.805940 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:11.806356 12494 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:11.778404 12678 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s	user 0.017s	sys 0.012s
W20260709 12:13:11.791364 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:11.742415 20215 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:11.871809 20353 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:11.872534 20353 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:11.818608 20336 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.359s	user 0.000s	sys 0.003s
W20260709 12:13:11.775820 20317 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.580s	user 0.005s	sys 0.020s
W20260709 12:13:11.671475 20243 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.634s	user 0.001s	sys 0.019s
W20260709 12:13:11.876744 20243 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.839s	user 0.002s	sys 0.020s
I20260709 12:13:11.877825 20352 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:11.878567 20352 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:11.881209 20352 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:11.880228 12425 catalog_manager.cc:5697] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 15 to 19. New cstate: current_term: 19 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:13:11.808360 12678 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s	user 0.017s	sys 0.013s
I20260709 12:13:11.811842 20359 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:11.813727 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3048) took 1980 ms. Trace:
W20260709 12:13:11.814019 20350 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:13:11.815480 20120 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:11.816083 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 825) took 1202 ms. Trace:
I20260709 12:13:11.816622 12680 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 17.
I20260709 12:13:11.816923 12523 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 16. Current term is 17. Ops: [16.9-16.9]
I20260709 12:13:11.818506 12675 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 18 index: 12. (index mismatch)
I20260709 12:13:11.797912 20349 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.029s
I20260709 12:13:11.822507 12646 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:11.825026 20358 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:13:11.827463 20262 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:11.835175 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:11.801201 20265 consensus_meta.cc:287] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.765s	user 0.017s	sys 0.000s
I20260709 12:13:11.835772 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:11.842947 13331 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:13:11.846268 20122 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:11.749266 20329 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
W20260709 12:13:11.848982 12522 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 3047) took 2015 ms (client timeout 2182 ms). Trace:
W20260709 12:13:11.929352 12522 rpcz_store.cc:269] 0709 12:13:09.833232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.382915 (+549683us) service_pool.cc:224] Handling call
0709 12:13:11.848715 (+1465800us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b54003ca5d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:11.848899 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1065458432}
I20260709 12:13:11.806869 12674 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
W20260709 12:13:11.930092 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 107689us
I20260709 12:13:11.788383 20250 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:11.859916 12496 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:11.862385 20064 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:11.845902 20206 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:11.866526 20139 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:13:11.868420 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:11.875020 20353 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:11.885061 12425 catalog_manager.cc:5697] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 15 to 16, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 16 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:11.908833 20120 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:11.897209 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:11.917304 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:11.908656 12597 rpcz_store.cc:276] 0709 12:13:10.613558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.613707 (+   149us) service_pool.cc:224] Handling call
0709 12:13:10.618083 (+  4376us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:10.635285 (+ 17202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:10.635318 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:10.645350 (+ 10032us) log.cc:844] Serialized 52 byte log entry
0709 12:13:10.809698 (+164348us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:13:10.809765 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.809879 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:11.815817 (+1005938us) raft_consensus.cc:1685] finished
0709 12:13:11.815842 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:11.816014 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":311,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":6712,"thread_start_us":146507,"threads_started":1,"wal-append.queue_time_us":1093018}
I20260709 12:13:11.922727 13338 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:11.852255 20354 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:11.896347 12525 rpcz_store.cc:276] 0709 12:13:09.833461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.527715 (+694254us) service_pool.cc:224] Handling call
0709 12:13:10.528276 (+   561us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:10.528812 (+   536us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:10.528864 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:10.529564 (+   700us) log.cc:844] Serialized 26 byte log entry
0709 12:13:10.842615 (+313051us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:10.842671 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:10.842761 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:11.813418 (+970657us) raft_consensus.cc:1685] finished
0709 12:13:11.813448 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:11.813641 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":140,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":127,"thread_start_us":312587,"threads_started":1,"wal-append.queue_time_us":1167316}
I20260709 12:13:11.899928 20128 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:11.935017 12531 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:13:11.896000 20133 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:11.896226 20352 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:13:11.896839 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 90609.7us
W20260709 12:13:11.944737 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 84897.1us
I20260709 12:13:11.956504 12496 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:11.946331 12646 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:11.951215 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 61398.1us
I20260709 12:13:11.957652 20354 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:11.935000 20238 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:11.965191 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:11.985733 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 46259.5us
I20260709 12:13:11.952690 20122 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:11.965484 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:11.972891 20079 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:11.952885 20349 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:11.932812 20262 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:11.978046 20326 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:11.963514 20369 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.019s
I20260709 12:13:12.009429 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:11.998919 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:11.934412 20351 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 12:13:12.004765 12604 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:13:12.005759 20264 consensus_meta.cc:287] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.521s	user 0.008s	sys 0.005s
I20260709 12:13:12.011462 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:12.001161 20206 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:11.900362 20359 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:11.983362 12646 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:11.966653 20128 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:11.955394 20353 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:13:12.038904 12494 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:11.981316 20340 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:12.016011 12606 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:12.007854 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:12.021723 12601 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:13:12.000805 20265 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:12.027659 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:12.029240 12645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.734s	user 0.019s	sys 0.028s
W20260709 12:13:12.044305 12645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.768s	user 0.020s	sys 0.028s
W20260709 12:13:12.012969 13304 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.003s	sys 0.028s
I20260709 12:13:12.044703 12672 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
W20260709 12:13:12.044780 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:11.966430 20064 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.049s
I20260709 12:13:11.961004 20133 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:12.061628 20133 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:12.062222 20265 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:12.065073 20265 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:12.043251 20329 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:12.066504 20293 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:12.067271 20293 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:12.069612 20293 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:12.027293 20349 raft_consensus.cc:2764] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:13:12.070044 20100 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:12.074393 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:12.074594 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3543) took 1016 ms. Trace:
I20260709 12:13:12.075069 12600 rpcz_store.cc:276] 0709 12:13:11.058503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:11.060932 (+  2429us) service_pool.cc:224] Handling call
0709 12:13:11.061451 (+   519us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:11.079099 (+ 17648us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:11.079141 (+    42us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:11.090155 (+ 11014us) log.cc:844] Serialized 52 byte log entry
0709 12:13:11.433274 (+343119us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:11.449063 (+ 15789us) log.cc:844] Serialized 14 byte log entry
0709 12:13:11.449671 (+   608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:11.449780 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:12.074359 (+624579us) raft_consensus.cc:1685] finished
0709 12:13:12.074381 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:12.074535 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17082,"prepare.queue_time_us":14923,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":56,"raft.queue_time_us":17350,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":16844,"thread_start_us":277775,"threads_started":1,"wal-append.queue_time_us":548808}
I20260709 12:13:12.079180 12604 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 22 index: 18. (index mismatch)
I20260709 12:13:12.079413 20262 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:12.080022 20064 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:12.006672 20238 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:12.011519 20354 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:12.083081 20133 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:12.033978 12496 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:12.083631 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:11.965492 20352 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:12.045218 12672 pending_rounds.cc:85] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 11
I20260709 12:13:12.088058 12672 pending_rounds.cc:107] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 12:13:12.088450 12672 raft_consensus.cc:2889] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:13:12.088594 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54408.1us
I20260709 12:13:12.030313 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:12.089850 12496 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:12.045017 13304 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.003s	sys 0.028s
W20260709 12:13:12.090611 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 866457us
I20260709 12:13:12.091019 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:12.093416 13304 leader_election.cc:336] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.850s (SENT)
I20260709 12:13:12.048524 20128 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:12.055797 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3152) took 1121 ms. Trace:
I20260709 12:13:12.055590 20286 consensus_queue.cc:1059] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 17.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:13:12.057339 20243 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:12.080506 20064 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:12.099501 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:12.095536 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:12.038264 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 54988.4us
W20260709 12:13:12.103452 12524 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.672s	user 0.006s	sys 0.015s
I20260709 12:13:12.103369 12678 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:13:12.103909 12646 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:12.046979 20122 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:12.018453 20209 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.068s
I20260709 12:13:12.088415 19689 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:12.102072 12675 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:13:12.119480 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:12.096179 12529 rpcz_store.cc:276] 0709 12:13:10.934563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.934875 (+   312us) service_pool.cc:224] Handling call
0709 12:13:10.961676 (+ 26801us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:11.050716 (+ 89040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:12.055521 (+1004805us) spinlock_profiling.cc:243] Waited 77.5 ms on lock 0x7b540038ead4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:12.055718 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":178242304}
I20260709 12:13:12.100405 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:12.083817 20333 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:12.102953 12569 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:12.083657 20133 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:12.126237 20264 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:12.125847 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:12.106730 20064 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:12.126876 20264 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:12.139532 12529 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: [21.11-21.11]
W20260709 12:13:12.141410 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:12.115942 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:12.119457 20353 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:12.094096 13304 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:12.042721 20206 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:12.151671 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:12.116506 12646 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:12.120697 12674 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:12.098874 19618 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:12.093792 20329 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:12.105821 20265 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:12.095741 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:12.098773 20238 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:12.133386 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:12.112116 20065 raft_consensus.cc:3055] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:13:12.134582 12530 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 916) took 2712 ms (client timeout 2753 ms). Trace:
W20260709 12:13:12.103943 12524 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.672s	user 0.006s	sys 0.015s
W20260709 12:13:12.137703 12572 connection.cc:256] RPC call timeout handler was delayed by -1.2123768060000657s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.202s for a response
I20260709 12:13:12.106366 20243 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:12.018533 20120 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.18, Last appended by leader: 18, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:12.157238 20354 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:13:12.155936 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 62598.6us
I20260709 12:13:12.157262 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:12.162460 12530 rpcz_store.cc:269] 0709 12:13:09.422435 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:09.444845 (+ 22410us) service_pool.cc:224] Handling call
0709 12:13:12.134363 (+2689518us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54003cb9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:12.134516 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2665566464}
I20260709 12:13:12.146031 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:12.163004 12572 connection.cc:256] RPC call timeout handler was delayed by -1.175443641999891s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18720000000000003s for a response
W20260709 12:13:12.166874 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 71240.4us
I20260709 12:13:12.161505 20265 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:12.149989 12679 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:12.188194 12531 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:13:12.197067 12529 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
W20260709 12:13:12.200946 12572 connection.cc:256] RPC call timeout handler was delayed by -1.1331768059999376s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21080000000000002s for a response
W20260709 12:13:12.202505 12572 connection.cc:256] RPC call timeout handler was delayed by -1.0665768060000573s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2182s for a response
I20260709 12:13:12.196934 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:12.129141 20333 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:12.194072 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:12.162286 20286 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
I20260709 12:13:12.132644 20272 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.026s
W20260709 12:13:12.203984 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:12.167547 20065 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:12.141026 19689 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.054s
I20260709 12:13:12.194813 20202 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.018s
I20260709 12:13:12.214620 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:12.161751 19827 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:13:12.161375 20215 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 12:13:12.157648 20341 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 12:13:12.167214 20369 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 12:13:12.205592 20265 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:12.223560 12601 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:13:12.199079 20238 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:12.215063 20352 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:12.227898 12599 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:12.233651 12674 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:12.217891 20359 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:12.248147 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3637) took 1160 ms. Trace:
I20260709 12:13:12.285472 12672 rpcz_store.cc:276] 0709 12:13:11.087522 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:11.089321 (+  1799us) service_pool.cc:224] Handling call
0709 12:13:11.089650 (+   329us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:12.089332 (+999682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:12.247843 (+158511us) spinlock_profiling.cc:243] Waited 954 ms on lock 0x7b540008e154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:12.248073 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2195312000}
W20260709 12:13:12.286495 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 87743.6us
I20260709 12:13:12.260268 19827 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:13:12.249890 13304 leader_election.cc:336] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.068s (SENT)
W20260709 12:13:12.257040 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 291628us
I20260709 12:13:12.230173 20354 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:13:12.299543 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49685.1us
I20260709 12:13:12.270583 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:12.220021 20264 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:12.273160 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 119690us
I20260709 12:13:12.224926 20202 consensus_queue.cc:1243] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 22.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:13:12.235268 20238 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:12.273265 20352 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:13:12.300383 13304 leader_election.cc:336] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.214s (SENT)
I20260709 12:13:12.304417 12643 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:12.312857 13304 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:12.290041 19827 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:13:12.321439 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:12.303308 20376 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.197s	user 0.000s	sys 0.002s
I20260709 12:13:12.333333 13332 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 15. (term mismatch)
I20260709 12:13:12.333940 13332 pending_rounds.cc:85] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 14
I20260709 12:13:12.334244 13332 pending_rounds.cc:107] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 20.15
I20260709 12:13:12.334546 13332 raft_consensus.cc:2889] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:12.298573 20354 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:12.252174 20329 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:12.365938 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:12.255071 20065 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:12.321547 20133 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:12.376452 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:12.312506 20352 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:12.379436 13331 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:12.392050 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:12.392834 12672 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:13:12.312263 20359 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:13:12.399319 20286 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:12.403721 12678 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:13:12.410440 19827 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:12.456471 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:12.416638 20206 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:12.456552 20286 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:12.418856 20383 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:13:12.392741 20317 consensus_queue.cc:1059] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 15.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 12:13:12.423786 20238 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:12.427979 20203 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:12.428942 20333 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:12.432019 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 3643) took 1340 ms. Trace:
I20260709 12:13:12.459138 20286 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:12.459216 12680 rpcz_store.cc:276] 0709 12:13:11.091968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:11.105821 (+ 13853us) service_pool.cc:224] Handling call
0709 12:13:12.431807 (+1325986us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b54003d4ad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:12.431949 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1631247104}
I20260709 12:13:12.460083 20349 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:12.434259 20133 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:13:12.438807 20380 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
W20260709 12:13:12.444995 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:12.446975 20342 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.618s	user 0.013s	sys 0.000s
I20260709 12:13:12.448248 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 925) took 1446 ms. Trace:
I20260709 12:13:12.479894 12523 rpcz_store.cc:276] 0709 12:13:11.001912 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:11.928657 (+926745us) service_pool.cc:224] Handling call
0709 12:13:11.929073 (+   416us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:11.934826 (+  5753us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:11.934858 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:11.943156 (+  8298us) log.cc:844] Serialized 26 byte log entry
0709 12:13:12.248717 (+305561us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:12.248765 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:12.248853 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:12.447927 (+199074us) raft_consensus.cc:1685] finished
0709 12:13:12.447960 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:12.448162 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7923,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":7975,"thread_start_us":241050,"threads_started":1,"wal-append.queue_time_us":317973}
W20260709 12:13:12.449409 20325 consensus_meta.cc:287] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.613s	user 0.013s	sys 0.001s
I20260709 12:13:12.343024 20385 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:13:12.450985 20388 log.cc:927] Time spent T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:13:12.454516 20372 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:13:12.454320 20064 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:12.464133 20243 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:12.453629 20375 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:13:12.461637 20133 raft_consensus.cc:3055] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:13:12.470855 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:12.493176 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 169378us
W20260709 12:13:12.500759 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 116775us
I20260709 12:13:12.501087 12572 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:12.502696 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:12.503988 20206 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:12.504448 20206 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:12.504822 20366 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:12.497929 20133 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:12.506546 20317 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:12.507570 20133 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:12.509052 20133 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:13:12.509869 20346 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.115s	user 0.007s	sys 0.004s
I20260709 12:13:12.493464 20325 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election lost for term 19. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20.
I20260709 12:13:12.504873 20206 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:12.515807 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:12.516556 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:12.519068 12678 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
I20260709 12:13:12.516759 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:12.516490 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:12.519400 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:12.520476 12672 raft_consensus.cc:2376] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24. Re-sending same reply.
I20260709 12:13:12.517371 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:12.517834 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 215334us
I20260709 12:13:12.521581 12675 raft_consensus.cc:2376] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18. Re-sending same reply.
I20260709 12:13:12.491477 20385 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:12.525813 12674 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 27.
I20260709 12:13:12.521697 13304 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:12.527160 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:12.534212 20365 log.cc:927] Time spent T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.546s	user 0.002s	sys 0.001s
I20260709 12:13:12.537397 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3619) took 2716 ms. Trace:
I20260709 12:13:12.540949 12673 rpcz_store.cc:276] 0709 12:13:09.820847 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:10.309208 (+488361us) service_pool.cc:224] Handling call
0709 12:13:10.309530 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:10.452721 (+143191us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:13:10.742275 (+289554us) log.cc:844] Serialized 14 byte log entry
0709 12:13:10.985680 (+243405us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:10.985745 (+    65us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:10.985796 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:12.537026 (+1551230us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b540016f3d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:12.537096 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:12.537275 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":693,"prepare.queue_time_us":212426,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":421535,"raft.run_cpu_time_us":170,"raft.run_wall_time_us":7063,"spinlock_wait_cycles":328417536,"thread_start_us":480169,"threads_started":2,"wal-append.queue_time_us":961563}
I20260709 12:13:12.496430 20333 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:12.543843 20333 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:12.547042 20333 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:12.548885 20333 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:12.554544 20320 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:12.555202 20320 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:12.555682 20320 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:12.570719 20265 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:12.572264 20064 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:12.573199 20265 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:12.573135 20385 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:12.493695 20342 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:12.574659 20250 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.631s	user 0.009s	sys 0.006s
I20260709 12:13:12.576962 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:12.584436 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:12.595813 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:12.576783 20250 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:12.596515 20206 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:12.598321 20250 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:12.584362 20262 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:12.583527 20408 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:13:12.599741 20408 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:12.600170 20408 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:12.221264 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:12.604049 20206 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:12.608500 20122 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:12.607949 12425 catalog_manager.cc:5697] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 15 to 22, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 22 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:12.611121 20342 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:12.620585 19827 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:12.627633 20170 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:13:12.644573 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:12.645352 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:12.647295 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 62941.8us
I20260709 12:13:12.649358 12679 raft_consensus.cc:3055] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:13:12.649799 12679 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:12.650662 12679 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:12.663311 12679 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:12.625478 20408 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:12.672906 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:12.651809 20357 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:12.692306 20370 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:12.692993 20370 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:12.700242 20385 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:12.702934 20385 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:12.682790 20369 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:12.711313 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 407344us
I20260709 12:13:12.706491 12676 raft_consensus.cc:2393] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 22: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:12.707028 20262 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:12.694087 20370 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:12.710376 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3065) took 1735 ms. Trace:
I20260709 12:13:12.711282 20326 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:13:12.712648 20373 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.705s	user 0.009s	sys 0.000s
I20260709 12:13:12.703855 12425 catalog_manager.cc:5697] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 13 to 17, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 17 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:13:12.720356 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:12.722415 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:12.722998 12528 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:13:12.723762 20128 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:12.727346 12527 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 21. Current term is 22. Ops: []
W20260709 12:13:12.744717 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:12.744867 12679 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 19 index: 14. (term mismatch)
I20260709 12:13:12.746187 12679 pending_rounds.cc:85] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 13
I20260709 12:13:12.746621 12679 pending_rounds.cc:107] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 18.14
I20260709 12:13:12.747090 12679 raft_consensus.cc:2889] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:12.720738 12526 rpcz_store.cc:276] 0709 12:13:10.975375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:11.294192 (+318817us) service_pool.cc:224] Handling call
0709 12:13:11.295224 (+  1032us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:11.439617 (+144393us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:13:11.725353 (+285736us) log.cc:844] Serialized 14 byte log entry
0709 12:13:12.278023 (+552670us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:12.290859 (+ 12836us) log.cc:844] Serialized 26 byte log entry
0709 12:13:12.291151 (+   292us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:13:12.291461 (+   310us) log.cc:844] Serialized 14 byte log entry
0709 12:13:12.709628 (+418167us) spinlock_profiling.cc:243] Waited 3.59 ms on lock 0x7b18002fe120. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:13:12.709861 (+   233us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:12.709950 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:12.710085 (+   135us) raft_consensus.cc:1685] finished
0709 12:13:12.710102 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:12.710279 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":283619,"prepare.queue_time_us":212826,"prepare.run_cpu_time_us":77,"prepare.run_wall_time_us":79,"raft.queue_time_us":689655,"raft.run_cpu_time_us":1471,"raft.run_wall_time_us":30594,"spinlock_wait_cycles":9238656,"thread_start_us":626331,"threads_started":2,"wal-append.queue_time_us":504599}
I20260709 12:13:12.721133 20262 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:12.761265 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:12.762046 12525 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
W20260709 12:13:12.762209 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:12.773191 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52156.6us
I20260709 12:13:12.773945 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:12.776872 13305 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:12.769534 20370 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:12.720706 20369 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:12.789516 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:12.766453 20128 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:12.771023 20317 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.015s
I20260709 12:13:12.798871 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:12.781982 20331 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:12.804188 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:12.804896 12604 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:12.805403 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:12.806600 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:12.801543 20331 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:12.810469 20331 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:12.780601 20375 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.016s
I20260709 12:13:12.819875 12675 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:12.820457 12675 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:12.821173 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:12.813158 12597 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:12.822710 12672 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:12.813506 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3063) took 1839 ms. Trace:
I20260709 12:13:12.814961 12605 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:12.825467 12524 rpcz_store.cc:276] 0709 12:13:10.974613 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:11.272101 (+297488us) service_pool.cc:224] Handling call
0709 12:13:11.272494 (+   393us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:11.396534 (+124040us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:13:12.162668 (+766134us) log.cc:844] Serialized 14 byte log entry
0709 12:13:12.711483 (+548815us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:12.712068 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 12:13:12.712235 (+   167us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:12.712279 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:12.712355 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:12.813208 (+100853us) raft_consensus.cc:1685] finished
0709 12:13:12.813238 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:12.813420 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":561505,"prepare.queue_time_us":273308,"prepare.run_cpu_time_us":26,"prepare.run_wall_time_us":27,"raft.queue_time_us":530121,"raft.run_cpu_time_us":259,"raft.run_wall_time_us":21648,"thread_start_us":823484,"threads_started":2,"wal-append.queue_time_us":196811}
I20260709 12:13:12.828459 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:12.835480 20374 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:12.836462 20379 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:13:12.839291 20128 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:12.844676 12604 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:12.845144 12677 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:13:12.821301 12675 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:12.851593 12599 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:13:12.851943 12675 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:12.787492 20320 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:12.781741 20409 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 12:13:12.863518 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 16 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:12.864264 12673 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:12.867542 12672 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:12.870273 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:12.873512 12673 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:13:12.874922 20408 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:12.875938 20100 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:12.785315 20420 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.071s
I20260709 12:13:12.894253 20100 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:12.896416 20100 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:12.814720 20133 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:12.897554 13331 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:13:12.897321 20133 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:12.899523 20133 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:12.899935 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:12.901448 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:12.902740 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:12.904624 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:12.905921 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:12.910295 20354 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:12.911916 20354 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:12.918069 20351 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:13:12.919837 20351 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:13:12.920789 20351 raft_consensus.cc:3055] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:12.921119 20351 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:13:12.923326 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 143754us
I20260709 12:13:12.864280 12675 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 12. (term mismatch)
I20260709 12:13:12.876888 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:12.877532 20250 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:12.877457 20340 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:12.878746 20065 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:12.884704 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:12.937076 12673 raft_consensus.cc:2410] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:13:12.890120 20122 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:12.939365 20122 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:12.939845 20122 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:12.891542 20120 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:12.893639 20369 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:12.926612 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49805.2us
I20260709 12:13:12.928596 12523 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:12.948408 12496 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:12.889043 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:12.954502 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 65542us
I20260709 12:13:12.942922 12675 pending_rounds.cc:85] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 11
I20260709 12:13:12.957058 12675 pending_rounds.cc:107] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 12:13:12.928603 20317 raft_consensus.cc:3055] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:12.940723 20122 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:12.957589 12675 raft_consensus.cc:2889] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:12.929646 20379 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:13:12.930194 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:12.939407 20320 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:12.963433 20218 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:12.934158 20340 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:13.015055 13334 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:12.946144 20215 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:12.966423 20081 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:12.965224 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:13.033764 12673 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:13:12.965396 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 443643us
W20260709 12:13:13.034662 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 86387.6us
W20260709 12:13:12.965265 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 464231us
I20260709 12:13:13.035511 12496 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 16 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:12.965601 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:12.966334 20265 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:12.967360 20341 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:12.921958 20351 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:13.022723 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:12.959275 20317 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:12.934857 20250 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:13.029690 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:13.054318 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:13.055236 12598 raft_consensus.cc:2410] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 23 because replica has last-logged OpId of term: 21 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:13:13.055377 20081 raft_consensus.cc:697] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:13.055761 20250 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:13.056358 20081 consensus_queue.cc:237] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:13.057937 20352 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:13.059101 20352 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:13.059224 12676 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 17 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:13:13.031512 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:13.063720 20341 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:13.047271 13331 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:13:13.058504 12680 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:13.069945 12523 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 19 index: 14. (index mismatch)
I20260709 12:13:13.075515 20203 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:13:13.078748 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:13.030197 12597 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:13:13.079807 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:13.081476 12675 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:13:13.083112 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:13.084748 12673 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 28.
I20260709 12:13:13.065160 20341 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:13.087076 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:13.082219 12496 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:13.093806 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:13.039724 20265 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:13.104166 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:13.042040 20327 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader election lost for term 16. Reason: could not achieve majority
I20260709 12:13:13.086932 20122 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:13.038506 20215 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:13.089967 20366 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:13.124164 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:13.124317 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:13.126046 12600 raft_consensus.cc:2410] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 18 because replica has last-logged OpId of term: 16 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 8.
I20260709 12:13:13.101421 20202 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:13:13.046680 20422 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.080s
I20260709 12:13:13.102229 20351 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:13.118856 20286 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:13:13.105105 19851 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:13.101032 20349 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:13.115651 20264 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:13.097296 20100 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:13.160022 20100 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:13.094218 20379 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.033s
I20260709 12:13:13.163108 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:13.163822 12679 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:13:13.163966 20218 raft_consensus.cc:3055] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:13.164613 20218 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:13.097733 20382 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:13:13.147902 20202 raft_consensus.cc:697] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:13.166994 20202 consensus_queue.cc:237] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:13.169480 20349 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:13.150942 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:13.171981 20349 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:13.172562 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:13.117445 20420 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.038s
I20260709 12:13:13.174360 20250 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:13.174865 20250 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:13.175308 20250 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:13.176527 20215 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:13:13.123219 20326 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 12:13:13.165901 19539 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.014s
I20260709 12:13:13.152207 20265 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:13.150998 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 15 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:13.184636 12673 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 12:13:13.185081 12673 pending_rounds.cc:85] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 13
I20260709 12:13:13.185329 12673 pending_rounds.cc:107] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 12:13:13.185583 12673 raft_consensus.cc:2889] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:13.191366 20264 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:13.192248 20264 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:13.098611 20262 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:13.193953 20390 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:13:13.194723 20262 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:13.157270 12494 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:13.196813 20128 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:13.199790 12527 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: [21.14-21.14]
I20260709 12:13:13.199978 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:13.200304 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:13.203135 20286 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:13.127753 20272 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:13.127524 20327 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.078s
I20260709 12:13:13.206907 12528 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: []
I20260709 12:13:13.202323 20317 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:13.208482 20317 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:13.211220 12597 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:13.202930 12529 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
W20260709 12:13:13.207108 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:13.216502 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:13.223546 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 66387.4us
I20260709 12:13:13.205308 12425 catalog_manager.cc:5697] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 13 to 16, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 16 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:13.236815 12529 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: []
W20260709 12:13:13.239442 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:13.242066 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:13.250007 12522 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 21 index: 18. Preceding OpId from leader: term: 22 index: 18. (term mismatch)
I20260709 12:13:13.250517 12522 pending_rounds.cc:85] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 17
I20260709 12:13:13.250758 12522 pending_rounds.cc:107] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 21.18
I20260709 12:13:13.251335 12522 raft_consensus.cc:2889] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:13.257670 12494 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:13.258486 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:13.260061 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:13.260735 12530 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:13:13.254626 19827 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.261431 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:13.261794 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:13.262445 20215 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:13.263022 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:13.262995 20215 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:13.264057 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:13.264673 12530 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 23. Current term is 24. Ops: [22.13-23.14]
I20260709 12:13:13.265470 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:13.265431 20215 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:13.261679 19827 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:13.267244 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3654) took 1812 ms. Trace:
I20260709 12:13:13.267656 12677 rpcz_store.cc:276] 0709 12:13:11.454848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:11.458387 (+  3539us) service_pool.cc:224] Handling call
0709 12:13:11.458805 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:12.845646 (+1386841us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:13.267010 (+421364us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54000ac8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:13.267186 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3187147136}
I20260709 12:13:13.268734 19827 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:13.278439 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:13.280628 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:13.300750 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:13.300750 13332 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
W20260709 12:13:13.302976 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:13.306258 12528 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:13:13.257349 20349 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:13.312661 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:13.283411 19539 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:13.315737 12527 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:13.318315 12675 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22.
I20260709 12:13:13.342188 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:13.345818 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:13.346870 12523 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:13.349555 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:13.357894 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:13.343932 13304 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:13.362761 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:13.363828 12530 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 22: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:13.316627 20354 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.365305 12680 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 12:13:13.365904 12680 pending_rounds.cc:85] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 13
I20260709 12:13:13.365423 20354 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:13.366225 12680 pending_rounds.cc:107] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 21.14
I20260709 12:13:13.366542 12680 raft_consensus.cc:2889] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:13:13.357923 20417 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:13:13.421419 12529 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:13.350404 20366 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:13.349498 20349 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:13.433647 20349 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:13.373085 20100 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:13.448637 12646 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:13.417836 20422 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:13.454005 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:13.424664 20370 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.639s	user 0.012s	sys 0.003s
I20260709 12:13:13.463740 12646 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:13.465219 19827 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:13:13.481222 20427 log.cc:927] Time spent T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:13:13.475934 20422 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:13.482853 20340 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:13.484139 20428 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:13:13.485836 20425 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
W20260709 12:13:13.485981 20423 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.115s	user 0.001s	sys 0.000s
W20260709 12:13:13.488106 20431 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:13:13.502858 20354 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:13.461676 20170 consensus_queue.cc:1059] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.042s
W20260709 12:13:13.507208 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 43559.4us
I20260709 12:13:13.483589 20422 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:13.509835 12646 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:13.448176 20366 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:13.517755 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 137686us
W20260709 12:13:13.521203 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:13.371430 20286 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:13.521658 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:13.521189 19827 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:13.533082 19827 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:13.533493 19827 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:13.511690 20370 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:13.447368 20202 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.081s
W20260709 12:13:13.525127 20331 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.714s	user 0.012s	sys 0.002s
I20260709 12:13:13.516279 20133 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:13.557971 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:13.564807 20133 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:13.568755 12422 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59006 (request call id 178) took 56605 ms (client timeout 15000 ms). Trace:
I20260709 12:13:13.569437 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:13.570432 12677 raft_consensus.cc:2410] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 23 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:13:13.573697 19610 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:13:13.576942 20440 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:13:13.577826 20379 consensus_queue.cc:1059] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 21.11, Next index: 12, Last known committed idx: 9, Time since last communication: 0.001s
W20260709 12:13:13.639099 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 53335.8us
I20260709 12:13:13.637842 20372 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:13.650919 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 526845us
I20260709 12:13:13.528213 20250 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:13.612854 20369 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:13.662554 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 81413.1us
I20260709 12:13:13.565230 20331 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:13.689416 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:13.692224 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:13.689095 19851 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:13:13.700903 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:13.646873 20326 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.044s
I20260709 12:13:13.672717 20379 raft_consensus.cc:3055] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:13.663424 20372 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:13.683137 20369 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:13.682641 19827 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:13.707046 20265 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.707887 20379 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:13.716780 20399 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:13.739729 20366 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:13.741909 20369 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:13.719756 20354 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.743705 20372 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:13.748925 20265 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:13.755264 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:13.720073 20262 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:13:13.744069 20122 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.621s	user 0.007s	sys 0.006s
I20260709 12:13:13.778236 12528 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:13:13.782070 12673 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:13.788524 12675 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
W20260709 12:13:13.791760 20448 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
I20260709 12:13:13.783190 12496 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:13.821586 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 59877.3us
I20260709 12:13:13.790297 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:13.777247 20262 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:13.833976 13332 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
W20260709 12:13:13.847054 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:13.757807 20352 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:13.860060 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 76972.6us
I20260709 12:13:13.831348 20357 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:13.763129 20399 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:13.778672 20354 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:13.852349 20372 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.857684 20369 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:13.871688 20357 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:13.828598 19851 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:13.874774 20455 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:13.838512 20379 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:13.864666 20352 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:13.959110 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 297937us
I20260709 12:13:13.953255 20357 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:13:13.960093 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52484us
I20260709 12:13:13.888664 20326 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.067s
W20260709 12:13:13.953233 20424 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:13:13.963817 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:13.906617 20331 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:13.897682 20250 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:13.951068 20203 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:13.967329 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 91328.1us
I20260709 12:13:13.899446 20372 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:13.984959 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 106239us
W20260709 12:13:13.990563 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:13.898082 20369 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:13.974802 20264 consensus_meta.cc:287] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.782s	user 0.015s	sys 0.000s
I20260709 12:13:14.002477 13338 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:14.004155 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:14.004528 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:14.004781 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:14.009900 12527 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:13:14.004155 12525 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:13:14.011390 12646 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:14.013037 20215 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:14.013825 12523 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 17: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:13.966818 20265 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:14.027179 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:14.028131 12680 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:14.017083 12530 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 15. Current term is 16. Ops: [15.9-15.9]
I20260709 12:13:14.030268 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:14.031910 12677 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 16 index: 10. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:13:14.031284 12531 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
W20260709 12:13:14.041592 20461 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
W20260709 12:13:14.041821 20317 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.833s	user 0.011s	sys 0.004s
I20260709 12:13:14.038487 20170 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.007s
W20260709 12:13:14.042909 20459 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.073s	user 0.000s	sys 0.000s
I20260709 12:13:14.025748 20215 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:14.044811 20215 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:13.997552 20203 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:14.034168 20081 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:14.048925 20462 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:13:14.049114 20081 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:14.007144 20374 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:14.051102 20374 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:14.052240 20081 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:14.053100 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:14.053527 20374 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:14.025352 20264 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:14.056205 20264 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:14.056486 20399 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:14.057178 20250 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:14.057744 20250 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:14.058068 20262 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:14.058823 19827 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:13:14.060030 20250 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:14.068651 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:14.071974 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:14.070241 12646 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:14.073030 12525 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:14.072539 12530 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:14.080392 12646 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:14.087787 12680 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:13:14.095144 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:14.096148 12531 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:13:14.097883 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:14.100998 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:14.115571 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:14.117236 12525 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:14.123854 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:14.125841 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:14.128455 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 104798us
W20260709 12:13:14.114779 20065 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.180s	user 0.008s	sys 0.010s
I20260709 12:13:14.076632 19827 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:14.139870 13336 raft_consensus.cc:2410] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 19 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 12:13:14.141455 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:14.142365 12680 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
W20260709 12:13:14.137074 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 59464.7us
I20260709 12:13:14.074849 20385 consensus_queue.cc:1059] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 14.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.060s
I20260709 12:13:14.146709 13304 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:14.108398 20215 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:14.155978 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:14.140259 20203 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:14.142864 19827 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:14.146036 20476 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:14.175158 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:14.180575 12531 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:13:14.184300 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:14.185159 13334 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
W20260709 12:13:14.186825 20457 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.273s	user 0.000s	sys 0.001s
W20260709 12:13:13.569248 12422 rpcz_store.cc:269] 0709 12:12:16.963378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:16.987157 (+ 23779us) service_pool.cc:224] Handling call
0709 12:13:13.568490 (+56581333us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:12:57.442240 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:12:57.443118 (+   878us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:12:57.443179 (+    61us) write_op.cc:435] Acquired schema lock
0709 12:12:57.443201 (+    22us) tablet.cc:662] Decoding operations
0709 12:12:57.444873 (+  1672us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:12:57.444987 (+   114us) write_op.cc:641] Partition lock acquired for write op
0709 12:12:57.445033 (+    46us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:12:57.497010 (+ 51977us) tablet.cc:701] Row locks acquired
0709 12:12:57.497096 (+    86us) write_op.cc:260] PREPARE: finished
0709 12:12:57.497518 (+   422us) write_op.cc:270] Start()
0709 12:12:57.503191 (+  5673us) write_op.cc:276] Timestamp: P: 1783599177497281 usec, L: 0
0709 12:12:57.522354 (+ 19163us) op_driver.cc:348] REPLICATION: starting
0709 12:12:57.523290 (+   936us) log.cc:844] Serialized 5577 byte log entry
0709 12:12:58.388012 (+864722us) op_driver.cc:464] REPLICATION: finished
0709 12:12:58.777808 (+389796us) write_op.cc:301] APPLY: starting
0709 12:12:58.777932 (+   124us) tablet.cc:1367] starting BulkCheckPresence
0709 12:12:58.937963 (+160031us) spinlock_profiling.cc:243] Waited 7.01 ms on lock 0x7b180044df40. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f148875f0ec 00007f14887abc0a
0709 12:13:00.280692 (+1342729us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b180044df40. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f148875f0ec 00007f14887abc0a 00007f14887ab4f7 00007f14887ab490 00007f1488762f48 00007f148c4771bb 00007f148c477532 00007f148c4beb93 00007f148c36bde3
0709 12:13:00.287798 (+  7106us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:00.287840 (+    42us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:00.696934 (+409094us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:00.697366 (+   432us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=28,mrs_lookups=0
0709 12:13:00.697442 (+    76us) write_op.cc:312] APPLY: finished
0709 12:13:00.697932 (+   490us) log.cc:844] Serialized 129 byte log entry
0709 12:13:01.733293 (+1035361us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:01.777232 (+ 43939us) write_op.cc:454] Released schema lock
0709 12:13:01.802675 (+ 25443us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:01.971346 (+168671us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:01.974033 (+  2687us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":14360,"reported_tablets":53,"child_traces":[["op",{"apply.queue_time_us":371204,"apply.run_cpu_time_us":55249,"apply.run_wall_time_us":3072607,"cfile_cache_hit":58,"cfile_cache_hit_bytes":62563,"num_ops":14,"prepare.queue_time_us":240827,"prepare.run_cpu_time_us":27735,"prepare.run_wall_time_us":398552,"raft.queue_time_us":403345,"raft.run_cpu_time_us":27370,"raft.run_wall_time_us":479384,"replication_time_us":834436,"spinlock_wait_cycles":20212736,"thread_start_us":1994987,"threads_started":5,"wal-append.queue_time_us":1387739,"wal-append.run_cpu_time_us":50454,"wal-append.run_wall_time_us":2826399}]]}
I20260709 12:13:14.179281 19827 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:14.190559 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3674) took 1314 ms. Trace:
I20260709 12:13:14.191095 12679 rpcz_store.cc:276] 0709 12:13:12.875657 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:13.164673 (+289016us) service_pool.cc:224] Handling call
0709 12:13:13.165036 (+   363us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:13.165454 (+   418us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:13.165480 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:13.531092 (+365612us) spinlock_profiling.cc:243] Waited 57.2 ms on lock 0x7b2c00037868. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:13:13.531596 (+   504us) log.cc:844] Serialized 26 byte log entry
0709 12:13:13.724494 (+192898us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:13.724535 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:13.724601 (+    66us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:14.190206 (+465605us) raft_consensus.cc:1685] finished
0709 12:13:14.190229 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:14.190488 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":289,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"spinlock_wait_cycles":131591424,"thread_start_us":192521,"threads_started":1,"wal-append.queue_time_us":325855}
W20260709 12:13:14.194474 20439 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.553s	user 0.005s	sys 0.000s
I20260709 12:13:14.195220 20385 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:14.195696 20385 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:14.196111 20385 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:14.182410 20476 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:14.200896 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 1.0828e+06us
I20260709 12:13:14.204794 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:14.184767 20354 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:14.218268 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 136744us
I20260709 12:13:14.222023 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:14.222896 12523 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:14.183609 20342 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:14.224869 20488 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:14.234509 20488 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:14.235045 20488 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:14.238703 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:14.239893 12672 raft_consensus.cc:2410] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:13:14.238750 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:14.241618 12677 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:13:14.170854 20374 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.046s
I20260709 12:13:14.244967 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:14.245960 12531 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:13:14.221225 12646 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:14.244915 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:14.214279 20422 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:14.256742 12523 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
W20260709 12:13:14.277868 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 314151us
I20260709 12:13:14.278824 12571 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:14.280418 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 59328.2us
I20260709 12:13:14.285171 20133 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:14.293864 20133 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:14.215858 20272 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.045s
I20260709 12:13:14.294562 20133 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:14.295099 20133 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:13:14.295138 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 93335us
I20260709 12:13:14.295663 20272 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:13:14.296135 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:14.297168 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:14.290365 20265 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:14.281117 12646 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:14.286005 20466 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:14.257412 20342 raft_consensus.cc:697] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:14.305878 20430 log.cc:927] Time spent T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.834s	user 0.003s	sys 0.000s
I20260709 12:13:14.306344 20487 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:14.306883 20487 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:14.306469 20342 consensus_queue.cc:237] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:14.307266 20487 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:14.307475 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3665) took 1477 ms. Trace:
I20260709 12:13:14.308781 20262 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:14.307926 12674 rpcz_store.cc:276] 0709 12:13:12.829677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:12.846393 (+ 16716us) service_pool.cc:224] Handling call
0709 12:13:12.846818 (+   425us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:12.856186 (+  9368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:12.856240 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:12.926228 (+ 69988us) log.cc:844] Serialized 26 byte log entry
0709 12:13:13.135142 (+208914us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:13.135184 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:13.135254 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:13.190805 (+ 55551us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b4800060858. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:13:14.305185 (+1114380us) spinlock_profiling.cc:243] Waited 2.09 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:14.307225 (+  2040us) raft_consensus.cc:1685] finished
0709 12:13:14.307247 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:14.307404 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":50650,"raft.queue_time_us":9025,"raft.run_cpu_time_us":2360,"raft.run_wall_time_us":314433,"spinlock_wait_cycles":40835712,"thread_start_us":186967,"threads_started":1,"wal-append.queue_time_us":424877}
I20260709 12:13:14.304368 12604 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 14 index: 9. Preceding OpId from leader: term: 18 index: 11. (index mismatch)
I20260709 12:13:14.305415 20466 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:14.302506 12646 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:14.269006 20422 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:14.343824 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 65126.3us
I20260709 12:13:14.346294 12571 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:14.309238 20262 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:14.294343 20490 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:14.367043 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:14.367596 20463 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:14.368283 20396 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:13:14.371066 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 70579.9us
I20260709 12:13:14.411271 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:14.414458 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 267816us
I20260709 12:13:14.418282 12599 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:14.312171 20250 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:14.424544 12606 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:14.377118 20272 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:14.382980 20354 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:14.375422 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:14.423616 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.425894 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.338811 20422 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:14.442691 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.388055 20262 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:14.446537 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.447422 12599 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
W20260709 12:13:14.443049 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 198205us
I20260709 12:13:14.451447 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:14.452338 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 76988.9us
I20260709 12:13:14.455308 12674 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 12:13:14.455760 12674 pending_rounds.cc:85] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 11
I20260709 12:13:14.456106 12674 pending_rounds.cc:107] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 12:13:14.456375 12674 raft_consensus.cc:2889] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:14.459296 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.460206 12600 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:14.462903 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.464304 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 15 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.465085 12604 raft_consensus.cc:2410] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 19 because replica has last-logged OpId of term: 17 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 14.
I20260709 12:13:14.424327 20250 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:14.466023 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.466511 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.464788 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:14.469897 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:14.471688 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.472608 12599 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 22: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:14.512202 12600 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:14.511165 20250 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:14.520730 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:14.521697 12678 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:14.523466 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:14.523590 12677 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:14.449681 20272 raft_consensus.cc:697] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:14.527824 12675 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:14.527370 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.460917 20463 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:14.460371 20354 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:14.530773 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:14.548156 12677 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:14.543191 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.550380 12600 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:14.567821 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:14.602897 12599 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:14.601440 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.533599 20396 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:14.627696 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.616848 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:14.637436 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:14.627187 12678 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:14.548589 20463 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:14.545243 20265 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:14.544725 19842 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:14.665560 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 17 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.673635 12678 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:13:14.675099 12675 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:14.682439 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.676664 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.676426 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.685985 12599 raft_consensus.cc:2410] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 20 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
W20260709 12:13:14.711745 19827 consensus_meta.cc:287] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.520s	user 0.011s	sys 0.001s
I20260709 12:13:14.712630 19827 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:14.665036 20065 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:14.673638 20463 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:13:14.727236 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:14.728955 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.719515 12678 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 27. Current term is 28. Ops: []
I20260709 12:13:14.737800 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:14.749207 12678 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:14.755012 12678 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:13:14.759819 12571 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:14.761358 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:14.762696 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:14.771147 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.615200 20218 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:14.750524 20457 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:13:14.421494 20490 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:14.689389 20265 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:14.774173 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:14.668176 20396 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:14.701653 19842 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:14.827844 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 16 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:14.825763 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.828867 12531 raft_consensus.cc:2393] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 16: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:14.829715 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:14.830858 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:14.831219 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:14.831900 12530 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:14.832849 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:14.729669 20065 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:14.838704 12597 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:13:14.831492 12525 raft_consensus.cc:2376] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23. Re-sending same reply.
I20260709 12:13:14.771348 19851 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:14.822288 20243 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:13:14.838214 20385 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.641s	user 0.011s	sys 0.003s
I20260709 12:13:14.720181 20272 consensus_queue.cc:237] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 18.8, Last appended by leader: 8, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:14.858964 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:14.864676 12599 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:13:14.835486 20218 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:14.827922 20100 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:14.776245 20374 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.072s
I20260709 12:13:14.842484 20264 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:14.877727 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:14.882362 20483 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.253s	user 0.000s	sys 0.001s
I20260709 12:13:14.759318 20354 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 24.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:14.888155 20393 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:14.888775 20393 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:14.828032 20464 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.134s
I20260709 12:13:14.891186 20393 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:14.891901 20465 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
W20260709 12:13:14.893139 20495 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.218s	user 0.000s	sys 0.001s
I20260709 12:13:14.894320 20398 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:14.884156 20100 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:14.894954 20398 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:14.896751 20100 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:14.876827 20385 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:14.899506 20385 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:14.900676 20385 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:14.897462 20494 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:14.901448 12680 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: []
W20260709 12:13:14.835707 20460 consensus_queue.cc:1175] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "193a40da6e364ead8ab561e14c3cc8d1" responder_term: 22 status { last_received { term: 21 index: 16 } last_committed_idx: 16 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:13:14.903144 20351 raft_consensus.cc:3055] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Stepping down as leader of term 15
I20260709 12:13:14.903635 20351 raft_consensus.cc:740] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:14.904620 20351 consensus_queue.cc:260] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 15, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:14.901561 20494 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:14.908437 20494 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:14.911294 20499 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
I20260709 12:13:14.911582 20490 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:14.923069 20351 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 15 FOLLOWER]: Advancing to term 16
W20260709 12:13:14.893280 20488 consensus_meta.cc:287] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.658s	user 0.009s	sys 0.004s
W20260709 12:13:14.924830 20492 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.295s	user 0.004s	sys 0.000s
I20260709 12:13:14.924643 20488 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:14.929091 20509 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:14.938326 20509 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:14.938917 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 3684) took 1383 ms. Trace:
I20260709 12:13:14.939378 12673 rpcz_store.cc:276] 0709 12:13:13.555908 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:13.557700 (+  1792us) service_pool.cc:224] Handling call
0709 12:13:14.938689 (+1380989us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b540016eed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:14.938850 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":507922816}
W20260709 12:13:14.941305 20442 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.589s	user 0.003s	sys 0.002s
I20260709 12:13:14.941463 20509 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:14.941929 20488 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:14.868445 20434 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:13:14.937250 20393 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:14.943253 20369 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:14.943208 20393 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:14.945640 20393 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:14.946166 12528 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3177) took 1638 ms (client timeout 2080 ms). Trace:
W20260709 12:13:14.950919 12528 rpcz_store.cc:269] 0709 12:13:13.307982 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:13.308187 (+   205us) service_pool.cc:224] Handling call
0709 12:13:14.945950 (+1637763us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b54003caad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:14.946075 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":997205888}
I20260709 12:13:14.952289 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:14.952977 20357 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.980s	user 0.012s	sys 0.000s
I20260709 12:13:14.953619 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:14.955924 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3597) took 1828 ms. Trace:
I20260709 12:13:14.954828 12528 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:14.953473 20357 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:14.958577 12601 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:14.956455 12602 rpcz_store.cc:276] 0709 12:13:13.127542 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:13.127677 (+   135us) service_pool.cc:224] Handling call
0709 12:13:13.128056 (+   379us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:13.877233 (+749177us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:13.877284 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:13.883403 (+  6119us) log.cc:844] Serialized 26 byte log entry
0709 12:13:14.136825 (+253422us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:14.136992 (+   167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:14.137149 (+   157us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:14.955594 (+818445us) raft_consensus.cc:1685] finished
0709 12:13:14.955627 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:14.955832 (+   205us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":547640,"raft.queue_time_us":536026,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":20748,"thread_start_us":402489,"threads_started":2,"wal-append.queue_time_us":386096}
I20260709 12:13:14.846752 20482 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.447s
I20260709 12:13:14.965803 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:14.887310 20264 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:14.966749 12528 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:14.964488 19827 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:14.959153 20385 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:14.986613 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 226851us
I20260709 12:13:14.984709 12528 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 19. Current term is 20. Ops: [17.12-19.13]
I20260709 12:13:15.000396 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:15.007498 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:14.968485 20250 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:14.978678 20512 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:15.011315 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:15.013777 12528 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:13:15.046732 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:15.048364 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:15.049710 12528 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:15.084048 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:15.058144 12602 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:15.010977 19851 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:15.032574 20516 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:15.009114 20369 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:15.009191 20399 consensus_queue.cc:1059] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:13:15.029299 20512 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:15.097968 20357 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:15.099694 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:15.113675 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 165321us
I20260709 12:13:15.114013 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:15.114079 12602 raft_consensus.cc:3055] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:15.114847 12602 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:15.008951 20393 consensus_queue.cc:1059] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.076s
I20260709 12:13:15.102067 12606 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:15.118067 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:15.119606 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 271408us
I20260709 12:13:15.095206 19851 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:15.128695 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.124192 13333 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: []
W20260709 12:13:15.104802 20133 consensus_meta.cc:287] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.809s	user 0.009s	sys 0.006s
W20260709 12:13:15.109386 20487 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.802s	user 0.008s	sys 0.006s
I20260709 12:13:15.132243 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:15.132695 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:15.115933 12602 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:15.138855 12602 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:15.107807 20512 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:15.151701 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:15.149287 13332 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:15.102320 20515 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:15.112996 20513 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:15.110805 20396 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:15.122879 20065 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:15.139213 20133 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:15.123203 19982 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:15.156625 20250 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:15.157636 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.109077 20463 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:15.131106 19851 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:15.156899 20262 consensus_meta.cc:287] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.690s	user 0.018s	sys 0.000s
I20260709 12:13:15.138062 20518 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:15.151320 20374 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.120242 20369 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:15.193023 20491 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:15.131158 20372 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:15.153644 20393 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:15.157794 20482 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.225252 12602 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:13:15.236397 20402 raft_consensus.cc:3055] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:15.237068 20396 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:15.253245 20402 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:15.237720 20516 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:13:15.253873 20396 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:15.152701 20515 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:15.255165 20372 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:15.255754 20372 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:15.256312 20396 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:15.256537 20515 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:15.257956 20422 consensus_meta.cc:287] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.799s	user 0.000s	sys 0.011s
I20260709 12:13:15.258718 20422 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:15.261791 20369 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:15.189213 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 178396us
I20260709 12:13:15.261956 20422 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:15.148551 20487 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:15.228932 20264 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:15.257896 20372 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:15.167956 20513 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:15.233141 19982 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:15.262746 20518 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:15.270799 20385 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:15.240147 20262 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:15.245863 20374 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:15.269474 20393 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:15.335011 20518 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:15.336725 20393 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:15.250530 20491 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:15.271267 20482 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:15.345938 20527 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 12:13:15.347281 20527 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:15.347400 20374 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:15.348871 20487 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:15.350428 20475 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 1.172s	user 0.007s	sys 0.001s
I20260709 12:13:15.351078 20463 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:15.351787 20476 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:15.301575 20402 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:15.347723 20527 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:15.352805 20402 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:15.352965 20527 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:15.354593 20527 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:15.355733 20491 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:15.353394 20513 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:15.371686 20482 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:15.410785 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 264408us
I20260709 12:13:15.413604 12496 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:15.432456 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 939) took 2505 ms. Trace:
I20260709 12:13:15.434435 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 947) took 1563 ms. Trace:
I20260709 12:13:15.434993 12524 rpcz_store.cc:276] 0709 12:13:13.870898 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:13.872458 (+  1560us) service_pool.cc:224] Handling call
0709 12:13:14.739430 (+866972us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:14.739725 (+   295us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:13:14.983669 (+243944us) log.cc:844] Serialized 14 byte log entry
0709 12:13:15.236783 (+253114us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:15.237016 (+   233us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:15.237166 (+   150us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.237269 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:15.369190 (+131921us) spinlock_profiling.cc:243] Waited 5.98 ms on lock 0x7b18004cab60. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:13:15.433994 (+ 64804us) spinlock_profiling.cc:243] Waited 867 ms on lock 0x7b5400105ed0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:15.434229 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5823,"prepare.queue_time_us":357865,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":3393,"raft.queue_time_us":8667,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":17832,"spinlock_wait_cycles":2006770304,"thread_start_us":457506,"threads_started":2,"wal-append.queue_time_us":279040}
W20260709 12:13:15.437526 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.443758 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 928) took 3495 ms. Trace:
I20260709 12:13:15.444492 12522 rpcz_store.cc:276] 0709 12:13:11.948570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:11.948697 (+   127us) service_pool.cc:224] Handling call
0709 12:13:12.135547 (+186850us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:13.251888 (+1116341us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:14.184337 (+932449us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b54003cb9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:15.443185 (+1258848us) spinlock_profiling.cc:243] Waited 186 ms on lock 0x7b54003cb9d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:15.443549 (+   364us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2991044224}
W20260709 12:13:15.445197 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 181868us
I20260709 12:13:15.448055 12605 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:15.433171 12526 rpcz_store.cc:276] 0709 12:13:12.926696 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:12.926849 (+   153us) service_pool.cc:224] Handling call
0709 12:13:12.927935 (+  1086us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:13.026659 (+ 98724us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:13:13.317380 (+290721us) log.cc:844] Serialized 14 byte log entry
0709 12:13:13.649651 (+332271us) log.cc:844] Serialized 14 byte log entry
0709 12:13:13.665544 (+ 15893us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:13.685717 (+ 20173us) log.cc:844] Serialized 26 byte log entry
0709 12:13:13.685915 (+   198us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:13:13.685972 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:13.740367 (+ 54395us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b18002e5dc0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:13:15.431564 (+1691197us) spinlock_profiling.cc:243] Waited 88.2 ms on lock 0x7b540027f6d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:15.431678 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:15.431777 (+    99us) raft_consensus.cc:1685] finished
0709 12:13:15.431803 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:15.432196 (+   393us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":113603,"prepare.queue_time_us":348373,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":88,"raft.queue_time_us":54944,"raft.run_cpu_time_us":459,"raft.run_wall_time_us":39238,"spinlock_wait_cycles":226940544,"thread_start_us":536554,"threads_started":3,"wal-append.queue_time_us":256812,"wal-append.run_cpu_time_us":15909,"wal-append.run_wall_time_us":1182040}
I20260709 12:13:15.461422 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 851) took 2019 ms. Trace:
I20260709 12:13:15.462216 12676 rpcz_store.cc:276] 0709 12:13:13.442042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:13.467146 (+ 25104us) service_pool.cc:224] Handling call
0709 12:13:13.469869 (+  2723us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:13.482302 (+ 12433us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:13.482351 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:13.482404 (+    53us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:13.482438 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.460661 (+1978223us) spinlock_profiling.cc:243] Waited 12.1 ms on lock 0x7b54003d4fd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:15.460789 (+   128us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:15.461151 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":27884928}
I20260709 12:13:15.469108 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3164) took 2215 ms. Trace:
I20260709 12:13:15.468617 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:15.469684 12529 rpcz_store.cc:276] 0709 12:13:13.253013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:13.253137 (+   124us) service_pool.cc:224] Handling call
0709 12:13:13.380218 (+127081us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:13.422070 (+ 41852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:14.737551 (+1315481us) spinlock_profiling.cc:243] Waited 40.8 ms on lock 0x7b5400105ed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:15.468355 (+730804us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b5400105ed0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:15.468693 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":385357056}
I20260709 12:13:15.478360 12675 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:15.489996 13331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38482 (request call id 752) took 1787 ms. Trace:
I20260709 12:13:15.519018 13331 rpcz_store.cc:276] 0709 12:13:13.701681 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:13.702111 (+   430us) service_pool.cc:224] Handling call
0709 12:13:13.702468 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:13.702709 (+   241us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:13:13.951731 (+249022us) log.cc:844] Serialized 14 byte log entry
0709 12:13:14.914389 (+962658us) spinlock_profiling.cc:243] Waited 16.8 ms on lock 0x7b180038c8c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:13:15.488006 (+573617us) spinlock_profiling.cc:243] Waited 52.6 ms on lock 0x7b180038c8c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287
0709 12:13:15.488650 (+   644us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:15.488813 (+   163us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:13:15.488982 (+   169us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.489151 (+   169us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:15.489424 (+   273us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15470,"prepare.queue_time_us":267023,"prepare.run_cpu_time_us":50,"prepare.run_wall_time_us":51,"raft.queue_time_us":55582,"raft.run_cpu_time_us":164,"raft.run_wall_time_us":168,"spinlock_wait_cycles":159750272,"thread_start_us":303167,"threads_started":2}
W20260709 12:13:15.492097 13336 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 749) took 2594 ms (client timeout 2235 ms). Trace:
W20260709 12:13:15.523245 13336 rpcz_store.cc:269] 0709 12:13:12.897747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:12.904320 (+  6573us) service_pool.cc:224] Handling call
0709 12:13:15.491312 (+2586992us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b5400690354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:15.491742 (+   430us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2838814464}
I20260709 12:13:15.495105 13338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 849) took 1803 ms. Trace:
I20260709 12:13:15.498242 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3179) took 1070 ms. Trace:
I20260709 12:13:15.525523 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:15.525758 12605 rpcz_store.cc:276] 0709 12:13:14.428244 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.521343 (+ 93099us) service_pool.cc:224] Handling call
0709 12:13:14.543867 (+ 22524us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:15.456560 (+912693us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.497815 (+ 41255us) spinlock_profiling.cc:243] Waited 903 ms on lock 0x7b54003110d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:15.498147 (+   332us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2077397376}
I20260709 12:13:15.507144 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3168) took 1096 ms. Trace:
I20260709 12:13:15.527563 12601 rpcz_store.cc:276] 0709 12:13:14.410061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.412817 (+  2756us) service_pool.cc:224] Handling call
0709 12:13:14.958322 (+545505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:14.959100 (+   778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.506474 (+547374us) spinlock_profiling.cc:243] Waited 545 ms on lock 0x7b540022fbd0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:15.506877 (+   403us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1252790656}
I20260709 12:13:15.518172 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3194) took 1402 ms. Trace:
I20260709 12:13:15.530680 19826 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:15.529323 12527 rpcz_store.cc:276] 0709 12:13:14.079622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.079780 (+   158us) service_pool.cc:224] Handling call
0709 12:13:14.129197 (+ 49417us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:14.147496 (+ 18299us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:14.147527 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:14.148019 (+   492us) log.cc:844] Serialized 26 byte log entry
0709 12:13:14.288141 (+140122us) spinlock_profiling.cc:243] Waited 5.18 ms on lock 0x7b4800002958. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:13:15.480984 (+1192843us) spinlock_profiling.cc:243] Waited 983 us on lock 0x7b18004caa40. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1
0709 12:13:15.481395 (+   411us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:13:15.481466 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.481591 (+   125us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:15.481639 (+    48us) raft_consensus.cc:1685] finished
0709 12:13:15.481677 (+    38us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:15.481940 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39812,"raft.queue_time_us":19417,"raft.run_cpu_time_us":2067,"raft.run_wall_time_us":134588,"spinlock_wait_cycles":14178560,"thread_start_us":162315,"threads_started":1,"wal-append.queue_time_us":297940}
I20260709 12:13:15.531044 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:15.510216 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:15.535372 12676 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 16.
I20260709 12:13:15.525238 13338 rpcz_store.cc:276] 0709 12:13:13.692020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:13.692865 (+   845us) service_pool.cc:224] Handling call
0709 12:13:13.704015 (+ 11150us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:14.008717 (+304702us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.494544 (+1485827us) spinlock_profiling.cc:243] Waited 298 ms on lock 0x7b54004e84d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:15.494914 (+   370us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":685657856}
I20260709 12:13:15.536978 19842 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:15.540182 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:15.541069 12676 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:13:15.541617 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:15.543758 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:15.543653 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:15.550531 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:15.551608 12679 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:13:15.557881 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:15.517437 19610 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.043s
I20260709 12:13:15.560672 19982 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:15.563016 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:15.564113 12603 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:15.566479 20065 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:15.566716 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:15.567906 13331 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 19: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:15.567487 20065 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:15.572543 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:15.575143 20133 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:15.576289 20133 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:15.576735 20250 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:15.577137 20133 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:15.577668 20250 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:15.580154 20133 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:15.580598 20250 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:15.582800 20264 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:13:15.359445 20352 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:15.588892 20262 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:15.593266 20393 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:13:15.593972 20398 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:15.593973 20393 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:15.585850 20264 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.010s
I20260709 12:13:15.595523 20170 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:15.596367 20264 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:15.596957 20264 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:15.596464 20393 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:15.596381 20170 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:15.599371 20393 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:15.601593 20170 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:15.604072 20264 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:15.607995 20264 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:13:15.609155 20122 consensus_queue.cc:1175] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "193a40da6e364ead8ab561e14c3cc8d1" responder_term: 21 status { last_received { term: 21 index: 11 } last_committed_idx: 9 last_received_current_leader { term: 21 index: 11 } }
I20260709 12:13:15.610440 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:15.615310 20215 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:15.621173 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:15.623016 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:15.632519 12679 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:15.668059 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:15.671020 12603 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
W20260709 12:13:15.673951 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.622026 19842 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 17.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:13:15.674877 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:15.680200 20442 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.059s	user 0.001s	sys 0.000s
W20260709 12:13:15.684974 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.686316 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:15.688522 20473 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:15.689378 20473 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:15.692474 20473 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:15.705246 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:15.706538 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:15.708194 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 281944us
I20260709 12:13:15.645368 20202 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:15.718572 12675 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:15.723989 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:15.725260 20202 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:15.750931 20065 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:15.726393 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:15.727010 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:15.727577 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:15.731163 12496 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:15.732730 20473 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:15.738468 20519 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.756584 20519 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:15.758219 20265 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:15.759366 20519 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:15.760085 20519 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:15.739717 20491 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:13:15.745098 12671 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:15.762394 20202 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:15.745286 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.766742 12646 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:15.768671 20487 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:15.769400 20487 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:15.770726 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:15.770911 20487 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:15.771643 12673 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
W20260709 12:13:15.775723 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:15.776320 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.778337 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:15.781424 20379 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:15.781860 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:15.781479 12530 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:15.747162 13302 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:15.747987 20466 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:15.748230 12679 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:15.733577 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:15.757150 20526 raft_consensus.cc:3055] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:15.739302 19851 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:15.798696 12605 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:13:15.810283 12597 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:13:15.795778 20519 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:15.805474 12677 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:13:15.798666 20526 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:15.820935 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:15.827847 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:15.829123 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:15.830421 12605 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:13:15.830982 12527 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:15.831457 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:15.832365 12522 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:13:15.829200 12526 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:13:15.836308 20357 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:15.821033 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:15.819106 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:15.839351 12529 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:15.816645 20519 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:15.844148 20530 raft_consensus.cc:3055] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:15.844367 20464 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:15.847838 20357 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:15.849509 12525 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:15.849514 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:15.859191 20526 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:15.860517 20526 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:13:15.862994 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 146708us
I20260709 12:13:15.863631 20215 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:15.864249 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:15.864205 20215 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:15.866066 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:15.874568 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:15.867991 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:15.874188 12601 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:13:15.820590 20473 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:15.924997 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:15.871965 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:15.868395 12598 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:15.924922 20466 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:15.879449 20357 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:15.871860 20393 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:15.933701 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:15.934661 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:15.855705 20530 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:15.929121 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:15.923310 20170 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.950837 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:15.966676 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:15.967593 12680 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:15.969313 12677 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:13:15.969352 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3150) took 1501 ms. Trace:
I20260709 12:13:15.973965 12671 rpcz_store.cc:276] 0709 12:13:14.468112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.468307 (+   195us) service_pool.cc:224] Handling call
0709 12:13:14.946904 (+478597us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:15.762593 (+815689us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:15.793075 (+ 30482us) spinlock_profiling.cc:243] Waited 798 ms on lock 0x7b540016eed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:15.969159 (+176084us) spinlock_profiling.cc:243] Waited 478 ms on lock 0x7b540016eed0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:15.969298 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2934407680}
I20260709 12:13:15.975389 20215 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:15.868202 20464 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:15.980576 12572 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 16 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:15.983333 20382 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 15 FOLLOWER]: Advancing to term 16
I20260709 12:13:15.983430 20466 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:15.986606 20081 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:15.987164 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:15.987214 20081 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:15.988677 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:15.969223 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:15.988421 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54166.1us
W20260709 12:13:15.958670 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 161050us
I20260709 12:13:15.993848 12604 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:13:15.996805 20170 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:15.997752 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:15.998626 12529 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:16.000180 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:16.000886 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:16.001163 12526 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:13:16.002857 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:16.003501 12527 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:16.007184 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:16.008080 12601 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:15.987279 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:16.012753 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:16.013867 12598 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:15.985940 20466 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:16.020740 20464 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:16.021795 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:16.018188 12571 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:16.026188 20451 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:16.026805 20451 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:16.027328 20451 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:16.021255 20170 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:16.040436 12529 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22.
I20260709 12:13:16.017328 12602 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:13:16.045897 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:16.052256 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 72035.9us
I20260709 12:13:16.056531 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:16.057663 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48095.7us
I20260709 12:13:16.054309 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:16.060765 12677 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:16.062275 12679 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:16.115360 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:16.024104 20393 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:16.127074 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:16.071637 19851 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:16.138890 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:16.141067 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:16.148551 12572 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:13:16.155869 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 107321us
I20260709 12:13:16.154433 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:16.138590 20466 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:16.160387 20396 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:16.160902 20516 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.754s	user 0.033s	sys 0.024s
I20260709 12:13:16.161616 12676 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 12:13:16.163594 12496 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:16.174633 20393 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:16.174940 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 111359us
I20260709 12:13:16.195426 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:16.175220 20464 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
W20260709 12:13:16.186262 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.191428 12676 pending_rounds.cc:85] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 12
W20260709 12:13:16.208811 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 45304.8us
W20260709 12:13:16.208897 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 84251.5us
I20260709 12:13:16.210497 12672 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:16.211261 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:16.195921 20516 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.898s	user 0.033s	sys 0.025s
I20260709 12:13:16.200522 12676 pending_rounds.cc:107] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 12:13:16.222554 12676 raft_consensus.cc:2889] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:16.223367 12598 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:16.196691 20466 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:16.221845 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:16.230046 12527 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
I20260709 12:13:16.234536 12599 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 20.
I20260709 12:13:16.199548 20464 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:16.237460 12571 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:16.206750 19689 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:16.229033 20393 raft_consensus.cc:2764] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election decision vote started in defunct term 23: won
W20260709 12:13:16.260339 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46900.2us
I20260709 12:13:16.236723 20466 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:16.265726 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:16.244397 20464 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:13:16.270342 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 56435.4us
I20260709 12:13:16.271155 12526 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
I20260709 12:13:16.232465 20170 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:16.283073 20170 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:16.269388 19689 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:16.288713 19689 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:16.277428 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:16.291998 20243 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:16.292472 20243 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:16.292867 20243 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:13:16.277544 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 54680.9us
I20260709 12:13:16.297487 12494 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:16.300433 20120 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:16.302295 20531 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:16.302857 20531 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:16.303305 20531 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:16.241806 19851 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:13:16.308899 20420 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:16.309478 20420 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:16.309890 20420 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:16.250919 20122 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:13:16.339300 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 207737us
W20260709 12:13:16.346786 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.276744 20396 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:16.345860 20122 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:16.416664 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 359264us
I20260709 12:13:16.415886 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:16.383280 20122 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:16.415843 12646 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.993s (SENT) [suppressed 19 similar messages]
I20260709 12:13:16.446660 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:16.447214 20327 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 12:13:16.383857 20516 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 24.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.075s
I20260709 12:13:16.356526 20496 consensus_queue.cc:1059] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.085s
W20260709 12:13:16.446698 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 72646.2us
W20260709 12:13:16.417313 12643 connection.cc:256] RPC call timeout handler was delayed by -1.4297237130001577s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2138s for a response
I20260709 12:13:16.438421 20170 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:16.490597 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:16.484558 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 189419us
I20260709 12:13:16.491801 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:16.495947 20385 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:16.497306 20379 consensus_meta.cc:287] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.715s	user 0.010s	sys 0.003s
W20260709 12:13:16.496186 12643 connection.cc:256] RPC call timeout handler was delayed by -1.2780660509999962s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2235s for a response
W20260709 12:13:16.497937 12643 connection.cc:256] RPC call timeout handler was delayed by -1.2434237130000838s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.23450000000000004s for a response
W20260709 12:13:16.498322 12643 connection.cc:256] RPC call timeout handler was delayed by -1.215550359999952s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2235s for a response
W20260709 12:13:16.499116 12646 leader_election.cc:336] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 2.080s (SENT)
W20260709 12:13:16.501960 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.403357 20530 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:16.511611 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:16.508270 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:16.506655 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:16.513303 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.526955 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:16.527150 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:16.526196 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:16.531877 19851 consensus_queue.cc:1243] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 23.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:13:16.532779 12677 raft_consensus.cc:2393] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 24: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:13:16.533660 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:16.534212 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.534863 20170 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:16.535871 20451 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.508s	user 0.006s	sys 0.006s
I20260709 12:13:16.535480 20170 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:16.536310 20451 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:16.537446 12525 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:16.538111 20487 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:16.538180 20170 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:16.538844 20487 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:16.540678 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:16.541451 12671 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:13:16.541698 20487 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:16.544697 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:16.551267 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:16.559774 20396 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:16.562776 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 60795.9us
I20260709 12:13:16.565091 20491 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:13:16.565254 13302 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:16.567097 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:16.567458 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:16.567893 12526 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:16.568089 12523 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:16.568945 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:16.555893 13336 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:16.542083 20531 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:16.571578 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:16.572700 20531 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:16.576745 20357 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.613s	user 0.012s	sys 0.001s
I20260709 12:13:16.577363 20357 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:16.578897 13302 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:16.579816 20065 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:16.580261 20065 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:16.580593 20065 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:16.587608 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:16.588096 13338 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:13:16.588398 13336 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:16.588829 20530 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:16.577925 12671 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
W20260709 12:13:16.595773 20519 consensus_meta.cc:287] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.723s	user 0.004s	sys 0.008s
I20260709 12:13:16.596333 20519 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:16.597805 12529 raft_consensus.cc:2393] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 23: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:16.598904 12672 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 19. Current term is 20. Ops: [19.13-19.13]
I20260709 12:13:16.581240 20243 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:16.600251 12598 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:16.599889 12523 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:16.582056 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:16.603372 12572 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:16.605309 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:16.582589 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:16.611673 20357 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:16.618646 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:16.618827 20065 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:16.620121 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:16.621306 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:16.620261 12601 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:16.624073 20243 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:16.579838 20385 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:16.625644 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:16.629299 20385 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:16.629812 12523 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:13:16.630999 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:16.571465 20550 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 12:13:16.577683 20466 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:16.641988 12526 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 19: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:16.708938 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:16.606113 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 58597.8us
I20260709 12:13:16.626389 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:16.727705 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 72414.8us
I20260709 12:13:16.667300 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:16.727033 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 176919us
I20260709 12:13:16.665652 20473 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:16.663884 20399 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:13:16.710306 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 194868us
I20260709 12:13:16.709810 20451 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:16.752600 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.582432 20491 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:16.632421 20065 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:16.735222 20399 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:13:16.662954 20526 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.802s	user 0.009s	sys 0.008s
I20260709 12:13:16.764581 12527 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:16.779395 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 43528.7us
I20260709 12:13:16.781368 12601 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:16.667415 20519 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:16.626235 20554 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:13:16.788870 12523 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:16.790890 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:16.793660 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:16.662814 20327 raft_consensus.cc:3055] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:13:16.794945 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.802974 12600 raft_consensus.cc:2393] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 24: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:16.710632 20243 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:13:16.801771 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:16.812853 12602 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 22: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:16.815763 12531 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:16.817492 12605 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
W20260709 12:13:16.829596 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 246804us
W20260709 12:13:16.809072 20264 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.184s	user 0.007s	sys 0.010s
I20260709 12:13:16.829816 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:16.803570 20526 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:16.799073 20327 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:16.836335 20243 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:16.842042 20243 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:16.843384 20202 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:16.844511 20243 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:16.806862 20554 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:16.844822 20202 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:16.845790 20554 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:16.847187 20554 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:16.850389 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:16.825788 20399 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:16.852771 20399 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:16.854326 20545 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 22.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.060s
I20260709 12:13:16.855120 19539 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:16.855616 19539 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:16.856647 20518 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:13:16.856808 20122 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:16.858022 12530 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 22. Current term is 23. Ops: [22.18-22.18]
W20260709 12:13:16.859882 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 66762.5us
I20260709 12:13:16.861152 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:16.862908 12603 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:16.867786 20122 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:16.851046 12680 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:16.887291 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:16.889233 12643 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:16.890369 20382 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.906s	user 0.011s	sys 0.006s
I20260709 12:13:16.890847 20382 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election lost for term 16. Reason: could not achieve majority
I20260709 12:13:16.891146 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:16.892517 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:16.893158 12523 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:13:16.894325 20487 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:16.894145 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:16.894058 20476 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:13:16.895664 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:16.892539 20202 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:16.899458 20546 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.399s	user 0.004s	sys 0.001s
W20260709 12:13:16.902086 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:16.902511 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:16.903345 12523 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:13:16.901903 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:16.904248 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:16.904863 12676 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:16.895102 20487 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:13:16.902308 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 132725us
I20260709 12:13:16.893738 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:16.838018 20516 consensus_queue.cc:1243] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 25.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:13:16.910395 12524 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:16.835691 20526 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:13:16.908787 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 70687.6us
I20260709 12:13:16.919730 12523 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:13:16.925014 20561 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:16.920946 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:16.952584 12673 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:16.918103 20526 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:16.971763 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:16.973121 20531 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:13:16.973672 20531 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:16.973438 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:16.979246 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:16.982278 12646 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:13:16.983024 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 62637.6us
I20260709 12:13:16.983774 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:16.985495 12646 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:13:16.986600 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 97532.3us
I20260709 12:13:16.987654 20215 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:16.990582 20466 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:16.858145 20556 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.200s
I20260709 12:13:16.955969 20518 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:17.013988 12494 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:17.013063 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:17.019683 12527 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:17.022840 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:17.024863 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:17.032233 12678 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:13:17.035789 12680 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:17.011377 20487 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:13:17.043154 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 210994us
W20260709 12:13:17.044174 20549 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.612s	user 0.000s	sys 0.004s
I20260709 12:13:17.044910 12496 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:17.022323 20215 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:13:17.047042 20420 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.737s	user 0.013s	sys 0.000s
I20260709 12:13:17.047200 20215 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:17.047518 20420 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:17.048100 20460 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:17.048596 20560 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:17.049176 20560 raft_consensus.cc:697] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:17.050333 20560 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:17.052713 12522 raft_consensus.cc:2368] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21.
I20260709 12:13:17.023634 20531 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:13:17.047745 20551 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:13:17.052897 20487 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
W20260709 12:13:17.056499 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 55613.1us
I20260709 12:13:17.056878 20531 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:17.049849 20420 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:17.067334 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 81904.7us
I20260709 12:13:17.022563 20518 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:13:17.102439 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 92647.3us
I20260709 12:13:16.977540 20327 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:17.116007 12494 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:17.120273 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:17.144663 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:17.145363 12677 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:17.142799 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:17.148654 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:17.149683 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3197) took 1461 ms. Trace:
I20260709 12:13:17.152266 12646 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:17.150333 12603 rpcz_store.cc:276] 0709 12:13:15.682538 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:15.701143 (+ 18605us) service_pool.cc:224] Handling call
0709 12:13:17.143898 (+1442755us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b5400116cd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:17.144080 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2595838336}
I20260709 12:13:17.154850 12600 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:13:17.158383 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 297340us
I20260709 12:13:17.168996 12524 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:17.174283 13305 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:17.114286 20491 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 20 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:17.145989 20202 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:13:17.178489 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 59551.8us
I20260709 12:13:17.159103 20556 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:17.184638 12646 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:13:17.182938 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 160175us
I20260709 12:13:17.142836 19826 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.037s
W20260709 12:13:17.207254 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:17.190667 20464 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.840s	user 0.013s	sys 0.002s
I20260709 12:13:17.182547 20487 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:17.188023 20327 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:17.178180 20202 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:13:17.220302 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 128938us
I20260709 12:13:17.226405 20518 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:17.228199 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 43654.9us
I20260709 12:13:17.228945 20571 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:17.210004 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:17.246328 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 86982.1us
I20260709 12:13:17.183394 20561 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:13:17.265443 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 47411.5us
I20260709 12:13:17.171905 20393 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:17.250655 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 65088.2us
I20260709 12:13:17.253472 12646 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:17.258776 12679 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:17.259181 20555 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:17.261003 20557 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:17.276147 12677 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:17.231081 20530 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:17.277921 12679 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21. Ops: []
W20260709 12:13:17.283578 20566 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:13:17.284991 12572 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:17.285658 20545 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:17.286082 20545 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:17.287134 20545 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:17.288232 20396 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:17.256349 20202 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:17.293335 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 948) took 1296 ms. Trace:
I20260709 12:13:17.293751 12525 rpcz_store.cc:276] 0709 12:13:15.996858 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:16.010202 (+ 13344us) service_pool.cc:224] Handling call
0709 12:13:16.023150 (+ 12948us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:16.537828 (+514678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:17.293086 (+755258us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b5400053554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:17.293271 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1182300032}
I20260709 12:13:17.299088 12678 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:13:17.251231 20487 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:17.257089 12600 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
I20260709 12:13:17.257964 20464 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:17.309468 12605 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:17.310829 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:17.311630 13331 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 19: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:17.314157 20487 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:17.314306 20464 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:17.300438 20243 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 22.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.024s
I20260709 12:13:17.261754 20491 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:17.256247 12673 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:17.327080 12680 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:17.280125 20170 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:17.335583 13305 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:17.334916 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:17.288260 20545 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:17.338433 12673 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:13:17.342964 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 133103us
I20260709 12:13:17.256433 20556 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:17.304047 20561 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:13:17.345065 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 91666.1us
I20260709 12:13:17.344645 20215 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:13:17.348276 20215 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:17.348711 20215 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:17.350319 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:17.313953 20530 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:17.370042 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:17.341177 20560 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:17.326273 20393 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:17.381929 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 97018.2us
I20260709 12:13:17.300803 20557 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:17.352560 20243 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:17.359356 20565 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:13:17.413079 20399 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:17.413620 20399 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:17.414600 20399 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.10, Last appended by leader: 10, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:17.415900 20399 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:17.383435 20170 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:17.400086 13305 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:13:17.429962 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:17.443384 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 47921.9us
W20260709 12:13:17.444383 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 44395.3us
I20260709 12:13:17.346771 20555 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:17.448596 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 92657.9us
I20260709 12:13:17.460997 12673 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:13:17.419153 20081 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:17.431636 20565 raft_consensus.cc:697] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:17.471035 12646 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:17.397480 20243 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:17.398830 20560 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:13:17.475633 12676 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.563s	user 0.012s	sys 0.000s
I20260709 12:13:17.482453 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:17.483358 20393 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:17.484431 20563 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.155s	user 0.001s	sys 0.001s
I20260709 12:13:17.499380 12676 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:17.462329 20491 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:17.499177 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:17.424012 20215 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:17.503191 12528 raft_consensus.cc:2410] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:13:17.505602 12523 raft_consensus.cc:2376] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23. Re-sending same reply.
I20260709 12:13:17.436901 20487 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:17.509840 12572 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:13:17.515930 12425 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47524 (request call id 102) took 18748 ms (client timeout 15000 ms). Trace:
I20260709 12:13:17.515744 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:17.517771 20587 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:17.519341 20215 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:17.520274 20215 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:17.520737 20215 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:17.526511 20243 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:17.516469 12425 rpcz_store.cc:269] 0709 12:12:58.772383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:58.772654 (+   271us) service_pool.cc:224] Handling call
0709 12:12:59.086221 (+313567us) spinlock_profiling.cc:243] Waited 19.4 ms on lock 0x7b54000104d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148898a090 00007f1488a28f3a 00007f148e7dee9c 00007f148e8dd04e 00007f148cd9e2a4 00007f148cd9e232 00007f148cd9e161 00007f148cd9e0dc 00007f148cd9d392 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:17.515643 (+18429422us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:13.363608 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:13.364349 (+   741us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:13.364407 (+    58us) write_op.cc:435] Acquired schema lock
0709 12:13:13.364442 (+    35us) tablet.cc:662] Decoding operations
0709 12:13:13.365985 (+  1543us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:13.369454 (+  3469us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:13.380707 (+ 11253us) tablet.cc:685] Acquiring locks for 10 operations
0709 12:13:13.502954 (+122247us) tablet.cc:701] Row locks acquired
0709 12:13:13.503006 (+    52us) write_op.cc:260] PREPARE: finished
0709 12:13:13.503209 (+   203us) write_op.cc:270] Start()
0709 12:13:13.503392 (+   183us) write_op.cc:276] Timestamp: P: 1783599193503139 usec, L: 0
0709 12:13:13.503429 (+    37us) op_driver.cc:348] REPLICATION: starting
0709 12:13:13.522817 (+ 19388us) log.cc:844] Serialized 3984 byte log entry
0709 12:13:14.217265 (+694448us) op_driver.cc:464] REPLICATION: finished
0709 12:13:14.723127 (+505862us) write_op.cc:301] APPLY: starting
0709 12:13:14.750847 (+ 27720us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:15.258062 (+507215us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:15.258219 (+   157us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:15.732156 (+473937us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:15.732754 (+   598us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=20,key_file_lookups=20,delta_file_lookups=20,mrs_lookups=0
0709 12:13:15.732868 (+   114us) write_op.cc:312] APPLY: finished
0709 12:13:15.733765 (+   897us) log.cc:844] Serialized 97 byte log entry
0709 12:13:16.049747 (+315982us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:16.061361 (+ 11614us) write_op.cc:454] Released schema lock
0709 12:13:16.118956 (+ 57595us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:16.170820 (+ 51864us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:16.170870 (+    50us) write_op.cc:454] Released schema lock
0709 12:13:16.898520 (+727650us) spinlock_profiling.cc:243] Waited 5.87 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
Metrics: {"mutex_wait_us":7042749,"reported_tablets":21,"spinlock_wait_cycles":44563200,"child_traces":[["op",{"apply.queue_time_us":385336,"apply.run_cpu_time_us":54030,"apply.run_wall_time_us":1476133,"cfile_cache_hit":42,"cfile_cache_hit_bytes":45867,"num_ops":10,"prepare.queue_time_us":102463,"prepare.run_cpu_time_us":17629,"prepare.run_wall_time_us":393168,"raft.queue_time_us":101542,"raft.run_cpu_time_us":19018,"raft.run_wall_time_us":286198,"replication_time_us":679338,"spinlock_wait_cycles":13493632,"thread_start_us":736561,"threads_started":5,"wal-append.queue_time_us":1068742,"wal-append.run_cpu_time_us":10783,"wal-append.run_wall_time_us":1171636}]]}
I20260709 12:13:17.521553 20215 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:17.529438 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:17.532276 20565 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 17.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:17.533738 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:17.535988 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:17.536513 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3632) took 1013 ms. Trace:
I20260709 12:13:17.536839 12522 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:17.538210 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:17.536998 12598 rpcz_store.cc:276] 0709 12:13:16.523598 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:16.523828 (+   230us) service_pool.cc:224] Handling call
0709 12:13:16.524221 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:16.612600 (+ 88379us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:17.536272 (+923672us) spinlock_profiling.cc:243] Waited 75.8 ms on lock 0x7b54002e5ed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:17.536454 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":174355328}
I20260709 12:13:17.547828 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:17.548959 12524 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 19: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:17.500922 20385 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:17.505213 20491 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:17.545957 19539 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:17.560388 20531 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:17.586828 20491 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:17.452893 20561 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:17.593500 20554 consensus_meta.cc:287] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.745s	user 0.006s	sys 0.007s
I20260709 12:13:17.576082 20476 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 20.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.030s
W20260709 12:13:17.632879 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 157405us
I20260709 12:13:17.635293 20487 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:17.635855 20560 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:17.635931 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 266019us
I20260709 12:13:17.636107 20556 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:17.603878 20582 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.237s
I20260709 12:13:17.639691 20587 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:13:17.649426 20579 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.208s	user 0.003s	sys 0.000s
I20260709 12:13:17.603587 20202 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:17.650682 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:17.651055 20557 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:17.651046 20357 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:13:17.654678 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:17.673327 12672 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:17.658349 12600 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:17.674026 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:17.599298 20531 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:17.660543 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:17.661346 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:17.662345 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:17.664445 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 279967us
I20260709 12:13:17.664582 12598 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:17.678615 12572 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:17.673064 20530 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:17.682650 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:17.686583 12603 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:13:17.676949 20531 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:17.724440 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:17.726526 12603 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
W20260709 12:13:17.690229 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 154195us
I20260709 12:13:17.682272 20557 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:13:17.731106 20591 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
W20260709 12:13:17.735584 12531 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 3086) took 2903 ms (client timeout 1594 ms). Trace:
W20260709 12:13:17.788784 12531 rpcz_store.cc:269] 0709 12:13:14.831814 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:14.857659 (+ 25845us) service_pool.cc:224] Handling call
0709 12:13:17.735302 (+2877643us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b54003cc8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:17.735471 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3719949184}
I20260709 12:13:17.738021 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:17.739425 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:17.791203 12672 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:17.793202 20491 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:17.743201 12677 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:13:17.743463 20202 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:17.750415 20555 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:17.712641 20556 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:17.752143 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:17.754009 12496 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:17.804644 13331 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:17.758172 19539 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:17.806177 19539 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:17.787089 20557 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:17.808094 20491 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:17.764853 20476 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:13:17.766294 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 87781.6us
I20260709 12:13:17.809018 19539 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:17.760031 20526 consensus_meta.cc:287] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.788s	user 0.008s	sys 0.004s
I20260709 12:13:17.809363 20476 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:17.809883 20526 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:17.810577 12572 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:17.810495 20476 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:17.811611 20556 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:17.811986 20476 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:17.812282 20526 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:17.769766 12602 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:13:17.771522 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:17.813057 12572 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:17.767550 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:17.814517 20582 raft_consensus.cc:2764] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:13:17.794152 20588 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:17.815452 20588 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:17.815680 20526 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:17.816473 20588 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:17.816970 12572 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:17.817844 20588 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:17.799923 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:17.818955 12572 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:17.819411 12569 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:17.801891 12524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:13:17.820237 20556 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:17.820484 12524 raft_consensus.cc:606] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:17.820849 20556 raft_consensus.cc:697] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:17.821064 20243 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:17.821750 20243 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:17.821897 20556 consensus_queue.cc:237] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:17.823050 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:13:17.813982 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 99885.6us
I20260709 12:13:17.774523 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:17.826694 12603 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:17.739675 12571 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:17.822383 20243 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:17.808828 20557 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:13:17.830667 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 91076.8us
I20260709 12:13:17.831568 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:17.832927 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:17.833899 20464 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:13:17.834352 20555 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:17.834710 20464 raft_consensus.cc:697] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:17.835047 20555 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:17.835168 20399 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:13:17.835825 20399 raft_consensus.cc:697] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:17.835933 20464 consensus_queue.cc:237] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 16.9, Last appended by leader: 9, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:17.836129 20555 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:17.836772 20399 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 17.15, Last appended by leader: 15, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:17.709048 20571 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:17.819890 20526 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:13:17.840137 20571 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:17.842610 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:17.843896 20518 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:17.844425 20518 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:17.844858 20518 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:13:17.845095 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:17.849220 12524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:13:17.850107 12524 raft_consensus.cc:606] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:17.850591 12524 raft_consensus.cc:614] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:13:17.850257 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 222581us
I20260709 12:13:17.829891 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:17.857326 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 103400us
I20260709 12:13:17.861756 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:17.863397 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:17.877324 12531 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:17.887310 12528 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:17.876158 12646 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:17.891747 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:17.855270 20081 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:17.861824 20582 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:13:17.897076 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 78246.6us
I20260709 12:13:17.862761 19539 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:17.861826 20500 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:17.900275 12603 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:13:17.905051 12523 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: []
W20260709 12:13:17.920616 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 66181.2us
I20260709 12:13:17.849274 20243 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:17.896723 20599 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:17.925050 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 95309.2us
I20260709 12:13:17.925737 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:17.928258 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:17.930750 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:17.924365 12646 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:17.945013 13336 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:17.898833 20531 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:17.927474 20500 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:17.936707 13333 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:17.910725 20081 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:17.939045 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:17.938875 20602 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:17.939752 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:17.941046 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:17.960950 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:17.942541 12522 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:13:17.911993 20379 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:17.955909 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:17.972892 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:17.954941 12646 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:17.976035 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:13:17.998847 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 44000.6us
I20260709 12:13:18.000085 20385 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:13:18.001111 20081 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:18.001408 20592 raft_consensus.cc:3055] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:17.976784 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:17.979528 12526 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 22. Current term is 23. Ops: [22.18-22.18]
I20260709 12:13:17.957594 20599 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:17.979779 20500 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:18.005143 12671 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 16 index: 7. Preceding OpId from leader: term: 20 index: 9. (index mismatch)
I20260709 12:13:18.006304 20599 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:18.006867 13332 raft_consensus.cc:2410] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 23 because replica has last-logged OpId of term: 22 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 17.
I20260709 12:13:17.979948 20243 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:17.923643 19539 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:18.009426 20243 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260709 12:13:17.927052 20215 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:17.984874 12605 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 26. Current term is 27. Ops: []
I20260709 12:13:17.984956 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:17.987964 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:17.989889 12672 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:17.993232 20379 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:17.997326 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:18.001816 20592 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:18.010906 12529 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:18.012583 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:17.954524 20582 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:18.018162 13336 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:18.021085 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:18.034852 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 16 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:18.035892 12524 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:18.039965 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:18.041193 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:18.041837 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:18.042744 12523 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:18.043970 12530 raft_consensus.cc:2410] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
W20260709 12:13:18.045609 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 77679.3us
I20260709 12:13:18.048749 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:17.998915 20594 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:13:18.020289 20582 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:18.051573 12531 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
W20260709 12:13:18.059199 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 121612us
I20260709 12:13:18.059841 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:18.009892 19539 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.066166 12496 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:18.039570 20215 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:18.064292 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:18.042626 20587 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:18.077057 12601 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:18.075789 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:18.073254 12529 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:18.046391 20599 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:18.065346 20340 consensus_queue.cc:1059] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:13:18.074283 12494 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:18.078085 12522 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:18.079021 12672 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:13:18.066790 20420 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 12:13:18.091722 20420 raft_consensus.cc:3055] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:13:18.092204 20420 raft_consensus.cc:740] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:18.091876 20592 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:18.093400 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:18.093151 20420 consensus_queue.cc:260] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:18.093637 20592 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:13:18.102537 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 128553us
I20260709 12:13:18.094823 20612 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:18.095275 20600 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:18.106371 20612 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:18.106542 20600 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:18.107100 20600 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:18.107525 12572 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:18.107383 20612 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.108762 20612 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:18.108804 12572 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:18.111754 12572 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:18.095635 20327 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:13:18.096310 20526 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:18.097375 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:18.097612 20556 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:18.097971 20560 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:18.097831 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:18.098423 20611 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:18.100903 20420 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:18.102074 12522 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:13:18.105894 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:18.105576 12643 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:18.095543 12674 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:13:18.086736 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:18.094841 12680 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
W20260709 12:13:18.116300 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:18.122323 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:18.059464 20582 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:18.124756 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:18.126735 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:18.046450 20243 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:13:18.122220 20516 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:18.161917 12675 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 23: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:18.122422 20555 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:18.161909 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:18.168452 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:18.127568 20396 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:13:18.174144 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 49505.2us
I20260709 12:13:18.178085 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:18.178395 20215 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:18.112061 20081 consensus_queue.cc:1175] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" responder_term: 24 status { last_received { term: 23 index: 14 } last_committed_idx: 14 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:13:18.186507 20530 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:18.187980 20243 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:18.154844 20473 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.034s
I20260709 12:13:18.189932 19826 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:18.190443 19826 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:18.190626 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 85142.6us
I20260709 12:13:18.191969 19826 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.192715 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:18.193528 19826 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:18.141628 20526 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:18.192808 20595 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 10, Time since last communication: 0.003s
I20260709 12:13:18.188472 20243 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:18.196725 20526 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:18.188201 20531 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:18.169021 20587 raft_consensus.cc:2764] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
W20260709 12:13:18.198637 20603 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.258s	user 0.000s	sys 0.005s
I20260709 12:13:18.199674 20599 raft_consensus.cc:993] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: : Instructing follower f5fc807f7ecc4d1ca0fd6be524f734ce to start an election
I20260709 12:13:18.200160 20599 raft_consensus.cc:1081] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Signalling peer f5fc807f7ecc4d1ca0fd6be524f734ce to start an election
I20260709 12:13:18.201114 20571 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:18.201660 20595 raft_consensus.cc:3055] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:18.168553 20357 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.034s
I20260709 12:13:18.202072 20595 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:18.203191 12643 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:18.203411 20357 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:13:18.203948 20357 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:18.204437 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:18.204839 20357 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:18.204476 20587 raft_consensus.cc:2764] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:13:18.209127 13338 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:18.209347 20557 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:18.204093 20595 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:18.213846 20608 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.166s	user 0.001s	sys 0.000s
W20260709 12:13:18.209152 20613 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.114s	user 0.000s	sys 0.001s
I20260709 12:13:18.137598 20556 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:18.218312 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 44947.5us
I20260709 12:13:18.133235 20379 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:18.221539 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 43774.1us
I20260709 12:13:18.246104 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:18.178299 20532 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:18.251521 12572 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:18.242648 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:18.248726 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:18.240365 20571 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Leader pre-election won for term 17
I20260709 12:13:18.237068 20530 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.024s
I20260709 12:13:18.256402 20571 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:18.256836 20571 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 16 FOLLOWER]: Advancing to term 17
W20260709 12:13:18.262760 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 58382.2us
I20260709 12:13:18.265058 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:18.265904 12603 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:18.268154 20557 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:18.271185 20556 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:18.249799 20357 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:18.279289 12680 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:13:18.280189 20531 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:13:18.238212 20396 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:18.281920 20567 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.050s
I20260709 12:13:18.282764 20396 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:18.295866 20530 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:18.248555 20600 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.296847 20530 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.299376 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:18.300895 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:18.280609 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:18.302330 20587 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:18.299399 20600 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:18.303539 20385 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:18.196743 20243 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.303936 12643 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:18.304169 20385 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:18.304986 20487 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:18.305667 20487 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:18.306828 20487 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:18.304141 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:18.302732 20170 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:18.310050 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:18.327811 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 76414.3us
I20260709 12:13:18.329771 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:18.305032 20243 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:18.305151 20385 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:18.283632 20567 raft_consensus.cc:3055] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:18.345283 12524 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:13:18.352200 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:18.354875 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:18.356909 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:18.357435 20582 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.358448 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:18.360141 20582 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:18.367807 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:18.293426 20396 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:18.372087 12674 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:18.376036 12572 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:18.335446 20556 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:18.381008 12646 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:18.382678 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:18.377806 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 220278us
I20260709 12:13:18.386008 12572 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:18.338094 20531 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:18.345618 20567 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:18.389520 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:18.345865 20555 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:18.391945 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:18.397950 12605 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:18.372692 20526 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:18.377354 20590 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:13:18.377241 20170 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:18.331735 20518 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.399269 12572 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:18.404860 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:18.410823 12524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:13:18.411586 12524 raft_consensus.cc:606] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:18.414427 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:18.415045 20170 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:18.420548 20516 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:18.421060 20516 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:18.421402 20516 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:18.423969 20518 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:18.415093 12527 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:18.391825 20396 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:18.391597 20464 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:13:18.427497 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 284390us
I20260709 12:13:18.396986 20582 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:18.431012 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:18.404147 20600 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:18.399880 20382 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:18.437216 12680 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:18.407660 20590 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:18.391506 20491 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:13:18.446059 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:18.447016 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:18.449085 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:18.452963 12522 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 15 index: 8. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:13:18.455255 20202 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:13:18.461333 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:18.430824 20170 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:18.464941 12680 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:18.433857 20555 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:18.435930 20518 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:18.494899 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 113979us
I20260709 12:13:18.410375 20526 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.412988 20531 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.506295 12527 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:18.444954 20590 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:18.502898 12646 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:18.511487 12526 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:13:18.479094 20170 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:18.521921 12646 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:18.529724 12494 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:18.448894 20600 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:18.541028 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:18.524408 12530 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:18.554257 12674 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:18.530534 20560 raft_consensus.cc:993] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: : Instructing follower f5fc807f7ecc4d1ca0fd6be524f734ce to start an election
I20260709 12:13:18.521809 20587 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election won for term 17
I20260709 12:13:18.565908 12674 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:13:18.530364 20170 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:18.486577 20622 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.074s
I20260709 12:13:18.571205 12528 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:18.512346 20555 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.574412 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:18.545586 20473 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:18.515187 20518 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.546943 20530 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:18.553640 20385 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:18.522486 20621 consensus_queue.cc:1059] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.064s
I20260709 12:13:18.587805 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:18.569953 20623 raft_consensus.cc:1081] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Signalling peer f5fc807f7ecc4d1ca0fd6be524f734ce to start an election
I20260709 12:13:18.573683 20582 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:18.600800 20530 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:13:18.576453 20587 raft_consensus.cc:697] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:18.586582 20382 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:13:18.615307 20341 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:18.631659 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 109806us
I20260709 12:13:18.614655 12679 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:18.576105 20379 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:13:18.684687 20595 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:18.611709 19539 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.083s
I20260709 12:13:18.632647 20341 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.660899 12605 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
W20260709 12:13:18.660897 20619 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 12:13:18.668962 20243 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:18.668047 20531 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:18.692333 20243 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:18.693553 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:18.667049 20600 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:18.695592 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:18.696043 12523 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:13:18.695781 20567 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:18.696856 20567 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:18.669242 12530 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:13:18.697809 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:18.698267 12530 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:13:18.698942 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:18.699553 12523 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:18.699988 20555 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:18.700302 20615 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.612s	user 0.004s	sys 0.000s
I20260709 12:13:18.650566 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:18.694459 12677 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:18.703958 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:18.644068 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:18.704586 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:18.705171 12672 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 19: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:18.705098 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:18.705725 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:18.719221 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:18.706135 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:18.706779 20526 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:18.707340 20600 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:18.709084 20385 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:18.702081 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:18.697026 20612 consensus_meta.cc:287] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.588s	user 0.010s	sys 0.002s
I20260709 12:13:18.705998 12673 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:18.698947 12530 raft_consensus.cc:606] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:18.732975 12530 raft_consensus.cc:614] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:18.668396 20557 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:18.721489 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:18.725689 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:18.727221 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:18.728601 20357 consensus_queue.cc:1059] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:13:18.729439 20476 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:13:18.729732 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:18.730435 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:18.722005 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:18.709779 20592 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.607s	user 0.011s	sys 0.002s
I20260709 12:13:18.711911 20243 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:18.619798 20620 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.076s	user 0.001s	sys 0.001s
I20260709 12:13:18.760751 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:18.639770 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:18.774880 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:18.735432 13338 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:13:18.748483 12527 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:13:18.748965 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:18.749425 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:18.749989 20357 raft_consensus.cc:3055] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:18.733808 12677 raft_consensus.cc:2393] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 23: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:18.794955 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:18.797845 12602 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
I20260709 12:13:18.798024 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:18.733870 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:18.616451 12494 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:13:18.797078 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:18.799216 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:18.798985 12522 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:18.806903 12605 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 21: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:13:18.809901 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 193537us
I20260709 12:13:18.811513 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:18.812182 12604 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:18.811749 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:18.812880 12599 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:18.813971 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:18.812898 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:18.820185 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:18.653337 20532 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:18.830811 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:18.831736 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:18.834288 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:18.837896 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 19 candidate_status { last_received { term: 17 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:18.839721 12572 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:18.846246 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 145780us
W20260709 12:13:18.886725 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 157079us
W20260709 12:13:18.887405 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 47757.1us
I20260709 12:13:18.740985 20379 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:18.885365 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:18.890832 12680 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 19: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:18.795077 20526 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:18.751673 20557 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.802940 20382 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:18.900115 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:18.915508 13331 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 22: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:18.797881 20357 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:18.616503 20611 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.281s
I20260709 12:13:18.839740 20624 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:18.830262 20243 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:18.913512 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:18.916426 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:18.925031 12602 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 24: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:18.910763 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:18.829917 20385 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:18.888054 20065 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:18.649461 20571 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:18.745903 20600 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.922138 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:18.923024 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 202935us
I20260709 12:13:18.898957 20530 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:18.894855 20396 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
I20260709 12:13:18.920643 20382 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.929505 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:18.930039 12673 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:18.922267 20624 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:18.898787 20379 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.945528 12674 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 16. Current term is 17. Ops: []
I20260709 12:13:18.931411 20635 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:18.932308 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:18.838946 20531 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.082s
I20260709 12:13:18.940353 20530 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:18.952087 20530 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:18.953130 20557 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:18.950784 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:18.653784 20587 consensus_queue.cc:237] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 15.9, Last appended by leader: 9, Current term: 17, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.884295 20264 consensus_queue.cc:1059] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.079s
I20260709 12:13:18.884943 20592 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.040s
I20260709 12:13:18.906037 20630 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.050s
I20260709 12:13:18.926522 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:18.936394 13304 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:18.924247 20526 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:18.961097 20564 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:13:18.961939 13304 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:18.962669 20526 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:18.962759 20564 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:13:18.946225 20624 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:18.964643 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:18.964834 20264 raft_consensus.cc:3055] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:18.965308 20264 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:18.966641 20571 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:18.905352 20582 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:18.968315 20630 consensus_queue.cc:1243] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 20.11 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 10
I20260709 12:13:18.968994 20473 raft_consensus.cc:3055] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:18.969475 20473 raft_consensus.cc:740] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:18.933171 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:18.970553 20473 consensus_queue.cc:260] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.971761 13339 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:13:18.972501 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:18.973101 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3110) took 1069 ms. Trace:
I20260709 12:13:18.973765 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:18.975486 12603 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
W20260709 12:13:18.977706 20487 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.670s	user 0.013s	sys 0.000s
I20260709 12:13:18.977854 12604 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 17.
I20260709 12:13:18.947754 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:18.939458 12671 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:18.959226 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:18.950363 12673 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 17 index: 12. Preceding OpId from leader: term: 19 index: 13. (index mismatch)
I20260709 12:13:18.983188 12606 raft_consensus.cc:2410] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 23 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 12:13:18.966207 20264 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.968729 20631 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:18.959352 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 16 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:18.978229 20487 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.984344 12494 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:18.973482 12528 rpcz_store.cc:276] 0709 12:13:17.903737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:17.903881 (+   144us) service_pool.cc:224] Handling call
0709 12:13:17.925087 (+ 21206us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:18.577151 (+652064us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:18.972870 (+395719us) spinlock_profiling.cc:243] Waited 646 ms on lock 0x7b54003caad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:18.973034 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1485209984}
I20260709 12:13:18.985710 20564 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:18.980225 12603 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:51782
I20260709 12:13:18.988514 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:18.967885 20582 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.993188 12603 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:18.990989 20631 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.955688 20357 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:18.995731 20600 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:18.993288 12600 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:19.019768 20631 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:19.020539 20530 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.991430 20473 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:19.023149 20170 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:18.999917 20382 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:19.005479 20264 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:19.028436 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 77687.1us
I20260709 12:13:19.029863 20382 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:19.007241 12603 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:19.030851 12675 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:13:19.029527 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:18.933723 20385 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:19.037081 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:19.037880 12680 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:19.038622 12679 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:13:19.001550 20379 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:19.074605 12673 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 15 index: 14. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
W20260709 12:13:19.075142 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 62951.6us
W20260709 12:13:19.054886 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:19.016265 19539 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:19.037932 12496 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:19.074496 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:19.036562 20571 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:19.103448 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:19.083840 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 54409.5us
I20260709 12:13:19.037365 20170 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:19.105388 20571 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:19.109048 12672 raft_consensus.cc:2393] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 23: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:19.104946 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:19.036317 20464 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:19.111552 12571 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:19.109516 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:19.113090 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:19.113343 12527 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:13:19.114239 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:19.118682 20630 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:19.120237 12604 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:51782
I20260709 12:13:19.121114 12604 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:19.121590 12604 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:19.114936 20555 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:19.112685 20571 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:19.037650 20632 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:19.126588 20632 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.129400 20632 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:19.041471 20341 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:19.077306 20516 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.620s	user 0.005s	sys 0.008s
I20260709 12:13:19.130537 20611 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:19.130817 20516 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:19.131779 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 93934.9us
I20260709 12:13:19.133212 20516 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:19.037202 20564 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:19.134641 12601 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:19.029407 20590 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:19.143002 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:19.145543 12680 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:19.037006 20382 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.156106 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:19.154520 12496 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:19.144641 12604 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.159149 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:19.115453 20396 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:19.164836 12677 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 28. Current term is 29. Ops: []
I20260709 12:13:19.166946 12604 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:19.174633 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:19.175707 12679 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:13:19.186416 12679 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:19.187604 12597 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
W20260709 12:13:19.186774 20636 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.154s	user 0.002s	sys 0.000s
I20260709 12:13:19.108946 20531 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.051s
I20260709 12:13:19.176566 12603 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.193621 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:19.148129 20592 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:19.192608 12604 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:13:19.156281 20357 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:19.163702 20623 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:19.134243 20564 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:19.204398 12600 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
W20260709 12:13:19.194177 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 87756us
I20260709 12:13:19.212065 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:19.212311 12679 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 16 index: 9. Preceding OpId from leader: term: 18 index: 10. (index mismatch)
I20260709 12:13:19.162727 20396 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:19.152004 20635 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:19.116129 20464 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.195611 20530 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:19.216698 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:19.186334 20516 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:19.233757 20516 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.236445 20516 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:19.237653 20634 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:19.238744 20632 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:19.239383 20632 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.241262 20382 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:19.242395 12603 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:19.246443 20170 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:19.248050 20632 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:19.221318 20518 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:19.220700 12604 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:19.222919 20531 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:19.223923 20623 raft_consensus.cc:697] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:19.230233 20564 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:19.208069 20582 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:19.261668 20582 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:19.262473 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:19.263439 12604 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:19.264534 20582 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:19.265265 20464 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:19.214854 20396 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:19.266450 20623 consensus_queue.cc:237] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.246495 20634 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:19.260543 20531 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.264565 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:19.272455 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:19.273509 12604 raft_consensus.cc:2410] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 22 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:13:19.274936 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:19.218184 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 102861us
W20260709 12:13:19.220805 20567 consensus_meta.cc:287] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.523s	user 0.008s	sys 0.004s
I20260709 12:13:19.249449 20614 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:13:19.259860 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:19.282711 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:19.284468 20487 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:19.277877 20396 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:19.284222 12571 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:19.298025 12494 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:19.296847 13335 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:19.273488 20531 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:19.302656 12494 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:19.303329 20464 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:19.303805 20464 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:19.303853 19826 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:19.304312 20464 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:19.305711 20611 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:19.306205 20611 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:19.306628 20611 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:19.294977 20590 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:19.301332 20595 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:13:19.292320 20624 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:19.316968 12524 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:19.327884 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:19.362483 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:19.363188 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:19.364845 12571 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:19.372442 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:19.373747 12531 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:13:19.376415 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:19.379474 12603 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:13:19.382414 13304 leader_election.cc:336] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.849s (SENT)
I20260709 12:13:19.382941 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:19.379674 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:19.383044 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:19.324887 20396 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:19.391661 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:19.393829 12571 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:19.394132 12526 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:13:19.397615 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:19.379333 20612 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:19.379920 20531 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:19.325634 20571 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.388844 20644 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:19.401613 20526 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:19.419473 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:19.400905 20643 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:19.362139 20460 consensus_queue.cc:1059] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:13:19.401307 20565 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:19.379047 20555 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 12:13:19.384812 20500 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 12:13:19.429418 12530 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:13:19.388979 20532 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.026s
W20260709 12:13:19.445431 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:19.454013 20464 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:19.458069 20385 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:19.463261 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:19.464303 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:19.463840 20464 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:19.415918 20396 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:19.478644 20396 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:19.479153 20396 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:19.438616 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:19.482661 20460 raft_consensus.cc:3055] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:19.483160 20460 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:19.484030 20460 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.436947 20526 raft_consensus.cc:697] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:19.486426 20526 consensus_queue.cc:237] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.429456 20382 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:19.443405 13338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38472 (request call id 869) took 1188 ms. Trace:
W20260709 12:13:19.456427 12671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 3791) took 1373 ms (client timeout 1500 ms). Trace:
I20260709 12:13:19.472388 12601 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:13:19.474171 20464 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:13:19.485380 20460 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:19.491149 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:19.491683 12524 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 22 index: 11. (index mismatch)
I20260709 12:13:19.384341 20556 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.029s
I20260709 12:13:19.492864 12531 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:13:19.464347 12599 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 26: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:19.502831 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:19.506989 13338 rpcz_store.cc:276] 0709 12:13:18.254561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:18.254789 (+   228us) service_pool.cc:224] Handling call
0709 12:13:19.443083 (+1188294us) spinlock_profiling.cc:243] Waited 416 ms on lock 0x7b54004439d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:19.443270 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":955773312}
I20260709 12:13:19.518712 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:19.507390 12671 rpcz_store.cc:269] 0709 12:13:18.082509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:18.084812 (+  2303us) service_pool.cc:224] Handling call
0709 12:13:19.456213 (+1371401us) spinlock_profiling.cc:243] Waited 690 ms on lock 0x7b540015dbd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:19.456329 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1586988416}
I20260709 12:13:19.521055 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:19.522874 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:19.520313 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:19.526870 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:19.523576 13332 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:19.531405 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:19.534905 12496 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:19.545537 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:19.524842 20645 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.088s	user 0.000s	sys 0.000s
I20260709 12:13:19.562507 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:19.564145 12599 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 12:13:19.564724 12599 pending_rounds.cc:85] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 12
I20260709 12:13:19.567428 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:19.567618 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:19.568894 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:19.570218 12601 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22.
I20260709 12:13:19.570278 12672 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 23: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:19.572230 12601 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 22 index: 17. (term mismatch)
I20260709 12:13:19.519905 20594 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:19.565081 12599 pending_rounds.cc:107] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 12:13:19.532809 20385 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:19.573832 12599 raft_consensus.cc:2889] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:19.574210 12601 pending_rounds.cc:85] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 16
I20260709 12:13:19.572146 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:19.515419 20635 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:13:19.536651 20487 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:13:19.586099 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 67886.9us
I20260709 12:13:19.534711 20590 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:13:19.542945 20532 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:19.576529 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: true dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:19.555275 20634 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:19.557441 20624 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:19.576085 12601 pending_rounds.cc:107] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 21.17
I20260709 12:13:19.599222 12601 raft_consensus.cc:2889] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:19.532668 20602 consensus_queue.cc:1059] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 12:13:19.589318 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:19.593519 12671 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 23: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:19.521461 20382 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:19.596839 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:19.558157 20632 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:13:19.597998 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:19.600016 12599 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:19.605608 20634 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.610335 20357 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:19.610888 20357 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:19.611364 20357 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:19.601032 20635 consensus_queue.cc:1243] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:13:19.615732 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:19.615468 20611 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.618942 20611 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:19.626740 20654 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.050s	user 0.000s	sys 0.000s
W20260709 12:13:19.627944 20648 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:13:19.633526 20382 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:19.650897 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:19.651780 12678 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:13:19.642061 20661 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:13:19.661401 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: true dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:19.665279 12671 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:19.661819 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:19.662451 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:19.603953 20532 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:19.667151 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:19.667905 13338 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:13:19.674265 13335 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 17 index: 15. Preceding OpId from leader: term: 19 index: 16. (index mismatch)
I20260709 12:13:19.669410 12569 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:19.677299 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:19.660351 20635 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:19.642063 20590 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:19.678733 20635 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.679585 20590 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.682582 20635 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:19.684221 20396 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.686551 12603 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 21: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:19.690378 12569 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:19.690713 12671 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:19.658190 20385 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:19.692377 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 862) took 1091 ms. Trace:
I20260709 12:13:19.625581 20357 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:19.696794 13334 raft_consensus.cc:2368] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24.
I20260709 12:13:19.681980 20590 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:19.698251 12680 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:19.694515 12571 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:19.695036 12674 rpcz_store.cc:276] 0709 12:13:18.601468 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:18.617764 (+ 16296us) service_pool.cc:224] Handling call
0709 12:13:18.618230 (+   466us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:18.954483 (+336253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:19.692145 (+737662us) spinlock_profiling.cc:243] Waited 327 ms on lock 0x7b54000924d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:19.692319 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":752151808}
I20260709 12:13:19.703442 12678 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:19.704985 12571 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:19.708477 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:19.692802 20630 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:13:19.704171 20644 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
W20260709 12:13:19.739355 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:19.704706 20518 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:19.747707 12674 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:13:19.701027 20385 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.717217 20530 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:19.750761 20530 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:19.752523 20385 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:19.752291 20530 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:19.759737 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:19.760896 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:19.774818 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:19.776250 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:19.779327 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:19.705018 20555 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 12:13:19.776901 12675 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:13:19.786904 12674 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:19.717061 20531 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:19.786890 19539 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:19.798413 12529 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:13:19.799909 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:19.796974 12675 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:19.710047 20382 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.068s
I20260709 12:13:19.807725 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:19.814937 12524 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:19.815908 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:19.794370 20630 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:19.804795 19539 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.725800 20264 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.082s
I20260709 12:13:19.810876 12496 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:19.831349 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:19.832386 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: true dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:19.841631 12525 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:19.841658 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:19.842491 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:19.799549 12531 pending_rounds.cc:85] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 10
I20260709 12:13:19.806357 12674 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:13:19.835240 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:19.841928 12523 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:13:19.843830 12531 pending_rounds.cc:107] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 20.11
I20260709 12:13:19.844000 12523 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:19.844347 12531 raft_consensus.cc:2889] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:19.843196 12527 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 23: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:19.838413 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: true dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:19.852267 12524 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:19.852751 12524 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:19.835630 20555 consensus_queue.cc:1243] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 22.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:13:19.812153 20531 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:19.890658 20531 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:19.846102 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:19.895884 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:19.905352 20614 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:19.905815 20587 raft_consensus.cc:3055] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:19.906548 20587 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:19.842905 12526 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:19.795727 20644 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:19.802912 20621 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:19.855265 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:19.907915 20587 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.855755 12674 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:19.803511 20396 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:19.840026 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:19.860316 20357 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:19.914681 12523 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.777371 20532 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:19.861454 20500 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:19.863212 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 868) took 1217 ms. Trace:
I20260709 12:13:19.867352 12525 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:19.876070 12676 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:13:19.880775 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:19.884512 12422 catalog_manager.cc:5697] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 18 to 21, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 21 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:13:19.887109 20665 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.126s	user 0.000s	sys 0.000s
I20260709 12:13:19.887812 20660 raft_consensus.cc:3055] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:19.913529 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:19.935906 12528 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:19.860781 20555 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:19.962067 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:19.982817 12602 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:13:19.963424 12569 leader_election.cc:400] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24.
I20260709 12:13:19.984231 12569 leader_election.cc:403] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:13:19.964556 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 127303us
I20260709 12:13:19.967382 12523 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:19.964989 12677 rpcz_store.cc:276] 0709 12:13:18.645920 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:18.916432 (+270512us) service_pool.cc:224] Handling call
0709 12:13:18.916850 (+   418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:19.176080 (+259230us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:19.862924 (+686844us) spinlock_profiling.cc:243] Waited 248 ms on lock 0x7b5400092ed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:19.863134 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":569858432}
I20260709 12:13:19.941396 20621 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.963726 20644 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:19.976866 12526 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:13:19.967754 20518 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:19.989468 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:20.002827 12603 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:20.004712 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:20.005405 12604 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:13:19.967280 20500 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:20.006093 12572 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:19.964375 20587 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:20.010623 12525 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:13:20.005543 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:19.952301 20564 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.057s
W20260709 12:13:20.113132 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 195765us
I20260709 12:13:19.976256 20660 raft_consensus.cc:740] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:20.038607 12525 raft_consensus.cc:606] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:20.018567 12606 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 24: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:20.030201 12523 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:20.038776 12597 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
W20260709 12:13:20.137074 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 131079us
I20260709 12:13:19.967650 20476 consensus_queue.cc:1059] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 16.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.051s
I20260709 12:13:20.140122 12572 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:20.141223 20516 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:13:20.169509 12572 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:20.112075 20500 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:20.182243 12572 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:20.208807 12572 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:20.210433 20516 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:13:20.211032 20668 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.224s	user 0.000s	sys 0.000s
I20260709 12:13:20.212101 20674 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:20.212917 20674 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:20.216245 20674 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:20.221408 20674 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:20.222440 20674 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:20.252725 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 441859us
I20260709 12:13:20.255640 20526 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:20.256729 20526 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:20.265419 20526 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:20.294953 20526 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:20.295732 20526 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:20.310446 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:20.312759 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:20.330329 12422 catalog_manager.cc:5697] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 17 to 23. New cstate: current_term: 23 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:20.332804 20526 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:20.333062 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:20.332556 12422 catalog_manager.cc:5697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 23 to 28. New cstate: current_term: 28 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:20.333868 13335 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:20.335168 12569 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:20.336009 20526 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:20.336499 20526 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:20.336927 20526 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:20.341694 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3696) took 1039 ms. Trace:
I20260709 12:13:20.342250 12605 rpcz_store.cc:276] 0709 12:13:19.301834 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.476450 (+174616us) service_pool.cc:224] Handling call
0709 12:13:19.476719 (+   269us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:19.976200 (+499481us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:19.976234 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:19.976282 (+    48us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:19.976335 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:20.341416 (+365081us) spinlock_profiling.cc:243] Waited 499 ms on lock 0x7b5400310bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:20.341468 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:20.341625 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1148242176}
I20260709 12:13:20.342941 12646 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:20.345144 20243 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:20.345660 20243 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:20.346633 20243 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:20.348877 20571 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:20.349411 20243 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:20.354928 20634 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:20.364993 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:20.184935 20555 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:13:20.373405 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 119307us
I20260709 12:13:20.370563 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 976) took 1004 ms. Trace:
I20260709 12:13:20.380154 20526 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:20.332510 12524 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:20.375584 12530 rpcz_store.cc:276] 0709 12:13:19.365569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.365780 (+   211us) service_pool.cc:224] Handling call
0709 12:13:19.397156 (+ 31376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:19.444631 (+ 47475us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:20.360949 (+916318us) spinlock_profiling.cc:243] Waited 25.8 ms on lock 0x7b540027fbd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:20.370223 (+  9274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":59352832}
I20260709 12:13:20.412547 12524 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:20.367023 20516 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:20.010913 20630 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:20.419382 12494 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:20.455067 20679 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.270s	user 0.001s	sys 0.000s
W20260709 12:13:20.460485 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:20.463140 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49005.9us
W20260709 12:13:20.412949 20202 consensus_queue.cc:1175] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "193a40da6e364ead8ab561e14c3cc8d1" responder_term: 23 status { last_received { term: 22 index: 14 } last_committed_idx: 14 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:13:20.460959 12524 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:20.191766 20673 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:20.459357 20341 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:20.487017 19539 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:20.464593 20674 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:20.489862 19539 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:20.470199 20420 raft_consensus.cc:3055] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:20.474934 20672 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:20.500247 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 323628us
I20260709 12:13:20.466936 20340 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.033s
I20260709 12:13:20.536033 13333 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 23 index: 17. (index mismatch)
I20260709 12:13:20.500212 20420 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:20.546960 20682 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
W20260709 12:13:20.548919 20678 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.078s	user 0.000s	sys 0.000s
I20260709 12:13:20.497750 20341 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:20.550904 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3228) took 1040 ms. Trace:
I20260709 12:13:20.551990 20341 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:20.551306 12672 rpcz_store.cc:276] 0709 12:13:19.510503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.662394 (+151891us) service_pool.cc:224] Handling call
0709 12:13:19.662797 (+   403us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:19.663359 (+   562us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:19.663397 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:19.663996 (+   599us) log.cc:844] Serialized 26 byte log entry
0709 12:13:20.002738 (+338742us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:20.002789 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:20.002869 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:20.550647 (+547778us) raft_consensus.cc:1685] finished
0709 12:13:20.550672 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:20.550839 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1681,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":18608,"thread_start_us":338292,"threads_started":1,"wal-append.queue_time_us":786138}
I20260709 12:13:20.556016 20420 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:20.543493 20357 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:20.559101 20531 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.668s	user 0.009s	sys 0.001s
I20260709 12:13:20.558810 20357 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:20.559612 20531 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:20.543280 20621 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:20.565061 12602 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.580s	user 0.006s	sys 0.005s
I20260709 12:13:20.565508 12602 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
I20260709 12:13:20.566035 20644 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:20.557885 20420 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:20.572813 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3234) took 1039 ms. Trace:
I20260709 12:13:20.581678 20215 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:20.583273 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:20.591408 12677 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:13:20.582491 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:20.591982 12531 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.002s	sys 0.024s
I20260709 12:13:20.592391 12680 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:13:20.573264 12671 rpcz_store.cc:276] 0709 12:13:19.535656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.695014 (+159358us) service_pool.cc:224] Handling call
0709 12:13:19.695347 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:19.695521 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 9
0709 12:13:19.704521 (+  9000us) log.cc:844] Serialized 14 byte log entry
0709 12:13:19.807143 (+102622us) log.cc:844] Serialized 14 byte log entry
0709 12:13:20.572229 (+765086us) spinlock_profiling.cc:243] Waited 7.18 ms on lock 0x7b180043c8a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:13:20.572433 (+   204us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:20.572489 (+    56us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:20.572556 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:20.572625 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:20.572750 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":742717,"prepare.queue_time_us":14218,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":4494,"raft.queue_time_us":797753,"raft.run_cpu_time_us":318,"raft.run_wall_time_us":15504,"spinlock_wait_cycles":16518912,"thread_start_us":80335,"threads_started":1,"wal-append.queue_time_us":209567}
W20260709 12:13:20.590857 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:20.584153 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:20.593626 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:20.594550 12674 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: []
W20260709 12:13:20.592374 12531 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.002s	sys 0.024s
I20260709 12:13:20.597656 12676 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:20.382855 20660 consensus_queue.cc:260] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:20.593636 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:20.603909 12679 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
W20260709 12:13:20.605319 20683 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.059s	user 0.000s	sys 0.000s
I20260709 12:13:20.623011 20170 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:20.624327 20170 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:20.606796 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:20.626973 12680 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:20.627163 20170 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:20.382313 20518 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:20.631042 20602 raft_consensus.cc:3055] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Stepping down as leader of term 16
I20260709 12:13:20.631616 20602 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 16 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:20.632862 12679 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:20.632582 20602 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 16, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:20.639537 20531 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:20.640690 20526 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:20.640903 20687 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:13:20.611675 12671 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:20.615053 20635 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:20.590617 20627 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:20.617009 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:13:20.617385 20516 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:13:20.618650 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 275812us
I20260709 12:13:20.647095 19539 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:13:20.619247 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:20.620471 20215 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:20.569852 20643 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24.
I20260709 12:13:20.621467 20595 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:20.508261 20672 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:20.622965 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:20.610314 12572 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:20.651365 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:20.665681 12524 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:20.646739 12646 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:20.645427 20635 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:20.654623 20627 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:20.658671 12603 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 22. Current term is 23. Ops: [22.17-22.17]
W20260709 12:13:20.659072 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:20.692370 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 45774.8us
I20260709 12:13:20.659509 12671 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
W20260709 12:13:20.663136 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:20.665942 12600 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:20.699435 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:20.700268 12527 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:20.701460 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:20.668466 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 58251.4us
I20260709 12:13:20.702821 12572 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:20.680197 12677 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:20.704510 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:20.705543 12677 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 20: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:20.706525 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:20.707398 12530 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
I20260709 12:13:20.711042 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:20.712819 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:20.714984 13333 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:20.715099 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:20.716051 13332 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:20.717444 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:20.672118 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:20.692370 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 69521.2us
I20260709 12:13:20.718710 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:20.718837 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:20.717579 12572 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:20.651995 20516 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:20.720999 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 17 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:20.721920 12646 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:20.669575 20602 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 16 FOLLOWER]: Advancing to term 17
I20260709 12:13:20.715677 13333 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:20.662598 20460 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:20.731801 19539 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:13:20.733894 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 61914.7us
I20260709 12:13:20.733307 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:20.742007 12527 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:20.687480 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 268827us
I20260709 12:13:20.693802 20531 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:20.729080 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:20.739316 12524 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
I20260709 12:13:20.719560 20592 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:20.748124 12528 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 12:13:20.748533 12528 pending_rounds.cc:85] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 12
I20260709 12:13:20.748751 12528 pending_rounds.cc:107] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 12:13:20.748986 12528 raft_consensus.cc:2889] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:20.719794 20385 raft_consensus.cc:2764] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:13:20.756466 12571 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:20.735170 20487 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:20.739491 12671 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 11. Preceding OpId from leader: term: 23 index: 12. (index mismatch)
I20260709 12:13:20.729168 20630 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:20.721035 20170 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:13:20.751539 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:20.768500 12526 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:20.752067 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:20.737566 20631 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:20.729455 20674 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:20.740005 20476 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:20.757596 19539 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:20.780611 19539 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:20.666313 20564 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.074s
I20260709 12:13:20.758373 12679 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:20.736096 20357 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:20.691162 20595 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:20.720646 20594 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:20.750273 13333 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:20.770165 20487 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:20.800195 20170 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.008s
I20260709 12:13:20.810256 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3819) took 1063 ms. Trace:
I20260709 12:13:20.810441 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3149) took 1014 ms. Trace:
I20260709 12:13:20.823024 12531 rpcz_store.cc:276] 0709 12:13:19.796482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.799014 (+  2532us) service_pool.cc:224] Handling call
0709 12:13:19.799350 (+   336us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:19.973539 (+174189us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:19.973748 (+   209us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:19.974633 (+   885us) log.cc:844] Serialized 52 byte log entry
0709 12:13:20.598033 (+623400us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:20.615805 (+ 17772us) log.cc:844] Serialized 14 byte log entry
0709 12:13:20.616243 (+   438us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:20.616324 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:20.810204 (+193880us) raft_consensus.cc:1685] finished
0709 12:13:20.810227 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:20.810375 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":17219,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":205,"raft.run_cpu_time_us":178,"raft.run_wall_time_us":181,"thread_start_us":622846,"threads_started":1,"wal-append.queue_time_us":798352}
I20260709 12:13:20.738070 20081 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.050s
I20260709 12:13:20.740101 20571 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 25.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.048s
I20260709 12:13:20.820215 12494 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:20.830838 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 109911us
I20260709 12:13:20.747407 20460 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:20.832363 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 96588.5us
I20260709 12:13:20.786545 20592 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader election lost for term 17. Reason: could not achieve majority
W20260709 12:13:20.847700 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 130219us
I20260709 12:13:20.841886 12643 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:20.843459 12646 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:20.821496 12678 rpcz_store.cc:276] 0709 12:13:19.746555 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.746704 (+   149us) service_pool.cc:224] Handling call
0709 12:13:19.799130 (+ 52426us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:20.579768 (+780638us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:20.579810 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:20.580342 (+   532us) log.cc:844] Serialized 26 byte log entry
0709 12:13:20.740562 (+160220us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:20.740623 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:20.740700 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:20.810000 (+ 69300us) raft_consensus.cc:1685] finished
0709 12:13:20.810022 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:20.810186 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":784244,"raft.queue_time_us":772136,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":9983,"thread_start_us":159753,"threads_started":1,"wal-append.queue_time_us":227675}
I20260709 12:13:20.826946 19539 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:20.853057 13332 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:20.854446 12569 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:20.858692 12527 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
W20260709 12:13:20.850723 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 94361.1us
I20260709 12:13:20.849937 12572 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:20.820581 20487 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:20.835879 20644 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:20.775568 20630 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:20.879105 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 58973.8us
I20260709 12:13:20.906257 12494 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:20.907727 19539 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:20.865705 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:20.815054 20476 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 24 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:20.838709 20555 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:20.900739 12680 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
W20260709 12:13:20.910707 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 45092us
W20260709 12:13:20.911178 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47810.6us
W20260709 12:13:20.978649 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:20.990424 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 84247.1us
I20260709 12:13:20.837399 20688 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 20.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.112s
I20260709 12:13:20.903667 20592 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:20.907760 20571 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:20.902936 20644 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:20.999962 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 56036.8us
I20260709 12:13:20.880203 20341 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.127s
I20260709 12:13:20.913620 20555 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:20.863448 20532 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.134s
I20260709 12:13:20.907356 20202 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:13:20.813992 20594 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:21.011104 13305 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:13:21.009399 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:20.992748 20690 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:21.037868 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 187998us
I20260709 12:13:21.002249 20612 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:13:21.029467 20587 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.905s	user 0.012s	sys 0.000s
W20260709 12:13:21.050086 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:21.015172 20500 consensus_meta.cc:287] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.810s	user 0.005s	sys 0.007s
I20260709 12:13:21.007854 20700 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:21.031145 20644 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:21.013090 20571 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:21.050817 20555 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:21.061982 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 45392.8us
I20260709 12:13:21.073084 20710 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:21.061822 20692 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:21.059628 20531 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:21.081194 20635 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:21.087198 20460 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:21.034422 20170 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 12:13:21.052937 20564 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:21.114173 20564 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:21.115648 12530 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 12:13:21.116757 20564 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:21.117445 20564 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:21.117843 20564 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:21.118423 20564 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:21.111097 20692 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:21.119204 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:21.120486 20692 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:21.121444 20516 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:21.050565 20215 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:21.059267 20672 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:21.139535 20703 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:21.140252 20703 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:21.059945 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:21.064100 20592 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:21.141466 20592 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:21.113114 20706 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:13:21.127960 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:21.128481 20696 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.216s	user 0.000s	sys 0.000s
I20260709 12:13:21.128223 20564 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:21.129488 20695 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:13:21.110091 20689 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:21.130450 20698 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.097s	user 0.001s	sys 0.000s
I20260709 12:13:21.074054 20627 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:21.090360 20700 raft_consensus.cc:697] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:21.060990 20702 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:21.151533 20565 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:13:21.151993 20518 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:13:21.153486 20630 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:21.155105 20594 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:21.155704 20595 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:21.156582 20680 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.690s	user 0.005s	sys 0.000s
I20260709 12:13:21.157608 20170 consensus_queue.cc:1243] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 24.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:13:21.168548 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:21.221959 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:21.222633 12531 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:13:21.222823 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:21.223817 12523 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:13:21.223903 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:21.224442 12524 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:13:21.224566 12526 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:21.225206 12523 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 23. Current term is 24. Ops: [23.11-23.11]
I20260709 12:13:21.225369 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:21.227159 12524 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 14 index: 8. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
W20260709 12:13:21.151116 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:21.188261 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 185969us
I20260709 12:13:21.140712 20516 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:21.093394 20614 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:21.233470 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:21.155750 20621 raft_consensus.cc:993] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: : Instructing follower f5fc807f7ecc4d1ca0fd6be524f734ce to start an election
I20260709 12:13:21.235802 12675 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 12:13:21.196244 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:21.077533 20500 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:21.065128 20476 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:21.244015 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:21.244975 12528 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:21.245431 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:21.246299 12530 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 23: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:21.158982 20706 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:21.064319 20690 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:21.194442 20587 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:21.218247 20564 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:21.219244 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 159489us
I20260709 12:13:21.220489 12674 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:21.215456 20702 raft_consensus.cc:697] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:21.238111 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:21.211545 20710 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:21.262036 12494 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:21.247493 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:21.268241 12531 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
W20260709 12:13:21.259034 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 316071us
I20260709 12:13:21.274672 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:21.225394 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:21.240900 20688 consensus_queue.cc:1243] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 21.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:13:21.288085 12572 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:21.287120 12529 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 21: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:13:21.294219 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 404398us
I20260709 12:13:21.239460 20592 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:21.243798 20703 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:21.202409 20689 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:21.200258 20571 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:13:21.306599 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 44659.1us
I20260709 12:13:21.277246 20718 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:21.276161 20643 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:21.270153 20565 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:21.261224 20623 raft_consensus.cc:1081] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Signalling peer f5fc807f7ecc4d1ca0fd6be524f734ce to start an election
I20260709 12:13:21.309573 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:21.272341 20706 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:21.340728 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:21.298498 20594 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:21.240278 20614 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:21.357448 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 47924.6us
I20260709 12:13:21.308374 20595 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:21.289752 20587 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:21.239624 20531 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:21.347728 20703 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:21.338074 20592 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:21.402338 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:21.337723 20718 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:13:21.388809 12494 connection.cc:256] RPC call timeout handler was delayed by -1.0084400549999373s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1715s for a response
I20260709 12:13:21.403887 12526 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:21.377766 12600 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 12. (term mismatch)
I20260709 12:13:21.403265 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:21.414649 20674 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.016s	sys 0.036s
I20260709 12:13:21.419932 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:21.420498 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:21.437601 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:21.413153 20698 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.071s	user 0.000s	sys 0.000s
I20260709 12:13:21.428560 12600 pending_rounds.cc:85] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 11
I20260709 12:13:21.444211 12600 pending_rounds.cc:107] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 22.12
I20260709 12:13:21.444722 12600 raft_consensus.cc:2889] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:13:21.421380 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 51797.4us
I20260709 12:13:21.427003 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:21.404125 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:21.342300 20643 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:21.354204 20700 consensus_queue.cc:237] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:21.438867 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:21.426472 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 125935us
W20260709 12:13:21.440629 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:21.417537 20680 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:13:21.416908 20711 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.080s
I20260709 12:13:21.452224 13335 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 27: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:21.455124 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:21.461527 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:21.422260 20703 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:21.460711 13339 raft_consensus.cc:2368] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21.
I20260709 12:13:21.458039 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:21.349414 20565 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:21.437002 20674 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.525s	user 0.016s	sys 0.036s
I20260709 12:13:21.457746 20643 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:21.498857 20709 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:21.499585 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 44641.7us
I20260709 12:13:21.412161 20564 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.051s
I20260709 12:13:21.500486 20722 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:21.501005 20587 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:21.502318 20693 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:13:21.503343 20715 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:13:21.358404 20594 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:21.455389 20621 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:21.544783 13305 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:21.499940 20699 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:21.552682 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:21.557169 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:21.567312 12528 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:21.578454 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3147) took 1782 ms. Trace:
I20260709 12:13:21.578603 13332 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:21.471859 20703 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:21.425522 20595 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:21.467422 20202 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:21.589956 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:21.591059 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 186800us
I20260709 12:13:21.437069 20592 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:21.572861 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:21.591341 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:21.701666 12531 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:13:21.702592 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:21.607154 12601 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc"
dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:51782
I20260709 12:13:21.608721 12526 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:21.701666 12530 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
W20260709 12:13:21.687237 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 142521us
I20260709 12:13:21.572608 20722 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:21.715212 12494 leader_election.cc:400] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21.
I20260709 12:13:21.791599 12494 leader_election.cc:403] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:21.792337 19539 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21.
W20260709 12:13:21.792524 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 77538.8us
W20260709 12:13:21.715613 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:21.716732 12603 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 22: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:13:21.606206 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 159806us
I20260709 12:13:21.609064 12522 rpcz_store.cc:276] 0709 12:13:19.795916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.798338 (+  2422us) service_pool.cc:224] Handling call
0709 12:13:19.798636 (+   298us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:19.808470 (+  9834us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:19.808500 (+    30us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:19.848334 (+ 39834us) log.cc:844] Serialized 52 byte log entry
0709 12:13:19.995472 (+147138us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:20.232299 (+236827us) log.cc:844] Serialized 14 byte log entry
0709 12:13:20.373358 (+141059us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:20.373443 (+    85us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:21.126033 (+752590us) spinlock_profiling.cc:243] Waited 2.57 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:21.568697 (+442664us) raft_consensus.cc:1685] finished
0709 12:13:21.578147 (+  9450us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:21.578376 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":179788,"prepare.queue_time_us":185081,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":17098,"raft.queue_time_us":150495,"raft.run_cpu_time_us":386,"raft.run_wall_time_us":32529,"spinlock_wait_cycles":5909760,"thread_start_us":292213,"threads_started":2,"wal-append.queue_time_us":368694}
I20260709 12:13:21.800546 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:21.801610 12603 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 21: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:21.803434 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:21.803850 12531 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:21.721302 20587 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.100s
I20260709 12:13:21.821188 20663 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:21.684026 20621 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:21.822082 20663 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:21.826748 12524 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:21.713842 20734 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:21.876847 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:21.877180 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:21.877645 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:21.824923 20202 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:21.799260 20722 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:21.503814 20710 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:21.465188 20702 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:21.829066 20500 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:21.896982 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 103433us
I20260709 12:13:21.866609 20672 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:21.926154 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:21.581881 20709 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:21.930065 12527 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
W20260709 12:13:21.940486 12529 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 3162) took 2106 ms (client timeout 2331 ms). Trace:
I20260709 12:13:21.890379 20202 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:13:21.941195 12529 rpcz_store.cc:269] 0709 12:13:19.814298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:19.895432 (+ 81134us) service_pool.cc:224] Handling call
0709 12:13:21.919928 (+2024496us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b54003c1754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:21.920120 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3197121920}
I20260709 12:13:21.715432 20699 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:21.899282 19826 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:21.953289 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:21.949608 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:21.949961 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:21.910394 20740 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:21.897609 20340 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:21.943576 20663 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:21.897990 20621 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:21.971246 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 64927.6us
I20260709 12:13:21.968122 12672 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 20.
I20260709 12:13:21.911018 20690 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:21.956154 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:21.608031 20718 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:21.946713 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 152025us
I20260709 12:13:21.976444 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:21.977236 12673 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22.
I20260709 12:13:21.958688 20670 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:13:21.987962 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:21.991379 12646 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:21.891642 20734 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:21.990092 20693 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:21.998987 12646 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:21.971858 20614 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:21.944756 20672 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:21.963410 20460 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:13:21.977154 20565 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:21.968452 19826 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.026465 20733 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 22.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.036s
I20260709 12:13:22.027987 20735 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.117s
I20260709 12:13:22.028750 20728 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:22.029870 20753 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:22.030573 20737 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:13:22.031474 20739 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.157s	user 0.000s	sys 0.000s
I20260709 12:13:22.034951 20709 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:22.035907 20725 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.207s	user 0.000s	sys 0.000s
I20260709 12:13:22.039031 20663 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:22.041874 20487 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.134s	user 0.011s	sys 0.000s
I20260709 12:13:21.987111 20664 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:22.042781 20723 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.214s	user 0.000s	sys 0.001s
I20260709 12:13:22.002698 20594 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:22.005201 20670 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:21.982506 20732 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:22.006753 20719 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:13:21.997766 20673 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:22.062254 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:22.046509 20644 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.921s	user 0.004s	sys 0.008s
I20260709 12:13:22.097941 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:22.193619 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 180718us
I20260709 12:13:22.062791 20749 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:13:22.118563 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 131577us
I20260709 12:13:22.059109 20753 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:22.114606 20592 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:22.163681 12526 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:22.101114 20737 raft_consensus.cc:697] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 21 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:22.169817 12601 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:13:22.227526 12601 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:22.052958 20744 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.134s
I20260709 12:13:21.978503 20740 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.118381 20557 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:22.058701 20728 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:22.127712 20602 consensus_meta.cc:287] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.388s	user 0.013s	sys 0.002s
I20260709 12:13:22.201467 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:22.203156 12605 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:13:22.199932 20531 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:22.203742 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:22.203967 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:22.318009 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:22.204234 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:22.221383 20594 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:22.217832 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:22.119992 20487 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:22.115952 20663 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:22.343586 20663 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:22.127640 20733 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:22.225036 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:22.344877 20487 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:22.344962 20733 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:22.345854 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:22.345961 20487 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:22.347041 12671 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:13:22.346698 20487 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:22.146906 20709 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.080s
I20260709 12:13:22.347877 20733 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:22.348132 20709 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:22.348462 20734 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:22.348685 20709 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.349398 20487 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:22.349547 20699 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:22.214797 20555 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.056s	user 0.010s	sys 0.004s
I20260709 12:13:22.205910 20644 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:22.168785 20719 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:22.353991 20719 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:22.355406 20719 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:13:22.285015 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:22.237303 20670 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:22.236575 20763 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:22.311949 12528 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.629s	user 0.008s	sys 0.004s
I20260709 12:13:22.239342 20737 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:22.244823 20750 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:22.234772 20748 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:22.205001 20757 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:22.315052 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:22.203063 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:22.168617 20732 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.316661 12422 catalog_manager.cc:5697] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 19 to 22. New cstate: current_term: 22 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:22.317639 12601 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.215463 20749 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:22.168540 20674 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s	user 0.003s	sys 0.009s
I20260709 12:13:22.214382 20673 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.356719 20500 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:22.331352 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:22.333324 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:22.337352 12527 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:13:22.337386 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:22.338600 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:22.347468 20755 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
I20260709 12:13:22.335186 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:22.346732 12526 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 21: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:22.124976 20664 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:22.385058 12600 raft_consensus.cc:2393] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:13:22.346159 20754 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.176s	user 0.000s	sys 0.000s
W20260709 12:13:22.378350 20674 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.812s	user 0.003s	sys 0.009s
I20260709 12:13:22.350472 20555 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.355543 20644 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:22.368196 12531 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:22.368233 12599 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
W20260709 12:13:22.361994 20706 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.006s	user 0.008s	sys 0.005s
I20260709 12:13:22.330826 20592 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:22.383054 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:22.441475 12527 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:13:22.324486 20557 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:22.381285 12604 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:13:22.363525 20672 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:22.372516 20722 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.335502 20531 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:22.435578 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:22.455156 12605 raft_consensus.cc:2410] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 18 because replica has last-logged OpId of term: 15 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 14 index: 8.
I20260709 12:13:22.436319 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:22.378644 20749 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:22.364758 12528 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:13:22.437247 19826 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:22.457055 19539 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:22.352144 20594 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:22.465447 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:22.444346 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:22.466437 20664 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:22.449681 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:22.439927 20460 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:22.486578 12604 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:22.453872 20766 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:22.434973 20750 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:22.443380 20761 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:22.386288 20500 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.373950 20763 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.380044 20757 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:22.512969 20757 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:13:22.517352 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3258) took 1227 ms. Trace:
I20260709 12:13:22.517822 12523 rpcz_store.cc:276] 0709 12:13:21.290391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:21.290563 (+   172us) service_pool.cc:224] Handling call
0709 12:13:21.291026 (+   463us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:21.595847 (+304821us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:21.595906 (+    59us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:21.630504 (+ 34598us) log.cc:844] Serialized 26 byte log entry
0709 12:13:21.896480 (+265976us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:13:21.896614 (+   134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:21.897041 (+   427us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:22.517088 (+620047us) raft_consensus.cc:1685] finished
0709 12:13:22.517112 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:22.517284 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":53754,"raft.queue_time_us":67859,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":53834,"thread_start_us":310819,"threads_started":2,"wal-append.queue_time_us":282330}
I20260709 12:13:22.499647 20473 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:22.455896 20592 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.452073 20706 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.534363 20709 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:22.529349 20757 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:22.492959 20460 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.510006 20763 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:22.550798 12572 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 2.331s (SENT) [suppressed 14 similar messages]
I20260709 12:13:22.500398 20750 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.378607 20748 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.538329 20757 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.578769 20757 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:22.558347 20767 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:13:22.564970 20500 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:22.568410 20592 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:22.569182 20740 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:22.570027 20728 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:22.536741 20473 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:22.571655 20643 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.990s	user 0.006s	sys 0.006s
I20260709 12:13:22.573673 20750 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:22.573797 20460 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:22.575547 20531 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:22.576800 20673 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:22.500725 20761 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.580235 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:22.571877 20706 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:22.580616 20753 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.583642 20473 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:22.554047 20763 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:22.596446 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:22.584386 20643 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:22.600997 12679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3251) took 1944 ms (client timeout 1715 ms). Trace:
W20260709 12:13:22.623131 20202 consensus_meta.cc:287] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.669s	user 0.011s	sys 0.000s
I20260709 12:13:22.623844 20202 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:22.625890 20722 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:22.628785 20732 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:22.629767 20557 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:22.630678 20555 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:22.632893 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:22.633586 20643 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:22.634315 20720 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:13:22.635907 20720 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:22.636309 20720 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:22.636683 20720 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:22.592849 20357 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:13:22.601662 12679 rpcz_store.cc:269] 0709 12:13:20.656409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:20.664537 (+  8128us) service_pool.cc:224] Handling call
0709 12:13:22.600798 (+1936261us) spinlock_profiling.cc:243] Waited 39.9 ms on lock 0x7b54001414d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:22.600927 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":91859712}
W20260709 12:13:22.650995 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:22.647284 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:22.496493 20766 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.663883 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:22.681833 12677 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 21: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:22.664578 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:22.682546 12674 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
W20260709 12:13:22.638031 20703 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.860s	user 0.002s	sys 0.009s
I20260709 12:13:22.642329 20753 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:22.708003 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:22.713251 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 14 index: 8 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:22.715409 12677 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:22.713171 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:22.717176 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 166425us
I20260709 12:13:22.689270 20357 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:22.717391 12675 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
W20260709 12:13:22.723332 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:22.725651 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:22.730962 20600 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:22.731387 20611 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:22.731544 20600 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.731956 20611 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:22.734099 20760 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:22.734438 20611 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:22.743178 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:22.744202 12678 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:22.599902 20763 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.753194 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:22.706583 20690 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:22.727813 20357 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:22.708789 20703 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.722093 20643 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:22.773563 12598 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:22.776592 20703 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:22.768111 20766 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:22.768599 20763 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:22.769583 20600 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:22.791589 20748 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:22.810411 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 76261.1us
I20260709 12:13:22.734911 20760 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.785651 20690 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:22.773602 20643 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:22.818532 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:22.820457 12496 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:22.822258 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:22.824590 12524 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:13:22.830562 20760 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:22.830746 20532 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:22.871490 12496 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:13:22.866082 20719 consensus_meta.cc:287] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.502s	user 0.012s	sys 0.000s
I20260709 12:13:22.831091 20664 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.036s
I20260709 12:13:22.883665 20700 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:22.888185 20643 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:22.832500 20729 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:13:22.892761 20202 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:22.894511 20624 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:22.899032 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:22.899369 20693 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:22.899670 12601 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:22.899068 12496 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:22.915802 12496 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:22.900882 20690 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:22.901988 12598 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3717) took 1529 ms (client timeout 1755 ms). Trace:
I20260709 12:13:22.866421 20532 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:22.910065 13335 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:22.920719 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:51782 (request call id 3723) took 1308 ms. Trace:
I20260709 12:13:22.903362 20357 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:22.919728 12598 rpcz_store.cc:269] 0709 12:13:21.372335 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:21.378159 (+  5824us) service_pool.cc:224] Handling call
0709 12:13:22.901777 (+1523618us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54002ffb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:22.901921 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3051049856}
I20260709 12:13:22.921110 12601 rpcz_store.cc:276] 0709 12:13:21.606748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:21.606980 (+   232us) service_pool.cc:224] Handling call
0709 12:13:22.898429 (+1291449us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b5400271ad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a17a3d 00007f148d9e7cd0 00007f1486f6c254 00007f1486f6c1e2 00007f1486f6c111 00007f1486f6c08c 00007f1486f6b342 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:22.914909 (+ 16480us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1070429440}
W20260709 12:13:22.923524 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 91324.4us
I20260709 12:13:22.908078 20700 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:22.923681 20532 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:22.920502 20692 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:22.919253 20693 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:22.939417 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:22.937361 20202 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:22.940111 12679 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:22.940153 20202 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:22.946035 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:22.948882 12496 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:22.927325 20749 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.950284 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:22.918799 20624 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.938973 20692 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:22.942571 20693 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:22.946934 20202 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:22.965939 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:22.966782 12602 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:13:22.956532 20341 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:22.968020 20750 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:22.971477 20750 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:22.967924 20341 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:22.974676 20341 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:22.960549 20692 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:22.969354 20732 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:22.977308 20732 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.979488 20732 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:22.980679 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3849) took 2279 ms. Trace:
I20260709 12:13:22.981093 12671 rpcz_store.cc:276] 0709 12:13:20.701431 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:20.762415 (+ 60984us) service_pool.cc:224] Handling call
0709 12:13:20.762748 (+   333us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:22.347490 (+1584742us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:22.980378 (+632888us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b54003d4fd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:22.980584 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3642880384}
I20260709 12:13:22.982342 20750 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:22.343462 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:22.987120 20624 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:22.986541 12422 catalog_manager.cc:5697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 21 to 23, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 23 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:13:22.988816 12422 catalog_manager.cc:5697] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 10 to 24. New cstate: current_term: 24 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:22.991972 20720 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:22.993129 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3722) took 1395 ms. Trace:
I20260709 12:13:22.996297 20594 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:22.998790 12422 catalog_manager.cc:5697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 12 to 22. New cstate: current_term: 22 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:23.000346 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.000974 12671 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:13:23.001739 13335 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:23.003047 12496 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:23.002919 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:22.965794 20766 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.008292 20705 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:23.011124 20587 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:22.967280 20700 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.11, Last appended by leader: 11, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.009871 20744 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.014554 20740 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.015950 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:22.993530 12606 rpcz_store.cc:276] 0709 12:13:21.597747 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:21.598773 (+  1026us) service_pool.cc:224] Handling call
0709 12:13:21.717946 (+119173us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:21.867516 (+149570us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:21.867557 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:21.868210 (+   653us) log.cc:844] Serialized 26 byte log entry
0709 12:13:22.992434 (+1124224us) spinlock_profiling.cc:243] Waited 15.6 ms on lock 0x7b18003e4100. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1
0709 12:13:22.992621 (+   187us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:22.992663 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:22.992753 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:22.992812 (+    59us) raft_consensus.cc:1685] finished
0709 12:13:22.992825 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:22.993066 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68323,"raft.queue_time_us":179991,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":38669,"spinlock_wait_cycles":35773824,"thread_start_us":161348,"threads_started":2,"wal-append.queue_time_us":237191}
I20260709 12:13:23.027474 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.033250 12606 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:23.027320 20740 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.029109 20595 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:23.030558 20720 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:23.039547 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:23.041566 20726 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:23.042026 20726 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:23.033905 12643 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:23.015703 20500 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:13:23.009405 20623 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:23.035813 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:23.047879 12602 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:23.036857 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:23.043072 12572 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:23.045729 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:23.048648 12671 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:23.018678 20766 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:23.050714 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:23.008564 20624 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.035s
I20260709 12:13:23.051295 20732 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:23.036026 20594 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:13:23.053879 20711 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:13:23.018636 20750 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 12:13:23.059183 20532 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.047147 20700 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:23.045325 20726 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:23.066898 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:23.070250 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:23.049839 20623 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:23.046993 20595 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:23.021266 20587 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:23.022565 20744 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.055514 20357 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:23.074031 12671 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:23.056851 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:23.060828 20643 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:23.067375 20634 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
W20260709 12:13:23.061309 20787 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.110s	user 0.000s	sys 0.001s
I20260709 12:13:23.055681 20500 raft_consensus.cc:697] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:23.079788 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:23.021879 20705 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:23.081388 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:23.071221 20630 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:23.087396 13332 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 769) took 1685 ms (client timeout 1579 ms). Trace:
I20260709 12:13:23.085076 12602 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:23.088588 13331 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:13:23.076414 20623 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:13:23.090557 13332 rpcz_store.cc:269] 0709 12:13:21.401812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:21.402007 (+   195us) service_pool.cc:224] Handling call
0709 12:13:23.087174 (+1685167us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b540055ee54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:23.087310 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":355430656}
W20260709 12:13:23.094969 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:23.095324 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:23.091606 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:23.096956 12643 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:23.082085 20170 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:23.105262 12675 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:23.105441 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:23.107151 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.105041 20753 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:23.109325 20732 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:23.110934 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:23.112083 20594 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:23.112525 20594 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:23.112903 20594 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:23.109652 20753 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:23.114006 20753 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:13:23.114847 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:23.111176 20660 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:23.096539 20473 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:23.117558 12597 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 23: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:23.096688 20595 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:23.116255 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.101449 20643 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:23.104158 20634 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:13:23.125696 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:23.121855 12602 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:13:23.096882 20630 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:23.132040 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 113157us
I20260709 12:13:23.135588 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.120947 20750 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:23.122434 20718 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:23.124421 20749 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:23.138197 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:23.126241 20473 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:23.142961 12601 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:23.125380 20740 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:23.126708 20634 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:23.147472 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.152680 12677 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 24: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:23.148327 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.148561 20473 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.149506 20532 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:23.156978 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:23.157259 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.138945 20750 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:23.151114 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.159296 12673 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:13:23.159549 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.160257 12679 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
I20260709 12:13:23.154326 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:23.154851 12674 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 28. Current term is 29. Ops: [28.19-28.19]
I20260709 12:13:23.161258 12671 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:23.161839 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:23.130299 20595 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:23.162740 12677 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:13:23.161139 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:23.171378 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.172142 12679 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:13:23.172966 12646 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:23.165320 20688 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:23.165922 20761 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:23.131400 20341 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.173897 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.180722 12646 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:23.153288 20634 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:23.183164 20794 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.196463 12671 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
I20260709 12:13:23.196982 13304 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:23.199059 12674 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:13:23.160300 20750 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:23.159611 20766 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:23.203280 20766 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:23.203810 20766 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:23.204243 20766 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:23.201006 20795 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:23.207847 20795 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:23.208313 20795 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:23.180435 20595 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:23.213618 20595 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:23.137648 20500 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:23.201130 20794 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:23.218559 20753 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.219856 13304 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:23.180204 20740 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:13:23.224542 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:23.233714 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:23.222915 20081 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.209736 20341 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:23.247874 12677 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:23.249518 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:23.251377 13304 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:23.253937 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:23.250313 20081 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:23.254809 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 72330.9us
I20260709 12:13:23.230228 20357 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:23.247310 20688 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:23.256032 20744 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:23.270994 20081 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:23.257023 20664 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:23.259116 12646 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:23.251999 20736 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:23.255805 20666 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:23.250677 20740 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:23.279644 13333 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:23.267460 20721 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:23.183837 20643 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.282998 20744 raft_consensus.cc:2764] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election decision vote started in defunct term 21: won
I20260709 12:13:23.310948 20744 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:23.311597 20744 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:23.280354 20379 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.314754 20379 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:23.315693 20740 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.283427 20587 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:23.328403 13331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 973) took 1899 ms. Trace:
I20260709 12:13:23.328771 20202 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.268775 20357 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:23.317444 20379 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:23.336503 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 162412us
I20260709 12:13:23.277506 20688 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:23.338898 12530 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 16 index: 11. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:13:23.338238 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:23.346448 12525 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:23.346791 12524 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:23.341471 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:23.329095 13331 rpcz_store.cc:276] 0709 12:13:21.429521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:21.429690 (+   169us) service_pool.cc:224] Handling call
0709 12:13:23.088223 (+1658533us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:23.091610 (+  3387us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:23.328023 (+236413us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b540055ee50. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:23.328289 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3813635968}
I20260709 12:13:23.342525 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:23.280717 20341 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.277112 20664 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:23.343935 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:23.370441 20664 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:23.371523 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:23.373214 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:23.356302 20794 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:23.359015 12524 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:23.364373 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:23.374855 12526 raft_consensus.cc:2393] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 24: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:23.368047 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:23.370813 12523 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 26: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:23.370378 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:23.376528 12528 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:23.374946 12530 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:13:23.377314 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:23.354305 20341 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.336820 20744 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:23.351665 20705 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:23.369407 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:23.378057 12524 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 21: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:23.378155 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:23.380640 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:23.329159 20630 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:23.385066 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.388173 12523 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:13:23.336170 20357 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:23.395669 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:23.396531 12530 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:23.396824 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: true dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:23.401942 12531 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:23.402359 12531 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:23.402959 12677 raft_consensus.cc:2410] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 25 because replica has last-logged OpId of term: 23 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 10.
I20260709 12:13:23.348532 20688 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:23.403017 12528 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:23.403205 12531 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.411830 12531 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:23.414008 12677 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:23.416296 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:23.340039 20692 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.390652 20202 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:23.423622 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:23.406520 20690 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.427479 20693 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.428695 12529 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:23.430220 20802 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:23.430666 20693 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:23.431696 20341 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:23.416182 20766 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.426021 20797 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.010s
I20260709 12:13:23.426851 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.435298 12531 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:23.442811 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.389364 20594 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:23.453967 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 2.15998e+06us
I20260709 12:13:23.463155 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.444960 20630 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:23.475713 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.472458 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:23.476619 12601 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:23.435943 20690 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:23.468576 20671 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.479072 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.468923 20711 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:23.482291 12601 raft_consensus.cc:2410] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 25 because replica has last-logged OpId of term: 22 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 17.
I20260709 12:13:23.482336 12603 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 17 index: 10. (index mismatch)
I20260709 12:13:23.576403 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3244) took 1374 ms. Trace:
I20260709 12:13:23.483672 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.576861 12603 rpcz_store.cc:276] 0709 12:13:22.201852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.203646 (+  1794us) service_pool.cc:224] Handling call
0709 12:13:23.481833 (+1278187us) spinlock_profiling.cc:243] Waited 1.64 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fbeee 000000000036eb39 00007f148d96f1e0 00007f148da380c4 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:13:23.481976 (+   143us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:23.576111 (+ 94135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:23.576338 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3771520}
I20260709 12:13:23.483433 12494 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:23.473771 20766 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:23.579281 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 106900us
W20260709 12:13:23.579903 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 98734.6us
I20260709 12:13:23.571265 20711 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.577247 12597 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:23.477105 20710 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.084s
I20260709 12:13:23.481683 20795 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.571859 13333 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 21 index: 15. (index mismatch)
I20260709 12:13:23.580950 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.583007 12603 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
W20260709 12:13:23.512619 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:23.482085 20761 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:23.470045 20726 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:23.563639 20718 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.472906 20630 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.595417 12601 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 21: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:23.595466 13333 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 884) took 2917 ms. Trace:
I20260709 12:13:23.597602 13333 rpcz_store.cc:276] 0709 12:13:20.670299 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:20.670533 (+   234us) service_pool.cc:224] Handling call
0709 12:13:20.714852 (+ 44319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:23.278900 (+2564048us) spinlock_profiling.cc:243] Waited 496 ms on lock 0x7b5400644c20. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488995a1d 00007f148897c1a4 00007f14889fe447 00007f1488a16fe7 00007f1488a187de 00007f1488a22b49 00007f1488a23416 00007f1488a1fd8f 00007f1488a1f83e 00007f148d9e4419
0709 12:13:23.587003 (+308103us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:23.587231 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1140157696}
I20260709 12:13:23.593920 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:23.613921 20805 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.132s	user 0.000s	sys 0.000s
I20260709 12:13:23.620747 20726 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:23.584993 20473 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election lost for term 18. Reason: could not achieve majority
I20260709 12:13:23.589146 20752 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.629930 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:23.633988 12601 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:23.600183 20170 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:23.571590 20634 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.487398 20671 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:23.639662 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:23.571486 20623 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.645548 12680 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:23.642205 20752 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:23.654546 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:23.669035 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3854) took 2504 ms. Trace:
I20260709 12:13:23.669828 20711 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:23.632879 20799 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.609563 20761 raft_consensus.cc:697] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:23.607411 20693 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:23.638948 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.669497 12676 rpcz_store.cc:276] 0709 12:13:21.164660 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:21.166834 (+  2174us) service_pool.cc:224] Handling call
0709 12:13:22.602476 (+1435642us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:22.621706 (+ 19230us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:22.621742 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:22.622352 (+   610us) log.cc:844] Serialized 26 byte log entry
0709 12:13:22.753337 (+130985us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:22.753379 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:22.753447 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:22.762900 (+  9453us) raft_consensus.cc:1685] finished
0709 12:13:22.762934 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:23.668753 (+905819us) spinlock_profiling.cc:243] Waited 1.43 s on lock 0x7b54001414d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:23.668967 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23370,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"spinlock_wait_cycles":3300433152,"thread_start_us":112346,"threads_started":1,"wal-append.queue_time_us":134682}
I20260709 12:13:23.674360 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:23.678845 12603 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
W20260709 12:13:23.674348 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 93974.9us
W20260709 12:13:23.678056 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:23.679666 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:23.680151 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:23.680469 12606 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:13:23.677979 20721 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.683347 20721 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:23.684438 20671 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:23.684674 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.688429 12603 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:23.693348 20750 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.681567 20799 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:23.688730 20761 consensus_queue.cc:237] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:23.689752 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:23.692838 20664 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:23.685511 20721 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:23.609901 20718 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:23.717552 20718 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:23.695931 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.700976 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.719678 12676 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 17.
I20260709 12:13:23.701077 20594 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:23.704326 20795 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:23.679224 20711 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.044s
I20260709 12:13:23.723574 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.724471 12676 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 29.
I20260709 12:13:23.704600 20802 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:23.705600 20750 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:23.706460 12671 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:23.681684 20692 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:23.682181 20689 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:23.681869 20804 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:13:23.711153 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:23.712934 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:23.714051 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.712024 12675 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:23.728242 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:23.743320 12676 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 20
W20260709 12:13:23.706841 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:23.746573 20623 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:23.699565 20630 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:23.749269 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.751667 20760 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:23.751976 20634 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:23.752230 20760 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:23.751679 20664 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:23.753260 20760 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:23.695429 20693 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:23.757026 20693 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:23.757726 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.758528 12598 raft_consensus.cc:2393] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 22: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:23.742398 12674 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:13:23.759371 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.759549 20623 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:13:23.759738 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 48697.2us
I20260709 12:13:23.759815 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.760478 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:23.760704 12494 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:23.761615 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:23.764644 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:23.760188 20688 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:23.763684 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:23.767287 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.772420 12598 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:23.768303 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.761515 20802 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:23.769348 20760 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:23.765831 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:23.793390 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.714375 20809 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:23.776443 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:23.799597 12676 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:23.804395 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.803965 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.804940 12673 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:23.805881 12603 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 20: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:23.805737 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.800818 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.808022 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.809082 12598 raft_consensus.cc:2368] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24.
I20260709 12:13:23.811486 13333 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:23.813380 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.774070 20671 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.820695 13332 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:23.820376 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:23.813840 12680 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 17: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:23.816461 20799 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:23.817307 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:23.818537 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 47145.8us
I20260709 12:13:23.818902 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.824666 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:23.788885 20420 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:23.775252 20736 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:23.814997 12676 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 29: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:23.798620 20802 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:23.799055 20809 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:23.826560 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:23.825064 20671 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:23.831933 20660 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.713s	user 0.007s	sys 0.007s
I20260709 12:13:23.841074 20671 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:23.841697 20690 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:23.847270 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:23.848737 12531 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:23.849612 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:23.851111 12676 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:23.854969 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:23.855870 12524 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:23.860244 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:23.860888 12523 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:23.863279 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:23.864171 12523 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:23.846498 13334 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:23.866910 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:23.867974 13336 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:13:23.868849 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:23.866910 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 17 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:23.871135 13332 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 17.
I20260709 12:13:23.793941 20357 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:23.882339 12525 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:23.876762 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:23.887032 12524 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:23.886453 12572 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:23.845758 20587 consensus_queue.cc:1059] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.030s
I20260709 12:13:23.836627 20802 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:23.892300 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.866991 20564 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:23.893239 12676 raft_consensus.cc:2410] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 24 because replica has last-logged OpId of term: 23 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 10.
I20260709 12:13:23.835762 20809 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:23.894069 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:23.895714 13333 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 20: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:23.896274 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.895104 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.898067 12605 raft_consensus.cc:2410] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 30 because replica has last-logged OpId of term: 28 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 18.
I20260709 12:13:23.899829 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:23.901238 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.905130 12606 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:23.906895 12599 raft_consensus.cc:2368] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24.
I20260709 12:13:23.902063 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:23.908591 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: true dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:23.909230 12676 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:23.909410 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:23.910063 13335 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
I20260709 12:13:23.905892 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:23.912518 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3730) took 1705 ms. Trace:
I20260709 12:13:23.867316 20379 consensus_queue.cc:1059] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.030s
I20260709 12:13:23.915290 12600 rpcz_store.cc:276] 0709 12:13:22.207521 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.385729 (+178208us) service_pool.cc:224] Handling call
0709 12:13:22.386053 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:22.508543 (+122490us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:22.508576 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:22.508625 (+    49us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:22.508659 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:23.912306 (+1403647us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b5400310bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:23.912350 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:23.912466 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":281249664}
I20260709 12:13:23.920533 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:23.921901 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:23.922806 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:23.875423 20634 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.024s
I20260709 12:13:23.910617 12572 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:23.924414 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:23.907759 20732 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.012s	sys 0.028s
I20260709 12:13:23.906385 20726 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:23.962507 20726 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:23.963025 20726 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:13:23.964680 20810 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.148s	user 0.002s	sys 0.000s
I20260709 12:13:23.926189 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:23.972398 12526 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:23.887666 20630 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.022s
I20260709 12:13:23.927989 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:23.928498 12531 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:23.900020 20761 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:23.891988 20396 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.024s
I20260709 12:13:23.925575 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:23.924656 20746 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:23.900836 20564 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:24.003352 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:23.933275 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:23.933964 20753 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:23.934018 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:23.938366 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.938895 20688 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:23.939538 12680 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:13:23.940148 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.919034 20747 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:13:23.940850 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:23.940994 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.941306 20706 consensus_queue.cc:1059] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:13:23.941700 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:23.942488 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:24.032135 13332 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:23.942603 20379 raft_consensus.cc:3055] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:23.946812 20357 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:23.948498 13304 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:23.949137 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:24.011271 20714 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:23.952021 20812 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:13:23.953843 13337 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:23.948616 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:23.956427 20768 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 1.501s	user 0.001s	sys 0.007s
W20260709 12:13:23.956758 13334 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 972) took 2534 ms (client timeout 1653 ms). Trace:
I20260709 12:13:23.959321 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:24.112111 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 152889us
I20260709 12:13:23.967736 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:23.978153 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:23.978219 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:24.115476 12524 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:23.980020 12676 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:23.984190 12523 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 19: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:23.997725 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:23.999078 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 66755.6us
I20260709 12:13:23.899220 20671 consensus_queue.cc:1059] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 12:13:23.999671 12571 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:24.003157 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:24.117993 12526 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:24.119768 13333 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 24: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:13:23.931515 20732 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.809s	user 0.012s	sys 0.028s
I20260709 12:13:24.003990 12528 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:24.005363 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:24.017825 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:24.006098 20396 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:13:24.164039 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:24.164690 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:24.164994 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:24.165452 12675 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:24.165724 12676 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 26: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:24.043592 12672 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:13:23.983217 20761 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:24.044366 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:24.183295 12530 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
W20260709 12:13:24.048799 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 100439us
I20260709 12:13:24.184888 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:23.950151 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:24.097630 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:24.122747 12531 raft_consensus.cc:2368] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20.
W20260709 12:13:24.110667 13334 rpcz_store.cc:269] 0709 12:13:21.422131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:21.422383 (+   252us) service_pool.cc:224] Handling call
0709 12:13:23.956483 (+2534100us) spinlock_profiling.cc:243] Waited 2.38 s on lock 0x7b54000d7854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:23.956657 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5472553728}
I20260709 12:13:24.138487 13331 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:24.139734 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:24.106652 20747 raft_consensus.cc:697] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:24.141968 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:24.186834 13330 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:24.218704 13333 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:24.155504 20202 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:13:24.230600 13332 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:24.156658 20500 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:24.157539 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:24.164120 12671 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:24.167768 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:24.028946 12679 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:13:24.118579 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:24.170460 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:24.171422 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:24.172346 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:24.245265 12523 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:24.179538 13339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 974) took 2261 ms. Trace:
W20260709 12:13:24.210966 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:24.212809 12524 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:24.221256 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:24.248759 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:24.161628 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:24.233632 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47291.1us
I20260709 12:13:24.252312 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:24.235415 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:24.237948 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:24.042899 20379 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:24.255589 12676 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:24.108709 20688 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:24.259277 12671 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 20: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:13:24.260051 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 260477us
W20260709 12:13:24.260674 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 142194us
I20260709 12:13:24.262043 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:24.095608 20714 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:24.246397 13339 rpcz_store.cc:276] 0709 12:13:21.917825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:21.917967 (+   142us) service_pool.cc:224] Handling call
0709 12:13:21.947954 (+ 29987us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:22.172099 (+224145us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:22.200674 (+ 28575us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:22.201975 (+  1301us) log.cc:844] Serialized 26 byte log entry
0709 12:13:22.329936 (+127961us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:22.329973 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:22.330052 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:23.955717 (+1625665us) spinlock_profiling.cc:243] Waited 2.67 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:24.159759 (+204042us) raft_consensus.cc:1685] finished
0709 12:13:24.159790 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:24.179411 (+ 19621us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10432,"raft.queue_time_us":47106,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":14172,"spinlock_wait_cycles":6147840,"thread_start_us":339627,"threads_started":2,"wal-append.queue_time_us":160001}
W20260709 12:13:24.229514 20821 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:13:24.282680 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:24.217909 20761 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:24.285038 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:24.286901 12675 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:24.263221 20807 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:24.296567 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:24.261238 20736 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:13:24.352178 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 67300.9us
I20260709 12:13:24.358482 13334 raft_consensus.cc:2410] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 30 because replica has last-logged OpId of term: 28 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 18.
I20260709 12:13:24.358542 12571 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:24.359781 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:24.356925 12675 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:13:24.271970 20526 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:24.276822 20688 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:24.360188 12571 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:24.396519 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:24.398608 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:24.399811 20760 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.601s	user 0.010s	sys 0.000s
I20260709 12:13:24.385747 20526 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:24.400811 20526 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:24.292403 20500 raft_consensus.cc:697] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:24.406086 20500 consensus_queue.cc:237] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:24.407939 20379 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:24.300714 20396 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.109s
I20260709 12:13:24.410254 20630 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:24.410465 20396 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:13:24.410879 20396 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:24.411705 20396 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:24.281603 20634 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:24.414556 20634 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:24.416849 20634 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:24.270985 20666 consensus_queue.cc:1059] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.115s
I20260709 12:13:24.417997 20666 raft_consensus.cc:3055] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:13:24.418372 20666 raft_consensus.cc:740] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:24.351882 20753 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:24.428742 12527 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
W20260709 12:13:24.429225 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 69105.9us
I20260709 12:13:24.441680 12571 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:24.443070 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:24.430110 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:24.444350 13335 raft_consensus.cc:2368] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20.
I20260709 12:13:24.431079 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:24.436331 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:24.445540 13331 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 24: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:24.428516 20688 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:24.361557 20807 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:24.451722 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:24.458002 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:24.348582 20671 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.093s
I20260709 12:13:24.388648 20736 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:24.461650 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:24.482719 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:24.483752 12527 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:24.485803 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:24.455742 20634 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:13:24.465621 12604 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3259) took 1513 ms (client timeout 1798 ms). Trace:
I20260709 12:13:24.453984 20807 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:24.496049 12604 rpcz_store.cc:269] 0709 12:13:22.952226 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:22.952371 (+   145us) service_pool.cc:224] Handling call
0709 12:13:24.461452 (+1509081us) spinlock_profiling.cc:243] Waited 2.03 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:13:24.465550 (+  4098us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4660480}
I20260709 12:13:24.461133 20736 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:13:24.514609 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:24.464489 20571 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:24.460044 20396 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:24.532347 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:24.447783 20753 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:24.463796 20746 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:24.495836 20634 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:24.438537 20714 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:24.528869 20807 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:24.447078 20747 consensus_queue.cc:237] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:24.561035 20726 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.598s	user 0.010s	sys 0.003s
I20260709 12:13:24.567515 12527 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
W20260709 12:13:24.564023 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 296126us
I20260709 12:13:24.580487 20726 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:24.583935 12606 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:24.586573 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:24.587935 20726 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:24.588644 20726 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:24.589169 20726 raft_consensus.cc:697] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:24.591913 20726 consensus_queue.cc:237] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:24.607857 20744 raft_consensus.cc:3055] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:24.608616 20744 raft_consensus.cc:740] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:24.615623 20753 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:24.617125 20564 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:24.617792 20564 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:13:24.622454 20746 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:24.626567 20811 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.719s	user 0.006s	sys 0.005s
I20260709 12:13:24.632007 20710 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:24.635143 20752 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:24.644143 20518 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:24.644693 20518 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:13:24.652998 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 210017us
I20260709 12:13:24.578691 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:24.656255 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 77637.9us
I20260709 12:13:24.565430 20634 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:24.657042 12525 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.548s	user 0.009s	sys 0.026s
I20260709 12:13:24.656395 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:24.645926 20815 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:24.479739 20666 consensus_queue.cc:260] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:24.665511 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:24.666395 12526 raft_consensus.cc:2368] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24.
I20260709 12:13:24.652134 20710 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:24.660051 20714 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:24.679407 12526 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 24 index: 19. (index mismatch)
I20260709 12:13:24.645697 20518 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:24.681248 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:24.688627 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:24.712522 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:24.686496 12646 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:24.713691 12524 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 20: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:24.714222 12646 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:24.714581 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:24.715378 12531 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22.
I20260709 12:13:24.712746 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3185) took 1368 ms. Trace:
I20260709 12:13:24.716697 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:24.716890 12646 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:24.717115 12529 rpcz_store.cc:276] 0709 12:13:23.361634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:23.361972 (+   338us) service_pool.cc:224] Handling call
0709 12:13:23.362366 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:23.429105 (+ 66739us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:24.712475 (+1283370us) spinlock_profiling.cc:243] Waited 66.1 ms on lock 0x7b54003f7354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:24.712672 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":152112000}
I20260709 12:13:24.697512 12496 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:24.671870 20686 consensus_queue.cc:1059] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:13:24.721769 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:24.723052 20215 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:24.710232 12523 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:13:24.730424 20705 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:24.731350 20705 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:24.732697 20822 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:13:24.734380 20753 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:24.735783 20600 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:24.736524 20600 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:24.668107 20526 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:24.665088 20815 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:24.756774 20526 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:24.759068 20802 consensus_meta.cc:287] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.862s	user 0.007s	sys 0.004s
I20260709 12:13:24.759450 20526 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:24.662801 20744 consensus_queue.cc:260] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:24.717535 12672 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
W20260709 12:13:24.771476 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 54637us
I20260709 12:13:24.800032 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:24.771629 20215 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:13:24.801695 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 76698.5us
I20260709 12:13:24.757704 20815 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:24.783309 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 85880.8us
I20260709 12:13:24.800032 12646 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:24.804921 13305 leader_election.cc:400] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24.
I20260709 12:13:24.809706 13305 leader_election.cc:403] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
I20260709 12:13:24.801247 20215 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:24.673077 20823 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:13:24.766361 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 67725.6us
I20260709 12:13:24.783666 20473 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:24.760037 20526 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:24.771528 20600 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:24.801388 20749 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:24.780586 20518 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:24.773272 20705 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:24.834460 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:24.814677 20357 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:24.835366 20215 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:24.885322 20564 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:13:24.832989 20815 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:24.890506 20815 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:24.891007 20815 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:24.893608 12677 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:24.759918 20802 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:24.894608 20592 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:24.895308 20592 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:24.896565 20802 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:24.906191 20807 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:24.833721 20752 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24.
I20260709 12:13:24.849336 20761 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:24.914413 20761 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:24.915285 20761 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:13:24.848287 19826 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 19.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 12:13:24.917202 20802 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:24.872107 20826 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:24.917917 20688 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:24.918596 20826 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:24.919263 13332 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 18: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:13:24.920576 20809 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.012s	user 0.012s	sys 0.000s
I20260709 12:13:24.921137 20802 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:24.919880 20826 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:24.921195 20809 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:24.805896 20753 consensus_queue.cc:1059] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.070s
I20260709 12:13:24.845600 20473 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:24.873265 20671 consensus_queue.cc:1243] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 26.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:13:24.874918 20749 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:25.027405 12603 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:24.976722 20357 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:24.996642 20786 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:24.989259 20705 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:25.020112 20473 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:13:25.051612 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 217222us
W20260709 12:13:25.019157 20643 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.044s	sys 0.091s
I20260709 12:13:24.911427 20215 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:25.020351 20807 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:25.029047 20357 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:13:25.056336 20643 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.577s	user 0.044s	sys 0.092s
I20260709 12:13:24.922883 20802 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:25.048729 20786 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:25.107908 12602 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:25.075495 20786 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:25.025226 20526 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:25.114374 20708 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.027s	sys 0.029s
W20260709 12:13:25.210500 20736 consensus_meta.cc:287] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.672s	user 0.012s	sys 0.000s
I20260709 12:13:25.227042 20571 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:25.068409 20815 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:25.058457 20705 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:25.235361 20708 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.702s	user 0.028s	sys 0.030s
I20260709 12:13:25.230536 20802 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:25.228682 20722 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:25.263461 13302 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.014s	sys 0.017s
W20260709 12:13:25.264017 13302 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.737s	user 0.014s	sys 0.018s
I20260709 12:13:25.070791 20749 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:25.267879 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:25.268906 12672 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
W20260709 12:13:25.264657 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 778893us
I20260709 12:13:25.271023 13302 leader_election.cc:400] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24.
I20260709 12:13:25.236335 19826 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.026s
I20260709 12:13:25.253542 20526 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:13:25.265749 12571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.012s	sys 0.010s
W20260709 12:13:25.265050 12570 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.015s	sys 0.011s
W20260709 12:13:25.272967 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 399089us
W20260709 12:13:25.278703 20634 consensus_meta.cc:287] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.608s	user 0.014s	sys 0.000s
I20260709 12:13:25.244936 20736 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:25.265501 12572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.000s	sys 0.022s
W20260709 12:13:25.294734 12571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.624s	user 0.012s	sys 0.011s
W20260709 12:13:25.295104 12570 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.015s	sys 0.011s
I20260709 12:13:25.248757 20571 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:25.271171 20688 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:25.263243 20500 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.004s	sys 0.024s
W20260709 12:13:25.265331 20746 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.640s	user 0.014s	sys 0.011s
W20260709 12:13:25.295601 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 495666us
I20260709 12:13:25.317831 12672 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
W20260709 12:13:25.327705 12572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.657s	user 0.000s	sys 0.023s
W20260709 12:13:25.448232 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 788567us
W20260709 12:13:25.328800 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 672461us
I20260709 12:13:25.449890 12571 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:25.354761 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 82814.5us
I20260709 12:13:25.262814 20722 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:25.461035 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:25.462098 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:25.462774 12601 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:25.462849 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:25.454516 12646 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:13:25.312279 20396 consensus_meta.cc:287] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.774s	user 0.011s	sys 0.001s
I20260709 12:13:25.454363 13304 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:13:25.298838 20829 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:13:25.457048 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 895) took 1636 ms. Trace:
I20260709 12:13:25.292836 20693 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:25.458572 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 401404us
I20260709 12:13:25.297377 20705 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:25.473285 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:25.477600 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:25.479929 12604 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:13:25.482651 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:25.483295 12599 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:25.483350 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:25.484207 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:25.477435 12606 rpcz_store.cc:276] 0709 12:13:23.820516 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:23.916198 (+ 95682us) service_pool.cc:224] Handling call
0709 12:13:23.916469 (+   271us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:24.586016 (+669547us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:25.456717 (+870701us) spinlock_profiling.cc:243] Waited 667 ms on lock 0x7b540007b554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:25.456975 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1534545280}
W20260709 12:13:25.453002 20500 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.889s	user 0.006s	sys 0.024s
I20260709 12:13:25.306741 20708 raft_consensus.cc:3055] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:13:25.462960 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:25.492239 12600 raft_consensus.cc:2368] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
W20260709 12:13:25.458724 20746 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.833s	user 0.015s	sys 0.011s
I20260709 12:13:25.504370 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:25.469885 13302 leader_election.cc:400] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24.
I20260709 12:13:25.447817 20809 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:25.513706 12643 leader_election.cc:400] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20.
I20260709 12:13:25.441618 20634 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:25.478618 20839 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:25.518447 12643 leader_election.cc:403] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:25.523236 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:25.524742 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:25.529269 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:25.530241 12600 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 16.
W20260709 12:13:25.445312 12671 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.684s	user 0.012s	sys 0.032s
W20260709 12:13:25.537098 12671 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.775s	user 0.012s	sys 0.032s
W20260709 12:13:25.538298 12527 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.787s	user 0.013s	sys 0.024s
W20260709 12:13:25.538841 12527 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.830s	user 0.013s	sys 0.024s
W20260709 12:13:25.539697 13305 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.727s	user 0.011s	sys 0.026s
W20260709 12:13:25.540184 13305 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.727s	user 0.011s	sys 0.026s
W20260709 12:13:25.540604 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 727997us
I20260709 12:13:25.541585 13305 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:25.488127 20215 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:25.496089 20708 raft_consensus.cc:740] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:25.550477 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:25.551224 12605 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:25.546578 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:25.553222 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:25.556994 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:25.558596 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 45115.5us
W20260709 12:13:25.559564 12643 leader_election.cc:336] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801): Timed out: RequestConsensusVote RPC to 127.11.13.132:45801 timed out after 1.579s (SENT)
I20260709 12:13:25.560036 12643 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:25.561621 12571 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:25.564338 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:25.565184 12606 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:25.516420 20396 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:13:25.567934 12680 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:25.500128 20834 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:13:25.569967 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:25.572882 12600 raft_consensus.cc:2410] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
W20260709 12:13:25.574033 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:25.574927 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:25.575137 12494 leader_election.cc:336] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.755s (SENT)
I20260709 12:13:25.577484 12494 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:25.586763 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:25.587993 12600 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:13:25.554845 20630 consensus_meta.cc:287] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.141s	user 0.011s	sys 0.004s
I20260709 12:13:25.525146 20500 raft_consensus.cc:3055] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:25.485412 20693 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:25.593353 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 915) took 1339 ms. Trace:
W20260709 12:13:25.594364 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:25.595250 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:25.600234 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 58358.1us
I20260709 12:13:25.602478 13305 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:25.550504 20840 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:25.606673 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:25.597642 12679 rpcz_store.cc:276] 0709 12:13:24.254158 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:24.256095 (+  1937us) service_pool.cc:224] Handling call
0709 12:13:24.263803 (+  7708us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:24.265267 (+  1464us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:24.265317 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:24.281975 (+ 16658us) log.cc:844] Serialized 26 byte log entry
0709 12:13:24.873556 (+591581us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:24.873596 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:24.873684 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:25.593109 (+719425us) raft_consensus.cc:1685] finished
0709 12:13:25.593133 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:25.593288 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":51626,"raft.queue_time_us":1655,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":17419,"thread_start_us":487003,"threads_started":1,"wal-append.queue_time_us":709947}
I20260709 12:13:25.621304 12600 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:25.573267 20839 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:25.574064 20215 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20.
I20260709 12:13:25.609536 12606 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:13:25.624012 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:25.636216 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:25.638621 12599 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 23: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:25.603030 20500 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:25.622341 20752 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:25.625844 12600 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
W20260709 12:13:25.625358 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 146016us
I20260709 12:13:25.623464 12571 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:25.648588 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:25.649569 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:25.650703 13334 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:13:25.604907 20714 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:25.651494 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:25.651831 20714 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:13:25.642469 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:25.644945 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:25.652453 20714 consensus_queue.cc:1243] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 20.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:13:25.620464 20845 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:25.654299 20845 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:25.573415 20844 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:25.650677 20837 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:25.662245 20844 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:25.662858 20844 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:25.663285 20215 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:13:25.665501 20473 consensus_meta.cc:287] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.602s	user 0.010s	sys 0.000s
I20260709 12:13:25.662866 12599 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:25.666376 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:25.667011 12600 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:25.667899 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 996) took 1007 ms. Trace:
I20260709 12:13:25.624826 13305 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:25.569826 20835 consensus_queue.cc:1059] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 12:13:25.680261 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:25.679171 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:25.602205 20630 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:25.631045 20396 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:25.659528 20845 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:25.632648 20849 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:25.660723 20836 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
W20260709 12:13:25.661645 20816 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.719s	user 0.004s	sys 0.017s
I20260709 12:13:25.668303 12527 rpcz_store.cc:276] 0709 12:13:24.660892 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:24.661197 (+   305us) service_pool.cc:224] Handling call
0709 12:13:24.661664 (+   467us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:24.662175 (+   511us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:24.662212 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:24.667440 (+  5228us) log.cc:844] Serialized 26 byte log entry
0709 12:13:25.539392 (+871952us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:25.539455 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:25.539563 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:25.667661 (+128098us) raft_consensus.cc:1685] finished
0709 12:13:25.667682 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:25.667832 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2422,"raft.queue_time_us":126,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":118,"thread_start_us":830835,"threads_started":1,"wal-append.queue_time_us":986070}
W20260709 12:13:25.674769 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 97356.1us
I20260709 12:13:25.756505 12494 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:25.640178 20746 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:13:25.675555 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 221100us
I20260709 12:13:25.679145 20473 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:25.652210 12606 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 23: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:25.679556 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:25.652562 20747 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:25.646788 20752 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:25.766153 12646 leader_election.cc:400] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20.
I20260709 12:13:25.793953 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:25.693470 20837 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:25.797767 12599 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:25.829958 12600 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:25.740566 20643 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:25.745059 20215 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:25.763365 20847 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:25.755384 20571 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:25.843310 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3919) took 1147 ms. Trace:
I20260709 12:13:25.769276 20747 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:25.740446 20786 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:25.849337 12671 rpcz_store.cc:276] 0709 12:13:24.696278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:24.697225 (+   947us) service_pool.cc:224] Handling call
0709 12:13:24.761010 (+ 63785us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:24.761288 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:13:25.549474 (+788186us) log.cc:844] Serialized 14 byte log entry
0709 12:13:25.841865 (+292391us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:25.842080 (+   215us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:25.842382 (+   302us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:25.842674 (+   292us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:25.843178 (+   504us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":827049,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":972,"raft.queue_time_us":16231,"raft.run_cpu_time_us":101,"raft.run_wall_time_us":101,"thread_start_us":1066895,"threads_started":3,"wal-append.queue_time_us":259752}
I20260709 12:13:25.809500 20752 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:25.854763 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 186163us
I20260709 12:13:25.755420 20396 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:25.860397 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 103983us
I20260709 12:13:25.831801 20845 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:13:25.865250 20813 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.968s	user 0.027s	sys 0.022s
I20260709 12:13:25.858099 20747 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:25.767392 20849 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:25.856323 20571 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:25.882890 20396 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:25.883653 20860 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:13:25.884421 20623 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 12:13:25.884678 20838 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:13:25.852397 20847 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:25.885736 20845 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:25.844779 20643 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:25.893509 20863 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:25.894097 20855 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:25.898253 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:25.875208 20526 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:25.849205 20215 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:25.768285 20500 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:25.837308 20837 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:25.930989 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:25.932906 12678 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:13:25.875089 20833 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:25.888604 20634 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:25.941231 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:25.912122 20473 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:25.912523 20830 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.677s	user 0.000s	sys 0.008s
I20260709 12:13:25.912446 20847 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:25.913244 20849 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:25.914253 20845 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:25.938840 20726 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.336s	user 0.020s	sys 0.018s
I20260709 12:13:25.915380 20855 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:25.908625 20852 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:13:25.916345 20518 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.034s	user 0.010s	sys 0.001s
I20260709 12:13:25.886169 20850 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:25.917543 20863 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:25.908468 20859 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:25.914903 20688 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:25.938459 20643 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:25.943089 20627 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:25.953950 20726 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.356s	user 0.021s	sys 0.018s
I20260709 12:13:25.963990 20845 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:25.961069 20863 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:25.969925 20855 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:25.972410 20802 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.705s	user 0.010s	sys 0.000s
I20260709 12:13:25.972813 20840 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:25.966498 20786 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:25.942543 20833 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:25.961112 20850 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:25.976819 20833 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:25.965492 20844 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:25.956321 20852 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:25.941373 20864 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 12:13:25.977447 20852 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:25.977857 20864 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:13:25.978281 20864 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:25.979223 20864 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:25.979930 20844 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:25.977154 20850 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:25.980619 20864 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:25.961205 20859 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:25.982632 20859 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:25.980964 20865 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.022s
I20260709 12:13:25.984160 20837 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:25.984891 20872 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:13:25.986037 20863 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:25.987955 20852 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:25.988559 20865 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:13:25.989038 20863 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:25.989100 20865 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:25.990553 20852 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:25.990408 20859 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:25.990254 20865 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.19, Last appended by leader: 19, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:25.991715 20865 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:25.992374 20859 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:25.887493 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:25.908238 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 72014.7us
W20260709 12:13:26.033206 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 169456us
I20260709 12:13:25.916250 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3264) took 2957 ms. Trace:
I20260709 12:13:26.035326 12602 rpcz_store.cc:276] 0709 12:13:22.958501 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:23.134969 (+176468us) service_pool.cc:224] Handling call
0709 12:13:23.138344 (+  3375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:25.108603 (+1970259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:25.915952 (+807349us) spinlock_profiling.cc:243] Waited 1.97 s on lock 0x7b540007a654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:25.916144 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4528962304}
I20260709 12:13:26.038434 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3273) took 2113 ms. Trace:
I20260709 12:13:25.910414 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.042685 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.044628 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.039116 12525 rpcz_store.cc:276] 0709 12:13:23.920750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:23.922369 (+  1619us) service_pool.cc:224] Handling call
0709 12:13:23.931060 (+  8691us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:23.972933 (+ 41873us) raft_consensus.cc:1534] Early marking committed up to index 8
0709 12:13:24.005219 (+ 32286us) log.cc:844] Serialized 14 byte log entry
0709 12:13:24.614062 (+608843us) spinlock_profiling.cc:243] Waited 5.47 ms on lock 0x7b18002e5dc0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:13:24.665380 (+ 51318us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:24.667525 (+  2145us) log.cc:844] Serialized 52 byte log entry
0709 12:13:24.677092 (+  9567us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:24.677550 (+   458us) log.cc:844] Serialized 14 byte log entry
0709 12:13:24.677853 (+   303us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:24.677937 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:24.768606 (+ 90669us) spinlock_profiling.cc:243] Waited 25.7 ms on lock 0x7b18002e5dc0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:13:25.911854 (+1143248us) spinlock_profiling.cc:243] Waited 9.25 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:26.033405 (+121551us) raft_consensus.cc:1685] finished
0709 12:13:26.033485 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:26.033973 (+   488us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":68914,"prepare.queue_time_us":27700,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":82,"raft.queue_time_us":59916,"raft.run_cpu_time_us":529,"raft.run_wall_time_us":119316,"spinlock_wait_cycles":92936064,"thread_start_us":553833,"threads_started":1,"wal-append.queue_time_us":1021645}
I20260709 12:13:25.938516 12605 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:13:26.046334 12602 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 23: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:26.046711 12671 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: [19.13-21.14]
I20260709 12:13:26.047720 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.049666 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:26.050235 12675 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:26.052634 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.053854 12598 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 23: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:26.055071 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:26.043939 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:26.060184 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 172617us
I20260709 12:13:26.047745 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.049683 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.062254 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:26.063176 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:26.068138 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 435940us
W20260709 12:13:26.068264 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 443382us
I20260709 12:13:26.075457 12671 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:26.079595 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:25.898581 20571 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:25.966207 20688 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:26.093102 12672 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:26.094766 13338 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:13:26.097261 20688 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:25.954799 20693 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:26.099272 20357 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.010s	user 0.009s	sys 0.009s
I20260709 12:13:26.098570 20627 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.099990 20357 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:26.083184 12571 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:26.080591 12602 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 16.
I20260709 12:13:26.083423 20215 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:26.088299 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 53345.2us
I20260709 12:13:25.926473 20526 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.107990 20802 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.119982 20627 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:26.120280 20802 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.145234 20802 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:26.108738 20847 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
W20260709 12:13:26.154471 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 71554.6us
I20260709 12:13:26.157220 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 18 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:26.158172 12602 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 16 FOLLOWER]: Advancing to term 18
I20260709 12:13:26.166576 20844 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:26.173012 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3764) took 2482 ms. Trace:
I20260709 12:13:26.173542 12603 rpcz_store.cc:276] 0709 12:13:23.691038 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:23.819882 (+128844us) service_pool.cc:224] Handling call
0709 12:13:23.834264 (+ 14382us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:25.028827 (+1194563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:26.172421 (+1143594us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540007a154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:26.172768 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2743425920}
I20260709 12:13:26.093107 20571 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:26.109261 20357 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:26.183095 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:26.183416 12677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 3899) took 2761 ms (client timeout 3083 ms). Trace:
I20260709 12:13:26.183679 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3921) took 1486 ms. Trace:
W20260709 12:13:26.183844 12677 rpcz_store.cc:269] 0709 12:13:23.421848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:23.426519 (+  4671us) service_pool.cc:224] Handling call
0709 12:13:26.182932 (+2756413us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b540016eed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:26.183159 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3355425536}
I20260709 12:13:26.177765 12675 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:13:26.191164 20852 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:26.192040 20852 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.194504 20833 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.020s
I20260709 12:13:26.195678 20672 raft_consensus.cc:3055] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:13:26.196676 20837 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:26.196838 20672 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:26.197567 20837 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:26.184245 12680 rpcz_store.cc:276] 0709 12:13:24.696837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:24.697775 (+   938us) service_pool.cc:224] Handling call
0709 12:13:24.698031 (+   256us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:25.574386 (+876355us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:26.183158 (+608772us) spinlock_profiling.cc:243] Waited 857 ms on lock 0x7b540016f8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:26.183505 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1970702592}
I20260709 12:13:26.199033 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.200050 12603 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:26.201421 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:26.202327 12605 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:13:26.203123 20706 raft_consensus.cc:3055] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:26.203856 20706 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:26.206780 12605 raft_consensus.cc:3055] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:26.207262 12605 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:26.208664 12605 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:26.210294 20837 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:26.212713 20722 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:26.205418 20706 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:26.216039 12569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.903s	user 0.047s	sys 0.040s
W20260709 12:13:26.221093 12569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.935s	user 0.047s	sys 0.040s
I20260709 12:13:26.214259 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.223158 12605 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:26.224658 12678 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:13:26.220142 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:26.122979 20643 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.104s
I20260709 12:13:26.227170 12677 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:26.195827 20833 consensus_queue.cc:1243] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 20.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:13:26.228976 20357 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.229822 20357 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:26.231279 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.232770 20357 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:26.232877 20815 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:26.233623 20357 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:26.107177 20611 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:26.264102 12571 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:26.233934 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:26.219172 20706 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:26.246492 20752 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:26.271677 20379 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:26.273680 20752 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:26.268376 20357 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:26.236635 20708 consensus_queue.cc:260] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.235733 20672 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.253232 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:26.235220 20852 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:26.241950 20812 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:26.235992 20833 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:13:26.281495 20643 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:26.292429 20722 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.293277 20722 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.296084 20600 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:26.296790 13332 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 783) took 1900 ms (client timeout 2508 ms). Trace:
I20260709 12:13:26.296980 20600 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:26.297312 13332 rpcz_store.cc:269] 0709 12:13:24.395767 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:24.396063 (+   296us) service_pool.cc:224] Handling call
0709 12:13:26.296214 (+1900151us) spinlock_profiling.cc:243] Waited 522 ms on lock 0x7b540063ecd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:26.296504 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1199532288}
I20260709 12:13:26.299726 20600 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:26.301175 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.302315 12525 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 27: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:26.291568 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:26.313529 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:26.291534 20643 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:26.315215 13334 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
W20260709 12:13:26.319438 20827 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.088s	user 0.006s	sys 0.009s
W20260709 12:13:26.320019 20827 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.091s	user 0.006s	sys 0.009s
I20260709 12:13:26.321831 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:26.322822 13337 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:26.328989 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.329911 12525 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:26.331099 20672 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:13:26.339706 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 56173us
I20260709 12:13:26.332842 20643 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:26.347445 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:13:26.346522 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 82631.2us
I20260709 12:13:26.341507 20722 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:26.341553 20823 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:26.350618 20823 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:26.351184 20823 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:26.343066 20357 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:26.335625 20708 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:26.337219 20812 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:26.346781 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:26.356642 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.357062 20812 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:26.345114 20749 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:26.358384 20711 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.351042 13302 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:26.366277 20736 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:26.366537 20812 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
I20260709 12:13:26.368633 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:26.369050 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:26.370208 12526 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:26.370456 12524 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:26.371786 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:26.372247 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:26.372663 12569 connection.cc:256] RPC call timeout handler was delayed by -1.3128120300000319s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1623s for a response
W20260709 12:13:26.373101 12569 connection.cc:256] RPC call timeout handler was delayed by -1.257012030000169s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1685s for a response
W20260709 12:13:26.373457 12569 connection.cc:256] RPC call timeout handler was delayed by -1.0887120300001243s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18720000000000003s for a response
I20260709 12:13:26.376369 20848 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.377322 20848 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:26.380890 20870 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.381774 20870 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:26.394567 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 43682.6us
I20260709 12:13:26.398361 20711 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.407439 20711 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:26.408193 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 18 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:26.410684 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:26.411085 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.411574 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:26.412199 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:26.413250 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:26.413817 12525 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:26.413868 12522 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:13:26.415637 20834 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.004s	sys 0.008s
W20260709 12:13:26.416095 20834 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.780s	user 0.004s	sys 0.008s
I20260709 12:13:26.416402 20834 raft_consensus.cc:697] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:26.404354 20837 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.423900 12526 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:26.428949 20473 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.429572 20473 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.433327 20473 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:26.434266 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:26.434907 13337 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:26.435825 20833 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:13:26.437211 20839 consensus_queue.cc:1059] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:13:26.438727 20839 raft_consensus.cc:3055] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:26.438974 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.439092 20839 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:26.440583 20611 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:26.440363 20839 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 19.16, Last appended by leader: 16, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:26.441278 20611 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.441787 20839 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:26.443730 20866 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:26.440601 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:26.444391 20866 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:26.444979 12531 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:26.445564 20866 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:26.447602 20866 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:26.450343 12603 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:26.450810 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:26.453454 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.436335 12602 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:13:26.454083 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.454514 12525 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 24: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:26.443850 20611 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:26.431170 12569 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:26.467301 12522 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:26.472452 12531 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:13:26.476038 20741 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.476516 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:26.476636 20741 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:26.478094 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.480126 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:26.481331 12523 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:26.484504 20526 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:26.487958 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3272) took 1026 ms. Trace:
I20260709 12:13:26.488411 12604 rpcz_store.cc:276] 0709 12:13:25.461557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:25.462627 (+  1070us) service_pool.cc:224] Handling call
0709 12:13:26.487582 (+1024955us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b54002f1a54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:26.487773 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":37637376}
I20260709 12:13:26.492365 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.491814 12422 catalog_manager.cc:5697] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 15 to 21, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 21 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:26.496970 20837 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:26.498265 20643 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.500689 20643 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:26.501544 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.511138 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:26.512099 12674 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:26.512293 20852 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:26.513062 20852 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.514477 20705 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.514727 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:26.515277 20705 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:26.516577 12679 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: [19.13-21.14]
I20260709 12:13:26.519160 12677 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:26.519199 20852 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:26.520049 20744 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:26.523398 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:26.523936 20592 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:13:26.524740 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:26.525542 12673 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:26.524927 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:26.526484 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:26.526635 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:26.526813 12677 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:26.527262 13337 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 23: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:26.528664 13334 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:26.529191 20500 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:26.527267 12606 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 23: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:26.525641 20840 raft_consensus.cc:3055] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:26.531747 12605 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 17. (term mismatch)
I20260709 12:13:26.540307 12605 pending_rounds.cc:85] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 16
I20260709 12:13:26.540709 13332 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:13:26.529414 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:26.476043 12524 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:26.417587 20834 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 16.13, Last appended by leader: 13, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:26.549553 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:26.548209 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:26.542138 12422 catalog_manager.cc:5697] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 13 to 18. New cstate: current_term: 18 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:13:26.540593 12605 pending_rounds.cc:107] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 25.17
I20260709 12:13:26.554947 12605 raft_consensus.cc:2889] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:26.544967 12494 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:26.559304 12601 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 27: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:26.561216 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.557317 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:26.540249 20840 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:26.532325 20705 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:26.560405 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 166581us
I20260709 12:13:26.565845 12673 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:26.567340 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.546600 20711 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:26.575189 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.559082 20706 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.022s
I20260709 12:13:26.580658 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.580840 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:26.581205 20706 consensus_queue.cc:1243] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 20.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:13:26.582082 20799 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:26.582643 20799 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:26.583134 20799 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:26.586356 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.586903 20840 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.588627 12679 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:26.589702 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:26.590271 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:26.590489 12679 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:26.591022 13331 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:26.591670 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:26.592516 12679 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 22: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:13:26.593183 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 117190us
W20260709 12:13:26.594033 20710 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 26 current term: 26 is higher.
I20260709 12:13:26.594358 20379 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:13:26.595917 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:26.587237 13338 raft_consensus.cc:2410] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 25 because replica has last-logged OpId of term: 24 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 18.
I20260709 12:13:26.596230 20460 raft_consensus.cc:3055] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:13:26.601136 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 45374.5us
W20260709 12:13:26.602444 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.602802 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:26.607262 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.601804 12494 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:26.609786 12676 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:26.611084 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:26.611439 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.611704 12606 raft_consensus.cc:2376] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25. Re-sending same reply.
I20260709 12:13:26.612690 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.612949 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:26.613823 12680 raft_consensus.cc:2393] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:26.617162 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:26.618444 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:26.619108 12680 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:26.619318 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.620042 12599 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 28.
I20260709 12:13:26.624315 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:26.625408 12604 raft_consensus.cc:2410] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
I20260709 12:13:26.625663 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 22 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.626958 12605 raft_consensus.cc:2410] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 25 because replica has last-logged OpId of term: 23 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 10.
I20260709 12:13:26.629631 12605 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:26.632182 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.634455 12672 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:26.635550 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.636390 12646 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:26.637009 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.612697 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:26.638687 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.638854 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:26.638559 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:26.639271 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.639216 13331 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:26.642599 12598 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:13:26.649696 12606 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:26.655745 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3776) took 1014 ms. Trace:
I20260709 12:13:26.657755 12601 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:26.656412 12605 rpcz_store.cc:276] 0709 12:13:25.628138 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:26.628896 (+1000758us) service_pool.cc:224] Handling call
0709 12:13:26.629313 (+   417us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:26.630058 (+   745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:26.642369 (+ 12311us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:26.660662 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.662892 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:26.666178 12528 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 27: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:26.667328 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3771) took 1031 ms. Trace:
I20260709 12:13:26.608925 20840 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:26.668082 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3774) took 1040 ms. Trace:
I20260709 12:13:26.671444 12604 rpcz_store.cc:276] 0709 12:13:25.627601 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:26.625983 (+998382us) service_pool.cc:224] Handling call
0709 12:13:26.648249 (+ 22266us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:26.667536 (+ 19287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:26.667586 (+    50us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:26.667660 (+    74us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:26.667710 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:26.667804 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:26.667990 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:26.671787 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 240844us
I20260709 12:13:26.667816 12606 rpcz_store.cc:276] 0709 12:13:25.626686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:26.613635 (+986949us) service_pool.cc:224] Handling call
0709 12:13:26.614073 (+   438us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:26.658208 (+ 44135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:26.658484 (+   276us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:26.662328 20706 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:26.675132 20706 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:26.675625 20706 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:26.678186 12597 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:13:26.678341 20705 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:26.678997 20705 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:26.679378 20705 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:26.679852 12646 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:26.680908 12646 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:26.661664 20848 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:26.685544 20592 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:26.609158 20460 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:26.686184 20592 raft_consensus.cc:697] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:26.686810 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:26.687098 12531 raft_consensus.cc:2393] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 23: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:26.687268 20592 consensus_queue.cc:237] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.690415 20852 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:26.691090 20852 raft_consensus.cc:697] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:26.692138 20852 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:26.693197 12676 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
W20260709 12:13:26.694952 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:26.695650 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:26.696197 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.697494 12601 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:26.687949 20827 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:13:26.698285 20827 raft_consensus.cc:697] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:26.706844 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:26.699748 12571 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:26.670193 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3773) took 1042 ms. Trace:
I20260709 12:13:26.649430 20870 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:26.715052 12599 rpcz_store.cc:276] 0709 12:13:25.627309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:26.621768 (+994459us) service_pool.cc:224] Handling call
0709 12:13:26.638189 (+ 16421us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:26.669785 (+ 31596us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:26.669824 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:26.669876 (+    52us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:13:26.669906 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:26.669964 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:26.670090 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:26.717221 12673 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:26.719592 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.722206 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:26.666420 20747 consensus_meta.cc:287] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.775s	user 0.010s	sys 0.010s
I20260709 12:13:26.723953 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.723162 12601 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:26.725886 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:26.726822 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:26.719712 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 117999us
I20260709 12:13:26.731058 12605 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:26.729658 12677 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:13:26.733002 12602 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:26.733624 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3779) took 1104 ms. Trace:
I20260709 12:13:26.733937 12602 rpcz_store.cc:276] 0709 12:13:25.628985 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:26.716594 (+1087609us) service_pool.cc:224] Handling call
0709 12:13:26.732806 (+ 16212us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:26.733391 (+   585us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:26.733573 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:26.739993 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49011.3us
I20260709 12:13:26.740091 20839 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:13:26.741046 12643 leader_election.cc:336] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.749s (SENT)
I20260709 12:13:26.741195 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3782) took 1109 ms. Trace:
I20260709 12:13:26.741514 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:26.741612 12601 rpcz_store.cc:276] 0709 12:13:25.631485 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:26.732396 (+1100911us) service_pool.cc:224] Handling call
0709 12:13:26.740132 (+  7736us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:26.740787 (+   655us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:13:26.740821 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:26.740867 (+    46us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:26.740930 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:26.741006 (+    76us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:26.741129 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":99,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99}
W20260709 12:13:26.743348 12643 leader_election.cc:336] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.798s (SENT)
I20260709 12:13:26.743695 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.744120 12643 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:26.699360 20827 consensus_queue.cc:237] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.745728 12601 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:13:26.745661 20631 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:26.757474 20823 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.760107 12601 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:26.724084 20710 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:26.762575 20710 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:26.763888 12606 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:26.763744 20710 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.764580 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3778) took 1135 ms. Trace:
W20260709 12:13:26.764727 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.764961 12606 rpcz_store.cc:276] 0709 12:13:25.628693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:26.715390 (+1086697us) service_pool.cc:224] Handling call
0709 12:13:26.737905 (+ 22515us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:26.764310 (+ 26405us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:26.764498 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:26.765520 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.754778 12569 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 17 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:26.766625 20460 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:26.768656 20460 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:26.769061 12569 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:26.732393 20747 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:26.772354 20747 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:26.773236 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 124624us
I20260709 12:13:26.747387 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3765) took 1126 ms. Trace:
I20260709 12:13:26.774541 12597 rpcz_store.cc:276] 0709 12:13:25.621135 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:25.621574 (+   439us) service_pool.cc:224] Handling call
0709 12:13:26.747174 (+1125600us) spinlock_profiling.cc:243] Waited 827 ms on lock 0x7b54002710d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:26.747308 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1901047680}
I20260709 12:13:26.739979 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:26.777967 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.766309 12678 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:26.785800 12602 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:13:26.786391 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.786582 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3289) took 1069 ms. Trace:
I20260709 12:13:26.786926 12602 rpcz_store.cc:276] 0709 12:13:25.717397 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:26.734685 (+1017288us) service_pool.cc:224] Handling call
0709 12:13:26.776341 (+ 41656us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:26.786331 (+  9990us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:26.786531 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:26.785925 12524 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:13:26.787735 12531 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:13:26.788839 20595 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.789361 20595 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.789748 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.790561 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.792258 12599 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:13:26.746393 20844 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:26.794305 20879 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:13:26.794673 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:26.794898 20879 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:26.795282 12677 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:26.795938 20879 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.797257 20672 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.797804 20672 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:26.798535 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.798914 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.799911 12522 raft_consensus.cc:2410] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 28 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 17.
I20260709 12:13:26.800037 20672 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:26.799767 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.801785 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:26.802014 12674 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:26.803117 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:26.751850 20843 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.179s	user 0.004s	sys 0.000s
I20260709 12:13:26.804391 12674 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:26.803192 13305 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:26.780915 20823 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:26.816488 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:13:26.816737 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 48033.6us
I20260709 12:13:26.818919 12528 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:26.822852 12524 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:26.823443 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:26.821671 20595 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:26.825346 12643 leader_election.cc:336] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.827s (SENT)
I20260709 12:13:26.810042 12422 catalog_manager.cc:5697] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:26.826553 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.828065 12494 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:26.809978 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:26.781191 20870 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:26.828706 20853 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:26.832090 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:26.845742 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:26.835055 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.824081 20705 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.826572 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:26.842618 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:26.757050 20799 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:26.846632 12522 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 23: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:26.847500 12603 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:26.845741 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:26.834196 20863 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:26.863909 12531 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:26.815268 20711 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:26.864320 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.789376 20706 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.881430 13334 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 30: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:26.882161 12671 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 21 index: 12. (term mismatch)
I20260709 12:13:26.882645 12671 pending_rounds.cc:85] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 11
I20260709 12:13:26.882994 12671 pending_rounds.cc:107] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 20.12
I20260709 12:13:26.883272 12671 raft_consensus.cc:2889] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:26.883960 12598 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 30.
I20260709 12:13:26.882185 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.885564 13302 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:26.889590 20595 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:26.895718 12527 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:26.897109 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:26.897447 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:26.895943 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:26.899173 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:26.901532 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:26.905332 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 77394.2us
I20260709 12:13:26.866945 20823 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.884833 20747 raft_consensus.cc:2764] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election decision vote started in defunct term 28: won
I20260709 12:13:26.881300 20886 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:13:26.926359 12529 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:26.926957 13305 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:26.910009 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47559.3us
I20260709 12:13:26.895931 20672 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:26.881165 20711 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.940389 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:26.942003 20845 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:13:26.943737 20600 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:13:26.944370 20706 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:26.946151 20726 raft_consensus.cc:2764] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 16: won
I20260709 12:13:26.947108 20864 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.947727 20864 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.948587 20799 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:26.951505 20863 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:13:26.951826 20666 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:26.952714 20863 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:13:26.953980 20856 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:13:26.954428 20856 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:26.955108 12422 catalog_manager.cc:5697] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 16 to 19. New cstate: current_term: 19 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:26.955384 20856 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:26.956934 20856 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:26.957334 12422 catalog_manager.cc:5697] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 16 to 22, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 22 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:13:26.959867 12496 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:26.960330 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:26.961021 12673 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:13:26.936057 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 110772us
I20260709 12:13:26.963182 12677 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:26.928916 20473 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:26.895272 20870 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:26.932262 20866 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:26.970737 12674 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:26.925988 20823 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:26.931578 20672 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:26.933633 20867 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.141s
I20260709 12:13:26.980674 20711 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:26.965268 20845 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 30 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:27.000819 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:27.011101 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 45376.7us
I20260709 12:13:26.983620 20799 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:13:26.982970 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:26.988359 12531 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
I20260709 12:13:26.995880 12529 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:13:26.984777 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:26.946204 20857 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.116s
I20260709 12:13:27.019958 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:26.982944 20473 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:27.024330 12496 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:27.031611 20845 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:27.033145 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 165770us
I20260709 12:13:26.996706 20672 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:27.059985 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:13:27.067195 12643 leader_election.cc:336] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.883s (SENT)
I20260709 12:13:27.067803 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:27.033556 12429 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58980 (request call id 183) took 137437 ms (client timeout 15000 ms). Trace:
I20260709 12:13:27.034809 12422 catalog_manager.cc:5697] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:27.039117 20705 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:27.041980 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:27.072907 12525 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:27.074901 20823 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:27.048337 13305 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:27.017935 20799 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:27.059767 20379 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:13:26.998091 20866 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:27.063781 20870 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:27.057303 20857 raft_consensus.cc:3055] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Stepping down as leader of term 19
W20260709 12:13:27.069926 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50083.4us
I20260709 12:13:27.071516 12526 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:13:26.970885 20848 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.038s
I20260709 12:13:27.060634 20473 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:27.053629 20711 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:27.088096 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:27.077073 20839 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:13:27.094187 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 46008us
W20260709 12:13:27.069068 12429 rpcz_store.cc:269] 0709 12:11:09.653526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:09.653825 (+   299us) service_pool.cc:224] Handling call
0709 12:11:11.140670 (+1486845us) spinlock_profiling.cc:243] Waited 226 ms on lock 0x7b54000104d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148898a090 00007f1488a28f3a 00007f148e7dee9c 00007f148e8dd04e 00007f148cd9e2a4 00007f148cd9e232 00007f148cd9e161 00007f148cd9e0dc 00007f148cd9d392 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:27.033244 (+135892574us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":111275,"reported_tablets":52,"spinlock_wait_cycles":520651520}
I20260709 12:13:27.080438 20460 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:27.081449 20799 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:27.089372 20886 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:27.084889 20857 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:27.093924 20595 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:27.102512 20379 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:27.100543 20460 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:27.121295 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:27.102988 20839 raft_consensus.cc:697] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:13:27.161639 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:27.109504 20711 raft_consensus.cc:697] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:27.172341 12572 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 18 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:27.173643 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 106497us
I20260709 12:13:27.174937 12673 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
W20260709 12:13:27.174990 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:27.202783 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 148626us
I20260709 12:13:27.204523 12598 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
W20260709 12:13:27.206208 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 123467us
I20260709 12:13:27.119818 20886 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 24 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:27.130025 20741 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:27.169468 20460 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:27.209182 20870 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:27.208367 20643 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:27.242352 12494 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:27.243780 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:27.247838 20886 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:27.249337 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:27.250403 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 129255us
I20260709 12:13:27.220858 20473 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:27.226063 20706 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election lost for term 18. Reason: could not achieve majority
W20260709 12:13:27.266264 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 256173us
I20260709 12:13:27.278326 20202 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:27.278968 20202 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:27.279419 20202 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:27.226090 20595 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:27.283316 12525 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:13:27.285436 12646 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:27.292656 12673 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:27.256386 20848 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:27.300967 12572 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:27.306721 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:27.299532 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:27.264094 20870 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:27.325953 12671 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:27.299876 20749 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:13:27.336472 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 53991.1us
W20260709 12:13:27.331075 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 104010us
I20260709 12:13:27.335039 12604 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
W20260709 12:13:27.343963 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:27.347347 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:27.356818 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:27.359386 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:27.359673 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 58851.6us
I20260709 12:13:27.287597 20643 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:27.324947 20866 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:27.346550 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:27.370543 12672 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:27.368573 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:27.268620 20379 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 25.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:27.333106 20812 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:27.335961 20706 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:27.375564 12678 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:27.310586 20595 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:27.382443 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:27.311313 20741 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.058s
W20260709 12:13:27.386837 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:27.385095 13305 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:27.360736 20749 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:27.325482 20711 consensus_queue.cc:237] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:27.395671 20834 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:13:27.396382 20899 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 12:13:27.398397 20870 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:27.400020 20749 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 23.11, Last appended by leader: 11, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:27.402024 20863 raft_consensus.cc:2764] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:13:27.402210 20812 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:27.398846 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3777) took 1770 ms. Trace:
I20260709 12:13:27.404951 20812 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:27.405900 20706 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:13:27.406492 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:27.408846 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:27.412467 20710 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:13:27.413188 20710 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:27.414469 20710 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:27.404652 12604 rpcz_store.cc:276] 0709 12:13:25.628430 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:26.672873 (+1044443us) service_pool.cc:224] Handling call
0709 12:13:26.730084 (+ 57211us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:27.344469 (+614385us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:27.398609 (+ 54140us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b54002e63d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:27.398781 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1390531968}
W20260709 12:13:27.418735 20894 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.247s	user 0.000s	sys 0.001s
I20260709 12:13:27.409842 20857 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:27.420389 20857 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:27.305097 20839 consensus_queue.cc:237] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:27.424795 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:27.436514 12678 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: []
W20260709 12:13:27.440652 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:27.450356 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:27.451282 12526 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:13:27.458765 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:27.470996 12673 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:13:27.470863 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:27.472564 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:27.472786 12604 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:13:27.473229 12525 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:27.473747 12675 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
W20260709 12:13:27.476630 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 89552.8us
I20260709 12:13:27.423870 20864 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:27.390492 20592 raft_consensus.cc:2764] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election decision vote started in defunct term 17: won
I20260709 12:13:27.470542 20706 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:27.479557 20866 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:27.482568 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:27.448496 20611 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:27.482491 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:27.487732 12678 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:27.479025 13305 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:27.459653 20744 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:27.480723 20900 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.141s	user 0.000s	sys 0.001s
W20260709 12:13:27.497774 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 130095us
I20260709 12:13:27.480142 20705 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election won for term 20
W20260709 12:13:27.525336 20897 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.314s	user 0.001s	sys 0.000s
I20260709 12:13:27.497668 20706 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:27.526084 12572 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:27.528426 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:27.534473 20898 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.320s	user 0.001s	sys 0.001s
W20260709 12:13:27.534826 20902 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:13:27.535061 20823 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:27.534262 20726 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:27.538805 20823 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:27.539335 20823 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:27.535817 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:27.541647 20726 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:27.542173 20706 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:27.544274 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:27.497749 20812 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:27.542109 20726 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:27.507645 20747 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:27.487371 20611 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:27.506971 20866 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:13:27.551220 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 65484.9us
I20260709 12:13:27.546425 20812 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:27.560817 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:27.567850 12572 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:27.518916 20744 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:27.554420 20726 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:27.549221 20706 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:27.527832 20827 consensus_queue.cc:1059] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 12:13:27.563639 20812 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:13:27.572927 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 307651us
W20260709 12:13:27.571427 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 167976us
W20260709 12:13:27.573134 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 95065.3us
I20260709 12:13:27.572652 20747 raft_consensus.cc:697] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:27.617849 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3937) took 1079 ms. Trace:
I20260709 12:13:27.632896 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3300) took 1103 ms. Trace:
I20260709 12:13:27.616703 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:27.639325 20705 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:27.639415 12673 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:13:27.604642 20600 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:27.642968 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:27.632899 12680 rpcz_store.cc:276] 0709 12:13:26.538438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:26.620959 (+ 82521us) service_pool.cc:224] Handling call
0709 12:13:26.621281 (+   322us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:26.640238 (+ 18957us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:26.640269 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:26.698517 (+ 58248us) log.cc:844] Serialized 26 byte log entry
0709 12:13:26.931384 (+232867us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:13:26.931424 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:26.931505 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:27.617585 (+686080us) raft_consensus.cc:1685] finished
0709 12:13:27.617614 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:27.617774 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11293,"raft.queue_time_us":18431,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":9709,"thread_start_us":232393,"threads_started":1,"wal-append.queue_time_us":412291}
I20260709 12:13:27.605190 20643 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:27.634593 12679 rpcz_store.cc:276] 0709 12:13:26.529884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:26.593120 (+ 63236us) service_pool.cc:224] Handling call
0709 12:13:26.593476 (+   356us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:26.609288 (+ 15812us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:26.609320 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:26.719049 (+109729us) log.cc:844] Serialized 26 byte log entry
0709 12:13:27.092177 (+373128us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:27.092218 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:27.092304 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:27.632581 (+540277us) raft_consensus.cc:1685] finished
0709 12:13:27.632610 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:27.632805 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":14410,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":6294,"thread_start_us":325282,"threads_started":1,"wal-append.queue_time_us":507888}
I20260709 12:13:27.628684 20711 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:27.650533 20666 consensus_meta.cc:287] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.691s	user 0.016s	sys 0.001s
I20260709 12:13:27.670620 12643 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:27.641146 20837 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:27.682364 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:27.676496 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 108742us
I20260709 12:13:27.693601 20643 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:27.694190 20643 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:27.692122 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:27.665868 20913 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:27.628017 20611 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:27.666225 20600 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:27.701622 20643 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:27.686837 20840 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.968s	user 0.004s	sys 0.012s
I20260709 12:13:27.668797 20711 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:27.680017 20705 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 19.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:27.731765 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 103388us
W20260709 12:13:27.704298 13304 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.627s (SENT) [suppressed 36 similar messages]
I20260709 12:13:27.682622 20870 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:13:27.691145 20741 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 12:13:27.707409 20595 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:27.671604 20866 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 22.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:27.781770 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 111268us
W20260709 12:13:27.769712 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:27.789824 20910 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.162s	user 0.000s	sys 0.000s
W20260709 12:13:27.792295 20906 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.309s	user 0.004s	sys 0.000s
W20260709 12:13:27.793394 20202 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.514s	user 0.005s	sys 0.005s
W20260709 12:13:27.729208 20904 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.169s	user 0.000s	sys 0.000s
W20260709 12:13:27.797391 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:27.798986 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 103859us
I20260709 12:13:27.807497 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:27.793900 20856 consensus_meta.cc:287] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.837s	user 0.011s	sys 0.000s
I20260709 12:13:27.733201 20845 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:27.798856 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 206897us
I20260709 12:13:27.728677 20872 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:27.777539 20827 consensus_queue.cc:1243] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 22.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 12:13:27.813584 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 102500us
I20260709 12:13:27.831228 20595 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:27.834429 20845 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:27.835585 20711 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:27.835100 20872 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:27.731592 20919 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:13:27.709903 20500 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.101s
I20260709 12:13:27.698100 20837 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.107s
I20260709 12:13:27.839063 12494 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:27.830621 12601 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:13:27.728600 20747 consensus_queue.cc:237] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:27.836740 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 139178us
I20260709 12:13:27.710268 20848 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:27.793892 20202 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:27.849498 12672 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:27.820986 20870 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:27.874660 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 71404.8us
I20260709 12:13:27.795039 20710 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.037s
W20260709 12:13:27.876124 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 67843.8us
I20260709 12:13:27.885210 12525 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
W20260709 12:13:27.890291 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:27.884187 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:27.892979 12671 raft_consensus.cc:2410] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 28 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 14.
I20260709 12:13:27.890955 12646 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:27.850217 20744 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:27.882961 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:27.903010 12529 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:27.904729 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:27.799340 20913 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:27.901118 20595 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:27.908824 20595 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:27.909297 20595 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:27.909466 20845 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:27.910665 20930 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.040s
W20260709 12:13:27.911237 20672 consensus_meta.cc:287] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.829s	user 0.013s	sys 0.000s
I20260709 12:13:27.911672 20672 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:27.914037 20672 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:27.918362 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 77818.6us
I20260709 12:13:27.921079 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:27.923228 12526 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:13:27.889868 20711 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:27.938764 20711 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:27.939678 20711 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:27.940958 20711 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:27.891796 20870 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:27.875134 20848 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:27.925695 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:27.921977 12530 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
W20260709 12:13:27.927824 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:27.919862 20706 raft_consensus.cc:3055] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:27.930215 20879 raft_consensus.cc:3055] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:13:27.931687 20611 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:27.935052 20600 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:27.935321 20940 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:27.884321 20919 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:27.950783 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:27.996711 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 75441.6us
I20260709 12:13:27.859447 20473 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.058s
W20260709 12:13:27.952239 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 61336.3us
I20260709 12:13:27.894982 20643 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.064s
I20260709 12:13:27.915354 20856 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.067s
I20260709 12:13:28.001127 12672 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:28.002261 12673 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:27.953198 20706 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:27.953480 20848 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:28.011482 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:27.969344 20930 consensus_queue.cc:1243] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 22.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:13:28.012797 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:28.019088 12677 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:28.003178 20460 consensus_meta.cc:287] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.754s	user 0.003s	sys 0.011s
I20260709 12:13:28.018013 12680 raft_consensus.cc:2410] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 26 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 12:13:28.013327 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:28.024770 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:27.956552 20879 raft_consensus.cc:740] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:28.002409 20592 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:28.020051 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:13:28.021896 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 200361us
I20260709 12:13:28.029605 20920 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:28.030705 20924 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:13:28.025449 13338 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
W20260709 12:13:28.031884 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 133294us
I20260709 12:13:28.032079 13334 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:28.015981 20853 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.046619 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:28.048373 20913 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:28.052313 12494 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:28.054450 20917 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
I20260709 12:13:28.055837 20592 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:28.056090 20473 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:13:28.057194 20473 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:13:28.056916 20592 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.054400 20672 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:28.057920 20473 raft_consensus.cc:3055] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:28.015074 20823 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.105227 20913 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:28.084659 20672 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:28.008508 20726 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.026250 20460 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.040773 20870 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:28.135179 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:28.135511 12672 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:13:28.113306 12677 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:28.078088 20473 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:28.139592 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:28.124796 20857 consensus_meta.cc:287] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.704s	user 0.014s	sys 0.001s
I20260709 12:13:28.141371 12598 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
I20260709 12:13:28.144125 12673 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:13:28.119562 20666 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.072817 20853 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.146833 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:28.063236 20920 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.109135 20595 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.199144 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:28.198645 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 81479.6us
I20260709 12:13:28.150236 20845 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.041s
I20260709 12:13:28.187687 20913 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:28.208666 12494 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:28.209990 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 98453.9us
I20260709 12:13:28.161115 20666 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:28.213043 12494 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:28.213624 20933 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:28.213694 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:28.214180 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:28.198546 20856 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.218151 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:28.220738 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:28.149243 20919 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.233395 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:28.206826 20913 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:28.198909 20879 consensus_queue.cc:260] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 18.12, Last appended by leader: 12, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.238735 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:28.240434 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 51371.7us
I20260709 12:13:28.241585 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:28.241822 12522 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: []
W20260709 12:13:28.242096 20938 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.124s	user 0.001s	sys 0.000s
I20260709 12:13:28.242760 20925 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:28.244990 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:28.232213 20867 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:28.247210 12643 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:28.235553 20913 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:28.248858 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:28.249974 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:28.250759 12524 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
W20260709 12:13:28.240525 20799 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.124s	user 0.016s	sys 0.000s
I20260709 12:13:28.279598 20918 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:28.252319 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:28.252552 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:28.252399 20949 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:28.256626 12676 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:13:28.259159 20920 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:28.259299 20595 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:28.260910 20812 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.655s	user 0.011s	sys 0.001s
I20260709 12:13:28.261395 20726 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:28.269488 20837 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:13:28.247265 20936 log.cc:927] Time spent T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:13:28.206493 20672 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:28.245740 12529 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:28.314543 20937 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:28.296680 20706 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.311260 20949 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:28.316479 20706 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:28.316733 20949 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.291767 20799 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.319084 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:28.320463 20799 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:28.323203 20914 raft_consensus.cc:3055] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:28.323987 20914 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:28.325045 20914 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.310873 20812 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:28.288350 13334 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:28.231846 20856 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:28.296633 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:28.241096 20500 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.053s
I20260709 12:13:28.287407 20741 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:28.303304 20945 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:13:28.308276 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 55951us
W20260709 12:13:28.308116 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 59351.3us
W20260709 12:13:28.311190 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 153408us
I20260709 12:13:28.338359 12673 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:28.307080 20823 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:28.311354 20837 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:28.352458 12676 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:13:28.350488 20726 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.353585 12677 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:28.354307 20595 consensus_queue.cc:1059] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 15.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260709 12:13:28.352506 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:28.353974 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:28.349393 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:28.311700 20460 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:28.364461 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.342770 20705 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:13:28.395292 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:28.401746 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:28.368062 12494 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:28.403492 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.404511 12598 raft_consensus.cc:2368] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
W20260709 12:13:28.397362 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:28.409859 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.313603 20913 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.412993 12673 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:28.414921 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.416704 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:28.355348 20812 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:28.360430 20853 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:28.418862 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:28.418159 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:28.428560 12675 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:28.426688 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:28.405527 20845 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:28.434373 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:28.405767 20747 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:28.350751 20741 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.436385 12672 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:13:28.359871 20914 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:28.437987 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.437319 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.439566 12602 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:28.365662 20744 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 10, Time since last communication: 0.062s
I20260709 12:13:28.438650 12604 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:13:28.442997 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.426031 20958 raft_consensus.cc:3055] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Stepping down as leader of term 17
W20260709 12:13:28.456288 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 102529us
I20260709 12:13:28.431094 20460 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:13:28.457832 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 109089us
I20260709 12:13:28.433717 20595 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:28.408957 20848 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:28.414306 20726 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.436203 20845 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:28.469996 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.473553 12598 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 19: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:28.431236 20823 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.034s
I20260709 12:13:28.435218 20711 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.032s
I20260709 12:13:28.476305 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.438333 20930 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.032s
I20260709 12:13:28.455175 20958 raft_consensus.cc:740] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:28.438215 20827 consensus_queue.cc:1059] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.032s
I20260709 12:13:28.476989 12602 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:13:28.465276 20927 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:28.482837 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.467810 20666 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:28.489580 12675 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:28.473739 20845 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:13:28.477937 20611 consensus_meta.cc:287] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.508s	user 0.009s	sys 0.003s
I20260709 12:13:28.488139 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:28.492905 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.492859 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.494932 12603 raft_consensus.cc:2393] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 23: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:28.474756 20460 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:28.496012 12679 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:28.498634 20937 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:13:28.502403 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:28.503669 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:28.503813 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:28.504484 12680 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 23: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:28.504894 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:28.505378 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:28.505694 12679 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:28.506599 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:28.506721 20848 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:28.507388 12679 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:28.508306 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:28.510643 20726 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:28.510242 20927 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.510756 20958 consensus_queue.cc:260] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 17, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.512265 20958 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:28.512943 20927 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:28.514922 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:28.515838 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.516201 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:28.516671 12599 raft_consensus.cc:2410] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:13:28.517230 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:28.518106 12678 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:13:28.457808 20747 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.518549 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.497682 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:28.519960 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.520282 20747 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:28.520761 12597 raft_consensus.cc:2410] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 23 because replica has last-logged OpId of term: 21 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:13:28.521708 20799 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:28.522300 20799 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:28.522519 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:28.523000 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.523598 12604 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:13:28.525643 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.523206 20799 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.11, Last appended by leader: 11, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.526929 20741 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:28.528993 20913 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:28.531837 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:28.533303 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.532949 20460 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.534011 12675 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:28.535187 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:28.491778 12672 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:28.506107 12680 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 23: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:28.514986 20856 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:28.533324 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:28.542415 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:28.543259 12680 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:28.543725 20473 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.545375 20473 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:28.548429 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:28.549146 12674 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:13:28.550549 20925 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:28.550928 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:28.552662 20500 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 18.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:28.554284 20500 consensus_queue.cc:1243] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 19.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
W20260709 12:13:28.552134 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 66489.7us
I20260709 12:13:28.554672 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:28.555887 12671 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:28.557267 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:28.552448 12603 raft_consensus.cc:2368] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
I20260709 12:13:28.559460 12675 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
I20260709 12:13:28.562680 12678 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 30.
I20260709 12:13:28.564206 12671 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:13:28.564255 12646 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:28.557508 20744 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:13:28.570168 20744 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:28.570632 20744 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:28.571053 20744 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:28.571417 20879 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:28.573698 20500 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:28.454648 20837 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:28.574746 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:28.575506 12679 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 16 index: 13. Preceding OpId from leader: term: 18 index: 14. (index mismatch)
I20260709 12:13:28.568296 12599 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:13:28.578611 12494 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:28.580184 20500 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:28.587544 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:28.591984 12601 raft_consensus.cc:2410] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:13:28.591835 12572 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:28.591539 20500 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:28.598407 20379 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:13:28.599129 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:28.599400 20930 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:13:28.599989 20379 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:13:28.600432 20379 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:28.600828 20379 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:28.567212 20920 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.035s
I20260709 12:13:28.602397 20867 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:13:28.603636 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:28.605218 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:28.605439 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:28.605935 12530 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:28.606024 13338 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
W20260709 12:13:28.607079 20960 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.098s	user 0.001s	sys 0.000s
I20260709 12:13:28.607345 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:28.609119 12528 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 22 index: 10. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 12:13:28.587101 20962 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.613299 20962 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:28.615571 12530 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:28.622860 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:28.623226 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:28.623862 12677 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 22: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:28.624006 13331 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:28.624466 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:28.624033 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:28.626430 13337 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:13:28.627089 20856 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.627694 20856 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:28.604890 20961 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.084s	user 0.001s	sys 0.000s
I20260709 12:13:28.625782 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:28.637687 13331 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:28.639029 12680 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:28.626658 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:28.644464 12677 raft_consensus.cc:2393] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:28.620064 12526 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:28.644642 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.646098 12599 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 20.
I20260709 12:13:28.656456 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:28.657804 12530 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:13:28.657933 12674 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:28.660743 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:28.661644 12530 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:13:28.656981 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 153251us
W20260709 12:13:28.664888 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 73129.5us
I20260709 12:13:28.667514 12572 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:28.677542 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:28.678833 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:28.681267 20600 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.677127 13302 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:28.656404 20962 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:28.690696 12643 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:28.692178 12572 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:28.673221 20839 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:28.699139 12494 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:28.699626 12530 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26.
I20260709 12:13:28.699039 12529 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:28.706005 13305 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:28.694973 20600 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.690804 20643 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:28.712540 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:28.713536 12646 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:28.720563 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:28.720888 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:28.724962 20930 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.725363 20711 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:28.725700 20930 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.725992 20711 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.703073 20666 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:28.707171 20706 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:13:28.728065 12496 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:28.713461 20919 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:28.715111 20799 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:28.739485 12529 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:13:28.717185 20812 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:13:28.720403 20840 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.730024 20741 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:28.731801 20949 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:28.734788 12569 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:28.766889 13331 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:28.729530 20666 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:28.732767 20705 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:28.719653 20864 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.041s
I20260709 12:13:28.751402 20799 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:28.781874 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:28.779402 13338 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:13:28.758213 20867 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:28.731312 20595 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.035s
I20260709 12:13:28.717819 20379 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:28.727711 20951 consensus_queue.cc:1059] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.049s
I20260709 12:13:28.739284 20500 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:28.773075 20812 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:13:28.793643 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:28.777709 20666 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:28.777994 20706 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:28.795802 13331 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:13:28.776611 20741 raft_consensus.cc:697] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:28.766695 20840 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.787719 20705 raft_consensus.cc:697] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:28.786474 20799 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:28.794984 20943 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:13:28.837543 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 60312.3us
I20260709 12:13:28.843888 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:28.834561 12524 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:28.793049 20867 raft_consensus.cc:697] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:28.788178 20744 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.828871 20706 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:28.791045 20839 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 12:13:28.851912 20672 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.830538 20747 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:13:28.856509 20706 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:28.860852 20958 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:28.874650 12643 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:28.848413 20867 consensus_queue.cc:237] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.874318 20866 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.854369 20932 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:13:28.877096 20930 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:28.893522 20726 raft_consensus.cc:3055] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Stepping down as leader of term 21
W20260709 12:13:28.895182 20972 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
I20260709 12:13:28.926355 20726 raft_consensus.cc:740] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:28.927450 20726 consensus_queue.cc:260] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.928581 20726 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:28.929436 20706 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:28.934345 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:28.898488 12525 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:28.898869 20940 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:28.902644 13335 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:13:28.887691 20711 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:28.903846 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:28.892782 20929 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:28.918888 20840 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:28.923916 12531 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:13:28.875105 20672 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.890223 20919 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:13:28.889027 20958 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:13:28.948132 20974 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.075s	user 0.000s	sys 0.001s
I20260709 12:13:28.951074 20595 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:28.939738 20940 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:28.961530 12602 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27. Ops: [26.17-26.17]
I20260709 12:13:28.939662 20930 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:28.960093 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.876250 20812 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:28.968937 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.932541 20866 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.970669 12605 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:28.970047 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:28.951719 20929 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.976097 12604 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:28.884922 20705 consensus_queue.cc:237] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.978909 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.964375 20953 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:28.981485 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.979806 12603 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:13:28.982429 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:28.888696 20741 consensus_queue.cc:237] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.9, Last appended by leader: 9, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.982113 12605 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:13:28.977393 20202 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:28.966289 20600 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:28.996469 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:28.991587 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:28.999941 12528 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:28.996466 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:29.002002 12599 raft_consensus.cc:2368] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24.
W20260709 12:13:28.996665 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 92887.9us
I20260709 12:13:28.970088 20823 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
I20260709 12:13:29.006956 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.987551 20666 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:29.009207 20744 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:29.010268 20940 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:29.014828 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:28.975368 20473 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:29.023113 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:29.023499 12603 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:13:29.024051 12605 raft_consensus.cc:2410] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
I20260709 12:13:29.021651 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:29.027740 12606 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 24: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:29.030575 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:29.031656 12602 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:29.033566 12603 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:13:29.035408 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:29.038671 12604 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 27: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:28.970829 20930 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:29.050570 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:29.026948 20473 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:28.983479 20500 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:29.056010 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:29.022598 20710 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:29.028179 20940 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:28.989064 20953 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:29.064961 20706 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:29.065146 20473 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:29.071163 20845 consensus_meta.cc:287] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.529s	user 0.004s	sys 0.008s
I20260709 12:13:29.072414 20866 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:29.079347 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:29.079645 20929 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:29.082808 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:29.084281 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:29.085588 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:29.092245 12604 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:29.092355 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:29.087311 20672 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:29.089092 20930 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:29.056013 20958 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:29.091501 20953 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:29.093047 12597 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:13:29.067361 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:29.106307 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:29.107524 12597 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:29.072981 20710 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:29.111796 20856 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:29.120191 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:29.130978 20666 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:29.132170 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:29.154604 12599 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 19 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:13:29.120009 20710 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:29.076066 20845 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:29.148576 20643 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:29.166823 13338 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:29.175503 12602 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
W20260709 12:13:29.166239 12643 connection.cc:256] RPC call timeout handler was delayed by -1.1118524040000466s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15500000000000003s for a response
W20260709 12:13:29.178421 12643 connection.cc:256] RPC call timeout handler was delayed by -1.073152403999984s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1593s for a response
W20260709 12:13:29.178980 12643 connection.cc:256] RPC call timeout handler was delayed by -1.0713524039999811s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1595s for a response
I20260709 12:13:29.162009 20950 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:29.179551 12643 connection.cc:256] RPC call timeout handler was delayed by -1.0596524039999622s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16080000000000003s for a response
I20260709 12:13:29.181180 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:29.182358 12643 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:29.182849 12602 raft_consensus.cc:2393] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:29.188010 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:29.188143 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:29.189978 13334 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:29.189214 12602 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:29.187039 20929 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:29.206347 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:29.213095 13330 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 20 index: 14. (index mismatch)
I20260709 12:13:29.174983 20643 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:29.214797 12599 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:29.216794 20643 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:29.217387 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:29.220248 20879 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:13:29.221819 20845 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:29.206337 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:29.234267 20975 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.327s	user 0.000s	sys 0.005s
I20260709 12:13:29.240404 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:29.241302 12527 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:13:29.245724 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:29.231189 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:29.246670 12527 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:29.254771 12643 leader_election.cc:400] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
I20260709 12:13:29.263716 12602 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
I20260709 12:13:29.198447 20592 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:29.267735 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:29.267558 12494 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:29.189571 20950 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:29.255281 12643 leader_election.cc:403] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:29.273640 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:29.286329 12528 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:13:29.297755 12529 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: [24.14-25.15]
I20260709 12:13:29.289023 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3239) took 1034 ms. Trace:
I20260709 12:13:29.255256 20672 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:29.298659 12531 rpcz_store.cc:276] 0709 12:13:28.254325 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:28.254479 (+   154us) service_pool.cc:224] Handling call
0709 12:13:28.923560 (+669081us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:28.946233 (+ 22673us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:29.288692 (+342459us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b54003f6e50. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:29.288925 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1448790400}
I20260709 12:13:29.307934 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:29.323359 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:29.324625 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:29.328603 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 55074.6us
I20260709 12:13:29.329804 12494 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:29.331677 12494 leader_election.cc:400] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24.
I20260709 12:13:29.279521 20834 raft_consensus.cc:3055] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:29.245129 20929 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:29.331465 12646 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:29.333513 12494 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:29.333840 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 130792us
I20260709 12:13:29.337428 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:29.337461 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:29.335057 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:29.334947 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:29.319445 20879 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:29.338937 12597 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:29.338279 12528 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:29.342263 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:29.327790 20812 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:13:29.335265 20987 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.110s	user 0.000s	sys 0.002s
I20260709 12:13:29.333496 20919 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
I20260709 12:13:29.333420 20867 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:29.347267 12602 raft_consensus.cc:2393] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 18: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:29.333523 20473 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:29.348212 12494 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:29.336895 20834 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:29.353861 13302 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:29.343649 20864 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:13:29.359951 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:29.344893 20812 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:29.338091 20595 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.012s
I20260709 12:13:29.363907 12531 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:29.382268 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:29.383082 12531 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:13:29.384106 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:29.385318 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:29.381224 20812 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:13:29.380636 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:29.357688 20857 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:29.347795 20600 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:13:29.385246 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:29.348416 20705 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:29.380849 20845 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:29.364542 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:29.351426 20867 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:29.335748 20741 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.023s
I20260709 12:13:29.358035 20913 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:29.421458 20913 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:29.422013 20913 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:29.425518 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:29.389664 20864 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:29.432390 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:29.432454 20864 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:29.433357 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:29.412957 20744 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:29.416939 20977 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.502s	user 0.000s	sys 0.002s
I20260709 12:13:29.434540 20744 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:29.418285 20973 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.512s	user 0.000s	sys 0.003s
I20260709 12:13:29.419205 20379 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:29.419359 20964 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:29.356217 20473 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:29.437466 20964 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:29.437702 20473 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:29.439597 20834 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:29.441095 20834 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:29.442894 20744 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:29.428427 20837 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:29.452030 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:29.459056 12597 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:29.456350 20950 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:29.408530 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:29.433903 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:29.475971 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 65924.3us
I20260709 12:13:29.476433 12531 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:13:29.375962 20953 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:29.454859 20929 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:29.480201 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 46947.6us
I20260709 12:13:29.468194 12527 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:29.469300 12604 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:29.482353 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:29.483160 12604 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:29.485096 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:29.405009 20952 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:29.489144 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:29.489529 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:29.411429 20867 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:29.359539 20943 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.119s
W20260709 12:13:29.499020 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:29.489043 20672 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:13:29.728569 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:29.686071 12603 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 12:13:29.672225 12599 pending_rounds.cc:85] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 12
I20260709 12:13:29.766363 12599 pending_rounds.cc:107] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:13:29.766788 12599 raft_consensus.cc:2889] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:29.769565 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:29.770012 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:29.770857 12528 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:29.772240 12528 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:29.773555 12528 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:13:29.690955 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 310099us
W20260709 12:13:29.636996 20992 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.062s	user 0.000s	sys 0.000s
I20260709 12:13:29.590473 20710 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.063s
I20260709 12:13:29.650050 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:29.571306 20799 consensus_meta.cc:287] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.678s	user 0.014s	sys 0.000s
W20260709 12:13:29.604898 12525 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 3231) took 1391 ms (client timeout 1843 ms). Trace:
W20260709 12:13:29.832829 12525 rpcz_store.cc:269] 0709 12:13:28.213604 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:28.213773 (+   169us) service_pool.cc:224] Handling call
0709 12:13:29.604674 (+1390901us) spinlock_profiling.cc:243] Waited 630 ms on lock 0x7b540034fb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:29.604801 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1448746880}
I20260709 12:13:29.609416 12598 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 17. Current term is 18. Ops: [17.10-17.10]
I20260709 12:13:29.486726 20866 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.093s
W20260709 12:13:29.596720 20984 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.139s	user 0.000s	sys 0.000s
W20260709 12:13:29.851533 20976 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.953s	user 0.002s	sys 0.002s
I20260709 12:13:29.853415 20866 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:29.853863 20866 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:29.854832 20866 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:29.855436 20996 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.114s	user 0.000s	sys 0.000s
I20260709 12:13:29.856177 20866 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:29.579420 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 102716us
I20260709 12:13:29.859370 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:29.860438 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:29.810137 20845 raft_consensus.cc:697] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 21 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:29.862352 20943 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:29.863278 20943 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:29.863435 20845 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 19.16, Last appended by leader: 16, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:29.864693 20943 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 22.11, Last appended by leader: 11, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:29.866734 13337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 910) took 1114 ms. Trace:
I20260709 12:13:29.867233 13337 rpcz_store.cc:276] 0709 12:13:28.752427 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:28.752552 (+   125us) service_pool.cc:224] Handling call
0709 12:13:28.764108 (+ 11556us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:28.773741 (+  9633us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:28.773772 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:28.789055 (+ 15283us) log.cc:844] Serialized 26 byte log entry
0709 12:13:28.867872 (+ 78817us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:28.867903 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:28.867954 (+    51us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:29.850846 (+982892us) spinlock_profiling.cc:243] Waited 879 us on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:29.866299 (+ 15453us) raft_consensus.cc:1685] finished
0709 12:13:29.866323 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:29.866638 (+   315us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1024,"raft.queue_time_us":9678,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":10222,"spinlock_wait_cycles":2028800,"thread_start_us":34636,"threads_started":1,"wal-append.queue_time_us":99043}
W20260709 12:13:29.726402 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 317968us
I20260709 12:13:29.611904 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3302) took 1251 ms. Trace:
I20260709 12:13:29.871434 20913 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:29.842257 20799 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:29.871999 12601 rpcz_store.cc:276] 0709 12:13:28.360829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:28.391835 (+ 31006us) service_pool.cc:224] Handling call
0709 12:13:29.610114 (+1218279us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b5400270bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:29.611821 (+  1707us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":375743744}
I20260709 12:13:29.574361 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:29.824121 20912 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:13:29.874292 12523 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 21: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:29.487600 20953 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:29.876717 20953 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:29.590813 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 3981) took 1050 ms. Trace:
I20260709 12:13:29.877341 12678 rpcz_store.cc:276] 0709 12:13:28.540023 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:28.567703 (+ 27680us) service_pool.cc:224] Handling call
0709 12:13:28.589002 (+ 21299us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:28.643157 (+ 54155us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:28.643231 (+    74us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:28.671587 (+ 28356us) log.cc:844] Serialized 26 byte log entry
0709 12:13:28.870401 (+198814us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:28.870449 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:28.870713 (+   264us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:29.416244 (+545531us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:29.536265 (+120021us) raft_consensus.cc:1685] finished
0709 12:13:29.536299 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:29.590716 (+ 54417us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1535,"raft.queue_time_us":68217,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":6137,"spinlock_wait_cycles":56769792,"thread_start_us":185157,"threads_started":1,"wal-append.queue_time_us":229793}
W20260709 12:13:29.490208 20983 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.302s	user 0.000s	sys 0.007s
I20260709 12:13:29.871549 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:29.880378 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:29.602669 20952 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:29.652607 20930 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:29.882534 12600 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:13:29.882772 20930 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:29.883373 21005 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:29.885305 20930 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:13:29.729727 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:29.886834 20929 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:29.887001 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 923) took 1487 ms. Trace:
I20260709 12:13:29.585237 20839 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:13:29.528425 20726 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.599s	user 0.010s	sys 0.004s
W20260709 12:13:29.716164 20940 consensus_meta.cc:287] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.642s	user 0.008s	sys 0.006s
W20260709 12:13:29.739318 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 401937us
I20260709 12:13:29.888402 12606 raft_consensus.cc:2410] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 11.
I20260709 12:13:29.887400 12600 rpcz_store.cc:276] 0709 12:13:28.399214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:28.401572 (+  2358us) service_pool.cc:224] Handling call
0709 12:13:28.417957 (+ 16385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:29.885738 (+1467781us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:29.886707 (+   969us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54002e59d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:29.886911 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3367305088}
I20260709 12:13:29.887415 20929 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:29.770388 12676 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:13:29.890470 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:29.891628 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:29.894938 12606 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:29.895484 12528 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:13:29.895866 12606 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:29.774683 12673 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 18 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:13:29.775702 12597 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:13:29.897761 12524 raft_consensus.cc:2410] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:13:29.658805 20500 raft_consensus.cc:3055] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Stepping down as leader of term 26
W20260709 12:13:29.815034 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:29.823762 12531 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:29.833668 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:29.900545 12671 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
I20260709 12:13:29.901044 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:29.911144 12679 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:13:29.912142 12601 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27. Ops: [26.17-26.17]
I20260709 12:13:29.893227 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:29.913213 12676 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:29.914202 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:29.835955 12603 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:29.873839 20913 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:29.839526 12672 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:29.757268 20872 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:29.870370 20706 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:29.874027 12527 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 23: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:29.476502 20857 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:29.882881 20473 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:29.755364 12604 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:13:29.898416 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:29.898332 12571 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:29.899869 20500 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:29.837361 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:29.918022 12526 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 14. (term mismatch)
I20260709 12:13:29.919745 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:29.920903 12524 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:29.942119 12531 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 27. Current term is 28. Ops: []
I20260709 12:13:29.945092 12673 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:29.922879 12528 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:29.920308 12606 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:29.949160 12606 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:29.942464 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:29.873842 20799 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:29.931895 12494 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:13:29.932771 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 201300us
W20260709 12:13:29.899401 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 169781us
I20260709 12:13:29.911331 20940 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:29.911032 20379 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:29.901757 21008 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:29.911120 21006 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:29.944737 12526 pending_rounds.cc:85] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 13
I20260709 12:13:30.016136 12526 pending_rounds.cc:107] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 25.14
I20260709 12:13:30.016484 12526 raft_consensus.cc:2889] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:29.929450 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3337) took 1224 ms. Trace:
I20260709 12:13:29.950484 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:29.951157 12672 raft_consensus.cc:2410] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:13:29.920763 12496 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:29.913538 20202 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:13:29.962843 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 64640.2us
I20260709 12:13:30.001323 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:30.004990 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 106639us
I20260709 12:13:30.009140 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:30.024369 12524 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:13:30.022825 13338 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:29.919893 20952 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 12:13:29.965262 20913 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:13:30.032420 20202 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:30.034516 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:30.035434 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:29.926911 20886 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.074s
I20260709 12:13:30.039717 20799 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:30.040532 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:30.089658 12672 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:30.041355 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:30.042136 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:30.044560 12523 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 1021) took 1341 ms (client timeout 1700 ms). Trace:
I20260709 12:13:30.048008 21000 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Leader pre-election won for term 18
I20260709 12:13:30.017493 12522 rpcz_store.cc:276] 0709 12:13:28.705027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:28.913075 (+208048us) service_pool.cc:224] Handling call
0709 12:13:28.913503 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:28.925069 (+ 11566us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:28.925101 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:28.925525 (+   424us) log.cc:844] Serialized 26 byte log entry
0709 12:13:29.144938 (+219413us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:29.145001 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:29.145119 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:29.929233 (+784114us) raft_consensus.cc:1685] finished
0709 12:13:29.929266 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:29.929389 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":116,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":90,"thread_start_us":218870,"threads_started":1,"wal-append.queue_time_us":417398}
I20260709 12:13:29.942607 20706 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:30.056659 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:30.091214 12523 rpcz_store.cc:269] 0709 12:13:28.702909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:28.705519 (+  2610us) service_pool.cc:224] Handling call
0709 12:13:30.044270 (+1338751us) spinlock_profiling.cc:243] Waited 1.15 s on lock 0x7b5400106dd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:30.044474 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2650315392}
I20260709 12:13:30.080386 12680 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:30.036331 20857 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:30.081144 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:29.965708 20953 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:30.080652 12671 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
W20260709 12:13:29.962751 20726 consensus_queue.cc:1175] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "e56d487365394ac5ac5f1002926a5645" responder_term: 26 status { last_received { term: 26 index: 20 } last_committed_idx: 18 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
W20260709 12:13:30.093545 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:30.094659 12598 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:13:30.099442 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:30.100518 12680 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:30.102450 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:30.103962 12671 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:13:30.118477 12678 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:13:30.037950 20473 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:30.110545 12571 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:30.130414 12606 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
W20260709 12:13:30.103905 20812 consensus_meta.cc:287] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.648s	user 0.008s	sys 0.002s
W20260709 12:13:30.142294 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:30.141188 12571 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:30.144819 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:30.092324 20886 consensus_queue.cc:1243] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 23.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:13:30.026693 20379 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:30.111851 20827 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:30.151074 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:30.089102 20799 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:30.131086 20988 log.cc:927] Time spent T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:13:30.094209 21000 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:30.100724 21001 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:30.022444 20940 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:30.162974 12672 raft_consensus.cc:2410] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 19 because replica has last-logged OpId of term: 17 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 15 index: 9.
I20260709 12:13:30.130877 20864 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:30.166553 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:30.141284 20953 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:30.166692 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:30.173671 12673 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
W20260709 12:13:30.164306 20867 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.588s	user 0.011s	sys 0.001s
W20260709 12:13:30.189608 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50673.3us
I20260709 12:13:30.190591 12643 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:30.166280 20799 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:30.194582 12528 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:30.195748 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:30.166673 21000 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 17 FOLLOWER]: Advancing to term 18
I20260709 12:13:30.159467 21009 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:30.152644 20837 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.023s
I20260709 12:13:30.095498 20706 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:30.169718 21001 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:30.199954 12524 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:13:30.118216 20500 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:30.175668 20953 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:30.160893 20985 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 20.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.036s
W20260709 12:13:30.212311 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:30.158070 20812 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:30.168246 20827 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:30.207389 21009 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:30.216578 20953 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:30.211688 21001 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:30.193686 20867 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:30.217736 20919 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:30.287494 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 146395us
W20260709 12:13:30.268544 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 347768us
I20260709 12:13:30.278924 12527 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:13:30.297135 12527 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 20 index: 17. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:13:30.298084 12524 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:30.283244 20927 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:30.243607 20500 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:30.251428 21009 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
W20260709 12:13:30.310402 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:30.293361 12571 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:30.337370 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:30.342427 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:30.320261 20927 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:30.327158 20202 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:30.374305 12601 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:30.291939 20919 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:30.390559 21018 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
W20260709 12:13:30.401332 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 210818us
I20260709 12:13:30.363713 20927 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:30.439185 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1022) took 1057 ms. Trace:
I20260709 12:13:30.439638 12531 rpcz_store.cc:276] 0709 12:13:29.381924 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:29.951704 (+569780us) service_pool.cc:224] Handling call
0709 12:13:30.016865 (+ 65161us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:30.017065 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:13:30.083488 (+ 66423us) log.cc:844] Serialized 14 byte log entry
0709 12:13:30.438666 (+355178us) log.cc:844] Serialized 14 byte log entry
0709 12:13:30.438844 (+   178us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:30.438897 (+    53us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:13:30.438942 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:30.439007 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:30.439124 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11005,"thread_start_us":343541,"threads_started":1}
I20260709 12:13:30.444787 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:30.342921 20473 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.059s
I20260709 12:13:30.449751 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:30.452054 12677 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:13:30.455884 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3823) took 1293 ms. Trace:
I20260709 12:13:30.402755 21006 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:30.461264 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:30.503288 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:30.504280 12531 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:30.431483 21034 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:30.503531 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:30.512331 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:30.513105 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:30.449127 20940 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:30.518036 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 115892us
I20260709 12:13:30.518930 12643 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:30.527735 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:30.528764 12675 raft_consensus.cc:2410] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 25 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:13:30.499519 20930 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:13:30.556752 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 214389us
I20260709 12:13:30.562006 12571 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:30.505520 20913 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:30.599964 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:30.508754 20837 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:30.516441 20379 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:30.449353 21014 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:30.557986 20955 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:30.456786 12602 rpcz_store.cc:276] 0709 12:13:29.162720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:29.674300 (+511580us) service_pool.cc:224] Handling call
0709 12:13:29.674612 (+   312us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:29.674928 (+   316us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:13:30.386159 (+711231us) log.cc:844] Serialized 14 byte log entry
0709 12:13:30.455336 (+ 69177us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:30.455476 (+   140us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:30.455559 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:30.455685 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:30.455821 (+   136us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":620378,"thread_start_us":116164,"threads_started":2}
W20260709 12:13:30.637322 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 75400us
I20260709 12:13:30.541515 20930 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:30.643978 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:30.586520 20913 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:30.607347 20837 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:30.686762 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 156688us
I20260709 12:13:30.657800 20955 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:30.670380 21017 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.419s	user 0.002s	sys 0.000s
I20260709 12:13:30.724421 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:30.662060 20812 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:30.673100 20930 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:30.655431 20827 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:30.693800 21027 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:30.735586 21016 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.445s	user 0.002s	sys 0.000s
I20260709 12:13:30.671358 20867 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:30.712565 20837 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:30.775957 12677 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:30.728844 20955 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:30.737730 12531 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26.
I20260709 12:13:30.790563 12572 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:30.758721 21031 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:30.670012 21014 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:30.802742 12572 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:13:30.813576 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 169671us
I20260709 12:13:30.795133 20827 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:30.779574 21045 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:30.810478 20983 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
W20260709 12:13:30.957710 21026 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.457s	user 0.000s	sys 0.002s
I20260709 12:13:30.795426 20913 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:30.959996 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 157344us
I20260709 12:13:31.024578 12680 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:13:31.025789 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:30.779920 21027 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:30.827581 21022 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.402s	user 0.001s	sys 0.000s
W20260709 12:13:30.823855 21035 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.350s	user 0.002s	sys 0.000s
I20260709 12:13:30.795941 21028 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:30.868314 20827 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:31.019958 12672 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:13:31.020936 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3339) took 1248 ms. Trace:
I20260709 12:13:30.911484 20919 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:31.020509 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3817) took 1965 ms. Trace:
I20260709 12:13:31.026625 20929 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:31.076049 12599 rpcz_store.cc:276] 0709 12:13:29.055011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:29.671438 (+616427us) service_pool.cc:224] Handling call
0709 12:13:29.671972 (+   534us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:29.853280 (+181308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:29.853316 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:29.853796 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 12:13:30.077529 (+223733us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:30.692776 (+615247us) log.cc:844] Serialized 14 byte log entry
0709 12:13:30.693049 (+   273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:30.693127 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:31.020189 (+327062us) raft_consensus.cc:1685] finished
0709 12:13:31.020219 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:31.020427 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":429149,"prepare.queue_time_us":716972,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":64,"thread_start_us":389617,"threads_started":2,"wal-append.queue_time_us":352012}
I20260709 12:13:31.070765 12674 rpcz_store.cc:276] 0709 12:13:29.772552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:29.773461 (+   909us) service_pool.cc:224] Handling call
0709 12:13:29.773772 (+   311us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:29.836856 (+ 63084us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:29.836892 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:29.854507 (+ 17615us) log.cc:844] Serialized 26 byte log entry
0709 12:13:30.080081 (+225574us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:30.080121 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:30.080186 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:31.020699 (+940513us) raft_consensus.cc:1685] finished
0709 12:13:31.020722 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:31.020868 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1548,"raft.queue_time_us":80256,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":113,"thread_start_us":225132,"threads_started":1,"wal-append.queue_time_us":340625}
I20260709 12:13:31.095947 12674 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 20 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
W20260709 12:13:31.097617 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 394337us
I20260709 12:13:31.098558 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:31.035320 20848 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 2.126s	user 0.011s	sys 0.007s
W20260709 12:13:31.030048 20500 consensus_meta.cc:287] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.697s	user 0.007s	sys 0.001s
W20260709 12:13:31.108007 13338 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.888s	user 0.022s	sys 0.062s
W20260709 12:13:30.874599 12416 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 70b8c9b89e3d43fbb70422110d76702c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica a2fdbe7eef6748a9b4611f6c9bc9b201 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:13:31.118381 13338 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.899s	user 0.024s	sys 0.064s
I20260709 12:13:31.121022 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:31.120038 13338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 1003) took 1092 ms. Trace:
I20260709 12:13:31.039794 21028 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:31.100587 21039 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.264s	user 0.002s	sys 0.011s
W20260709 12:13:31.136683 12645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.882s	user 0.010s	sys 0.004s
W20260709 12:13:31.107976 20848 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 2.199s	user 0.011s	sys 0.007s
W20260709 12:13:31.142050 12645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.887s	user 0.011s	sys 0.004s
I20260709 12:13:31.097374 20929 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.097136 20202 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:31.146478 12416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 332775us
I20260709 12:13:31.132483 21028 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:31.148406 20964 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:31.122402 20913 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:13:31.160050 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:31.091454 20827 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:31.122702 20919 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:31.163650 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3272) took 1144 ms. Trace:
W20260709 12:13:31.181360 12600 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.008s	sys 0.008s
I20260709 12:13:31.023948 21045 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.125742 13338 rpcz_store.cc:276] 0709 12:13:30.027383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:30.027547 (+   164us) service_pool.cc:224] Handling call
0709 12:13:30.027869 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:30.028100 (+   231us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:13:30.219200 (+191100us) log.cc:844] Serialized 14 byte log entry
0709 12:13:31.119393 (+900193us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:31.119467 (+    74us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:13:31.119667 (+   200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:31.119768 (+   101us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:31.119944 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":49400,"prepare.queue_time_us":124204,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":94,"thread_start_us":1023020,"threads_started":2}
W20260709 12:13:31.185113 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:31.185581 12600 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.755s	user 0.008s	sys 0.008s
W20260709 12:13:31.192739 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 71831.4us
I20260709 12:13:31.193980 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:31.157362 20643 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:13:31.175063 20985 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.925s	user 0.006s	sys 0.011s
I20260709 12:13:31.202988 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:31.169718 21027 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:31.217216 21050 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:31.217911 21050 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:31.218447 21050 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:31.228291 20643 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:13:31.211015 20985 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.963s	user 0.006s	sys 0.012s
I20260709 12:13:31.168845 20929 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.182997 12529 rpcz_store.cc:276] 0709 12:13:30.006402 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:30.006720 (+   318us) service_pool.cc:224] Handling call
0709 12:13:30.024637 (+ 17917us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:30.084831 (+ 60194us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:30.084879 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:30.098636 (+ 13757us) log.cc:844] Serialized 26 byte log entry
0709 12:13:30.301485 (+202849us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:30.441815 (+140330us) log.cc:844] Serialized 14 byte log entry
0709 12:13:30.442035 (+   220us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:30.442107 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:31.150763 (+708656us) raft_consensus.cc:1685] finished
0709 12:13:31.150786 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:31.150931 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":59729,"prepare.queue_time_us":368253,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":16406,"raft.queue_time_us":69991,"raft.run_cpu_time_us":232,"raft.run_wall_time_us":234,"thread_start_us":302066,"threads_started":2,"wal-append.queue_time_us":276267}
I20260709 12:13:31.174571 20202 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:31.247309 21044 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.211s	user 0.000s	sys 0.000s
I20260709 12:13:31.186923 20964 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:31.211781 21029 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.264726 21063 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
W20260709 12:13:31.269917 21026 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.083s	user 0.000s	sys 0.000s
I20260709 12:13:31.193311 20592 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:31.317137 13304 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.971s	user 0.012s	sys 0.035s
W20260709 12:13:31.344074 13304 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.012s	sys 0.035s
W20260709 12:13:31.344584 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 998842us
W20260709 12:13:31.321722 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 295997us
I20260709 12:13:31.348289 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:31.332512 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 138638us
I20260709 12:13:31.349888 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:31.359149 12643 leader_election.cc:400] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
I20260709 12:13:31.359594 12643 leader_election.cc:403] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
W20260709 12:13:31.327945 21001 consensus_meta.cc:287] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.037s	user 0.016s	sys 0.007s
I20260709 12:13:31.318073 20919 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:13:31.338195 20955 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.524s	user 0.011s	sys 0.001s
I20260709 12:13:31.385159 12572 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:31.331657 20799 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.121s	user 0.009s	sys 0.011s
W20260709 12:13:31.379522 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 223960us
W20260709 12:13:31.348408 20866 consensus_meta.cc:287] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.492s	user 0.017s	sys 0.007s
I20260709 12:13:31.438789 12603 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 13. (term mismatch)
I20260709 12:13:31.439283 12603 pending_rounds.cc:85] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 12
I20260709 12:13:31.439512 12603 pending_rounds.cc:107] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 22.13
I20260709 12:13:31.439733 12603 raft_consensus.cc:2889] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:13:31.451195 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 66178.7us
I20260709 12:13:31.352342 21063 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:31.362648 21014 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:31.454404 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 935) took 2065 ms. Trace:
W20260709 12:13:31.453409 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54826.1us
I20260709 12:13:31.472373 12600 rpcz_store.cc:276] 0709 12:13:29.388830 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:29.889154 (+500324us) service_pool.cc:224] Handling call
0709 12:13:29.889732 (+   578us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:29.889937 (+   205us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:13:31.190154 (+1300217us) log.cc:844] Serialized 14 byte log entry
0709 12:13:31.453914 (+263760us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:31.453992 (+    78us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:13:31.454068 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:31.454165 (+    97us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:31.454328 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":534186,"prepare.queue_time_us":851824,"prepare.run_cpu_time_us":66,"prepare.run_wall_time_us":70,"thread_start_us":972468,"threads_started":2}
I20260709 12:13:31.375819 20592 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:31.396939 21001 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.475696 20919 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
I20260709 12:13:31.457943 21045 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:31.481727 20935 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.458236 21068 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:13:31.468080 21027 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:31.395399 20839 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
W20260709 12:13:31.540570 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 66674.7us
I20260709 12:13:31.467960 21063 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:13:31.546703 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 60576.3us
I20260709 12:13:31.567902 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:31.568948 12674 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:31.483530 20866 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.074s
I20260709 12:13:31.413414 20799 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.414160 20955 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.573690 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:31.482580 20985 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 20.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.068s
I20260709 12:13:31.361321 21029 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:31.584108 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:31.586894 21083 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.608691 12674 raft_consensus.cc:2393] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 22: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:13:31.613531 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 50794.9us
I20260709 12:13:31.614826 12572 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:31.619797 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 260774us
I20260709 12:13:31.631207 12643 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:31.646060 21029 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:31.647683 21058 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:13:31.648361 21064 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.095s	user 0.000s	sys 0.001s
I20260709 12:13:31.592058 21073 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:13:31.649401 21078 raft_consensus.cc:3055] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:13:31.649852 21078 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:31.650769 21078 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:31.652132 21078 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:31.627687 20919 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:31.653033 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:31.653366 20919 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.653860 20985 consensus_queue.cc:1243] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 21.10 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 9
I20260709 12:13:31.654479 20985 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:31.654999 20985 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.655881 20919 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:31.657116 21009 consensus_meta.cc:287] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.306s	user 0.010s	sys 0.007s
I20260709 12:13:31.657205 20985 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:31.578191 20935 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:31.658646 21031 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.832s	user 0.008s	sys 0.005s
I20260709 12:13:31.659088 21031 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:31.659926 20935 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:31.660185 20927 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.208s	user 0.010s	sys 0.004s
I20260709 12:13:31.660637 21069 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.661160 21069 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:31.661568 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:31.662107 20953 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.371s	user 0.010s	sys 0.005s
I20260709 12:13:31.662549 20953 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:31.662943 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 48296us
I20260709 12:13:31.663388 21069 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:31.663929 21069 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:31.664263 21069 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:31.664579 20953 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:31.664875 21077 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:31.665232 20953 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:13:31.633464 20202 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:31.666177 21000 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.453s	user 0.013s	sys 0.004s
W20260709 12:13:31.633720 21034 consensus_meta.cc:287] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.094s	user 0.012s	sys 0.003s
I20260709 12:13:31.666627 21000 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.667433 20839 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:13:31.668404 21027 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.668797 21000 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:31.666971 21034 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:31.668964 21027 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:31.670243 20930 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.895s	user 0.014s	sys 0.000s
I20260709 12:13:31.670457 21070 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:13:31.670648 20930 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.671281 21027 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:31.671859 21027 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:31.672750 20930 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:31.673291 12675 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:13:31.660607 20927 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.674921 20955 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:31.675798 21072 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.676187 20927 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:31.676280 21072 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.657626 21009 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.616750 20643 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.680364 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:31.619473 21062 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:31.672194 21027 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:31.675556 20827 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.715986 20929 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:31.719491 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 98883.8us
I20260709 12:13:31.722718 12677 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:31.705533 20985 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:31.664592 21069 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:31.729212 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:31.707118 20930 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.741552 21034 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:31.695617 20799 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:31.727473 21029 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:31.761296 21081 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:31.625941 21083 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:31.741510 20837 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.964s	user 0.015s	sys 0.000s
W20260709 12:13:31.762028 21080 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.185s	user 0.000s	sys 0.001s
W20260709 12:13:31.780354 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 51221.2us
I20260709 12:13:31.781551 12643 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:31.694587 20643 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:31.843098 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:31.742249 20985 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.829083 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3835) took 1779 ms. Trace:
I20260709 12:13:31.831480 20935 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.845425 20985 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:31.734866 21062 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:31.845535 20935 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:31.808072 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:31.822319 20919 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:31.804082 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3255) took 2629 ms. Trace:
I20260709 12:13:31.796522 20837 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.838958 20592 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:31.776432 21081 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:31.803388 21087 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:31.805826 21065 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.178s	user 0.000s	sys 0.001s
I20260709 12:13:31.811583 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3338) took 2039 ms. Trace:
I20260709 12:13:31.858505 12679 rpcz_store.cc:276] 0709 12:13:29.772190 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:29.772615 (+   425us) service_pool.cc:224] Handling call
0709 12:13:29.772916 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:29.914749 (+141833us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:31.811281 (+1896532us) spinlock_profiling.cc:243] Waited 137 ms on lock 0x7b54000ab9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:31.811519 (+   238us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":315555584}
I20260709 12:13:31.838979 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:31.823975 21028 consensus_meta.cc:287] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.639s	user 0.000s	sys 0.010s
I20260709 12:13:31.817972 20600 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.811024 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:31.844717 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:31.754602 21027 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.846339 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:31.794659 21029 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:13:31.864964 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:31.806476 21037 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.087s	user 0.000s	sys 0.001s
I20260709 12:13:31.846994 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:31.847214 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48163.6us
I20260709 12:13:31.764516 20930 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:31.815114 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 1.84386e+06us
I20260709 12:13:31.856230 20799 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.853034 20919 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:31.853592 21062 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:31.812814 12528 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 3273) took 1789 ms (client timeout 2072 ms). Trace:
I20260709 12:13:31.801683 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:31.824684 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3347) took 1291 ms. Trace:
I20260709 12:13:31.844416 12601 rpcz_store.cc:276] 0709 12:13:30.049464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:30.079925 (+ 30461us) service_pool.cc:224] Handling call
0709 12:13:30.080267 (+   342us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:30.374930 (+294663us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:31.828805 (+1453875us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b5400116cd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:31.829019 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":675563264}
I20260709 12:13:31.845887 12531 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:31.847000 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:31.852044 12526 rpcz_store.cc:276] 0709 12:13:29.174646 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:29.881975 (+707329us) service_pool.cc:224] Handling call
0709 12:13:29.882350 (+   375us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:30.017152 (+134802us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:31.803767 (+1786615us) spinlock_profiling.cc:243] Waited 29.2 ms on lock 0x7b5400054454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:31.803992 (+   225us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":67206912}
I20260709 12:13:31.850488 20958 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:31.802682 21009 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:31.828440 21001 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:31.827713 21072 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:31.830902 20827 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:31.865453 21070 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.850689 21000 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.845788 20643 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:31.870445 12494 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:31.887532 12527 rpcz_store.cc:276] 0709 12:13:30.533228 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:30.533464 (+   236us) service_pool.cc:224] Handling call
0709 12:13:30.533893 (+   429us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:30.534233 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:13:30.710282 (+176049us) log.cc:844] Serialized 14 byte log entry
0709 12:13:30.710930 (+   648us) log.cc:844] Serialized 14 byte log entry
0709 12:13:31.823993 (+1113063us) spinlock_profiling.cc:243] Waited 9.21 ms on lock 0x7b18003fdc60. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f14885d6d70 00007f14885ca293 00007f14885bfcea 00007f14885c023a 00007f1488a2aa2a 00007f1488a3a0bf 00007f1488a3a038
0709 12:13:31.824267 (+   274us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:31.824339 (+    72us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:31.824394 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:31.824489 (+    95us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:31.824619 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12491,"prepare.queue_time_us":728765,"prepare.run_cpu_time_us":6153,"prepare.run_wall_time_us":116245,"raft.queue_time_us":13776,"raft.run_cpu_time_us":308,"raft.run_wall_time_us":20271,"spinlock_wait_cycles":21190528,"thread_start_us":111558,"threads_started":1}
I20260709 12:13:31.853447 20985 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:31.865257 21058 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:31.866439 21086 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:31.904908 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:31.884865 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 83328.2us
I20260709 12:13:31.888851 12530 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:13:31.880375 12673 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 23: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:31.882133 20202 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.820433 21083 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:31.902550 12672 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.509s	user 0.027s	sys 0.024s
I20260709 12:13:31.864554 20726 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:31.864948 20950 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:31.889820 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:31.869264 20913 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:31.881110 12528 rpcz_store.cc:269] 0709 12:13:30.023492 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:30.023696 (+   204us) service_pool.cc:224] Handling call
0709 12:13:31.812647 (+1788951us) spinlock_profiling.cc:243] Waited 93.3 ms on lock 0x7b540038ead4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:31.812758 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":214527232}
I20260709 12:13:31.918529 20726 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.909114 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:31.908715 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:31.894667 20997 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:31.891395 20919 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:31.864082 21028 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:31.916203 20964 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.565s	user 0.012s	sys 0.001s
I20260709 12:13:31.914753 21086 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.864519 21087 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:31.866201 20600 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.917785 21058 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.918460 20202 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:31.919312 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:31.917122 12679 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:31.908707 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:31.885548 20953 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 12:13:31.921692 20827 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.933893 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:31.884459 20848 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:13:31.931867 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3839) took 1876 ms. Trace:
I20260709 12:13:31.890162 20799 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.893019 20958 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.944883 20997 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:31.953152 21028 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:31.955271 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4010) took 1598 ms. Trace:
I20260709 12:13:31.998191 12677 rpcz_store.cc:276] 0709 12:13:30.357237 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:30.809617 (+452380us) service_pool.cc:224] Handling call
0709 12:13:30.984170 (+174553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:31.728259 (+744089us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:31.954929 (+226670us) spinlock_profiling.cc:243] Waited 738 ms on lock 0x7b54000ac8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:31.955188 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1697925504}
I20260709 12:13:31.976486 12673 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:31.923071 20950 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:31.975982 12672 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 22: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:31.955981 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:31.867385 21034 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.050s
I20260709 12:13:31.930516 21083 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:31.935592 21089 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:31.986563 12603 rpcz_store.cc:276] 0709 12:13:30.055005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:30.055130 (+   125us) service_pool.cc:224] Handling call
0709 12:13:30.055508 (+   378us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:31.464766 (+1409258us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:31.931518 (+466752us) spinlock_profiling.cc:243] Waited 1.38 s on lock 0x7b5400271ad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:31.931785 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3180380544}
I20260709 12:13:31.989497 20600 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:31.990373 21087 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:31.955262 21086 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:31.958570 20930 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:31.991196 20202 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:31.905509 21000 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:31.906042 21070 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:32.000712 20997 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:32.020800 21058 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:32.021276 21070 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:31.954017 21079 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
I20260709 12:13:31.924353 20853 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:31.996142 21008 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:32.003610 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 86631.5us
I20260709 12:13:32.044574 12643 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:32.008632 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:32.046216 12679 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:13:31.956955 20935 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:32.048938 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:31.933470 20913 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:31.915534 20985 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:31.992033 20953 consensus_queue.cc:1243] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 26.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:13:32.016444 21087 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:32.022919 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:32.017884 21050 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.799s	user 0.014s	sys 0.000s
W20260709 12:13:32.057067 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 153413us
I20260709 12:13:31.994433 20827 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.055124 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.042289 20853 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:32.052134 21068 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:32.012220 21083 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:32.059244 21032 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:13:32.100965 13338 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:32.004506 21029 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.112205 12673 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:13:32.113819 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.114341 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:32.114888 12673 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
W20260709 12:13:32.097229 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 226874us
I20260709 12:13:32.098380 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:32.116022 12677 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:32.096575 12496 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:13:32.116561 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 67733.3us
I20260709 12:13:32.076936 20837 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:32.076714 20997 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:32.119154 12643 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:32.088518 20853 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:32.122606 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:32.131839 12416 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 70b8c9b89e3d43fbb70422110d76702c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica a2fdbe7eef6748a9b4611f6c9bc9b201 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:13:32.138833 13338 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:32.086807 21001 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:32.013614 21089 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:32.132411 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3345) took 2001 ms. Trace:
I20260709 12:13:32.145084 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:32.146940 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:32.076414 21087 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:32.150861 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:32.052188 21008 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:32.151662 13334 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:32.148380 12524 rpcz_store.cc:276] 0709 12:13:30.131156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:30.219730 (+ 88574us) service_pool.cc:224] Handling call
0709 12:13:30.243103 (+ 23373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:30.298674 (+ 55571us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:32.132153 (+1833479us) spinlock_profiling.cc:243] Waited 54.8 ms on lock 0x7b5400408654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:32.132349 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126011136}
I20260709 12:13:32.148334 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:32.077765 21050 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:32.119920 21084 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:13:32.161580 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:32.163200 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:32.164155 13337 raft_consensus.cc:2410] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 28 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
W20260709 12:13:32.166879 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49966.4us
I20260709 12:13:32.136732 20848 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:32.135566 21058 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:32.172219 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:32.167711 13305 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:32.151855 21032 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:32.154356 20997 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:32.175738 12680 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:32.158677 20726 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:32.151593 20964 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:13:32.175006 21032 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:32.184612 21061 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.851s	user 0.000s	sys 0.004s
I20260709 12:13:32.159905 21001 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:32.209919 21078 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.557s	user 0.013s	sys 0.001s
I20260709 12:13:32.176834 21058 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.215545 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.216840 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.192322 21031 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.018s
I20260709 12:13:32.217326 20919 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.219250 21089 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:32.219906 20950 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:32.174805 20848 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:32.222335 20799 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:32.194512 20202 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.029s
I20260709 12:13:32.223009 20848 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:32.210057 13364 heartbeater.cc:646] Failed to heartbeat to 127.11.13.190:46629 (5 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.11.13.190:46629 timed out after 15.000s (SENT)
I20260709 12:13:32.211530 21087 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:32.222358 21063 consensus_meta.cc:287] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.611s	user 0.005s	sys 0.006s
I20260709 12:13:32.211093 20913 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:32.170095 21034 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 12:13:32.226501 21050 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:32.175480 21069 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:32.227342 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:32.227746 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:32.230392 12646 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:32.231909 21049 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 20.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:13:32.233078 21049 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:13:32.233091 20839 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:13:32.234675 12524 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:13:32.212639 21083 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:32.233932 21049 raft_consensus.cc:3055] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:32.235467 21049 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:32.236371 21049 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.211514 21008 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:32.225896 21063 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.213308 20943 raft_consensus.cc:3055] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:32.234623 20839 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:32.237843 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.238366 21027 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:32.196194 21091 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.240147 20913 consensus_queue.cc:1059] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:13:32.241552 20950 raft_consensus.cc:3055] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:32.241940 20950 raft_consensus.cc:740] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:32.239252 21091 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:32.196631 21032 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:32.187476 20600 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.047s
I20260709 12:13:32.247320 21089 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:32.247485 21087 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:32.247964 21089 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 25 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:32.248044 21087 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:32.248692 21087 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:32.244632 21091 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:32.249171 21089 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:32.237834 20943 raft_consensus.cc:740] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:32.242863 20950 consensus_queue.cc:260] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.245368 12679 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:32.199308 12677 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:32.222604 21099 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.028s
I20260709 12:13:32.281255 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.287379 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.292932 12673 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 18.
W20260709 12:13:32.294059 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 160577us
I20260709 12:13:32.288836 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.182817 20958 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:32.243691 21021 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.919s	user 0.011s	sys 0.000s
I20260709 12:13:32.345036 12496 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 18 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:13:32.295135 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 133674us
I20260709 12:13:32.299692 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.280186 21038 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:32.295897 21098 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:32.242093 21001 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:32.243176 21031 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.059s
I20260709 12:13:32.303530 20943 consensus_queue.cc:260] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 20.9, Last appended by leader: 9, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:32.352058 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:32.306093 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:32.313949 21091 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:13:32.314749 21050 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:32.314750 20985 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:32.315706 21000 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:32.279518 21049 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:32.374922 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:32.316349 20839 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:32.323206 12601 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:13:32.331178 12602 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:13:32.377852 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:32.378952 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:32.333197 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:32.379568 12602 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:32.333113 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.335332 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:32.335739 21095 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.379897 12597 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 18.
I20260709 12:13:32.336388 21105 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.360421 21092 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.337107 21034 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:32.337164 21009 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:32.337438 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:32.336922 20864 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:32.341909 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:32.342847 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:32.350517 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 182888us
I20260709 12:13:32.356657 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:32.429334 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 87516.4us
I20260709 12:13:32.349911 20958 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
I20260709 12:13:32.361863 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.363847 12569 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:32.435873 12679 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
W20260709 12:13:32.435978 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 72264us
I20260709 12:13:32.364831 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:32.340636 20913 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.399219 20943 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:32.369349 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:32.439183 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.359534 21098 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:32.346403 20950 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:32.299006 20853 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.346532 20592 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:32.348626 20927 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:13:32.381462 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:32.336031 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:32.378799 12601 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:32.443917 12599 raft_consensus.cc:2368] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
I20260709 12:13:32.336880 12600 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:32.444422 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:32.445344 12601 raft_consensus.cc:2368] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21.
I20260709 12:13:32.394973 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.370602 21100 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.447676 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 16 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:32.401598 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3267) took 3116 ms. Trace:
I20260709 12:13:32.376868 12524 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:13:32.448912 12599 raft_consensus.cc:2410] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 19 because replica has last-logged OpId of term: 18 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 16 index: 13.
I20260709 12:13:32.449246 12524 raft_consensus.cc:606] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:32.351541 21008 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.042s
I20260709 12:13:32.419560 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.419615 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.423203 21034 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.427134 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:32.429826 13338 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 26 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:13:32.448541 12525 rpcz_store.cc:276] 0709 12:13:29.285398 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:29.911277 (+625879us) service_pool.cc:224] Handling call
0709 12:13:29.911614 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:29.911801 (+   187us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:29.911824 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:29.912255 (+   431us) log.cc:844] Serialized 26 byte log entry
0709 12:13:30.174727 (+262472us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:30.174766 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:30.174834 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:32.243066 (+2068232us) spinlock_profiling.cc:243] Waited 4.76 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:32.393420 (+150354us) raft_consensus.cc:1685] finished
0709 12:13:32.393444 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:32.401529 (+  8085us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":10942336,"thread_start_us":206767,"threads_started":1,"wal-append.queue_time_us":309759}
I20260709 12:13:32.429420 12602 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:13:32.358108 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:32.430588 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 89978.1us
I20260709 12:13:32.415588 12598 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:32.451404 12523 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:32.437368 21094 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:13:32.450685 12526 raft_consensus.cc:2410] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 25 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
I20260709 12:13:32.432284 12603 raft_consensus.cc:2410] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 26 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 14.
I20260709 12:13:32.429518 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:32.455559 12597 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:32.455842 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:32.434168 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.351788 21038 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.340698 20744 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.439235 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.439942 12679 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 20.
I20260709 12:13:32.403422 20839 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:32.373718 21050 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:32.371697 20202 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Leader election won for term 18
I20260709 12:13:32.453817 13333 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:32.442243 21027 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:32.459079 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:32.396929 21095 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.465770 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.465883 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.467824 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.430500 21009 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.452364 21008 consensus_queue.cc:1243] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 25.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:13:32.436699 20864 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.477628 21009 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:32.477816 20864 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:32.402778 21105 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:32.475680 12676 raft_consensus.cc:2410] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 12.
I20260709 12:13:32.478264 21009 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:32.479199 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.486357 12673 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:32.475630 12677 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
I20260709 12:13:32.461578 21031 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:32.443470 20913 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.465903 20744 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.414739 21092 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:32.445194 21000 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.025s
I20260709 12:13:32.472695 20919 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:32.472574 21027 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:32.454614 21100 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.512840 13304 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:32.517139 12602 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 12:13:32.571741 12602 pending_rounds.cc:85] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 11
I20260709 12:13:32.571993 12602 pending_rounds.cc:107] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 12:13:32.572201 12530 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 12. (term mismatch)
I20260709 12:13:32.541742 12527 raft_consensus.cc:2376] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21. Re-sending same reply.
I20260709 12:13:32.572690 12530 pending_rounds.cc:85] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 11
I20260709 12:13:32.573029 12530 pending_rounds.cc:107] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 23.12
I20260709 12:13:32.573328 12530 raft_consensus.cc:2889] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:32.575570 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:32.577895 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 65154us
I20260709 12:13:32.579874 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:32.581349 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:32.524026 12673 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26.
W20260709 12:13:32.544574 21107 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.114s	user 0.001s	sys 0.000s
I20260709 12:13:32.547029 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.586866 12676 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:32.492599 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:32.549434 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:32.503896 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.588882 12598 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 21: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:32.589761 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:32.589876 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:32.590965 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:32.591246 21095 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:32.591533 12524 raft_consensus.cc:2376] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25. Re-sending same reply.
I20260709 12:13:32.591738 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3836) took 2542 ms. Trace:
I20260709 12:13:32.572239 12602 raft_consensus.cc:2889] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:32.591130 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:32.593238 12598 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:13:32.515053 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 85790.8us
I20260709 12:13:32.530632 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.596447 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.597506 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:32.526542 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:32.598398 21113 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:32.494091 20705 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.106s
W20260709 12:13:32.601565 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:32.550797 21108 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.089s	user 0.000s	sys 0.000s
I20260709 12:13:32.567700 20913 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:32.568555 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:32.604658 12598 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:32.598073 12679 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:32.542455 12597 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:32.592186 12604 rpcz_store.cc:276] 0709 12:13:30.050003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:30.050613 (+   610us) service_pool.cc:224] Handling call
0709 12:13:30.051077 (+   464us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:30.079691 (+ 28614us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:30.079733 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:30.093503 (+ 13770us) log.cc:844] Serialized 26 byte log entry
0709 12:13:32.234486 (+2140983us) spinlock_profiling.cc:243] Waited 19.3 ms on lock 0x7b180038f140. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:13:32.347105 (+112619us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:32.347143 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:32.347666 (+   523us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:32.591428 (+243762us) raft_consensus.cc:1685] finished
0709 12:13:32.591449 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:32.591677 (+   228us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":39127,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":22978,"spinlock_wait_cycles":44348288,"thread_start_us":112240,"threads_started":1,"wal-append.queue_time_us":2281309}
I20260709 12:13:32.592391 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:32.620236 12524 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:32.625298 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:32.567662 21060 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.805s	user 0.003s	sys 0.000s
I20260709 12:13:32.627110 12676 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 27: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:32.629559 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:32.629740 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 155453us
I20260709 12:13:32.555374 20744 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:32.633047 12572 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:32.563253 21034 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:32.644577 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:32.633963 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 141449us
I20260709 12:13:32.494712 21083 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.143s
I20260709 12:13:32.634811 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:32.634892 21095 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:32.635072 21098 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:32.592517 21050 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:32.531629 21000 consensus_queue.cc:1243] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 24.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:13:32.639706 12606 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:32.583827 20799 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:32.541210 21027 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:32.562319 20853 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:32.583279 20202 raft_consensus.cc:697] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:32.640646 12679 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:13:32.650350 12524 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:13:32.651605 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:32.652736 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:32.656136 12572 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:32.627624 21089 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:32.625669 21073 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:32.625967 21105 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:32.567752 21092 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:32.625321 21114 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:32.661499 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:32.664848 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.664963 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.663400 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:32.659648 20943 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:13:32.666383 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 65182.4us
I20260709 12:13:32.502657 21101 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.185s
W20260709 12:13:32.668622 21045 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.008s	user 0.008s	sys 0.015s
I20260709 12:13:32.674611 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:32.674860 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:32.673458 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:32.660238 21095 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:32.687749 21095 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:32.674825 21089 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:32.695013 21113 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:32.695187 21089 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:32.694733 20202 consensus_queue.cc:237] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 18, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.679355 20799 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.672168 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:32.544734 21087 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:32.697643 13337 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:32.698020 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.697643 13333 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: [20.14-20.14]
I20260709 12:13:32.698745 12679 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:13:32.673198 13338 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 26: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:32.701288 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:32.680505 12643 leader_election.cc:400] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
I20260709 12:13:32.673624 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.704089 12643 leader_election.cc:403] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:32.704747 12523 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
W20260709 12:13:32.675855 12494 connection.cc:256] RPC call timeout handler was delayed by -1.4388869499998691s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16100000000000003s for a response
I20260709 12:13:32.673502 13330 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:32.682941 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:32.705736 12494 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:32.677609 12527 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:32.707437 12572 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:32.677235 21081 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:32.684746 12680 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:32.676030 21092 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:32.724650 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:32.684950 12524 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:13:32.685984 21045 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.026s	user 0.009s	sys 0.015s
I20260709 12:13:32.686671 21032 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.703208 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.676054 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:32.677814 12676 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:13:32.681537 21105 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:32.681125 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:32.708323 20964 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:32.733131 20943 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:13:32.738765 21109 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.156s	user 0.001s	sys 0.000s
I20260709 12:13:32.677178 21058 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:32.710705 12572 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:32.675325 21073 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:13:32.764559 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 53937.8us
W20260709 12:13:32.747496 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 67136.2us
I20260709 12:13:32.749311 12526 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:32.765950 12527 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:32.748718 12529 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:32.752156 12524 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
W20260709 12:13:32.752954 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 71905.2us
I20260709 12:13:32.767709 12678 raft_consensus.cc:2393] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 19: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:32.768757 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:32.745985 21092 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.671800 21119 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.079s
I20260709 12:13:32.748605 21099 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:32.774518 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.767107 21087 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:32.751368 21050 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
I20260709 12:13:32.775864 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.752444 21105 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:32.793007 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:32.767421 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.764592 21121 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:13:32.704376 20839 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:13:32.786352 12524 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 27.
I20260709 12:13:32.790807 21034 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:32.810932 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:32.791550 12643 leader_election.cc:400] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21.
I20260709 12:13:32.798451 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.799464 12527 raft_consensus.cc:2376] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27. Re-sending same reply.
I20260709 12:13:32.803332 12526 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:32.811911 12598 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:32.812161 12529 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:32.806638 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:32.812947 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:32.775009 21099 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:32.771976 20710 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:13:32.810328 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:32.811853 12643 leader_election.cc:403] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Cancelling election due to peer responding with higher term
I20260709 12:13:32.773767 21083 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:32.774256 21058 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.806793 12680 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
I20260709 12:13:32.821972 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:32.821841 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.814850 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.823354 12529 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22.
I20260709 12:13:32.823472 12524 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:13:32.818148 20744 raft_consensus.cc:3055] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:32.824101 13338 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:13:32.824388 20744 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:13:32.825665 21122 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:13:32.818910 20710 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:32.819226 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:32.796427 21105 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:32.858283 20799 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:32.858754 21118 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:13:32.830960 21034 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election lost for term 20. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21.
I20260709 12:13:32.792544 21049 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:32.819252 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.863219 12522 raft_consensus.cc:2393] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 20: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:32.864631 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.865820 12522 raft_consensus.cc:2410] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 26 because replica has last-logged OpId of term: 25 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 11.
I20260709 12:13:32.819986 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:32.822729 12604 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:32.814615 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:32.825511 20744 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.867514 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.867678 20744 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:32.868777 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.826634 21032 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:32.827859 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:32.808568 21100 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:32.823751 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:32.869808 13339 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:32.869951 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.870138 12527 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:32.787405 21073 raft_consensus.cc:697] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:32.870538 12522 raft_consensus.cc:2376] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25. Re-sending same reply.
I20260709 12:13:32.871469 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:32.871325 21073 consensus_queue.cc:237] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.834504 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:32.873694 21027 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:32.817159 21099 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:32.816656 21121 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:32.875981 21027 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:32.852138 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.876883 12529 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 28: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:32.877522 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:32.855469 20710 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.879302 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:32.880039 12528 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 22: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:32.854014 21095 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.880447 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:32.880858 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.881711 12528 raft_consensus.cc:2410] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 28 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:13:32.882270 21095 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:32.877638 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.883018 12599 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 15. (term mismatch)
I20260709 12:13:32.883136 12529 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 20.
I20260709 12:13:32.883504 12599 pending_rounds.cc:85] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 14
I20260709 12:13:32.883843 12599 pending_rounds.cc:107] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 25.15
I20260709 12:13:32.884220 12599 raft_consensus.cc:2889] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:32.875593 21121 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:32.893416 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 74282.9us
I20260709 12:13:32.894325 21001 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.894395 12643 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:32.895097 21001 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.896940 12643 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:32.884975 12673 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:13:32.898351 21009 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:32.898756 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:32.886137 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.843873 21058 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:32.905352 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3362) took 1244 ms. Trace:
I20260709 12:13:32.899590 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:32.905821 12679 rpcz_store.cc:276] 0709 12:13:31.661177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.703170 (+1041993us) service_pool.cc:224] Handling call
0709 12:13:32.703558 (+   388us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:32.703776 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:13:32.739610 (+ 35834us) log.cc:844] Serialized 14 byte log entry
0709 12:13:32.904711 (+165101us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:32.904783 (+    72us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:32.904837 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:32.905170 (+   333us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:32.905295 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":38303,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":749,"thread_start_us":164729,"threads_started":1}
I20260709 12:13:32.897897 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:32.891990 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.832499 21083 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:32.911368 12494 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:32.913932 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:32.897459 21001 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:32.897977 21111 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:32.909260 12679 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 21 index: 17. (index mismatch)
I20260709 12:13:32.898790 21009 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:32.914762 12528 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:32.913117 13305 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:32.916599 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:32.903044 12529 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:13:32.923806 12529 raft_consensus.cc:606] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:32.924139 12529 raft_consensus.cc:614] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:32.921545 12679 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:32.918934 12494 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:32.920511 12597 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:13:32.911263 12522 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:13:32.925386 12527 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:32.937292 13339 raft_consensus.cc:2410] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 26 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
I20260709 12:13:32.906677 21029 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:32.938339 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:32.938804 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3364) took 1261 ms. Trace:
I20260709 12:13:32.938962 21113 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:32.969218 21113 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:32.969663 21113 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:32.899209 21032 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:13:32.921751 20913 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:13:32.921975 21092 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:32.966214 12676 rpcz_store.cc:276] 0709 12:13:31.661735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.751429 (+1089694us) service_pool.cc:224] Handling call
0709 12:13:32.751779 (+   350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:32.751986 (+   207us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:13:32.804784 (+ 52798us) log.cc:844] Serialized 14 byte log entry
0709 12:13:32.908220 (+103436us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:32.908289 (+    69us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:32.908366 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:32.908940 (+   574us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:32.922965 (+ 14025us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26023,"prepare.queue_time_us":17779,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":1612,"raft.run_cpu_time_us":167,"raft.run_wall_time_us":5699,"thread_start_us":140523,"threads_started":2}
I20260709 12:13:32.939708 21111 raft_consensus.cc:697] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:33.016326 12602 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:32.970239 20848 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:13:32.923053 21095 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:32.936057 21009 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:32.975663 21001 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:32.917232 21105 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:32.934514 20799 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.081s
I20260709 12:13:32.952001 21045 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:33.027608 21089 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:33.029568 13331 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 23: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:13:33.030526 21066 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.526s	user 0.001s	sys 0.004s
I20260709 12:13:32.980680 21029 raft_consensus.cc:697] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:33.021592 20848 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:33.042268 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 144446us
I20260709 12:13:33.043962 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:33.014633 21099 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:33.059013 13304 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.712s (SENT) [suppressed 18 similar messages]
I20260709 12:13:33.056546 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:33.060297 12531 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:33.068181 12531 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 27. Current term is 28. Ops: []
I20260709 12:13:33.070448 21111 consensus_queue.cc:237] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:33.041628 20848 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:33.074896 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:33.076290 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:33.078346 21087 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:33.078631 20952 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:33.078819 21087 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:33.079042 20952 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:33.079473 20952 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:33.063355 21038 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:33.083215 21058 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:33.038691 21095 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:33.039450 21089 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:33.087018 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:33.096805 12603 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:33.087961 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:33.078260 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:33.097965 12598 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:33.102795 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.031663 21001 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:33.107005 12526 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 24: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:33.079303 21087 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:33.038968 21045 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:33.113718 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.096060 21103 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:33.125385 12598 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:13:33.126456 20997 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:33.130108 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:33.138311 20592 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:33.138862 20592 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:33.152057 21105 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:33.153239 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:33.153304 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:33.154202 13331 raft_consensus.cc:2368] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23.
I20260709 12:13:33.099583 20872 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.030s
I20260709 12:13:33.178318 12603 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:33.193002 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:33.192724 20950 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:33.196391 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:33.114714 21029 consensus_queue.cc:237] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:33.129282 21103 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:33.209462 13331 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:33.175132 21099 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:33.174054 21045 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:33.218228 21047 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 1.743s	user 0.004s	sys 0.001s
I20260709 12:13:33.171828 21095 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:33.213618 21083 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:33.222656 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3374) took 1366 ms. Trace:
I20260709 12:13:33.223066 12678 rpcz_store.cc:276] 0709 12:13:31.855852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.216576 (+360724us) service_pool.cc:224] Handling call
0709 12:13:33.222480 (+1005904us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b54003d4ad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:33.222601 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1265258880}
I20260709 12:13:33.224259 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:33.224802 12678 raft_consensus.cc:2376] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26. Re-sending same reply.
I20260709 12:13:33.225804 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:33.227830 12678 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:33.230420 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:33.232056 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:33.232301 21049 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:13:33.232679 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:33.233794 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:33.234385 21049 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:33.234957 21049 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:33.235491 12599 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:13:33.234539 12602 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 23: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:33.227887 21069 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:33.240003 12643 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:33.248353 12643 leader_election.cc:400] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23.
I20260709 12:13:33.242859 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.251932 12643 leader_election.cc:403] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:13:33.252297 12604 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:33.245582 21103 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:33.253186 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:33.246984 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:33.242074 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.248574 12673 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:33.249204 21001 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:33.255390 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:33.249688 21105 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:33.256142 21105 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:33.256385 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 18 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:33.256646 21105 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:33.255154 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.254016 12604 raft_consensus.cc:2368] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23.
I20260709 12:13:33.265007 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.256052 12597 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:33.247087 12678 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:13:33.269665 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:33.273479 12678 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
I20260709 12:13:33.263590 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:33.285101 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.285001 12599 raft_consensus.cc:2393] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 18: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:33.285908 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.292191 21001 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:33.296070 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.288228 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:33.295353 12571 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:33.304915 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:33.306353 12604 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 27: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:33.306703 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.307832 12603 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:33.293810 21134 raft_consensus.cc:3055] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:33.319949 12597 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:13:33.321390 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:33.322201 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.313290 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:33.323555 12603 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:33.307253 21008 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:33.325102 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:33.325719 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:33.326493 12671 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 18.
I20260709 12:13:33.323223 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4006) took 3001 ms. Trace:
I20260709 12:13:33.327332 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:33.324252 12571 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:33.286176 20744 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23.
I20260709 12:13:33.327471 12675 rpcz_store.cc:276] 0709 12:13:30.307766 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:30.794677 (+486911us) service_pool.cc:224] Handling call
0709 12:13:30.824618 (+ 29941us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:31.673724 (+849106us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:33.309162 (+1635438us) spinlock_profiling.cc:243] Waited 848 ms on lock 0x7b540036cc54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:33.309333 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1950965120}
I20260709 12:13:33.328727 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:33.306916 21058 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:33.337073 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.191963 21113 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:33.332160 20710 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:33.338929 21008 raft_consensus.cc:697] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 23 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:33.339565 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:33.387653 12678 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:33.339720 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:33.340728 20839 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:33.338946 20952 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:33.342243 12604 pending_rounds.cc:85] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 16
I20260709 12:13:33.394317 12604 pending_rounds.cc:107] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 23.17
I20260709 12:13:33.394582 12604 raft_consensus.cc:2889] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:33.344728 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.324944 21073 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:13:33.383674 12603 raft_consensus.cc:2410] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 27 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 12:13:33.322710 21134 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:33.398515 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.393931 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:33.400107 12675 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
I20260709 12:13:33.401010 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:33.332003 21127 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:33.401297 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:33.419369 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.420159 12597 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 27: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:33.421029 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.421635 12603 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 28.
I20260709 12:13:33.422435 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.422925 12603 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22.
I20260709 12:13:33.423668 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:33.405572 21008 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:33.383232 21058 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:33.428279 21058 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:33.407788 21113 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:33.408602 21134 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.14, Last appended by leader: 14, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:33.435379 21134 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:33.437175 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.410492 21073 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:33.438505 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:33.439244 12601 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 22: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:33.438946 21073 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:33.440039 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.440795 12601 raft_consensus.cc:2410] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 26 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
I20260709 12:13:33.441591 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 20 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.412250 21009 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:33.393997 20710 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:33.390898 20839 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:33.455902 20799 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:33.413519 20848 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:33.418335 20952 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:33.424909 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:33.425216 12678 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:33.427168 21102 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:13:33.411976 21143 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:13:33.444054 12601 raft_consensus.cc:2410] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 27 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 17.
I20260709 12:13:33.447204 21127 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:33.456548 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:33.485443 12531 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:33.458392 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:33.451087 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.453091 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:33.450052 13305 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:33.449699 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.468214 12673 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:13:33.479291 12494 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:33.457427 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:33.492195 12526 raft_consensus.cc:2368] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 27. Current term is 28.
I20260709 12:13:33.456974 20710 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:33.492565 13305 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:33.470803 21014 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:33.493446 21014 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:33.494050 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:33.494943 13305 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:33.495674 21098 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:33.495455 21049 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:33.496248 21098 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:33.496733 21103 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:33.496834 21098 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:33.456208 21100 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:33.498375 20827 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:33.482973 21102 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:33.499081 21102 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:33.497223 21103 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:33.500428 21103 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:33.503168 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:33.504992 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:33.506359 21091 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:33.507206 21091 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:33.507448 13337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38482 (request call id 794) took 1151 ms. Trace:
I20260709 12:13:33.507700 21091 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:33.507928 13337 rpcz_store.cc:276] 0709 12:13:32.355720 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.355896 (+   176us) service_pool.cc:224] Handling call
0709 12:13:32.362683 (+  6787us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:32.698151 (+335468us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:33.507145 (+808994us) spinlock_profiling.cc:243] Waited 335 ms on lock 0x7b540046a5d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:33.507355 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":769852672}
I20260709 12:13:33.469884 20997 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:33.509413 20997 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:33.456403 20839 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:33.457521 20799 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:33.513377 20799 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:33.514325 20872 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:13:33.514756 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.516544 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4030) took 1223 ms. Trace:
I20260709 12:13:33.516629 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.517001 12676 rpcz_store.cc:276] 0709 12:13:32.293092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:33.059830 (+766738us) service_pool.cc:224] Handling call
0709 12:13:33.060128 (+   298us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:33.060523 (+   395us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:33.060555 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:33.061094 (+   539us) log.cc:844] Serialized 26 byte log entry
0709 12:13:33.119257 (+ 58163us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:33.119298 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:33.119371 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:33.516331 (+396960us) raft_consensus.cc:1685] finished
0709 12:13:33.516351 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:33.516478 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2518,"raft.run_cpu_time_us":111,"raft.run_wall_time_us":112,"thread_start_us":57765,"threads_started":1,"wal-append.queue_time_us":121145}
I20260709 12:13:33.519064 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:33.520228 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:33.498428 21100 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:33.506044 12601 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:33.523159 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:33.523859 12673 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:33.524279 21100 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:33.524680 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:33.525285 12673 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:33.526597 20985 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:33.527094 20985 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:33.497323 21063 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:33.528307 21063 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:33.518481 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.529503 20985 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:33.474916 20741 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:13:33.530762 21084 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:13:33.532220 21084 raft_consensus.cc:3055] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:33.532737 21084 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:33.537925 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:33.538450 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 59242.1us
I20260709 12:13:33.539372 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:33.541291 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:33.528709 21063 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:33.558108 12601 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
W20260709 12:13:33.559413 21059 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 2.019s	user 0.001s	sys 0.008s
I20260709 12:13:33.560248 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:33.560906 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:33.561209 12601 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:33.562065 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:33.546396 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:33.550077 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:33.551908 21005 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:13:33.552647 21098 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:33.553596 21127 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:33.455999 21095 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:33.552699 12416 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 70b8c9b89e3d43fbb70422110d76702c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica a2fdbe7eef6748a9b4611f6c9bc9b201 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:13:33.562219 12572 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:33.583844 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:33.585014 12572 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:33.545189 13305 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:33.565517 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:33.571277 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:33.572587 12531 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:13:33.578747 12525 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:13:33.586594 13337 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:33.586984 12525 raft_consensus.cc:606] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:33.495546 21014 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:33.530105 20985 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:33.600464 20985 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:33.602843 20985 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:33.467736 21009 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:33.409989 20592 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:33.590272 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3279) took 1181 ms. Trace:
I20260709 12:13:33.603780 20935 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:33.603863 12530 rpcz_store.cc:276] 0709 12:13:32.408421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.408567 (+   146us) service_pool.cc:224] Handling call
0709 12:13:32.409039 (+   472us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:32.573788 (+164749us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:33.589876 (+1016088us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54003c1c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:33.590183 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":374278016}
I20260709 12:13:33.590572 12603 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:13:33.587513 12525 raft_consensus.cc:614] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:33.597359 12601 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
W20260709 12:13:33.606654 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:33.608718 20592 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:33.634732 12598 pending_rounds.cc:85] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 13
I20260709 12:13:33.599002 20950 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:33.582854 21000 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:33.461555 20848 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:33.560381 20600 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:33.568041 21100 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:13:33.533728 21084 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:33.611105 12525 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:13:33.659821 12525 raft_consensus.cc:606] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:33.660311 12525 raft_consensus.cc:614] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:13:33.661237 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:33.611958 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3344) took 1284 ms. Trace:
I20260709 12:13:33.661960 12606 rpcz_store.cc:276] 0709 12:13:32.327362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.335067 (+  7705us) service_pool.cc:224] Handling call
0709 12:13:33.611792 (+1276725us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b54002ff654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:33.611904 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":595649408}
I20260709 12:13:33.663196 21063 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:33.605449 21098 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:33.664363 12606 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:13:33.612263 12602 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 20 index: 9. Preceding OpId from leader: term: 21 index: 10. (index mismatch)
I20260709 12:13:33.618914 12530 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:33.665920 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:33.666527 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:33.667640 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:33.622445 21014 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
W20260709 12:13:33.624789 21123 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.833s	user 0.000s	sys 0.004s
I20260709 12:13:33.625383 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:33.627008 12522 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:33.627228 21049 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:33.629899 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:33.630954 12603 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:33.592427 20853 raft_consensus.cc:2764] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:13:33.567370 21113 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:13:33.635581 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3335) took 4586 ms. Trace:
I20260709 12:13:33.692665 12605 rpcz_store.cc:276] 0709 12:13:29.048921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:29.605802 (+556881us) service_pool.cc:224] Handling call
0709 12:13:29.606233 (+   431us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:29.606761 (+   528us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:13:30.284081 (+677320us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b1800194140. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf 00007f1488a3a038
0709 12:13:30.597964 (+313883us) log.cc:844] Serialized 14 byte log entry
0709 12:13:31.146022 (+548058us) log.cc:844] Serialized 14 byte log entry
0709 12:13:31.146229 (+   207us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:31.156344 (+ 10115us) log.cc:844] Serialized 52 byte log entry
0709 12:13:31.156520 (+   176us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:31.156796 (+   276us) log.cc:844] Serialized 14 byte log entry
0709 12:13:31.438174 (+281378us) log.cc:844] Serialized 14 byte log entry
0709 12:13:31.581877 (+143703us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:31.582180 (+   303us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:33.558800 (+1976620us) spinlock_profiling.cc:243] Waited 7.33 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:33.635350 (+ 76550us) raft_consensus.cc:1685] finished
0709 12:13:33.635374 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:33.635523 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":222403,"prepare.queue_time_us":950399,"prepare.run_cpu_time_us":188,"prepare.run_wall_time_us":7791,"raft.queue_time_us":626510,"raft.run_cpu_time_us":732,"raft.run_wall_time_us":4227,"spinlock_wait_cycles":23100032,"thread_start_us":1008511,"threads_started":4,"wal-append.queue_time_us":887819}
I20260709 12:13:33.588161 20827 raft_consensus.cc:2764] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election decision vote started in defunct term 19: won
I20260709 12:13:33.695873 21031 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:33.592741 20747 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:33.636065 12671 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:33.698431 12603 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:33.698841 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:33.700475 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:33.701228 12671 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:33.638803 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:33.592857 20952 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:13:33.640244 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:33.641947 12598 pending_rounds.cc:107] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 24.14
I20260709 12:13:33.706477 12598 raft_consensus.cc:2889] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:33.641866 12569 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:33.707633 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 65848.2us
I20260709 12:13:33.642863 12672 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:13:33.591130 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:33.592232 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 47182.2us
I20260709 12:13:33.712029 12531 raft_consensus.cc:2368] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23.
I20260709 12:13:33.597662 12494 leader_election.cc:400] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23.
I20260709 12:13:33.581106 21081 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:33.654736 20950 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:33.668464 20592 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:33.715641 12603 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
W20260709 12:13:33.670245 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:33.670665 12606 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: [23.13-23.13]
I20260709 12:13:33.675091 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3361) took 2107 ms. Trace:
I20260709 12:13:33.718730 12606 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: []
W20260709 12:13:33.679864 12422 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47524 (request call id 103) took 17679 ms (client timeout 15000 ms). Trace:
I20260709 12:13:33.681300 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:33.682241 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:33.721328 12529 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:33.722466 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:33.675053 13335 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 930) took 1539 ms. Trace:
I20260709 12:13:33.683924 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:33.668026 21084 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:33.703325 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:33.713706 12494 leader_election.cc:403] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:13:33.725206 20950 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:33.725775 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:33.727679 21091 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:33.728827 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 137799us
I20260709 12:13:33.780055 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:33.731036 21102 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:33.705994 13337 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:13:33.733966 21103 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:33.595046 21095 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:33.696458 21031 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:33.735401 21087 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.617s	user 0.006s	sys 0.005s
I20260709 12:13:33.709600 21105 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:33.834761 12677 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:33.835922 21105 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:33.716269 21081 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:33.736207 21058 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:33.836670 21081 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:33.733664 20839 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:33.838368 21058 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:33.739600 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:33.839278 12676 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:33.739527 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:33.840471 12599 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:33.750164 20592 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:33.742383 21063 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:33.841503 21063 raft_consensus.cc:697] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:33.741390 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:33.841656 20592 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:33.753425 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:33.752661 12605 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:33.842885 12527 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:13:33.843595 12527 raft_consensus.cc:606] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:33.843945 12527 raft_consensus.cc:614] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260709 12:13:33.762517 21123 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:13:33.759958 21000 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:33.845475 21000 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:33.762897 20827 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:33.846365 20827 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:33.847152 20827 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:33.846599 21000 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:33.696979 21113 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:33.764578 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:33.848883 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:33.766631 12529 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 26: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:33.768700 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:33.723472 13335 rpcz_store.cc:276] 0709 12:13:32.135676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.135814 (+   138us) service_pool.cc:224] Handling call
0709 12:13:32.154891 (+ 19077us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:32.172914 (+ 18023us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:32.172948 (+    34us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:32.172991 (+    43us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:13:32.173031 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:33.674758 (+1501727us) spinlock_profiling.cc:243] Waited 17.8 ms on lock 0x7b54004439d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:33.674825 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:33.674978 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":40951680}
I20260709 12:13:33.781674 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:33.849534 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:33.734195 12671 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:13:33.789770 12572 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:33.717468 12674 rpcz_store.cc:276] 0709 12:13:31.567287 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:31.640149 (+ 72862us) service_pool.cc:224] Handling call
0709 12:13:31.640744 (+   595us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:32.474530 (+833786us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:13:32.536626 (+ 62096us) log.cc:844] Serialized 14 byte log entry
0709 12:13:32.642806 (+106180us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:32.642862 (+    56us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:32.642905 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:33.674849 (+1031944us) spinlock_profiling.cc:243] Waited 833 ms on lock 0x7b540016f3d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:33.674908 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:33.675027 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13019,"prepare.queue_time_us":114277,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":39,"raft.queue_time_us":13132,"raft.run_cpu_time_us":112,"raft.run_wall_time_us":1926,"spinlock_wait_cycles":1916929792,"thread_start_us":121294,"threads_started":2,"wal-append.queue_time_us":93060,"wal-append.run_cpu_time_us":1150,"wal-append.run_wall_time_us":1005702}
I20260709 12:13:33.792748 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:33.786649 12646 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:33.777773 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 180235us
W20260709 12:13:33.852727 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 66191.9us
I20260709 12:13:33.800624 13331 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 26: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:33.853525 12494 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:33.849448 12597 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:33.802448 21102 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:33.808245 21095 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:33.795192 20985 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:33.809226 21049 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:33.824127 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:33.826710 21031 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:33.831194 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:33.835951 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4038) took 1419 ms. Trace:
I20260709 12:13:33.880403 12677 rpcz_store.cc:276] 0709 12:13:32.416422 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:33.461343 (+1044921us) service_pool.cc:224] Handling call
0709 12:13:33.461631 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:33.835185 (+373554us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:33.835711 (+   526us) spinlock_profiling.cc:243] Waited 373 ms on lock 0x7b54002c4f54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:33.835889 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":857652992}
I20260709 12:13:33.840564 12523 raft_consensus.cc:2368] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23.
I20260709 12:13:33.763880 13305 leader_election.cc:400] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 27. Current term is 28.
I20260709 12:13:33.881474 13305 leader_election.cc:403] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 election: Cancelling election due to peer responding with higher term
I20260709 12:13:33.849185 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3351) took 1519 ms. Trace:
I20260709 12:13:33.763945 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:33.782325 21141 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23.
I20260709 12:13:33.885726 12677 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:33.885926 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:33.887122 12677 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:33.888475 12677 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:33.805261 21103 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:33.858860 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:33.890081 12676 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:13:33.860404 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:13:33.860947 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 71307.4us
I20260709 12:13:33.892107 12572 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:33.862032 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:33.862908 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:33.868423 12646 leader_election.cc:400] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23.
I20260709 12:13:33.893042 12524 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:33.873978 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:33.875341 12674 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:13:33.894958 12524 raft_consensus.cc:3055] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:33.895386 12524 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:33.896240 12674 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 24 index: 12. (index mismatch)
I20260709 12:13:33.882331 12600 rpcz_store.cc:276] 0709 12:13:32.329418 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.450576 (+121158us) service_pool.cc:224] Handling call
0709 12:13:32.461361 (+ 10785us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:32.461745 (+   384us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:13:32.623744 (+161999us) log.cc:844] Serialized 14 byte log entry
0709 12:13:32.703262 (+ 79518us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:32.709820 (+  6558us) log.cc:844] Serialized 26 byte log entry
0709 12:13:32.746603 (+ 36783us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:32.746856 (+   253us) log.cc:844] Serialized 14 byte log entry
0709 12:13:32.762708 (+ 15852us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:32.762797 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:33.460552 (+697755us) spinlock_profiling.cc:243] Waited 7.29 ms on lock 0x7b1800170200. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:13:33.624204 (+163652us) spinlock_profiling.cc:243] Waited 1.98 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:33.848970 (+224766us) raft_consensus.cc:1685] finished
0709 12:13:33.848993 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:33.849123 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":131337,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":83,"raft.queue_time_us":19347,"raft.run_cpu_time_us":385,"raft.run_wall_time_us":10374,"spinlock_wait_cycles":21321344,"thread_start_us":152303,"threads_started":2,"wal-append.queue_time_us":106531}
I20260709 12:13:33.892269 12672 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 22: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:33.842782 20592 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:13:33.898398 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 134771us
I20260709 12:13:33.899259 13305 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:33.904628 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 281894us
I20260709 12:13:33.892462 12677 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:13:33.905015 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:33.894446 12646 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:33.899533 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:33.920658 12671 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:13:33.922230 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:33.848882 21113 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:33.869545 21000 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:33.936664 12494 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:33.882161 21009 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:33.940096 12523 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:13:33.930234 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:33.941511 12529 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:33.933069 12646 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:33.882483 21073 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:33.881065 21101 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:33.947252 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:33.878717 20985 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:33.953061 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:33.896016 20839 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:33.921968 12524 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:33.948735 20747 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:33.943487 21103 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:33.940655 21100 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:33.961529 12494 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:33.961794 12646 leader_election.cc:400] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23.
W20260709 12:13:33.875229 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 338527us
I20260709 12:13:33.861037 21087 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:34.019075 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:34.019361 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:33.842643 21063 consensus_queue.cc:237] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.10, Last appended by leader: 10, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:33.947063 21095 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:34.012648 12524 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:33.882972 20952 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.074s
I20260709 12:13:34.023402 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:34.013028 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 43830.7us
I20260709 12:13:34.035095 12496 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:33.891623 21049 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:34.031317 13332 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:13:33.950780 21009 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:34.020537 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 58900.5us
I20260709 12:13:34.045514 12672 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
W20260709 12:13:34.035539 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 74071.7us
I20260709 12:13:34.024691 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:34.047868 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:33.968448 20705 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:13:34.047192 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:33.720080 12422 rpcz_store.cc:269] 0709 12:13:16.000395 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:16.000774 (+   379us) service_pool.cc:224] Handling call
0709 12:13:33.679708 (+17678934us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:27.686071 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:27.732548 (+ 46477us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:27.732614 (+    66us) write_op.cc:435] Acquired schema lock
0709 12:13:27.732649 (+    35us) tablet.cc:662] Decoding operations
0709 12:13:27.914634 (+181985us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:27.914753 (+   119us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:27.914778 (+    25us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:13:28.020707 (+105929us) tablet.cc:701] Row locks acquired
0709 12:13:28.036755 (+ 16048us) write_op.cc:260] PREPARE: finished
0709 12:13:28.128615 (+ 91860us) write_op.cc:270] Start()
0709 12:13:28.158041 (+ 29426us) write_op.cc:276] Timestamp: P: 1783599208117759 usec, L: 0
0709 12:13:28.163037 (+  4996us) op_driver.cc:348] REPLICATION: starting
0709 12:13:28.277942 (+114905us) log.cc:844] Serialized 5196 byte log entry
0709 12:13:28.628181 (+350239us) spinlock_profiling.cc:243] Waited 3.11 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14859160ac 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b
0709 12:13:28.919695 (+291514us) op_driver.cc:464] REPLICATION: finished
0709 12:13:29.072742 (+153047us) spinlock_profiling.cc:243] Waited 21.1 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14859160ac 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f148c456ef0
0709 12:13:29.345236 (+272494us) write_op.cc:301] APPLY: starting
0709 12:13:29.350403 (+  5167us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:29.600518 (+250115us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:29.600565 (+    47us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:29.941410 (+340845us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:30.048732 (+107322us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=26,mrs_lookups=0
0709 12:13:30.048816 (+    84us) write_op.cc:312] APPLY: finished
0709 12:13:30.049334 (+   518us) log.cc:844] Serialized 121 byte log entry
0709 12:13:30.624827 (+575493us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:30.669786 (+ 44959us) write_op.cc:454] Released schema lock
0709 12:13:30.710044 (+ 40258us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:30.820783 (+110739us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:30.827852 (+  7069us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":7424470,"reported_tablets":21,"child_traces":[["op",{"apply.queue_time_us":407972,"apply.run_cpu_time_us":48432,"apply.run_wall_time_us":1439247,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58389,"num_ops":13,"prepare.queue_time_us":340629,"prepare.run_cpu_time_us":54725,"prepare.run_wall_time_us":1095407,"raft.queue_time_us":70704,"raft.run_cpu_time_us":26087,"raft.run_wall_time_us":275080,"replication_time_us":744179,"spinlock_wait_cycles":55683712,"thread_start_us":1447899,"threads_started":5,"wal-append.queue_time_us":1294319,"wal-append.run_cpu_time_us":36029,"wal-append.run_wall_time_us":2710673}]]}
I20260709 12:13:33.935055 21014 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:33.928202 21102 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader election lost for term 27. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 27. Current term is 28.
I20260709 12:13:34.046676 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:33.936299 21069 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:13:33.935262 21058 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:34.013615 20848 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:33.968111 20747 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:34.058804 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:34.061828 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:34.049223 20997 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.535s	user 0.011s	sys 0.000s
I20260709 12:13:34.202685 12524 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:13:34.024190 21115 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:13:33.839556 12423 catalog_manager.cc:5697] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 9 to 13, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 13 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:13:34.219873 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:34.061844 21101 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:33.956794 21105 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:33.957405 20985 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:34.056298 21008 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.159s
I20260709 12:13:34.051082 21009 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:13:34.203172 21134 consensus_meta.cc:287] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.767s	user 0.011s	sys 0.000s
I20260709 12:13:34.206573 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:34.226171 12676 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:13:34.228011 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:34.215371 20747 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:34.236250 20710 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.595s	user 0.024s	sys 0.059s
I20260709 12:13:34.212111 21095 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:34.214017 21141 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:33.970196 21083 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.252s
I20260709 12:13:34.276258 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 969) took 1059 ms. Trace:
I20260709 12:13:34.276687 12678 rpcz_store.cc:276] 0709 12:13:33.221507 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:33.477586 (+256079us) service_pool.cc:224] Handling call
0709 12:13:33.478014 (+   428us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:33.481141 (+  3127us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:33.481179 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:33.487438 (+  6259us) log.cc:844] Serialized 26 byte log entry
0709 12:13:33.948396 (+460958us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:13:33.948452 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:33.948523 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:34.276007 (+327484us) raft_consensus.cc:1685] finished
0709 12:13:34.276031 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:34.276195 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2704,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":1685,"thread_start_us":278800,"threads_started":1,"wal-append.queue_time_us":415404}
I20260709 12:13:34.219090 20872 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.051s
I20260709 12:13:34.280905 20600 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:34.281421 20600 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:34.281800 20600 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:34.215094 21005 consensus_queue.cc:1059] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 22.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.054s
I20260709 12:13:34.283054 12423 catalog_manager.cc:5697] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 8 to 11. New cstate: current_term: 11 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
W20260709 12:13:34.266690 20710 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.626s	user 0.025s	sys 0.059s
I20260709 12:13:34.246268 21091 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:34.292236 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 330171us
I20260709 12:13:34.261922 20848 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:34.218943 20997 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:34.287088 21155 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:13:34.273835 21058 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:34.285347 12423 catalog_manager.cc:5697] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 7 to 12. New cstate: current_term: 12 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:34.292411 21087 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:34.330839 21101 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:34.332403 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 319167us
I20260709 12:13:34.345010 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:34.347412 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:34.270633 21095 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:34.318941 20799 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:34.301362 21103 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.037s
I20260709 12:13:34.363947 12529 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:13:34.364737 12571 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:34.372138 12522 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:34.365728 13305 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:34.381240 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:34.373448 12571 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:34.379026 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:34.380847 21159 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
I20260709 12:13:34.324836 21058 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:34.392577 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:34.395972 12530 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
W20260709 12:13:34.395964 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 349331us
I20260709 12:13:34.400092 12522 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26.
I20260709 12:13:34.369994 21100 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:34.404663 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:34.422564 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 56958.1us
I20260709 12:13:34.406142 12496 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:13:34.426985 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:34.410648 12606 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:13:34.434659 12646 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:34.412137 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:34.413535 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:34.438289 12600 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:34.414804 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:34.420158 12599 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:13:34.425737 12496 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:34.447680 12603 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
W20260709 12:13:34.439960 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47468.3us
I20260709 12:13:34.449966 12571 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:34.454782 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:34.432716 21008 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:13:34.457026 21014 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:13:34.460083 13332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 1015) took 1335 ms. Trace:
I20260709 12:13:34.460515 13332 rpcz_store.cc:276] 0709 12:13:33.124678 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:33.124803 (+   125us) service_pool.cc:224] Handling call
0709 12:13:33.126850 (+  2047us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:34.043562 (+916712us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:34.459774 (+416212us) spinlock_profiling.cc:243] Waited 904 ms on lock 0x7b54001b49d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:34.460015 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2079158528}
I20260709 12:13:34.447026 20799 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:34.463853 20799 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:34.464234 20799 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:34.422549 20710 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:13:34.451565 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 390504us
I20260709 12:13:34.381423 21101 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:34.447832 20839 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Leader pre-election won for term 19
I20260709 12:13:34.433416 21049 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:34.381588 20848 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:34.447273 21098 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
I20260709 12:13:34.488983 12597 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:13:34.475569 21174 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:34.479434 21154 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
W20260709 12:13:34.483654 21153 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.268s	user 0.001s	sys 0.000s
I20260709 12:13:34.474592 12531 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
W20260709 12:13:34.492055 21156 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.226s	user 0.001s	sys 0.000s
W20260709 12:13:34.513713 21161 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.239s	user 0.000s	sys 0.001s
I20260709 12:13:34.440598 21172 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 12:13:34.500764 21170 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:34.528198 12531 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:34.564693 12531 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:13:34.503679 20839 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:34.478549 21141 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.054s
W20260709 12:13:34.484272 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 256343us
I20260709 12:13:34.507517 20741 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:34.507723 20997 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:34.491300 21115 consensus_queue.cc:1059] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 21.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.071s
I20260709 12:13:34.569952 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:34.503732 20872 consensus_queue.cc:1059] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.063s
W20260709 12:13:34.589491 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:34.591729 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4019) took 2743 ms. Trace:
I20260709 12:13:34.592159 21031 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:34.592612 21031 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:34.592114 12680 rpcz_store.cc:276] 0709 12:13:31.848748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:32.846096 (+997348us) service_pool.cc:224] Handling call
0709 12:13:32.846586 (+   490us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:33.616178 (+769592us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:13:33.935805 (+319627us) log.cc:844] Serialized 14 byte log entry
0709 12:13:34.590863 (+655058us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:34.590918 (+    55us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:34.590963 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:34.591511 (+   548us) spinlock_profiling.cc:243] Waited 769 ms on lock 0x7b540016eed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:34.591552 (+    41us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:34.591671 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":309010,"prepare.queue_time_us":130052,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"spinlock_wait_cycles":1768443136,"thread_start_us":656505,"threads_started":2,"wal-append.queue_time_us":497464}
I20260709 12:13:34.503999 20853 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 12:13:34.489807 21184 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.089s
I20260709 12:13:34.598634 12676 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:34.601001 12675 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:34.608160 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:34.573549 20839 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:34.491163 21162 consensus_queue.cc:1059] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 18.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.090s
I20260709 12:13:34.613698 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:34.613803 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:34.615737 12680 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
W20260709 12:13:34.618109 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 168247us
W20260709 12:13:34.604534 21165 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
W20260709 12:13:34.623034 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 188442us
W20260709 12:13:34.627339 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:34.654054 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:34.569978 20741 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:34.537032 21174 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:34.600880 21091 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:34.612658 21084 consensus_meta.cc:287] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.845s	user 0.012s	sys 0.000s
I20260709 12:13:34.685127 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 970) took 1463 ms. Trace:
W20260709 12:13:34.617766 21168 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
I20260709 12:13:34.714248 12600 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 19.
I20260709 12:13:34.611891 21179 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:34.698907 21182 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:13:34.700554 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 92444.3us
I20260709 12:13:34.732918 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:34.702106 21169 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
W20260709 12:13:34.733685 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 133216us
I20260709 12:13:34.703449 20935 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:34.684881 21095 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:34.685578 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 44885.6us
I20260709 12:13:34.649948 21127 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:34.702836 20202 raft_consensus.cc:3055] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:34.734015 12494 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:34.490732 12423 catalog_manager.cc:5697] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 10 to 15, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 15 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:34.839793 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:34.696074 12673 rpcz_store.cc:276] 0709 12:13:33.221792 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:33.526013 (+304221us) service_pool.cc:224] Handling call
0709 12:13:33.526365 (+   352us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:33.526748 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:33.526781 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:33.527374 (+   593us) log.cc:844] Serialized 52 byte log entry
0709 12:13:33.981323 (+453949us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:13:34.032741 (+ 51418us) log.cc:844] Serialized 14 byte log entry
0709 12:13:34.046519 (+ 13778us) log.cc:844] Serialized 14 byte log entry
0709 12:13:34.046765 (+   246us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:34.046848 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:34.588668 (+541820us) spinlock_profiling.cc:243] Waited 6.69 ms on lock 0x7b18002cfb00. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f148c8f458f 00007f1485933571 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd
0709 12:13:34.684857 (+ 96189us) raft_consensus.cc:1685] finished
0709 12:13:34.684901 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:34.685066 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24805,"prepare.queue_time_us":256685,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":47,"raft.queue_time_us":27213,"raft.run_cpu_time_us":392,"raft.run_wall_time_us":149544,"spinlock_wait_cycles":54177536,"thread_start_us":453498,"threads_started":1,"wal-append.queue_time_us":744144}
I20260709 12:13:34.712920 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4037) took 2351 ms. Trace:
I20260709 12:13:34.841429 12679 rpcz_store.cc:276] 0709 12:13:32.361341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:33.233711 (+872370us) service_pool.cc:224] Handling call
0709 12:13:33.233976 (+   265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:33.249136 (+ 15160us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:33.249165 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:33.249662 (+   497us) log.cc:844] Serialized 26 byte log entry
0709 12:13:33.712142 (+462480us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:33.712191 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:33.712308 (+   117us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:34.712681 (+1000373us) raft_consensus.cc:1685] finished
0709 12:13:34.712701 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:34.712852 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1019,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"spinlock_wait_cycles":349056,"thread_start_us":461451,"threads_started":1,"wal-append.queue_time_us":772699}
W20260709 12:13:34.712460 21163 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.331s	user 0.000s	sys 0.001s
W20260709 12:13:34.730242 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 110579us
I20260709 12:13:34.683137 20705 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:13:34.793700 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 51529us
I20260709 12:13:34.849270 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:34.839308 12530 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
W20260709 12:13:34.741228 21177 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.320s	user 0.001s	sys 0.001s
I20260709 12:13:34.735221 20853 raft_consensus.cc:3055] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:13:34.848207 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3378) took 1187 ms. Trace:
W20260709 12:13:34.803534 21167 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.370s	user 0.001s	sys 0.001s
I20260709 12:13:34.772511 21058 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:34.722708 21091 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:34.794540 20202 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:34.841778 21182 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:13:34.798592 21031 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:34.851078 12527 rpcz_store.cc:276] 0709 12:13:33.684496 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:33.846089 (+161593us) service_pool.cc:224] Handling call
0709 12:13:33.858170 (+ 12081us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:33.858436 (+   266us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:13:33.893915 (+ 35479us) log.cc:844] Serialized 14 byte log entry
0709 12:13:34.280561 (+386646us) spinlock_profiling.cc:243] Waited 17 ms on lock 0x7b18003c8aa0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a 00007f1488a2aa2a
0709 12:13:34.556279 (+275718us) log.cc:844] Serialized 14 byte log entry
0709 12:13:34.845604 (+289325us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:34.845819 (+   215us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:34.846179 (+   360us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:34.847938 (+  1759us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:34.848146 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":141849,"prepare.queue_time_us":80779,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":9071,"raft.queue_time_us":134410,"raft.run_cpu_time_us":283,"raft.run_wall_time_us":20460,"spinlock_wait_cycles":54272640,"thread_start_us":413100,"threads_started":3,"wal-append.queue_time_us":610124}
I20260709 12:13:34.864589 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:34.865316 12529 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:13:34.802933 21134 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.065s
I20260709 12:13:34.855682 20853 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:34.859469 21113 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.909s	user 0.012s	sys 0.000s
I20260709 12:13:34.879417 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:34.895138 12529 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:13:34.881706 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3863) took 1438 ms. Trace:
I20260709 12:13:34.886653 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:34.889997 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:34.739655 21179 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:34.900408 12643 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:34.895969 12598 rpcz_store.cc:276] 0709 12:13:33.443391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:33.633969 (+190578us) service_pool.cc:224] Handling call
0709 12:13:33.634406 (+   437us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:13:33.729525 (+ 95119us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:33.729558 (+    33us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:13:33.730350 (+   792us) log.cc:844] Serialized 78 byte log entry
0709 12:13:34.227015 (+496665us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:34.264477 (+ 37462us) log.cc:844] Serialized 14 byte log entry
0709 12:13:34.466720 (+202243us) spinlock_profiling.cc:243] Waited 6.31 ms on lock 0x7b1800486d00. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf 00007f1488a3a038
0709 12:13:34.466942 (+   222us) log.cc:844] Serialized 14 byte log entry
0709 12:13:34.648093 (+181151us) log.cc:844] Serialized 14 byte log entry
0709 12:13:34.740847 (+ 92754us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:34.793946 (+ 53099us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:34.881367 (+ 87421us) raft_consensus.cc:1685] finished
0709 12:13:34.881401 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:34.881620 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":192216,"prepare.queue_time_us":38651,"prepare.run_wall_time_us":33,"raft.queue_time_us":276451,"raft.run_cpu_time_us":549,"raft.run_wall_time_us":42584,"spinlock_wait_cycles":24944640,"thread_start_us":676162,"threads_started":2,"wal-append.queue_time_us":640061}
W20260709 12:13:34.905092 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 171146us
I20260709 12:13:34.906517 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:34.905448 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:34.917821 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:34.887959 20705 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:34.920485 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:34.887358 21031 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:34.895902 20997 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:34.895710 21084 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:34.939049 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3366) took 1314 ms. Trace:
I20260709 12:13:34.939741 12526 rpcz_store.cc:276] 0709 12:13:33.624750 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:33.843754 (+219004us) service_pool.cc:224] Handling call
0709 12:13:33.844184 (+   430us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:33.844623 (+   439us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:33.844669 (+    46us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:33.849843 (+  5174us) log.cc:844] Serialized 52 byte log entry
0709 12:13:34.326391 (+476548us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:34.364801 (+ 38410us) log.cc:844] Serialized 14 byte log entry
0709 12:13:34.367292 (+  2491us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:34.367374 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:34.938688 (+571314us) raft_consensus.cc:1685] finished
0709 12:13:34.938732 (+    44us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:34.938952 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2142,"prepare.queue_time_us":32198,"prepare.run_cpu_time_us":69,"prepare.run_wall_time_us":3933,"raft.queue_time_us":17828,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":11415,"thread_start_us":381641,"threads_started":1,"wal-append.queue_time_us":570094}
I20260709 12:13:34.916393 21184 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:13:34.946727 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 56806us
I20260709 12:13:34.944001 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:34.915961 20872 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:34.947763 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:34.915488 21180 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:13:34.921463 21036 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:34.954866 21174 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:34.955008 21063 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:13:34.956013 21202 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260709 12:13:34.944527 20950 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.163s	user 0.010s	sys 0.003s
I20260709 12:13:34.956918 21049 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:34.930051 20744 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:13:34.957065 21198 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.097s	user 0.000s	sys 0.000s
I20260709 12:13:34.963016 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:34.902060 21113 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:34.979493 12526 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
W20260709 12:13:34.964234 12416 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 70b8c9b89e3d43fbb70422110d76702c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica a2fdbe7eef6748a9b4611f6c9bc9b201 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:13:34.945979 20827 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.098s	user 0.016s	sys 0.000s
I20260709 12:13:34.957495 21190 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:34.919236 21162 consensus_queue.cc:1059] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.039s
I20260709 12:13:34.944800 20997 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:34.968958 20600 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.687s	user 0.011s	sys 0.000s
I20260709 12:13:34.921212 20202 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:34.949308 20853 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:34.979106 21036 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:34.978631 20744 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:35.002089 20853 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:34.930243 12423 catalog_manager.cc:5697] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 8 to 12, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 12 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
W20260709 12:13:34.970515 20799 consensus_meta.cc:287] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.506s	user 0.012s	sys 0.000s
I20260709 12:13:34.957324 21127 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.014s
W20260709 12:13:34.972107 20592 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.042s	user 0.011s	sys 0.000s
I20260709 12:13:34.920010 20913 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.038s
W20260709 12:13:34.958390 20985 consensus_meta.cc:287] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.696s	user 0.007s	sys 0.004s
I20260709 12:13:34.923774 21068 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.039s
I20260709 12:13:34.922839 21105 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.039s
I20260709 12:13:34.945750 21172 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:13:34.963742 21099 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.698s	user 0.020s	sys 0.027s
W20260709 12:13:34.972595 21186 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.335s	user 0.000s	sys 0.001s
I20260709 12:13:34.984011 12529 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:13:34.933224 20705 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:34.981773 21194 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:35.031998 20913 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:35.032461 20913 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:35.033330 20913 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.035151 20913 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:35.002250 21036 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.035964 21172 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:34.973222 21180 raft_consensus.cc:697] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:34.988507 21174 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:13:35.001180 21009 consensus_meta.cc:287] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.732s	user 0.009s	sys 0.000s
I20260709 12:13:34.951375 21084 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:34.994344 21184 raft_consensus.cc:3055] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:13:34.975661 20950 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.054036 21184 raft_consensus.cc:740] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:35.058835 21184 consensus_queue.cc:260] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 18.11, Last appended by leader: 11, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.059861 20950 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:35.060163 21184 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 18 FOLLOWER]: Advancing to term 19
I20260709 12:13:35.010840 20799 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.063228 12522 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: [25.13-25.13]
W20260709 12:13:35.065083 21187 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.343s	user 0.000s	sys 0.002s
I20260709 12:13:34.961755 20872 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 23 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:35.066512 20872 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:35.001274 20600 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:34.982421 21175 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.088s
I20260709 12:13:34.995488 21193 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:13:35.070990 12525 raft_consensus.cc:2368] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22.
I20260709 12:13:34.996559 21113 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:34.997829 21179 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:35.054409 21068 consensus_queue.cc:1243] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 26.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:13:34.985145 20827 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:35.077334 20600 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:35.078783 20827 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:35.038976 21091 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:35.003257 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:34.941866 21031 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.079802 12531 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:13:35.081408 21031 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:35.035326 21099 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.789s	user 0.020s	sys 0.028s
I20260709 12:13:34.986616 21190 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:35.082875 20705 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:35.035181 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:35.057739 12423 catalog_manager.cc:5697] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 11 to 13, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 13 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:35.083704 12529 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 28: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:35.084149 21190 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:34.981230 21182 raft_consensus.cc:697] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:35.084519 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:35.085491 12529 raft_consensus.cc:2410] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 24 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:13:34.994011 21162 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:35.085238 21182 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:35.086289 21162 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:35.042862 20985 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.087169 21162 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.038442 20741 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:34.983069 21063 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:35.002559 20744 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:35.089147 21063 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:35.091238 21084 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:35.035063 21083 raft_consensus.cc:3055] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:35.065387 20799 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:35.054386 21009 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:35.073051 12525 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3369) took 1447 ms (client timeout 1626 ms). Trace:
W20260709 12:13:35.107416 12525 rpcz_store.cc:269] 0709 12:13:33.625536 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:33.660964 (+ 35428us) service_pool.cc:224] Handling call
0709 12:13:35.072761 (+1411797us) spinlock_profiling.cc:243] Waited 1.41 s on lock 0x7b540034fb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:35.072980 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3240574848}
I20260709 12:13:34.994961 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:35.075752 12524 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:35.076496 12527 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: [24.12-24.12]
W20260709 12:13:35.078881 21081 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.242s	user 0.015s	sys 0.000s
I20260709 12:13:34.999940 20997 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:35.112593 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 226038us
I20260709 12:13:34.976516 21049 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:35.031383 21181 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:35.104635 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:35.117049 12528 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
W20260709 12:13:35.105592 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 185180us
I20260709 12:13:35.085175 12423 catalog_manager.cc:5697] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 9 to 12, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 12 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:35.119347 12643 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:35.108651 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 115093us
W20260709 12:13:35.093225 21000 consensus_meta.cc:287] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.148s	user 0.013s	sys 0.000s
I20260709 12:13:35.111581 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:35.089160 20985 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:35.114323 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 197717us
I20260709 12:13:35.120605 12571 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:35.098053 21083 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:35.103163 21172 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:35.123407 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:35.154954 12643 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:35.113621 21084 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:35.158630 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:35.117825 21049 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:35.170511 12599 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:35.157783 12603 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:13:35.160501 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:35.173020 12597 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:35.159893 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:35.161183 12571 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:35.161921 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:35.169842 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:35.174077 12606 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:35.174890 12602 raft_consensus.cc:2393] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 24: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:35.175185 13339 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:13:35.121243 20997 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:35.177757 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:35.150169 21031 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:35.181396 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:35.181975 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:35.179360 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:35.184911 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:35.192420 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:35.188306 12525 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:13:35.114619 21081 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:35.194603 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:35.195561 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:35.208218 12606 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:13:35.212711 13335 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:35.178359 21190 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:35.177407 21068 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:35.170559 21099 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:35.216037 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54936.2us
I20260709 12:13:35.216508 21073 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:35.176342 21134 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:35.221021 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:35.177130 21084 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.187731 20997 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:35.151456 20848 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.054s
I20260709 12:13:35.178824 21100 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.222568 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:35.121953 21181 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:35.156845 20799 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.057s
I20260709 12:13:35.224887 12601 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 26. Current term is 27. Ops: [24.19-26.20]
I20260709 12:13:35.225368 12603 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:13:35.228140 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:35.086513 21180 consensus_queue.cc:237] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 18.11, Last appended by leader: 11, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.183079 21172 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:13:35.237284 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:35.247606 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:35.223320 21134 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:35.226390 21099 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.220765 21031 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.026s
I20260709 12:13:35.226225 21068 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:35.248355 21063 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:35.232873 21100 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:35.286945 21219 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
I20260709 12:13:35.235284 12423 catalog_manager.cc:5697] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 7 to 9, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 9 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:13:35.263608 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:35.291838 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 44319.6us
W20260709 12:13:35.292363 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46163us
I20260709 12:13:35.293525 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:35.294556 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:35.295173 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:35.295547 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:35.296541 12598 raft_consensus.cc:2393] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 23: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:35.304087 12677 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:35.304518 12677 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:35.247287 21103 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.308081 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:35.305238 12677 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.337237 12677 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:13:35.340431 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 47295.8us
I20260709 12:13:35.309487 12671 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:35.341296 12494 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:35.343187 21099 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:35.271929 20997 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:13:35.310560 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:35.344480 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 101757us
I20260709 12:13:35.346664 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 937) took 2005 ms. Trace:
I20260709 12:13:35.350283 21213 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.134s
I20260709 12:13:35.352335 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:35.352856 21216 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.127s
W20260709 12:13:35.354152 21211 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.134s	user 0.000s	sys 0.001s
I20260709 12:13:35.354732 21009 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:35.435300 12675 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:13:35.270519 21210 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.188s	user 0.003s	sys 0.000s
I20260709 12:13:35.358516 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:35.236886 21000 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:35.343425 21212 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:35.376720 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:35.413703 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:35.446354 12677 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 16. (term mismatch)
I20260709 12:13:35.446660 12678 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:35.447094 12677 pending_rounds.cc:85] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 15
I20260709 12:13:35.447559 12677 pending_rounds.cc:107] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 25.16
W20260709 12:13:35.428804 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 87602.8us
I20260709 12:13:35.313560 20913 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.027s
I20260709 12:13:35.430568 12674 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:13:35.386154 21194 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:35.265223 21081 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:35.439184 12671 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:13:35.307602 20985 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:13:35.302275 21134 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:35.267537 21084 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:35.249076 21190 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:35.410259 12604 rpcz_store.cc:276] 0709 12:13:33.341498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:33.341621 (+   123us) service_pool.cc:224] Handling call
0709 12:13:33.341976 (+   355us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:33.399019 (+ 57043us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:33.399050 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:33.399545 (+   495us) log.cc:844] Serialized 52 byte log entry
0709 12:13:33.875838 (+476293us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:13:33.967028 (+ 91190us) log.cc:844] Serialized 14 byte log entry
0709 12:13:34.612473 (+645445us) log.cc:844] Serialized 14 byte log entry
0709 12:13:34.638599 (+ 26126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:34.638692 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:35.268695 (+630003us) spinlock_profiling.cc:243] Waited 47.1 ms on lock 0x7b240033f7b8. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14857b99c4 00007f14857b7acb 00007f1485972dc9 00007f1485972a55 00007f1485971fb0 00007f14885e8488 00007f14885e8be8 00007f14885c0398 00007f14885bb76a 00007f14885babb6
0709 12:13:35.346286 (+ 77591us) raft_consensus.cc:1685] finished
0709 12:13:35.346345 (+    59us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:35.346575 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":167221,"prepare.queue_time_us":576165,"prepare.run_cpu_time_us":164,"prepare.run_wall_time_us":12216,"raft.queue_time_us":177225,"raft.run_cpu_time_us":615,"raft.run_wall_time_us":27791,"spinlock_wait_cycles":108401024,"thread_start_us":914202,"threads_started":3,"wal-append.queue_time_us":620025}
I20260709 12:13:35.442366 21174 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.448661 12677 raft_consensus.cc:2889] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:35.449190 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:35.451965 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:35.462602 12674 raft_consensus.cc:2393] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 23: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:35.375792 21182 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:35.431844 20950 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:35.432600 20710 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:35.435743 21073 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:35.455487 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 264469us
I20260709 12:13:35.486132 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:35.487174 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:35.488576 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:35.295759 21100 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:35.355922 21068 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.466529 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:35.230672 21083 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:35.468116 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:35.469777 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:35.377666 21103 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:35.513242 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:35.513798 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:35.491362 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:35.501614 12606 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26.
I20260709 12:13:35.495421 12643 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:35.479894 20950 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:35.521729 20950 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:35.522104 12678 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:13:35.525461 20744 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.512735 21084 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:35.527822 20744 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:35.528452 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:35.529462 12527 raft_consensus.cc:2368] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22.
I20260709 12:13:35.530602 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:35.471041 21000 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:13:35.526382 12529 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:35.537719 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:35.540808 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:35.543381 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:35.532490 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:35.547230 12646 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:35.550693 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:35.552733 12643 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:35.467891 21190 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:13:35.535288 20839 consensus_meta.cc:287] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.897s	user 0.008s	sys 0.004s
I20260709 12:13:35.470851 21174 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.554587 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:35.479583 20853 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.065s
I20260709 12:13:35.538400 21009 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:35.533924 21181 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:35.532411 21000 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:35.535539 21083 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:35.589872 12571 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:35.481451 21182 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:35.546640 20747 consensus_meta.cc:287] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.276s	user 0.009s	sys 0.005s
I20260709 12:13:35.554504 12675 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:13:35.604928 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:35.605553 12678 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:13:35.603497 21194 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:35.620354 21216 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:35.620458 21194 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:13:35.620954 21216 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.611099 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:35.610435 21058 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:13:35.622716 12676 raft_consensus.cc:2410] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 22 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:13:35.494171 20710 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.623785 21174 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:35.623615 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:35.547964 20997 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
I20260709 12:13:35.625196 20710 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:35.506631 21073 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.557752 20705 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader election won for term 19
I20260709 12:13:35.620193 21063 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:13:35.546512 21095 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.613375 21000 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:35.593380 21036 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:35.588994 20958 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.052s
I20260709 12:13:35.598410 20839 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:35.554028 20872 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:13:35.636751 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:35.691901 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48382.5us
I20260709 12:13:35.607470 21009 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:35.604208 21180 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:13:35.636026 21184 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.575s	user 0.011s	sys 0.002s
I20260709 12:13:35.657150 12598 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:13:35.601723 12646 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:35.695426 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58783.4us
I20260709 12:13:35.617290 21081 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.041s
W20260709 12:13:35.697458 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 95881.5us
I20260709 12:13:35.698763 20710 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:13:35.699172 20592 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:13:35.699857 21216 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:35.643512 21218 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:35.700629 21213 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:13:35.704437 21190 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:13:35.733163 12602 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 18. Current term is 19. Ops: []
I20260709 12:13:35.704753 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:35.704493 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:35.646656 20705 raft_consensus.cc:697] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:35.685110 20997 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:35.738643 12676 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 22: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:35.680852 21214 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:13:35.643581 21191 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.109s
I20260709 12:13:35.633598 20747 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.706389 21009 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:35.643442 21224 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.180s
I20260709 12:13:35.643748 21095 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.726492 20592 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:35.641703 21179 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.085s
W20260709 12:13:35.730686 21220 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.282s	user 0.001s	sys 0.000s
I20260709 12:13:35.729821 21184 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.769289 21184 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.773636 21184 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:35.776062 20950 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.729727 21218 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:35.778348 21218 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:35.782485 20705 consensus_queue.cc:237] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 17.10, Last appended by leader: 10, Current term: 19, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.783962 21073 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:35.791119 21182 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:35.745558 20997 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:35.795974 20997 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:35.735370 20710 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:35.797333 20710 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.800446 20747 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:35.765471 20592 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:13:35.802361 21221 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 12:13:35.802631 20839 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:35.782078 12423 catalog_manager.cc:5697] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 11 to 14, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 14 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:13:35.804287 12423 catalog_manager.cc:5697] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 6 to 9. New cstate: current_term: 9 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:13:35.727586 20872 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:35.826195 21162 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:35.827651 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3324) took 1083 ms. Trace:
I20260709 12:13:35.738032 21172 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.847141 12530 rpcz_store.cc:276] 0709 12:13:34.744560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:34.744706 (+   146us) service_pool.cc:224] Handling call
0709 12:13:34.745004 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:34.850414 (+105410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.827314 (+976900us) spinlock_profiling.cc:243] Waited 51.6 ms on lock 0x7b5400407c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:35.827570 (+   256us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118651776}
I20260709 12:13:35.839690 12676 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:35.840063 12675 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:35.842433 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:35.744499 21191 consensus_queue.cc:1243] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 20.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:13:35.849192 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:35.851388 12676 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:13:35.852509 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:35.856732 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:35.744731 21222 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.277s	user 0.000s	sys 0.002s
I20260709 12:13:35.843946 20997 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.859542 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:35.864419 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:35.866194 12606 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:35.873390 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:35.873616 12494 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:35.871035 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:35.875573 13335 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 21: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:35.877000 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:35.869632 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:35.880260 12676 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:13:35.881619 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:35.867242 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:35.900332 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:35.900941 12672 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:35.901391 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:35.902460 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:35.902992 12675 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
W20260709 12:13:35.847141 21049 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.657s	user 0.006s	sys 0.006s
I20260709 12:13:35.883980 13302 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:35.890298 12496 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:35.891027 20913 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:13:35.892390 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 359996us
I20260709 12:13:35.864245 20958 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.882412 20997 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:35.911273 20958 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.912995 20997 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:35.913326 20958 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:35.894132 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:35.914206 20950 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:35.916121 13305 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:35.916860 21098 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:35.917294 21098 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:35.917637 21098 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:35.871371 12423 catalog_manager.cc:5697] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 13 to 14, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 14 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:13:35.920325 21049 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.921604 12671 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 24: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:35.864624 21182 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:35.905615 12675 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:35.889232 21216 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:35.930332 21102 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:13:35.930470 21216 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:35.914278 21174 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:35.864508 21214 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:35.924161 12676 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
W20260709 12:13:35.938068 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47878us
W20260709 12:13:35.927987 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 44078.4us
I20260709 12:13:35.939440 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:35.940999 12496 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:35.929273 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:35.943337 12606 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:35.948532 12496 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:35.953796 20848 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:35.945084 12494 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:35.850544 21172 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.931258 21216 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:35.898962 21014 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 23.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.045s
I20260709 12:13:35.965744 13337 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:35.966794 12672 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:35.932355 21182 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:35.969058 13335 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:13:35.938014 20827 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:35.909657 20592 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.954888 21214 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:35.959082 21036 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:35.982504 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:35.982785 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:35.983260 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:35.984225 13333 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:35.984995 13336 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:35.954572 20950 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:13:35.921845 12423 catalog_manager.cc:5697] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 9 to 13. New cstate: current_term: 13 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:13:35.954381 21099 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:35.986285 21191 consensus_queue.cc:1059] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:13:35.987049 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:35.987991 21095 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:35.990599 13335 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7bd864b75c34027b05582a55b152877"
dest_uuid: "e56d487365394ac5ac5f1002926a5645"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:34880
I20260709 12:13:35.955582 20853 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:36.030416 20853 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:36.030910 20853 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:36.030925 12423 catalog_manager.cc:5697] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 10 to 12, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 12 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:36.033303 21218 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:36.034508 12674 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 26. Current term is 27. Ops: []
I20260709 12:13:36.034991 21103 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:36.036890 21115 raft_consensus.cc:3055] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:36.037326 21115 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:36.037873 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:36.040586 12676 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:13:36.042755 12675 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:13:35.950554 20872 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:36.048401 12423 catalog_manager.cc:5697] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 10 to 13, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 13 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:13:36.053361 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 945) took 2709 ms. Trace:
I20260709 12:13:36.053807 12601 rpcz_store.cc:276] 0709 12:13:33.343573 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:33.606424 (+262851us) service_pool.cc:224] Handling call
0709 12:13:33.607007 (+   583us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:35.233792 (+1626785us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:36.053065 (+819273us) spinlock_profiling.cc:243] Waited 1.62 s on lock 0x7b540007b554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:36.053297 (+   232us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3720080256}
I20260709 12:13:36.056041 21214 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:36.057317 12601 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:13:36.057610 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:36.000202 13336 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:35.977172 20913 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:36.058928 20913 raft_consensus.cc:697] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:36.058746 13336 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:36.059839 13336 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:36.060072 20913 consensus_queue.cc:237] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:35.977598 21182 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:35.942358 21174 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:36.067714 12601 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:13:36.054854 13339 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
W20260709 12:13:35.968395 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 259015us
I20260709 12:13:36.023423 13335 raft_consensus.cc:606] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 24 LEADER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:36.064749 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:36.071269 12672 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
I20260709 12:13:36.067256 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:36.079039 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:36.080199 12672 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 27. Current term is 28. Ops: [25.14-27.15]
I20260709 12:13:36.076573 12646 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:36.087643 12569 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:36.007829 21099 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:36.095237 12675 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 20 index: 11. (index mismatch)
I20260709 12:13:36.089743 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:36.090399 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:36.091274 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:13:36.078989 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:36.149756 12674 raft_consensus.cc:2410] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 25 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:13:36.222280 12601 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 21. Current term is 22. Ops: [21.17-21.17]
I20260709 12:13:36.316720 12606 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:36.090588 12646 leader_election.cc:400] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22.
I20260709 12:13:36.089993 21214 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.288s
I20260709 12:13:36.374255 21214 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:36.374814 21214 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:36.377641 21214 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:36.378264 21036 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:36.378801 12418 maintenance_manager.cc:419] P d5d8248387cf49cfbd3416fcacedae86: Scheduling FlushDeltaMemStoresOp(00000000000000000000000000000000): perf score=0.033374
I20260709 12:13:36.378921 21036 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:36.381417 21036 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:36.382997 20592 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:36.383756 21218 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:36.147037 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:36.372732 12646 leader_election.cc:403] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:36.394698 12601 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:36.075275 20848 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.321s
I20260709 12:13:36.396869 12674 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: [21.12-21.12]
I20260709 12:13:36.398597 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:36.090193 21095 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:36.401925 12674 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:13:36.403926 12674 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:36.083393 21049 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:36.370507 21099 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:36.142256 21191 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:13:36.093379 21113 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:36.414772 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:36.415655 12674 pending_rounds.cc:85] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 12
I20260709 12:13:36.416007 12674 pending_rounds.cc:107] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 12:13:36.416378 12674 raft_consensus.cc:2889] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:13:36.407910 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 320817us
I20260709 12:13:36.144459 21176 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:13:36.417315 12598 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.526s	user 0.015s	sys 0.039s
I20260709 12:13:36.074594 21009 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:36.419915 12598 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.016s	sys 0.039s
I20260709 12:13:36.388372 20710 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:36.432377 13336 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 17. (term mismatch)
W20260709 12:13:36.437369 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 61876.6us
I20260709 12:13:36.470531 13336 pending_rounds.cc:85] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 16
I20260709 12:13:36.470857 13336 pending_rounds.cc:107] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 25.17
I20260709 12:13:36.471191 13336 raft_consensus.cc:2889] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:36.474923 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:36.476725 21174 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:36.094700 21194 consensus_queue.cc:1059] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.321s
I20260709 12:13:36.085780 21000 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:36.086076 21216 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:36.408244 21084 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22.
W20260709 12:13:36.414896 21083 consensus_meta.cc:287] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.800s	user 0.007s	sys 0.004s
I20260709 12:13:36.503901 21014 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:36.504419 21014 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:36.508030 21000 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:36.486078 20710 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:36.509626 21009 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:36.415280 21113 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:36.510926 21113 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:36.511533 20710 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:36.513449 20985 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:13:36.521747 13332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 944) took 1903 ms. Trace:
I20260709 12:13:36.488860 12601 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 28.
W20260709 12:13:36.490010 21093 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 26 current term: 26 is higher.
I20260709 12:13:36.523874 13332 rpcz_store.cc:276] 0709 12:13:34.618388 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:34.618537 (+   149us) service_pool.cc:224] Handling call
0709 12:13:34.686810 (+ 68273us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:34.931947 (+245137us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:34.931996 (+    49us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:34.932057 (+    61us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:34.932108 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:36.521506 (+1589398us) spinlock_profiling.cc:243] Waited 245 ms on lock 0x7b540047b8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:36.521551 (+    45us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:36.521682 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":563291904}
I20260709 12:13:36.095057 21115 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:36.537873 21179 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.584s	user 0.028s	sys 0.046s
W20260709 12:13:36.539232 21179 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.587s	user 0.028s	sys 0.046s
I20260709 12:13:36.541090 12597 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 25. Current term is 26. Ops: [25.17-25.17]
I20260709 12:13:36.542687 20202 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:36.609386 21172 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:36.611058 12678 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: [23.14-23.14]
I20260709 12:13:36.558279 21095 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:36.508030 21216 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:36.618382 21095 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:13:36.601895 12496 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.652s	user 0.020s	sys 0.064s
W20260709 12:13:36.621127 12496 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.671s	user 0.021s	sys 0.064s
W20260709 12:13:36.621862 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 673386us
I20260709 12:13:36.622977 12496 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:36.624801 12496 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:36.605159 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3887) took 1382 ms. Trace:
W20260709 12:13:36.607128 13305 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.630s	user 0.055s	sys 0.049s
W20260709 12:13:36.626657 13305 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.055s	sys 0.049s
W20260709 12:13:36.576370 21091 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.029s	sys 0.051s
W20260709 12:13:36.627164 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 650879us
W20260709 12:13:36.627467 21091 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.637s	user 0.029s	sys 0.052s
W20260709 12:13:36.534144 21184 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.617s	user 0.013s	sys 0.034s
W20260709 12:13:36.629007 21184 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.712s	user 0.014s	sys 0.034s
I20260709 12:13:36.629154 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:36.629899 12529 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
I20260709 12:13:36.626518 12603 rpcz_store.cc:276] 0709 12:13:35.222553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.222778 (+   225us) service_pool.cc:224] Handling call
0709 12:13:35.223075 (+   297us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.234080 (+ 11005us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:36.604759 (+1370679us) spinlock_profiling.cc:243] Waited 2.02 ms on lock 0x7b5400232154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:36.605066 (+   307us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4641152}
W20260709 12:13:36.605397 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 78509.8us
I20260709 12:13:36.632164 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:36.625849 21216 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:36.505308 21014 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:36.640775 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:36.641455 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:36.635210 12601 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:13:36.644551 13305 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:36.601042 20848 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:36.522477 21176 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:36.609828 21083 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:36.639577 21216 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:36.661943 21216 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:36.667538 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 983) took 1063 ms. Trace:
I20260709 12:13:36.668025 12677 rpcz_store.cc:276] 0709 12:13:35.604603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.615863 (+ 11260us) service_pool.cc:224] Handling call
0709 12:13:35.616309 (+   446us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:35.616772 (+   463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:35.616813 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:35.633887 (+ 17074us) log.cc:844] Serialized 26 byte log entry
0709 12:13:36.077824 (+443937us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:36.077894 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:36.078000 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:36.667281 (+589281us) raft_consensus.cc:1685] finished
0709 12:13:36.667306 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:36.667466 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8739,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":87,"thread_start_us":225912,"threads_started":1,"wal-append.queue_time_us":992914}
W20260709 12:13:36.685670 21162 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.836s	user 0.009s	sys 0.002s
W20260709 12:13:36.688186 21234 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.202s	user 0.000s	sys 0.000s
I20260709 12:13:36.659566 21089 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:36.689427 21089 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:36.689924 21089 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:36.608873 20710 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.052s
I20260709 12:13:36.650314 21091 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:36.671824 20853 consensus_meta.cc:287] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.640s	user 0.010s	sys 0.005s
I20260709 12:13:36.614925 21212 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.068s
I20260709 12:13:36.690524 21115 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:36.660041 20848 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:36.771250 21162 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:13:36.824199 21141 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 2.111s	user 0.011s	sys 0.011s
I20260709 12:13:36.683182 21083 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:36.788100 21095 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:36.785006 21091 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:36.845458 21141 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 2.150s	user 0.012s	sys 0.011s
I20260709 12:13:36.850996 21194 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:13:36.852891 21194 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:13:36.854471 21194 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:13:36.854984 21176 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:36.872792 21141 consensus_queue.cc:1243] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 21.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
W20260709 12:13:36.952955 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 479268us
I20260709 12:13:36.958329 12526 raft_consensus.cc:2410] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 25 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:13:36.964844 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3888) took 1501 ms. Trace:
I20260709 12:13:36.966014 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3420) took 1263 ms. Trace:
I20260709 12:13:36.974972 12529 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:13:36.785754 20600 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:36.986613 20848 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:36.785389 20853 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.081777 21250 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:37.082998 21250 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.084540 20853 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:36.840926 21058 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Stepping down as leader of term 24
W20260709 12:13:36.988044 20872 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.903s	user 0.027s	sys 0.043s
I20260709 12:13:36.853504 21089 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:36.997241 21095 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:37.000655 21091 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:37.002771 21083 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:37.093462 12523 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3390) took 3034 ms (client timeout 2092 ms). Trace:
W20260709 12:13:36.900977 21239 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.123s	user 0.000s	sys 0.001s
I20260709 12:13:37.094005 20799 consensus_queue.cc:1059] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:37.095577 20799 raft_consensus.cc:3055] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 12:13:37.047533 12673 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.577s	user 0.000s	sys 0.012s
W20260709 12:13:36.901173 21185 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 2.330s	user 0.031s	sys 0.037s
I20260709 12:13:37.059031 21216 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:37.066548 21134 consensus_meta.cc:287] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.599s	user 0.005s	sys 0.021s
W20260709 12:13:37.067111 12495 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.005s	sys 0.009s
W20260709 12:13:37.067966 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 435049us
W20260709 12:13:37.076747 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 112181us
I20260709 12:13:37.077364 20600 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:37.085814 21212 raft_consensus.cc:3055] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:37.086625 20741 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 20.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
W20260709 12:13:37.091748 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 101901us
I20260709 12:13:37.096177 20799 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:37.096508 20872 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.014s	user 0.029s	sys 0.043s
I20260709 12:13:37.111245 21246 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.111850 20600 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:37.102144 12673 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.632s	user 0.001s	sys 0.012s
I20260709 12:13:37.102164 21212 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:13:37.094069 12523 rpcz_store.cc:269] 0709 12:13:34.058800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:34.058988 (+   188us) service_pool.cc:224] Handling call
0709 12:13:37.091736 (+3032748us) spinlock_profiling.cc:243] Waited 872 ms on lock 0x7b5400408654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:37.093143 (+  1407us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2004546304}
I20260709 12:13:37.106179 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 955) took 1807 ms. Trace:
I20260709 12:13:37.106493 21162 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.118990 21212 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.119479 21162 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.107174 20848 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:37.122650 20848 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.073305 12604 rpcz_store.cc:276] 0709 12:13:35.463121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.464122 (+  1001us) service_pool.cc:224] Handling call
0709 12:13:35.464443 (+   321us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:35.464801 (+   358us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:35.464828 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:35.505411 (+ 40583us) log.cc:844] Serialized 26 byte log entry
0709 12:13:35.936373 (+430962us) raft_consensus.cc:1645] Marking committed up to 9
0709 12:13:35.936414 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.936484 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:36.963962 (+1027478us) raft_consensus.cc:1685] finished
0709 12:13:36.964034 (+    72us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:36.964533 (+   499us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":3538,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":13231,"thread_start_us":405591,"threads_started":1,"wal-append.queue_time_us":855665}
I20260709 12:13:37.073412 12680 rpcz_store.cc:276] 0709 12:13:35.702884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.703604 (+   720us) service_pool.cc:224] Handling call
0709 12:13:35.705540 (+  1936us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:35.705788 (+   248us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:35.705812 (+    24us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:35.706291 (+   479us) log.cc:844] Serialized 26 byte log entry
0709 12:13:36.083833 (+377542us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:13:36.084655 (+   822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:36.084719 (+    64us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:36.965322 (+880603us) raft_consensus.cc:1685] finished
0709 12:13:36.965361 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:36.965712 (+   351us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":377242,"threads_started":1,"wal-append.queue_time_us":963995}
I20260709 12:13:37.106230 21089 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.118673 12602 rpcz_store.cc:276] 0709 12:13:35.298843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.309870 (+ 11027us) service_pool.cc:224] Handling call
0709 12:13:35.310153 (+   283us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:35.737749 (+427596us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:37.105545 (+1367796us) spinlock_profiling.cc:243] Waited 422 ms on lock 0x7b54002f1054. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:37.105921 (+   376us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":970966272}
I20260709 12:13:37.112946 20202 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.113651 21000 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:37.129109 20202 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:37.129281 21000 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:37.129860 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 472563us
I20260709 12:13:37.131373 12646 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:37.110428 21058 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:37.125576 20848 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:37.102491 12495 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.544s	user 0.006s	sys 0.009s
I20260709 12:13:37.127974 21216 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:37.131294 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:37.179628 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:37.132805 21212 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:37.144483 20202 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:37.174993 20600 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:37.191354 21249 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.192476 21249 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:37.199033 21058 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.200592 21058 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:13:37.204998 21099 consensus_meta.cc:287] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.788s	user 0.009s	sys 0.015s
I20260709 12:13:37.174328 20848 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:37.219031 21173 raft_consensus.cc:3055] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:37.219327 20202 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:37.219653 21173 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:37.220166 20202 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.222235 12571 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 19 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:37.224253 21174 raft_consensus.cc:3055] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:37.224802 21174 raft_consensus.cc:740] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:37.225821 21265 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader election lost for term 19. Reason: could not achieve majority
I20260709 12:13:37.230093 21249 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.235930 20744 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:37.236897 20744 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:37.234519 20202 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.189950 21246 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.180375 21250 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.218645 20848 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:37.143949 21134 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.252095 21000 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.232400 21257 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.052s
I20260709 12:13:37.221212 21173 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 21.17, Last appended by leader: 17, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:37.259922 20744 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:37.232892 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:37.267824 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:37.248307 21162 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.274938 21089 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.009s
W20260709 12:13:37.287737 21226 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.598s	user 0.025s	sys 0.022s
W20260709 12:13:37.292989 21232 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.314s	user 0.002s	sys 0.005s
I20260709 12:13:37.295742 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 980) took 2402 ms. Trace:
I20260709 12:13:37.296252 12679 rpcz_store.cc:276] 0709 12:13:34.899868 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:34.900092 (+   224us) service_pool.cc:224] Handling call
0709 12:13:34.900438 (+   346us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:34.900766 (+   328us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:34.900794 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:34.915846 (+ 15052us) log.cc:844] Serialized 26 byte log entry
0709 12:13:37.114743 (+2198897us) spinlock_profiling.cc:243] Waited 5.65 ms on lock 0x7b180037e280. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:13:37.263604 (+148861us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:13:37.266653 (+  3049us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:37.273937 (+  7284us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:37.273984 (+    47us) raft_consensus.cc:1685] finished
0709 12:13:37.273994 (+    10us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:37.295504 (+ 21510us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":463,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":11482,"spinlock_wait_cycles":13006080,"thread_start_us":55725,"threads_started":1,"wal-append.queue_time_us":2295015}
W20260709 12:13:37.297653 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 166513us
I20260709 12:13:37.298367 21068 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:37.299295 21068 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:37.302482 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:37.302690 20592 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.303424 12530 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:37.303535 20592 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.305116 21031 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:37.305826 21031 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.306392 20744 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:37.308275 20705 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:37.308722 20705 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:37.309141 20705 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:37.307147 20744 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:37.310741 21105 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 26 current term: 26 is higher.
I20260709 12:13:37.313962 21095 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:37.296267 21240 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.793s	user 0.010s	sys 0.006s
I20260709 12:13:37.314800 21095 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.316630 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3878) took 2161 ms. Trace:
I20260709 12:13:37.317750 21218 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.318586 21218 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:37.320393 20848 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:37.235076 20799 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.234158 21099 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.324751 21099 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.317071 12600 rpcz_store.cc:276] 0709 12:13:35.155529 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.156435 (+   906us) service_pool.cc:224] Handling call
0709 12:13:35.156782 (+   347us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:35.188597 (+ 31815us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:35.188629 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:35.189271 (+   642us) log.cc:844] Serialized 26 byte log entry
0709 12:13:35.470101 (+280830us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:35.470182 (+    81us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:35.470247 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:37.286606 (+1816359us) spinlock_profiling.cc:243] Waited 3.95 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:37.316061 (+ 29455us) raft_consensus.cc:1685] finished
0709 12:13:37.316084 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:37.316422 (+   338us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25364,"raft.queue_time_us":32367,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":16327,"spinlock_wait_cycles":9079040,"thread_start_us":239407,"threads_started":1,"wal-append.queue_time_us":424737}
W20260709 12:13:37.328382 12526 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 3330) took 2362 ms (client timeout 2866 ms). Trace:
W20260709 12:13:37.329005 12526 rpcz_store.cc:269] 0709 12:13:34.965517 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:34.980187 (+ 14670us) service_pool.cc:224] Handling call
0709 12:13:37.328042 (+2347855us) spinlock_profiling.cc:243] Waited 1.85 s on lock 0x7b54002107d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:37.328283 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4249943296}
I20260709 12:13:37.273499 12523 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 20.
I20260709 12:13:37.339694 12529 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
I20260709 12:13:37.341239 21089 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.341522 21246 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.342671 21246 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:37.342309 21089 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:37.343726 21246 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.347198 21089 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.351015 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:13:37.351890 21098 consensus_meta.cc:287] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.434s	user 0.017s	sys 0.011s
I20260709 12:13:37.322268 20799 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:37.356559 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4062) took 1517 ms. Trace:
I20260709 12:13:37.357175 12678 rpcz_store.cc:276] 0709 12:13:35.839206 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.840154 (+   948us) service_pool.cc:224] Handling call
0709 12:13:36.556774 (+716620us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:36.617596 (+ 60822us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:37.356070 (+738474us) spinlock_profiling.cc:243] Waited 716 ms on lock 0x7b54001414d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:37.356323 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1647042944}
I20260709 12:13:37.360225 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3894) took 1653 ms. Trace:
I20260709 12:13:37.360836 12606 rpcz_store.cc:276] 0709 12:13:35.706816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.945718 (+238902us) service_pool.cc:224] Handling call
0709 12:13:35.946802 (+  1084us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:36.317328 (+370526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:37.359602 (+1042274us) spinlock_profiling.cc:243] Waited 370 ms on lock 0x7b54002ffb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:37.359998 (+   396us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":850141568}
I20260709 12:13:37.363689 21173 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:13:37.368321 13339 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 804) took 1515 ms (client timeout 1864 ms). Trace:
W20260709 12:13:37.369015 13339 rpcz_store.cc:269] 0709 12:13:35.852558 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.852700 (+   142us) service_pool.cc:224] Handling call
0709 12:13:37.367784 (+1515084us) spinlock_profiling.cc:243] Waited 189 ms on lock 0x7b540046a5d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:37.368011 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":434465664}
I20260709 12:13:37.370282 21174 consensus_queue.cc:260] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:37.372648 21174 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:37.379416 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:37.384797 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 262668us
W20260709 12:13:37.387491 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 61418.6us
I20260709 12:13:37.384961 20592 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:37.419152 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:37.418330 13302 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:37.401688 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:37.406756 20710 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:37.407384 21194 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:37.414700 21068 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:37.416638 21089 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:13:37.352514 21098 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:37.443322 12529 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 12. (index mismatch)
I20260709 12:13:37.444505 21099 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:13:37.434531 12531 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:37.433408 21162 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:37.441447 21134 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:37.469398 21100 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.010s
I20260709 12:13:37.468097 21099 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.470808 21099 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.471856 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:37.472820 12528 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 21: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:37.474066 21255 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:13:37.474615 21255 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:37.474208 12526 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
W20260709 12:13:37.475086 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 57057.6us
I20260709 12:13:37.479559 13338 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 22: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:37.457509 21089 consensus_queue.cc:1243] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 25.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
W20260709 12:13:37.483752 21227 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.872s	user 0.005s	sys 0.014s
I20260709 12:13:37.486641 21099 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.487201 21176 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:13:37.487742 21176 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:37.488751 21176 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.491991 20741 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 12:13:37.494051 21031 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:37.494204 20710 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.496243 21218 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:37.496960 20710 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.462536 20872 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:37.497704 21095 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:37.500532 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3436) took 1087 ms. Trace:
I20260709 12:13:37.503669 12530 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:13:37.475921 21255 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.506147 21255 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:37.511181 20744 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:37.511415 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:37.454275 21194 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.514950 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:37.516469 20827 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:37.516626 21170 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.518440 21191 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.540973 21191 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.514072 12673 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22.
I20260709 12:13:37.542986 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:37.543900 21194 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.519025 21274 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:37.544756 20827 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.546072 21274 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.547873 12522 raft_consensus.cc:3055] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:37.548100 20827 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.548364 12522 raft_consensus.cc:740] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:37.514969 21115 consensus_meta.cc:287] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.573s	user 0.008s	sys 0.011s
I20260709 12:13:37.526722 21099 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.527276 20744 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.544816 21190 raft_consensus.cc:3055] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:37.515729 12496 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:37.550963 21099 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.553766 21115 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:37.554397 21115 raft_consensus.cc:697] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:37.555712 21099 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.500945 12674 rpcz_store.cc:276] 0709 12:13:36.413099 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:36.415051 (+  1952us) service_pool.cc:224] Handling call
0709 12:13:36.415423 (+   372us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:13:37.230890 (+815467us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:37.230953 (+    63us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:13:37.232525 (+  1572us) log.cc:844] Serialized 78 byte log entry
0709 12:13:37.391140 (+158615us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:37.391173 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:37.400641 (+  9468us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:37.500169 (+ 99528us) raft_consensus.cc:1685] finished
0709 12:13:37.500190 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:37.500359 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":755741,"raft.queue_time_us":763912,"raft.run_cpu_time_us":327,"raft.run_wall_time_us":329,"thread_start_us":155101,"threads_started":1,"wal-append.queue_time_us":219036}
W20260709 12:13:37.552978 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:37.517442 20705 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:37.532821 21202 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:37.560737 21202 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.555863 21115 consensus_queue.cc:237] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.563742 21202 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.533638 21095 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:37.537933 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:37.564855 21095 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.565615 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:37.566376 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:37.567044 12526 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:37.537180 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:37.540820 21170 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:37.528342 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:37.551620 20744 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.552522 21190 raft_consensus.cc:740] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:37.570967 21005 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:13:37.502624 21236 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:37.513492 21113 consensus_meta.cc:287] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.881s	user 0.005s	sys 0.022s
I20260709 12:13:37.561278 21249 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Stepping down as leader of term 26
W20260709 12:13:37.586753 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:37.533644 12646 connection.cc:256] RPC call timeout handler was delayed by -1.369426939999812s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20920000000000002s for a response
I20260709 12:13:37.570686 21190 consensus_queue.cc:260] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:37.573544 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:37.573951 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:37.517132 21162 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.578704 21191 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.578877 21005 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:37.550058 12522 consensus_queue.cc:260] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.580300 21274 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.559252 21176 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:37.580747 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:37.578613 21093 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.003s
I20260709 12:13:37.580345 21170 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:37.581843 21246 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.567277 21095 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:37.589330 12528 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:13:37.602267 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:37.571580 20744 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:37.588291 21249 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:37.608481 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:37.611037 12529 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:13:37.616959 12679 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:13:37.626626 20872 consensus_queue.cc:1059] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:13:37.595527 21113 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:37.628319 20872 raft_consensus.cc:3055] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Stepping down as leader of term 18
I20260709 12:13:37.628832 20872 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 18 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:37.631736 12522 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:37.635142 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:37.604096 21134 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:37.635656 20985 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.635947 21005 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:37.636574 21134 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:37.637029 20985 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.637693 21212 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.638425 21212 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.638948 21134 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:37.639384 20985 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.648803 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:37.649721 12531 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:13:37.647071 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 182416us
W20260709 12:13:37.649269 13338 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 808) took 1592 ms (client timeout 1616 ms). Trace:
I20260709 12:13:37.636111 21254 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:37.654407 13338 rpcz_store.cc:269] 0709 12:13:36.057187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:36.057352 (+   165us) service_pool.cc:224] Handling call
0709 12:13:37.619052 (+1561700us) spinlock_profiling.cc:243] Waited 1.39 s on lock 0x7b5400145fd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:37.649207 (+ 30155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3198079360}
I20260709 12:13:37.669046 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:37.651223 20710 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.667871 12572 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:37.675956 12529 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:13:37.680142 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:37.682886 12526 raft_consensus.cc:2410] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 28 because replica has last-logged OpId of term: 27 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
I20260709 12:13:37.686396 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:37.686980 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:37.671357 21191 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.687862 12524 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:37.688150 21249 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:37.656157 21170 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.021s
W20260709 12:13:37.691519 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 82191.6us
I20260709 12:13:37.692631 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:37.692502 21254 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.651134 20985 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:37.694204 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:37.706027 12599 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 26. Current term is 27. Ops: [24.19-26.20]
I20260709 12:13:37.646337 21236 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:37.672063 21098 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.689914 21249 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:37.714977 12604 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:13:37.629983 20872 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 18, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:37.692535 21102 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 1.637s	user 0.011s	sys 0.013s
I20260709 12:13:37.706063 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:37.729478 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:37.704991 20913 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.110s	user 0.006s	sys 0.017s
W20260709 12:13:37.734270 20913 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.236s	user 0.008s	sys 0.017s
I20260709 12:13:37.705463 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:37.738211 20710 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.740926 20710 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.738245 12597 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:37.742967 20705 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:37.746922 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:37.747730 12527 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:37.748795 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:37.749688 12527 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:37.750865 20985 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.753266 20799 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:13:37.729203 21102 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.750s	user 0.012s	sys 0.013s
I20260709 12:13:37.754402 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:37.755398 13330 raft_consensus.cc:2410] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 29 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 16.
I20260709 12:13:37.755569 12671 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:37.742267 21162 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.732515 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:37.773476 12524 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:37.755234 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:37.687933 21191 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:37.783810 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:37.784780 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:37.782549 12676 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22.
I20260709 12:13:37.790171 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3893) took 2154 ms. Trace:
I20260709 12:13:37.791225 12600 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:37.790606 12598 rpcz_store.cc:276] 0709 12:13:35.635321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.887940 (+252619us) service_pool.cc:224] Handling call
0709 12:13:35.888601 (+   661us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:35.891310 (+  2709us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:13:36.470721 (+579411us) log.cc:844] Serialized 14 byte log entry
0709 12:13:37.076930 (+606209us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:37.681319 (+604389us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b2c00000e98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:13:37.742699 (+ 61380us) log.cc:844] Serialized 26 byte log entry
0709 12:13:37.742908 (+   209us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:37.742942 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:37.743057 (+   115us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:37.776228 (+ 33171us) raft_consensus.cc:1685] finished
0709 12:13:37.784116 (+  7888us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:37.789954 (+  5838us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":112359,"prepare.queue_time_us":797938,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":44,"raft.queue_time_us":115813,"raft.run_cpu_time_us":463,"raft.run_wall_time_us":464,"spinlock_wait_cycles":25182080,"thread_start_us":976959,"threads_started":2,"wal-append.queue_time_us":548378}
W20260709 12:13:37.796514 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 146704us
I20260709 12:13:37.768229 21212 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.798574 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:37.804167 12522 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 21 index: 12. (index mismatch)
I20260709 12:13:37.806615 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:37.814525 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:37.818874 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:37.785933 20985 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.824954 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3407) took 2576 ms. Trace:
I20260709 12:13:37.822302 12597 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:37.817744 12603 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:13:37.825580 12522 rpcz_store.cc:276] 0709 12:13:35.243744 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:35.515937 (+272193us) service_pool.cc:224] Handling call
0709 12:13:35.516223 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:37.547593 (+2031370us) spinlock_profiling.cc:243] Waited 1.54 s on lock 0x7b54002ad620. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488997dc0 00007f1488a221c0 00007f1488a23231 00007f1488a1fd8f 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c
0709 12:13:37.805508 (+257915us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:37.820181 (+ 14673us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3537808640}
I20260709 12:13:37.836761 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:37.844465 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:37.845510 12526 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 26: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:37.855765 21261 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:13:37.857359 21183 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 21 current term: 21 is higher.
I20260709 12:13:37.858994 12646 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:37.859612 21009 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:13:37.860409 21031 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:37.860988 21031 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:37.861363 21031 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:37.862255 21215 consensus_queue.cc:1059] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:37.862766 21170 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:13:37.863497 21215 raft_consensus.cc:3055] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:37.863940 21215 raft_consensus.cc:740] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:37.865913 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:37.867324 21182 consensus_meta.cc:287] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.792s	user 0.017s	sys 0.009s
I20260709 12:13:37.868011 21182 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:37.870530 12525 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 26. Current term is 27. Ops: []
I20260709 12:13:37.871980 21191 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:37.872169 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:37.872917 12677 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:37.873385 21254 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:37.873842 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:37.867254 21236 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:37.866773 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:37.876194 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:37.880875 12600 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:13:37.850970 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:37.874523 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:37.884647 12531 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:13:37.887238 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:37.889281 20958 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:13:37.890645 21174 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.517s	user 0.010s	sys 0.003s
I20260709 12:13:37.851612 20872 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 18 FOLLOWER]: Advancing to term 19
W20260709 12:13:37.894878 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:37.896899 21162 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:37.897336 21162 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:37.899576 12677 raft_consensus.cc:2393] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 21: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:37.904366 12680 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 28: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:37.906705 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:37.907241 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:37.908665 21182 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:37.909004 21162 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.911171 21162 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:37.874910 21173 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.038s
I20260709 12:13:37.913079 12528 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:13:37.864961 21215 consensus_queue.cc:260] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:37.915309 21215 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:37.915472 12528 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:37.916534 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:37.917759 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:37.918413 12673 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:37.922559 12522 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 21: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:37.924010 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:37.925022 12522 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:37.926631 13339 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26. Ops: [25.18-25.18]
I20260709 12:13:37.926950 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:37.927695 12603 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
W20260709 12:13:37.928566 12416 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 70b8c9b89e3d43fbb70422110d76702c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica a2fdbe7eef6748a9b4611f6c9bc9b201 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 20 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:13:37.929838 12672 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
W20260709 12:13:37.931144 21284 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:13:37.933478 21183 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:37.934011 21183 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:37.918900 21285 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.129s	user 0.000s	sys 0.003s
I20260709 12:13:37.919896 21014 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:37.952358 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:37.921501 12675 raft_consensus.cc:2410] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 23 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:13:37.904479 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:37.938673 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:37.955983 21214 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:13:37.951012 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 43887us
I20260709 12:13:37.957445 13302 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:37.959705 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:37.960032 21173 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:37.960438 21173 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:37.960494 13339 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 26: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:37.960664 12673 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:13:37.961640 13305 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:37.960866 21173 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:37.943816 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:37.965430 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:37.963371 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:37.965555 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:37.990011 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:37.990505 13336 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 21: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:37.991369 21214 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:37.992110 21214 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:37.988073 12522 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:37.992908 21214 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:37.962100 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:37.994020 12673 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:37.995489 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:37.970937 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:37.999336 12522 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:38.000707 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:38.001830 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:38.006742 12531 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:37.952224 12646 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:13:38.009131 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 70535us
I20260709 12:13:38.010883 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:38.009164 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:37.996640 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:37.966588 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:37.966871 21098 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:38.019711 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:38.021425 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:38.024297 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:37.861908 21009 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.090s
I20260709 12:13:38.026649 12526 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 19 index: 11. (term mismatch)
I20260709 12:13:38.022239 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:38.028822 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 78047.7us
I20260709 12:13:38.029165 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:38.034370 12527 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:38.027233 12526 pending_rounds.cc:85] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 10
I20260709 12:13:38.044589 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:38.048805 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:38.040858 20744 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:38.039806 12526 pending_rounds.cc:107] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 18.11
I20260709 12:13:38.052731 12674 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:38.061110 12526 raft_consensus.cc:2889] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:38.019781 21098 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 25 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:38.074612 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:38.076542 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:38.078146 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:38.080397 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:38.083016 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:38.084615 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:38.094074 12646 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:38.095491 12605 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:38.079475 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:38.085988 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:38.103777 12527 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22.
I20260709 12:13:38.111001 21031 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:38.111922 21218 consensus_queue.cc:1059] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:13:38.112419 21173 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:38.114439 12531 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:13:38.038506 21255 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.067s
W20260709 12:13:38.119089 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 362631us
I20260709 12:13:38.119689 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:38.060664 20744 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:38.061993 20705 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:38.060832 20958 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:38.123588 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 133714us
I20260709 12:13:38.124564 21098 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:38.132627 12571 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:38.137979 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:38.140331 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:38.137256 12646 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:38.152648 12677 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 22: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:13:38.154572 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:38.138252 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:38.156378 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:38.157138 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 359778us
I20260709 12:13:38.159022 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:38.160840 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:38.130400 20799 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:38.137060 21216 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:38.186470 13336 raft_consensus.cc:2368] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 23. Current term is 24.
W20260709 12:13:38.188768 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 65963.2us
I20260709 12:13:38.190132 12680 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 27: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:38.132414 21009 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:13:38.141439 20744 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:38.190284 13304 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:38.119251 21278 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.081s
I20260709 12:13:38.161093 21218 consensus_queue.cc:1243] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 23.13 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 12
I20260709 12:13:38.180296 21093 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:38.186184 20827 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election won for term 21
W20260709 12:13:38.205591 12643 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.511s (SENT) [suppressed 4 similar messages]
I20260709 12:13:38.207787 12646 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:38.220073 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:38.216658 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:38.212539 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:38.185119 21261 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:38.222980 12674 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:38.224146 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:38.207126 21068 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:13:38.142702 21115 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.070s
I20260709 12:13:38.202338 21134 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:38.222193 20827 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:38.199226 20705 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:38.239252 21105 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:38.196239 20958 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:38.269198 20958 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:38.251597 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:38.260545 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:38.270535 12603 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:13:38.261300 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:38.272006 12605 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 23: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:38.273108 12601 raft_consensus.cc:2410] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 27 because replica has last-logged OpId of term: 24 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 17.
I20260709 12:13:38.263382 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:38.211858 21216 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:38.285108 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:13:38.288830 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:38.239100 21093 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:38.260200 21068 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:38.153563 12417 maintenance_manager.cc:643] P d5d8248387cf49cfbd3416fcacedae86: FlushDeltaMemStoresOp(00000000000000000000000000000000) complete. Timing: real 1.773s	user 0.034s	sys 0.026s Metrics: {"bytes_written":3855,"delete_count":0,"lbm_write_time_us":5046,"lbm_writes_1-10_ms":2,"lbm_writes_lt_1ms":3,"raft.queue_time_us":268731,"raft.run_cpu_time_us":28679,"raft.run_wall_time_us":361693,"reinsert_count":0,"thread_start_us":223432,"threads_started":2,"update_count":139,"wal-append.queue_time_us":160838}
I20260709 12:13:38.294692 21102 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:38.296553 21236 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:13:38.308216 21290 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.173s	user 0.000s	sys 0.001s
I20260709 12:13:38.308585 21292 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:38.325876 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:38.326303 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:38.326725 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1060) took 1684 ms. Trace:
I20260709 12:13:38.326974 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:38.328573 13305 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:38.348942 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:38.329588 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:38.266431 21105 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:38.331373 21218 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:13:38.261600 20827 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:38.347059 21236 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:38.362586 21236 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:38.329581 21113 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:38.334971 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:38.276028 21182 raft_consensus.cc:3055] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:38.245573 20799 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:38.347740 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:38.343690 12494 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:38.353320 13332 raft_consensus.cc:2410] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 23 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
I20260709 12:13:38.349663 12601 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:38.348789 12525 rpcz_store.cc:276] 0709 12:13:36.642799 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:36.642969 (+   170us) service_pool.cc:224] Handling call
0709 12:13:36.643375 (+   406us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:37.871062 (+1227687us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:38.326368 (+455306us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540027f6d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:38.326642 (+   274us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2821357952}
I20260709 12:13:38.252703 21261 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:38.354434 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:38.360181 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:38.354516 21105 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:38.361423 12418 maintenance_manager.cc:419] P d5d8248387cf49cfbd3416fcacedae86: Scheduling MajorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.226224
I20260709 12:13:38.332909 21191 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:13:38.414578 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 43818.9us
I20260709 12:13:38.378741 12673 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 28: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:38.379520 12522 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 22: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:13:38.384140 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 254454us
I20260709 12:13:38.286998 21134 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:38.393445 21031 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:38.397414 12528 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 28. Current term is 29. Ops: []
I20260709 12:13:38.347759 21292 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:38.400313 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 103929us
I20260709 12:13:38.398689 21113 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:38.430795 21068 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:38.431712 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:38.458082 21113 raft_consensus.cc:697] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:38.379350 21182 raft_consensus.cc:740] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:38.425937 12496 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:38.456362 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:38.467536 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:38.468655 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49985.6us
I20260709 12:13:38.417601 21246 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:38.433678 21183 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:38.474754 20913 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:38.475838 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:38.458626 21102 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 12:13:38.477380 21102 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:13:38.477869 21102 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:38.479017 21102 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:38.454694 21292 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:38.480507 12600 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
W20260709 12:13:38.481186 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:38.486793 12673 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:38.481348 20913 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:38.534200 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 81381.8us
I20260709 12:13:38.512804 21183 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:13:38.559819 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:38.524629 20913 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:38.536708 21102 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:38.571110 21172 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:38.538508 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 416886us
I20260709 12:13:38.512563 21246 raft_consensus.cc:697] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:38.614269 12680 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28. Ops: []
I20260709 12:13:38.577764 21183 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:38.645006 21095 raft_consensus.cc:3055] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:38.645515 21095 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:38.650715 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:38.651438 12678 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:38.542862 21113 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:38.661171 21246 consensus_queue.cc:237] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:38.702847 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 54866us
W20260709 12:13:38.716316 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 526094us
I20260709 12:13:38.716941 12680 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: []
W20260709 12:13:38.717921 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 354272us
I20260709 12:13:38.719815 12673 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: []
W20260709 12:13:38.716368 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 456139us
I20260709 12:13:38.598063 21182 consensus_queue.cc:260] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:38.747833 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:38.750168 12496 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:38.757314 12569 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:38.730314 21190 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:38.759339 21191 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:38.744431 20872 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:38.754244 21009 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.517s	user 0.003s	sys 0.038s
W20260709 12:13:38.773754 12679 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.011s	sys 0.036s
I20260709 12:13:38.632385 21172 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:38.780392 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:38.782431 12679 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.547s	user 0.012s	sys 0.036s
I20260709 12:13:38.766068 21115 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:38.765553 20799 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:38.781929 21009 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.551s	user 0.003s	sys 0.038s
I20260709 12:13:38.770231 21216 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:38.802048 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 55153.1us
I20260709 12:13:38.803591 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:38.810307 12673 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:13:38.817039 12679 raft_consensus.cc:2393] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:38.791821 21173 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:38.851939 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 120357us
I20260709 12:13:38.807046 21182 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:13:38.855046 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 46327.7us
I20260709 12:13:38.781826 21255 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.038s
I20260709 12:13:38.785636 21295 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 22.11, Next index: 12, Last known committed idx: 11, Time since last communication: 0.036s
I20260709 12:13:38.863152 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:38.786552 20872 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:38.865901 20872 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:38.807322 20705 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:38.808432 21009 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:38.868391 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:38.885572 12597 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:38.853967 21134 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:38.859356 21261 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:38.905673 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:38.894835 20799 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:38.941231 20799 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:38.902037 21093 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:38.906438 12601 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
W20260709 12:13:38.903733 21215 consensus_meta.cc:287] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.951s	user 0.021s	sys 0.000s
I20260709 12:13:38.882544 21058 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.064s
I20260709 12:13:38.947083 12672 raft_consensus.cc:2410] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:13:38.948772 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 4078) took 1017 ms. Trace:
W20260709 12:13:38.944872 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:38.989418 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 1069) took 1236 ms. Trace:
I20260709 12:13:38.989889 12531 rpcz_store.cc:276] 0709 12:13:37.748700 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:37.938329 (+189629us) service_pool.cc:224] Handling call
0709 12:13:38.984906 (+1046577us) spinlock_profiling.cc:243] Waited 51.4 ms on lock 0x7b540038e5d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:38.985057 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":118210304}
W20260709 12:13:38.911834 21162 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.000s	user 0.014s	sys 0.007s
W20260709 12:13:38.940357 21014 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.930s	user 0.016s	sys 0.015s
I20260709 12:13:38.847319 21183 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:38.968330 21214 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.975s	user 0.010s	sys 0.010s
W20260709 12:13:39.005424 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 142329us
I20260709 12:13:39.012842 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:39.016489 12643 leader_election.cc:400] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 23. Current term is 24.
I20260709 12:13:39.016927 12643 leader_election.cc:403] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:38.949143 12672 rpcz_store.cc:276] 0709 12:13:37.931165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:37.951410 (+ 20245us) service_pool.cc:224] Handling call
0709 12:13:38.948508 (+997098us) spinlock_profiling.cc:243] Waited 922 ms on lock 0x7b54000ac8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:38.948705 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2119570688}
I20260709 12:13:38.912354 21134 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:13:39.101478 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 245229us
W20260709 12:13:39.028201 12643 connection.cc:256] RPC call timeout handler was delayed by -1.1173454979998496s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2565s for a response
I20260709 12:13:39.031008 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:39.102614 12643 connection.cc:256] RPC call timeout handler was delayed by -1.0544267249999848s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2127s for a response
I20260709 12:13:39.076793 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:39.033471 12529 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:13:39.022082 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:39.110196 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:39.115211 12494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.537s	user 0.005s	sys 0.005s
W20260709 12:13:39.115862 12494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.538s	user 0.005s	sys 0.005s
W20260709 12:13:39.118456 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 542717us
I20260709 12:13:39.122071 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:39.111023 12598 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
W20260709 12:13:39.124365 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 102361us
I20260709 12:13:39.010684 21295 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:13:39.030874 21162 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.064079 20799 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:39.032888 21278 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:13:39.243019 21246 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.243723 21246 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:39.248250 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:39.248572 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:39.251848 21349 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.021320 21214 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:39.128572 21330 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:39.165729 21312 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
I20260709 12:13:39.171172 21306 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:39.172748 21326 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.159s	user 0.001s	sys 0.000s
W20260709 12:13:39.174952 21321 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
W20260709 12:13:39.176126 21318 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 12:13:39.179406 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:39.057257 21215 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:13:39.013723 20958 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:13:39.229130 12678 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.568s	user 0.010s	sys 0.008s
W20260709 12:13:39.230273 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 117255us
I20260709 12:13:39.231477 21162 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:39.154400 21255 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:39.283162 12605 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 12. (term mismatch)
I20260709 12:13:39.154654 21093 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:39.283653 12605 pending_rounds.cc:85] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 11
I20260709 12:13:39.283958 12605 pending_rounds.cc:107] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 21.12
I20260709 12:13:39.284044 21093 raft_consensus.cc:697] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 25 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:39.285233 21246 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:39.249994 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:39.023468 21216 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:39.287129 12606 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 27: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:39.286967 21216 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:39.289503 21216 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:39.231040 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 442060us
W20260709 12:13:39.180992 20744 consensus_meta.cc:287] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.959s	user 0.003s	sys 0.015s
I20260709 12:13:39.290349 20744 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:39.293129 20744 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:39.256035 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:39.139158 21295 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:39.295696 21162 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:39.296380 21295 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:39.298105 21295 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:39.274255 21330 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:39.300490 21330 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:39.301141 21344 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:13:39.301894 21115 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.502s	user 0.004s	sys 0.004s
I20260709 12:13:39.262399 12526 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:39.265710 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:39.269248 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:39.065062 21009 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:39.278439 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:39.278589 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:39.281397 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 285189us
I20260709 12:13:39.259677 21172 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:39.301712 21344 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:39.273048 21105 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.873s	user 0.017s	sys 0.003s
I20260709 12:13:39.275936 20958 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:39.284236 12605 raft_consensus.cc:2889] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:39.286022 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4094) took 1150 ms. Trace:
I20260709 12:13:39.255012 12494 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:39.249830 12599 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:13:39.272843 21306 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:39.302589 21214 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:39.296427 12601 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:39.342332 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:39.303105 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:39.321810 12528 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:39.321862 12531 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:13:39.322925 21005 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:39.324505 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 51796.4us
W20260709 12:13:39.347064 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 92142.8us
I20260709 12:13:39.348695 12494 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:39.316792 12678 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.656s	user 0.010s	sys 0.008s
I20260709 12:13:39.329716 21344 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:39.339075 12671 rpcz_store.cc:276] 0709 12:13:38.135743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:38.614813 (+479070us) service_pool.cc:224] Handling call
0709 12:13:38.615175 (+   362us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:38.615674 (+   499us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:38.615705 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:38.616250 (+   545us) log.cc:844] Serialized 26 byte log entry
0709 12:13:38.791410 (+175160us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:38.791452 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:38.791521 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:39.285804 (+494283us) raft_consensus.cc:1685] finished
0709 12:13:39.285826 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:39.285960 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1367,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":20116,"thread_start_us":174720,"threads_started":1,"wal-append.queue_time_us":257412}
I20260709 12:13:39.331003 21115 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:39.285527 21093 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:39.258513 21349 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:39.349757 20913 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.777s	user 0.014s	sys 0.005s
I20260709 12:13:39.352886 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3464) took 1062 ms. Trace:
W20260709 12:13:39.359416 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 56378.4us
I20260709 12:13:39.375440 21344 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:13:39.353049 21341 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:13:39.357218 20958 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:39.380951 12597 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:39.382239 21212 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:13:39.383215 12494 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:13:39.384951 21292 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.870s	user 0.017s	sys 0.003s
I20260709 12:13:39.398458 21349 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:39.411309 12494 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:39.401729 12678 rpcz_store.cc:276] 0709 12:13:38.290825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:38.659589 (+368764us) service_pool.cc:224] Handling call
0709 12:13:38.659969 (+   380us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:38.660375 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:38.660404 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:38.660994 (+   590us) log.cc:844] Serialized 26 byte log entry
0709 12:13:39.350470 (+689476us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:39.350524 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:39.350599 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:39.350642 (+    43us) raft_consensus.cc:1685] finished
0709 12:13:39.350653 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:39.352810 (+  2157us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":121,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"thread_start_us":689053,"threads_started":1,"wal-append.queue_time_us":642046}
I20260709 12:13:39.357318 21330 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:39.429077 12494 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:13:39.407560 20827 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.045s	user 0.012s	sys 0.027s
I20260709 12:13:39.434154 12494 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:39.432607 12496 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:39.342042 21105 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:39.352857 21005 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:39.446249 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 124065us
I20260709 12:13:39.435150 21246 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:39.456552 21246 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:39.457087 21246 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:39.438448 21349 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:39.462533 21349 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:39.462990 21349 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:39.464247 21105 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:39.440642 21102 consensus_meta.cc:287] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.870s	user 0.006s	sys 0.010s
I20260709 12:13:39.406531 20913 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:39.440685 21115 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:39.438063 21341 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.016s
I20260709 12:13:39.437160 21336 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.431504 21014 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:13:39.445814 21099 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:39.483383 12572 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:39.415235 21292 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:39.439505 20827 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:39.489957 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50844.6us
I20260709 12:13:39.490954 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3462) took 1542 ms. Trace:
I20260709 12:13:39.444988 21344 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.045s
I20260709 12:13:39.482187 21115 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:39.493813 12674 rpcz_store.cc:276] 0709 12:13:37.948556 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:38.547183 (+598627us) service_pool.cc:224] Handling call
0709 12:13:38.547547 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:38.547787 (+   240us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:13:38.548140 (+   353us) log.cc:844] Serialized 14 byte log entry
0709 12:13:39.414890 (+866750us) spinlock_profiling.cc:243] Waited 3.01 ms on lock 0x7b18003db7c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf
0709 12:13:39.415054 (+   164us) log.cc:844] Serialized 14 byte log entry
0709 12:13:39.470404 (+ 55350us) log.cc:844] Serialized 14 byte log entry
0709 12:13:39.478061 (+  7657us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:39.478181 (+   120us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:13:39.478245 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:39.478337 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:39.490894 (+ 12557us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10744,"prepare.queue_time_us":46961,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":134,"raft.queue_time_us":139509,"raft.run_cpu_time_us":431,"raft.run_wall_time_us":36596,"spinlock_wait_cycles":6934272,"thread_start_us":352388,"threads_started":2,"wal-append.queue_time_us":559074}
W20260709 12:13:39.492807 21190 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.716s	user 0.011s	sys 0.002s
I20260709 12:13:39.490444 21295 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:39.500445 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:39.510491 12675 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:39.482084 21212 consensus_queue.cc:1059] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 18.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.015s
W20260709 12:13:39.509236 21315 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.571s	user 0.001s	sys 0.003s
I20260709 12:13:39.509663 21344 consensus_queue.cc:1243] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:13:39.515403 21292 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:39.498658 21115 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:13:39.516321 21236 consensus_meta.cc:287] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.153s	user 0.009s	sys 0.012s
I20260709 12:13:39.497061 21215 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 23. Current term is 24.
I20260709 12:13:39.524919 13330 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:39.530414 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:39.531589 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 97539.5us
I20260709 12:13:39.509179 21191 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.487043 21336 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:39.533274 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:39.533975 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:39.537599 13332 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:39.526732 21095 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:39.534540 21014 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:13:39.527626 21182 consensus_meta.cc:287] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.655s	user 0.009s	sys 0.002s
I20260709 12:13:39.525436 21102 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:39.541262 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:39.543764 21336 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:39.544322 21341 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:13:39.544739 21342 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.292s	user 0.000s	sys 0.002s
I20260709 12:13:39.525102 21093 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:39.532320 21236 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:39.548920 21212 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:39.528087 21337 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:39.561666 21337 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:39.562142 21337 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:39.564289 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3444) took 1246 ms. Trace:
I20260709 12:13:39.534560 21098 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.022s
I20260709 12:13:39.546094 21014 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:39.566599 21307 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:13:39.554881 21102 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:39.567109 21307 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:39.568308 21014 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:39.568058 21307 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:39.569005 21102 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:39.538976 21344 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.031s
I20260709 12:13:39.561054 21341 raft_consensus.cc:2764] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election decision vote started in defunct term 26: won
I20260709 12:13:39.560642 21212 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:39.560739 21093 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:39.571087 12603 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 24: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:39.576109 21236 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:39.577751 21093 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:39.577761 21005 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:39.584574 12529 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3433) took 1840 ms (client timeout 1901 ms). Trace:
I20260709 12:13:39.564836 12523 rpcz_store.cc:276] 0709 12:13:38.318018 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:38.318193 (+   175us) service_pool.cc:224] Handling call
0709 12:13:38.318681 (+   488us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:38.319150 (+   469us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:38.319190 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:38.319829 (+   639us) log.cc:844] Serialized 26 byte log entry
0709 12:13:38.741871 (+422042us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:38.741913 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:38.741993 (+    80us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:39.505547 (+763554us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:39.563920 (+ 58373us) raft_consensus.cc:1685] finished
0709 12:13:39.563953 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:39.564210 (+   257us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":8349,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":146,"spinlock_wait_cycles":32158720,"thread_start_us":421612,"threads_started":1,"wal-append.queue_time_us":502938}
I20260709 12:13:39.536572 21191 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:39.585148 12529 rpcz_store.cc:269] 0709 12:13:37.744371 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:37.769670 (+ 25299us) service_pool.cc:224] Handling call
0709 12:13:39.584305 (+1814635us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400053554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:39.584480 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2866046848}
I20260709 12:13:39.578822 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:39.593228 13336 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 24: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:13:39.594547 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 103532us
I20260709 12:13:39.596405 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:39.601441 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:39.602155 12677 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:39.600399 12602 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
I20260709 12:13:39.604519 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:39.606488 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:39.607978 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 17 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:39.592231 21095 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:39.610160 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:39.612493 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:39.613862 12671 raft_consensus.cc:2410] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 26 because replica has last-logged OpId of term: 24 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 11.
I20260709 12:13:39.616804 12679 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 13. (term mismatch)
I20260709 12:13:39.609710 21344 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:39.618991 21344 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:39.621451 21246 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:39.626847 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:39.617365 12679 pending_rounds.cc:85] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 12
I20260709 12:13:39.638661 12679 pending_rounds.cc:107] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 20.13
I20260709 12:13:39.638907 12680 raft_consensus.cc:2410] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 27 because replica has last-logged OpId of term: 25 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 12.
I20260709 12:13:39.638921 12679 raft_consensus.cc:2889] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:39.637964 20985 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:13:39.641215 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:39.642045 21215 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.642598 21215 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:39.643483 20985 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:13:39.614426 20913 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:39.650494 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:39.637985 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 96812.7us
I20260709 12:13:39.655099 12494 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:39.645737 21261 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:39.659052 12494 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:39.660454 20913 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:39.662295 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:39.665391 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:39.658459 21274 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:39.670399 21274 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:39.670781 21274 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:39.673979 21005 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.674582 21005 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:39.674888 20985 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 19.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:13:39.675954 20985 raft_consensus.cc:3055] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:39.676283 20985 raft_consensus.cc:740] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:39.677183 20985 consensus_queue.cc:260] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:39.677989 21005 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:39.649962 21337 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:39.680917 21337 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:39.661746 20913 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:39.686041 12676 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 23 index: 13. (index mismatch)
I20260709 12:13:39.686264 20913 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:39.621910 21344 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:39.632524 20827 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:39.631989 21246 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:39.693560 12572 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:39.660668 21261 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:39.701625 20799 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:39.702294 20799 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:39.699616 12494 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:39.707690 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:39.702711 20799 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:39.709436 21337 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:39.697431 21362 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election lost for term 20. Reason: could not achieve majority
W20260709 12:13:39.713868 21320 log.cc:927] Time spent T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.460s	user 0.002s	sys 0.002s
I20260709 12:13:39.706390 21099 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:39.710183 20985 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:39.750092 12494 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:39.752538 12603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 972) took 1390 ms (client timeout 1719 ms). Trace:
I20260709 12:13:39.709924 21360 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:39.725041 21246 raft_consensus.cc:2764] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election decision vote started in defunct term 27: won
W20260709 12:13:39.753014 12603 rpcz_store.cc:269] 0709 12:13:38.353909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:38.354070 (+   161us) service_pool.cc:224] Handling call
0709 12:13:39.743767 (+1389697us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54001167d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:39.743934 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2721618176}
I20260709 12:13:39.707015 21349 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:39.767513 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 74058.6us
I20260709 12:13:39.775355 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:39.754302 12494 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:39.778242 12494 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:39.758889 21216 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:39.759383 21360 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:39.780053 20913 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:39.780260 21014 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:13:39.793910 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:39.798873 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3450) took 1445 ms. Trace:
I20260709 12:13:39.799041 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:39.799261 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:39.800087 12603 raft_consensus.cc:2410] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 26 because replica has last-logged OpId of term: 24 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 11.
I20260709 12:13:39.801142 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:39.801602 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:39.801755 12603 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:39.802791 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:39.802958 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:39.795965 21337 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:39.799870 12680 raft_consensus.cc:2393] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 22: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:39.806818 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:39.787155 21014 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:39.807858 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:39.808120 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:39.808610 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:39.808914 12600 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 24: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:39.807878 21014 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:39.810601 12604 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:39.813488 12603 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:39.814365 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:39.809194 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:39.815117 12597 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:39.815248 12598 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:39.816200 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:39.816915 12599 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:39.819371 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:39.805987 21337 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:39.820678 21337 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:39.820861 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:39.821728 13330 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 24: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:39.827065 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:39.827713 21313 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.775s	user 0.003s	sys 0.000s
I20260709 12:13:39.827940 12680 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:39.809790 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:39.832824 12600 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:39.784448 21009 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:39.783548 20913 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:39.785385 21360 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:39.799338 12525 rpcz_store.cc:276] 0709 12:13:38.353467 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:38.399009 (+ 45542us) service_pool.cc:224] Handling call
0709 12:13:38.428553 (+ 29544us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:38.453216 (+ 24663us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:38.453266 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:38.458108 (+  4842us) log.cc:844] Serialized 26 byte log entry
0709 12:13:38.787372 (+329264us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:38.787421 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:38.787493 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:39.713230 (+925737us) spinlock_profiling.cc:243] Waited 3.36 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:39.785968 (+ 72738us) raft_consensus.cc:1685] finished
0709 12:13:39.785992 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:39.798797 (+ 12805us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":107,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":3276,"spinlock_wait_cycles":7739264,"thread_start_us":328857,"threads_started":1,"wal-append.queue_time_us":607754}
I20260709 12:13:39.825505 21014 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:39.759810 21261 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:39.782047 21099 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.053s
I20260709 12:13:39.803658 20827 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:39.857067 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:39.842432 20913 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:39.869382 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:39.839494 21176 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:39.867928 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3461) took 1919 ms. Trace:
W20260709 12:13:39.869071 21363 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.096s	user 0.000s	sys 0.000s
I20260709 12:13:39.870045 12679 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:39.912315 12603 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:39.882175 12675 rpcz_store.cc:276] 0709 12:13:37.948283 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:38.273511 (+325228us) service_pool.cc:224] Handling call
0709 12:13:38.277449 (+  3938us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:39.523315 (+1245866us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:39.867675 (+344360us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54003bc754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:39.867860 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2835245312}
W20260709 12:13:39.889029 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 82293.6us
I20260709 12:13:39.939869 12572 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:39.947130 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:39.947355 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:39.948856 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:39.949592 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:39.954186 12600 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:39.956723 12677 raft_consensus.cc:2368] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22.
I20260709 12:13:39.956969 12494 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:39.963925 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:39.969379 12602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 965) took 2324 ms (client timeout 2117 ms). Trace:
I20260709 12:13:39.969702 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:13:39.969986 12602 rpcz_store.cc:269] 0709 12:13:37.633372 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:37.634723 (+  1351us) service_pool.cc:224] Handling call
0709 12:13:39.957549 (+2322826us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b540023f854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:39.957660 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4517632768}
I20260709 12:13:39.843442 21009 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:39.973711 12599 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 21 index: 13. (index mismatch)
I20260709 12:13:39.974964 12572 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:39.977442 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:39.979332 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:39.960249 21328 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:39.982088 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:39.969916 21261 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:39.968298 21194 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:39.898459 20827 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:39.990581 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:39.991415 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:39.989908 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.000837 12597 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
I20260709 12:13:40.000315 12677 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
I20260709 12:13:40.000695 12602 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22.
I20260709 12:13:40.005169 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:40.006079 12679 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26.
I20260709 12:13:40.007282 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:39.995673 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:39.971994 21324 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:39.972271 21341 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:13:40.019996 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 45113.8us
I20260709 12:13:40.037556 12671 raft_consensus.cc:2368] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
I20260709 12:13:39.986445 21182 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:40.043393 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:40.036159 21361 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.282s	user 0.000s	sys 0.001s
W20260709 12:13:40.005221 21364 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:13:40.000274 21295 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:39.966450 21115 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:40.052649 12599 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
I20260709 12:13:40.073287 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.042068 21324 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:39.993808 21328 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:39.982511 20799 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:40.073824 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:40.080178 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 110545us
I20260709 12:13:40.084288 12598 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:40.052330 21182 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:40.054559 21349 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:40.066350 21194 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:40.114526 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.070849 21255 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:40.130716 13302 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:40.131353 12571 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:40.094841 21324 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:40.142307 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.143464 12602 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 27.
I20260709 12:13:40.144593 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:40.145960 21332 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.893s	user 0.000s	sys 0.003s
I20260709 12:13:40.147428 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3451) took 1784 ms. Trace:
I20260709 12:13:40.148327 21362 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:13:40.152144 12680 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:40.156536 21212 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:40.157347 21255 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:40.158764 21014 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:40.131752 21361 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.059s	user 0.003s	sys 0.005s
I20260709 12:13:40.160513 13302 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:40.161111 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3924) took 1145 ms. Trace:
I20260709 12:13:40.163596 21098 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:40.164125 21098 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:40.164542 21330 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:13:40.164973 21330 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:40.165155 21115 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:40.161573 12601 rpcz_store.cc:276] 0709 12:13:39.016080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:39.099658 (+ 83578us) service_pool.cc:224] Handling call
0709 12:13:40.160930 (+1061272us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b540023fd54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:40.161054 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":398797696}
I20260709 12:13:40.165913 21330 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:40.167265 21337 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:40.168612 21330 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:40.171036 21190 raft_consensus.cc:3055] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:40.171465 21190 raft_consensus.cc:740] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:40.171993 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.172668 12601 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:40.147917 12524 rpcz_store.cc:276] 0709 12:13:38.362745 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:38.479555 (+116810us) service_pool.cc:224] Handling call
0709 12:13:38.479944 (+   389us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:38.491128 (+ 11184us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:38.491177 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:38.742164 (+250987us) log.cc:844] Serialized 26 byte log entry
0709 12:13:38.892048 (+149884us) raft_consensus.cc:1645] Marking committed up to 10
0709 12:13:38.892274 (+   226us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:38.892504 (+   230us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:40.145340 (+1252836us) spinlock_profiling.cc:243] Waited 6.9 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:40.147007 (+  1667us) raft_consensus.cc:1685] finished
0709 12:13:40.147038 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:40.147349 (+   311us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10747,"raft.queue_time_us":17653,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":3258,"spinlock_wait_cycles":15873920,"thread_start_us":67825,"threads_started":1,"wal-append.queue_time_us":488121}
I20260709 12:13:40.161582 21328 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:40.176429 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:40.177359 12672 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 22: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:40.177449 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.176267 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:40.178334 21322 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.925s	user 0.001s	sys 0.006s
I20260709 12:13:40.178344 12605 raft_consensus.cc:2410] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 27 because replica has last-logged OpId of term: 25 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 12.
I20260709 12:13:40.115634 21182 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:40.180482 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:40.186218 13302 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:40.132148 21341 consensus_queue.cc:1059] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:40.164546 21098 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:40.133863 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:40.208417 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:40.142612 21369 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.075s	user 0.000s	sys 0.000s
I20260709 12:13:40.209785 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:40.234426 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 100640us
I20260709 12:13:40.214473 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:40.222390 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:40.226233 21249 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:40.228303 21068 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:40.228526 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:40.229741 21014 raft_consensus.cc:3055] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:40.231063 21358 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:40.231896 12496 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:40.240478 12674 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:40.233281 12646 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:40.233572 12598 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:13:40.207175 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:40.195518 21102 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:40.144639 21009 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:40.240571 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.235416 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:40.245381 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.251991 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.261178 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:40.262929 12600 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:40.263717 12602 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 22: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:40.251098 21358 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:40.269276 21358 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:40.240571 21014 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:40.238444 21249 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:40.282778 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:40.228843 21360 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:40.286033 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:40.195716 21274 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:40.270781 12646 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:40.301460 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:40.304500 12522 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 22: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:40.305951 12522 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:13:40.307890 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:40.308718 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:40.251389 20958 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:40.309867 12527 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:13:40.310478 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:40.249455 21102 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:40.315784 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 45138.8us
I20260709 12:13:40.316721 12646 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:40.318895 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:40.319208 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 87401.9us
I20260709 12:13:40.319561 12528 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:40.320304 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:40.320351 12496 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:40.321455 12530 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 27.
I20260709 12:13:40.271353 20872 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:13:40.326596 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:40.333446 12527 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:13:40.288272 20827 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:40.287331 21249 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:40.329360 20872 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:40.359504 12527 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:40.324949 20958 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:40.371024 21360 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:40.374010 21376 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.065s	user 0.000s	sys 0.000s
I20260709 12:13:40.332106 21373 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:40.344985 21349 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:40.344926 21009 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:13:40.377130 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50636.1us
I20260709 12:13:40.364384 21246 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:40.404678 12528 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:40.406373 12528 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:40.408813 12528 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27. Ops: []
W20260709 12:13:40.363386 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 54882.8us
I20260709 12:13:40.413244 13305 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:40.383000 21115 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:40.379076 21216 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:40.418767 21216 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:40.419250 21216 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:40.423429 21274 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:40.396265 21014 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:40.390792 20958 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:40.436296 20913 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:40.387009 20872 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:40.325785 21341 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:40.381274 12530 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:40.329439 21102 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:40.396798 12524 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:40.248634 21068 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:40.456022 21068 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:40.456672 21068 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:40.457062 21068 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:40.457397 21068 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:40.459515 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:13:40.461290 12416 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 70b8c9b89e3d43fbb70422110d76702c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica a2fdbe7eef6748a9b4611f6c9bc9b201 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 21 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:13:40.461568 21386 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:13:40.411161 20799 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:40.378391 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 143091us
I20260709 12:13:40.395416 21358 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:40.449641 21384 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:13:40.500988 21384 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:13:40.501420 21384 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:40.501749 21384 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:13:40.502040 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 517950us
I20260709 12:13:40.450779 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:40.506829 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 56124.5us
I20260709 12:13:40.507885 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4104) took 1728 ms. Trace:
I20260709 12:13:40.508306 12676 rpcz_store.cc:276] 0709 12:13:38.780483 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:38.781741 (+  1258us) service_pool.cc:224] Handling call
0709 12:13:38.799982 (+ 18241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:39.686573 (+886591us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:40.507616 (+821043us) spinlock_profiling.cc:243] Waited 885 ms on lock 0x7b5400013bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:40.507818 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2036646784}
I20260709 12:13:40.512010 21190 consensus_queue.cc:260] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:40.513108 21190 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:13:40.451833 21357 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.911s	user 0.002s	sys 0.000s
I20260709 12:13:40.397900 21373 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:40.514809 21373 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:40.516109 21255 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:40.516764 21115 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:40.345192 21344 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:40.518424 21344 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:40.520676 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:40.466171 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 149558us
I20260709 12:13:40.523129 12680 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:13:40.528538 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:40.537631 12673 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:40.537837 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:40.538655 20913 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:40.539162 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.540221 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.540943 12606 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 23: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:40.541654 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.448392 21360 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:13:40.497072 21182 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:40.544260 21182 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:40.545763 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:40.546414 12525 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:40.489431 21341 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:40.547832 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.548463 12600 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 19.
I20260709 12:13:40.549408 21341 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:40.402905 21349 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:40.551739 21349 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:40.553815 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.554105 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.495012 21249 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:40.555054 12603 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:40.555279 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.556051 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:40.556665 12496 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:40.549793 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:40.557825 21249 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:40.546427 12602 raft_consensus.cc:2376] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25. Re-sending same reply.
I20260709 12:13:40.558736 21349 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:40.549752 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.587033 12600 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 22: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:40.556547 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.559618 21099 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:13:40.560199 21341 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:13:40.560330 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:40.560658 13302 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:40.536473 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:40.597510 12526 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:13:40.597090 12603 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:13:40.599254 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:40.561393 12606 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:13:40.568827 21324 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:40.578300 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:13:40.578522 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 119082us
I20260709 12:13:40.498003 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:40.592176 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:40.601964 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:40.609926 12600 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:40.611785 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:40.613041 12530 raft_consensus.cc:2368] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22.
I20260709 12:13:40.613727 12528 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:13:40.602244 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:40.614017 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 91089.7us
I20260709 12:13:40.614928 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.615991 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:40.617712 12606 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:13:40.607191 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:40.618273 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 125162us
I20260709 12:13:40.617964 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:40.619375 12494 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:13:40.619971 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:40.622846 12602 raft_consensus.cc:2410] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 28 because replica has last-logged OpId of term: 27 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
I20260709 12:13:40.622880 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:40.623889 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:40.586640 21349 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:40.627770 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:40.629103 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.626444 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:40.631906 12526 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:40.632782 12531 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 27.
I20260709 12:13:40.612566 21341 consensus_queue.cc:1243] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:13:40.639979 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:40.640810 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:40.640715 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:40.642030 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:40.650049 12603 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:40.642776 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:40.650161 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 136345us
I20260709 12:13:40.650991 12677 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
W20260709 12:13:40.651539 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:40.652885 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:40.656139 12600 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22.
I20260709 12:13:40.657712 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:40.658263 12603 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:40.658768 12600 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:40.659279 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:40.659796 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:40.662711 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.623232 21249 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Leader pre-election won for term 20
I20260709 12:13:40.625356 21358 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:40.669333 12603 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:40.669605 12600 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:40.631752 21349 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:40.628458 21274 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:13:40.675882 12602 raft_consensus.cc:2368] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 23. Current term is 24.
I20260709 12:13:40.634361 20913 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:40.689735 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:40.692164 12599 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:40.695000 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:40.702309 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:40.702692 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 20 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:40.703289 12602 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:40.703696 21390 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:40.704622 21390 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:40.707033 21324 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:40.707759 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:40.709162 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.713171 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.713078 21216 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:40.707854 21390 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:40.716109 21215 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:40.718063 21397 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:40.673228 21341 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:40.672183 21249 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:40.747440 21249 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:40.747092 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:40.748461 12530 raft_consensus.cc:2368] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
I20260709 12:13:40.662616 20799 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:13:40.690734 21391 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.045s
I20260709 12:13:40.722325 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:40.722643 21393 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:40.723691 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:40.625658 21384 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:40.769053 21246 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:40.727505 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:40.714005 12601 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:40.777352 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:40.778255 12601 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 22: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:40.715075 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:40.709836 12605 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:40.732695 12526 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22.
I20260709 12:13:40.630295 21102 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:40.734014 21274 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:40.779855 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.746905 21341 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:40.780428 21274 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:40.749394 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:40.676272 21358 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:40.781559 21358 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:40.782217 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:40.783540 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 20 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:40.783681 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:40.784279 12530 raft_consensus.cc:2393] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 20: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:40.784850 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:40.753114 21382 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.135s	user 0.000s	sys 0.000s
I20260709 12:13:40.757011 12572 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:40.785732 12526 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 27 because replica has last-logged OpId of term: 25 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
I20260709 12:13:40.757978 12602 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:13:40.786897 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.763661 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:40.771127 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 265047us
I20260709 12:13:40.780990 21341 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:40.788233 12599 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:40.791744 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:40.792044 12603 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 20.
I20260709 12:13:40.793111 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:40.794405 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:40.781783 21102 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:40.795715 21214 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:40.800011 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:40.801298 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:40.785130 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:40.793954 12603 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:40.803364 12530 raft_consensus.cc:2410] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 22 because replica has last-logged OpId of term: 21 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:13:40.701639 12600 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
W20260709 12:13:40.838341 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 167886us
I20260709 12:13:40.840593 12494 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:40.842962 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:40.843184 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 86271.8us
I20260709 12:13:40.844237 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:40.844370 12600 raft_consensus.cc:2410] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 24 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 13.
I20260709 12:13:40.850643 12603 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:13:40.839139 12646 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:40.858291 13305 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:40.862216 12571 leader_election.cc:400] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22.
I20260709 12:13:40.862530 12571 leader_election.cc:403] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Cancelling election due to peer responding with higher term
I20260709 12:13:40.761384 21009 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:40.860491 13302 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:40.759531 20913 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:40.876237 12526 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 12:13:40.883026 12530 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:40.886065 12571 leader_election.cc:400] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
I20260709 12:13:40.886480 12571 leader_election.cc:403] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:13:40.888412 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.734499 21098 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:40.897171 12677 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:40.898104 21215 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:13:40.899948 21215 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:40.900354 21215 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:40.901283 21215 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:40.844282 21393 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:40.904088 21393 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:40.904136 12522 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: [25.13-25.13]
I20260709 12:13:40.904693 21341 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:40.906924 21341 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:40.907054 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.908489 12598 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:40.909965 12606 raft_consensus.cc:2410] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:13:40.861896 21255 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:13:40.910522 12531 raft_consensus.cc:2368] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
I20260709 12:13:40.911235 12606 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:40.912411 12606 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:40.865887 20827 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 12:13:40.866137 21386 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:40.915115 21386 raft_consensus.cc:697] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:40.910470 21255 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:40.861979 21236 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:13:40.801832 21358 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:40.901633 21068 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:40.937851 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:40.889217 21102 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:40.924746 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:40.931602 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:40.882812 21246 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:13:40.925694 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:40.897647 21194 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election lost for term 21. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22.
I20260709 12:13:40.917356 21337 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:40.875290 21336 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.060s
I20260709 12:13:40.877820 20958 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:40.916435 21386 consensus_queue.cc:237] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:40.881726 12417 maintenance_manager.cc:643] P d5d8248387cf49cfbd3416fcacedae86: MajorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 2.410s	user 0.122s	sys 0.029s Metrics: {"cfile_cache_miss":8,"cfile_cache_miss_bytes":25500,"cfile_init":3,"delete_count":0,"delta_blocks_compacted":2,"delta_iterators_relevant":2,"lbm_read_time_us":38971,"lbm_reads_10-100_ms":1,"lbm_reads_lt_1ms":19,"lbm_write_time_us":9177,"lbm_writes_1-10_ms":2,"lbm_writes_lt_1ms":8,"peak_mem_usage":259963,"reinsert_count":0,"thread_start_us":194016,"threads_started":2,"update_count":172}
I20260709 12:13:40.939863 12603 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:40.941941 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 101374us
W20260709 12:13:40.981071 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 45736.5us
I20260709 12:13:41.007238 12526 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:40.924742 21255 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:41.009804 12676 raft_consensus.cc:2393] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:40.922338 21399 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:40.981389 21215 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:41.002101 20872 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.564s	user 0.003s	sys 0.007s
I20260709 12:13:40.992100 21162 raft_consensus.cc:3055] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:41.011111 12600 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
W20260709 12:13:40.964569 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 107710us
I20260709 12:13:41.076602 21306 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:41.083631 21358 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:41.085507 21392 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.155s	user 0.002s	sys 0.000s
W20260709 12:13:41.091142 21402 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:13:41.094003 21068 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:41.095275 21216 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:41.096467 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:41.097908 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:41.100996 12525 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21. Ops: [20.11-20.11]
I20260709 12:13:41.010496 21393 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.092s
I20260709 12:13:41.038442 21246 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.104357 21191 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:41.105640 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:41.106519 20958 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:41.107577 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:41.108578 21190 consensus_meta.cc:287] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.595s	user 0.011s	sys 0.002s
I20260709 12:13:41.109519 21330 raft_consensus.cc:3055] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:41.110109 21330 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:41.111382 21330 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.112666 12599 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:41.113368 12605 raft_consensus.cc:2410] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
W20260709 12:13:41.117910 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:41.107510 12602 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 28. Current term is 29. Ops: []
W20260709 12:13:41.121282 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 81794.1us
I20260709 12:13:41.134951 12600 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:41.147763 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:41.127942 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:41.149220 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:41.151978 12599 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:41.152256 12600 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 20. Current term is 21. Ops: [20.13-20.13]
W20260709 12:13:41.154002 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 45886.2us
I20260709 12:13:41.154989 12418 maintenance_manager.cc:419] P d5d8248387cf49cfbd3416fcacedae86: Scheduling MinorDeltaCompactionOp(00000000000000000000000000000000): perf score=0.002000
I20260709 12:13:40.942764 20799 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:41.128785 12572 leader_election.cc:400] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22.
W20260709 12:13:41.169947 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 94777.6us
I20260709 12:13:41.171273 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:41.171857 12672 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:41.155839 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:41.173270 12526 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:13:41.174218 12680 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: [20.12-20.12]
I20260709 12:13:41.112775 21330 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:41.110141 21246 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:41.181130 12572 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:41.192615 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:41.119968 21399 raft_consensus.cc:697] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:41.165940 12600 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:41.204283 12602 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 27: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:41.209791 12572 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:41.166203 12417 maintenance_manager.cc:643] P d5d8248387cf49cfbd3416fcacedae86: MinorDeltaCompactionOp(00000000000000000000000000000000) complete. Timing: real 0.000s	user 0.000s	sys 0.000s Metrics: {}
I20260709 12:13:41.214344 12572 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:41.122431 21162 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:41.121012 21102 raft_consensus.cc:697] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 25 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:41.217233 21414 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:41.135393 20872 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:41.232570 21102 consensus_queue.cc:237] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:41.235656 20872 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:41.236403 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:41.236912 12600 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:41.237568 13336 raft_consensus.cc:2368] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28.
I20260709 12:13:41.218703 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:41.221611 21336 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:41.174763 21384 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:41.222239 21410 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:13:41.240842 21410 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:41.241343 21410 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.243654 21410 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:41.223012 21399 consensus_queue.cc:237] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.248436 21336 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:41.250772 12572 leader_election.cc:400] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
I20260709 12:13:41.093891 21255 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.252192 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3948) took 1444 ms. Trace:
I20260709 12:13:41.254709 12572 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:41.255187 21349 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.559s	user 0.004s	sys 0.006s
I20260709 12:13:41.255596 21336 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:41.213734 21068 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 25.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.048s
I20260709 12:13:41.255851 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:41.256814 21068 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:41.255605 21349 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.257210 21068 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:41.258745 21349 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:41.259872 12572 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:41.253839 21255 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:41.260191 21014 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:41.261174 21336 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:41.263038 12572 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:41.198002 21383 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.088s
I20260709 12:13:41.264256 21383 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:13:41.264669 21383 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:41.264758 21255 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:41.267371 20958 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:13:41.267916 20958 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:41.269088 20958 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:41.209537 21373 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.688s	user 0.013s	sys 0.000s
I20260709 12:13:41.271636 21373 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.273289 12528 raft_consensus.cc:2368] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26.
I20260709 12:13:41.274097 12605 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:41.274816 21307 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:13:41.225605 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:41.228003 21249 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:40.999152 20913 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.188099 21395 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.125s
I20260709 12:13:41.280042 21395 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:13:41.280462 21395 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:41.283156 21344 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:41.252779 12604 rpcz_store.cc:276] 0709 12:13:39.807382 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:39.813365 (+  5983us) service_pool.cc:224] Handling call
0709 12:13:39.813763 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:40.042087 (+228324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:40.042141 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:40.069297 (+ 27156us) log.cc:844] Serialized 26 byte log entry
0709 12:13:40.239406 (+170109us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:40.239456 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:41.251597 (+1012141us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b5400270bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:41.251689 (+    92us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:41.251758 (+    69us) raft_consensus.cc:1685] finished
0709 12:13:41.251781 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:41.252106 (+   325us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":521739008,"thread_start_us":97936,"threads_started":1,"wal-append.queue_time_us":200382}
I20260709 12:13:41.286633 20958 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:13:41.287060 21406 log.cc:927] Time spent T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.191s	user 0.001s	sys 0.001s
I20260709 12:13:41.264180 12572 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:41.308444 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:41.299368 12672 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:41.155294 12494 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:41.316931 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 52832.4us
I20260709 12:13:41.318436 12603 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:41.168167 20827 consensus_queue.cc:1243] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 26.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
W20260709 12:13:41.327116 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 252801us
W20260709 12:13:41.329162 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 443226us
I20260709 12:13:41.329376 12603 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:13:41.329198 12572 leader_election.cc:400] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
W20260709 12:13:41.353492 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:41.353983 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:13:41.356164 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:41.356355 12572 leader_election.cc:403] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:41.343397 12599 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
W20260709 12:13:41.358819 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:41.359793 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:41.360131 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:41.360651 12527 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:13:41.360906 12528 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 20: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:41.361497 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 977) took 1372 ms. Trace:
I20260709 12:13:41.361804 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:41.363168 12527 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:41.322716 21337 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
W20260709 12:13:41.366160 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 210961us
I20260709 12:13:41.366489 12572 leader_election.cc:400] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26.
I20260709 12:13:41.366971 12572 leader_election.cc:403] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:41.367000 12494 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:41.365257 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 56933us
I20260709 12:13:41.265661 21383 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:41.370204 12527 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20. Ops: []
I20260709 12:13:41.374805 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:41.361884 12597 rpcz_store.cc:276] 0709 12:13:39.989108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.020558 (+ 31450us) service_pool.cc:224] Handling call
0709 12:13:40.047310 (+ 26752us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:40.047572 (+   262us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:13:40.069686 (+ 22114us) log.cc:844] Serialized 14 byte log entry
0709 12:13:41.079067 (+1009381us) spinlock_profiling.cc:243] Waited 1.3 ms on lock 0x7b18004d3320. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a 00007f1488a2aa2a
0709 12:13:41.361151 (+282084us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:41.361206 (+    55us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:41.361254 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:41.361320 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:41.361438 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10167,"prepare.queue_time_us":21836,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":4688,"raft.queue_time_us":20301,"spinlock_wait_cycles":2981632,"thread_start_us":271657,"threads_started":1}
I20260709 12:13:41.384083 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:41.258029 21068 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.12, Last appended by leader: 12, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:41.370618 21274 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.569s	user 0.009s	sys 0.016s
I20260709 12:13:41.386667 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:41.393882 12527 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 22: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:41.398622 12526 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 26: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:41.368887 21191 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23.
I20260709 12:13:41.328608 21336 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:13:41.394977 12496 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:41.456781 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 61921.3us
W20260709 12:13:41.457145 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 64121.8us
I20260709 12:13:41.457934 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:41.397495 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:41.458755 12527 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
W20260709 12:13:41.459304 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 64055.6us
I20260709 12:13:41.459698 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:41.460047 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:41.461740 12494 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:41.365748 21373 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:41.460278 12527 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:41.354784 21410 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.023s
I20260709 12:13:41.371369 21236 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:13:41.398072 21274 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s	user 0.010s	sys 0.017s
I20260709 12:13:41.394233 21255 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:41.473533 12494 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:41.393815 21249 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:41.397317 21414 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 26
I20260709 12:13:41.462033 21383 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:41.492648 12494 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:41.493561 21306 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:41.494086 21306 raft_consensus.cc:697] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:41.493990 12494 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:41.494391 21246 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:41.495239 21306 consensus_queue.cc:237] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.12, Last appended by leader: 12, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.495980 21099 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:41.497251 12494 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:41.505774 12527 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 28.
I20260709 12:13:41.473678 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:41.477465 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:41.385514 21415 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:41.366380 21162 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.399399 21412 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:41.479856 21373 raft_consensus.cc:2764] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:13:41.488307 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:41.397574 21349 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:41.467664 21183 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:13:41.455447 20799 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:41.522315 20799 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.525269 20799 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:41.526289 21405 consensus_queue.cc:1059] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 19.11, Next index: 12, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:41.527489 21009 raft_consensus.cc:3055] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:41.477807 21216 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:13:41.534624 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46434.1us
I20260709 12:13:41.509857 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:41.484938 21068 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:41.496508 21099 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:41.532279 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:41.473358 12522 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.004s	sys 0.012s
I20260709 12:13:41.528012 21009 raft_consensus.cc:740] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:41.508481 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:41.555441 12527 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:41.508066 21396 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:13:41.552119 12522 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s	user 0.004s	sys 0.012s
I20260709 12:13:41.558323 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:41.561249 12678 raft_consensus.cc:2410] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 23 because replica has last-logged OpId of term: 20 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 13.
I20260709 12:13:41.505858 21246 raft_consensus.cc:697] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:41.512344 21393 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:41.563295 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:41.564267 21246 consensus_queue.cc:237] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 19.11, Last appended by leader: 11, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.564494 21009 consensus_queue.cc:260] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 19.11, Last appended by leader: 11, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.564941 21415 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:41.557736 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:41.566174 21377 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:41.566372 21115 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:41.566704 12675 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 26: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:41.566974 21115 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.568010 21194 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:41.568449 21194 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:41.562507 21099 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.513988 21236 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:41.570690 21415 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 22.17, Last appended by leader: 17, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:41.575304 12602 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:41.576061 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3424) took 1052 ms. Trace:
I20260709 12:13:41.572674 21236 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.576412 12602 rpcz_store.cc:276] 0709 12:13:40.523406 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:41.574677 (+1051271us) service_pool.cc:224] Handling call
0709 12:13:41.575072 (+   395us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:41.575796 (+   724us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:41.576002 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:41.577373 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:41.579800 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:41.516813 21375 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:41.553763 12526 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:13:41.532637 21412 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:41.565918 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 987) took 1154 ms. Trace:
I20260709 12:13:41.569357 21194 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.605734 12601 rpcz_store.cc:276] 0709 12:13:40.411034 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:41.565120 (+1154086us) service_pool.cc:224] Handling call
0709 12:13:41.565435 (+   315us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:41.565597 (+   162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:41.565620 (+    23us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:41.565657 (+    37us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:13:41.565688 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:41.565735 (+    47us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:41.565851 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:41.606489 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:41.620012 12680 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:41.619264 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:41.608973 12524 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:41.606588 13305 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:41.517939 21344 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.086s
I20260709 12:13:41.508569 21215 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.074s
I20260709 12:13:41.506255 21404 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.091s
I20260709 12:13:41.579715 21098 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:41.495767 21395 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.566061 21387 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:13:41.596400 12527 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 28.
I20260709 12:13:41.557072 21349 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:41.565788 21009 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:41.556372 21249 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:13:41.596608 21373 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:41.647097 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:41.555481 21403 consensus_queue.cc:1059] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 19.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.116s
I20260709 12:13:41.655264 13305 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:13:41.759649 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 104469us
I20260709 12:13:41.597098 21416 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:41.615037 21412 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:41.716082 12599 raft_consensus.cc:2368] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22.
I20260709 12:13:41.783993 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:41.647836 12524 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:41.617343 21375 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:41.716522 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:41.713578 21194 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:41.718590 12602 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:13:41.714376 21362 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:41.641433 21093 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:13:41.726419 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 249014us
I20260709 12:13:41.729454 12604 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:41.736161 20913 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:41.739383 21377 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:41.750972 21249 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:41.755918 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3396) took 1583 ms. Trace:
I20260709 12:13:41.758731 21349 raft_consensus.cc:697] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:41.781005 21416 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:41.781450 21400 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:13:41.787389 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 134173us
W20260709 12:13:41.839740 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:41.840358 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3368) took 1246 ms. Trace:
W20260709 12:13:41.787604 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:41.800022 12597 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 22: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:13:41.841449 12671 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.553s	user 0.018s	sys 0.016s
I20260709 12:13:41.797937 21395 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:41.666267 21344 consensus_queue.cc:1243] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:13:41.809020 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:41.809190 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3954) took 1173 ms. Trace:
I20260709 12:13:41.796494 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:41.753870 21410 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:41.816257 12598 rpcz_store.cc:276] 0709 12:13:40.172746 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.909135 (+736389us) service_pool.cc:224] Handling call
0709 12:13:40.909428 (+   293us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:40.909819 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:40.909842 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:40.921595 (+ 11753us) log.cc:844] Serialized 26 byte log entry
0709 12:13:41.458567 (+536972us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:41.458629 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:41.458723 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:41.755627 (+296904us) raft_consensus.cc:1685] finished
0709 12:13:41.755651 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:41.755845 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":33053,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":32722,"thread_start_us":485015,"threads_started":1,"wal-append.queue_time_us":684591}
I20260709 12:13:41.772049 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:41.812788 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:41.853698 12602 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 23: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:41.818953 12680 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:41.783829 21413 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:41.853951 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:41.782990 21236 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:41.820335 21377 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:41.860787 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:41.861630 12597 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 30.
W20260709 12:13:41.845536 12671 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.557s	user 0.018s	sys 0.016s
W20260709 12:13:41.803682 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 306487us
I20260709 12:13:41.848523 12604 rpcz_store.cc:276] 0709 12:13:40.635352 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:41.728883 (+1093531us) service_pool.cc:224] Handling call
0709 12:13:41.729227 (+   344us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:41.808867 (+ 79640us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:41.809117 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:41.798321 21399 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:41.776791 21373 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.840828 12530 rpcz_store.cc:276] 0709 12:13:40.594356 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.919642 (+325286us) service_pool.cc:224] Handling call
0709 12:13:40.920032 (+   390us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:40.920510 (+   478us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:40.920557 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:40.928249 (+  7692us) log.cc:844] Serialized 26 byte log entry
0709 12:13:41.402896 (+474647us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:41.836430 (+433534us) log.cc:844] Serialized 14 byte log entry
0709 12:13:41.839178 (+  2748us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:41.840083 (+   905us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:41.840129 (+    46us) raft_consensus.cc:1685] finished
0709 12:13:41.840144 (+    15us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:41.840291 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":7466,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":6072,"thread_start_us":765365,"threads_started":2,"wal-append.queue_time_us":665140}
W20260709 12:13:41.835757 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 467602us
I20260709 12:13:41.880587 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:41.808187 21405 consensus_queue.cc:1059] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 12:13:41.870316 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:41.881928 12598 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:13:41.873376 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:41.884935 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:41.882934 13336 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:41.816649 20913 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.857646 21400 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:41.879418 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:41.940798 21115 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:41.901453 12597 raft_consensus.cc:2410] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 28 because replica has last-logged OpId of term: 27 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
I20260709 12:13:41.818939 21362 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.859915 21413 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.871228 21215 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:13:41.930480 12528 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.556s	user 0.015s	sys 0.021s
I20260709 12:13:41.959322 21405 raft_consensus.cc:3055] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:13:41.989953 12528 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.616s	user 0.015s	sys 0.022s
W20260709 12:13:41.907712 21419 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:13:41.941808 12494 leader_election.cc:400] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28.
I20260709 12:13:41.990633 12494 leader_election.cc:403] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Cancelling election due to peer responding with higher term
I20260709 12:13:41.942058 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:41.946578 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:41.958362 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:41.952550 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:41.992869 12604 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:41.932282 21349 consensus_queue.cc:237] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 21.10, Last appended by leader: 10, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:41.963411 21420 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.302s	user 0.000s	sys 0.001s
I20260709 12:13:41.947628 21393 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.054s
I20260709 12:13:41.888981 21396 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.063s
I20260709 12:13:41.905853 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:41.980064 12602 raft_consensus.cc:2376] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27. Re-sending same reply.
I20260709 12:13:41.879521 21410 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:41.965996 21307 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.690s	user 0.009s	sys 0.000s
I20260709 12:13:41.888404 21399 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:41.985129 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1078) took 1705 ms. Trace:
I20260709 12:13:41.924880 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:41.931445 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 122501us
W20260709 12:13:41.995570 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 58037.3us
I20260709 12:13:42.037477 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:41.926831 21249 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:42.033799 12523 rpcz_store.cc:276] 0709 12:13:40.279366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.279489 (+   123us) service_pool.cc:224] Handling call
0709 12:13:40.315590 (+ 36101us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:40.315814 (+   224us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:13:40.369830 (+ 54016us) log.cc:844] Serialized 14 byte log entry
0709 12:13:41.248677 (+878847us) spinlock_profiling.cc:243] Waited 7.43 ms on lock 0x7b18004513c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a 00007f1488a2aa2a
0709 12:13:41.956067 (+707390us) log.cc:844] Serialized 14 byte log entry
0709 12:13:41.984734 (+ 28667us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:41.984810 (+    76us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:13:41.984878 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:41.984959 (+    81us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:41.985070 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":93186,"prepare.queue_time_us":48210,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":43,"raft.queue_time_us":95963,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":14864,"spinlock_wait_cycles":17078144,"thread_start_us":641117,"threads_started":2,"wal-append.queue_time_us":1427620}
I20260709 12:13:41.996701 12603 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 28: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:41.859453 21375 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:41.998301 12524 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
W20260709 12:13:41.964152 21427 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
I20260709 12:13:41.999034 12530 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:13:42.027275 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:42.060922 12597 raft_consensus.cc:2410] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 21 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:13:42.040843 13330 raft_consensus.cc:2410] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 30 because replica has last-logged OpId of term: 28 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 18.
I20260709 12:13:41.998008 21246 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election lost for term 27. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28.
W20260709 12:13:42.029553 21414 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.538s	user 0.006s	sys 0.006s
I20260709 12:13:42.068567 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:42.057488 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:41.986922 21216 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:13:41.992692 21014 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.732s	user 0.011s	sys 0.003s
I20260709 12:13:41.996241 21405 raft_consensus.cc:740] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:42.056372 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 19 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:42.072999 12602 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 19: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:42.069851 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:42.075966 12603 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:42.083732 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:42.083892 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:42.087630 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:42.088635 12598 raft_consensus.cc:2368] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 25.
I20260709 12:13:42.064152 21390 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.092573 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4133) took 1565 ms. Trace:
I20260709 12:13:42.065745 21336 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:42.091569 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 210228us
I20260709 12:13:42.096168 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:42.098949 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:42.099472 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1086) took 1450 ms. Trace:
I20260709 12:13:42.099860 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:42.077771 21425 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.264s	user 0.000s	sys 0.001s
I20260709 12:13:42.071584 21337 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.100737 12675 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 28: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:42.093532 12671 rpcz_store.cc:276] 0709 12:13:40.527017 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.527186 (+   169us) service_pool.cc:224] Handling call
0709 12:13:41.287334 (+760148us) spinlock_profiling.cc:243] Waited 1.78 ms on lock 0x7b58001f2c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fbeee 000000000036eb39 00007f148d96f1e0 00007f148da380c4 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:13:41.287496 (+   162us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:41.287706 (+   210us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:13:41.288024 (+   318us) log.cc:844] Serialized 14 byte log entry
0709 12:13:42.092226 (+804202us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:42.092285 (+    59us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:42.092334 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:42.092404 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:42.092514 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":16537,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"spinlock_wait_cycles":11254144,"thread_start_us":802994,"threads_started":2,"wal-append.queue_time_us":702262}
W20260709 12:13:42.101478 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 249992us
I20260709 12:13:42.101890 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:42.102211 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:42.100039 12531 rpcz_store.cc:276] 0709 12:13:40.648749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.912301 (+263552us) service_pool.cc:224] Handling call
0709 12:13:40.912601 (+   300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:40.912792 (+   191us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:40.912815 (+    23us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:40.923120 (+ 10305us) log.cc:844] Serialized 26 byte log entry
0709 12:13:41.353816 (+430696us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:41.353876 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:41.353969 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:42.099195 (+745226us) raft_consensus.cc:1685] finished
0709 12:13:42.099225 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:42.099399 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":372701,"threads_started":1,"wal-append.queue_time_us":590682}
W20260709 12:13:42.081025 21424 log.cc:927] Time spent T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.244s	user 0.001s	sys 0.000s
W20260709 12:13:42.108304 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:42.108323 12678 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 19.
I20260709 12:13:42.119493 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:42.120301 12603 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:42.092871 12494 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:42.111316 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3952) took 1477 ms. Trace:
I20260709 12:13:42.104879 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:42.121568 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:42.107652 12677 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
I20260709 12:13:42.117467 12672 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:13:42.127791 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:42.129109 12571 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:42.081103 21414 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26.
I20260709 12:13:42.055620 21115 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:13:42.139552 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:42.140854 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:42.144626 12676 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
W20260709 12:13:42.143066 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 50265.3us
I20260709 12:13:42.140395 21445 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.149698 21445 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:42.153802 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:42.154512 12598 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:42.121894 12605 rpcz_store.cc:276] 0709 12:13:40.634491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:41.724151 (+1089660us) service_pool.cc:224] Handling call
0709 12:13:41.724485 (+   334us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:41.724700 (+   215us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:13:41.955631 (+230931us) log.cc:844] Serialized 14 byte log entry
0709 12:13:42.107132 (+151501us) log.cc:844] Serialized 14 byte log entry
0709 12:13:42.107283 (+   151us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:42.110036 (+  2753us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:13:42.110299 (+   263us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:42.110877 (+   578us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:42.111204 (+   327us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":339,"prepare.queue_time_us":145035,"prepare.run_cpu_time_us":58,"prepare.run_wall_time_us":59,"thread_start_us":380854,"threads_started":2}
W20260709 12:13:42.141944 21427 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.058s	user 0.000s	sys 0.001s
I20260709 12:13:42.081231 21216 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:42.133536 21373 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:42.166375 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:42.171000 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:42.157552 21362 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:42.179504 12605 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:42.159174 21396 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:42.106652 21336 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:42.192298 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:42.136245 21337 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:42.185451 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:42.192689 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:42.117238 21390 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:42.204352 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3399) took 2030 ms. Trace:
I20260709 12:13:42.204749 12494 leader_election.cc:400] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 25.
I20260709 12:13:42.205147 12494 leader_election.cc:403] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 12:13:42.204779 12606 rpcz_store.cc:276] 0709 12:13:40.173590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.912985 (+739395us) service_pool.cc:224] Handling call
0709 12:13:40.913250 (+   265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:41.011231 (+ 97981us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:41.011267 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:41.040266 (+ 28999us) log.cc:844] Serialized 26 byte log entry
0709 12:13:41.393727 (+353461us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:41.393783 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:41.393876 (+    93us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:42.204124 (+810248us) raft_consensus.cc:1685] finished
0709 12:13:42.204145 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:42.204292 (+   147us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":135280,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":30940,"thread_start_us":317281,"threads_started":2,"wal-append.queue_time_us":477435}
I20260709 12:13:42.207368 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:42.184448 21450 raft_consensus.cc:3055] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:42.209640 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:42.197515 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1084) took 1581 ms. Trace:
I20260709 12:13:42.186161 21416 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:42.216926 12602 raft_consensus.cc:2410] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 21 because replica has last-logged OpId of term: 20 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 10.
I20260709 12:13:42.094215 21400 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:42.190845 21445 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:42.210186 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:42.211431 12522 rpcz_store.cc:276] 0709 12:13:40.616068 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.905049 (+288981us) service_pool.cc:224] Handling call
0709 12:13:40.905314 (+   265us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:40.905676 (+   362us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:40.905703 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:40.921802 (+ 16099us) log.cc:844] Serialized 26 byte log entry
0709 12:13:41.561888 (+640086us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:13:41.582025 (+ 20137us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:41.582166 (+   141us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:42.197280 (+615114us) raft_consensus.cc:1685] finished
0709 12:13:42.197302 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:42.197446 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1920,"raft.run_cpu_time_us":86,"raft.run_wall_time_us":85,"thread_start_us":588727,"threads_started":1,"wal-append.queue_time_us":681831}
I20260709 12:13:42.228472 12606 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:42.228531 12603 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 20.
I20260709 12:13:42.190012 21449 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:13:42.214699 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 226826us
I20260709 12:13:42.206423 12494 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:42.201689 21115 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:13:42.207973 21255 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:13:42.214561 21442 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.226s	user 0.000s	sys 0.001s
W20260709 12:13:42.197453 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 133339us
I20260709 12:13:42.246289 21362 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:42.247861 21336 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:42.251734 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:42.255474 21457 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:13:42.256942 12606 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
W20260709 12:13:42.266039 21452 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
W20260709 12:13:42.267156 21447 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:13:42.269258 21337 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:42.211474 21306 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 25.
W20260709 12:13:42.274963 21454 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:13:42.277591 21403 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:42.209774 21450 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:42.331499 21389 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:13:42.332000 21389 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:42.285655 21255 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:42.336074 21255 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:13:42.338028 13330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38472 (request call id 962) took 1818 ms (client timeout 1909 ms). Trace:
I20260709 12:13:42.338325 13333 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 19. (term mismatch)
I20260709 12:13:42.338788 13333 pending_rounds.cc:85] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 18
W20260709 12:13:42.338518 13330 rpcz_store.cc:269] 0709 12:13:40.520067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.520401 (+   334us) service_pool.cc:224] Handling call
0709 12:13:42.337804 (+1817403us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54004439d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:42.337964 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3493601536}
I20260709 12:13:42.339247 13333 pending_rounds.cc:107] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 28.19
I20260709 12:13:42.198823 21455 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:42.339946 21455 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:42.340319 21455 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:42.208827 21246 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:13:42.346423 21246 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:42.346850 21246 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:42.176167 21405 consensus_queue.cc:260] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:42.208560 21415 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:13:42.275048 21430 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.317s	user 0.003s	sys 0.000s
I20260709 12:13:42.300704 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:42.353966 12604 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:42.301021 12529 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:13:42.299505 12598 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 28.
I20260709 12:13:42.307878 12606 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:42.355178 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:42.309429 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:42.312330 12496 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:42.316365 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:42.358842 12672 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:42.318986 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:42.326380 21461 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.339560 13333 raft_consensus.cc:2889] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:42.323238 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:42.362161 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 61567.6us
I20260709 12:13:42.351138 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:42.357395 12677 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
W20260709 12:13:42.320693 21009 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.654s	user 0.012s	sys 0.000s
I20260709 12:13:42.363134 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:42.366997 12494 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:42.312922 21413 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:42.379124 12679 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
W20260709 12:13:42.374594 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 62375.8us
I20260709 12:13:42.380231 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:42.380923 12672 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 19.
I20260709 12:13:42.382792 12606 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 22 index: 16. (index mismatch)
I20260709 12:13:42.333431 21389 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:42.384027 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:42.388859 21438 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:42.389130 12494 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:42.324105 21410 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:42.392128 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:42.354632 21274 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.353868 21415 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:42.375342 21403 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:42.407131 12672 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 19 index: 11. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:13:42.410734 12606 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:13:42.385234 21344 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:13:42.406431 21194 consensus_meta.cc:287] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.594s	user 0.012s	sys 0.000s
I20260709 12:13:42.410269 21415 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:42.395682 21438 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:42.390934 21395 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:42.429735 21455 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:42.408078 12496 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:42.330327 21450 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:42.412199 21274 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:42.436884 21389 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:13:42.439431 21068 consensus_meta.cc:287] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.857s	user 0.008s	sys 0.004s
I20260709 12:13:42.398430 21399 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:42.445076 12601 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:42.373524 21461 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:42.449117 12599 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:13:42.454324 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:42.407425 21099 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:42.384169 21215 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:42.472880 21455 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:42.466998 21009 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.030s
W20260709 12:13:42.462283 21412 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.662s	user 0.012s	sys 0.000s
I20260709 12:13:42.477517 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3967) took 1358 ms. Trace:
I20260709 12:13:42.511121 12600 rpcz_store.cc:276] 0709 12:13:41.119105 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:41.347995 (+228890us) service_pool.cc:224] Handling call
0709 12:13:41.348507 (+   512us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:41.349035 (+   528us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:41.349089 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:41.354881 (+  5792us) log.cc:844] Serialized 26 byte log entry
0709 12:13:41.645893 (+291012us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:41.645939 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:41.646012 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:42.274419 (+628407us) spinlock_profiling.cc:243] Waited 3.29 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:42.468030 (+193611us) raft_consensus.cc:1685] finished
0709 12:13:42.468056 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:42.477444 (+  9388us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7998,"raft.queue_time_us":2312,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":2417,"spinlock_wait_cycles":7560064,"thread_start_us":263416,"threads_started":1,"wal-append.queue_time_us":520129}
I20260709 12:13:42.461530 21344 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:42.497012 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:42.497543 12676 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:13:42.488842 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3478) took 1123 ms. Trace:
I20260709 12:13:42.495679 12680 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 20 index: 14. Preceding OpId from leader: term: 22 index: 14. (term mismatch)
I20260709 12:13:42.518672 12680 pending_rounds.cc:85] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 13
I20260709 12:13:42.519052 12680 pending_rounds.cc:107] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 20.14
I20260709 12:13:42.519397 12680 raft_consensus.cc:2889] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:13:42.483090 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:42.473140 12605 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 21: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:13:42.493872 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:42.476919 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 69041.5us
I20260709 12:13:42.523931 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:42.484707 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:42.499559 12672 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 18.
I20260709 12:13:42.487275 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:42.525691 12673 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:13:42.526397 12677 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 27 because replica has last-logged OpId of term: 25 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 17.
I20260709 12:13:42.464576 21438 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:42.531119 21375 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.531687 21375 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:42.533849 21375 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:42.485349 21377 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.615s	user 0.006s	sys 0.004s
I20260709 12:13:42.485457 21404 raft_consensus.cc:3055] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Stepping down as leader of term 19
I20260709 12:13:42.558596 21404 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:13:42.559073 21439 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.494s	user 0.001s	sys 0.000s
I20260709 12:13:42.559845 21399 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.477864 21182 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:42.560401 21399 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:42.560828 21182 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:42.561388 21182 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:42.563119 21390 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:42.565117 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:42.517550 12678 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
W20260709 12:13:42.471542 20958 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.184s	user 0.011s	sys 0.004s
I20260709 12:13:42.534288 21411 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.536289 21194 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:13:42.537554 12522 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:13:42.541066 21461 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:42.525315 12496 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:42.518532 12526 rpcz_store.cc:276] 0709 12:13:41.365723 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:41.598574 (+232851us) service_pool.cc:224] Handling call
0709 12:13:42.488518 (+889944us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b54002107d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:42.488757 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1613212032}
I20260709 12:13:42.547966 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:42.550530 12673 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22.
I20260709 12:13:42.551765 12677 pending_rounds.cc:85] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 14
I20260709 12:13:42.496068 21215 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:42.553491 12571 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:42.581707 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:42.584638 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:42.476886 21362 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.076s
I20260709 12:13:42.556628 21377 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:42.557667 21344 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:42.568332 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:42.464825 21403 consensus_queue.cc:1059] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.088s
I20260709 12:13:42.569732 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3482) took 1186 ms. Trace:
I20260709 12:13:42.572697 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:42.567198 21411 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:42.577688 12677 pending_rounds.cc:107] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 23.15
W20260709 12:13:42.508404 21383 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.005s	user 0.010s	sys 0.003s
W20260709 12:13:42.578682 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 53464us
I20260709 12:13:42.618346 12602 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:42.618021 12530 rpcz_store.cc:276] 0709 12:13:41.382953 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:41.906624 (+523671us) service_pool.cc:224] Handling call
0709 12:13:41.909072 (+  2448us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:42.060036 (+150964us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:42.569398 (+509362us) spinlock_profiling.cc:243] Waited 89.6 ms on lock 0x7b540038ead4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:42.569652 (+   254us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":206077184}
W20260709 12:13:42.619735 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 93056.5us
I20260709 12:13:42.588677 12606 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:13:42.621317 21360 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:13:42.621960 21387 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:42.563053 21399 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:42.609493 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:42.625221 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:42.625197 21399 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:13:42.626231 12672 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:42.626668 21194 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:42.618462 12677 raft_consensus.cc:2889] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:42.627230 21194 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:42.615171 12676 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 24: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:42.610343 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:42.629294 12680 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:13:42.629974 21194 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:42.631629 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:42.634222 21014 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:42.634855 21014 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:42.635326 21014 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:42.622736 21387 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:42.642171 21387 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:42.650352 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:42.657528 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 150399us
W20260709 12:13:42.656703 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49197.2us
I20260709 12:13:42.584055 21349 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:42.606873 21455 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:42.518539 21412 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:42.664759 12680 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 28.
I20260709 12:13:42.666615 13302 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 20 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:42.609261 21468 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:42.617417 21461 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.061s
I20260709 12:13:42.685405 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:42.669312 21455 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:42.685596 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:42.690888 12672 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:13:42.654582 20799 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.062s
W20260709 12:13:42.692929 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:42.696659 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:42.697631 12680 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 18 FOLLOWER]: Advancing to term 20
W20260709 12:13:42.703151 12529 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 1077) took 2424 ms (client timeout 1709 ms). Trace:
I20260709 12:13:42.680840 21468 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:42.724452 21468 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:42.724713 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:42.725689 12677 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 21
I20260709 12:13:42.726955 21412 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:42.731264 21377 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:42.733929 21438 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:42.738437 21411 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:42.703732 12529 rpcz_store.cc:269] 0709 12:13:40.279013 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:40.279150 (+   137us) service_pool.cc:224] Handling call
0709 12:13:42.702941 (+2423791us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b5400015c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:42.703073 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4061498624}
I20260709 12:13:42.711340 12598 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 18 index: 11. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
W20260709 12:13:42.713227 13304 leader_election.cc:336] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.719s (SENT)
I20260709 12:13:42.743187 13304 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:42.717818 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4152) took 1157 ms. Trace:
I20260709 12:13:42.685669 21375 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:13:42.694283 21455 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:42.722286 20913 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:42.723301 21246 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:42.744426 12598 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 15 index: 9. Preceding OpId from leader: term: 20 index: 12. (index mismatch)
I20260709 12:13:42.743187 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:42.747567 12524 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 24: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:42.744272 12674 rpcz_store.cc:276] 0709 12:13:41.560638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:41.560839 (+   201us) service_pool.cc:224] Handling call
0709 12:13:41.561133 (+   294us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:42.536295 (+975162us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:42.536327 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:42.536380 (+    53us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:42.536412 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:42.717489 (+181077us) spinlock_profiling.cc:243] Waited 942 ms on lock 0x7b54000aafd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:42.717556 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:42.717740 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32200,"raft.queue_time_us":32653,"raft.run_cpu_time_us":87,"raft.run_wall_time_us":88,"spinlock_wait_cycles":2167638912}
I20260709 12:13:42.749663 12680 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 20.
I20260709 12:13:42.750842 12674 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:13:42.754698 12672 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:13:42.768971 12522 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
W20260709 12:13:42.771709 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 100213us
I20260709 12:13:42.772559 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:42.773877 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:42.774539 12598 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:42.788726 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:42.790688 12527 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 27. Current term is 28. Ops: [27.15-27.15]
I20260709 12:13:42.793072 12672 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:42.791898 12598 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
W20260709 12:13:42.796231 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 83042.3us
I20260709 12:13:42.796656 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:42.798559 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:42.799263 12680 raft_consensus.cc:2410] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 25 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:13:42.687316 21014 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:42.764436 21438 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:42.820546 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 20 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:42.821254 12680 raft_consensus.cc:2376] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 20. Re-sending same reply.
I20260709 12:13:42.822336 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:42.822907 12680 raft_consensus.cc:2376] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22. Re-sending same reply.
I20260709 12:13:42.824311 21246 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:42.825357 21014 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:42.825445 12672 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:42.826259 21255 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:42.826898 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:42.827941 21249 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:13:42.828554 21396 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.828685 21255 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:42.829842 12604 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 30. Current term is 31. Ops: []
I20260709 12:13:42.793099 21307 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader election won for term 20
I20260709 12:13:42.822919 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:42.847074 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:42.880887 21461 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.883029 12680 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:13:42.885596 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:42.801659 21410 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:42.836067 20827 raft_consensus.cc:3055] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:13:42.887483 21420 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:13:42.817229 21386 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:13:42.889123 21438 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:42.940315 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:42.944901 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 115851us
W20260709 12:13:42.971241 12643 leader_election.cc:336] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.541s (SENT)
I20260709 12:13:42.971839 12680 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:42.904142 20827 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:13:42.972939 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 136399us
I20260709 12:13:42.881595 21182 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:42.975534 12643 leader_election.cc:336] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.593s (SENT)
I20260709 12:13:42.977420 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:42.978559 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:42.930331 21466 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:42.898741 21115 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.047s
I20260709 12:13:42.829128 21396 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:42.896055 21461 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:42.986677 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:42.900298 21307 raft_consensus.cc:697] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 20 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:13:42.993793 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 221948us
W20260709 12:13:42.995326 12643 leader_election.cc:336] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.691s (SENT)
I20260709 12:13:43.050619 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:42.993193 20913 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:13:43.010607 12571 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:42.940665 21246 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.054s
I20260709 12:13:42.993131 21215 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:13:43.021996 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:43.027087 21476 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.090s
I20260709 12:13:42.986902 21466 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:43.035562 12677 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:42.914186 21410 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:43.045097 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:43.046867 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 60264.2us
I20260709 12:13:43.048437 21496 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:13:43.058145 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 62869.1us
W20260709 12:13:43.066319 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 399788us
I20260709 12:13:43.060060 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:43.077943 13330 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:43.056445 20958 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:13:43.085964 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 256126us
I20260709 12:13:43.031929 20827 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:43.087323 12494 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:13:43.086110 12572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.086s	user 0.022s	sys 0.048s
W20260709 12:13:43.090075 12572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.090s	user 0.022s	sys 0.048s
I20260709 12:13:43.090756 12572 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:43.059393 21255 raft_consensus.cc:2764] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:13:43.088258 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:43.095250 12572 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:43.085788 13302 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:43.084789 21428 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.018s	user 0.001s	sys 0.003s
W20260709 12:13:43.164713 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 64793.6us
W20260709 12:13:43.088608 21485 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.109s	user 0.001s	sys 0.000s
I20260709 12:13:43.056938 21468 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:43.084465 21496 raft_consensus.cc:697] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:43.076783 20913 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:43.084460 21389 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.100637 21191 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:43.099050 21377 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:43.122892 21215 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.088907 21478 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.085s
I20260709 12:13:43.195700 21501 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:43.201223 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:43.202178 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:43.196949 21490 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.203s	user 0.001s	sys 0.000s
I20260709 12:13:43.168500 21194 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:13:43.254472 21194 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:13:43.199867 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 114163us
I20260709 12:13:43.208572 21495 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 12:13:43.255918 21194 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:43.203249 21182 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:43.223345 21461 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:43.215597 21501 raft_consensus.cc:697] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 23 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:43.211984 21215 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:43.235473 21415 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.767s	user 0.012s	sys 0.001s
I20260709 12:13:43.176055 21307 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 18.14, Last appended by leader: 14, Current term: 20, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:43.226218 21487 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 12:13:43.222324 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 3985) took 1060 ms. Trace:
I20260709 12:13:43.268931 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:43.269362 12600 rpcz_store.cc:276] 0709 12:13:42.162041 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.513957 (+351916us) service_pool.cc:224] Handling call
0709 12:13:42.514489 (+   532us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:42.519510 (+  5021us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:42.519562 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:42.551149 (+ 31587us) log.cc:844] Serialized 26 byte log entry
0709 12:13:42.761770 (+210621us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:13:42.761814 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:42.761886 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:43.221952 (+460066us) raft_consensus.cc:1685] finished
0709 12:13:43.221989 (+    37us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:43.222213 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":673,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":5037,"thread_start_us":140083,"threads_started":1,"wal-append.queue_time_us":300759}
I20260709 12:13:43.272441 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:43.239986 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:43.240876 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:43.222285 21488 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.142s	user 0.000s	sys 0.001s
I20260709 12:13:43.277849 13336 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:13:43.280702 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:43.280520 21501 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 21.17, Last appended by leader: 17, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:43.227838 21494 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:13:43.210412 21216 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:43.285418 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:43.286712 12597 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:43.231045 21468 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:43.199891 21466 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:43.251809 21455 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:43.201715 21389 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:43.208377 21191 raft_consensus.cc:697] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:43.226958 21438 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.226385 21497 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:43.211913 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 134176us
W20260709 12:13:43.300949 12643 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.740s (SENT) [suppressed 32 similar messages]
I20260709 12:13:43.226692 21496 consensus_queue.cc:237] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:43.258477 21477 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:13:43.302572 12643 leader_election.cc:336] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.839s (SENT)
I20260709 12:13:43.168040 21375 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:43.263763 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:43.303746 12598 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:43.260634 21182 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.308219 21433 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.274050 12673 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:13:43.290580 21377 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.264881 21190 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:43.223762 21396 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:43.280797 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:43.281769 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 183198us
I20260709 12:13:43.248047 20913 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:43.242950 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1089) took 2324 ms. Trace:
W20260709 12:13:43.287582 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:43.287716 12676 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 21: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:43.288182 13330 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:43.292287 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:43.292819 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:43.331176 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:43.331884 12597 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:43.327979 13336 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
I20260709 12:13:43.279579 21478 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:43.232164 21410 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:43.275712 12600 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:43.290482 21236 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:43.311808 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:43.315009 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:43.314332 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:43.308632 21182 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:43.308290 21191 consensus_queue.cc:237] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 19.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:43.319638 21190 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:43.333123 12673 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:13:43.278698 21415 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:43.352458 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:43.339390 12598 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:43.325654 12525 rpcz_store.cc:276] 0709 12:13:40.918360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:41.101948 (+183588us) service_pool.cc:224] Handling call
0709 12:13:41.119170 (+ 17222us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:41.181457 (+ 62287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:41.181506 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:41.203476 (+ 21970us) log.cc:844] Serialized 26 byte log entry
0709 12:13:41.587001 (+383525us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:41.587065 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:41.588426 (+  1361us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:42.764960 (+1176534us) spinlock_profiling.cc:243] Waited 6.03 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:43.242593 (+477633us) raft_consensus.cc:1685] finished
0709 12:13:43.242621 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:43.242876 (+   255us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":160909,"raft.queue_time_us":27128,"raft.run_cpu_time_us":208,"raft.run_wall_time_us":17516,"spinlock_wait_cycles":13858176,"thread_start_us":362521,"threads_started":1,"wal-append.queue_time_us":714818}
I20260709 12:13:43.310514 21500 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.344877 13338 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:43.308919 21433 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:43.356988 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:43.303292 21438 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:43.340448 21375 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:43.365789 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:43.368616 12602 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 12:13:43.375021 21491 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:43.327757 21377 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:43.309553 21497 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:43.344183 21478 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:43.349799 21236 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:43.403214 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:13:43.411022 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 108462us
I20260709 12:13:43.372532 21416 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.367136 21461 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:43.398670 21375 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:43.469767 21429 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.631s	user 0.006s	sys 0.011s
I20260709 12:13:43.371912 21500 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:43.471369 21190 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:43.471683 21478 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:43.462566 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:43.472999 12643 leader_election.cc:336] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.863s (SENT)
I20260709 12:13:43.471168 21404 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 19.12, Last appended by leader: 12, Current term: 19, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:43.482162 21416 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:43.484728 21215 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:43.488936 12600 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
W20260709 12:13:43.491909 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 94152.5us
I20260709 12:13:43.539907 21190 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.540612 21190 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:43.541759 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:43.544572 21190 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:43.546728 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:43.556049 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:43.551764 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 78810.9us
I20260709 12:13:43.556857 12569 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:43.490446 21162 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:13:43.570717 12643 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.917s (SENT)
I20260709 12:13:43.571810 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:43.528592 21434 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:13:43.593868 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3483) took 1071 ms. Trace:
I20260709 12:13:43.594514 12527 rpcz_store.cc:276] 0709 12:13:42.522441 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.522595 (+   154us) service_pool.cc:224] Handling call
0709 12:13:42.522940 (+   345us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:42.792594 (+269654us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.593405 (+800811us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b5400054454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:43.593614 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":615148416}
I20260709 12:13:43.584379 21404 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 19 FOLLOWER]: Advancing to term 20
I20260709 12:13:43.581681 21215 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:43.579686 21506 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:43.607075 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:43.608353 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:43.608724 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:43.609150 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:43.609436 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:43.612867 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:43.615370 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:43.586820 21455 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:43.624158 12531 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 21: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:43.622929 12529 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:43.623100 12522 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:43.623647 12523 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:43.627583 12671 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 23 index: 11. (index mismatch)
I20260709 12:13:43.625922 12494 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:43.622378 12525 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:13:43.615998 21182 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:43.615855 21389 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:43.646567 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 89806.5us
I20260709 12:13:43.620289 21433 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:43.651542 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:43.664919 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:43.648232 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:43.669129 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:43.679816 12671 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:13:43.681806 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:43.684513 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:43.692123 12606 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:13:43.693276 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:43.695286 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 47146.3us
I20260709 12:13:43.696811 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:43.700004 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:43.634394 21377 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:43.708539 12606 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 22: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:43.650132 21358 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:43.714908 12603 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
W20260709 12:13:43.708959 12599 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3441) took 2058 ms (client timeout 1541 ms). Trace:
W20260709 12:13:43.701810 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 141621us
W20260709 12:13:43.716988 12643 leader_election.cc:336] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.020s (SENT)
I20260709 12:13:43.717410 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:43.718914 12643 leader_election.cc:336] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.077s (SENT)
I20260709 12:13:43.719421 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:43.715878 12599 rpcz_store.cc:269] 0709 12:13:41.650187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:41.783654 (+133467us) service_pool.cc:224] Handling call
0709 12:13:43.708735 (+1925081us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b540022fbd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:43.708884 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1506090752}
I20260709 12:13:43.721961 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:43.713531 12679 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:13:43.724275 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:43.729626 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:43.771047 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 86395.1us
I20260709 12:13:43.690771 21497 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:43.690168 21433 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.775094 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:43.648435 21455 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.069s
I20260709 12:13:43.683334 21517 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:13:43.629256 21506 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:43.792991 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:43.715108 21182 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:42.476804 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:43.806401 12528 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:43.798511 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:43.801433 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:43.803524 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 15 index: 9 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:43.897430 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:43.845613 13302 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.012s	sys 0.052s
W20260709 12:13:43.898481 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 179580us
I20260709 12:13:43.822461 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:43.900568 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:43.774463 21512 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:43.899528 12527 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:13:43.908958 12529 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
W20260709 12:13:43.902740 12643 leader_election.cc:336] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.105s (SENT)
I20260709 12:13:43.913015 12643 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:43.902827 12679 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:13:43.900888 12522 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:43.905902 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:43.901785 13302 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.564s	user 0.012s	sys 0.061s
W20260709 12:13:43.881690 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:43.773532 21009 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:13:43.731722 21496 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:13:43.915589 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 578238us
I20260709 12:13:43.920696 12525 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 23: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:13:43.910871 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 213841us
I20260709 12:13:43.921281 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:43.923166 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:43.956538 12569 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:13:43.957203 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:43.930565 12643 leader_election.cc:336] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.127s (SENT)
I20260709 12:13:43.794030 21517 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:43.933225 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 18 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:43.822075 21501 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:43.824326 21182 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:43.776038 21274 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:43.956542 12529 raft_consensus.cc:2393] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 27: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:43.795099 21236 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:43.966521 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:43.793687 21377 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:43.995345 13333 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 1038) took 2839 ms. Trace:
I20260709 12:13:43.995766 21377 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:43.996560 21519 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.202s	user 0.001s	sys 0.000s
I20260709 12:13:43.997787 21516 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:43.972831 21455 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:43.998782 21455 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:43.998389 21516 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:43.999169 21455 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:44.000804 21516 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:43.938987 21500 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:43.995826 13333 rpcz_store.cc:276] 0709 12:13:41.155838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:41.155987 (+   149us) service_pool.cc:224] Handling call
0709 12:13:42.337938 (+1181951us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:42.362300 (+ 24362us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:43.988983 (+1626683us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b54004439d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:43.995242 (+  6259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2675397504}
I20260709 12:13:43.989677 21517 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:13:44.005177 12680 raft_consensus.cc:2410] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 21 because replica has last-logged OpId of term: 19 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 11.
I20260709 12:13:43.791081 21389 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:13:43.979512 12601 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3977) took 2095 ms (client timeout 1802 ms). Trace:
I20260709 12:13:43.923812 21512 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:43.972105 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:43.971191 12522 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:43.975574 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:44.019649 21236 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:13:44.019528 21516 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.023049 21516 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:44.014415 20958 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.026659 20958 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:44.027004 21516 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:43.995729 21509 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.301s	user 0.001s	sys 0.000s
I20260709 12:13:43.981178 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 19 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:43.971091 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:43.982347 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:43.940836 21497 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:44.037709 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 68172.9us
I20260709 12:13:44.002197 21524 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:43.778469 21433 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:43.991420 21533 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.004s
I20260709 12:13:43.936856 21461 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:13:44.013275 12601 rpcz_store.cc:269] 0709 12:13:41.884360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:41.925388 (+ 41028us) service_pool.cc:224] Handling call
0709 12:13:43.979246 (+2053858us) spinlock_profiling.cc:243] Waited 351 ms on lock 0x7b5400374954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:43.979398 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":806712064}
I20260709 12:13:44.051295 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:44.005909 21383 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.040s
I20260709 12:13:43.930279 21416 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:44.034322 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 310160us
I20260709 12:13:43.970256 21496 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:43.993168 21182 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:13:44.042956 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 86557.1us
I20260709 12:13:43.985181 12528 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:13:44.044742 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:44.015163 21377 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:44.052450 12671 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:44.067965 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:44.052580 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:44.071013 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.068562 12599 raft_consensus.cc:2376] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23. Re-sending same reply.
I20260709 12:13:44.072907 13332 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:44.067553 12572 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:44.073967 12569 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:44.007156 21501 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:44.101624 12569 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:44.108404 21516 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:13:44.109745 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:44.113271 21461 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:44.114449 21461 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:44.116638 21506 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:44.116884 21461 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:44.117442 21506 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:44.075906 21162 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.502s	user 0.007s	sys 0.006s
I20260709 12:13:44.118023 21506 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:44.069423 21507 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:13:44.054430 21389 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election lost for term 19. Reason: could not achieve majority
I20260709 12:13:44.056354 21405 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:44.103314 12572 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:44.060760 21383 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:13:44.153461 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 223319us
I20260709 12:13:44.065277 21433 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:44.099895 21102 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:44.145347 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:44.046077 21497 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:44.160652 12643 leader_election.cc:336] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.218s (SENT)
I20260709 12:13:44.161249 12643 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:44.106559 21191 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:44.149854 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:44.164923 12601 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 24: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:13:44.107257 21375 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.619s	user 0.004s	sys 0.008s
I20260709 12:13:44.076793 21399 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.080s
I20260709 12:13:44.105486 21377 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:44.161435 12602 raft_consensus.cc:2368] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23.
I20260709 12:13:44.140837 21533 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:44.144191 21461 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.184414 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 990) took 2084 ms. Trace:
I20260709 12:13:44.184862 12604 rpcz_store.cc:276] 0709 12:13:42.099731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.379338 (+279607us) service_pool.cc:224] Handling call
0709 12:13:42.399558 (+ 20220us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:42.830681 (+431123us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.184091 (+1353410us) spinlock_profiling.cc:243] Waited 430 ms on lock 0x7b54002e68d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:44.184341 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":989094144}
I20260709 12:13:44.163281 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:44.162338 12496 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:44.188661 12671 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:44.189774 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:44.191795 21518 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
I20260709 12:13:44.199290 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:44.200577 12604 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 22: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:44.201706 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:44.202733 12604 raft_consensus.cc:2410] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 26 because replica has last-logged OpId of term: 24 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 11.
I20260709 12:13:44.206576 21496 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:44.208516 21518 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.001s
I20260709 12:13:44.212451 21501 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:44.217463 21544 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:44.170403 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3487) took 1623 ms. Trace:
I20260709 12:13:44.236995 12524 rpcz_store.cc:276] 0709 12:13:42.547342 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.547553 (+   211us) service_pool.cc:224] Handling call
0709 12:13:44.170166 (+1622613us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54003f7d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:44.170330 (+   164us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":377542400}
I20260709 12:13:44.171027 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:44.238621 12674 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
W20260709 12:13:44.159523 21529 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.234s	user 0.001s	sys 0.000s
I20260709 12:13:44.070815 21524 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:44.239717 21455 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:44.242156 21524 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:44.177006 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:44.179556 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:44.243039 12679 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:44.243379 12601 raft_consensus.cc:2393] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:44.144662 21465 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.244201 21465 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:44.244750 12601 raft_consensus.cc:1217] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Deduplicated request from leader. Original: 18.11->[19.12-19.12]   Dedup: 19.12->[]
I20260709 12:13:44.181529 21521 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.245337 12601 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 19. Current term is 20. Ops: [19.12-19.12]
I20260709 12:13:44.180406 21389 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:13:44.246235 21506 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:44.182874 21533 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:44.184108 21461 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:44.185956 21236 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:44.156687 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 53501.1us
I20260709 12:13:44.187966 12673 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 24 index: 18. (index mismatch)
I20260709 12:13:44.186570 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:44.256819 12676 raft_consensus.cc:2410] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 26 because replica has last-logged OpId of term: 23 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:13:44.222805 21512 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:44.258353 12572 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:44.229955 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3438) took 2580 ms. Trace:
I20260709 12:13:44.259634 12603 rpcz_store.cc:276] 0709 12:13:41.649378 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.230825 (+581447us) service_pool.cc:224] Handling call
0709 12:13:43.709026 (+1478201us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:43.715432 (+  6406us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.229702 (+514270us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b540022fbd0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:44.229874 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3220964736}
I20260709 12:13:44.261186 12603 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:44.151831 21383 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:44.181989 21503 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:13:44.167569 21507 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:44.168720 21102 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:44.235769 21520 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:13:44.237129 21404 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:13:44.183223 21523 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 12:13:44.245954 21521 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:44.272802 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3490) took 1627 ms. Trace:
I20260709 12:13:44.273288 12526 rpcz_store.cc:276] 0709 12:13:42.645929 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.647773 (+  1844us) service_pool.cc:224] Handling call
0709 12:13:42.648330 (+   557us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:42.686785 (+ 38455us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:13:42.686835 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:42.703552 (+ 16717us) log.cc:844] Serialized 26 byte log entry
0709 12:13:42.841411 (+137859us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:13:42.841471 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.268558 (+1427087us) spinlock_profiling.cc:243] Waited 33.8 ms on lock 0x7b540038ead4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:44.268764 (+   206us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:44.268824 (+    60us) raft_consensus.cc:1685] finished
0709 12:13:44.268848 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:44.272728 (+  3880us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4111,"raft.queue_time_us":4293,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":12174,"spinlock_wait_cycles":77664512,"thread_start_us":108016,"threads_started":1,"wal-append.queue_time_us":261751}
I20260709 12:13:44.274878 21521 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:44.176348 21433 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:44.275993 12604 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 21 index: 15. Preceding OpId from leader: term: 23 index: 16. (index mismatch)
I20260709 12:13:44.275858 21433 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:44.276132 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:44.168473 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:44.254997 21236 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:13:44.264380 12605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 3983) took 2109 ms (client timeout 2535 ms). Trace:
I20260709 12:13:44.271752 21383 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:44.229712 21497 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:44.280999 12598 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 26: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:13:44.173770 21375 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.682s	user 0.005s	sys 0.009s
I20260709 12:13:44.281062 12599 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 12:13:44.280660 12605 rpcz_store.cc:269] 0709 12:13:42.155413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.209419 (+ 54006us) service_pool.cc:224] Handling call
0709 12:13:44.264169 (+2054750us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b54002e63d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:44.264307 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":603722496}
I20260709 12:13:44.166169 21191 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:44.283213 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:44.253950 21461 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:44.292506 12524 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:13:44.302309 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:44.302980 12599 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 28.
I20260709 12:13:44.299206 12598 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24. Ops: []
W20260709 12:13:44.162652 21434 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.563s	user 0.005s	sys 0.010s
I20260709 12:13:44.269603 21512 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:44.302781 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:44.296064 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:44.305665 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 44529.1us
I20260709 12:13:44.310302 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:44.278075 21433 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:44.301548 12571 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:44.310648 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:44.311286 12524 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:13:44.285338 21507 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:44.308192 21191 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:44.349895 21412 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.367036 21412 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:44.368757 12525 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:13:44.342743 21102 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:44.373943 20958 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:44.284083 21533 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:44.292017 21383 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:44.283814 21542 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.403642 12603 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:13:44.283267 21521 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.294672 21547 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:44.371358 21412 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:44.306334 21434 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.717s	user 0.005s	sys 0.010s
I20260709 12:13:44.412956 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:44.413261 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 111805us
I20260709 12:13:44.414881 12571 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:44.410355 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 181207us
I20260709 12:13:44.304775 21377 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:44.409111 12646 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:44.352655 21236 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
W20260709 12:13:44.435952 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 245077us
W20260709 12:13:44.450742 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 348872us
W20260709 12:13:44.452600 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 43554.2us
I20260709 12:13:44.449893 12496 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:44.474763 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:44.475548 12677 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:44.476172 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 22 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:44.435019 21532 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:44.478658 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:44.477373 12676 raft_consensus.cc:2410] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 26 because replica has last-logged OpId of term: 24 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 11.
I20260709 12:13:44.480438 12673 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 12:13:44.481472 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 46393.8us
I20260709 12:13:44.348850 21517 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:44.431573 21434 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:44.472344 21377 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:44.479359 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:44.418316 21389 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.052s
W20260709 12:13:44.484175 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 69638.8us
I20260709 12:13:44.473764 21455 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:44.477558 20958 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:44.481228 21532 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:44.419184 21521 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:44.495395 12571 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:44.504004 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:44.499993 12597 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 29. Current term is 30. Ops: []
I20260709 12:13:44.411796 21542 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:44.489584 21236 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:44.504657 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:44.476979 21545 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:13:44.508129 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.492081 20827 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:44.496196 21377 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:44.519863 21554 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:13:44.509845 21492 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.011s
I20260709 12:13:44.522162 21236 raft_consensus.cc:697] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:44.523255 21236 consensus_queue.cc:237] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.11, Last appended by leader: 11, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:44.526711 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 214439us
I20260709 12:13:44.516522 21545 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:44.548043 21545 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:44.509629 21433 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:44.549368 21433 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:44.549842 21433 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:44.507242 21532 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:44.559657 21517 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:44.498862 20958 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:44.562500 20958 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:44.563619 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:44.564394 13335 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:13:44.434446 21547 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:44.569639 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:44.570488 12527 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
W20260709 12:13:44.579687 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.582248 12571 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:44.584705 21517 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:13:44.584929 21543 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.176s	user 0.003s	sys 0.000s
I20260709 12:13:44.585261 21517 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:44.585893 21517 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:13:44.596376 12675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3492) took 2345 ms (client timeout 2149 ms). Trace:
I20260709 12:13:44.596043 21434 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:44.596838 12675 rpcz_store.cc:269] 0709 12:13:42.251265 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:42.251438 (+   173us) service_pool.cc:224] Handling call
0709 12:13:44.596121 (+2344683us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b54000f00d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:44.596266 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1453227904}
I20260709 12:13:44.598860 21194 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:13:44.599493 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.608058 21449 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:13:44.608572 21449 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:13:44.633388 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:44.642325 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 44809us
I20260709 12:13:44.643570 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:44.647162 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:44.642488 12675 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:13:44.609493 21449 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:44.654731 12675 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:13:44.664057 12673 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:13:44.675761 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:44.690265 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:44.693656 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:44.695436 12496 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:44.695572 12675 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:44.712929 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:44.651248 21455 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:44.667896 21307 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:13:44.716578 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:44.715042 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:44.724459 12674 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:44.729480 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:44.687942 21375 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.667691 21478 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 30.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.058s
I20260709 12:13:44.713889 21306 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.781169 12675 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
W20260709 12:13:44.771628 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 292367us
I20260709 12:13:44.720355 21396 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:44.719326 21449 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:44.673599 21550 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.057s
I20260709 12:13:44.804569 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:44.725344 21194 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.734503 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:44.726552 21307 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:44.808339 12571 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:44.813818 12674 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:13:44.818897 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:44.825464 12675 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:13:44.826527 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:44.829823 12674 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:13:44.827263 12675 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:44.725098 20799 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.089s
I20260709 12:13:44.841755 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:44.866737 12674 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:44.805924 21396 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:44.834978 21307 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:44.780783 21375 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:44.882175 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 88009.7us
I20260709 12:13:44.898619 12674 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:44.808837 21306 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:44.731264 21546 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:44.879284 21396 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:44.888049 21465 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:44.920622 13335 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
W20260709 12:13:44.938551 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.940577 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:44.941447 12675 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:44.937239 12496 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:44.945869 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:44.947415 12496 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:44.914692 21524 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:44.966533 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:44.842964 21410 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:44.926201 21516 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:44.963519 12643 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:44.945989 21405 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.794s	user 0.008s	sys 0.007s
I20260709 12:13:44.842170 21542 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:44.836059 21194 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:44.926462 21411 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.047s
W20260709 12:13:44.971383 21191 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.606s	user 0.009s	sys 0.004s
W20260709 12:13:44.971560 21182 consensus_meta.cc:287] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.871s	user 0.015s	sys 0.001s
I20260709 12:13:44.968641 21330 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:13:44.981665 12678 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:44.970014 21507 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:13:44.988467 12643 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:44.972429 21255 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.972098 21521 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:44.963035 21465 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 12:13:44.931039 21546 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:44.975103 20827 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:44.976660 21516 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:44.993829 21215 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:44.979185 21554 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:45.006908 21527 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.190s
I20260709 12:13:44.962733 21183 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.041s
I20260709 12:13:44.971350 21545 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:44.973073 21524 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:44.985249 21486 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:44.993534 21330 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:45.033720 20827 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.033941 21377 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:45.036605 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:45.027994 21557 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.136s	user 0.001s	sys 0.000s
I20260709 12:13:45.035400 21236 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:45.061827 12674 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
W20260709 12:13:45.071022 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 82657.3us
W20260709 12:13:45.064983 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 256742us
I20260709 12:13:44.993068 21191 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:45.035666 21516 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:45.074601 21330 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:45.092133 12643 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:45.059053 21405 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:44.996945 21182 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.060110 21579 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.031744 21461 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:13:45.096832 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:45.028944 21255 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.106503 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.063776 21215 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.133219 12675 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
W20260709 12:13:45.124073 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:45.108371 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:45.075189 21486 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:45.102934 21559 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.259s	user 0.001s	sys 0.001s
I20260709 12:13:45.119315 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:45.151116 12531 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 23. Current term is 25. Ops: []
I20260709 12:13:45.148424 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:45.165431 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.166438 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.167618 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:45.097878 21236 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.111842 21554 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:13:45.151706 21558 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.073s	user 0.000s	sys 0.001s
I20260709 12:13:45.174680 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.180186 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.135203 21434 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:45.182753 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 68749.3us
I20260709 12:13:45.181929 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:45.141042 21306 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:45.165088 21561 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.236s	user 0.000s	sys 0.001s
I20260709 12:13:45.144847 21574 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.114312 21405 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:45.176791 21461 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:45.156926 21389 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:45.134717 21579 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:45.156620 21576 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:45.200935 21568 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:13:45.201107 21524 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:45.201676 21565 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.181s	user 0.000s	sys 0.001s
W20260709 12:13:45.202493 21567 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.102s	user 0.001s	sys 0.000s
I20260709 12:13:45.239720 21450 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:45.239871 21579 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:45.203564 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.193346 21434 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.204515 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.207366 21415 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:45.208146 21236 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:45.210700 21545 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:45.211163 21578 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:13:45.214336 21410 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:13:45.217432 21532 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.659s	user 0.008s	sys 0.006s
I20260709 12:13:45.190426 21562 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:13:45.219444 21546 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:45.219472 20827 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:45.221673 21517 consensus_meta.cc:287] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.635s	user 0.009s	sys 0.004s
I20260709 12:13:45.282233 21517 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.284415 21517 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:45.284591 12672 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:45.228581 21307 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:45.287297 21307 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:45.194604 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.289197 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:45.290990 21009 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.291852 21009 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.301389 21330 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:45.236881 20958 consensus_meta.cc:287] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.674s	user 0.012s	sys 0.000s
I20260709 12:13:45.227051 21389 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:45.239400 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1000) took 1703 ms. Trace:
I20260709 12:13:45.266919 21461 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:45.229830 21563 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.149s	user 0.000s	sys 0.001s
I20260709 12:13:45.249557 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3515) took 1560 ms. Trace:
I20260709 12:13:45.228232 21577 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.319193 12680 rpcz_store.cc:276] 0709 12:13:43.681345 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.681528 (+   183us) service_pool.cc:224] Handling call
0709 12:13:45.241118 (+1559590us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b54003d4ad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:45.241267 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":740848384}
W20260709 12:13:45.311187 21433 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.761s	user 0.013s	sys 0.001s
I20260709 12:13:45.250855 21377 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:45.242059 21576 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:45.251565 21434 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:45.253639 21564 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.277s	user 0.000s	sys 0.002s
I20260709 12:13:45.254035 21396 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:45.263207 21405 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:45.256350 21507 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:45.273841 12572 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:45.274950 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:45.331573 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 57805.3us
I20260709 12:13:45.314702 20958 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.280702 21410 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:45.332444 21576 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:45.333141 21576 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:45.333467 21194 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:45.334215 20958 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:45.335109 12671 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 26: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:45.280928 21532 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.337743 21532 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:45.304652 21383 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.896s	user 0.012s	sys 0.004s
I20260709 12:13:45.339278 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.198920 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:45.314965 12600 rpcz_store.cc:276] 0709 12:13:43.535546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.557867 (+ 22321us) service_pool.cc:224] Handling call
0709 12:13:43.558219 (+   352us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:44.537898 (+979679us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b18004e8800. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f1488a18060 00007f1488a18bfb 00007f1488a2364b
0709 12:13:44.538055 (+   157us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.538086 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:44.540946 (+  2860us) log.cc:844] Serialized 26 byte log entry
0709 12:13:44.825755 (+284809us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:13:44.825813 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.825913 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:45.239101 (+413188us) raft_consensus.cc:1685] finished
0709 12:13:45.239121 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:45.239330 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":70236,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":12304,"spinlock_wait_cycles":47815552,"thread_start_us":385653,"threads_started":2,"wal-append.queue_time_us":421224}
I20260709 12:13:45.241657 12677 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 20 index: 13. (index mismatch)
I20260709 12:13:45.250654 21182 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:45.317520 21389 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:13:45.321184 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 153641us
I20260709 12:13:45.344578 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:45.346550 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:45.347723 12643 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:45.348415 20958 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:45.349336 20958 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:45.350759 12643 leader_election.cc:400] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23.
I20260709 12:13:45.350584 21562 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:13:45.351132 12643 leader_election.cc:403] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Cancelling election due to peer responding with higher term
I20260709 12:13:45.352209 21562 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:13:45.353078 21562 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:13:45.354303 21587 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election lost for term 22. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23.
I20260709 12:13:45.354839 21562 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:45.355257 21562 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:45.355422 21383 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.356055 21562 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.321413 21577 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.357277 21562 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:45.325740 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.332309 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.332334 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:45.358880 12569 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:45.228660 21574 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:45.311453 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:45.332362 21396 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:45.350584 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.341504 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:45.350057 20958 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:13:45.363696 12572 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:45.357641 21410 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:45.364923 12525 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:13:45.377717 21404 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:45.383728 21404 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.365136 12530 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:45.364874 12522 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 29: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:45.366670 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.341049 21507 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:45.367991 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:45.368345 21577 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:45.368462 21455 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:45.356061 21383 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:45.322335 21433 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.368396 21434 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election won for term 30
W20260709 12:13:45.370050 21560 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.438s	user 0.000s	sys 0.004s
I20260709 12:13:45.382290 21377 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:45.370723 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:45.373385 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:45.365559 21468 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:45.365690 21405 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:45.408336 21405 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.374183 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:45.376668 12531 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 18 index: 10. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:13:45.409488 13338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 976) took 1336 ms. Trace:
I20260709 12:13:45.410701 21405 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:45.365947 21450 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.411531 21433 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:45.411664 21450 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:45.412292 21434 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:45.320160 21461 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.378571 21554 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:45.410187 13338 rpcz_store.cc:276] 0709 12:13:44.073506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.073642 (+   136us) service_pool.cc:224] Handling call
0709 12:13:44.102942 (+ 29300us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:44.279234 (+176292us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.279285 (+    51us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:44.285647 (+  6362us) log.cc:844] Serialized 26 byte log entry
0709 12:13:44.670945 (+385298us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:13:44.670993 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:44.671074 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:45.369471 (+698397us) spinlock_profiling.cc:243] Waited 9.81 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:45.409134 (+ 39663us) raft_consensus.cc:1685] finished
0709 12:13:45.409168 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:45.409403 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1253,"raft.queue_time_us":80738,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"spinlock_wait_cycles":22554880,"thread_start_us":437677,"threads_started":2,"wal-append.queue_time_us":568151}
I20260709 12:13:45.378571 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:45.417022 21554 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.417846 13339 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:45.379391 21587 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:45.381001 21555 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.446s	user 0.000s	sys 0.001s
I20260709 12:13:45.360428 21449 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.398911 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4014) took 1354 ms. Trace:
I20260709 12:13:45.427096 21449 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:45.427039 12604 rpcz_store.cc:276] 0709 12:13:44.044789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.278389 (+233600us) service_pool.cc:224] Handling call
0709 12:13:44.278846 (+   457us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.310333 (+ 31487us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:44.310366 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:44.310409 (+    43us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:44.310441 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:45.398664 (+1088223us) spinlock_profiling.cc:243] Waited 31.3 ms on lock 0x7b5400079c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:45.398717 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:45.398849 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":71939456}
I20260709 12:13:45.369910 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:45.408475 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:45.378376 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:45.416121 21450 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:45.394383 21404 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:45.418417 21461 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:45.425788 21383 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:45.422995 21587 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.422636 21554 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:45.439299 21468 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:13:45.400718 21455 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:45.413280 21434 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:45.442977 13332 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:45.426642 21330 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:45.431778 21449 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:45.434942 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 56684.7us
I20260709 12:13:45.435266 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3396) took 1635 ms. Trace:
I20260709 12:13:45.448208 12523 rpcz_store.cc:276] 0709 12:13:43.799588 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.821374 (+ 21786us) service_pool.cc:224] Handling call
0709 12:13:43.821743 (+   369us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:43.925081 (+103338us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:43.925133 (+    52us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:43.925199 (+    66us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:13:43.925256 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:45.435030 (+1509774us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b54003c0354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:45.435080 (+    50us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:45.435206 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":236922752}
I20260709 12:13:45.437104 21438 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:45.437415 21587 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:45.444523 12569 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:45.406636 20958 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:45.441970 21516 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.449908 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:45.453604 21574 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:45.454874 21516 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:45.455163 21410 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:45.461285 21009 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:45.453490 20958 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:45.450349 12643 connection.cc:256] RPC call timeout handler was delayed by -1.0674965349999184s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1632s for a response
I20260709 12:13:45.466929 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:45.468969 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:45.469538 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:45.471068 12678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3509) took 2231 ms (client timeout 1730 ms). Trace:
W20260709 12:13:45.471510 12678 rpcz_store.cc:269] 0709 12:13:43.239508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.240576 (+  1068us) service_pool.cc:224] Handling call
0709 12:13:45.470888 (+2230312us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b540015e0d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:45.470998 (+   110us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3925846400}
I20260709 12:13:45.471849 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:45.472913 12643 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:45.469892 21466 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:45.473778 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:45.451711 21389 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.473392 21516 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:45.491837 21516 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:45.492811 21516 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:45.494203 21389 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:45.495625 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 532023us
I20260709 12:13:45.496912 21373 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:13:45.497778 12496 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:45.498705 12496 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:13:45.482898 12606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3460) took 1829 ms (client timeout 2086 ms). Trace:
W20260709 12:13:45.503276 12606 rpcz_store.cc:269] 0709 12:13:43.653119 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.773433 (+120314us) service_pool.cc:224] Handling call
0709 12:13:45.482642 (+1709209us) spinlock_profiling.cc:243] Waited 1.3 s on lock 0x7b5400375d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:45.482823 (+   181us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2992540032}
W20260709 12:13:45.483461 12672 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 4181) took 2210 ms (client timeout 1715 ms). Trace:
W20260709 12:13:45.504782 12672 rpcz_store.cc:269] 0709 12:13:43.272793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:43.272963 (+   170us) service_pool.cc:224] Handling call
0709 12:13:45.483273 (+2210310us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b54002c5454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:45.483386 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4500574464}
I20260709 12:13:45.474303 21587 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:45.512231 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:45.517710 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1101) took 1233 ms. Trace:
I20260709 12:13:45.518316 12528 rpcz_store.cc:276] 0709 12:13:44.284534 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:44.284664 (+   130us) service_pool.cc:224] Handling call
0709 12:13:44.284907 (+   243us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:44.285099 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:13:44.656984 (+371885us) log.cc:844] Serialized 14 byte log entry
0709 12:13:45.517236 (+860252us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:45.517309 (+    73us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:13:45.517385 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:45.517481 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:45.517633 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":457150,"prepare.queue_time_us":233784,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"thread_start_us":714124,"threads_started":2,"wal-append.queue_time_us":450991}
I20260709 12:13:45.519639 21358 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:45.486284 21255 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:45.520202 21587 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:45.520609 21587 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:45.475411 21249 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:13:45.520984 21587 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:45.521323 21249 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:45.521663 21358 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:45.522377 21249 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:45.523427 21249 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:13:45.528200 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:45.498973 21373 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:45.533941 12673 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
W20260709 12:13:45.535193 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:45.477056 21215 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:45.550205 12601 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 19 index: 16. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:13:45.489773 21592 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:45.568462 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.539347 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:45.569453 12523 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:13:45.569605 12675 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26.
I20260709 12:13:45.538420 12674 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:13:45.546276 12606 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:13:45.535414 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:45.479050 21375 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:45.476958 20958 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:45.571558 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:45.570775 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:45.474779 21410 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:13:45.572820 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:45.573164 12416 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 70b8c9b89e3d43fbb70422110d76702c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica a2fdbe7eef6748a9b4611f6c9bc9b201 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 22 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:13:45.552522 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:45.573560 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:45.560444 12604 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 30. Current term is 31. Ops: []
I20260709 12:13:45.561054 12679 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:13:45.562541 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:45.565327 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 66684.5us
I20260709 12:13:45.561252 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:45.565951 21387 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:45.525872 21191 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:45.572280 12674 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:45.515287 21009 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:45.574172 12601 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:45.570000 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:45.575181 13332 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:13:45.566596 21465 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.559038 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:45.571873 12678 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:45.585776 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:45.572880 12675 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:45.575763 12524 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 28 index: 21. (index mismatch)
I20260709 12:13:45.577406 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 20 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.589229 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.590270 12603 raft_consensus.cc:2410] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 21 because replica has last-logged OpId of term: 20 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:13:45.580682 21387 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:45.578482 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.591686 21387 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:45.592391 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:45.579826 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.593647 12597 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 26. Current term is 27. Ops: []
I20260709 12:13:45.592393 12599 raft_consensus.cc:2410] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 26 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 14.
I20260709 12:13:45.582719 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:45.551450 21373 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:45.579178 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.527746 21358 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:45.558238 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:45.598790 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:45.599355 12598 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:45.579473 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.594296 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:45.595278 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:45.600682 12528 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:45.595968 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.530788 21547 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:45.602854 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.600709 12674 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:45.603825 12602 raft_consensus.cc:2393] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 24: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:45.604601 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:45.479461 21507 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:45.602566 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.596983 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.593068 12673 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:45.606106 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.609099 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.605893 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:45.614166 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.614889 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:45.615501 12673 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:45.613204 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:45.616324 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:45.616511 12674 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:13:45.618211 12674 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 23 index: 18. (index mismatch)
I20260709 12:13:45.613487 12602 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:45.620012 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.620775 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.620980 12599 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:45.622659 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.624398 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.624604 12572 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:45.625552 12571 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:45.625447 12599 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:13:45.627022 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.627723 12599 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:45.628861 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.589731 21465 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.631058 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.632490 12606 raft_consensus.cc:2393] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 31: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:45.638973 12598 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:13:45.639999 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.640578 12598 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:45.641655 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.643967 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.646014 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.646831 12530 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:45.650502 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:45.643157 12598 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:45.621387 12602 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 20.
I20260709 12:13:45.659341 12602 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:45.657795 12523 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 28. Current term is 29. Ops: []
I20260709 12:13:45.602617 21373 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:45.603222 21306 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:45.663095 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:45.660578 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 19 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:45.660609 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.633123 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:45.666472 12529 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 28. Current term is 29. Ops: []
I20260709 12:13:45.656773 21191 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:45.682582 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.588243 21587 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:45.684260 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 51340.7us
I20260709 12:13:45.670204 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:45.686074 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 20 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:45.664908 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:45.688072 12522 raft_consensus.cc:2393] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 27: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:45.607684 21410 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:45.680375 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:45.681012 12673 raft_consensus.cc:2410] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 24 because replica has last-logged OpId of term: 21 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 16.
W20260709 12:13:45.671494 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 75980.6us
I20260709 12:13:45.693833 12598 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: [23.11-23.11]
I20260709 12:13:45.694947 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.696201 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.700506 12530 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:45.703076 12643 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:45.703908 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.697834 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:45.631254 21255 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:13:45.702292 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:45.724642 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.596369 21532 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.066s
I20260709 12:13:45.733103 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:45.725674 12524 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:45.731698 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:45.738238 12598 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 26: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:45.670239 21586 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:45.606823 21534 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.092s
I20260709 12:13:45.612617 21375 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.086s
I20260709 12:13:45.719231 12528 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:13:45.716746 12601 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 29.
I20260709 12:13:45.679515 21399 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:45.744168 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.772375 12598 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 29: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:45.741135 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.773221 12524 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:45.746871 12677 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:45.681991 21546 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.682085 21191 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:45.753726 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.728971 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:45.770196 13305 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:45.776517 12601 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:13:45.776543 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.777735 12598 raft_consensus.cc:2393] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 22: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:45.778025 12601 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:13:45.780261 12601 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
W20260709 12:13:45.781689 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52825.3us
I20260709 12:13:45.781718 12598 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: []
I20260709 12:13:45.782809 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:45.784998 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:45.786201 12527 raft_consensus.cc:2410] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 29 because replica has last-logged OpId of term: 26 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 19.
I20260709 12:13:45.696770 21330 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.042s
I20260709 12:13:45.790935 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:45.792162 12678 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:13:45.733991 21562 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:45.807435 12525 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:13:45.807775 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:45.657167 21539 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 9, Time since last communication: 0.118s
I20260709 12:13:45.682482 21528 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.095s
I20260709 12:13:45.812474 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:45.697393 21383 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.119s
I20260709 12:13:45.789136 21191 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:45.835868 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:45.837706 12571 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:45.842280 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:45.867317 12673 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:45.843391 12604 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:13:45.843523 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:45.748175 21513 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.083s
I20260709 12:13:45.845505 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:45.805778 21389 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:45.871961 12571 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:45.873221 13332 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:13:45.873414 12571 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:45.848158 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:45.776535 21527 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.062s
I20260709 12:13:45.829597 21403 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:45.862723 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:45.815084 21009 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:45.865875 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:45.875536 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:45.834455 21477 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:45.880142 12571 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:45.797535 21546 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:45.926882 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52081.9us
I20260709 12:13:45.890630 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:45.892081 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:45.927783 12571 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:45.928189 12604 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:45.850725 21449 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:45.929170 12571 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:45.835052 21455 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.930732 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:45.875319 21586 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:45.931854 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:45.861122 21383 consensus_queue.cc:1243] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 25.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:13:45.938633 12643 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:45.932907 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:45.875854 20958 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 20. Reason: could not achieve majority
I20260709 12:13:45.935526 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:45.939610 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:45.942289 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:45.943166 12671 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:45.924671 21562 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:45.957314 21562 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:45.957749 21562 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:45.872553 21525 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.087s
I20260709 12:13:45.936776 21468 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:45.966187 21468 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:45.966630 21468 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
W20260709 12:13:45.966722 21595 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.186s	user 0.001s	sys 0.000s
I20260709 12:13:45.934453 21410 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:45.970885 21410 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:45.971058 21594 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.136s	user 0.000s	sys 0.000s
I20260709 12:13:45.971390 21410 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:45.973163 21597 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:13:45.974692 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:45.975801 21455 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:45.976008 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:45.976938 12680 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
W20260709 12:13:45.978242 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:45.979749 21009 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:13:45.980352 21009 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:45.981483 21546 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:45.981299 21009 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:45.879937 21574 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:45.951876 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:45.940136 21576 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:13:45.954789 21434 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:45.954928 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:45.995581 12673 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:13:45.950143 21591 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:45.996213 13339 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:45.955849 21587 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:45.936993 21586 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:45.921787 12427 catalog_manager.cc:5697] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 8 to 16, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 16 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:13:45.923440 21477 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:45.976052 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:45.950644 21389 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:45.977900 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:46.002534 12680 raft_consensus.cc:2410] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 29 because replica has last-logged OpId of term: 27 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 14.
I20260709 12:13:46.005584 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 18 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:46.006816 12680 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 22: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:45.988456 12675 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
I20260709 12:13:46.028435 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:45.989082 12671 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:45.992427 12679 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:13:45.987798 12678 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:45.898437 21486 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:45.995297 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:45.937546 21403 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:45.990300 21478 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.009s
I20260709 12:13:46.000600 21586 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:46.007787 21576 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:45.990234 21574 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:46.010485 21387 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:46.014318 21465 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:46.016764 21375 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:13:46.024736 21249 consensus_meta.cc:287] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.501s	user 0.009s	sys 0.002s
I20260709 12:13:45.997772 21434 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:46.001561 12604 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 29: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:46.035058 13332 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:45.950040 21433 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:45.998786 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:46.041716 13339 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:45.997603 21591 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:46.079602 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 308663us
I20260709 12:13:46.004271 21477 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:46.003988 21389 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:46.075822 21403 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:46.117774 12494 connection.cc:256] RPC call timeout handler was delayed by -1.261496717s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2535s for a response
I20260709 12:13:46.119563 12529 raft_consensus.cc:2393] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 22: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:46.093041 21574 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:46.125312 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:46.115751 21434 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:46.121273 21389 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:46.119215 21591 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:46.136688 21562 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:46.123274 21403 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:13:46.125967 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 186420us
I20260709 12:13:46.142349 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:46.143983 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:46.144800 21554 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:13:46.113173 21375 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:46.158027 21562 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:46.158871 21562 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:46.159359 21562 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:46.161067 12496 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:46.162070 12496 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:46.163544 12496 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:46.166313 21547 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:13:46.166837 21547 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:46.167695 21547 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:46.169629 21534 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:46.170985 21534 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:46.171509 21534 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:46.172451 21534 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:46.174544 21607 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:13:46.176362 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:46.121428 21433 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:46.128417 12494 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:46.181507 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:46.150683 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:46.189524 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 61173.7us
W20260709 12:13:46.193379 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 241595us
I20260709 12:13:46.194175 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:46.176594 21562 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:46.193068 21099 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:46.191277 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:46.184707 21546 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:13:46.207194 12603 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s	user 0.006s	sys 0.016s
W20260709 12:13:46.207628 12603 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s	user 0.006s	sys 0.016s
W20260709 12:13:46.194780 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50865us
I20260709 12:13:46.222981 12643 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:46.229219 21562 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:13:46.233927 12605 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.622s	user 0.004s	sys 0.025s
W20260709 12:13:46.237537 21358 consensus_meta.cc:287] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.625s	user 0.008s	sys 0.003s
I20260709 12:13:46.262984 12598 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:13:46.269474 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:46.213208 21396 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:46.269284 21516 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:46.234987 21099 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.043s
I20260709 12:13:46.271271 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:46.247506 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:46.293613 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:46.289687 12569 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:46.286715 21596 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.307s	user 0.001s	sys 0.000s
W20260709 12:13:46.291155 21603 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.282s	user 0.002s	sys 0.000s
W20260709 12:13:46.321803 12605 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.004s	sys 0.025s
W20260709 12:13:46.285404 21606 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.276s	user 0.003s	sys 0.000s
I20260709 12:13:46.235519 21546 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:13:46.328459 13339 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 31.
I20260709 12:13:46.246721 21545 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.073s
I20260709 12:13:46.273617 21433 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:46.333237 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 86237.1us
I20260709 12:13:46.306557 21387 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:46.333758 21532 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:46.354071 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:46.273986 21330 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 21.10, Next index: 11, Last known committed idx: 10, Time since last communication: 0.061s
I20260709 12:13:46.361169 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:46.336462 21562 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:46.363777 12569 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:46.367283 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:46.372148 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:13:46.378685 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:46.331743 21358 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:46.365298 21532 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:46.384058 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 313344us
I20260709 12:13:46.394191 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:46.387888 13339 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:46.366008 21009 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:46.340143 21468 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:46.375006 21592 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:46.369750 21562 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:46.378186 21576 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election won for term 32
W20260709 12:13:46.400239 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 218857us
I20260709 12:13:46.391148 21449 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:46.408300 12600 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:13:46.380779 20913 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:13:46.415230 12529 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:46.395964 21009 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:46.409943 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:46.362476 21389 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:46.422513 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:46.430970 12600 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:13:46.431969 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:46.423501 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:46.395234 21532 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:46.434216 21215 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:46.393713 21587 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:13:46.435443 21587 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:46.435977 21587 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:46.436303 21591 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:46.428562 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:46.376443 21191 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:46.438393 21614 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:13:46.438498 12674 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:46.440105 12675 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 24: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:13:46.433205 21620 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 12:13:46.419278 21562 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:46.422480 21576 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:46.419602 21194 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:46.427919 20913 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:46.430311 21619 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.096s
I20260709 12:13:46.531234 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1033) took 1001 ms. Trace:
I20260709 12:13:46.439373 21009 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:46.532374 12677 rpcz_store.cc:276] 0709 12:13:45.529567 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.529793 (+   226us) service_pool.cc:224] Handling call
0709 12:13:45.530200 (+   407us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:45.774490 (+244290us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.530763 (+756273us) spinlock_profiling.cc:243] Waited 216 ms on lock 0x7b540036d154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:46.531121 (+   358us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":497645952}
I20260709 12:13:46.513597 21576 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:13:46.619521 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 225395us
I20260709 12:13:46.792527 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:46.817536 12602 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:13:46.797125 12674 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:13:46.418257 21249 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.087s
I20260709 12:13:46.418272 21528 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.090s
W20260709 12:13:46.803236 21434 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.662s	user 0.015s	sys 0.011s
I20260709 12:13:46.844527 21215 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:46.844767 21434 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:46.845254 21215 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:46.803339 21547 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:46.847611 21621 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.404s	user 0.000s	sys 0.001s
I20260709 12:13:46.847887 21547 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:46.849318 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:46.812742 21307 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:46.850075 21547 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:46.851641 21605 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.889s	user 0.001s	sys 0.001s
I20260709 12:13:46.821717 21387 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:46.852451 21387 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:46.822151 21586 consensus_meta.cc:287] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.724s	user 0.012s	sys 0.017s
I20260709 12:13:46.853736 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 3435) took 1183 ms. Trace:
W20260709 12:13:46.807113 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 397250us
I20260709 12:13:46.853632 21586 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:46.854153 12530 rpcz_store.cc:276] 0709 12:13:45.670693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.682315 (+ 11622us) service_pool.cc:224] Handling call
0709 12:13:46.853549 (+1171234us) spinlock_profiling.cc:243] Waited 17.2 ms on lock 0x7b5400210cd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:46.853661 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39618816}
I20260709 12:13:46.854696 21387 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:46.855856 21586 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:46.856216 21307 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:13:46.857656 21434 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:46.821966 21613 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:46.860553 21587 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:46.856277 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1109) took 1289 ms. Trace:
I20260709 12:13:46.862100 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4213) took 1186 ms. Trace:
I20260709 12:13:46.862442 12526 rpcz_store.cc:276] 0709 12:13:45.567081 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.568642 (+  1561us) service_pool.cc:224] Handling call
0709 12:13:45.597176 (+ 28534us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:45.699191 (+102015us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:45.699281 (+    90us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:45.699411 (+   130us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:13:45.699537 (+   126us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.856039 (+1156502us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b5400407754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:46.856090 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:46.856216 (+   126us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":233815296}
I20260709 12:13:46.822817 12528 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:13:46.862625 12680 rpcz_store.cc:276] 0709 12:13:45.675278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.034029 (+358751us) service_pool.cc:224] Handling call
0709 12:13:46.039173 (+  5144us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:46.039638 (+   465us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:46.039678 (+    40us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:46.079360 (+ 39682us) log.cc:844] Serialized 52 byte log entry
0709 12:13:46.396536 (+317176us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:46.396587 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.396656 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:46.861853 (+465197us) raft_consensus.cc:1685] finished
0709 12:13:46.861875 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:46.862038 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":186,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":6881,"thread_start_us":256076,"threads_started":1,"wal-append.queue_time_us":748418}
W20260709 12:13:46.824558 21373 consensus_meta.cc:287] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.126s	user 0.013s	sys 0.011s
I20260709 12:13:46.850567 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4028) took 1119 ms. Trace:
I20260709 12:13:46.823230 12679 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27. Ops: []
I20260709 12:13:46.858150 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4215) took 1182 ms. Trace:
I20260709 12:13:46.870587 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:46.842968 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:46.847613 21215 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:46.860679 21613 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:46.869293 12599 rpcz_store.cc:276] 0709 12:13:45.731549 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.178708 (+447159us) service_pool.cc:224] Handling call
0709 12:13:46.179120 (+   412us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:46.179554 (+   434us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:46.179595 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:46.194506 (+ 14911us) log.cc:844] Serialized 26 byte log entry
0709 12:13:46.391169 (+196663us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:46.391215 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.391280 (+    65us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:46.850257 (+458977us) raft_consensus.cc:1685] finished
0709 12:13:46.850280 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:46.850482 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":32640,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":20910,"thread_start_us":177345,"threads_started":1,"wal-append.queue_time_us":631785}
I20260709 12:13:46.869537 12671 rpcz_store.cc:276] 0709 12:13:45.675812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.080050 (+404238us) service_pool.cc:224] Handling call
0709 12:13:46.080489 (+   439us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:46.080984 (+   495us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:46.081027 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:46.081553 (+   526us) log.cc:844] Serialized 26 byte log entry
0709 12:13:46.380731 (+299178us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:46.380822 (+    91us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.381000 (+   178us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:46.857883 (+476883us) raft_consensus.cc:1685] finished
0709 12:13:46.857906 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:46.858061 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2705,"raft.queue_time_us":145,"raft.run_cpu_time_us":222,"raft.run_wall_time_us":22588,"thread_start_us":260018,"threads_started":1,"wal-append.queue_time_us":739493}
I20260709 12:13:46.876075 12674 raft_consensus.cc:2410] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 25 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:13:46.866752 12604 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:46.868178 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:46.871510 12602 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 23: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:46.433911 21358 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:46.879371 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:46.884029 12571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.024s	sys 0.055s
W20260709 12:13:46.884779 12571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.674s	user 0.024s	sys 0.055s
I20260709 12:13:46.884941 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:46.885609 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 691502us
I20260709 12:13:46.888338 12522 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 19 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:13:46.889541 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:46.890591 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4214) took 1215 ms. Trace:
I20260709 12:13:46.892510 12680 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28. Ops: [27.19-27.19]
I20260709 12:13:46.894613 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:46.881547 12572 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:46.898577 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3553) took 1192 ms. Trace:
I20260709 12:13:46.899132 12674 rpcz_store.cc:276] 0709 12:13:45.708495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.897620 (+1189125us) service_pool.cc:224] Handling call
0709 12:13:46.897946 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:46.898190 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:46.898254 (+    64us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:46.898298 (+    44us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:46.898330 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.898384 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:46.898504 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:46.891006 12678 rpcz_store.cc:276] 0709 12:13:45.675566 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.036214 (+360648us) service_pool.cc:224] Handling call
0709 12:13:46.036592 (+   378us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:46.037442 (+   850us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:46.037484 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:46.038054 (+   570us) log.cc:844] Serialized 26 byte log entry
0709 12:13:46.246378 (+208324us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:46.246633 (+   255us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.247016 (+   383us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:46.847078 (+600062us) spinlock_profiling.cc:243] Waited 50.4 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:46.865383 (+ 18305us) raft_consensus.cc:1685] finished
0709 12:13:46.865411 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:46.890521 (+ 25110us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":350,"raft.queue_time_us":437,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":7769,"spinlock_wait_cycles":115852544,"thread_start_us":199626,"threads_started":1,"wal-append.queue_time_us":361149}
I20260709 12:13:46.901257 12674 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 15. (term mismatch)
I20260709 12:13:46.901710 12674 pending_rounds.cc:85] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 14
I20260709 12:13:46.893473 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4217) took 1215 ms. Trace:
I20260709 12:13:46.887138 12571 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:46.902341 12680 rpcz_store.cc:276] 0709 12:13:45.678436 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.880263 (+1201827us) service_pool.cc:224] Handling call
0709 12:13:46.882289 (+  2026us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:46.893089 (+ 10800us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.893390 (+   301us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:46.832826 12494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.576s	user 0.007s	sys 0.056s
I20260709 12:13:46.901983 12674 pending_rounds.cc:107] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 27.15
I20260709 12:13:46.910166 12674 raft_consensus.cc:2889] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:46.910181 21377 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:13:46.911013 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4225) took 1185 ms. Trace:
I20260709 12:13:46.912513 12674 rpcz_store.cc:276] 0709 12:13:45.729005 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.900444 (+1171439us) service_pool.cc:224] Handling call
0709 12:13:46.900785 (+   341us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:46.910761 (+  9976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.910954 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:46.914431 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:46.915220 12531 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:46.919456 21542 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:46.920558 21215 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:13:46.920977 21215 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:46.921310 21215 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:46.922269 21389 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:46.922567 21191 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:46.923611 21591 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:46.924484 21591 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
W20260709 12:13:46.924913 21396 consensus_meta.cc:287] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.592s	user 0.009s	sys 0.006s
I20260709 12:13:46.925127 21591 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:46.925383 21396 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:46.925879 21551 raft_consensus.cc:3055] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:13:46.883639 21373 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:46.928177 21373 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:46.928262 12525 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:46.928684 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 4216) took 1250 ms. Trace:
I20260709 12:13:46.929272 12675 rpcz_store.cc:276] 0709 12:13:45.677916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.028099 (+350183us) service_pool.cc:224] Handling call
0709 12:13:46.928477 (+900378us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b540015e0d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:46.928626 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":943888128}
I20260709 12:13:46.931339 21468 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:46.906689 21622 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.086s	user 0.000s	sys 0.001s
I20260709 12:13:46.931793 21562 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:46.932121 21532 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:46.932358 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4227) took 1188 ms. Trace:
I20260709 12:13:46.933552 21389 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:13:46.934378 12603 raft_consensus.cc:2368] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
I20260709 12:13:46.934819 21613 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:46.935133 12678 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:46.936532 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4226) took 1194 ms. Trace:
I20260709 12:13:46.926323 21551 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:46.937709 21551 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:46.936689 21574 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.798s	user 0.014s	sys 0.012s
I20260709 12:13:46.911150 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:46.938709 21626 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.119s	user 0.001s	sys 0.000s
W20260709 12:13:46.906629 12494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.650s	user 0.007s	sys 0.056s
I20260709 12:13:46.938908 12678 rpcz_store.cc:276] 0709 12:13:45.742145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.930440 (+1188295us) service_pool.cc:224] Handling call
0709 12:13:46.930799 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:46.935544 (+  4745us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.936299 (+   755us) spinlock_profiling.cc:243] Waited 4.14 ms on lock 0x7b54002c5954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:46.936474 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9518464}
I20260709 12:13:46.933336 12680 rpcz_store.cc:276] 0709 12:13:45.744369 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.931598 (+1187229us) service_pool.cc:224] Handling call
0709 12:13:46.931887 (+   289us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:46.932058 (+   171us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:46.932079 (+    21us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:46.932120 (+    41us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:46.932149 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.932215 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:46.932308 (+    93us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:46.939023 21586 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:46.950712 12571 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:46.944705 21554 consensus_meta.cc:287] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.799s	user 0.017s	sys 0.006s
I20260709 12:13:46.925688 21591 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:46.956383 12604 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:46.919457 21449 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:46.964668 12571 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:46.948964 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 692355us
I20260709 12:13:46.936585 21507 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:46.945421 21532 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:46.973450 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4039) took 1102 ms. Trace:
W20260709 12:13:46.948899 21477 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.824s	user 0.019s	sys 0.001s
I20260709 12:13:46.975255 12494 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:46.983417 12571 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:46.947640 21551 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:46.984151 12678 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 20 index: 11. Preceding OpId from leader: term: 22 index: 12. (index mismatch)
I20260709 12:13:46.991231 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3558) took 1154 ms. Trace:
I20260709 12:13:46.991592 12678 rpcz_store.cc:276] 0709 12:13:45.836520 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.950066 (+1113546us) service_pool.cc:224] Handling call
0709 12:13:46.975305 (+ 25239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:46.990947 (+ 15642us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.991171 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:46.993934 12531 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:13:46.963464 21586 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:46.944914 21574 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:46.998948 21438 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:47.007587 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:46.982306 21587 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:47.005862 21586 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:47.008793 12678 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:47.009954 12601 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:47.011297 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4036) took 1178 ms. Trace:
I20260709 12:13:47.011650 12601 rpcz_store.cc:276] 0709 12:13:45.833445 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.126734 (+293289us) service_pool.cc:224] Handling call
0709 12:13:47.011109 (+884375us) spinlock_profiling.cc:243] Waited 822 ms on lock 0x7b54002e5ed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:47.011239 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1890434944}
W20260709 12:13:46.947741 21592 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.013s	sys 0.000s
W20260709 12:13:47.013332 21592 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.583s	user 0.014s	sys 0.000s
I20260709 12:13:47.013621 21592 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:47.014602 21592 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.016346 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:47.017565 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:46.873206 20913 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:46.986333 12604 rpcz_store.cc:276] 0709 12:13:45.871241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.126339 (+255098us) service_pool.cc:224] Handling call
0709 12:13:46.849100 (+722761us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:13:46.973347 (+124247us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":23492352}
I20260709 12:13:46.974898 21527 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:13:46.996148 21434 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:13:47.008083 21574 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:46.985044 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 161110us
I20260709 12:13:46.996778 12494 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:47.023021 21438 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:47.024040 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:47.023808 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:46.982318 21449 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:47.026773 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:47.027420 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3480) took 1429 ms. Trace:
I20260709 12:13:47.028215 21633 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:13:47.027433 12677 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:47.030578 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1029) took 1503 ms. Trace:
I20260709 12:13:47.031016 12672 rpcz_store.cc:276] 0709 12:13:45.526839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.526986 (+   147us) service_pool.cc:224] Handling call
0709 12:13:45.555400 (+ 28414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:45.604446 (+ 49046us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:45.604478 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:45.605267 (+   789us) log.cc:844] Serialized 26 byte log entry
0709 12:13:45.770326 (+165059us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:45.770372 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:46.851065 (+1080693us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:47.030181 (+179116us) spinlock_profiling.cc:243] Waited 48.7 ms on lock 0x7b54003d45d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:47.030254 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:47.030318 (+    64us) raft_consensus.cc:1685] finished
0709 12:13:47.030334 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:47.030499 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":363,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":11326,"spinlock_wait_cycles":157873536,"thread_start_us":90671,"threads_started":1,"wal-append.queue_time_us":293698}
I20260709 12:13:47.034299 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1113) took 1459 ms. Trace:
I20260709 12:13:47.034622 21633 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:13:47.034674 21586 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.034727 12523 rpcz_store.cc:276] 0709 12:13:45.574862 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.575511 (+   649us) service_pool.cc:224] Handling call
0709 12:13:45.575797 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:45.670628 (+ 94831us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:47.034000 (+1363372us) spinlock_profiling.cc:243] Waited 81.8 ms on lock 0x7b540027f6d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:47.034212 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188066048}
I20260709 12:13:47.036818 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:47.037540 12523 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 23: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:47.037851 21478 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:13:47.038709 21486 raft_consensus.cc:3055] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:13:46.983574 21554 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:13:46.963974 21631 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:13:47.025894 21373 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:47.027858 12606 rpcz_store.cc:276] 0709 12:13:45.597884 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.644363 (+ 46479us) service_pool.cc:224] Handling call
0709 12:13:45.644706 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:45.739032 (+ 94326us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:45.739078 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:45.739668 (+   590us) log.cc:844] Serialized 26 byte log entry
0709 12:13:45.846924 (+107256us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:13:46.125738 (+278814us) log.cc:844] Serialized 14 byte log entry
0709 12:13:46.131614 (+  5876us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:47.025285 (+893671us) spinlock_profiling.cc:243] Waited 18.6 ms on lock 0x7b1800002ec0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419
0709 12:13:47.027080 (+  1795us) spinlock_profiling.cc:243] Waited 93.9 ms on lock 0x7b540022fbd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:47.027134 (+    54us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:47.027182 (+    48us) raft_consensus.cc:1685] finished
0709 12:13:47.027196 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:47.027359 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24443,"prepare.queue_time_us":135524,"raft.queue_time_us":4383,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":27154,"spinlock_wait_cycles":279360256,"thread_start_us":351529,"threads_started":2,"wal-append.queue_time_us":212129}
I20260709 12:13:46.989030 21619 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:47.059168 13339 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 29. Current term is 30. Ops: [29.19-29.19]
I20260709 12:13:47.056416 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:47.038878 21249 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:13:47.065301 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:47.057721 21527 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:47.061054 21249 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:47.067937 21527 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.066464 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:47.069396 12529 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:13:47.073679 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.077786 12572 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:47.048717 21486 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:47.091737 21591 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:47.092577 21586 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:47.092267 21438 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:13:47.093313 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:47.064801 21547 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:13:47.038738 21478 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.040s
I20260709 12:13:47.057894 21434 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:47.093784 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:46.992937 21477 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:47.107163 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:47.076632 21465 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.013s	sys 0.020s
I20260709 12:13:47.100871 21562 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:47.120630 21465 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.706s	user 0.014s	sys 0.020s
I20260709 12:13:47.115502 21434 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.117110 12529 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
W20260709 12:13:47.109191 21403 consensus_meta.cc:287] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.929s	user 0.016s	sys 0.005s
I20260709 12:13:47.118547 21532 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:13:47.101620 21410 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.129s	user 0.014s	sys 0.012s
I20260709 12:13:47.093227 21619 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:47.100378 21554 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:13:47.122530 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 240621us
I20260709 12:13:47.098639 21307 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:47.129926 12572 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:47.133677 13337 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 26: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:47.123057 21505 raft_consensus.cc:3055] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:47.069063 21249 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:47.131281 21629 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:47.128257 21586 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:47.130030 21358 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:47.122135 21191 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.023s
I20260709 12:13:47.151471 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:47.134852 21410 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.134470 21403 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:47.152760 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 267898us
I20260709 12:13:47.191557 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:47.196139 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:47.221149 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 365339us
I20260709 12:13:47.144397 21532 raft_consensus.cc:697] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:47.244719 21532 consensus_queue.cc:237] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.096486 12678 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:47.261667 12571 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:47.263872 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:47.264819 12678 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:47.145857 21505 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:47.270817 21639 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:47.271049 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:47.272431 21613 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:47.147102 21468 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:47.229612 21249 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:47.280344 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:47.274504 21542 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:47.148994 21591 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:47.283139 12673 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:47.284466 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:47.284811 12605 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:47.135516 21486 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:47.182274 21577 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:47.186262 21636 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:47.138299 21307 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:47.274756 21635 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:13:47.287907 21542 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.182699 21631 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:47.193686 21586 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:47.137717 21554 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:47.268231 20958 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:47.296761 21191 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:47.160168 21215 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.290074 21468 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:47.275326 21358 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:47.298614 21546 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:47.184319 21629 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:47.304028 21486 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:47.295771 21591 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:47.325390 21534 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.926s	user 0.016s	sys 0.006s
I20260709 12:13:47.275985 21619 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.299623 21577 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.304592 21636 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.342705 20958 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.337970 21586 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:47.336840 21631 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.371681 12678 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:13:47.510361 13332 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:47.588346 13330 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
W20260709 12:13:47.539423 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.577389 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4230) took 1417 ms. Trace:
I20260709 12:13:47.516774 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4211) took 1842 ms. Trace:
I20260709 12:13:47.347409 21547 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:47.540593 21449 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.514s	user 0.010s	sys 0.001s
I20260709 12:13:47.635121 13332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 994) took 1263 ms. Trace:
I20260709 12:13:47.635691 13332 rpcz_store.cc:276] 0709 12:13:46.372162 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.372680 (+   518us) service_pool.cc:224] Handling call
0709 12:13:47.510054 (+1137374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:47.633347 (+123293us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:47.634853 (+  1506us) spinlock_profiling.cc:243] Waited 1.14 s on lock 0x7b5400469950. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:47.635040 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2614666112}
I20260709 12:13:47.498622 21255 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 LEADER]: Stepping down as leader of term 28
W20260709 12:13:47.550947 21551 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.557s	user 0.012s	sys 0.000s
I20260709 12:13:47.490620 20913 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:13:47.491525 21215 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:47.637591 21255 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:47.387116 21477 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:47.475804 21638 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.184s	user 0.000s	sys 0.001s
W20260709 12:13:47.547446 21576 consensus_meta.cc:287] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.725s	user 0.010s	sys 0.002s
W20260709 12:13:47.385916 21534 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.990s	user 0.019s	sys 0.006s
W20260709 12:13:47.477389 21009 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.653s	user 0.008s	sys 0.003s
W20260709 12:13:47.481690 21602 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.504s	user 0.001s	sys 0.001s
W20260709 12:13:47.524062 21600 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.545s	user 0.005s	sys 0.000s
I20260709 12:13:47.516472 21554 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:47.527419 21629 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:47.509883 13337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 1054) took 1582 ms. Trace:
I20260709 12:13:47.376859 21191 raft_consensus.cc:697] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:47.498788 21403 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:47.519721 21307 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:47.478109 21410 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:47.519493 21375 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:47.515409 21542 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:47.639680 21099 raft_consensus.cc:3055] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:47.652344 13337 rpcz_store.cc:276] 0709 12:13:45.927503 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.932590 (+  5087us) service_pool.cc:224] Handling call
0709 12:13:47.509579 (+1576989us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b5400469954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:47.509779 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2738944256}
I20260709 12:13:47.609293 21546 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.639866 21449 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.461828 21468 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.352877 21505 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:47.647621 21255 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.649488 21547 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:47.665094 21255 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:47.658203 21099 raft_consensus.cc:740] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:47.648921 21576 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:47.651697 21009 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:47.668128 21358 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:47.695469 21546 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:47.696883 21591 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:47.697306 21636 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:47.697649 21586 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:13:47.683753 21613 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:47.698303 21577 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:47.708029 21631 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:47.749187 12494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.649s	user 0.007s	sys 0.015s
W20260709 12:13:47.750381 12494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.670s	user 0.007s	sys 0.017s
W20260709 12:13:47.750841 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 674239us
I20260709 12:13:47.752360 12494 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:47.754911 12522 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.015s	sys 0.007s
W20260709 12:13:47.755458 12522 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.693s	user 0.016s	sys 0.008s
W20260709 12:13:47.755560 12530 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.781s	user 0.004s	sys 0.019s
W20260709 12:13:47.755615 12526 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.750s	user 0.004s	sys 0.011s
W20260709 12:13:47.756198 12530 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.782s	user 0.004s	sys 0.019s
W20260709 12:13:47.756264 12526 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.004s	sys 0.011s
W20260709 12:13:47.754930 12528 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.755s	user 0.001s	sys 0.022s
W20260709 12:13:47.757046 12528 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.758s	user 0.001s	sys 0.022s
I20260709 12:13:47.293845 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:47.770829 12572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.585s	user 0.001s	sys 0.009s
I20260709 12:13:47.297125 12643 leader_election.cc:400] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
I20260709 12:13:47.508420 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 4219) took 1829 ms. Trace:
I20260709 12:13:47.772437 12643 leader_election.cc:403] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Cancelling election due to peer responding with higher term
I20260709 12:13:47.773029 12673 rpcz_store.cc:276] 0709 12:13:45.679107 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.998472 (+319365us) service_pool.cc:224] Handling call
0709 12:13:47.508135 (+1509663us) spinlock_profiling.cc:243] Waited 1.17 s on lock 0x7b54002c5454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:47.508338 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2682281216}
W20260709 12:13:47.774362 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 480236us
I20260709 12:13:47.775911 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:47.778196 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:47.476660 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4233) took 1196 ms. Trace:
I20260709 12:13:47.783491 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:47.784708 12674 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:47.780339 12672 rpcz_store.cc:276] 0709 12:13:46.280285 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:47.032921 (+752636us) service_pool.cc:224] Handling call
0709 12:13:47.033335 (+   414us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:47.038237 (+  4902us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:47.038284 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:47.038785 (+   501us) log.cc:844] Serialized 26 byte log entry
0709 12:13:47.340515 (+301730us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:47.340562 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:47.340648 (+    86us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:47.476384 (+135736us) raft_consensus.cc:1685] finished
0709 12:13:47.476423 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:47.476593 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1693,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":95,"thread_start_us":301406,"threads_started":1,"wal-append.queue_time_us":426406}
W20260709 12:13:47.771622 12572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.586s	user 0.001s	sys 0.009s
I20260709 12:13:47.485285 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:47.795642 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3537) took 2261 ms. Trace:
I20260709 12:13:47.796307 12675 raft_consensus.cc:2368] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24.
W20260709 12:13:47.502743 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:47.495862 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4232) took 1335 ms. Trace:
I20260709 12:13:47.798944 12677 rpcz_store.cc:276] 0709 12:13:46.159925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:47.030894 (+870969us) service_pool.cc:224] Handling call
0709 12:13:47.031237 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:47.031973 (+   736us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:47.032003 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:47.032481 (+   478us) log.cc:844] Serialized 26 byte log entry
0709 12:13:47.307236 (+274755us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:47.307297 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:47.307518 (+   221us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:47.495643 (+188125us) raft_consensus.cc:1685] finished
0709 12:13:47.495664 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:47.495802 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":90,"raft.run_cpu_time_us":89,"raft.run_wall_time_us":91,"thread_start_us":274321,"threads_started":1,"wal-append.queue_time_us":402256}
I20260709 12:13:47.608798 12679 rpcz_store.cc:276] 0709 12:13:45.674597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.031131 (+356534us) service_pool.cc:224] Handling call
0709 12:13:46.087848 (+ 56717us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:46.869464 (+781616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:47.516417 (+646953us) spinlock_profiling.cc:243] Waited 735 ms on lock 0x7b54000aafd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:47.516682 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1690787584}
I20260709 12:13:47.607403 12680 rpcz_store.cc:276] 0709 12:13:46.159475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.971766 (+812291us) service_pool.cc:224] Handling call
0709 12:13:46.972136 (+   370us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:46.972387 (+   251us) raft_consensus.cc:1534] Early marking committed up to index 11
0709 12:13:47.213500 (+241113us) log.cc:844] Serialized 14 byte log entry
0709 12:13:47.576801 (+363301us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:47.576930 (+   129us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:47.577010 (+    80us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:47.577108 (+    98us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:47.577303 (+   195us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":382007,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":66,"raft.queue_time_us":144,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":2305,"thread_start_us":582733,"threads_started":2}
I20260709 12:13:47.796267 12676 rpcz_store.cc:276] 0709 12:13:45.534213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:45.534336 (+   123us) service_pool.cc:224] Handling call
0709 12:13:45.534678 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:45.535045 (+   367us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:45.535153 (+   108us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:45.547485 (+ 12332us) log.cc:844] Serialized 26 byte log entry
0709 12:13:45.871692 (+324207us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:45.871731 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:45.871803 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:47.481135 (+1609332us) spinlock_profiling.cc:243] Waited 4.42 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:47.794420 (+313285us) raft_consensus.cc:1685] finished
0709 12:13:47.794460 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:47.795379 (+   919us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17011,"raft.queue_time_us":199,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":21571,"spinlock_wait_cycles":10160000,"thread_start_us":269379,"threads_started":1,"wal-append.queue_time_us":395766}
I20260709 12:13:47.789804 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:47.779297 12673 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 25
W20260709 12:13:47.772524 12598 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.516s	user 0.000s	sys 0.000s
W20260709 12:13:47.831130 12497 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.592s	user 0.011s	sys 0.015s
W20260709 12:13:47.838310 12598 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.001s	sys 0.001s
I20260709 12:13:47.838587 21215 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:13:47.837407 12497 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.598s	user 0.011s	sys 0.016s
I20260709 12:13:47.809500 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:47.842489 12496 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.009s	sys 0.005s
W20260709 12:13:47.676699 21465 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.528s	user 0.007s	sys 0.008s
I20260709 12:13:47.843993 12675 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 28. Current term is 29. Ops: [28.17-28.17]
I20260709 12:13:47.801975 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:47.836804 12571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.574s	user 0.015s	sys 0.000s
W20260709 12:13:47.852104 12571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.016s	sys 0.000s
I20260709 12:13:47.853718 21478 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:47.855099 21358 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:47.855047 21478 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:47.850477 12496 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.681s	user 0.010s	sys 0.006s
I20260709 12:13:47.861272 21477 consensus_queue.cc:1059] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:13:47.863229 21009 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:47.869673 21554 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:47.869963 21099 consensus_queue.cc:260] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.870820 21554 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.871809 21099 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:47.873765 21554 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:47.914492 12529 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:47.919891 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:47.936739 12675 raft_consensus.cc:2393] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 22: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:47.937024 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:47.909134 21631 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:47.917333 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:47.875732 20958 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:47.875949 21449 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:47.883390 21619 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:47.889051 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3440) took 1014 ms. Trace:
W20260709 12:13:47.919359 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 117887us
I20260709 12:13:47.920190 21641 raft_consensus.cc:3055] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:47.915648 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:47.900158 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4033) took 2093 ms. Trace:
I20260709 12:13:47.922508 21191 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 20.15, Last appended by leader: 15, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:47.943444 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 155593us
W20260709 12:13:47.945829 12606 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.706s	user 0.003s	sys 0.002s
I20260709 12:13:47.907403 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1116) took 1743 ms. Trace:
I20260709 12:13:47.928687 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:47.946982 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4030) took 2203 ms. Trace:
I20260709 12:13:47.929941 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:47.952564 12525 rpcz_store.cc:276] 0709 12:13:46.163735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.164432 (+   697us) service_pool.cc:224] Handling call
0709 12:13:46.164763 (+   331us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:46.928930 (+764167us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:47.906693 (+977763us) spinlock_profiling.cc:243] Waited 763 ms on lock 0x7b54001068d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:47.907102 (+   409us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1755433216}
I20260709 12:13:47.919831 21477 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader election won for term 29
W20260709 12:13:47.945020 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 776148us
I20260709 12:13:47.912834 21670 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:47.914866 21399 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:47.967170 21670 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.967209 21399 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.897949 21591 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:47.970988 21591 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:47.974390 12679 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:47.978897 20958 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:47.979199 21399 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:47.928192 21478 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:47.979671 20958 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:47.983001 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:47.983891 12603 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:47.983444 21619 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.986022 21399 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:47.987085 21399 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:47.988852 12677 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:47.992489 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:47.993396 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:47.994455 12602 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:47.996762 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:47.996452 21477 raft_consensus.cc:697] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 29 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:47.997725 12599 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:47.998546 21477 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:47.995937 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:48.000324 12602 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:47.908825 21562 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:48.000711 20958 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:48.002208 21562 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:47.993558 12604 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:48.004781 21562 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:13:47.946628 12528 rpcz_store.cc:276] 0709 12:13:46.875086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.875215 (+   129us) service_pool.cc:224] Handling call
0709 12:13:46.909062 (+ 33847us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:46.948327 (+ 39265us) raft_consensus.cc:1534] Early marking committed up to index 10
0709 12:13:46.974529 (+ 26202us) log.cc:844] Serialized 14 byte log entry
0709 12:13:47.500733 (+526204us) spinlock_profiling.cc:243] Waited 2.48 ms on lock 0x7b1800310fc0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:13:47.757963 (+257230us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:47.759521 (+  1558us) log.cc:844] Serialized 52 byte log entry
0709 12:13:47.759815 (+   294us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:47.760297 (+   482us) log.cc:844] Serialized 14 byte log entry
0709 12:13:47.760500 (+   203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:47.760610 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:47.888619 (+128009us) raft_consensus.cc:1685] finished
0709 12:13:47.888647 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:47.888939 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125726,"prepare.queue_time_us":81812,"prepare.run_cpu_time_us":168,"prepare.run_wall_time_us":5726,"raft.queue_time_us":193383,"raft.run_cpu_time_us":772,"raft.run_wall_time_us":20655,"spinlock_wait_cycles":5713280,"thread_start_us":758203,"threads_started":1,"wal-append.queue_time_us":495375}
I20260709 12:13:48.006846 21562 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:48.010658 12603 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:48.009821 21562 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:48.016299 12495 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.689s	user 0.000s	sys 0.014s
I20260709 12:13:48.017825 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:48.022367 12597 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:47.951807 12605 rpcz_store.cc:276] 0709 12:13:45.806250 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.342993 (+536743us) service_pool.cc:224] Handling call
0709 12:13:46.343418 (+   425us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:47.293652 (+950234us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:47.899473 (+605821us) spinlock_profiling.cc:243] Waited 939 ms on lock 0x7b540022ecd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:47.899868 (+   395us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2159779584}
I20260709 12:13:47.915679 21576 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:47.962936 12606 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.827s	user 0.004s	sys 0.003s
I20260709 12:13:47.924991 21577 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:48.035105 21562 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:47.951246 21641 raft_consensus.cc:740] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:48.019646 12676 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26.
I20260709 12:13:47.909721 21434 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:48.042223 12529 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:47.938025 21631 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:47.929450 21505 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:13:48.038625 12495 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.001s	sys 0.014s
I20260709 12:13:48.033567 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4224) took 2150 ms. Trace:
W20260709 12:13:48.040009 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 778433us
I20260709 12:13:48.078112 12674 raft_consensus.cc:2410] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 26 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 12:13:48.071389 12671 rpcz_store.cc:276] 0709 12:13:45.682910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.883880 (+1200970us) service_pool.cc:224] Handling call
0709 12:13:46.884264 (+   384us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:47.294891 (+410627us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:13:47.294944 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:47.317572 (+ 22628us) log.cc:844] Serialized 26 byte log entry
0709 12:13:47.823653 (+506081us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:47.823778 (+   125us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:47.823920 (+   142us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:47.827414 (+  3494us) raft_consensus.cc:1685] finished
0709 12:13:47.827449 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:47.833427 (+  5978us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":39623,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":104,"thread_start_us":875996,"threads_started":2,"wal-append.queue_time_us":481766}
I20260709 12:13:48.103937 12599 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:48.095800 20827 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.112650 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:48.120431 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1017) took 1049 ms. Trace:
W20260709 12:13:48.123943 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.077378 21434 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:48.140879 21587 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.141548 21587 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:48.143411 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.143671 12671 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:48.148588 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:48.117347 20827 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:48.155359 20913 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.512s	user 0.012s	sys 0.005s
I20260709 12:13:48.156536 12527 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:13:48.158448 21534 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:13:48.160666 21438 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:13:48.162572 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.162775 21438 consensus_queue.cc:1059] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 10, Time since last communication: 0.001s
I20260709 12:13:48.165769 21661 raft_consensus.cc:3055] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:48.166285 21661 raft_consensus.cc:740] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:48.167335 21661 consensus_queue.cc:260] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:48.169032 21661 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:48.172348 21507 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:13:48.174484 12674 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24. Ops: [23.18-23.18]
I20260709 12:13:48.176337 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:48.177254 21641 consensus_queue.cc:260] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:48.178908 21641 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:48.181052 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:48.182407 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:48.183072 12604 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:48.183962 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:48.185039 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:48.187027 21399 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:48.188596 21619 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:48.190930 12494 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:48.193100 21468 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.193730 21468 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:48.196426 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:48.201288 12571 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:48.202589 21306 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:13:48.203194 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:48.203504 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:48.204262 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.204779 12597 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:13:47.956825 12598 rpcz_store.cc:276] 0709 12:13:45.743150 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.327583 (+584433us) service_pool.cc:224] Handling call
0709 12:13:46.328114 (+   531us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:13:46.331566 (+  3452us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:46.331607 (+    41us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:13:46.337545 (+  5938us) log.cc:844] Serialized 78 byte log entry
0709 12:13:46.842765 (+505220us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:47.137346 (+294581us) log.cc:844] Serialized 14 byte log entry
0709 12:13:47.154944 (+ 17598us) log.cc:844] Serialized 14 byte log entry
0709 12:13:47.227605 (+ 72661us) log.cc:844] Serialized 14 byte log entry
0709 12:13:47.492085 (+264480us) spinlock_profiling.cc:243] Waited 6.69 ms on lock 0x7b480024c258. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:13:47.839818 (+347733us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:47.858154 (+ 18336us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:47.868434 (+ 10280us) raft_consensus.cc:1685] finished
0709 12:13:47.868450 (+    16us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:47.946891 (+ 78441us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":106150,"prepare.queue_time_us":941542,"prepare.run_cpu_time_us":169,"prepare.run_wall_time_us":1965,"raft.queue_time_us":75173,"raft.run_cpu_time_us":3850,"raft.run_wall_time_us":1207667,"spinlock_wait_cycles":15377664,"thread_start_us":1277794,"threads_started":3,"wal-append.queue_time_us":610356}
I20260709 12:13:48.227278 12673 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:48.229333 12679 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:48.223246 21306 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:48.231760 21306 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:48.233505 21586 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:13:48.234731 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:48.235378 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1122) took 1442 ms. Trace:
I20260709 12:13:48.235782 12524 rpcz_store.cc:276] 0709 12:13:46.792816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.792944 (+   128us) service_pool.cc:224] Handling call
0709 12:13:46.793250 (+   306us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:46.793600 (+   350us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:46.793627 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:46.794073 (+   446us) log.cc:844] Serialized 26 byte log entry
0709 12:13:47.792205 (+998132us) spinlock_profiling.cc:243] Waited 997 us on lock 0x7b18002aa9c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:13:48.053252 (+261047us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:13:48.053326 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.081603 (+ 28277us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:48.234880 (+153277us) raft_consensus.cc:1685] finished
0709 12:13:48.234900 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:48.235203 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":184,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":14540,"spinlock_wait_cycles":2294016,"thread_start_us":53802,"threads_started":1,"wal-append.queue_time_us":1291924}
I20260709 12:13:48.237565 21534 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:13:48.237994 21534 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:48.240964 12680 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:48.228094 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3572) took 1132 ms. Trace:
I20260709 12:13:48.258363 12673 rpcz_store.cc:276] 0709 12:13:47.095765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:47.777673 (+681908us) service_pool.cc:224] Handling call
0709 12:13:48.228011 (+450338us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:48.122208 12606 rpcz_store.cc:276] 0709 12:13:47.071020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:47.073219 (+  2199us) service_pool.cc:224] Handling call
0709 12:13:47.074029 (+   810us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:47.106786 (+ 32757us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:47.106841 (+    55us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:47.114549 (+  7708us) log.cc:844] Serialized 52 byte log entry
0709 12:13:48.077402 (+962853us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:13:48.087687 (+ 10285us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.101841 (+ 14154us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:48.101890 (+    49us) raft_consensus.cc:1685] finished
0709 12:13:48.101904 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:48.116354 (+ 14450us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":45902,"raft.queue_time_us":33902,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":3384,"thread_start_us":922794,"threads_started":1,"wal-append.queue_time_us":688197}
I20260709 12:13:48.257690 12602 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29.
I20260709 12:13:48.260545 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:48.261406 12679 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 23: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:48.262249 12677 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:48.108006 21591 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:48.267760 12571 leader_election.cc:400] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24.
I20260709 12:13:48.272173 12571 leader_election.cc:403] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Cancelling election due to peer responding with higher term
W20260709 12:13:48.264210 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 45645.7us
I20260709 12:13:48.276428 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:48.239223 21534 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:48.282012 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:48.291430 12603 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:48.294543 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:48.298151 12675 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:13:48.299786 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:48.301496 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:48.302415 21249 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.303025 21249 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:48.305409 21249 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:48.312631 12427 catalog_manager.cc:5697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 8 to 16. New cstate: current_term: 16 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:13:48.314694 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 111771us
I20260709 12:13:48.298233 21661 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:13:48.316123 12675 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:48.317368 21534 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:48.325446 13304 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:48.335117 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:48.224825 21577 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:48.346823 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:48.349900 12676 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 28. Current term is 29. Ops: [28.18-28.18]
I20260709 12:13:48.351259 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:48.352142 12524 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:13:48.352245 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 19 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:48.355454 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:48.352949 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:48.345549 21670 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:48.363190 13332 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 29 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: []
I20260709 12:13:48.363274 12597 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:48.359311 12605 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:13:48.366705 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:48.353461 21586 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election lost for term 23. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24.
I20260709 12:13:48.368705 12604 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:48.394502 12675 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:48.369798 21661 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:13:48.401118 12423 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58992 (request call id 173) took 105682 ms (client timeout 15000 ms). Trace:
I20260709 12:13:48.388476 12603 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:48.404704 12603 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 17. (term mismatch)
I20260709 12:13:48.405071 12603 pending_rounds.cc:85] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 16
I20260709 12:13:48.405313 12603 pending_rounds.cc:107] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 28.17
I20260709 12:13:48.394186 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:13:48.406548 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 65688.9us
I20260709 12:13:48.405994 12603 raft_consensus.cc:2889] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:48.410635 21587 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:48.411469 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 143962us
I20260709 12:13:48.419854 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:48.422197 12677 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:13:48.416155 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:48.430799 21403 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.431617 21403 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:48.436043 12598 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 29. Current term is 30. Ops: []
I20260709 12:13:48.436851 21644 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.437754 12598 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: [23.11-23.11]
I20260709 12:13:48.441448 13331 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:13:48.446928 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:48.447940 13333 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:13:48.457634 21099 consensus_meta.cc:287] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.585s	user 0.011s	sys 0.001s
I20260709 12:13:48.459452 12604 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 26. Current term is 27. Ops: []
I20260709 12:13:48.460919 12604 pending_rounds.cc:85] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 18
I20260709 12:13:48.461251 12604 pending_rounds.cc:107] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 25.19
I20260709 12:13:48.461490 12604 raft_consensus.cc:2889] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:48.411593 21661 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:48.368018 21551 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:48.413362 21399 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.437426 21644 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:48.388571 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 76110.6us
I20260709 12:13:48.389096 12427 catalog_manager.cc:5697] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 11 to 16, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 16 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:48.469722 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:48.475349 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:48.465201 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 55401.7us
I20260709 12:13:48.373346 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:48.487121 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:48.488047 13338 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:48.488647 12601 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 26: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:13:48.501019 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.504520 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:48.505692 13335 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
W20260709 12:13:48.508762 21215 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.661s	user 0.013s	sys 0.001s
I20260709 12:13:48.509351 21215 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
I20260709 12:13:48.501070 21306 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:48.511205 12427 catalog_manager.cc:5697] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 12 to 14, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 14 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:13:48.511958 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:48.512353 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:48.512910 12427 catalog_manager.cc:5697] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 13 to 19. New cstate: current_term: 19 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:13:48.514282 21478 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.514508 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:48.514597 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:48.514914 21478 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:48.513695 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:48.516077 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:48.517529 21478 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:48.518958 21306 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:48.513458 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:48.524505 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:48.526085 21665 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.045s
I20260709 12:13:48.526839 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:48.529935 21551 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:48.530486 21551 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:48.534554 21477 consensus_queue.cc:1059] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 21.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:13:48.535382 21684 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.178s	user 0.000s	sys 0.000s
I20260709 12:13:48.537163 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3560) took 1534 ms. Trace:
I20260709 12:13:48.476799 21586 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:13:48.539772 13333 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 32.
I20260709 12:13:48.537628 12678 rpcz_store.cc:276] 0709 12:13:47.003019 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:47.795351 (+792332us) service_pool.cc:224] Handling call
0709 12:13:47.796208 (+   857us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:47.797033 (+   825us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:47.797077 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:47.798091 (+  1014us) log.cc:844] Serialized 26 byte log entry
0709 12:13:48.089074 (+290983us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:48.089128 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.089218 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:48.536772 (+447554us) raft_consensus.cc:1685] finished
0709 12:13:48.536828 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:48.537001 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":101530,"raft.run_cpu_time_us":309,"raft.run_wall_time_us":309,"thread_start_us":221801,"threads_started":1,"wal-append.queue_time_us":420619}
I20260709 12:13:48.550712 12672 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:48.537344 21640 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.015s
I20260709 12:13:48.489282 21587 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.541414 21478 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.540625 21403 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:48.572464 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.574729 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:48.579352 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:48.580132 12528 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
W20260709 12:13:48.580999 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 66779.8us
I20260709 12:13:48.582266 21586 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:13:48.490842 20913 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.071s
I20260709 12:13:48.585340 12524 raft_consensus.cc:2410] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 26 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 18.
I20260709 12:13:48.586548 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:48.552825 21215 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:48.600450 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:48.556102 21399 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:48.619614 12677 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 32.
I20260709 12:13:48.595677 21403 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:48.648310 12678 raft_consensus.cc:2393] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 27: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:48.657303 21255 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.644538 21665 raft_consensus.cc:2764] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 23: won
I20260709 12:13:48.658054 21255 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:48.660315 21255 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:48.662633 21591 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:48.662660 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:48.663431 21591 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:48.664529 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:48.665534 12523 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 26: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:48.666385 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:48.666589 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:48.667245 12531 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:48.667927 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 20 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:48.668074 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:48.668231 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:48.668658 21591 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:48.668893 12522 raft_consensus.cc:2410] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 23 because replica has last-logged OpId of term: 22 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 11.
I20260709 12:13:48.671505 21465 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:48.672595 21465 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:48.669854 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:48.675954 12530 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:48.676532 12427 catalog_manager.cc:5697] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 14 to 18, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 18 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:13:48.678524 12530 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 22 index: 11. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:13:48.679160 13337 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:48.680644 21641 raft_consensus.cc:3055] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:48.681129 21641 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:48.683084 21591 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:48.683643 21644 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:48.683689 21591 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:48.401715 12423 rpcz_store.cc:269] 0709 12:12:02.719160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:02.719583 (+   423us) service_pool.cc:224] Handling call
0709 12:13:48.400821 (+105681238us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:37.182606 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:37.260232 (+ 77626us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:37.260309 (+    77us) write_op.cc:435] Acquired schema lock
0709 12:13:37.260349 (+    40us) tablet.cc:662] Decoding operations
0709 12:13:37.359266 (+ 98917us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:37.359392 (+   126us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:37.359421 (+    29us) tablet.cc:685] Acquiring locks for 14 operations
0709 12:13:37.360925 (+  1504us) tablet.cc:701] Row locks acquired
0709 12:13:37.360979 (+    54us) write_op.cc:260] PREPARE: finished
0709 12:13:37.361287 (+   308us) write_op.cc:270] Start()
0709 12:13:37.361718 (+   431us) write_op.cc:276] Timestamp: P: 1783599217361157 usec, L: 0
0709 12:13:37.361768 (+    50us) op_driver.cc:348] REPLICATION: starting
0709 12:13:37.362870 (+  1102us) log.cc:844] Serialized 5586 byte log entry
0709 12:13:37.796770 (+433900us) op_driver.cc:464] REPLICATION: finished
0709 12:13:38.294476 (+497706us) write_op.cc:301] APPLY: starting
0709 12:13:38.335720 (+ 41244us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:38.989278 (+653558us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:38.989402 (+   124us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:39.379438 (+390036us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:39.379827 (+   389us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=28,key_file_lookups=28,delta_file_lookups=56,mrs_lookups=0
0709 12:13:39.379891 (+    64us) write_op.cc:312] APPLY: finished
0709 12:13:39.380374 (+   483us) log.cc:844] Serialized 129 byte log entry
0709 12:13:39.450970 (+ 70596us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:39.451178 (+   208us) write_op.cc:454] Released schema lock
0709 12:13:39.451272 (+    94us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:39.451925 (+   653us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:39.451974 (+    49us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":365440,"reported_tablets":52,"spinlock_wait_cycles":3328,"child_traces":[["op",{"apply.queue_time_us":497381,"apply.run_cpu_time_us":53421,"apply.run_wall_time_us":1157364,"cfile_cache_hit":58,"cfile_cache_hit_bytes":62563,"cfile_init":1,"lbm_read_time_us":47103,"lbm_reads_10-100_ms":2,"lbm_reads_lt_1ms":2,"num_ops":14,"prepare.queue_time_us":112284,"prepare.run_cpu_time_us":7304,"prepare.run_wall_time_us":255378,"replication_time_us":434817,"thread_start_us":484385,"threads_started":3,"wal-append.queue_time_us":113134,"wal-append.run_cpu_time_us":1620,"wal-append.run_wall_time_us":2027096}]]}
I20260709 12:13:48.686708 21577 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:48.612931 21587 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:48.669731 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:48.692512 13338 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
W20260709 12:13:48.698267 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 114303us
I20260709 12:13:48.690505 12522 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
W20260709 12:13:48.698174 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47954us
I20260709 12:13:48.672799 21551 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:48.739413 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:48.682153 21641 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 23.18, Last appended by leader: 18, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:48.627835 21478 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:48.753796 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:48.741705 21665 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 26.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.018s
I20260709 12:13:48.763494 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:48.770964 12531 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:13:48.747037 21644 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.690790 21215 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:48.769826 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:48.780753 12679 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:48.728965 21403 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:48.793931 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:48.774374 20913 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:13:48.808807 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 66571.1us
I20260709 12:13:48.787835 21591 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:48.788319 21665 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:13:48.802696 21587 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:48.812258 12427 catalog_manager.cc:5697] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 14 to 17. New cstate: current_term: 17 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:48.820873 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:48.786705 21644 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:48.827266 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:48.811657 20913 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:48.830986 12679 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 12:13:48.830688 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 323778us
I20260709 12:13:48.816946 21478 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:48.818145 21665 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:48.820246 21591 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:48.841851 12427 catalog_manager.cc:5697] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 15 to 18, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 18 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:13:48.852823 12427 catalog_manager.cc:5697] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 14 to 17, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 17 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
W20260709 12:13:48.833045 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 319323us
I20260709 12:13:48.838882 12494 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:48.869940 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 366424us
I20260709 12:13:48.872409 12679 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: [28.15-28.15]
I20260709 12:13:48.850523 21478 raft_consensus.cc:3055] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:48.871747 21551 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:48.880534 21689 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.063s	user 0.000s	sys 0.000s
I20260709 12:13:48.881381 21577 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:13:48.883910 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:48.884500 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.885851 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3439) took 2092 ms. Trace:
I20260709 12:13:48.886487 12527 rpcz_store.cc:276] 0709 12:13:46.793308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:46.802560 (+  9252us) service_pool.cc:224] Handling call
0709 12:13:46.802905 (+   345us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:48.156896 (+1353991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.885380 (+728484us) spinlock_profiling.cc:243] Waited 1.35 s on lock 0x7b54003c1254. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:48.885647 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3111683968}
I20260709 12:13:48.887267 21551 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.850198 21465 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:48.893111 21534 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.546s	user 0.009s	sys 0.005s
I20260709 12:13:48.860565 21591 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:48.914810 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 76026us
I20260709 12:13:48.915668 12494 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:48.874678 21587 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.916462 12427 catalog_manager.cc:5697] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 12 to 18. New cstate: current_term: 18 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:13:48.918329 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.919153 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:48.920179 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:48.878786 21478 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:48.914670 21665 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:48.925159 21587 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:48.925990 21665 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:48.927887 21587 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:48.927667 12427 catalog_manager.cc:5697] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 13 to 16, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 16 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:13:48.929384 12569 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:48.907887 21399 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:48.925638 21478 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:48.930547 12569 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:48.930970 21478 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:48.931648 12569 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:48.932227 12646 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:48.932642 21619 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:13:48.933151 21619 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:48.933563 21619 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:48.909271 21465 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:48.934623 12646 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:48.934896 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:48.928772 21587 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:48.856205 21358 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.051s
I20260709 12:13:48.936089 12646 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:13:48.947531 21687 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.162s	user 0.001s	sys 0.000s
I20260709 12:13:48.948247 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:48.948979 12524 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26.
I20260709 12:13:48.950502 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:48.952133 12531 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:13:48.952229 20827 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:48.917790 21586 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:48.953351 20827 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:48.953476 21586 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:48.954416 21591 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:48.955459 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:48.930554 21399 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:48.959159 12427 catalog_manager.cc:5697] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 17 to 19, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 19 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:13:48.896184 21551 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:48.954061 20827 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:48.982712 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:48.983477 13330 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
W20260709 12:13:48.982733 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 67171.8us
I20260709 12:13:48.987378 12494 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:48.988303 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:48.960521 21215 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:48.966135 21577 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:13:48.992867 12427 catalog_manager.cc:5697] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 9 to 11, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 11 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:13:48.965570 21591 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:13:48.981462 21701 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election won for term 24
W20260709 12:13:49.037719 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49516.6us
I20260709 12:13:49.039278 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:49.001775 21215 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:48.976322 21587 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:49.003906 21577 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:49.004776 21403 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:49.049623 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:49.052419 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 137938us
I20260709 12:13:49.052758 12675 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
I20260709 12:13:48.974824 20913 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:49.038113 21644 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:49.059309 12679 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:49.001225 21399 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:49.060747 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:49.061407 13337 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:13:49.058779 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:49.064000 12671 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:49.047616 21700 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:49.074196 13338 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 26. Current term is 27. Ops: []
W20260709 12:13:49.076627 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 140757us
I20260709 12:13:49.049396 21438 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:49.082243 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:49.074317 12427 catalog_manager.cc:5697] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 8 to 17. New cstate: current_term: 17 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:13:49.082168 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 94947us
W20260709 12:13:49.136956 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 97810.7us
I20260709 12:13:49.081552 12569 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:49.060341 21591 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:49.145594 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:49.144790 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:49.148473 13305 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:49.165081 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:49.165947 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:49.165242 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:49.170320 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:49.166468 12427 catalog_manager.cc:5697] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 9 to 12, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 12 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:13:49.171552 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:49.172230 21690 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.191s	user 0.000s	sys 0.001s
I20260709 12:13:49.172139 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:49.172729 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:49.174383 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:49.174952 12675 raft_consensus.cc:2410] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
I20260709 12:13:49.168467 12522 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:13:49.171552 12427 catalog_manager.cc:5697] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 16 to 19, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 19 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } }
I20260709 12:13:49.178598 12674 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
W20260709 12:13:49.184453 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46431.6us
I20260709 12:13:49.186267 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:49.147651 21591 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:49.201973 12529 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:13:49.151247 21644 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:13:49.199540 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 118103us
I20260709 12:13:49.216267 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:49.218147 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:49.203023 12427 catalog_manager.cc:5697] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 9 to 14, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 14 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:49.164338 21701 raft_consensus.cc:697] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:13:49.229276 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 90051.2us
W20260709 12:13:49.232987 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 61683.2us
I20260709 12:13:49.065527 20827 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:49.237299 12522 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:13:49.215091 21591 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:49.148545 21700 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:49.212373 21644 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:49.165562 21697 consensus_queue.cc:1059] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 18.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.258s
I20260709 12:13:49.248986 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:49.256724 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.260609 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3569) took 2159 ms. Trace:
I20260709 12:13:49.256446 12427 catalog_manager.cc:5697] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 10 to 19, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 19 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:49.261055 12672 rpcz_store.cc:276] 0709 12:13:47.094866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:47.788969 (+694103us) service_pool.cc:224] Handling call
0709 12:13:49.253743 (+1464774us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b540036d154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:49.253882 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1566102912}
I20260709 12:13:49.262708 12524 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:13:49.259697 12496 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:13:49.270308 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 84322.4us
I20260709 12:13:49.273041 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:49.249207 21644 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:49.245072 21586 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:49.277011 12522 raft_consensus.cc:2410] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
I20260709 12:13:49.277649 12606 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:13:49.276137 12675 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:13:49.279703 21403 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.280279 21403 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:49.283948 21403 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:49.270339 12646 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:49.257458 21215 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:49.315415 21215 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:49.265681 21702 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.270507 21551 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:49.296856 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 137840us
I20260709 12:13:49.321251 12494 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:49.297362 21721 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election won for term 21
W20260709 12:13:49.298506 21709 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:13:49.298811 21707 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:49.299285 21718 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.300629 21399 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:49.302104 21693 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.266s	user 0.001s	sys 0.000s
I20260709 12:13:49.302263 21696 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:13:49.306998 21705 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.125s	user 0.000s	sys 0.000s
I20260709 12:13:49.309381 21700 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:49.310920 21704 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.120s	user 0.000s	sys 0.001s
I20260709 12:13:49.244364 21358 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:13:49.311088 21716 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:13:49.327764 21403 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:13:49.311110 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 55932.9us
I20260709 12:13:49.346487 21403 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:49.346985 21403 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:49.347268 12496 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:13:49.311782 21710 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:13:49.312652 21099 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
W20260709 12:13:49.313109 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 47523.1us
I20260709 12:13:49.285506 21577 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:49.314141 21708 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.292217 12427 catalog_manager.cc:5697] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 16 to 17, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 17 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:13:49.322861 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:49.323032 21701 consensus_queue.cc:237] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:49.324366 21587 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:49.320168 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49075.7us
I20260709 12:13:49.331478 12646 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:49.295030 21586 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.331138 21707 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:49.361745 21707 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:49.341887 21722 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:49.341744 21718 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:49.356824 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.356602 12671 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 27.
I20260709 12:13:49.351554 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:49.341696 21721 raft_consensus.cc:697] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:49.339540 21465 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:13:49.381593 12674 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 26. Current term is 27. Ops: []
W20260709 12:13:49.384543 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:49.407908 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 159246us
I20260709 12:13:49.410547 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:49.548229 12494 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:49.346516 21716 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:49.360128 21099 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:13:49.552079 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 141613us
I20260709 12:13:49.368763 21722 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:49.561275 12427 catalog_manager.cc:5697] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 15 to 17, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 17 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:13:49.574647 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.352393 21399 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.203s
I20260709 12:13:49.328009 21702 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:49.375298 20913 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.216s
I20260709 12:13:49.591854 21711 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:13:49.640198 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4058) took 1449 ms. Trace:
I20260709 12:13:49.556476 21215 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:49.640761 12603 rpcz_store.cc:276] 0709 12:13:48.190563 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.452083 (+261520us) service_pool.cc:224] Handling call
0709 12:13:48.452459 (+   376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:48.494028 (+ 41569us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:48.494062 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:48.569969 (+ 75907us) log.cc:844] Serialized 26 byte log entry
0709 12:13:48.973648 (+403679us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:48.973708 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.973785 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:49.639889 (+666104us) raft_consensus.cc:1685] finished
0709 12:13:49.639921 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:49.640118 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17530,"raft.queue_time_us":23857,"spinlock_wait_cycles":7039872,"thread_start_us":335049,"threads_started":1,"wal-append.queue_time_us":469420}
I20260709 12:13:49.375463 21438 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.199s
I20260709 12:13:49.369657 21586 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:49.687220 21729 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.095s	user 0.001s	sys 0.000s
I20260709 12:13:49.688668 21586 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:49.677551 21711 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:49.690315 21465 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.690549 21711 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:49.691066 21465 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:49.694540 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 146539us
I20260709 12:13:49.685864 20913 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:13:49.695734 20913 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:49.696925 20913 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.15, Last appended by leader: 15, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:49.641069 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:49.563472 21722 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:49.683221 21215 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:49.683310 21698 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.684671 21399 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:49.639086 21358 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:49.697829 12494 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:49.755112 12677 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:49.707273 13337 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: [25.19-25.19]
I20260709 12:13:49.715699 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3573) took 1477 ms. Trace:
I20260709 12:13:49.794006 12676 rpcz_store.cc:276] 0709 12:13:48.204260 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.356535 (+152275us) service_pool.cc:224] Handling call
0709 12:13:48.372326 (+ 15791us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:48.559142 (+186816us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:48.559189 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:48.635837 (+ 76648us) log.cc:844] Serialized 26 byte log entry
0709 12:13:48.939057 (+303220us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:48.939123 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:48.939227 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:49.681556 (+742329us) raft_consensus.cc:1685] finished
0709 12:13:49.681578 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:49.681729 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":12894,"raft.run_cpu_time_us":229,"raft.run_wall_time_us":25116,"thread_start_us":179195,"threads_started":1,"wal-append.queue_time_us":322159}
I20260709 12:13:49.557013 21696 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.124s
I20260709 12:13:49.682500 12427 catalog_manager.cc:5697] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 12 to 15, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 15 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:13:49.693936 21465 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:49.802716 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 441685us
I20260709 12:13:49.714990 20913 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:13:49.827548 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:49.873045 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.873118 12643 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:49.558087 21708 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:49.754330 21737 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:49.879383 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:49.881485 12677 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:49.699796 21700 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:49.698585 21438 consensus_queue.cc:1059] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.174s
W20260709 12:13:49.883400 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 322080us
I20260709 12:13:49.749758 21591 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:49.893080 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:49.690026 21707 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:49.798133 21215 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:49.868309 21099 raft_consensus.cc:697] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:49.699219 21717 consensus_queue.cc:1059] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.509s
W20260709 12:13:49.891451 20827 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.540s	user 0.072s	sys 0.109s
W20260709 12:13:49.897925 13305 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.529s	user 0.006s	sys 0.032s
W20260709 12:13:49.904024 13305 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.535s	user 0.006s	sys 0.032s
W20260709 12:13:49.904486 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 535762us
I20260709 12:13:49.797380 21358 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:49.699086 21719 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.510s
I20260709 12:13:49.699707 21739 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.295s
I20260709 12:13:49.901365 12676 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:49.811751 21399 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:49.698917 21727 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.187s
W20260709 12:13:49.906085 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:49.894531 21478 consensus_meta.cc:287] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.963s	user 0.016s	sys 0.000s
I20260709 12:13:49.708397 21716 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:49.899868 12572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.021s	sys 0.039s
I20260709 12:13:49.887804 21700 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:49.878234 21465 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.029s
W20260709 12:13:49.937407 12572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.679s	user 0.021s	sys 0.039s
I20260709 12:13:49.870131 21698 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:49.907953 21730 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:49.910964 20827 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.072s	sys 0.110s
W20260709 12:13:49.942832 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49900.8us
W20260709 12:13:49.959978 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 57588.5us
I20260709 12:13:49.961555 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:49.965415 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 267663us
I20260709 12:13:49.976958 12677 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:13:49.942106 21700 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:13:49.951611 21619 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.011s	user 0.017s	sys 0.000s
W20260709 12:13:49.991995 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:50.037726 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 48125.2us
I20260709 12:13:49.995570 21727 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.038981 20827 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:50.066886 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 101395us
I20260709 12:13:50.058947 21746 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:50.004272 21619 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:49.941883 21721 consensus_queue.cc:237] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 20.12, Last appended by leader: 12, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.066531 21731 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:50.073572 21702 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:50.090830 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.034452 21719 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:50.077028 21478 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:50.094229 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:49.991788 21730 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.107543 21742 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:13:50.092136 13302 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:50.104971 21753 raft_consensus.cc:3055] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:50.107226 21740 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.133869 21740 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:50.134935 21734 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.261s	user 0.004s	sys 0.006s
I20260709 12:13:50.135728 21707 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.107888 21738 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:50.104462 21744 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.064s
I20260709 12:13:50.106606 21547 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:50.109053 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:50.092918 21438 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:50.106474 21591 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:50.080794 20827 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.075268 21727 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.075291 21725 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.021s
W20260709 12:13:50.156042 12646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.002s	sys 0.006s
W20260709 12:13:50.156577 12646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.595s	user 0.002s	sys 0.006s
W20260709 12:13:50.148568 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 57173us
W20260709 12:13:50.140985 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 179585us
I20260709 12:13:50.129920 12597 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 28: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:50.152688 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1134) took 1078 ms. Trace:
I20260709 12:13:50.153558 21661 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.154511 21742 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:13:50.154891 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 63073.5us
I20260709 12:13:50.067322 21099 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:50.170902 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:50.157457 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:50.171267 21661 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.177651 20827 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.178254 21661 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.177675 21619 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:50.178906 21661 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:13:50.158387 21719 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.179852 21756 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.180392 21756 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:50.182366 21358 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:50.182541 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1127) took 1614 ms. Trace:
W20260709 12:13:50.182708 21733 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.241s	user 0.001s	sys 0.001s
I20260709 12:13:50.157336 12531 rpcz_store.cc:276] 0709 12:13:49.074170 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.074325 (+   155us) service_pool.cc:224] Handling call
0709 12:13:49.139315 (+ 64990us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:49.139540 (+   225us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:13:49.406378 (+266838us) log.cc:844] Serialized 14 byte log entry
0709 12:13:50.152133 (+745755us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:50.152215 (+    82us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:13:50.152288 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.152408 (+   120us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:50.152591 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7690,"prepare.queue_time_us":80955,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":49,"raft.queue_time_us":187560,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":8634,"thread_start_us":926940,"threads_started":3,"wal-append.queue_time_us":578102}
I20260709 12:13:50.179955 21631 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.147787 21438 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:50.182996 12525 rpcz_store.cc:276] 0709 12:13:48.568118 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.568310 (+   192us) service_pool.cc:224] Handling call
0709 12:13:48.599924 (+ 31614us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:48.674387 (+ 74463us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:48.674448 (+    61us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:48.674624 (+   176us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:48.674724 (+   100us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.182294 (+1507570us) spinlock_profiling.cc:243] Waited 74 ms on lock 0x7b5400105ed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:50.182349 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:50.182472 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":170282880}
I20260709 12:13:50.149665 21465 raft_consensus.cc:3055] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:50.184866 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.196383 12680 raft_consensus.cc:2410] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 25 because replica has last-logged OpId of term: 23 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 10.
W20260709 12:13:50.196382 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.189546 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3535) took 1526 ms. Trace:
W20260709 12:13:50.195209 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 869490us
I20260709 12:13:50.139285 21753 raft_consensus.cc:740] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:50.197788 12646 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:50.157943 21215 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:50.163174 21731 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:13:50.163911 21697 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:50.132429 21730 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.162024 21751 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:13:50.161794 21703 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:50.136588 21740 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.182808 21719 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.250435 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:50.261169 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:50.149540 21478 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:50.264302 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 63547.6us
I20260709 12:13:50.187461 21399 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:50.268318 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 70623.8us
I20260709 12:13:50.187106 21551 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:13:50.188911 21255 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:50.194278 21761 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.163899 21708 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:50.280426 12496 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:50.249907 21702 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.286026 12646 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:50.138530 21738 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:50.197176 12526 rpcz_store.cc:276] 0709 12:13:48.663414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.663543 (+   129us) service_pool.cc:224] Handling call
0709 12:13:48.663915 (+   372us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:48.697630 (+ 33715us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:13:49.036876 (+339246us) log.cc:844] Serialized 14 byte log entry
0709 12:13:49.197178 (+160302us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:50.186867 (+989689us) spinlock_profiling.cc:243] Waited 2.93 ms on lock 0x7b2c00058028. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:13:50.187268 (+   401us) log.cc:844] Serialized 26 byte log entry
0709 12:13:50.187446 (+   178us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:50.187484 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.187555 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:50.189243 (+  1688us) raft_consensus.cc:1685] finished
0709 12:13:50.189350 (+   107us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:50.189483 (+   133us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37083,"prepare.queue_time_us":333854,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":61,"raft.queue_time_us":101840,"raft.run_cpu_time_us":669,"raft.run_wall_time_us":30692,"spinlock_wait_cycles":6746496,"thread_start_us":402393,"threads_started":2,"wal-append.queue_time_us":251914}
I20260709 12:13:50.289512 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.195884 21465 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:50.278463 21759 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:50.339787 12680 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
W20260709 12:13:50.340402 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 54697.5us
I20260709 12:13:50.341645 12646 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:50.281549 21756 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:50.274854 21751 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:50.297897 21754 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.169s
I20260709 12:13:50.281795 21591 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:50.367621 21591 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:50.368125 21591 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:50.298702 21755 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.295317 21255 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:50.371480 21755 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:50.372408 21760 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.372074 21255 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:50.372983 21760 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:50.373649 21255 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:13:50.375306 13304 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.882s (SENT) [suppressed 5 similar messages]
I20260709 12:13:50.375491 21760 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:50.299813 21736 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.103s	user 0.000s	sys 0.000s
W20260709 12:13:50.300251 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 107018us
W20260709 12:13:50.377080 21644 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.059s	user 0.013s	sys 0.000s
I20260709 12:13:50.376812 21751 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:50.365424 21753 consensus_queue.cc:260] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.354349 12571 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:50.379271 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.318742 21730 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:50.319586 21711 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.628s	user 0.006s	sys 0.006s
W20260709 12:13:50.320123 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 162766us
W20260709 12:13:50.320919 21748 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.254s	user 0.001s	sys 0.000s
I20260709 12:13:50.322417 21722 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:50.325654 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.279091 21215 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:13:50.286307 21740 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
W20260709 12:13:50.336081 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 55713.5us
I20260709 12:13:50.312466 21438 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.286046 21698 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.350092 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.356520 21764 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:13:50.395722 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:50.396113 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:50.345907 21708 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.277086 21358 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 12:13:50.353596 21551 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.279534 21703 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.322989 21468 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.360659 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.357829 21478 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.362790 21727 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.312834 21766 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.086s
W20260709 12:13:50.363523 21403 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.016s	user 0.014s	sys 0.000s
I20260709 12:13:50.354426 21477 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.346213 21767 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:50.357995 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.350178 21759 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.388805 12676 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 24: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:50.399729 21723 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:50.396996 13331 raft_consensus.cc:2410] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 30 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 17.
I20260709 12:13:50.390758 12646 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:50.402706 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:50.378554 21753 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:50.354051 21702 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.414098 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.383193 21644 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.387858 21755 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:50.391984 21740 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:50.484540 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:50.514739 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:50.391865 21746 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:50.403169 21215 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:50.482990 21767 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:50.524547 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 149267us
I20260709 12:13:50.474164 21764 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:50.513813 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 123170us
I20260709 12:13:50.513942 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:50.514314 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.437134 12671 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
I20260709 12:13:50.531351 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:50.532012 12599 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
I20260709 12:13:50.515530 13334 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:13:50.392900 21711 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:50.388556 21730 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.522739 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.483606 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:50.507454 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:50.536919 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.492817 21723 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:50.389514 21761 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:50.533082 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.529906 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.534289 12673 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:50.436105 21708 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:50.505668 21763 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:50.520633 21740 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:50.521621 21438 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:50.540782 12676 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:50.558171 12603 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: [28.15-28.15]
I20260709 12:13:50.545485 12598 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:50.495044 21403 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.500937 21551 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:50.549700 12643 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:50.527007 21767 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:50.524873 21760 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.565485 13337 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 28: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:50.552798 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.527551 21701 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.552204 13339 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:13:50.547150 21723 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:50.555822 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:50.437780 21465 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.544796 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.527954 21765 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:50.542735 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:50.552119 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:50.562610 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:50.578560 12672 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:50.562513 21714 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:50.576206 12605 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:50.563140 21702 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.502439 21477 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:50.566866 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.574066 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:50.549754 21746 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.577000 12680 raft_consensus.cc:2410] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 26 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:13:50.585510 20913 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.024s
I20260709 12:13:50.580775 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:50.581079 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.581585 21761 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:50.582664 21722 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:50.509236 21742 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:50.605666 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 122150us
I20260709 12:13:50.623844 21477 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.604615 21551 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.610538 13334 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:50.626204 21746 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.558051 21763 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.626935 21738 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.628337 21759 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.628875 21763 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.629375 13333 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:13:50.596827 21718 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:50.611270 12676 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 29.
I20260709 12:13:50.631219 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.543933 21700 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.610562 12599 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:13:50.632119 12494 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:50.586611 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:50.567152 21099 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:50.568225 21507 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:50.626747 21756 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:13:50.639034 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52542.3us
I20260709 12:13:50.585665 21714 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:50.642447 12676 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 28: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:50.643395 12494 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:50.644834 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.605450 21708 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:50.647840 21641 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:50.594784 21764 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.648257 21644 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:50.651413 21591 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.581601 21730 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.680794 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:50.602938 21703 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.658032 21507 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:50.683331 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.684278 12676 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:50.685643 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.689153 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.690421 12676 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:13:50.691810 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:50.692881 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.693521 12571 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:50.694486 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.597981 21215 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:50.697688 21750 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.501s	user 0.001s	sys 0.002s
I20260709 12:13:50.576890 21701 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:50.603299 21711 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:50.597795 21403 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:50.574612 21760 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.674559 21438 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:50.691700 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:50.696229 12676 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:13:50.700043 21534 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Stepping down as leader of term 25
W20260709 12:13:50.737020 12597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1027) took 1554 ms (client timeout 1565 ms). Trace:
I20260709 12:13:50.702289 21586 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:13:50.654352 21700 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.710718 13332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 1060) took 2564 ms. Trace:
I20260709 12:13:50.712540 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3502) took 2718 ms. Trace:
I20260709 12:13:50.714003 21434 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:50.659771 21714 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:13:50.718358 21712 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.126s	user 0.008s	sys 0.003s
I20260709 12:13:50.679525 21707 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:50.675240 21744 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:50.676347 21756 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.763391 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.722857 21665 raft_consensus.cc:3055] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:50.724424 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4044) took 2546 ms. Trace:
I20260709 12:13:50.749181 21738 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:50.642076 21099 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:50.765905 12602 rpcz_store.cc:276] 0709 12:13:48.178420 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.278330 (+ 99910us) service_pool.cc:224] Handling call
0709 12:13:48.278902 (+   572us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:48.363119 (+ 84217us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:13:48.363509 (+   390us) log.cc:844] Serialized 14 byte log entry
0709 12:13:48.750858 (+387349us) log.cc:844] Serialized 14 byte log entry
0709 12:13:48.772561 (+ 21703us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b18000a05a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:13:48.773725 (+  1164us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:48.773776 (+    51us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:48.773841 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.724152 (+1950311us) spinlock_profiling.cc:243] Waited 80.9 ms on lock 0x7b5400310bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:50.724210 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:50.724361 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6866,"prepare.queue_time_us":382,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":85,"raft.queue_time_us":22406,"raft.run_cpu_time_us":271,"raft.run_wall_time_us":5851,"spinlock_wait_cycles":198657280,"thread_start_us":386194,"threads_started":1,"wal-append.queue_time_us":499212,"wal-append.run_cpu_time_us":2016,"wal-append.run_wall_time_us":1085967}
I20260709 12:13:50.656088 21761 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:50.661163 21763 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:50.658905 12671 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26.
W20260709 12:13:50.741860 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 199207us
I20260709 12:13:50.782168 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:50.782259 13304 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:50.782779 12671 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:50.745160 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:50.741322 12597 rpcz_store.cc:269] 0709 12:13:49.170655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.180051 (+  9396us) service_pool.cc:224] Handling call
0709 12:13:50.709907 (+1529856us) spinlock_profiling.cc:243] Waited 911 ms on lock 0x7b540022f6d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:50.725373 (+ 15466us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2095025408}
W20260709 12:13:50.751983 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.748155 13332 rpcz_store.cc:276] 0709 12:13:48.146788 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.146999 (+   211us) service_pool.cc:224] Handling call
0709 12:13:48.147813 (+   814us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:48.363770 (+215957us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.710356 (+2346586us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b54006419d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:50.710615 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":493801728}
I20260709 12:13:50.736858 21577 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:50.748368 12600 rpcz_store.cc:276] 0709 12:13:47.993586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:47.996323 (+  2737us) service_pool.cc:224] Handling call
0709 12:13:47.996827 (+   504us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:48.162621 (+165794us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:48.162652 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:48.163431 (+   779us) log.cc:844] Serialized 52 byte log entry
0709 12:13:48.608664 (+445233us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:48.615776 (+  7112us) log.cc:844] Serialized 14 byte log entry
0709 12:13:49.007293 (+391517us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.711714 (+1704421us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54001946d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:50.711795 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:50.711866 (+    71us) raft_consensus.cc:1685] finished
0709 12:13:50.712025 (+   159us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:50.712256 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6519,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":80209,"raft.run_cpu_time_us":263,"raft.run_wall_time_us":263,"spinlock_wait_cycles":244140928,"thread_start_us":835878,"threads_started":2,"wal-append.queue_time_us":610168,"wal-append.run_cpu_time_us":4107,"wal-append.run_wall_time_us":1155843}
I20260709 12:13:50.741114 21534 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:50.748360 21586 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:50.748795 21719 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:50.798538 21719 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.800239 21249 raft_consensus.cc:3055] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:50.800786 21249 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:50.801409 21719 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.802237 21586 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:13:50.804117 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:50.805380 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:50.763773 21707 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.808369 21707 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.809477 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:50.811829 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3590) took 1897 ms. Trace:
I20260709 12:13:50.812852 21744 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:13:50.813372 21744 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:50.813766 21744 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:13:50.815248 21760 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:50.816273 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3588) took 2379 ms. Trace:
I20260709 12:13:50.765674 21665 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:50.772449 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:50.771184 21099 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:50.760859 21749 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:13:50.744460 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52847.6us
I20260709 12:13:50.783499 13330 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:13:50.795565 21534 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.796968 21766 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:13:50.786348 21577 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:50.770720 21738 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:50.805939 12605 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:50.660247 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:50.820231 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:50.812649 12679 rpcz_store.cc:276] 0709 12:13:48.914307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.934319 (+ 20012us) service_pool.cc:224] Handling call
0709 12:13:50.811405 (+1877086us) spinlock_profiling.cc:243] Waited 123 ms on lock 0x7b54003d4ad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:50.811636 (+   231us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":284000000}
I20260709 12:13:50.743818 21438 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.851475 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:50.853154 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:50.853719 12602 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:50.833235 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:50.834589 12496 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:50.866497 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1025) took 1727 ms. Trace:
I20260709 12:13:50.867105 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3583) took 2431 ms. Trace:
I20260709 12:13:50.837073 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.842350 13338 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:50.856832 21746 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:50.836184 21749 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:50.843115 21577 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:50.859367 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.859597 21701 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:50.774765 21761 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:50.866922 12598 rpcz_store.cc:276] 0709 12:13:49.138995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.183922 (+ 44927us) service_pool.cc:224] Handling call
0709 12:13:49.216958 (+ 33036us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:50.561038 (+1344080us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.866154 (+305116us) spinlock_profiling.cc:243] Waited 1.33 s on lock 0x7b54002f1054. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:50.866437 (+   283us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3054785920}
I20260709 12:13:50.781348 21763 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:50.816977 12675 rpcz_store.cc:276] 0709 12:13:48.436413 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.312492 (+876079us) service_pool.cc:224] Handling call
0709 12:13:49.313038 (+   546us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:49.316924 (+  3886us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:13:49.317309 (+   385us) log.cc:844] Serialized 14 byte log entry
0709 12:13:49.884941 (+567632us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:49.885004 (+    63us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:49.891788 (+  6784us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.816020 (+924232us) spinlock_profiling.cc:243] Waited 3.55 ms on lock 0x7b54003bd154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:50.816074 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:50.816203 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39503,"prepare.queue_time_us":744,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":55,"raft.queue_time_us":291314,"raft.run_cpu_time_us":195,"raft.run_wall_time_us":52353,"spinlock_wait_cycles":8165248,"thread_start_us":526662,"threads_started":2,"wal-append.queue_time_us":574973}
I20260709 12:13:50.867890 12674 rpcz_store.cc:276] 0709 12:13:48.434735 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.201991 (+767256us) service_pool.cc:224] Handling call
0709 12:13:49.379043 (+177052us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:49.382242 (+  3199us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.865874 (+1483632us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54002c5e50. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:50.866061 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":405968640}
I20260709 12:13:50.848827 21766 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 27 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:50.873912 12676 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 23: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:50.868584 21640 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.876479 12494 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:13:50.859877 21781 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:13:50.878700 12678 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:50.925305 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.845785 21738 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:50.853897 21438 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.917982 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:50.903218 12571 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:50.936884 12602 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 28: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:50.922310 12671 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:50.859684 21249 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 20.15, Last appended by leader: 15, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:50.939337 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:50.927886 12678 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 27: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:50.927817 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.841179 21507 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:50.932147 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 55743.5us
I20260709 12:13:50.941890 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:50.891731 21306 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:13:50.907460 21505 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:50.912456 21761 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:50.876039 21749 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:50.922544 21763 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:50.921211 21534 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:50.949494 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:50.950757 12598 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 28: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:13:50.954873 21782 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.093s	user 0.000s	sys 0.001s
I20260709 12:13:50.938656 21735 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:50.959255 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4258) took 1368 ms. Trace:
I20260709 12:13:50.937635 21719 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:50.986408 21719 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:50.986845 21719 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:50.988080 21763 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.960028 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3585) took 2524 ms. Trace:
I20260709 12:13:50.990371 21763 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:50.990190 21767 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:50.992113 12572 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:50.992468 21767 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:50.993444 12572 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:51.002979 12672 rpcz_store.cc:276] 0709 12:13:48.435341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.270720 (+835379us) service_pool.cc:224] Handling call
0709 12:13:49.296105 (+ 25385us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:50.579067 (+1282962us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.959809 (+380742us) spinlock_profiling.cc:243] Waited 1.28 s on lock 0x7b540016f3d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:50.959975 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2948965632}
I20260709 12:13:51.006101 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:50.951543 21306 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:50.962725 12678 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:50.965070 12605 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:50.965728 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:50.973253 12675 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:50.961824 21249 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:50.982704 21766 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:50.984521 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:51.020084 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4255) took 1466 ms. Trace:
I20260709 12:13:50.934762 21640 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:50.985493 21735 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:50.972337 21703 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.014s
I20260709 12:13:51.008446 12679 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:13:50.977532 12680 rpcz_store.cc:276] 0709 12:13:49.590785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.958384 (+1367599us) service_pool.cc:224] Handling call
0709 12:13:50.958735 (+   351us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:50.958937 (+   202us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:50.958963 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:50.959005 (+    42us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:50.959034 (+    29us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:50.959089 (+    55us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:50.959198 (+   109us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:50.957862 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.003842 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:51.014655 12600 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:51.018045 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:51.020498 12678 rpcz_store.cc:276] 0709 12:13:49.553401 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.962011 (+1408610us) service_pool.cc:224] Handling call
0709 12:13:50.962443 (+   432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:51.019785 (+ 57342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.020021 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:50.969411 21591 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:51.022347 21763 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:13:50.957832 21744 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:50.963698 21641 consensus_queue.cc:1175] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" responder_term: 21 status { last_received { term: 21 index: 16 } last_committed_idx: 14 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 20. Current term is 21. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:13:51.039745 13336 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:13:51.040508 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:51.040956 12675 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:50.937378 21665 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:51.068163 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 74808.4us
I20260709 12:13:51.043186 12678 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 27. Current term is 28. Ops: []
I20260709 12:13:51.067999 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4260) took 1352 ms. Trace:
I20260709 12:13:51.069734 12675 rpcz_store.cc:276] 0709 12:13:49.716187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.019903 (+1303716us) service_pool.cc:224] Handling call
0709 12:13:51.040732 (+ 20829us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:51.067676 (+ 26944us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.067913 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:51.041033 21255 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.667s	user 0.012s	sys 0.001s
I20260709 12:13:51.049005 12598 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:50.964322 21505 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:51.059309 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:13:51.046980 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 212506us
I20260709 12:13:51.039651 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4257) took 1485 ms. Trace:
I20260709 12:13:51.073333 12680 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:13:51.017272 21306 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:13:51.076184 12679 rpcz_store.cc:276] 0709 12:13:49.554027 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.962410 (+1408383us) service_pool.cc:224] Handling call
0709 12:13:51.007921 (+ 45511us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:51.039386 (+ 31465us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.039595 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:51.077647 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:51.078094 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.067898 13331 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
W20260709 12:13:51.081910 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.082258 12680 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 19 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:13:51.025000 21735 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:51.026229 21434 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:13:51.069676 21753 consensus_meta.cc:287] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.557s	user 0.008s	sys 0.004s
I20260709 12:13:51.084128 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:51.084220 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:51.038057 21670 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:51.084863 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:51.041463 21576 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:13:51.085124 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:51.097429 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:51.085995 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:51.086194 12599 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:51.086684 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:51.087535 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:51.089347 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:51.037420 21745 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:51.085321 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:51.094530 12494 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:51.100884 12530 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:51.101665 12680 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 26. Current term is 27. Ops: []
I20260709 12:13:51.090837 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:51.018841 21740 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:51.163044 21099 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:51.165495 21099 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:51.166572 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 77351.6us
I20260709 12:13:51.167644 12572 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:51.093382 12496 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:51.038398 21438 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.057s
I20260709 12:13:51.102447 21763 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:51.105260 21756 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:51.146903 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:51.147672 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:51.168557 21730 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:51.191861 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:51.192629 12529 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:13:51.191952 21714 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:51.193665 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:51.024968 21755 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.067s
I20260709 12:13:51.134563 21789 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:13:51.153017 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:51.142302 21507 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:51.154356 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:51.196465 12524 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:51.155200 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:51.157749 21723 consensus_meta.cc:287] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.576s	user 0.009s	sys 0.000s
I20260709 12:13:51.158824 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:51.159171 21505 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:51.159683 21670 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:51.130594 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:51.105067 12494 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:51.208896 12598 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:13:51.209105 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 104149us
I20260709 12:13:51.109495 13332 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:51.175663 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3600) took 1043 ms. Trace:
W20260709 12:13:51.180088 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 86814.8us
I20260709 12:13:51.180281 12600 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:51.130261 21387 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:50.973315 12671 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:45864
I20260709 12:13:51.139339 21719 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:51.183632 21740 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:51.184526 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:51.187711 21738 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:51.179366 21763 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:51.225600 12529 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:13:51.227098 20913 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:13:51.210793 12680 rpcz_store.cc:276] 0709 12:13:50.132311 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.101114 (+968803us) service_pool.cc:224] Handling call
0709 12:13:51.101466 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:51.175322 (+ 73856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.175588 (+   266us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:51.215162 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:51.182806 21434 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:13:51.218513 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 222849us
I20260709 12:13:51.220376 12671 raft_consensus.cc:606] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:51.210108 12606 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 21.
W20260709 12:13:51.167356 21717 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.212s	user 0.016s	sys 0.016s
I20260709 12:13:51.108019 21753 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:51.208456 21099 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:51.219368 21438 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:51.240760 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:51.238945 12526 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:13:51.240330 21438 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:51.236703 21744 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:51.168756 21745 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:51.237782 21717 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.292s	user 0.016s	sys 0.016s
I20260709 12:13:51.244464 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:51.253669 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:51.246572 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:51.249930 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:51.243530 21099 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:51.256161 21099 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:51.256422 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:51.257421 12671 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:13:51.243049 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:51.247584 21753 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:51.225999 21789 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:51.258615 21306 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:51.259598 21789 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:51.260545 21753 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:51.260870 21306 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:51.262488 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:13:51.262755 12522 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 30: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:51.224457 21387 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:51.265089 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:51.265537 21387 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:51.266188 21756 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:13:51.246157 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 100164us
I20260709 12:13:51.232470 21763 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:51.268606 21745 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:51.251549 13339 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:13:51.241032 21507 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.271344 21507 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:51.273751 21507 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:51.271359 12606 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 27: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:51.252460 13304 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:51.210546 21723 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:51.286729 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 102277us
I20260709 12:13:51.291831 12572 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:51.243793 21703 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:13:51.294224 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:51.250931 21547 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:51.251077 21438 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:51.298523 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:51.306195 12671 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:13:51.290352 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:51.315366 13335 raft_consensus.cc:2410] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 22 because replica has last-logged OpId of term: 20 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 18 index: 14.
I20260709 12:13:51.322419 12605 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:51.327226 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4259) took 1727 ms. Trace:
I20260709 12:13:51.323961 13332 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:13:51.315713 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3598) took 1184 ms. Trace:
I20260709 12:13:51.330974 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:51.333210 12529 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:51.248597 21577 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:51.303771 21714 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:51.310161 21306 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.335898 12643 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:51.336246 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:51.316658 21745 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:51.350775 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:51.345553 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:51.356432 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:51.357704 12572 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:13:51.362080 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.305215 21586 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:51.365535 12572 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:51.355541 12671 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26.
I20260709 12:13:51.324996 21099 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:51.363014 21730 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:51.356578 21703 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.019s
I20260709 12:13:51.370599 21576 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:51.357648 21306 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:51.371663 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:51.375314 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3602) took 1116 ms. Trace:
I20260709 12:13:51.425181 13335 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:13:51.376139 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.327533 21756 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 12:13:51.379210 21737 raft_consensus.cc:3055] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:13:51.424924 12680 rpcz_store.cc:276] 0709 12:13:50.259161 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.231959 (+972798us) service_pool.cc:224] Handling call
0709 12:13:51.244350 (+ 12391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:51.247034 (+  2684us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:13:51.261575 (+ 14541us) log.cc:844] Serialized 14 byte log entry
0709 12:13:51.374930 (+113355us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:51.374985 (+    55us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:51.375037 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.375105 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:51.375251 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":15071,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":170,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":107,"thread_start_us":112907,"threads_started":1}
I20260709 12:13:51.380297 21719 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:51.381234 21738 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:51.388042 21586 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:51.439169 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:51.390287 21722 raft_consensus.cc:2764] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election decision vote started in defunct term 26: won
I20260709 12:13:51.373055 21761 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:51.439754 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:51.392282 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:51.395066 21465 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:51.397223 21717 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:13:51.401376 13337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38472 (request call id 1010) took 1204 ms. Trace:
I20260709 12:13:51.361249 21701 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:51.356845 21745 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:51.412029 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:51.420143 12598 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:51.429103 12529 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:51.438475 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:51.388883 12602 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 29. Current term is 30. Ops: [29.17-29.17]
I20260709 12:13:51.439999 12530 raft_consensus.cc:2410] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 23 because replica has last-logged OpId of term: 20 index: 12, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:13:51.398216 21723 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:51.441499 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:51.437590 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:51.444839 12522 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:51.446568 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:51.450963 12671 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 20.
I20260709 12:13:51.447345 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:51.443773 13337 rpcz_store.cc:276] 0709 12:13:50.197391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.197511 (+   120us) service_pool.cc:224] Handling call
0709 12:13:51.401022 (+1203511us) spinlock_profiling.cc:243] Waited 278 ms on lock 0x7b54006450d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:51.401244 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":640386432}
I20260709 12:13:51.423686 21670 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:51.440528 12679 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 32.
W20260709 12:13:51.471699 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.528895 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:51.433215 21737 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:51.453466 12569 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:51.424521 21255 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:13:51.447902 12525 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:13:51.619813 12527 raft_consensus.cc:2368] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
I20260709 12:13:51.620865 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:51.622174 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:51.620618 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 103833us
I20260709 12:13:51.623637 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:51.634574 12530 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:51.535467 12680 raft_consensus.cc:2393] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 25: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:51.624595 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4074) took 1152 ms. Trace:
I20260709 12:13:51.636281 12600 rpcz_store.cc:276] 0709 12:13:50.472321 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.042275 (+569954us) service_pool.cc:224] Handling call
0709 12:13:51.042634 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:51.219511 (+176877us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.624225 (+404714us) spinlock_profiling.cc:243] Waited 130 ms on lock 0x7b540022ecd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:51.624515 (+   290us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":298951040}
I20260709 12:13:51.443244 21717 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:51.631810 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 178453us
I20260709 12:13:51.444195 21249 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:13:51.635366 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:51.454749 21766 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:51.450837 21745 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:51.447510 21701 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:51.649603 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4077) took 1084 ms. Trace:
I20260709 12:13:51.637959 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:51.648661 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:51.607995 21762 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:51.659809 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:51.657037 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:51.633426 21735 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.530s	user 0.006s	sys 0.009s
I20260709 12:13:51.618750 21719 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.453131 21749 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:51.637916 21756 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:51.672274 21756 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:51.651432 21434 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:51.675463 21762 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:51.621730 21724 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 12:13:51.649809 21547 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:13:51.683183 12673 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3606) took 1329 ms (client timeout 1529 ms). Trace:
I20260709 12:13:51.687575 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:51.651190 21766 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:51.644068 21723 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:51.718807 12680 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:51.696935 21763 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:51.697378 21387 consensus_queue.cc:1059] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:13:51.721308 21766 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:51.724864 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:51.725535 12680 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:13:51.699810 21804 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:13:51.658110 21551 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Leader pre-election won for term 21
I20260709 12:13:51.729472 21744 consensus_queue.cc:1059] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 23.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:13:51.730528 12522 raft_consensus.cc:2368] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 28. Current term is 29.
I20260709 12:13:51.680912 21505 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:13:51.690301 21713 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.668080 21769 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:51.708854 21703 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:51.715328 12673 rpcz_store.cc:269] 0709 12:13:50.353564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.357594 (+  4030us) service_pool.cc:224] Handling call
0709 12:13:51.682996 (+1325402us) spinlock_profiling.cc:243] Waited 56.2 ms on lock 0x7b54002c4554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:51.683121 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":129309952}
I20260709 12:13:51.653883 21701 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:51.696605 12494 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:51.719462 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:51.722764 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:51.631316 21670 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.039s
W20260709 12:13:51.742664 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46146.3us
I20260709 12:13:51.744313 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:51.693445 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:51.729012 21753 raft_consensus.cc:3055] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:51.729750 21551 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:51.701867 21507 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:13:51.703461 21534 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:13:51.707132 21717 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:51.681725 21735 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:51.741649 12529 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:13:51.748942 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:51.755582 12673 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 24: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:51.733357 21505 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:51.691812 21722 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:51.751734 12494 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:51.752506 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 59186.5us
I20260709 12:13:51.695158 21719 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:51.669479 21737 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:51.745364 21577 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:51.758260 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.759655 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:51.748709 21764 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:51.751802 21551 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:51.747013 21753 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:51.738706 21713 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:51.783664 12598 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:51.758239 21505 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:13:51.790544 12680 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:51.784140 12646 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:51.758064 21215 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:13:51.782927 12600 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:51.795562 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:51.800144 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:51.749400 21703 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:51.800396 12646 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:51.813153 12529 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 28.
I20260709 12:13:51.758126 21755 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.041s
I20260709 12:13:51.782702 21791 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:51.785760 21764 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:51.796758 21534 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.041s
I20260709 12:13:51.819541 21387 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:51.839135 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:51.839813 12680 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:51.838333 21403 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:51.839332 21763 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:51.815198 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:51.846866 12530 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:13:51.816303 12646 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:51.820753 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:51.850513 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.829512 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:51.843267 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 208176us
I20260709 12:13:51.851086 12528 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26.
I20260709 12:13:51.854238 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:51.855093 12673 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 25
I20260709 12:13:51.805850 21761 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:51.813781 21714 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:13:51.823102 21801 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:13:51.813560 21587 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:51.822386 21764 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:51.843822 21737 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:51.874795 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:51.822454 21749 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:51.837972 21703 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:51.835912 21713 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:51.825794 21735 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:51.844367 21403 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:51.880616 12529 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:13:51.888850 21766 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:51.865078 21438 consensus_meta.cc:287] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.537s	user 0.008s	sys 0.004s
I20260709 12:13:51.892467 12680 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:13:51.860982 21738 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:13:51.853652 21387 raft_consensus.cc:697] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:51.899968 12673 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:51.889896 21723 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:51.902384 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:51.862633 21714 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:51.914345 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:51.913424 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:51.916965 12529 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:13:51.896265 21587 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:51.919497 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:51.920094 12673 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:13:51.920578 21587 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:51.921312 21551 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:51.923027 21713 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:51.923588 21713 raft_consensus.cc:697] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:51.924619 21713 consensus_queue.cc:237] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 23.11, Last appended by leader: 11, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:51.328627 12672 rpcz_store.cc:276] 0709 12:13:49.599630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.003995 (+1404365us) service_pool.cc:224] Handling call
0709 12:13:51.025428 (+ 21433us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:51.025632 (+   204us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:13:51.229250 (+203618us) log.cc:844] Serialized 14 byte log entry
0709 12:13:51.326256 (+ 97006us) log.cc:844] Serialized 14 byte log entry
0709 12:13:51.326889 (+   633us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:51.326938 (+    49us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:51.326985 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.327050 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:51.327169 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":120136,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":79,"raft.queue_time_us":244,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":7684,"thread_start_us":244749,"threads_started":2}
I20260709 12:13:51.923625 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1029) took 1191 ms. Trace:
I20260709 12:13:51.926846 21714 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:13:51.932135 13338 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38472 (request call id 1006) took 2611 ms (client timeout 1817 ms). Trace:
W20260709 12:13:51.932575 13338 rpcz_store.cc:269] 0709 12:13:49.320886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.321061 (+   175us) service_pool.cc:224] Handling call
0709 12:13:51.931928 (+2610867us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b5400690354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:51.932034 (+   106us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3411288960}
I20260709 12:13:51.934146 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:51.934707 13331 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 20. Current term is 21. Ops: []
I20260709 12:13:51.934832 12678 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:51.935868 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:51.936419 21766 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:51.937255 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4062) took 1531 ms. Trace:
I20260709 12:13:51.937628 12603 rpcz_store.cc:276] 0709 12:13:50.405533 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.434621 (+ 29088us) service_pool.cc:224] Handling call
0709 12:13:50.485339 (+ 50718us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:50.558882 (+ 73543us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.936990 (+1378108us) spinlock_profiling.cc:243] Waited 72.5 ms on lock 0x7b54002f2954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:51.937174 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":166712704}
I20260709 12:13:51.939033 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:51.941239 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4266) took 1472 ms. Trace:
I20260709 12:13:51.957918 12673 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:13:51.959017 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:51.963447 12673 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:51.946781 12600 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:51.964109 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4267) took 1488 ms. Trace:
I20260709 12:13:51.896176 21403 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:51.964596 12673 rpcz_store.cc:276] 0709 12:13:50.475933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.962998 (+1487065us) service_pool.cc:224] Handling call
0709 12:13:51.963288 (+   290us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:51.963894 (+   606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.964059 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:51.941646 12674 rpcz_store.cc:276] 0709 12:13:50.468794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.940470 (+1471676us) service_pool.cc:224] Handling call
0709 12:13:51.940755 (+   285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:51.940929 (+   174us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:51.940954 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:51.940996 (+    42us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:13:51.941026 (+    30us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.941074 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:51.941165 (+    91us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:51.926745 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:51.936581 12678 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 27: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:51.944504 12523 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:51.945595 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:51.962013 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 110515us
I20260709 12:13:51.835342 21753 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:51.971185 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 201090us
I20260709 12:13:51.957727 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1019) took 3759 ms. Trace:
I20260709 12:13:51.926467 12597 rpcz_store.cc:276] 0709 12:13:50.740060 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.382632 (+642572us) service_pool.cc:224] Handling call
0709 12:13:51.382959 (+   327us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:51.422541 (+ 39582us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:51.422574 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:51.436313 (+ 13739us) log.cc:844] Serialized 26 byte log entry
0709 12:13:51.731096 (+294783us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:13:51.731149 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.731226 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:51.923403 (+192177us) raft_consensus.cc:1685] finished
0709 12:13:51.923426 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:51.923565 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":39191,"raft.queue_time_us":39742,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":8397,"thread_start_us":111040,"threads_started":1,"wal-append.queue_time_us":415456}
I20260709 12:13:51.992014 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:51.909379 21577 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.994510 12494 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:51.909328 21700 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:51.993335 12646 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:13:51.995482 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 134863us
W20260709 12:13:52.018432 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 72898.9us
I20260709 12:13:52.039654 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:52.040448 12603 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:52.041380 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:52.043403 21587 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:52.043905 21587 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:52.044293 21587 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:51.906566 21438 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:52.046384 21387 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:52.048754 21438 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:51.962946 21749 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:13:51.992601 12604 rpcz_store.cc:276] 0709 12:13:48.198301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.460284 (+261983us) service_pool.cc:224] Handling call
0709 12:13:48.460629 (+   345us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:13:48.485519 (+ 24890us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:48.485553 (+    34us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:13:48.518348 (+ 32795us) log.cc:844] Serialized 78 byte log entry
0709 12:13:50.770482 (+2252134us) spinlock_profiling.cc:243] Waited 20.6 ms on lock 0x7b18004c6780. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:13:50.940941 (+170459us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:13:50.971649 (+ 30708us) log.cc:844] Serialized 14 byte log entry
0709 12:13:50.999873 (+ 28224us) log.cc:844] Serialized 14 byte log entry
0709 12:13:51.037603 (+ 37730us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.037698 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:51.953971 (+916273us) spinlock_profiling.cc:243] Waited 5.49 ms on lock 0x7b18004c6780. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:13:51.957459 (+  3488us) raft_consensus.cc:1685] finished
0709 12:13:51.957484 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:51.957657 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":141638,"prepare.queue_time_us":58001,"prepare.run_cpu_time_us":71,"prepare.run_wall_time_us":78,"raft.queue_time_us":62522,"raft.run_cpu_time_us":453,"raft.run_wall_time_us":21309,"spinlock_wait_cycles":59974272,"thread_start_us":156535,"threads_started":1,"wal-append.queue_time_us":2567289}
I20260709 12:13:52.054188 12571 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:52.051342 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 56902.9us
I20260709 12:13:52.021229 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4064) took 1614 ms. Trace:
I20260709 12:13:52.037029 12646 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:52.053336 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:51.970921 21591 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:13:52.055476 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:52.055037 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4269) took 1562 ms. Trace:
I20260709 12:13:52.060967 12597 raft_consensus.cc:2368] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28.
I20260709 12:13:52.061147 12678 rpcz_store.cc:276] 0709 12:13:50.493094 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.054166 (+1561072us) service_pool.cc:224] Handling call
0709 12:13:52.054490 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:52.054706 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:52.054741 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:52.054787 (+    46us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:52.054827 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.054889 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:52.054985 (+    96us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:52.062229 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:52.062642 12678 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:52.063035 12597 raft_consensus.cc:2368] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27.
I20260709 12:13:52.063314 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1055) took 1517 ms. Trace:
I20260709 12:13:52.058527 12601 rpcz_store.cc:276] 0709 12:13:50.406800 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.406955 (+   155us) service_pool.cc:224] Handling call
0709 12:13:50.472889 (+ 65934us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:50.510872 (+ 37983us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:13:50.536554 (+ 25682us) log.cc:844] Serialized 14 byte log entry
0709 12:13:50.762115 (+225561us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:50.762199 (+    84us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:50.766721 (+  4522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.020905 (+1254184us) spinlock_profiling.cc:243] Waited 26.1 ms on lock 0x7b54002e6dd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:52.020963 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:52.021145 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":24956,"prepare.run_cpu_time_us":75,"prepare.run_wall_time_us":78,"raft.queue_time_us":1116,"raft.run_cpu_time_us":142,"raft.run_wall_time_us":5329,"spinlock_wait_cycles":60058368,"thread_start_us":201529,"threads_started":1,"wal-append.queue_time_us":252617,"wal-append.run_cpu_time_us":1454,"wal-append.run_wall_time_us":1005596}
I20260709 12:13:51.971266 21719 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:52.063966 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:52.067595 12597 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:13:52.068168 12680 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 18 index: 14. Preceding OpId from leader: term: 20 index: 15. (index mismatch)
I20260709 12:13:52.063671 12678 rpcz_store.cc:276] 0709 12:13:50.545856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.062188 (+1516332us) service_pool.cc:224] Handling call
0709 12:13:52.062463 (+   275us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:52.063050 (+   587us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.063260 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:52.039042 21791 raft_consensus.cc:697] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:52.075181 21791 consensus_queue.cc:237] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 20.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:52.070498 21576 consensus_queue.cc:1059] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.006s
I20260709 12:13:52.068794 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3610) took 1583 ms. Trace:
I20260709 12:13:51.942022 21505 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:52.070151 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:52.070406 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:52.077874 12680 rpcz_store.cc:276] 0709 12:13:50.485818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.014421 (+1528603us) service_pool.cc:224] Handling call
0709 12:13:52.066033 (+ 51612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:52.068531 (+  2498us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.068742 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:52.078466 12673 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:52.070607 21718 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:52.054963 21700 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:52.071092 21703 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:13:52.071213 21769 consensus_queue.cc:1059] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:13:52.070166 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:52.078811 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:52.093948 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:52.095436 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:52.096101 12604 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:13:52.079917 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:51.995631 21358 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:13:52.038084 21717 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:52.079133 21215 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:52.050494 21735 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:52.049926 21762 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:52.081324 12646 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:13:52.103685 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:52.093119 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:52.065510 21438 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:52.081943 12678 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 20. Current term is 21. Ops: [20.15-20.15]
I20260709 12:13:52.094594 21738 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:52.088855 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:52.105736 12597 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:52.106932 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:52.051812 21577 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:52.107532 13336 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:52.087738 21718 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:52.100979 21719 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:13:52.108132 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1056) took 1561 ms. Trace:
I20260709 12:13:52.090408 21753 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:52.112718 12673 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 27.
I20260709 12:13:52.067155 21795 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:13:52.118280 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4271) took 1462 ms. Trace:
I20260709 12:13:52.121147 12678 rpcz_store.cc:276] 0709 12:13:50.546247 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.069711 (+1523464us) service_pool.cc:224] Handling call
0709 12:13:52.080618 (+ 10907us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:52.107679 (+ 27061us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.107998 (+   319us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:52.126277 12680 rpcz_store.cc:276] 0709 12:13:50.656307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.093137 (+1436830us) service_pool.cc:224] Handling call
0709 12:13:52.110837 (+ 17700us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:52.117729 (+  6892us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:52.117776 (+    47us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:52.117858 (+    82us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:52.117907 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.117995 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:52.118201 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:52.122481 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:52.079576 21714 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:52.128594 12678 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:13:52.127878 13334 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Advancing to term 29
I20260709 12:13:52.100126 21749 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:52.121779 21755 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.134853 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:52.134850 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3614) took 1402 ms. Trace:
I20260709 12:13:52.138187 12604 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:52.138609 12678 rpcz_store.cc:276] 0709 12:13:50.732925 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.126895 (+1393970us) service_pool.cc:224] Handling call
0709 12:13:52.127216 (+   321us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:52.128994 (+  1778us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.134782 (+  5788us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:52.119966 21718 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:52.141191 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3611) took 1411 ms. Trace:
I20260709 12:13:52.145838 12673 rpcz_store.cc:276] 0709 12:13:50.730106 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.126560 (+1396454us) service_pool.cc:224] Handling call
0709 12:13:52.140429 (+ 13869us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:52.140702 (+   273us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:52.140759 (+    57us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:52.140827 (+    68us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:52.140879 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.140959 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:52.141110 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:52.155508 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:52.156937 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:52.163547 21505 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:52.164769 21749 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:52.120337 21735 raft_consensus.cc:697] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:52.171182 12529 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 29. Current term is 30. Ops: [29.19-29.19]
I20260709 12:13:52.158103 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:52.172219 12604 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:52.177693 13336 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
W20260709 12:13:52.218861 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 137498us
I20260709 12:13:52.220525 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:52.181064 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:52.255156 21823 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:52.255908 21823 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:52.256388 21823 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:52.120433 21745 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:52.264722 21739 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:13:52.111719 21814 consensus_queue.cc:1059] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.064s
W20260709 12:13:52.287027 13332 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38472 (request call id 1016) took 1314 ms (client timeout 1555 ms). Trace:
W20260709 12:13:52.287657 13332 rpcz_store.cc:269] 0709 12:13:50.972327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.003582 (+ 31255us) service_pool.cc:224] Handling call
0709 12:13:52.286754 (+1283172us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b54000d7854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:52.286939 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":72841088}
I20260709 12:13:52.289296 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3557) took 1218 ms. Trace:
I20260709 12:13:52.289664 12527 rpcz_store.cc:276] 0709 12:13:51.071131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.071248 (+   117us) service_pool.cc:224] Handling call
0709 12:13:52.289059 (+1217811us) spinlock_profiling.cc:243] Waited 516 ms on lock 0x7b54003ca5d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:52.289209 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1187501952}
I20260709 12:13:52.290005 12597 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:13:52.290944 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:52.292033 12527 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:52.305972 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4254) took 2752 ms. Trace:
I20260709 12:13:52.307751 12674 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27. Ops: []
I20260709 12:13:52.318365 12677 rpcz_store.cc:276] 0709 12:13:49.553140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:49.998063 (+444923us) service_pool.cc:224] Handling call
0709 12:13:50.818308 (+820245us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:50.830825 (+ 12517us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:13:51.069927 (+239102us) log.cc:844] Serialized 14 byte log entry
0709 12:13:51.928873 (+858946us) spinlock_profiling.cc:243] Waited 49.1 ms on lock 0x7b18003e4b80. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a 00007f1488a2aa2a
0709 12:13:52.081462 (+152589us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:52.081515 (+    53us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:52.081560 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.081629 (+    69us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:52.305794 (+224165us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b54003d4ad0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:52.305912 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":276711,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":53,"raft.queue_time_us":30923,"raft.run_wall_time_us":13547,"spinlock_wait_cycles":1913187712,"thread_start_us":361844,"threads_started":2,"wal-append.queue_time_us":1045521}
I20260709 12:13:52.309477 21534 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:52.320343 21534 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:52.321182 12527 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 28. Current term is 29. Ops: []
I20260709 12:13:52.302389 12672 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:13:52.322494 12527 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:13:52.265468 21739 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:52.338718 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:52.342010 12673 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:52.342528 12672 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:13:52.278316 21576 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:13:52.169237 21755 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:52.357288 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 136889us
I20260709 12:13:52.326191 21215 raft_consensus.cc:697] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:52.301968 21403 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:52.375008 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1034) took 1477 ms. Trace:
I20260709 12:13:52.375598 12605 rpcz_store.cc:276] 0709 12:13:50.902622 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.400076 (+497454us) service_pool.cc:224] Handling call
0709 12:13:51.400438 (+   362us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:51.444306 (+ 43868us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:51.444351 (+    45us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:51.444525 (+   174us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:51.444653 (+   128us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.374677 (+930024us) spinlock_profiling.cc:243] Waited 43.6 ms on lock 0x7b540023f854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:52.374744 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:52.374924 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":100315392}
W20260709 12:13:52.376947 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 288239us
I20260709 12:13:52.381721 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:13:52.379947 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 312064us
W20260709 12:13:52.389355 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:52.389569 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 106615us
I20260709 12:13:52.393261 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3567) took 1318 ms. Trace:
I20260709 12:13:52.393877 12523 rpcz_store.cc:276] 0709 12:13:51.074963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.146537 (+ 71574us) service_pool.cc:224] Handling call
0709 12:13:52.393006 (+1246469us) spinlock_profiling.cc:243] Waited 754 ms on lock 0x7b5400054e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:52.393176 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1733315840}
I20260709 12:13:52.404428 21534 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:52.407946 12673 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:52.413443 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:52.414534 12673 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:13:52.417624 21099 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:52.419621 21641 raft_consensus.cc:3055] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 23 LEADER]: Stepping down as leader of term 23
W20260709 12:13:52.438088 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50180.7us
I20260709 12:13:52.421150 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4075) took 1907 ms. Trace:
I20260709 12:13:52.437647 12673 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 33.
I20260709 12:13:52.439047 12602 rpcz_store.cc:276] 0709 12:13:50.514179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.042822 (+528643us) service_pool.cc:224] Handling call
0709 12:13:51.043717 (+   895us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:51.448946 (+405229us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.420660 (+971714us) spinlock_profiling.cc:243] Waited 345 ms on lock 0x7b54002f2454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:52.421059 (+   399us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":792971648}
I20260709 12:13:52.439821 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:52.426975 21722 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:13:52.434130 12678 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:52.350297 21576 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:52.424042 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4083) took 1451 ms. Trace:
I20260709 12:13:52.441144 12598 rpcz_store.cc:276] 0709 12:13:50.972698 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.516968 (+544270us) service_pool.cc:224] Handling call
0709 12:13:51.517399 (+   431us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:51.797538 (+280139us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.423736 (+626198us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b5400310bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:52.423965 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":611690240}
W20260709 12:13:52.390031 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:52.438429 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 77287us
I20260709 12:13:52.445469 12646 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:52.458703 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:52.459619 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:52.534693 12605 raft_consensus.cc:2393] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 21: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:52.458827 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:52.536247 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:52.537046 12598 raft_consensus.cc:2410] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 24 because replica has last-logged OpId of term: 23 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 12:13:52.390444 21812 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:52.473974 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:52.438169 21641 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:13:52.481038 21819 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
W20260709 12:13:52.481865 21813 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.395s	user 0.000s	sys 0.001s
I20260709 12:13:52.438546 21820 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:13:52.338256 21738 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:52.513012 12572 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51792 dropped due to backpressure: service queue is full with 50 items [suppressed 1 similar messages]
I20260709 12:13:52.531920 21814 raft_consensus.cc:3055] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:52.573511 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:52.574258 12603 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:52.388638 21823 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:52.527060 21215 consensus_queue.cc:237] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:52.364063 21825 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:52.522681 21828 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:13:52.533526 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 88635.7us
I20260709 12:13:52.536842 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:52.470795 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:52.534850 12602 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:52.546592 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:52.546840 21755 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:52.547010 21403 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:52.548751 21835 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:13:52.550971 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:52.586620 12530 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:52.450503 21739 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:52.378707 21735 consensus_queue.cc:237] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 23.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:52.569254 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:52.515625 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:52.574920 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 139677us
I20260709 12:13:52.554445 21769 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.029s
I20260709 12:13:52.579979 12605 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:13:52.573956 21814 raft_consensus.cc:740] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:52.579931 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:51.329238 12675 rpcz_store.cc:276] 0709 12:13:50.131742 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.094775 (+963033us) service_pool.cc:224] Handling call
0709 12:13:51.173919 (+ 79144us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:51.174423 (+   504us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:13:51.183958 (+  9535us) log.cc:844] Serialized 14 byte log entry
0709 12:13:51.315237 (+131279us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:51.315320 (+    83us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:51.315386 (+    66us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.315474 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:51.315636 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":9071,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":6765,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":6106,"thread_start_us":130654,"threads_started":1}
I20260709 12:13:52.601807 12525 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:52.559270 21586 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.028s
I20260709 12:13:52.585187 21714 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:52.578899 21820 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 27 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:52.611377 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:52.607831 12604 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:52.613646 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:52.612375 12527 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:13:52.575619 21700 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:52.587759 21577 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:52.607856 12530 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:13:52.615029 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:52.562224 21812 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:52.620028 12524 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:13:52.621611 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:52.621549 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:52.622547 12530 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:52.626474 12494 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:52.587193 21795 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.026s
I20260709 12:13:52.600805 21835 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:52.570585 12572 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51792 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3516)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3553)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4100)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3535)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3508)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3536)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3528)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4101)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3547)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3539)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3554)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3509)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3527)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3555)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3541)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3530)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3556)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3557)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3558)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3526)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3545)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4099)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4092)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3532)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3544)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3543)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3540)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4103)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3529)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4093)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1036)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3511)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3514)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3518)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3537)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3538)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3542)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3548)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3550)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3551)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3552)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4095)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4097)
I20260709 12:13:52.637823 12529 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 28: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:52.637477 12494 leader_election.cc:400] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28.
I20260709 12:13:52.637974 12603 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:52.644167 12494 leader_election.cc:403] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:52.638433 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:52.642918 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3608) took 2248 ms. Trace:
I20260709 12:13:52.645036 12679 rpcz_store.cc:276] 0709 12:13:50.394719 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.681424 (+1286705us) service_pool.cc:224] Handling call
0709 12:13:51.726086 (+ 44662us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:51.741042 (+ 14956us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:13:51.741075 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:51.757174 (+ 16099us) log.cc:844] Serialized 26 byte log entry
0709 12:13:51.916185 (+159011us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:51.916235 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.916311 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:52.642677 (+726366us) raft_consensus.cc:1685] finished
0709 12:13:52.642702 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:52.642858 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1719,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":6265,"thread_start_us":112882,"threads_started":1,"wal-append.queue_time_us":214976}
I20260709 12:13:52.653658 12527 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 30: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:52.654701 12494 leader_election.cc:400] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27.
I20260709 12:13:52.655177 12494 leader_election.cc:403] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
W20260709 12:13:52.653972 13334 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.519s	user 0.014s	sys 0.000s
I20260709 12:13:52.656029 12529 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:13:52.656337 13334 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:13:52.658001 12527 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:52.658826 12523 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 27. Current term is 28. Ops: []
W20260709 12:13:52.642283 21753 consensus_meta.cc:287] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.509s	user 0.009s	sys 0.002s
I20260709 12:13:52.660132 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:52.661341 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 221653us
I20260709 12:13:52.666003 12529 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: [21.13-21.13]
I20260709 12:13:52.586402 21576 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:52.681074 12529 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 27 index: 14. (index mismatch)
I20260709 12:13:52.642867 21745 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:52.672101 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:52.685885 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 70880.2us
I20260709 12:13:52.666566 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:52.659085 21701 consensus_meta.cc:287] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.909s	user 0.010s	sys 0.004s
I20260709 12:13:52.683903 13304 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:52.687659 12494 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:52.641803 21099 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:13:52.638813 21795 consensus_queue.cc:1243] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 19.12 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 11
I20260709 12:13:52.623379 21833 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.066s
I20260709 12:13:52.657675 21640 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:52.617327 21641 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:52.636880 21832 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.070s
I20260709 12:13:52.873312 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4290) took 1031 ms. Trace:
W20260709 12:13:52.839888 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 320574us
W20260709 12:13:52.885030 12526 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3568) took 1790 ms (client timeout 1596 ms). Trace:
I20260709 12:13:52.919229 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4275) took 2075 ms. Trace:
I20260709 12:13:52.692997 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:52.747844 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:52.756819 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 43851.9us
I20260709 12:13:52.657457 21791 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28.
W20260709 12:13:52.923408 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 214143us
W20260709 12:13:52.929505 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 134806us
I20260709 12:13:52.954783 21745 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:52.687104 21577 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:13:52.956545 12526 rpcz_store.cc:269] 0709 12:13:51.094484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.103543 (+  9059us) service_pool.cc:224] Handling call
0709 12:13:52.884801 (+1781258us) spinlock_profiling.cc:243] Waited 41.4 ms on lock 0x7b5400407c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:52.884943 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":95333760}
I20260709 12:13:52.667209 21707 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:53.005589 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:52.686429 21714 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:52.871484 21840 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.122s	user 0.002s	sys 0.000s
I20260709 12:13:53.010849 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:52.749130 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:52.929232 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 182466us
I20260709 12:13:53.013715 21306 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:52.955617 12677 rpcz_store.cc:276] 0709 12:13:51.841673 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.320113 (+478440us) service_pool.cc:224] Handling call
0709 12:13:52.320483 (+   370us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:52.512673 (+192190us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:52.512726 (+    53us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:52.513506 (+   780us) log.cc:844] Serialized 52 byte log entry
0709 12:13:52.625709 (+112203us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:13:52.625752 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.625828 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:52.872975 (+247147us) raft_consensus.cc:1685] finished
0709 12:13:52.873004 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:52.873226 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7198,"raft.queue_time_us":163890,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":128,"thread_start_us":242693,"threads_started":2,"wal-append.queue_time_us":326281}
W20260709 12:13:52.863976 21787 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 1.681s	user 0.000s	sys 0.013s
I20260709 12:13:53.014441 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:53.015162 12530 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:13:53.015697 12522 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 1140) took 1561 ms (client timeout 1909 ms). Trace:
W20260709 12:13:53.016131 12522 rpcz_store.cc:269] 0709 12:13:51.454327 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.606990 (+152663us) service_pool.cc:224] Handling call
0709 12:13:53.015492 (+1408502us) spinlock_profiling.cc:243] Waited 71.9 ms on lock 0x7b5400053554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:53.015630 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":165466368}
I20260709 12:13:52.956609 12680 rpcz_store.cc:276] 0709 12:13:50.843461 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.127274 (+1283813us) service_pool.cc:224] Handling call
0709 12:13:52.143576 (+ 16302us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:52.329224 (+185648us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:52.329281 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:52.363645 (+ 34364us) log.cc:844] Serialized 26 byte log entry
0709 12:13:52.682982 (+319337us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:52.683024 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:52.683087 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:52.918853 (+235766us) raft_consensus.cc:1685] finished
0709 12:13:52.918893 (+    40us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:52.919133 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":256977,"spinlock_wait_cycles":14467840,"thread_start_us":386071,"threads_started":2,"wal-append.queue_time_us":451155}
W20260709 12:13:52.924793 21718 consensus_meta.cc:287] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.702s	user 0.013s	sys 0.000s
I20260709 12:13:52.676884 21547 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.319s
I20260709 12:13:53.018184 21306 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:13:53.018617 21306 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:53.018361 21718 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.019825 12603 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 24: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:53.019495 21306 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.020845 21306 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:13:53.020970 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3535) took 1068 ms. Trace:
I20260709 12:13:53.021473 12603 rpcz_store.cc:276] 0709 12:13:51.952109 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.644607 (+692498us) service_pool.cc:224] Handling call
0709 12:13:53.020744 (+376137us) spinlock_profiling.cc:243] Waited 340 ms on lock 0x7b5400232154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:53.020901 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":781220096}
I20260709 12:13:53.022881 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:53.023788 12603 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 22: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:53.023904 12530 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:13:53.024350 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3528) took 1090 ms. Trace:
I20260709 12:13:53.024870 12603 rpcz_store.cc:276] 0709 12:13:51.934064 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.022545 (+1088481us) service_pool.cc:224] Handling call
0709 12:13:53.024287 (+  1742us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:53.025594 12530 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 27. Current term is 28. Ops: []
I20260709 12:13:53.025979 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:53.026739 12603 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:53.027885 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:53.028714 12603 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:53.029294 21551 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:53.030365 21714 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.031132 21507 raft_consensus.cc:3055] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:53.031591 21507 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:53.031966 21718 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:53.032413 21507 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.033277 21714 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:53.033510 21507 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:53.034827 21723 raft_consensus.cc:3055] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:53.035475 21723 raft_consensus.cc:740] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:52.864118 21099 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:53.036355 21723 consensus_queue.cc:260] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:53.036490 21099 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.037653 21723 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:52.718513 21465 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:13:52.801234 21826 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.193s	user 0.000s	sys 0.001s
W20260709 12:13:52.748728 21764 consensus_meta.cc:287] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.853s	user 0.011s	sys 0.001s
I20260709 12:13:52.864761 21823 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:53.039764 21465 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:52.860421 21587 consensus_meta.cc:287] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.816s	user 0.015s	sys 0.004s
I20260709 12:13:53.040550 21587 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.040796 21465 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.042918 21587 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:53.045300 21717 consensus_meta.cc:287] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.938s	user 0.012s	sys 0.000s
I20260709 12:13:53.047192 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:52.868733 21812 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:53.012329 21845 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:53.048012 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3536) took 1095 ms. Trace:
I20260709 12:13:53.048419 12604 rpcz_store.cc:276] 0709 12:13:51.952380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.046922 (+1094542us) service_pool.cc:224] Handling call
0709 12:13:53.047952 (+  1030us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:52.752171 21822 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.258s
I20260709 12:13:53.006909 21577 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.048187 21845 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:52.688978 21824 consensus_queue.cc:1059] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 20.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.329s
I20260709 12:13:53.049361 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:52.982647 21843 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Leader election won for term 33
W20260709 12:13:52.916417 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 232572us
I20260709 12:13:53.051743 13337 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
W20260709 12:13:53.081845 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.082233 12603 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:13:52.925312 21576 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:52.711815 21753 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:13:52.747411 21701 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:53.028280 21818 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.745s	user 0.000s	sys 0.001s
I20260709 12:13:52.712231 21700 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.299s
I20260709 12:13:52.974565 21505 consensus_queue.cc:1059] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.056s
I20260709 12:13:53.071842 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3539) took 1118 ms. Trace:
I20260709 12:13:53.092298 12604 rpcz_store.cc:276] 0709 12:13:51.953182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.049158 (+1095976us) service_pool.cc:224] Handling call
0709 12:13:53.071775 (+ 22617us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:52.802093 21755 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:13:53.075837 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.084190 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:53.083762 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3547) took 1006 ms. Trace:
I20260709 12:13:52.677347 21820 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:53.095158 12603 rpcz_store.cc:276] 0709 12:13:52.077182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.027609 (+950427us) service_pool.cc:224] Handling call
0709 12:13:53.083665 (+ 56056us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:53.106226 12525 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
W20260709 12:13:53.095985 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 99378.9us
I20260709 12:13:53.114020 12522 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:53.009451 21745 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:52.865029 21835 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.123884 12675 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:13:52.750715 21814 consensus_queue.cc:260] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 21.13, Last appended by leader: 13, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.090931 21843 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 33 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:53.117178 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:53.117539 12671 raft_consensus.cc:2393] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 27: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:53.051915 21764 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.120591 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:53.136541 12604 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:13:53.120693 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:53.138603 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:53.139269 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:53.141541 12494 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:53.121704 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:53.087960 21753 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:53.116459 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:53.148379 12680 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
I20260709 12:13:53.149013 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3509) took 1240 ms. Trace:
I20260709 12:13:53.149665 12603 rpcz_store.cc:276] 0709 12:13:51.908085 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.116163 (+1208078us) service_pool.cc:224] Handling call
0709 12:13:53.148948 (+ 32785us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:53.132287 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:53.150950 12673 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:53.156697 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:53.159845 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:53.160008 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:53.160835 12604 raft_consensus.cc:2368] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
I20260709 12:13:53.163014 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:53.163857 12676 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:53.168876 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3527) took 1235 ms. Trace:
I20260709 12:13:53.169461 12604 rpcz_store.cc:276] 0709 12:13:51.933819 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.137430 (+1203611us) service_pool.cc:224] Handling call
0709 12:13:53.168789 (+ 31359us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:53.171438 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:53.171730 12527 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:53.172081 12603 raft_consensus.cc:2376] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25. Re-sending same reply.
W20260709 12:13:53.174906 12643 leader_election.cc:336] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.614s (SENT)
I20260709 12:13:53.175566 12643 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:53.155697 21814 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:53.108016 21755 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.155371 21753 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:13:53.161149 21824 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:13:53.185194 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:53.190714 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:53.193457 12604 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:53.194031 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:53.199522 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:53.200279 12673 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:53.115036 21306 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.074s
I20260709 12:13:53.210503 12603 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:13:53.182659 21587 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:53.181934 21853 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.214882 12526 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:53.217074 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:53.213336 12525 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:53.218271 12526 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 29: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:53.192281 21824 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:53.178820 21743 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.034s
W20260709 12:13:53.401841 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 129109us
I20260709 12:13:53.270988 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:53.224838 12673 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:53.219949 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:53.465119 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 106582us
W20260709 12:13:53.469485 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.416411 12604 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:53.470300 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3541) took 1516 ms. Trace:
I20260709 12:13:53.470633 12604 rpcz_store.cc:276] 0709 12:13:51.953692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.170390 (+1216698us) service_pool.cc:224] Handling call
0709 12:13:53.470236 (+299846us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:53.275285 13304 leader_election.cc:336] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.936s (SENT)
I20260709 12:13:53.471652 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:53.471769 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:53.465361 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:53.468243 12526 raft_consensus.cc:2410] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 30 because replica has last-logged OpId of term: 29 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 16.
W20260709 12:13:53.474973 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 66254.7us
I20260709 12:13:53.474143 12571 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 21 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:53.362397 21846 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.237s	user 0.001s	sys 0.000s
W20260709 12:13:53.355875 21849 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:13:53.486366 21249 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:53.486598 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:53.300834 21099 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.484669 21822 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election won for term 21
I20260709 12:13:51.651872 12606 rpcz_store.cc:276] 0709 12:13:50.565045 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.276188 (+711143us) service_pool.cc:224] Handling call
0709 12:13:51.276545 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:51.276765 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:13:51.277064 (+   299us) log.cc:844] Serialized 14 byte log entry
0709 12:13:51.643072 (+366008us) log.cc:844] Serialized 14 byte log entry
0709 12:13:51.649061 (+  5989us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:51.649137 (+    76us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:51.649281 (+   144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:51.649425 (+   144us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:51.649542 (+   117us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":1899,"prepare.run_cpu_time_us":86,"prepare.run_wall_time_us":4186,"raft.queue_time_us":5367,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":3277,"thread_start_us":365421,"threads_started":1}
I20260709 12:13:53.465761 21829 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.498102 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:53.504487 12530 raft_consensus.cc:2410] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 26 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
W20260709 12:13:53.502717 12599 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4073) took 3079 ms (client timeout 2032 ms). Trace:
I20260709 12:13:53.499125 13305 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:53.355029 21852 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:53.507725 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:53.489588 21701 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:53.476476 21824 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:53.513301 12676 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:13:53.507448 12646 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:53.515844 21249 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:13:53.531637 21714 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:13:53.534386 21764 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:53.535360 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:53.536023 21707 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.527s	user 0.012s	sys 0.000s
I20260709 12:13:53.536541 21707 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27.
I20260709 12:13:53.536720 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:53.537415 12679 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:53.500613 21837 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:13:53.538816 21837 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:13:53.539198 21837 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:53.524031 21822 raft_consensus.cc:697] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:53.540148 21837 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.541451 21837 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:53.541250 21822 consensus_queue.cc:237] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 21, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.501396 21742 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.024s
W20260709 12:13:53.543927 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.545874 12523 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:13:53.474156 21745 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:53.274535 21853 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:53.502570 21855 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.143s	user 0.001s	sys 0.000s
W20260709 12:13:53.519245 12599 rpcz_store.cc:269] 0709 12:13:50.410037 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:50.637469 (+227432us) service_pool.cc:224] Handling call
0709 12:13:53.488795 (+2851326us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54002e59d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:53.488924 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":554164992}
I20260709 12:13:53.496824 21665 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:53.523381 12673 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:53.490567 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:53.501812 21789 raft_consensus.cc:3055] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:13:53.569499 21789 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:53.516542 12496 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:53.528209 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:53.361687 12570 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51782 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3557)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3558)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3526)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3545)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3572)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4099)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3565)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4092)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3532)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3559)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3573)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4109)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3563)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3568)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3560)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4108)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3566)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3567)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3564)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3570)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3569)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3544)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3543)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3561)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3562)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3540)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4103)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3571)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4106)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3529)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4093)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1036)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3511)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3514)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3518)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3537)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3538)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3542)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3548)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3550)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3551)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3552)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4095)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4097)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4104)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4105)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4107)
I20260709 12:13:53.570845 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:53.571617 12526 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:13:53.571691 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 55237.6us
I20260709 12:13:53.530185 12571 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:53.504487 21587 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 26.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.023s
W20260709 12:13:53.577332 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.561556 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:53.562345 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:53.564927 21586 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Stepping down as leader of term 26
W20260709 12:13:53.566306 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.569839 21812 raft_consensus.cc:3055] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Stepping down as leader of term 20
I20260709 12:13:53.572458 12496 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:53.586123 12526 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:13:53.587296 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:53.573134 12676 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:53.576457 12673 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
W20260709 12:13:53.573695 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 44664.6us
I20260709 12:13:53.578545 12598 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:53.369561 21843 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.21, Last appended by leader: 21, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:53.552685 21215 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:13:53.626816 21215 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:53.628006 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:53.645018 21641 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:53.647217 21591 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:53.648813 21591 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:13:53.654561 21859 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.102s	user 0.000s	sys 0.001s
I20260709 12:13:53.591926 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3553) took 1209 ms. Trace:
I20260709 12:13:53.655224 12598 rpcz_store.cc:276] 0709 12:13:52.382605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.561281 (+1178676us) service_pool.cc:224] Handling call
0709 12:13:53.591854 (+ 30573us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:53.656248 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:53.656757 12671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 1065) took 2326 ms (client timeout 1936 ms). Trace:
W20260709 12:13:53.657356 12671 rpcz_store.cc:269] 0709 12:13:51.330455 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.620575 (+290120us) service_pool.cc:224] Handling call
0709 12:13:53.656454 (+2035879us) spinlock_profiling.cc:243] Waited 1.48 s on lock 0x7b54002c4f54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:53.656648 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3411959040}
W20260709 12:13:53.660573 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.661499 12674 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27. Ops: []
I20260709 12:13:53.663192 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4287) took 1957 ms. Trace:
I20260709 12:13:53.663614 12674 rpcz_store.cc:276] 0709 12:13:51.705385 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.318994 (+613609us) service_pool.cc:224] Handling call
0709 12:13:53.661233 (+1342239us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:53.662091 (+   858us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:53.662922 (+   831us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b54002c4f50. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:53.663111 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3086096640}
I20260709 12:13:53.665230 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:53.593072 12571 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:53.665918 21824 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:53.666610 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 73650.2us
I20260709 12:13:53.668604 21465 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:53.668933 12571 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51782 dropped due to backpressure: service queue is full with 50 items [suppressed 1 similar messages]
I20260709 12:13:53.589516 21764 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:13:53.589777 21306 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.006s
I20260709 12:13:53.595708 21717 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:13:53.680428 21764 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:53.596601 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:53.598420 21755 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:53.681628 21764 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.15, Last appended by leader: 15, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.682739 21755 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:53.683233 21755 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:53.684384 21755 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.685710 21577 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:53.686794 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:53.686957 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:53.598891 21802 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.961s	user 0.000s	sys 0.007s
I20260709 12:13:53.589619 21587 consensus_queue.cc:1243] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 27.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 17
I20260709 12:13:53.586318 21833 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:13:53.599459 21576 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.509s	user 0.007s	sys 0.003s
I20260709 12:13:53.585752 21586 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:53.551117 21852 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:53.601764 21723 consensus_meta.cc:287] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.564s	user 0.010s	sys 0.003s
I20260709 12:13:53.581645 21812 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:53.602763 21862 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:13:53.604653 12572 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:53.722771 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 118267us
I20260709 12:13:53.523571 21829 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:53.610522 21789 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:53.727447 13338 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 27 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27. Ops: []
I20260709 12:13:53.614826 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:53.728817 12605 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
W20260709 12:13:53.729579 12605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3516) took 1798 ms (client timeout 2034 ms). Trace:
W20260709 12:13:53.730165 12605 rpcz_store.cc:269] 0709 12:13:51.930996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.614502 (+1683506us) service_pool.cc:224] Handling call
0709 12:13:53.729488 (+114986us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:53.617280 21249 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:53.617750 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:53.733527 12680 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:53.619491 21751 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:53.614717 21756 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:53.624020 12524 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3577) took 2216 ms (client timeout 1645 ms). Trace:
W20260709 12:13:53.746676 12524 rpcz_store.cc:269] 0709 12:13:51.407360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.424952 (+ 17592us) service_pool.cc:224] Handling call
0709 12:13:53.623519 (+2198567us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b54003c0354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:53.623736 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2536160768}
I20260709 12:13:53.582480 21739 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:53.628659 21215 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.588323 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3558) took 1200 ms. Trace:
W20260709 12:13:53.637568 21863 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:13:53.639312 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:53.654441 12679 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:53.759828 12523 raft_consensus.cc:2410] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 27 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 12:13:53.759932 12672 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: [25.16-25.16]
W20260709 12:13:53.760248 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 168637us
I20260709 12:13:53.657012 12598 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 30: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:53.758556 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:53.520251 21099 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.672202 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3618) took 2624 ms. Trace:
I20260709 12:13:53.761783 12678 rpcz_store.cc:276] 0709 12:13:51.048044 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.141937 (+1093893us) service_pool.cc:224] Handling call
0709 12:13:52.142289 (+   352us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:52.449399 (+307110us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:53.671846 (+1222447us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b54003bd154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:53.672123 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":565377792}
W20260709 12:13:53.673664 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 143698us
I20260709 12:13:53.670343 21255 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:13:53.688227 12675 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
W20260709 12:13:53.765239 12598 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3526) took 1831 ms (client timeout 2362 ms). Trace:
W20260709 12:13:53.689997 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.671221 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:53.765555 12598 rpcz_store.cc:269] 0709 12:13:51.933569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.655983 (+1722414us) service_pool.cc:224] Handling call
0709 12:13:53.765173 (+109190us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:53.693079 12671 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 29. Current term is 30. Ops: [25.14-29.16]
I20260709 12:13:53.694106 12674 raft_consensus.cc:2410] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 27 because replica has last-logged OpId of term: 25 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 12:13:53.766470 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:53.706084 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:53.706453 12531 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
W20260709 12:13:53.694973 21814 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.503s	user 0.011s	sys 0.000s
I20260709 12:13:53.669209 21465 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.636003 21753 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.726387 12604 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:13:53.740353 12680 pending_rounds.cc:85] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 11
I20260709 12:13:53.708110 21358 raft_consensus.cc:3055] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:13:53.713461 21833 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:53.669373 12571 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51782 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3545)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3572)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4099)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3565)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4092)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3532)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3559)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3573)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4109)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3563)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3568)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3560)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4108)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3566)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3567)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3564)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3570)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3569)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3544)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3543)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3561)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3562)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3540)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4115)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4112)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1038)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4103)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1037)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3571)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4111)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3529)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4093)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1036)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3511)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3514)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3518)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3537)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3538)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3542)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3548)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3550)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3551)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3552)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4095)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4097)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4104)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4105)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4107)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4113)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4114)
I20260709 12:13:53.753894 12599 rpcz_store.cc:276] 0709 12:13:52.387816 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.558140 (+1170324us) service_pool.cc:224] Handling call
0709 12:13:53.588219 (+ 30079us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:53.754281 12602 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:13:53.580200 12494 leader_election.cc:336] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51782 dropped due to backpressure: service queue is full with 50 items
I20260709 12:13:53.763270 12524 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: [23.15-25.16]
W20260709 12:13:53.672777 21861 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.116s	user 0.000s	sys 0.000s
I20260709 12:13:53.769006 12646 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:53.741488 21576 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:53.771890 12572 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:53.802026 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 221852us
I20260709 12:13:53.774401 12679 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 32. Current term is 33. Ops: [32.21-32.21]
I20260709 12:13:53.780249 12496 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:13:53.815232 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 46307.3us
I20260709 12:13:53.781078 12526 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 29. Current term is 30. Ops: [29.16-29.16]
I20260709 12:13:53.753489 21739 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:53.770390 21723 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:13:53.783241 12680 pending_rounds.cc:107] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 19.12
I20260709 12:13:53.821372 12680 raft_consensus.cc:2889] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:53.801662 12525 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:53.770825 21789 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:53.798305 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:53.797588 21821 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.470s	user 0.017s	sys 0.013s
I20260709 12:13:53.799618 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:53.816012 12646 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:53.819674 21591 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:53.820222 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 48420.8us
I20260709 12:13:53.769421 21740 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.805162 12674 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 26. Current term is 27. Ops: []
I20260709 12:13:53.804008 12524 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 26. Current term is 27. Ops: []
I20260709 12:13:53.828231 12529 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 29. Current term is 30. Ops: [29.15-29.15]
I20260709 12:13:53.822605 12672 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 26. Current term is 27. Ops: []
I20260709 12:13:53.824361 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4099) took 1682 ms. Trace:
I20260709 12:13:53.832093 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:53.832460 12599 rpcz_store.cc:276] 0709 12:13:52.141592 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.797951 (+1656359us) service_pool.cc:224] Handling call
0709 12:13:53.824296 (+ 26345us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:53.832975 12679 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:13:53.832960 12572 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:53.761804 21586 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.835007 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3591) took 1278 ms. Trace:
I20260709 12:13:53.834950 21740 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.835630 12527 rpcz_store.cc:276] 0709 12:13:52.556547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.679004 (+122457us) service_pool.cc:224] Handling call
0709 12:13:52.679309 (+   305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:53.178518 (+499209us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:53.834745 (+656227us) spinlock_profiling.cc:243] Waited 492 ms on lock 0x7b54003f8754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:53.834941 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1131937024}
I20260709 12:13:53.838297 21702 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:13:53.838502 21703 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:53.839272 21702 raft_consensus.cc:3055] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:53.839733 21702 raft_consensus.cc:740] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:53.839951 21756 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:53.840420 21756 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:53.840829 21756 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:53.840651 21702 consensus_queue.cc:260] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.844195 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:53.844944 12673 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:13:53.825294 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 45148.8us
I20260709 12:13:53.832114 12524 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24. Ops: []
W20260709 12:13:53.847819 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48328.8us
I20260709 12:13:53.782557 21255 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:53.841904 21702 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:53.794938 21249 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:53.802309 21358 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:53.850312 12602 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:53.820684 21814 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:13:53.852267 12646 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:53.771070 21837 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
I20260709 12:13:53.820984 21739 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:53.860343 21814 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:53.860905 12672 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:53.867591 12674 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 27. Current term is 28. Ops: []
I20260709 12:13:53.858695 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:53.861718 21852 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:53.773667 21577 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.069s
I20260709 12:13:53.877640 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:53.835031 21586 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:53.783968 21795 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 27.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.071s
I20260709 12:13:53.823652 21822 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:13:53.853758 21576 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:53.887392 12673 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:53.885134 21743 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.007s
I20260709 12:13:53.795321 21833 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.894763 12676 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:13:53.898017 21591 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:53.898905 21255 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.927680 21751 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.771396 21812 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 20, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.854947 21745 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.046s
I20260709 12:13:53.904209 21724 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:53.905017 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:53.931839 12599 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 27: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:53.907054 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:53.867950 21723 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.042s
I20260709 12:13:53.909899 21701 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:53.910732 21795 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:13:53.911491 21099 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:53.911839 21358 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:53.913009 21753 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:53.914294 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 639016us
W20260709 12:13:53.915179 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 416138us
I20260709 12:13:53.915676 21505 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:53.916148 21640 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:13:53.918424 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:53.866603 21587 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:13:53.926429 12676 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:53.926402 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:53.889065 21832 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:53.968300 21848 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:53.991346 21848 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:53.993539 12571 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:53.932555 21249 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:53.937516 12599 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4092) took 2245 ms (client timeout 2966 ms). Trace:
W20260709 12:13:53.939549 12606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3557) took 1555 ms (client timeout 1827 ms). Trace:
I20260709 12:13:53.928586 21769 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:54.009747 12606 rpcz_store.cc:269] 0709 12:13:52.384189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.906709 (+1522520us) service_pool.cc:224] Handling call
0709 12:13:53.939481 (+ 32772us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:53.965284 13304 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:53.967176 12673 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:13:53.975594 13334 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 25. Ops: [24.17-24.17]
I20260709 12:13:53.975375 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:53.977066 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:53.985368 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:53.986402 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:53.988878 21755 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:53.989187 21756 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:53.938258 21824 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:53.934252 21724 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:53.969475 12676 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:54.031373 12673 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 23 index: 11. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 12:13:54.040763 12673 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:13:53.953855 21735 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.070s
I20260709 12:13:53.996922 13337 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
W20260709 12:13:53.997444 12643 leader_election.cc:336] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.596s (SENT)
W20260709 12:13:54.053846 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 56459.9us
I20260709 12:13:54.054898 12676 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:13:54.055215 12643 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.624s (SENT)
I20260709 12:13:54.057366 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:53.930824 21751 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:53.938259 21795 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:13:53.976004 12416 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 70b8c9b89e3d43fbb70422110d76702c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica a2fdbe7eef6748a9b4611f6c9bc9b201 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 22 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:13:54.020248 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 45958.5us
I20260709 12:13:54.021039 12671 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:13:53.957810 21701 raft_consensus.cc:697] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:13:54.023627 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58444.5us
W20260709 12:13:54.031419 21873 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:13:54.019661 21743 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 30.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.006s
I20260709 12:13:54.025104 21641 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:13:54.070004 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 193807us
I20260709 12:13:54.023667 21670 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:13:53.998030 21740 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:54.025665 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:54.020418 21249 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:54.019881 21812 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 20 FOLLOWER]: Advancing to term 21
I20260709 12:13:54.018173 21358 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:13:54.031350 21871 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.148s	user 0.000s	sys 0.000s
I20260709 12:13:54.023267 21723 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.056s
I20260709 12:13:53.995465 21722 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:13:54.010432 21587 raft_consensus.cc:697] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:13:54.032275 21874 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:13:54.032928 21829 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:54.034319 21876 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
I20260709 12:13:53.964403 21853 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:54.037524 21756 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:53.925644 21576 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:54.008847 12599 rpcz_store.cc:269] 0709 12:13:51.691842 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.904715 (+2212873us) service_pool.cc:224] Handling call
0709 12:13:53.937445 (+ 32730us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:53.994042 21848 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:54.059799 12602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4100) took 1917 ms (client timeout 1877 ms). Trace:
I20260709 12:13:54.061401 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:13:54.064095 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 146137us
W20260709 12:13:54.062510 12416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 138768us
I20260709 12:13:54.024498 21852 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:13:54.067216 12597 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:13:54.069048 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:54.154368 12602 rpcz_store.cc:269] 0709 12:13:52.141997 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.627455 (+1485458us) service_pool.cc:224] Handling call
0709 12:13:54.059610 (+432155us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b54002e59d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:54.059721 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":251118336}
I20260709 12:13:54.059234 21845 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
W20260709 12:13:54.134868 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 45000.1us
I20260709 12:13:54.062896 21591 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:53.998493 21814 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:54.071303 21868 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:54.059172 21577 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.077s
I20260709 12:13:54.086889 21641 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:54.147068 12676 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:13:54.142436 12606 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:54.020298 21769 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:54.090533 21670 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:54.160854 12601 raft_consensus.cc:2410] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 24 because replica has last-logged OpId of term: 23 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 12:13:54.204620 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3559) took 1403 ms. Trace:
I20260709 12:13:54.161228 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:54.161401 13337 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 33 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 32. Current term is 33. Ops: []
I20260709 12:13:54.190838 12679 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 27. Current term is 28. Ops: []
I20260709 12:13:54.205029 12601 rpcz_store.cc:276] 0709 12:13:52.801778 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.042718 (+1240940us) service_pool.cc:224] Handling call
0709 12:13:54.204549 (+161831us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:54.153291 21576 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:54.160544 21811 raft_consensus.cc:3055] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:13:54.192057 21465 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:54.192276 13331 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 20 index: 15. Preceding OpId from leader: term: 21 index: 16. (index mismatch)
W20260709 12:13:54.194707 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 133373us
I20260709 12:13:54.198108 13305 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:54.215893 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:54.190933 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:54.154310 21745 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:13:54.209441 21215 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:54.198163 21868 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:54.205050 21641 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:54.211009 12600 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
W20260709 12:13:54.191886 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.134375 21843 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 28.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.067s
W20260709 12:13:54.201130 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:54.244570 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.189826 21722 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:13:54.217344 12602 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:54.245869 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4109) took 1095 ms. Trace:
I20260709 12:13:54.246358 12602 rpcz_store.cc:276] 0709 12:13:53.150366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.162667 (+1012301us) service_pool.cc:224] Handling call
0709 12:13:54.245613 (+ 82946us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:54.221066 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:54.202497 21852 consensus_queue.cc:1243] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 24.14 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:13:54.210194 13330 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 27 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 26. Current term is 27. Ops: []
I20260709 12:13:54.247534 13334 raft_consensus.cc:2376] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27. Re-sending same reply.
I20260709 12:13:54.228282 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:54.191247 21848 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.020s
I20260709 12:13:54.206321 21670 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:13:54.269431 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 444478us
I20260709 12:13:54.249853 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:54.272425 12601 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:54.272970 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3563) took 1294 ms. Trace:
I20260709 12:13:54.273289 12601 rpcz_store.cc:276] 0709 12:13:52.978861 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.221267 (+1242406us) service_pool.cc:224] Handling call
0709 12:13:54.272908 (+ 51641us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:54.253432 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:54.145449 21795 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:54.254988 21587 consensus_queue.cc:237] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:54.214473 21811 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:54.225653 21868 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:54.262739 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:54.224342 21879 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 21.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.045s
I20260709 12:13:54.217020 21745 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:54.230392 21835 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.293984 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:54.264307 21751 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:54.265031 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:54.300921 13330 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 27: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:54.301081 12676 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
W20260709 12:13:54.273932 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.302510 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:54.279646 12671 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a"
dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:45864
I20260709 12:13:54.221784 21827 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.197669 21591 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:54.222092 21823 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:54.285912 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:54.245491 21586 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.331436 21745 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 32.21, Last appended by leader: 21, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:54.304311 12671 raft_consensus.cc:606] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:54.270640 21722 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:54.293898 21746 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:54.337025 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:54.343467 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3568) took 1296 ms. Trace:
I20260709 12:13:54.343441 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:54.210990 21701 consensus_queue.cc:237] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:54.300172 21820 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:13:54.348284 12602 rpcz_store.cc:276] 0709 12:13:53.047464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.247099 (+1199635us) service_pool.cc:224] Handling call
0709 12:13:54.343367 (+ 96268us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:54.332981 21745 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:13:54.351210 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:54.351995 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4108) took 1202 ms. Trace:
I20260709 12:13:54.352272 12602 rpcz_store.cc:276] 0709 12:13:53.150035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.350856 (+1200821us) service_pool.cc:224] Handling call
0709 12:13:54.351937 (+  1081us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:54.353022 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:54.308140 21795 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:54.332113 21586 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:54.338471 21848 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.342657 21215 raft_consensus.cc:697] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:54.306053 21835 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:54.333966 21827 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:54.328193 21739 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:54.435537 21887 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.099s	user 0.000s	sys 0.000s
I20260709 12:13:54.337904 21823 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:54.442389 21730 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:13:54.443223 21730 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:54.448750 21828 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:13:54.456151 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:54.464022 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 235817us
W20260709 12:13:54.470235 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 272259us
I20260709 12:13:54.471948 12602 raft_consensus.cc:2410] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 28 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 15.
I20260709 12:13:54.474898 12523 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23. Ops: [22.13-22.13]
I20260709 12:13:54.478206 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
W20260709 12:13:54.479275 12643 leader_election.cc:336] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.827s (SENT)
I20260709 12:13:54.479697 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:54.480034 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:13:54.481290 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:13:54.440305 21823 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:54.462544 21853 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:54.489921 21879 raft_consensus.cc:3055] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:13:54.503307 21879 raft_consensus.cc:740] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:54.504515 21879 consensus_queue.cc:260] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:54.485026 21249 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:54.545004 12676 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
W20260709 12:13:54.513993 21878 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.292s	user 0.001s	sys 0.000s
I20260709 12:13:54.401440 21722 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 29.16, Last appended by leader: 16, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:54.543803 21586 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:54.510953 21742 raft_consensus.cc:3055] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:54.554198 21742 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:54.555423 21742 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:54.556581 21742 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:54.556864 21714 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
I20260709 12:13:54.506088 21879 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:54.463058 21848 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:54.562543 21848 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:54.506841 21591 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:54.520083 21853 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:54.533006 21886 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.515591 21739 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:54.517220 21769 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:54.579897 12603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3530) took 2629 ms (client timeout 2248 ms). Trace:
W20260709 12:13:54.580482 12603 rpcz_store.cc:269] 0709 12:13:51.950475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.172734 (+1222259us) service_pool.cc:224] Handling call
0709 12:13:54.579646 (+1406912us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b540023fd54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:54.579801 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":41099136}
I20260709 12:13:54.575222 12672 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 26. Current term is 27. Ops: []
I20260709 12:13:54.516759 21827 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:54.580992 12643 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:54.585840 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:54.588183 12676 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:13:54.593755 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3492) took 1512 ms. Trace:
W20260709 12:13:54.596426 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 749974us
I20260709 12:13:54.598520 12496 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:13:54.601378 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.594213 12528 rpcz_store.cc:276] 0709 12:13:53.081296 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.521743 (+440447us) service_pool.cc:224] Handling call
0709 12:13:53.538463 (+ 16720us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:53.538691 (+   228us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:13:53.720547 (+181856us) log.cc:844] Serialized 14 byte log entry
0709 12:13:54.593229 (+872682us) spinlock_profiling.cc:243] Waited 22.2 ms on lock 0x7b18004de5a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:13:54.593377 (+   148us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:54.593429 (+    52us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:54.593485 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:54.593552 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:54.593679 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8092,"prepare.queue_time_us":244876,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":41,"spinlock_wait_cycles":51056896,"thread_start_us":261056,"threads_started":2,"wal-append.queue_time_us":216155}
I20260709 12:13:54.566430 21670 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:54.576746 21853 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:13:54.610078 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.601785 12496 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:54.629762 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:54.627192 12571 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:54.631318 12603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3567) took 1584 ms (client timeout 1885 ms). Trace:
W20260709 12:13:54.632011 12603 rpcz_store.cc:269] 0709 12:13:53.047020 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.600353 (+1553333us) service_pool.cc:224] Handling call
0709 12:13:54.631209 (+ 30856us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:54.641683 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:54.642354 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:54.642817 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.514727 21215 consensus_queue.cc:237] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:54.649876 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:54.651080 12526 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:54.585444 21191 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.054s
I20260709 12:13:54.652436 12601 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 28: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:13:54.654922 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 74051.8us
I20260709 12:13:54.655820 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:54.658963 12672 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:13:54.660102 21740 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.660275 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:54.660676 21740 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:54.611488 21717 consensus_queue.cc:1059] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.051s
I20260709 12:13:54.662339 21701 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:13:54.663103 21701 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:54.663249 21740 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:54.663861 21740 raft_consensus.cc:3055] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:13:54.664182 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:54.664825 12528 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:54.648290 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:54.666373 12529 raft_consensus.cc:2393] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 22: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:54.641346 21822 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:13:54.669589 12531 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3553) took 3600 ms (client timeout 1899 ms). Trace:
I20260709 12:13:54.669891 21822 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:54.670696 12597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3512) took 2759 ms (client timeout 1877 ms). Trace:
W20260709 12:13:54.671218 12597 rpcz_store.cc:269] 0709 12:13:51.911688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:52.290676 (+378988us) service_pool.cc:224] Handling call
0709 12:13:54.670505 (+2379829us) spinlock_profiling.cc:243] Waited 1.78 s on lock 0x7b5400231254. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:54.670617 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4083605760}
W20260709 12:13:54.671861 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.637651 21718 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 12:13:54.676105 21755 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.676638 21755 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:54.678766 21739 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:54.678848 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 77144.4us
I20260709 12:13:54.679301 21791 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 12:13:54.678471 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.679515 21739 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:54.680475 21465 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:54.680975 21465 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:54.681367 21465 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:54.682782 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3642) took 1202 ms. Trace:
I20260709 12:13:54.683169 12679 rpcz_store.cc:276] 0709 12:13:53.480307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.844091 (+363784us) service_pool.cc:224] Handling call
0709 12:13:53.866812 (+ 22721us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:54.205635 (+338823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:54.682539 (+476904us) spinlock_profiling.cc:243] Waited 324 ms on lock 0x7b54003bcc54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:54.682717 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":744726912}
I20260709 12:13:54.684654 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:54.686323 21764 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:13:54.686647 21756 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:13:54.682066 21739 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:54.687987 21764 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 12:13:54.689666 21891 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.147s	user 0.000s	sys 0.000s
I20260709 12:13:54.509459 21811 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:54.690431 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:54.686766 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:54.691960 12603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3564) took 1692 ms (client timeout 1940 ms). Trace:
W20260709 12:13:54.692232 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:54.692438 12603 rpcz_store.cc:269] 0709 12:13:52.999227 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.690083 (+1690856us) service_pool.cc:224] Handling call
0709 12:13:54.691890 (+  1807us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:54.692912 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 46308.9us
I20260709 12:13:54.670500 21822 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:13:54.670110 12531 rpcz_store.cc:269] 0709 12:13:51.069301 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:51.069425 (+   124us) service_pool.cc:224] Handling call
0709 12:13:54.669403 (+3599978us) spinlock_profiling.cc:243] Waited 2.61 s on lock 0x7b540034fb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:54.669511 (+   108us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":5996781952}
I20260709 12:13:54.698879 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:54.699504 12529 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:13:54.692224 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3497) took 1208 ms. Trace:
I20260709 12:13:54.695103 21811 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:54.695191 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:54.696754 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:54.690745 12525 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 3493) took 1576 ms (client timeout 1715 ms). Trace:
I20260709 12:13:54.702467 12523 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
W20260709 12:13:54.703040 12525 rpcz_store.cc:269] 0709 12:13:53.114198 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.218708 (+104510us) service_pool.cc:224] Handling call
0709 12:13:54.690571 (+1471863us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b540007ce54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:54.690685 (+   114us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":756207360}
I20260709 12:13:54.697618 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:54.703980 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:54.704602 12643 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:54.700345 12522 rpcz_store.cc:276] 0709 12:13:53.483950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.642039 (+158089us) service_pool.cc:224] Handling call
0709 12:13:53.643211 (+  1172us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:53.643637 (+   426us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:53.643673 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:53.644200 (+   527us) log.cc:844] Serialized 26 byte log entry
0709 12:13:53.750895 (+106695us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:13:53.992023 (+241128us) spinlock_profiling.cc:243] Waited 4.36 ms on lock 0x7b1800482800. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf
0709 12:13:53.992239 (+   216us) log.cc:844] Serialized 14 byte log entry
0709 12:13:54.644659 (+652420us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:54.644775 (+   116us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:54.644836 (+    61us) raft_consensus.cc:1685] finished
0709 12:13:54.692012 (+ 47176us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:54.692166 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":652034,"prepare.queue_time_us":131214,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":130,"raft.queue_time_us":671311,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":216,"spinlock_wait_cycles":10020224,"thread_start_us":154319,"threads_started":2,"wal-append.queue_time_us":156564}
I20260709 12:13:54.710069 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:54.704258 12672 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:54.739704 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:54.740099 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:54.740583 13332 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 30: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:54.741019 12603 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:54.584118 21886 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:54.748476 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:54.750406 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:54.692777 21753 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:13:54.757539 12527 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 30: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:13:54.753823 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.758483 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 21 index: 10 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:54.759608 12643 leader_election.cc:336] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.869s (SENT)
I20260709 12:13:54.759785 12525 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:13:54.760246 12643 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:54.756548 12523 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 26. Current term is 27. Ops: []
I20260709 12:13:54.664198 21740 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:54.766219 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:54.767584 12531 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 29: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:54.767843 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:54.682019 21835 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:54.772017 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:54.763986 12643 leader_election.cc:336] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.899s (SENT)
I20260709 12:13:54.772696 12522 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:54.733777 21900 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:54.772859 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:54.771447 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:54.774369 12570 service_pool.cc:136] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51792 dropped due to backpressure: service queue is full with 50 items
I20260709 12:13:54.774734 12527 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 30: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:54.766350 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:54.775872 13334 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:13:54.739020 21827 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 21. Reason: could not achieve majority
W20260709 12:13:54.778817 12643 leader_election.cc:336] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.034s (SENT)
I20260709 12:13:54.779203 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:54.779502 12643 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:54.782243 12526 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:54.774806 12570 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51792 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3569)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3544)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3543)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3561)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3562)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3540)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4115)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4112)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1038)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4103)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1039)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1037)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3575)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3571)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1045)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1040)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4111)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3576)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4118)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4119)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1044)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1043)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4117)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3529)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4120)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4093)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1036)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3511)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3514)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3518)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3537)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3538)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3542)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3548)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3550)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3551)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3552)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4095)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4097)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4104)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4105)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4107)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4113)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4114)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3574)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4116)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1041)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1042)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1046)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1047)
I20260709 12:13:54.733269 21769 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.771538 21814 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:13:54.778029 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:54.787312 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:54.802233 12528 raft_consensus.cc:2376] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27. Re-sending same reply.
I20260709 12:13:54.788518 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:54.691740 21586 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.072s
I20260709 12:13:54.697870 21845 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 20.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 12:13:54.757530 21701 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.765235 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:54.811481 12643 leader_election.cc:336] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.152s (SENT)
I20260709 12:13:54.780021 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:54.794905 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:54.796743 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:54.786684 21896 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:13:54.798295 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:54.812592 12531 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:54.813670 12529 raft_consensus.cc:2376] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27. Re-sending same reply.
I20260709 12:13:54.816342 12523 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
W20260709 12:13:54.812517 12643 leader_election.cc:336] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.234s (SENT)
W20260709 12:13:54.816718 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 142708us
I20260709 12:13:54.821326 13338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 1084) took 1767 ms. Trace:
I20260709 12:13:54.821693 13338 rpcz_store.cc:276] 0709 12:13:53.047759 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.054194 (+  6435us) service_pool.cc:224] Handling call
0709 12:13:53.084193 (+ 29999us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:53.728104 (+643911us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:54.814389 (+1086285us) spinlock_profiling.cc:243] Waited 643 ms on lock 0x7b540046a5d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:54.814666 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1478750464}
I20260709 12:13:54.822237 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:54.824425 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:54.784662 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:54.771637 21753 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:54.779155 21900 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:54.801808 12527 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:54.793030 21587 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.762032 21843 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.037s
I20260709 12:13:54.829131 12603 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:13:54.807085 21835 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:13:54.830092 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:54.810325 21586 consensus_queue.cc:1243] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 30.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:13:54.834017 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:54.835366 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:54.834215 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:54.837090 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:54.837532 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:54.834682 12603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3570) took 1786 ms (client timeout 1899 ms). Trace:
W20260709 12:13:54.838335 12603 rpcz_store.cc:269] 0709 12:13:53.048312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.693337 (+1645025us) service_pool.cc:224] Handling call
0709 12:13:54.834569 (+141232us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:54.839903 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:54.841467 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:54.843037 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:54.843503 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:54.843813 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:54.823597 21718 raft_consensus.cc:3055] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:54.845319 12522 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 27. Current term is 28. Ops: [27.14-27.14]
I20260709 12:13:54.865921 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:54.853564 13332 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:13:54.858249 13338 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 29. Current term is 30. Ops: []
W20260709 12:13:54.858680 21907 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.063s	user 0.000s	sys 0.001s
I20260709 12:13:54.859243 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:54.862483 12675 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:54.862594 21670 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:13:54.850042 12570 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51782 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3544)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3543)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3561)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3562)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3540)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4115)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4112)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1038)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4103)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1039)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1037)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3575)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3571)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1045)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1040)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4111)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3576)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4118)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4119)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1044)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1043)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4117)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3529)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4120)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4093)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1036)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3511)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3514)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3518)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3537)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3538)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3542)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3548)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3550)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3551)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3552)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4095)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4097)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4104)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4105)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4107)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4113)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4114)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3574)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4116)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1041)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1042)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1046)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1047)
I20260709 12:13:54.807004 21769 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:54.942524 21769 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:54.872215 21895 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:13:54.880158 21465 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:54.945704 21465 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:54.886355 12530 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3593) took 1851 ms (client timeout 1514 ms). Trace:
I20260709 12:13:54.832115 21764 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:54.888617 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:54.889287 13330 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:13:54.817135 21701 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:54.889452 12570 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51782 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3544)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3543)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3561)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3562)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3540)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4115)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4112)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1038)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4103)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1039)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1037)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3575)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3571)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1045)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1040)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4111)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3576)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4118)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4122)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4119)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1044)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1043)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4117)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3529)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4120)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4093)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1036)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3511)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3514)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3518)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3537)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3538)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3542)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3548)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3550)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3551)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3552)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4095)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4097)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4104)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4105)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4107)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4113)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4114)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3574)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4116)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1041)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1042)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1046)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1047)
I20260709 12:13:54.896255 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:54.852941 21868 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.544s	user 0.009s	sys 0.002s
I20260709 12:13:54.900090 12571 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:48492 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3544)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3543)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3561)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3562)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3540)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4115)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4112)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1038)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4103)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1039)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1037)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3575)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3571)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1045)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1040)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4111)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3576)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4118)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4122)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4119)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1044)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1043)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4117)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3529)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4120)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4093)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1036)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3511)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3514)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3518)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3537)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3538)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3542)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3548)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3550)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3551)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3552)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4095)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4097)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4104)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4105)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4107)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4113)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4114)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3574)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4116)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1041)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1042)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1046)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1047)
I20260709 12:13:54.903815 21434 raft_consensus.cc:3055] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Stepping down as leader of term 25
W20260709 12:13:55.007493 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:54.909150 12604 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3556) took 2525 ms (client timeout 1827 ms). Trace:
W20260709 12:13:54.913913 21641 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.638s	user 0.012s	sys 0.000s
I20260709 12:13:54.911422 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:55.013176 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:54.916323 21791 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.918583 21753 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:54.924840 21881 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.707s	user 0.000s	sys 0.002s
I20260709 12:13:54.924552 21730 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:54.926537 21724 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:54.845280 21901 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:54.836040 21587 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:55.007629 21717 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:54.932785 21911 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
W20260709 12:13:54.935843 21576 consensus_meta.cc:287] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.705s	user 0.014s	sys 0.000s
I20260709 12:13:55.019095 21641 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:55.040958 21576 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:55.043470 21641 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:55.044006 21576 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:54.815533 21740 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:55.046247 12599 raft_consensus.cc:2368] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23.
W20260709 12:13:54.936705 21876 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:13:54.938576 21742 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.049857 21742 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:55.051713 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3633) took 1837 ms. Trace:
I20260709 12:13:54.827334 21900 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:55.053478 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3644) took 1566 ms. Trace:
I20260709 12:13:55.053908 12674 rpcz_store.cc:276] 0709 12:13:53.487177 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.845416 (+358239us) service_pool.cc:224] Handling call
0709 12:13:53.845712 (+   296us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:53.879036 (+ 33324us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.053235 (+1174199us) spinlock_profiling.cc:243] Waited 21.7 ms on lock 0x7b540008e154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:55.053415 (+   180us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":49879936}
I20260709 12:13:55.055217 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:55.055908 12674 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:55.058210 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:55.058759 21914 log.cc:927] Time spent T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.200s	user 0.002s	sys 0.000s
W20260709 12:13:54.947360 21815 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 27 current term: 27 is higher.
I20260709 12:13:54.954224 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:55.060264 13337 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 25: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:13:55.052699 21742 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:54.988204 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:54.963843 12606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3532) took 3012 ms (client timeout 2746 ms). Trace:
I20260709 12:13:55.062386 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:55.062538 12606 rpcz_store.cc:269] 0709 12:13:51.951164 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.011097 (+2059933us) service_pool.cc:224] Handling call
0709 12:13:54.963501 (+952404us) spinlock_profiling.cc:243] Waited 54.2 ms on lock 0x7b54002e63d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:54.963701 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":124604672}
W20260709 12:13:55.063633 12606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3544) took 3068 ms (client timeout 2967 ms). Trace:
W20260709 12:13:55.064179 12606 rpcz_store.cc:269] 0709 12:13:51.994786 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.063410 (+3068624us) service_pool.cc:211] Skipping call since client already timed out
0709 12:13:55.063550 (+   140us) inbound_call.cc:177] Queueing failure response
Metrics: {}
W20260709 12:13:55.064993 12606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3543) took 3110 ms (client timeout 3064 ms). Trace:
W20260709 12:13:54.954792 12600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3508) took 3046 ms (client timeout 2152 ms). Trace:
W20260709 12:13:54.979853 12601 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3560) took 2107 ms (client timeout 2034 ms). Trace:
W20260709 12:13:54.990284 12530 rpcz_store.cc:269] 0709 12:13:53.035061 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.054282 (+ 19221us) service_pool.cc:224] Handling call
0709 12:13:54.886101 (+1831819us) spinlock_profiling.cc:243] Waited 409 ms on lock 0x7b540038f4d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:54.886275 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":941158656}
I20260709 12:13:54.966951 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:54.904174 21756 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.016635 21465 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.052s
I20260709 12:13:54.828095 21814 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.049s
I20260709 12:13:55.070595 12674 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:13:55.073150 21911 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:13:55.081377 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:55.081810 21911 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:55.081923 12598 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3572) took 1992 ms (client timeout 1514 ms). Trace:
I20260709 12:13:55.015033 12675 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
W20260709 12:13:55.082434 12598 rpcz_store.cc:269] 0709 12:13:53.093145 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.766237 (+673092us) service_pool.cc:224] Handling call
0709 12:13:55.081690 (+1315453us) spinlock_profiling.cc:243] Waited 587 ms on lock 0x7b54003110d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:55.081828 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1350526720}
I20260709 12:13:55.082862 21911 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:55.083699 12598 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3561) took 2180 ms (client timeout 2152 ms). Trace:
I20260709 12:13:55.084275 21911 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:13:55.084656 21837 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 26.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
W20260709 12:13:54.997287 21795 consensus_meta.cc:287] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.633s	user 0.006s	sys 0.006s
W20260709 12:13:54.973340 21908 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.116s	user 0.003s	sys 0.000s
W20260709 12:13:55.072743 21745 consensus_meta.cc:287] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.717s	user 0.012s	sys 0.001s
I20260709 12:13:55.073858 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:55.088442 12677 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 23 index: 14. (index mismatch)
I20260709 12:13:54.972147 21915 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.090749 21815 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:13:55.091641 21815 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:54.949995 21722 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:13:55.067655 12605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3545) took 3050 ms (client timeout 2407 ms). Trace:
I20260709 12:13:55.092635 21815 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.14, Last appended by leader: 14, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:55.093266 12605 rpcz_store.cc:269] 0709 12:13:52.017638 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.067464 (+3049826us) service_pool.cc:211] Skipping call since client already timed out
0709 12:13:55.067586 (+   122us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:13:55.090246 21717 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:55.094659 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:55.008222 12604 rpcz_store.cc:269] 0709 12:13:52.383820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.471376 (+1087556us) service_pool.cc:224] Handling call
0709 12:13:54.908853 (+1437477us) spinlock_profiling.cc:243] Waited 254 ms on lock 0x7b54002e54d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:54.909008 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":584132480}
I20260709 12:13:55.096771 21717 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:55.097476 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:54.878211 21713 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:55.098537 12604 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3540) took 3145 ms (client timeout 3221 ms). Trace:
I20260709 12:13:55.098453 21713 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:55.099113 12604 rpcz_store.cc:269] 0709 12:13:51.953429 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.097167 (+3143738us) service_pool.cc:224] Handling call
0709 12:13:55.098456 (+  1289us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.032262 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:55.010643 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:54.836542 21755 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:54.954682 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:55.059504 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:55.063593 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:55.060969 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:55.079149 12600 rpcz_store.cc:269] 0709 12:13:51.907857 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.976720 (+2068863us) service_pool.cc:224] Handling call
0709 12:13:54.954451 (+977731us) spinlock_profiling.cc:243] Waited 199 ms on lock 0x7b54002ffb54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:54.954601 (+   150us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":457445248}
I20260709 12:13:55.023550 21791 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:55.079932 21756 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:55.080593 21814 consensus_queue.cc:1243] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 28.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:13:55.025522 21845 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.083455 12674 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:13:55.091177 21915 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:55.012352 21868 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:55.007637 21434 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:13:55.090219 12601 rpcz_store.cc:269] 0709 12:13:52.872757 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.307458 (+1434701us) service_pool.cc:224] Handling call
0709 12:13:54.979538 (+672080us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b5400079c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:54.979741 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":694566272}
I20260709 12:13:55.101042 12526 raft_consensus.cc:2368] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27.
I20260709 12:13:55.101135 12523 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:13:55.101629 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:55.188714 12600 raft_consensus.cc:2393] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 22: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:54.871704 21718 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:55.219590 21718 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:55.082548 12530 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 24: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:55.229426 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:55.095642 12605 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
W20260709 12:13:55.231287 12605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3562) took 2306 ms (client timeout 2234 ms). Trace:
W20260709 12:13:55.231818 12605 rpcz_store.cc:269] 0709 12:13:52.924590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.094273 (+2169683us) service_pool.cc:224] Handling call
0709 12:13:55.231194 (+136921us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.233121 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:55.065317 12606 rpcz_store.cc:269] 0709 12:13:51.964713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.064809 (+3100096us) service_pool.cc:211] Skipping call since client already timed out
0709 12:13:55.064936 (+   127us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:13:55.234827 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:55.235960 21915 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:55.100281 12678 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
W20260709 12:13:55.033072 21358 consensus_meta.cc:287] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.897s	user 0.012s	sys 0.002s
I20260709 12:13:55.052192 12680 rpcz_store.cc:276] 0709 12:13:53.214003 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:53.738013 (+524010us) service_pool.cc:224] Handling call
0709 12:13:53.740076 (+  2063us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:53.858336 (+118260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:53.858367 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:53.858986 (+   619us) log.cc:844] Serialized 52 byte log entry
0709 12:13:54.066962 (+207976us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:54.067818 (+   856us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:54.067928 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:54.909983 (+842055us) spinlock_profiling.cc:243] Waited 19 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:55.051330 (+141347us) raft_consensus.cc:1685] finished
0709 12:13:55.051365 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.051642 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":21074,"raft.queue_time_us":890,"raft.run_cpu_time_us":194,"raft.run_wall_time_us":19516,"spinlock_wait_cycles":43706752,"thread_start_us":133512,"threads_started":1,"wal-append.queue_time_us":219947}
I20260709 12:13:55.242082 12673 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 27: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:13:55.084070 12598 rpcz_store.cc:269] 0709 12:13:52.903168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.083515 (+2180347us) service_pool.cc:211] Skipping call since client already timed out
0709 12:13:55.083636 (+   121us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:13:55.243464 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:55.244267 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:55.026643 12570 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51782 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3544)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3543)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3561)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3562)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3540)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4115)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4112)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1038)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4103)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1039)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1037)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3575)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3571)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1045)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1040)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4111)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3576)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4118)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4122)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4119)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1044)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1043)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4117)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3529)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4120)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4093)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1036)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3511)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3514)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3518)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3537)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3538)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3542)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3548)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3550)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3551)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3552)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4095)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4097)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4104)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4105)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4107)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4113)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4114)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3574)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4116)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1041)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1042)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1046)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1047)
I20260709 12:13:55.078791 12646 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:55.106817 12531 raft_consensus.cc:2376] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27. Re-sending same reply.
I20260709 12:13:55.244972 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:55.116107 12603 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
W20260709 12:13:55.072065 21879 consensus_meta.cc:287] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.513s	user 0.010s	sys 0.000s
I20260709 12:13:55.119802 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:55.247432 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3575) took 1348 ms. Trace:
I20260709 12:13:55.102413 21815 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:55.065635 21701 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:55.077443 21822 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:55.253022 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:55.253774 12530 raft_consensus.cc:2410] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 28 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 15.
I20260709 12:13:55.190006 21745 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.257876 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:55.258880 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:55.259783 12677 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:13:55.228266 21886 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:55.179658 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:55.181325 12674 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
I20260709 12:13:55.181770 21755 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:55.182070 21756 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:55.182657 21099 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.184239 21791 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:55.190061 12600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4112) took 1618 ms (client timeout 1728 ms). Trace:
W20260709 12:13:55.288291 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 278871us
W20260709 12:13:55.292182 12600 rpcz_store.cc:269] 0709 12:13:53.571900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.101302 (+1529402us) service_pool.cc:224] Handling call
0709 12:13:55.189872 (+ 88570us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.254796 21742 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:55.216475 21640 raft_consensus.cc:3055] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:13:55.216842 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:55.218462 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:55.300341 12523 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 27: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:13:55.224599 12599 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3573) took 2092 ms (client timeout 1693 ms). Trace:
W20260709 12:13:55.301342 12599 rpcz_store.cc:269] 0709 12:13:53.131851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.160938 (+1029087us) service_pool.cc:224] Handling call
0709 12:13:55.224360 (+1063422us) spinlock_profiling.cc:243] Waited 841 ms on lock 0x7b540007b054. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:55.224521 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1933226240}
I20260709 12:13:55.302564 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 20 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.221132 21718 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:55.247205 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.123061 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.311807 12604 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
W20260709 12:13:55.312389 12604 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4115) took 1732 ms (client timeout 1712 ms). Trace:
W20260709 12:13:55.312812 12604 rpcz_store.cc:269] 0709 12:13:53.579605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.100003 (+1520398us) service_pool.cc:224] Handling call
0709 12:13:55.312312 (+212309us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.248044 12598 rpcz_store.cc:276] 0709 12:13:53.898882 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.244019 (+1345137us) service_pool.cc:224] Handling call
0709 12:13:55.247349 (+  3330us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.314976 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.316005 12598 raft_consensus.cc:2368] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27.
I20260709 12:13:55.317056 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:55.247395 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 168705us
I20260709 12:13:55.247403 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.321170 12597 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
W20260709 12:13:55.321830 12597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4103) took 3149 ms (client timeout 3282 ms). Trace:
W20260709 12:13:55.322312 12597 rpcz_store.cc:269] 0709 12:13:52.172497 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.223047 (+3050550us) service_pool.cc:224] Handling call
0709 12:13:55.321754 (+ 98707us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.236749 21837 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:13:55.336887 21868 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:55.263962 21745 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:55.339469 21745 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:55.238271 21764 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:55.273916 12672 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28. Ops: []
I20260709 12:13:55.193212 21743 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.064s
I20260709 12:13:55.340301 12572 service_pool.cc:137] RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51792 dropped due to backpressure: service queue is full with 50 items Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4119)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1044)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1043)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4117)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3529)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4120)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1049)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4125)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4093)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1036)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3511)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3514)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3518)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3537)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3538)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3542)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3548)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3550)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3551)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3552)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4095)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4097)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4104)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4105)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4107)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4113)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4114)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3574)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4116)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1041)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1042)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1046)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1047)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4126)
I20260709 12:13:55.289906 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:55.372411 12525 raft_consensus.cc:2410] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 26 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:13:55.132280 21845 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:55.290887 12527 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:55.293313 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:55.374177 12673 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 30 index: 16. (index mismatch)
I20260709 12:13:55.068708 21587 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:55.375886 12603 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3569) took 2327 ms (client timeout 1902 ms). Trace:
W20260709 12:13:55.376555 12603 rpcz_store.cc:269] 0709 12:13:53.047986 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.839668 (+1791682us) service_pool.cc:224] Handling call
0709 12:13:55.375675 (+536007us) spinlock_profiling.cc:243] Waited 246 ms on lock 0x7b5400375d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:55.375798 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":566564096}
W20260709 12:13:55.377530 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.216507 21434 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:55.296638 12674 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 17. (index mismatch)
I20260709 12:13:55.378017 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:55.381198 12603 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:13:55.339659 21917 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.069s	user 0.000s	sys 0.001s
I20260709 12:13:55.317775 12598 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:55.347518 12643 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:55.352180 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.353008 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.353570 12523 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:13:55.353020 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:55.298884 21640 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:55.294937 21099 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:55.278364 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 273460us
I20260709 12:13:55.364189 12679 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:55.291508 21755 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:55.331099 21837 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:55.367973 21889 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Stepping down as leader of term 28
W20260709 12:13:55.124065 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 106139us
I20260709 12:13:55.617559 13305 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:55.623781 12604 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
W20260709 12:13:55.624531 12604 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4111) took 2053 ms (client timeout 2711 ms). Trace:
W20260709 12:13:55.624984 12604 rpcz_store.cc:269] 0709 12:13:53.571619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.313695 (+1742076us) service_pool.cc:224] Handling call
0709 12:13:55.624447 (+310752us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.394052 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.622308 12569 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:55.621413 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 214771us
I20260709 12:13:55.626720 12597 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:13:55.600845 12601 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 33.
I20260709 12:13:55.621208 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.433290 12606 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
W20260709 12:13:55.636623 12601 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1038) took 1968 ms (client timeout 1670 ms). Trace:
W20260709 12:13:55.639068 12606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1037) took 1971 ms (client timeout 1842 ms). Trace:
W20260709 12:13:55.639606 12606 rpcz_store.cc:269] 0709 12:13:53.667954 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.234447 (+1566493us) service_pool.cc:224] Handling call
0709 12:13:55.638994 (+404547us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:55.639123 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 291720us
I20260709 12:13:55.642280 21822 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:55.647217 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.649686 12569 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:55.651664 12496 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:13:55.640820 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.641599 12673 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 13. (term mismatch)
I20260709 12:13:55.653594 12673 pending_rounds.cc:85] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 12
I20260709 12:13:55.653977 12673 pending_rounds.cc:107] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 21.13
I20260709 12:13:55.648597 12643 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:55.643663 12646 leader_election.cc:400] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27.
I20260709 12:13:55.655723 12646 leader_election.cc:403] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:13:55.635185 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1044) took 1132 ms. Trace:
I20260709 12:13:55.655864 12603 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:13:55.638571 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1040) took 1384 ms. Trace:
I20260709 12:13:55.656584 12599 rpcz_store.cc:276] 0709 12:13:54.254597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.303677 (+1049080us) service_pool.cc:224] Handling call
0709 12:13:55.638508 (+334831us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.656948 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.657536 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:13:55.644513 12602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3566) took 2597 ms (client timeout 1869 ms). Trace:
W20260709 12:13:55.658249 12602 rpcz_store.cc:269] 0709 12:13:53.046697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.352826 (+1306129us) service_pool.cc:224] Handling call
0709 12:13:55.644257 (+1291431us) spinlock_profiling.cc:243] Waited 118 ms on lock 0x7b540007a654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:55.644415 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":271658752}
W20260709 12:13:55.376631 21812 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.241s	user 0.015s	sys 0.000s
I20260709 12:13:55.654402 12673 raft_consensus.cc:2889] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:13:55.639292 12601 rpcz_store.cc:269] 0709 12:13:53.668351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.187011 (+1518660us) service_pool.cc:224] Handling call
0709 12:13:55.636556 (+449545us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.357426 21713 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:55.656250 12598 rpcz_store.cc:276] 0709 12:13:54.502652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.393756 (+891104us) service_pool.cc:224] Handling call
0709 12:13:55.635112 (+241356us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.480428 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:55.632000 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:55.660883 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:55.648602 12606 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3529) took 3700 ms (client timeout 4673 ms). Trace:
W20260709 12:13:55.627967 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:55.662379 12606 rpcz_store.cc:269] 0709 12:13:51.948743 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.646859 (+3698116us) service_pool.cc:224] Handling call
0709 12:13:55.648537 (+  1678us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.657799 12603 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
W20260709 12:13:55.630367 13302 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.669s (SENT) [suppressed 63 similar messages]
W20260709 12:13:55.593772 21920 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.200s	user 0.000s	sys 0.001s
I20260709 12:13:55.665555 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.666543 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1043) took 1202 ms. Trace:
I20260709 12:13:55.666949 12604 rpcz_store.cc:276] 0709 12:13:54.464523 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.625867 (+1161344us) service_pool.cc:224] Handling call
0709 12:13:55.666482 (+ 40615us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:55.669113 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.592969 21587 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:13:55.671597 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:55.672231 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.672405 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:55.647541 21738 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:55.648727 21743 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:13:55.673547 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.666108 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:55.655166 21434 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:55.678309 12601 raft_consensus.cc:2376] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27. Re-sending same reply.
I20260709 12:13:55.673049 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:55.679215 12529 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:13:55.679114 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:55.699333 21358 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.699931 21358 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:55.701048 21837 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 29.15, Last appended by leader: 15, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:55.621637 21889 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:55.680135 13302 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:55.676575 21904 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 27
I20260709 12:13:55.715924 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:55.498005 21900 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:55.670316 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:55.719018 12643 leader_election.cc:336] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51792 dropped due to backpressure: service queue is full with 50 items
I20260709 12:13:55.358152 21764 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:55.675200 12496 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:55.687572 12599 raft_consensus.cc:2368] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23.
I20260709 12:13:55.716861 12673 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:55.721170 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.687846 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:55.698364 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:55.699182 21551 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:13:55.702576 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:55.727197 12604 raft_consensus.cc:2376] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27. Re-sending same reply.
I20260709 12:13:55.727432 21755 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:55.704569 21837 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:13:55.728192 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:55.728802 12604 raft_consensus.cc:2376] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23. Re-sending same reply.
I20260709 12:13:55.679535 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.672151 21753 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:55.681656 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.671047 21745 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:55.673198 21868 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:55.680279 21812 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:55.743669 12676 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 24: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:55.745267 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.714915 21743 raft_consensus.cc:697] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:55.721835 12599 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:13:55.723814 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:55.724349 12526 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 27: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:55.750722 12604 raft_consensus.cc:2393] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 30: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:55.752153 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.725296 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:55.725626 12674 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:13:55.731031 12603 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:55.753100 12604 raft_consensus.cc:2410] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 31 because replica has last-logged OpId of term: 29 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 18.
I20260709 12:13:55.731551 12606 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:13:55.731174 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 11 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.755956 12601 raft_consensus.cc:2368] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
W20260709 12:13:55.735193 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 60036.2us
I20260709 12:13:55.719662 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:55.758530 12643 leader_election.cc:336] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51792 dropped due to backpressure: service queue is full with 50 items
I20260709 12:13:55.758934 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 18 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.721547 21358 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:55.759042 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.712656 21640 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:55.764997 12602 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:55.742369 21824 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:55.761991 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:55.739774 21868 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:55.739955 21812 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:55.800472 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 19 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.801692 12601 raft_consensus.cc:2410] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 19 index: 12.
I20260709 12:13:55.793818 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:55.802626 12597 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:55.804492 12602 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:55.805415 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4093) took 4113 ms. Trace:
I20260709 12:13:55.793736 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.806231 12603 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:13:55.805853 12602 rpcz_store.cc:276] 0709 12:13:51.692120 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.803944 (+4111824us) service_pool.cc:224] Handling call
0709 12:13:55.804304 (+   360us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.804912 (+   608us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.805306 (+   394us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.807237 12643 leader_election.cc:400] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27.
I20260709 12:13:55.808132 12602 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:55.808794 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3514) took 3878 ms. Trace:
I20260709 12:13:55.809180 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:55.819465 12606 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:13:55.820335 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3518) took 3888 ms. Trace:
I20260709 12:13:55.820766 12606 rpcz_store.cc:276] 0709 12:13:51.931494 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.818832 (+3887338us) service_pool.cc:224] Handling call
0709 12:13:55.819225 (+   393us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.820040 (+   815us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.820263 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.822589 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:55.823781 12604 raft_consensus.cc:2410] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 30 because replica has last-logged OpId of term: 29 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
I20260709 12:13:55.810098 12673 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
I20260709 12:13:55.825413 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:55.825977 12673 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:55.825975 12598 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:13:55.739555 21753 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:55.831223 12598 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:13:55.840593 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3538) took 3887 ms. Trace:
W20260709 12:13:55.806387 21853 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.156s	user 0.010s	sys 0.004s
I20260709 12:13:55.840939 12603 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:13:55.845889 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3511) took 3934 ms. Trace:
I20260709 12:13:55.846298 12603 rpcz_store.cc:276] 0709 12:13:51.911326 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.807011 (+3895685us) service_pool.cc:224] Handling call
0709 12:13:55.840578 (+ 33567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.845575 (+  4997us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.845814 (+   239us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.841017 12598 rpcz_store.cc:276] 0709 12:13:51.952923 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.827332 (+3874409us) service_pool.cc:224] Handling call
0709 12:13:55.827599 (+   267us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.840281 (+ 12682us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.840522 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.848448 12598 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:13:55.849202 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3551) took 3765 ms. Trace:
I20260709 12:13:55.849530 12598 rpcz_store.cc:276] 0709 12:13:52.083881 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.847924 (+3764043us) service_pool.cc:224] Handling call
0709 12:13:55.848256 (+   332us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.848950 (+   694us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.849138 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.793032 21814 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:13:55.851670 12598 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 29. Current term is 30. Ops: []
I20260709 12:13:55.798480 21824 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:55.801980 21868 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:55.801569 21701 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:13:55.800977 21812 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:55.852064 12673 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:13:55.851691 12599 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:13:55.838541 21640 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:13:55.865293 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3552) took 3771 ms. Trace:
I20260709 12:13:55.865192 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1036) took 4011 ms. Trace:
I20260709 12:13:55.865670 12598 rpcz_store.cc:276] 0709 12:13:52.084191 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.851117 (+3766926us) service_pool.cc:224] Handling call
0709 12:13:55.851511 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.855607 (+  4096us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.855829 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.866299 12599 rpcz_store.cc:276] 0709 12:13:51.854160 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.806325 (+3952165us) service_pool.cc:224] Handling call
0709 12:13:55.844661 (+ 38336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.864933 (+ 20272us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.865137 (+   204us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.867946 12598 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: []
I20260709 12:13:55.868754 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4095) took 3769 ms. Trace:
W20260709 12:13:55.858990 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 66833.4us
I20260709 12:13:55.859705 12606 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:55.869903 12598 rpcz_store.cc:276] 0709 12:13:52.098859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.866751 (+3767892us) service_pool.cc:224] Handling call
0709 12:13:55.867039 (+   288us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.868480 (+  1441us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.868706 (+   226us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.793144 21755 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.058s
I20260709 12:13:55.863596 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3548) took 3780 ms. Trace:
I20260709 12:13:55.858060 12601 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22. Ops: []
I20260709 12:13:55.874449 12598 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:13:55.875048 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4104) took 2726 ms. Trace:
I20260709 12:13:55.875048 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3542) took 3921 ms. Trace:
I20260709 12:13:55.875473 12598 rpcz_store.cc:276] 0709 12:13:53.148956 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.873828 (+2724872us) service_pool.cc:224] Handling call
0709 12:13:55.874136 (+   308us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.874827 (+   691us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.874994 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.875605 12601 rpcz_store.cc:276] 0709 12:13:51.953941 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.829350 (+3875409us) service_pool.cc:224] Handling call
0709 12:13:55.857818 (+ 28468us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.874827 (+ 17009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.874994 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.873478 12604 rpcz_store.cc:276] 0709 12:13:52.082856 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.839280 (+3756424us) service_pool.cc:224] Handling call
0709 12:13:55.862918 (+ 23638us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.863153 (+   235us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:55.863194 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:55.863253 (+    59us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:55.863312 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.863382 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.863519 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.870870 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3537) took 3918 ms. Trace:
I20260709 12:13:55.792904 21922 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.069s
I20260709 12:13:55.877236 12606 rpcz_store.cc:276] 0709 12:13:51.952655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.821746 (+3869091us) service_pool.cc:224] Handling call
0709 12:13:55.822105 (+   359us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.870616 (+ 48511us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.870819 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.879487 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4114) took 2307 ms. Trace:
I20260709 12:13:55.879967 12606 rpcz_store.cc:276] 0709 12:13:53.572394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.878235 (+2305841us) service_pool.cc:224] Handling call
0709 12:13:55.878599 (+   364us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.879050 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:55.879091 (+    41us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:55.879149 (+    58us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:55.879203 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.879277 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.879421 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.881662 12604 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:13:55.864212 21824 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:55.926282 21900 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:55.927456 21764 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:55.864351 21358 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.048s
I20260709 12:13:55.886560 12600 raft_consensus.cc:2410] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 26 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:13:55.882558 12598 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:13:55.929760 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4105) took 2780 ms. Trace:
I20260709 12:13:55.809104 12602 rpcz_store.cc:276] 0709 12:13:51.930404 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.807630 (+3877226us) service_pool.cc:224] Handling call
0709 12:13:55.807963 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.808552 (+   589us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.808734 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.770680 21889 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:55.907033 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4097) took 3784 ms. Trace:
I20260709 12:13:55.932942 12599 rpcz_store.cc:276] 0709 12:13:52.122686 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.867270 (+3744584us) service_pool.cc:224] Handling call
0709 12:13:55.906374 (+ 39104us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.906594 (+   220us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:55.906620 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:55.906676 (+    56us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:55.906727 (+    51us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.906804 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.906952 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:55.934388 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:55.872879 12496 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:55.917340 12601 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28. Ops: []
I20260709 12:13:55.937444 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4107) took 2787 ms. Trace:
I20260709 12:13:55.851773 21853 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:55.914587 21099 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:55.922173 12597 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:13:55.954300 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4117) took 1260 ms. Trace:
I20260709 12:13:55.954941 12597 rpcz_store.cc:276] 0709 12:13:54.694011 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.755677 (+1061666us) service_pool.cc:224] Handling call
0709 12:13:55.954230 (+198553us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.928766 12606 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:13:55.959163 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3574) took 2203 ms. Trace:
I20260709 12:13:55.960552 12606 rpcz_store.cc:276] 0709 12:13:53.755785 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.923284 (+2167499us) service_pool.cc:224] Handling call
0709 12:13:55.923590 (+   306us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.958790 (+ 35200us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.959092 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.923473 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4113) took 2351 ms. Trace:
I20260709 12:13:55.963177 12604 rpcz_store.cc:276] 0709 12:13:53.572139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.876940 (+2304801us) service_pool.cc:224] Handling call
0709 12:13:55.881181 (+  4241us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.923184 (+ 42003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.923421 (+   237us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:55.971989 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 100415us
I20260709 12:13:55.973362 12496 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:55.963884 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1046) took 1294 ms. Trace:
I20260709 12:13:55.937911 12601 rpcz_store.cc:276] 0709 12:13:53.149783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.876628 (+2726845us) service_pool.cc:224] Handling call
0709 12:13:55.917067 (+ 40439us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.937090 (+ 20023us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.937368 (+   278us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.954419 12603 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 27. Current term is 28. Ops: []
I20260709 12:13:55.976680 12599 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:13:55.988310 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1041) took 1733 ms. Trace:
I20260709 12:13:55.988761 12599 rpcz_store.cc:276] 0709 12:13:54.254938 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.934223 (+1679285us) service_pool.cc:224] Handling call
0709 12:13:55.976289 (+ 42066us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.988020 (+ 11731us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.988247 (+   227us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.931852 12602 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 10. Preceding OpId from leader: term: 25 index: 12. (index mismatch)
I20260709 12:13:55.980063 12605 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 25: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:55.990497 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4116) took 2174 ms. Trace:
I20260709 12:13:55.969655 21889 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:55.991309 12602 rpcz_store.cc:276] 0709 12:13:53.816080 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.931050 (+2114970us) service_pool.cc:224] Handling call
0709 12:13:55.931430 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.990170 (+ 58740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.990404 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.930214 12598 rpcz_store.cc:276] 0709 12:13:53.149259 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.876415 (+2727156us) service_pool.cc:224] Handling call
0709 12:13:55.882322 (+  5907us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:55.929200 (+ 46878us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.929694 (+   494us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:55.905675 21743 consensus_queue.cc:237] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:55.987294 21927 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.081s	user 0.000s	sys 0.000s
I20260709 12:13:55.980336 21756 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:13:55.979764 21099 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election won for term 27
W20260709 12:13:56.002018 21926 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.132s	user 0.000s	sys 0.001s
I20260709 12:13:56.002511 21764 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:56.020812 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:55.988602 21924 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.107s	user 0.000s	sys 0.000s
I20260709 12:13:56.010952 12599 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:13:55.983608 21755 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 27.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 12:13:56.053908 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3659) took 1375 ms. Trace:
I20260709 12:13:56.054435 12671 rpcz_store.cc:276] 0709 12:13:54.678214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.060262 (+382048us) service_pool.cc:224] Handling call
0709 12:13:55.060572 (+   310us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:55.060941 (+   369us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:55.060967 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:55.061466 (+   499us) log.cc:844] Serialized 26 byte log entry
0709 12:13:55.417455 (+355989us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:55.417515 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.417613 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:56.053548 (+635935us) raft_consensus.cc:1685] finished
0709 12:13:56.053575 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.053828 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":233,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":90,"thread_start_us":355599,"threads_started":1,"wal-append.queue_time_us":686941}
W20260709 12:13:56.053342 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:56.057298 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.017185 12528 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 28. Current term is 29. Ops: []
I20260709 12:13:56.079574 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1159) took 1097 ms. Trace:
I20260709 12:13:56.079932 12528 rpcz_store.cc:276] 0709 12:13:54.982050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.016555 (+1034505us) service_pool.cc:224] Handling call
0709 12:13:56.016977 (+   422us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:56.079302 (+ 62325us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.079510 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:56.019649 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:55.987679 21925 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.085s	user 0.000s	sys 0.000s
W20260709 12:13:56.021445 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:56.022549 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 363518us
W20260709 12:13:56.083174 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.084261 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:55.988034 12606 rpcz_store.cc:276] 0709 12:13:54.669812 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.963041 (+1293229us) service_pool.cc:224] Handling call
0709 12:13:55.963357 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.963554 (+   197us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:55.963579 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:55.963621 (+    42us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:55.963661 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.963713 (+    52us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:55.963833 (+   120us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:56.000217 21358 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.031s
I20260709 12:13:56.036978 21868 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:56.010191 21099 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:56.007925 21756 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:56.050899 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4323) took 1311 ms. Trace:
I20260709 12:13:56.051929 21853 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:56.015687 21845 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:55.993635 21922 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.027s
I20260709 12:13:56.069888 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:56.114988 13336 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 25: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:13:56.115360 21923 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.313s	user 0.005s	sys 0.000s
I20260709 12:13:56.073428 21714 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:56.116211 12600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3571) took 3067 ms (client timeout 3123 ms). Trace:
I20260709 12:13:56.116624 21714 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:56.027640 21911 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:56.116648 12600 rpcz_store.cc:269] 0709 12:13:53.048634 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.293016 (+2244382us) service_pool.cc:224] Handling call
0709 12:13:56.116018 (+823002us) spinlock_profiling.cc:243] Waited 512 ms on lock 0x7b5400230d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:56.116156 (+   138us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1176864512}
W20260709 12:13:56.078718 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 407585us
W20260709 12:13:56.085973 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:56.078181 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 383605us
I20260709 12:13:56.053347 21764 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:56.094437 12680 rpcz_store.cc:276] 0709 12:13:54.739054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.242373 (+503319us) service_pool.cc:224] Handling call
0709 12:13:55.242802 (+   429us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:55.243243 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:55.243278 (+    35us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:13:55.250736 (+  7458us) log.cc:844] Serialized 52 byte log entry
0709 12:13:55.817194 (+566458us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:13:55.817239 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.817308 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:56.050341 (+233033us) raft_consensus.cc:1685] finished
0709 12:13:56.050370 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.050797 (+   427us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6020,"raft.queue_time_us":2518,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":7070,"thread_start_us":224434,"threads_started":1,"wal-append.queue_time_us":672028}
I20260709 12:13:56.089249 21824 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:56.097129 21937 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:13:56.098078 21929 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.079s	user 0.000s	sys 0.000s
I20260709 12:13:56.102320 21756 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:13:56.105181 21868 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:56.106469 21904 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27.
I20260709 12:13:56.095273 21099 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:56.111492 21434 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.112036 12572 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:56.148828 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:56.119087 21714 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:56.127871 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:56.155627 21824 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:56.158720 21255 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election lost for term 22. Reason: could not achieve majority
I20260709 12:13:56.135387 21911 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:56.131749 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.162317 21911 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:56.164392 12528 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 16. (term mismatch)
I20260709 12:13:56.164880 12528 pending_rounds.cc:85] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 15
I20260709 12:13:56.165169 12528 pending_rounds.cc:107] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 26.16
I20260709 12:13:56.165469 12528 raft_consensus.cc:2889] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:56.166440 21853 raft_consensus.cc:2764] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:13:56.168296 21434 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:56.171386 21434 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:56.142313 21880 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:56.175124 21587 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:13:56.133360 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1047) took 1463 ms. Trace:
I20260709 12:13:56.111249 21730 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:56.140532 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:56.160351 21764 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:56.177209 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4324) took 1430 ms. Trace:
I20260709 12:13:56.178035 13304 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:56.178695 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:56.182699 12677 rpcz_store.cc:276] 0709 12:13:54.746869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.260805 (+513936us) service_pool.cc:224] Handling call
0709 12:13:55.261234 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:55.277452 (+ 16218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:55.277498 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:55.278174 (+   676us) log.cc:844] Serialized 26 byte log entry
0709 12:13:55.680939 (+402765us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:13:55.681001 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.681099 (+    98us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:56.176778 (+495679us) raft_consensus.cc:1685] finished
0709 12:13:56.176810 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.177112 (+   302us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15603,"raft.queue_time_us":25306,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":5218,"thread_start_us":334453,"threads_started":1,"wal-append.queue_time_us":657413}
I20260709 12:13:56.179280 12680 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:56.121762 21753 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:56.180250 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:56.185702 13336 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
W20260709 12:13:56.181025 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.132845 21937 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:56.176990 12604 rpcz_store.cc:276] 0709 12:13:54.670149 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.963911 (+1293762us) service_pool.cc:224] Handling call
0709 12:13:55.985274 (+ 21363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.985544 (+   270us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:13:56.028889 (+ 43345us) log.cc:844] Serialized 14 byte log entry
0709 12:13:56.132542 (+103653us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:56.132653 (+   111us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:13:56.132999 (+   346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.133091 (+    92us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.133273 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":36643,"prepare.queue_time_us":40329,"prepare.run_cpu_time_us":29,"prepare.run_wall_time_us":29,"raft.queue_time_us":37730,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":3716,"thread_start_us":108414,"threads_started":2,"wal-append.queue_time_us":117984}
W20260709 12:13:56.190950 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.189293 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:56.193791 13330 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:56.197265 12572 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:56.193346 21937 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:56.137893 12646 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:56.188980 21939 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:13:56.139866 21848 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.055s
I20260709 12:13:56.205873 12572 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:56.206725 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:56.187381 21886 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:56.141460 21755 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Stepping down as leader of term 27
W20260709 12:13:56.224972 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.196046 21853 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:56.205966 21751 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:13:56.216415 13338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 1093) took 1285 ms. Trace:
I20260709 12:13:56.244300 13338 rpcz_store.cc:276] 0709 12:13:54.930896 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.931116 (+   220us) service_pool.cc:224] Handling call
0709 12:13:54.931611 (+   495us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:54.978400 (+ 46789us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:54.978447 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:54.979142 (+   695us) log.cc:844] Serialized 26 byte log entry
0709 12:13:55.357919 (+378777us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:55.357963 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.109254 (+751291us) spinlock_profiling.cc:243] Waited 9.19 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:56.215961 (+106707us) spinlock_profiling.cc:243] Waited 46.3 ms on lock 0x7b5400478454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:56.216022 (+    61us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:56.216092 (+    70us) raft_consensus.cc:1685] finished
0709 12:13:56.216115 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.216331 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6948,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":8213,"spinlock_wait_cycles":127630464,"thread_start_us":378310,"threads_started":1,"wal-append.queue_time_us":753349}
W20260709 12:13:56.244225 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.197775 21255 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.032s
I20260709 12:13:56.217628 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:56.221020 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:56.222198 21936 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:13:56.230892 12523 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:13:56.261871 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:56.261741 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:56.237303 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:56.264755 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:56.265532 12600 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:56.239185 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:56.239602 12604 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 30 index: 16. (term mismatch)
I20260709 12:13:56.267328 12604 pending_rounds.cc:85] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 15
I20260709 12:13:56.240278 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:56.268435 12606 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:13:56.241990 12602 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:13:56.267656 12604 pending_rounds.cc:107] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 29.16
I20260709 12:13:56.270491 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:56.270623 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:56.269910 12604 raft_consensus.cc:2889] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:56.271524 12602 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:13:56.242216 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:56.252552 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1042) took 1997 ms. Trace:
I20260709 12:13:56.252661 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:56.215603 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 77815.4us
W20260709 12:13:56.253309 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 69840.5us
W20260709 12:13:56.254555 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.219671 21939 raft_consensus.cc:697] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 25 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:13:56.277635 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.228493 21845 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:13:56.274405 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.262746 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3617) took 1402 ms. Trace:
I20260709 12:13:56.224843 21755 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:56.277390 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:56.294013 12599 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:13:56.280186 12672 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 26: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:56.280224 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:56.281821 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:56.267597 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:56.296201 12601 raft_consensus.cc:2410] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 27 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:13:56.232894 21853 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:56.297133 21853 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:56.284546 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:56.285182 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:56.280447 12572 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:56.285166 12646 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:56.292927 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:56.304291 12604 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:13:56.296084 12598 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:56.282868 12523 rpcz_store.cc:276] 0709 12:13:54.860475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.211800 (+1351325us) service_pool.cc:224] Handling call
0709 12:13:56.230487 (+ 18687us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:56.262471 (+ 31984us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.262689 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:56.309785 12599 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:13:56.275940 12597 rpcz_store.cc:276] 0709 12:13:54.255221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.956538 (+1701317us) service_pool.cc:224] Handling call
0709 12:13:55.956932 (+   394us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.957192 (+   260us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:13:56.027319 (+ 70127us) log.cc:844] Serialized 14 byte log entry
0709 12:13:56.252207 (+224888us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:56.252259 (+    52us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:13:56.252316 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.252382 (+    66us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.252493 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":68407,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":54,"raft.queue_time_us":3093,"thread_start_us":272858,"threads_started":2}
I20260709 12:13:56.311689 12572 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:56.219542 21886 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:56.279171 21743 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:56.330478 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:56.332410 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:56.337296 12600 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:56.283427 21764 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:13:56.250852 21869 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.065s
I20260709 12:13:56.316247 12572 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:56.290663 21745 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:13:56.344534 12572 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:56.315243 21791 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.346524 21791 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:56.353358 12572 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:56.359433 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:56.329587 21434 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:13:56.332288 21751 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:56.329341 21868 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:13:56.355363 21255 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:13:56.368604 13330 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:13:56.366523 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:56.349195 21714 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.349377 21764 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:56.351029 21845 raft_consensus.cc:2764] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:13:56.329049 21900 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 12, Time since last communication: 0.036s
W20260709 12:13:56.407014 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 131695us
W20260709 12:13:56.451037 12494 leader_election.cc:336] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Remote error: Service unavailable: RequestConsensusVote request on kudu.consensus.ConsensusService from 127.0.0.1:51782 dropped due to backpressure: service queue is full with 50 items
I20260709 12:13:56.361593 21935 raft_consensus.cc:2764] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election decision vote started in defunct term 24: won
I20260709 12:13:56.453382 12494 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:56.464085 13302 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:56.463554 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 85658.1us
I20260709 12:13:56.372253 21751 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:56.485901 12494 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:56.490272 21751 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:56.491220 21751 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:56.491365 21753 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:56.492420 21753 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:56.402386 21434 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:56.492935 21753 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:56.431097 21764 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:13:56.530200 21944 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 12:13:56.533768 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:56.534890 12671 raft_consensus.cc:2410] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 28 because replica has last-logged OpId of term: 27 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 13.
I20260709 12:13:56.535315 12525 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 21 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:13:56.535808 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:56.537016 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:56.541934 12679 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 4305) took 1859 ms (client timeout 1728 ms). Trace:
W20260709 12:13:56.542759 21812 consensus_meta.cc:287] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.668s	user 0.007s	sys 0.005s
I20260709 12:13:56.543507 21812 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:56.544499 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:56.546217 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:56.546591 21791 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:56.378904 21755 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:56.445387 21714 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:56.542482 12679 rpcz_store.cc:269] 0709 12:13:54.682793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:54.779794 (+ 97001us) service_pool.cc:224] Handling call
0709 12:13:56.541727 (+1761933us) spinlock_profiling.cc:243] Waited 452 ms on lock 0x7b5400013e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:56.541852 (+   125us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1038809472}
I20260709 12:13:56.551203 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:56.553555 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.535950 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3619) took 1612 ms. Trace:
I20260709 12:13:56.554852 12525 rpcz_store.cc:276] 0709 12:13:54.923987 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.456528 (+1532541us) service_pool.cc:224] Handling call
0709 12:13:56.535006 (+ 78478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:56.535711 (+   705us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.535897 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:56.378453 21868 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:56.556849 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:56.565968 12671 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 25: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:13:56.570322 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 106358us
I20260709 12:13:56.573760 13302 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:56.575608 12599 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 32 index: 21. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
W20260709 12:13:56.601975 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:56.633090 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.583626 12672 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:13:56.557870 21465 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:13:56.674250 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.667263 12530 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 29. Current term is 30. Ops: [29.19-29.19]
I20260709 12:13:56.676319 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1157) took 1801 ms. Trace:
I20260709 12:13:56.676661 12530 rpcz_store.cc:276] 0709 12:13:54.874550 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.666366 (+1791816us) service_pool.cc:224] Handling call
0709 12:13:56.666898 (+   532us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:56.675982 (+  9084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.676245 (+   263us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:56.667625 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:56.678689 12600 raft_consensus.cc:2410] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 28 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 18.
W20260709 12:13:56.685654 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:56.571630 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 85859.2us
I20260709 12:13:56.698988 12494 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:56.700834 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:56.717594 12524 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:56.573295 21950 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:56.580613 21738 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.727923 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3606) took 1904 ms. Trace:
W20260709 12:13:56.685328 21930 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.112s	user 0.001s	sys 0.000s
I20260709 12:13:56.732024 12522 rpcz_store.cc:276] 0709 12:13:54.823409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.289233 (+465824us) service_pool.cc:224] Handling call
0709 12:13:55.312194 (+ 22961us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:55.744043 (+431849us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:55.744093 (+    50us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:55.749319 (+  5226us) log.cc:844] Serialized 26 byte log entry
0709 12:13:56.025526 (+276207us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:56.025588 (+    62us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.717048 (+691460us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b540042a9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:56.717111 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:56.717187 (+    76us) raft_consensus.cc:1685] finished
0709 12:13:56.717205 (+    18us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:56.727848 (+ 10643us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":992557440,"thread_start_us":209217,"threads_started":1,"wal-append.queue_time_us":285279}
I20260709 12:13:56.731338 21738 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:56.665725 21956 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:56.429914 21939 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:56.669845 21465 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:56.718367 21935 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.663370 21848 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.079s
I20260709 12:13:56.661825 21255 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:56.720588 21812 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:56.755453 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 54730.5us
I20260709 12:13:56.725871 21922 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:56.722150 21934 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.769053 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:56.770068 12604 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:13:56.766068 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 226308us
I20260709 12:13:56.771570 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:56.764580 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:13:56.765362 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 208676us
I20260709 12:13:56.766666 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:56.753412 21756 consensus_meta.cc:287] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.615s	user 0.010s	sys 0.007s
I20260709 12:13:56.760139 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:56.789321 12599 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:13:56.791155 12598 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:13:56.792366 12571 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:56.794727 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:56.758327 21465 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:56.802404 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:56.797089 12494 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:56.806073 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:56.808391 12600 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
W20260709 12:13:56.808506 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 148724us
I20260709 12:13:56.811772 12600 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:13:56.773876 21868 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:56.813992 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:56.821316 12600 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:56.819844 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:56.813858 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:56.823395 12679 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:56.824150 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:56.824884 12598 raft_consensus.cc:2368] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23.
I20260709 12:13:56.825343 12606 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:13:56.769788 21922 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:56.768704 21935 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:56.827448 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:56.829358 12602 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 16. (term mismatch)
I20260709 12:13:56.829909 12602 pending_rounds.cc:85] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 15
I20260709 12:13:56.830572 12602 pending_rounds.cc:107] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 25.16
I20260709 12:13:56.830873 12602 raft_consensus.cc:2889] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:56.829645 21935 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:56.836103 21935 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:13:56.840835 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:56.841598 12672 raft_consensus.cc:2376] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22. Re-sending same reply.
W20260709 12:13:56.845503 21937 consensus_meta.cc:287] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.647s	user 0.007s	sys 0.000s
I20260709 12:13:56.793062 21934 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:56.846588 21714 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:56.846488 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.805385 21812 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:56.859738 21812 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:56.860178 21812 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:56.845995 21937 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:56.752815 21950 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.062s
W20260709 12:13:56.866801 21961 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.126s	user 0.000s	sys 0.001s
I20260709 12:13:56.869277 21934 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:56.876075 12600 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:13:56.790524 21962 consensus_queue.cc:1059] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 27.14, Next index: 15, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:13:56.811170 21886 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:56.813287 21956 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:56.871778 21255 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:56.805048 21756 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:56.908135 12604 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:13:56.848876 21922 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:13:56.897118 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 100143us
W20260709 12:13:56.897769 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 103171us
W20260709 12:13:56.913683 12676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 4314) took 2228 ms (client timeout 2000 ms). Trace:
W20260709 12:13:56.914793 12676 rpcz_store.cc:269] 0709 12:13:54.685408 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.014454 (+329046us) service_pool.cc:224] Handling call
0709 12:13:56.913374 (+1898920us) spinlock_profiling.cc:243] Waited 643 ms on lock 0x7b5400091fd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:56.913576 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1479591296}
I20260709 12:13:56.918982 12679 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:56.922457 13336 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:13:56.914004 12494 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:56.874234 21937 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:56.850518 21845 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:13:56.940399 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:56.858516 21965 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election won for term 30
W20260709 12:13:56.941519 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 87938.8us
W20260709 12:13:56.914824 21730 consensus_meta.cc:287] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.726s	user 0.010s	sys 0.004s
W20260709 12:13:56.960057 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:56.962422 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47730.1us
W20260709 12:13:57.030658 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.913627 21956 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:57.224260 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.225308 21956 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:57.230007 21753 consensus_meta.cc:287] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.736s	user 0.016s	sys 0.013s
I20260709 12:13:57.231907 21753 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:57.236114 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:57.237015 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.237727 21753 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:57.244066 21753 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:57.247473 21947 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.800s	user 0.004s	sys 0.000s
I20260709 12:13:57.248201 21753 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:57.251202 12643 leader_election.cc:336] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.645s (SENT)
I20260709 12:13:57.251652 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:57.253026 21791 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.253284 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:57.253348 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.251327 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 4335) took 1125 ms. Trace:
I20260709 12:13:57.255412 13334 raft_consensus.cc:2410] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 28 because replica has last-logged OpId of term: 26 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 18.
W20260709 12:13:57.215025 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 301098us
I20260709 12:13:57.253597 21791 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.259016 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:13:57.260876 21791 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:57.260941 21824 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:13:57.261543 21791 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:13:56.963881 21434 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.262343 21791 consensus_queue.cc:1243] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 27.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:13:57.262512 21434 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.262928 21791 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:13:57.264160 12494 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:57.264631 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4340) took 1085 ms. Trace:
I20260709 12:13:57.269244 21946 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:13:57.270076 21946 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:57.270529 21791 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:13:57.271270 21946 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.13, Last appended by leader: 13, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.272153 21791 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 10, Time since last communication: 0.000s
I20260709 12:13:57.255045 12680 rpcz_store.cc:276] 0709 12:13:56.127465 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.127596 (+   131us) service_pool.cc:224] Handling call
0709 12:13:57.251144 (+1123548us) spinlock_profiling.cc:243] Waited 26.9 ms on lock 0x7b5400010c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:57.251260 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":61843200}
W20260709 12:13:57.215358 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.219209 21465 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.219973 21738 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:57.222203 21740 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:57.232424 21922 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.285503 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3531) took 2414 ms. Trace:
I20260709 12:13:57.285727 21922 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:56.964058 21965 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:57.286026 12524 rpcz_store.cc:276] 0709 12:13:54.871139 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.227113 (+1355974us) service_pool.cc:224] Handling call
0709 12:13:56.717366 (+490253us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:56.723022 (+  5656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:57.285253 (+562231us) spinlock_profiling.cc:243] Waited 490 ms on lock 0x7b540042a9d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:57.285430 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1126389504}
W20260709 12:13:57.287798 12605 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1039) took 3491 ms (client timeout 1669 ms). Trace:
I20260709 12:13:57.287521 21965 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.15, Last appended by leader: 14, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:57.288313 12605 rpcz_store.cc:269] 0709 12:13:53.796889 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.232769 (+1435880us) service_pool.cc:224] Handling call
0709 12:13:57.287570 (+2054801us) spinlock_profiling.cc:243] Waited 746 ms on lock 0x7b54001171d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:57.287716 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1715627776}
I20260709 12:13:57.289700 21465 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:57.293217 21434 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:57.293840 21434 raft_consensus.cc:3055] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:13:57.261657 21824 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:57.280262 12494 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:57.265226 12674 rpcz_store.cc:276] 0709 12:13:56.179391 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.192784 (+ 13393us) service_pool.cc:224] Handling call
0709 12:13:56.204371 (+ 11587us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:56.204875 (+   504us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:56.204913 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:56.213593 (+  8680us) log.cc:844] Serialized 26 byte log entry
0709 12:13:57.263884 (+1050291us) spinlock_profiling.cc:243] Waited 5.5 ms on lock 0x7b5400286f20. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148898a090 00007f14889976b8 00007f1488a2092b 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c
0709 12:13:57.263969 (+    85us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:13:57.264047 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:57.264151 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:57.264222 (+    71us) raft_consensus.cc:1685] finished
0709 12:13:57.264245 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:57.264548 (+   303us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4004,"raft.run_cpu_time_us":137,"raft.run_wall_time_us":7354,"spinlock_wait_cycles":12651392,"thread_start_us":117127,"threads_started":1,"wal-append.queue_time_us":134105}
I20260709 12:13:57.255237 21739 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.304289 12494 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:13:57.306455 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:57.311599 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:56.956754 21970 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:57.322592 21970 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:57.207508 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 255179us
I20260709 12:13:56.967682 21971 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.324868 12494 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:57.327594 21791 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:13:57.328472 21979 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:57.325436 21465 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 12:13:57.329833 21911 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.330446 21756 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:57.333281 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.316111 21922 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:57.338691 21714 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.339533 21587 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.340024 21815 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:57.361631 12496 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:57.355409 21934 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:57.362834 12494 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:57.363384 21753 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:57.364296 21740 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:13:57.406814 21740 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:57.407290 21740 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:13:57.331436 21751 consensus_meta.cc:287] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.832s	user 0.026s	sys 0.016s
W20260709 12:13:57.411367 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 56038.8us
I20260709 12:13:57.411624 21756 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:13:57.411926 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49199.3us
I20260709 12:13:57.412606 12496 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:57.412843 12494 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:57.412211 21756 raft_consensus.cc:697] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:57.414420 12494 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:57.376096 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.317708 21824 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:13:57.321532 21358 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:13:57.318467 21434 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:13:57.358156 21853 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.044s	user 0.012s	sys 0.008s
W20260709 12:13:57.357092 21951 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.603s	user 0.004s	sys 0.006s
I20260709 12:13:57.359840 21791 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:57.340873 21812 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.358486 21970 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:57.395422 21551 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:57.386801 21099 consensus_meta.cc:287] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.206s	user 0.017s	sys 0.021s
I20260709 12:13:57.401567 21815 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:57.395823 21714 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.396557 21911 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:57.387971 21743 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.036s	user 0.014s	sys 0.004s
W20260709 12:13:57.373392 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 157528us
W20260709 12:13:57.431583 13302 leader_election.cc:336] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.669s (SENT)
W20260709 12:13:57.418011 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 486661us
I20260709 12:13:57.334162 21739 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:57.442984 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:57.443902 21791 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:13:57.465538 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.442041 21845 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:57.410040 21587 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.418247 21751 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.417950 21738 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.448509 21993 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:13:57.482574 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 68202.9us
I20260709 12:13:57.436403 21934 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.491125 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4339) took 1312 ms. Trace:
I20260709 12:13:57.484520 21869 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:57.467001 21743 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:13:57.486883 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:57.491724 12677 rpcz_store.cc:276] 0709 12:13:56.179140 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.191224 (+ 12084us) service_pool.cc:224] Handling call
0709 12:13:56.191653 (+   429us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:56.192158 (+   505us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:56.192184 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:56.195344 (+  3160us) log.cc:844] Serialized 26 byte log entry
0709 12:13:56.354591 (+159247us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:56.354684 (+    93us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:56.354779 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:57.490806 (+1136027us) raft_consensus.cc:1685] finished
0709 12:13:57.490839 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:57.491036 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1272,"raft.run_cpu_time_us":119,"raft.run_wall_time_us":3957,"thread_start_us":158852,"threads_started":1,"wal-append.queue_time_us":488223}
I20260709 12:13:57.478819 21753 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:13:57.455463 21099 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.485656 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:57.440017 21853 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.455852 21551 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:57.526010 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:57.532790 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46023.8us
I20260709 12:13:57.464465 21981 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.053s
I20260709 12:13:57.484761 21358 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:13:57.536762 12494 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:57.532758 12674 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:13:57.374349 21971 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.485184 21982 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.486431 21465 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:13:57.484160 21984 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:13:57.486219 21993 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:57.523381 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:57.489152 21934 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:57.606608 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:57.485028 21975 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:57.545475 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.618455 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:57.556228 21745 consensus_meta.cc:287] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.212s	user 0.014s	sys 0.007s
I20260709 12:13:57.499497 21738 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:57.550568 21753 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:57.557230 21764 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.063s	user 0.013s	sys 0.016s
I20260709 12:13:57.558689 21992 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:57.558844 21971 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:57.559583 21966 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:13:57.559464 21845 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:57.559923 21789 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:13:57.559441 21935 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:13:57.499464 21756 consensus_queue.cc:237] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 21.12, Last appended by leader: 12, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.561378 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:57.562189 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:57.564122 21751 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:57.566203 21099 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:57.566807 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:57.521551 21853 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:57.568823 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:57.569468 21358 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
W20260709 12:13:57.571621 21928 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.568s	user 0.007s	sys 0.001s
I20260709 12:13:57.572826 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:57.575068 21824 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:57.576891 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:57.577229 21714 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:57.580325 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3550) took 5496 ms. Trace:
I20260709 12:13:57.763001 12603 rpcz_store.cc:276] 0709 12:13:52.083570 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.847131 (+3763561us) service_pool.cc:224] Handling call
0709 12:13:55.954224 (+107093us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:55.987696 (+ 33472us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:57.580066 (+1592370us) spinlock_profiling.cc:243] Waited 107 ms on lock 0x7b54001617d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:57.580255 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":245441920}
I20260709 12:13:57.488806 21962 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.065s
W20260709 12:13:57.631250 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46797.6us
I20260709 12:13:57.521817 21551 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:57.551661 21743 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.537979 21995 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:57.579442 21979 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:13:57.504611 21869 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:57.602548 21465 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:13:57.684224 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 220412us
I20260709 12:13:57.528031 21640 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.128s
I20260709 12:13:57.560519 21739 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:57.554539 21812 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:57.602691 21993 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:57.511173 21434 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.693153 12605 raft_consensus.cc:2376] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 33. Re-sending same reply.
I20260709 12:13:57.629640 21934 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:57.646350 21745 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:13:57.663277 21764 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.618063 21982 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.637135 21975 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:57.767179 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 194832us
I20260709 12:13:57.663244 21992 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.747180 21935 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.785939 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:57.840978 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 55177.3us
I20260709 12:13:57.751665 21996 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:13:57.847659 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:57.800412 21551 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.779788 21789 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:13:57.778637 21714 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.787101 21979 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:57.845477 21946 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.001s	sys 0.006s
W20260709 12:13:57.876212 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 433302us
W20260709 12:13:57.846658 21965 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.555s	user 0.018s	sys 0.021s
I20260709 12:13:57.834872 12524 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 21 index: 14. (index mismatch)
I20260709 12:13:57.836988 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:57.894765 12530 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:57.844419 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:57.902334 12605 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 29.
I20260709 12:13:57.766346 21751 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:57.849516 12494 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:57.762843 21358 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:57.856523 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:57.857600 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:57.822829 21815 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:57.858884 12522 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 26. Current term is 27. Ops: []
I20260709 12:13:57.872784 21979 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:13:57.892099 21965 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.600s	user 0.019s	sys 0.022s
W20260709 12:13:57.832619 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.879348 21587 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:57.891430 21853 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:57.859249 21971 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:13:57.924242 21358 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:13:57.924316 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 74887us
I20260709 12:13:57.751804 21753 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.785120 21995 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:57.880132 21946 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.539s	user 0.002s	sys 0.006s
I20260709 12:13:57.788215 21743 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:57.878850 21739 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:13:57.940610 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.847056 21935 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.890559 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4322) took 3154 ms. Trace:
I20260709 12:13:57.891103 21962 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:57.825531 21934 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:58.060544 21740 consensus_meta.cc:287] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.653s	user 0.014s	sys 0.000s
I20260709 12:13:58.001292 12494 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:13:57.877400 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:57.880604 21465 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.871127 21714 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:57.932133 21996 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:58.014285 21815 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:57.937014 21099 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:13:58.165808 12525 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 26. Current term is 27. Ops: []
W20260709 12:13:58.016458 21973 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.251s	user 0.001s	sys 0.000s
W20260709 12:13:58.017369 21991 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.192s	user 0.000s	sys 0.000s
I20260709 12:13:58.021478 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:58.022212 21587 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:13:58.022992 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:57.956068 21971 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:58.185470 21971 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:58.041272 21824 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:58.041694 21911 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:58.043202 21755 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:58.187328 22002 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.187935 22002 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:58.000285 21853 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:58.191780 21934 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:58.192466 21934 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:58.192493 21853 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:58.047396 12599 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 30.
I20260709 12:13:58.193063 21934 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:58.048386 21992 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:58.050421 21869 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:58.053798 21975 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:58.055359 21764 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:57.874348 21551 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:58.056531 21982 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:58.077852 12524 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:58.070650 12522 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:13:58.066272 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:58.204865 12674 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:57.873698 21812 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.066s
W20260709 12:13:58.082512 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.095958 12530 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:13:57.930590 21751 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:58.217247 21751 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:58.140206 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 262901us
I20260709 12:13:58.217566 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:58.218175 12643 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:58.071173 12678 rpcz_store.cc:276] 0709 12:13:54.735945 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:55.238153 (+502208us) service_pool.cc:224] Handling call
0709 12:13:55.238680 (+   527us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:55.239396 (+   716us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:55.239439 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:55.264471 (+ 25032us) log.cc:844] Serialized 26 byte log entry
0709 12:13:55.737687 (+473216us) raft_consensus.cc:1645] Marking committed up to 11
0709 12:13:55.737731 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:55.737805 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:57.570784 (+1832979us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:57.875160 (+304376us) raft_consensus.cc:1685] finished
0709 12:13:57.875193 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:57.890452 (+ 15259us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14151,"raft.queue_time_us":2203,"raft.run_cpu_time_us":193,"raft.run_wall_time_us":7110,"spinlock_wait_cycles":6091520,"thread_start_us":441281,"threads_started":1,"wal-append.queue_time_us":696975}
I20260709 12:13:58.219228 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:58.219789 12643 leader_election.cc:400] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23.
I20260709 12:13:58.220422 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:58.221092 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:58.221450 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:13:58.224699 21987 log.cc:927] Time spent T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.539s	user 0.001s	sys 0.012s
I20260709 12:13:58.225102 21935 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:58.226845 21970 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.771s	user 0.013s	sys 0.000s
I20260709 12:13:58.227300 21970 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:58.239449 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:58.222081 12678 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:58.244529 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:58.245501 12678 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
W20260709 12:13:58.145433 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58066.6us
W20260709 12:13:58.145776 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 144569us
W20260709 12:13:58.157846 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:58.256059 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.166579 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:58.174784 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:58.175654 21714 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:58.176393 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:58.169570 21815 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:58.169843 21713 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:58.065261 22009 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.154991 21640 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.082s
I20260709 12:13:58.142544 21738 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:58.300329 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:58.298442 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:58.304344 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:58.305262 13331 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:13:58.307204 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:58.308218 13337 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:13:58.247890 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:58.312819 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:58.314591 12598 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.741s	user 0.123s	sys 0.146s
I20260709 12:13:58.213478 21994 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.213960 21992 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:13:58.218403 12524 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 24: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:58.140457 12600 raft_consensus.cc:2376] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21. Re-sending same reply.
I20260709 12:13:58.172190 21465 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:58.290606 21714 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.199536 12423 catalog_manager.cc:5697] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 21 to 22, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 22 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:13:58.328313 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:58.266403 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:58.268512 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:58.329875 12673 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 31
I20260709 12:13:58.278541 12671 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 24: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:58.331379 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:58.332243 12497 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.783s	user 0.003s	sys 0.010s
I20260709 12:13:58.332491 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:58.333009 12497 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.784s	user 0.003s	sys 0.010s
I20260709 12:13:58.279882 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:58.334722 12604 raft_consensus.cc:2410] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 34 because replica has last-logged OpId of term: 32 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 20.
I20260709 12:13:58.281749 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:58.335232 12528 raft_consensus.cc:2376] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23. Re-sending same reply.
I20260709 12:13:58.282596 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:58.333829 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:58.289691 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:58.292074 12496 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:58.203948 21980 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.297183 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:58.167840 21996 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:58.340224 13338 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:58.154167 21740 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:58.144104 21962 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:58.296411 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:58.314716 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:58.349644 12672 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 33.
I20260709 12:13:58.317629 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:58.395311 12678 raft_consensus.cc:2410] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 28 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 15.
I20260709 12:13:58.396478 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:58.397691 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:58.257557 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:58.325946 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:58.194543 21587 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:58.398559 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 141161us
I20260709 12:13:58.272015 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:58.210372 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:58.284539 13330 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:58.400363 21997 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:58.400835 21587 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:58.400995 12674 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:58.400835 21997 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:58.402283 21751 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.338162 12671 raft_consensus.cc:2410] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 24 because replica has last-logged OpId of term: 23 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 12:13:58.402011 21997 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:58.337967 13334 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
W20260709 12:13:58.338738 12597 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.763s	user 0.006s	sys 0.009s
W20260709 12:13:58.407320 12597 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.831s	user 0.006s	sys 0.009s
I20260709 12:13:58.341696 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3609) took 1108 ms. Trace:
I20260709 12:13:58.409063 12604 rpcz_store.cc:276] 0709 12:13:57.233484 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:57.234714 (+  1230us) service_pool.cc:224] Handling call
0709 12:13:58.279595 (+1044881us) spinlock_profiling.cc:243] Waited 82 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:13:58.341630 (+ 62035us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":188560640}
I20260709 12:13:58.342103 12529 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:58.298166 22002 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:58.317112 12598 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.744s	user 0.124s	sys 0.146s
I20260709 12:13:58.362761 12677 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 28: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:58.411240 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:58.412267 12529 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 30.
I20260709 12:13:58.412989 12677 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 29 index: 15. Preceding OpId from leader: term: 28 index: 15. (term mismatch)
I20260709 12:13:58.371001 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:58.371861 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:58.346376 12570 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.714s	user 0.011s	sys 0.005s
I20260709 12:13:58.378643 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:58.413594 12677 pending_rounds.cc:85] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 14
I20260709 12:13:58.415620 12677 pending_rounds.cc:107] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 29.15
I20260709 12:13:58.415935 12677 raft_consensus.cc:2889] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:13:58.380280 12673 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
W20260709 12:13:58.387629 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.398598 12676 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:58.399838 12530 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:13:58.402799 21751 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:58.423408 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:58.424125 12530 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 30.
I20260709 12:13:58.414420 12528 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 27.
I20260709 12:13:58.425379 12530 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
W20260709 12:13:58.347757 21939 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.003s	sys 0.007s
I20260709 12:13:58.427199 12528 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 22. Current term is 23. Ops: []
I20260709 12:13:58.428676 21986 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:13:58.415095 12570 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.783s	user 0.011s	sys 0.006s
I20260709 12:13:58.430452 22012 consensus_queue.cc:1059] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 22.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:13:58.430423 21730 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:13:58.432775 22012 raft_consensus.cc:3055] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:58.433171 22012 raft_consensus.cc:740] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:58.433387 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:58.413400 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:58.435067 12605 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 27: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:58.430527 21981 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.436493 21981 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:58.434073 22012 consensus_queue.cc:260] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:58.437441 21964 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.437795 22012 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:58.438005 21964 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:58.435137 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:58.440140 12606 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:13:58.434518 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:58.441691 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:58.442471 12603 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 24: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:58.434739 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:58.443840 12604 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:58.443748 12599 raft_consensus.cc:2410] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 23 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 12.
I20260709 12:13:58.446506 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:58.447103 12602 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:58.465095 12604 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
W20260709 12:13:58.347375 21983 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.713s	user 0.000s	sys 0.015s
W20260709 12:13:58.478641 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 63822us
I20260709 12:13:58.348835 22017 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:13:58.386863 21993 consensus_meta.cc:287] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.560s	user 0.010s	sys 0.000s
I20260709 12:13:58.413666 21997 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:58.495170 21868 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:58.384692 21998 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.513003 21815 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:58.486933 21791 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.003s	user 0.013s	sys 0.000s
I20260709 12:13:58.489313 22002 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.024s
I20260709 12:13:58.514684 21743 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:58.478474 22020 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:58.375439 22019 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:13:58.299491 22009 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:58.522286 13302 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.722s	user 0.007s	sys 0.055s
W20260709 12:13:58.585732 13302 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.786s	user 0.008s	sys 0.056s
W20260709 12:13:58.586658 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 798566us
W20260709 12:13:58.530448 12679 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.892s	user 0.017s	sys 0.047s
W20260709 12:13:58.587924 12679 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.949s	user 0.017s	sys 0.047s
I20260709 12:13:58.552688 21970 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:58.498138 21969 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 1.512s	user 0.015s	sys 0.005s
I20260709 12:13:58.560196 22016 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:58.367678 21980 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:58.329579 21994 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:58.495970 21979 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.556s	user 0.008s	sys 0.001s
I20260709 12:13:58.388176 21935 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.138s
W20260709 12:13:58.547833 21983 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.969s	user 0.000s	sys 0.016s
I20260709 12:13:58.349344 21714 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:13:58.603690 21994 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:58.604709 21714 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:13:58.551220 22017 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:58.567924 21587 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:13:58.605775 22017 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:58.601887 21551 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:58.610515 21979 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:58.612824 21974 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.129s	user 0.008s	sys 0.005s
I20260709 12:13:58.613363 21979 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:58.580443 21939 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:58.614194 21939 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:58.618783 22017 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:58.625751 22017 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:58.626962 21751 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:58.561627 21713 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:13:58.586314 21995 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:58.593146 21934 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:13:58.597754 22018 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.259s	user 0.002s	sys 0.000s
I20260709 12:13:58.577816 21791 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:56.912519 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:13:58.707147 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 414866us
I20260709 12:13:58.759820 12496 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:58.765303 21099 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:13:58.766698 21739 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:58.768131 21739 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:58.712105 12526 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:58.321448 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:58.714934 12423 catalog_manager.cc:5697] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 16 to 23, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 23 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:58.374747 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:58.856568 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:58.857800 12522 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 23: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:58.723717 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:58.859561 12525 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 30: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:58.860898 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:58.726087 12529 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 29. Current term is 30. Ops: [29.15-29.15]
I20260709 12:13:58.864609 21739 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:58.864908 12680 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.533s	user 0.033s	sys 0.038s
W20260709 12:13:58.865507 12680 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.033s	sys 0.038s
I20260709 12:13:58.866511 13305 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:58.868245 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:58.474735 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:58.726177 12530 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 28. Current term is 29. Ops: [28.20-28.20]
I20260709 12:13:58.539665 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:58.589445 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3676) took 1051 ms. Trace:
I20260709 12:13:58.595113 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:58.900911 12599 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:13:58.903261 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 308096us
I20260709 12:13:58.904536 13305 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:58.574837 21868 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:58.616986 21939 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:58.660029 21962 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:58.664047 21964 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:58.935276 21939 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:13:58.935519 21868 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:58.935850 21939 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:58.936259 21984 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:58.937089 21984 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:58.652727 21971 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:58.945601 12679 rpcz_store.cc:276] 0709 12:13:57.537783 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:57.538404 (+   621us) service_pool.cc:224] Handling call
0709 12:13:57.557164 (+ 18760us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:57.557408 (+   244us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:13:57.638196 (+ 80788us) log.cc:844] Serialized 14 byte log entry
0709 12:13:58.588843 (+950647us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:58.588948 (+   105us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:58.589031 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:58.589139 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:58.589354 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26172,"prepare.queue_time_us":81090,"prepare.run_cpu_time_us":89,"prepare.run_wall_time_us":17950,"raft.queue_time_us":16601,"raft.run_cpu_time_us":93,"raft.run_wall_time_us":94,"thread_start_us":950070,"threads_started":1,"wal-append.queue_time_us":1009326}
I20260709 12:13:58.948400 21738 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:13:58.949429 21971 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:58.662952 21981 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:58.571581 21993 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:58.662801 21996 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:58.950816 22015 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:13:58.951629 22015 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:58.952076 22015 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:13:58.577297 21998 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:58.661731 21997 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:58.651651 22009 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:58.656908 22011 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:13:58.627995 22016 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:58.642158 22020 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:13:58.653018 22025 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.089s	user 0.001s	sys 0.000s
W20260709 12:13:58.751111 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:58.753394 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4349) took 1981 ms. Trace:
I20260709 12:13:58.756834 21740 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:58.735256 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1056) took 1167 ms. Trace:
I20260709 12:13:58.777333 21099 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:58.850239 12527 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:13:58.869958 21551 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:13:58.876302 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:58.880386 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:58.887866 12604 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:13:58.894260 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:58.900663 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:58.937614 12643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.005s	sys 0.017s
I20260709 12:13:58.918520 12524 raft_consensus.cc:2410] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 28 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
W20260709 12:13:58.918341 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 333995us
I20260709 12:13:58.918902 21791 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:58.936363 21939 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:58.609625 21980 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:58.945693 21935 consensus_queue.cc:1243] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 25.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:13:58.959254 12531 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 28. Current term is 29. Ops: []
W20260709 12:13:58.959944 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 183120us
W20260709 12:13:59.074864 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 170537us
I20260709 12:13:58.953866 21993 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:59.007928 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:59.096679 21434 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:13:59.097918 12602 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.650s	user 0.012s	sys 0.008s
I20260709 12:13:59.098430 21713 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:13:59.099690 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:59.099511 21713 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:58.979794 22016 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:59.103129 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:59.104171 13331 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.691s	user 0.016s	sys 0.008s
I20260709 12:13:59.104750 13331 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:13:59.105250 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3618) took 4181 ms. Trace:
I20260709 12:13:59.105789 12531 rpcz_store.cc:276] 0709 12:13:54.923600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:57.268249 (+2344649us) service_pool.cc:224] Handling call
0709 12:13:58.038931 (+770682us) spinlock_profiling.cc:243] Waited 47.5 ms on lock 0x7b5800002a98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:13:58.039343 (+   412us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:59.103949 (+1064606us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:59.104889 (+   940us) spinlock_profiling.cc:243] Waited 919 ms on lock 0x7b54001072d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:59.105141 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2223560832}
I20260709 12:13:59.110700 22016 raft_consensus.cc:2764] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader pre-election decision vote started in defunct term 29: won
W20260709 12:13:59.112762 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.114802 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:59.115325 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:59.117281 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:58.957484 21998 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:59.120261 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:59.098487 12602 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:13:59.121213 12604 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:13:59.121814 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:59.122304 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 3550) took 1047 ms. Trace:
I20260709 12:13:59.122886 12602 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 23: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:13:59.123399 22039 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.100s	user 0.000s	sys 0.000s
I20260709 12:13:59.124092 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:59.125149 12602 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:59.108349 12494 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:13:59.128422 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.128768 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:59.130013 12494 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:59.142277 22019 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:59.144409 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:59.144570 22019 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:59.118878 12531 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
W20260709 12:13:59.022486 12600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1054) took 1461 ms (client timeout 1842 ms). Trace:
I20260709 12:13:59.062201 12675 rpcz_store.cc:276] 0709 12:13:56.772050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.772237 (+   187us) service_pool.cc:224] Handling call
0709 12:13:56.772565 (+   328us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:56.773339 (+   774us) raft_consensus.cc:1534] Early marking committed up to index 12
0709 12:13:56.820557 (+ 47218us) log.cc:844] Serialized 14 byte log entry
0709 12:13:57.326200 (+505643us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:57.332020 (+  5820us) log.cc:844] Serialized 26 byte log entry
0709 12:13:57.332214 (+   194us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:57.332252 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:57.332326 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:58.612118 (+1279792us) spinlock_profiling.cc:243] Waited 5.27 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:58.752034 (+139916us) raft_consensus.cc:1685] finished
0709 12:13:58.752304 (+   270us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:58.752866 (+   562us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":217116,"prepare.queue_time_us":77193,"prepare.run_cpu_time_us":54,"prepare.run_wall_time_us":55,"raft.queue_time_us":230038,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":14049,"spinlock_wait_cycles":12118912,"thread_start_us":287845,"threads_started":1,"wal-append.queue_time_us":630036}
W20260709 12:13:59.150207 12643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.928s	user 0.010s	sys 0.022s
I20260709 12:13:58.943177 21984 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:59.069950 12598 rpcz_store.cc:276] 0709 12:13:57.567764 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:57.570720 (+  2956us) service_pool.cc:224] Handling call
0709 12:13:57.571404 (+   684us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:57.572255 (+   851us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:57.572304 (+    49us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:57.573107 (+   803us) log.cc:844] Serialized 26 byte log entry
0709 12:13:58.213102 (+639995us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b480024c258. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:13:58.411034 (+197932us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:13:58.411101 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:58.411208 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:58.496248 (+ 85040us) raft_consensus.cc:1685] finished
0709 12:13:58.496279 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:58.734830 (+238551us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":112133,"raft.run_cpu_time_us":3369,"raft.run_wall_time_us":525195,"spinlock_wait_cycles":46946048,"thread_start_us":837337,"threads_started":1,"wal-append.queue_time_us":753229}
I20260709 12:13:59.117497 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:58.958146 21358 consensus_meta.cc:287] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.947s	user 0.011s	sys 0.004s
I20260709 12:13:59.129853 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:59.157227 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.068780 21551 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:13:59.149390 12600 rpcz_store.cc:269] 0709 12:13:57.561307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:57.561874 (+   567us) service_pool.cc:224] Handling call
0709 12:13:59.022003 (+1460129us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b54002e54d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:59.022233 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1025842048}
I20260709 12:13:59.111457 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1057) took 1543 ms. Trace:
I20260709 12:13:59.159348 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:59.122799 12524 rpcz_store.cc:276] 0709 12:13:58.074755 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:58.321116 (+246361us) service_pool.cc:224] Handling call
0709 12:13:59.122169 (+801053us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:13:59.134696 22051 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.052s	user 0.000s	sys 0.000s
I20260709 12:13:59.076248 21099 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:59.164645 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3681) took 1311 ms. Trace:
W20260709 12:13:59.165688 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 261353us
I20260709 12:13:59.008450 21978 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.008093 12423 catalog_manager.cc:5697] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 12 to 21, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 21 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:59.182190 12604 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:13:59.184894 12599 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:13:59.128782 21979 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:13:59.169061 21551 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:13:59.190616 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:59.184268 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:59.213419 22020 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:59.215332 21815 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.215999 22049 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:59.216238 21815 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:59.217216 22049 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:59.222327 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:59.249626 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:59.219334 21815 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:59.287081 12604 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
W20260709 12:13:59.248404 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49339.8us
I20260709 12:13:59.176779 12680 rpcz_store.cc:276] 0709 12:13:57.844478 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:57.853165 (+  8687us) service_pool.cc:224] Handling call
0709 12:13:57.854755 (+  1590us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:58.208272 (+353517us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:58.208305 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:58.263652 (+ 55347us) log.cc:844] Serialized 26 byte log entry
0709 12:13:58.866334 (+602682us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:58.866436 (+   102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:58.866560 (+   124us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:59.148840 (+282280us) raft_consensus.cc:1685] finished
0709 12:13:59.148862 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:59.155510 (+  6648us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11583,"raft.queue_time_us":100760,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":10389,"thread_start_us":903643,"threads_started":2,"wal-append.queue_time_us":829159}
I20260709 12:13:59.239049 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:59.229866 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:59.165764 12597 rpcz_store.cc:276] 0709 12:13:57.568110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:57.573882 (+  5772us) service_pool.cc:224] Handling call
0709 12:13:57.574590 (+   708us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:57.575005 (+   415us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:57.575031 (+    26us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:57.575577 (+   546us) log.cc:844] Serialized 26 byte log entry
0709 12:13:58.407844 (+832267us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:13:59.110177 (+702333us) log.cc:844] Serialized 14 byte log entry
0709 12:13:59.110793 (+   616us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:59.110959 (+   166us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:59.111008 (+    49us) raft_consensus.cc:1685] finished
0709 12:13:59.111028 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:59.111229 (+   201us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":22706,"prepare.queue_time_us":706485,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":94,"raft.queue_time_us":114357,"raft.run_cpu_time_us":472,"raft.run_wall_time_us":22249,"spinlock_wait_cycles":12544,"thread_start_us":1318495,"threads_started":2,"wal-append.queue_time_us":747335}
I20260709 12:13:59.223425 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:59.218739 21970 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:13:59.236594 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:59.265429 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.206465 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:59.251772 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:59.229218 12522 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:13:59.227742 21978 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:59.296512 21977 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:13:59.297235 21977 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:59.297688 21977 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:13:59.219465 21722 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.298014 21978 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:59.298578 21722 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:59.300871 21722 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:59.301412 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 195900us
I20260709 12:13:59.181977 21358 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:59.229794 21935 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:13:59.272292 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 264862us
W20260709 12:13:59.259125 12527 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 3532) took 3608 ms (client timeout 1629 ms). Trace:
I20260709 12:13:59.253947 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:59.219799 21753 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:13:59.252409 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:13:59.305734 12527 rpcz_store.cc:269] 0709 12:13:55.650850 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:56.939945 (+1289095us) service_pool.cc:224] Handling call
0709 12:13:59.258860 (+2318915us) spinlock_profiling.cc:243] Waited 1.77 s on lock 0x7b540038e0d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:59.259027 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4071734144}
W20260709 12:13:59.287976 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 80061us
I20260709 12:13:59.285353 21966 raft_consensus.cc:3055] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Stepping down as leader of term 26
W20260709 12:13:59.301430 22039 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:13:59.330072 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:59.334234 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:59.341321 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:59.343667 12598 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 24: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:13:59.339375 13338 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:13:59.348022 12423 catalog_manager.cc:5697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 28 to 30. New cstate: current_term: 30 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:59.328544 21979 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.350596 12522 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 28: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:59.351919 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:59.352563 12604 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:59.317117 21966 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:13:59.344985 12572 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:59.347298 12600 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:13:59.357151 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:59.363279 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:59.366312 12529 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 23: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:13:59.369287 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46388.6us
I20260709 12:13:59.373817 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:59.376969 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:59.377554 12525 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:13:59.378695 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:59.379415 21972 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.378999 21966 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:59.380383 21966 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:13:59.380053 21972 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:59.385422 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:59.389432 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:59.384681 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:13:59.391907 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50714.4us
W20260709 12:13:59.382424 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 1.16126e+06us
W20260709 12:13:59.394593 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 250407us
I20260709 12:13:59.396943 12525 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 24: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:59.397709 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:59.398535 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:59.367630 21713 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:59.402608 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:59.402447 21640 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:13:59.413761 12605 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:59.403251 21815 consensus_queue.cc:1059] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.014s
I20260709 12:13:59.350611 21979 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:59.414821 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:59.426317 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:13:59.420287 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:59.427141 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4354) took 1197 ms. Trace:
I20260709 12:13:59.424933 21099 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:59.419612 12571 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:59.442715 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49840.7us
I20260709 12:13:59.423413 21358 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:59.448208 12603 raft_consensus.cc:1217] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Deduplicated request from leader. Original: 24.14->[26.15-28.16]   Dedup: 26.15->[28.16-28.16]
I20260709 12:13:59.449084 12603 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: [26.15-28.16]
I20260709 12:13:59.450800 12603 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:59.452185 21935 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:13:59.452745 21935 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:59.453158 21935 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:59.417681 21815 raft_consensus.cc:3055] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:13:59.454726 21815 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:59.455578 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:13:59.456302 21815 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:59.457897 12531 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:13:59.418100 21722 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:13:59.458784 21722 raft_consensus.cc:697] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:13:59.448305 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 119644us
I20260709 12:13:59.459558 12531 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:59.463995 12525 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 30.
I20260709 12:13:59.463553 12529 raft_consensus.cc:2410] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 26 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:13:59.461582 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:59.461063 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:59.492830 12526 raft_consensus.cc:2393] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 28: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:59.460109 21722 consensus_queue.cc:237] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.13, Last appended by leader: 13, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:59.463074 21551 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:59.464892 21939 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:59.508383 21551 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:59.509269 21939 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:59.501648 21972 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:59.514376 12527 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 27.
I20260709 12:13:59.510046 21434 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:59.516467 21434 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:59.517633 12527 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: [22.15-22.15]
I20260709 12:13:59.500516 21815 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:59.498734 12604 raft_consensus.cc:2410] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 29 because replica has last-logged OpId of term: 28 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 19.
I20260709 12:13:59.501212 12525 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:13:59.523262 12569 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:13:59.523999 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 78489.3us
I20260709 12:13:59.524394 12525 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:59.525857 12525 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:13:59.525669 21984 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:59.528342 21977 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:13:59.529636 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 88124.9us
I20260709 12:13:59.530489 21978 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:13:59.531138 21738 raft_consensus.cc:3055] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:13:59.541000 13337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 1106) took 1230 ms. Trace:
I20260709 12:13:59.546885 12496 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:59.542294 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4150) took 1106 ms. Trace:
W20260709 12:13:59.443851 13302 leader_election.cc:336] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.842s (SENT)
I20260709 12:13:59.548790 12605 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
W20260709 12:13:59.551368 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.552480 12679 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 26: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:13:59.502447 12572 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:59.496187 12599 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:13:59.556828 12494 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:13:59.472188 12423 catalog_manager.cc:5697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 23 to 25, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 25 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:59.558580 12602 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:13:59.559953 12643 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:59.564738 12530 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 27. Current term is 28. Ops: []
I20260709 12:13:59.566143 22049 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:13:59.566579 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.566526 21434 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:59.438822 12672 rpcz_store.cc:276] 0709 12:13:58.174440 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:58.397563 (+223123us) service_pool.cc:224] Handling call
0709 12:13:58.415747 (+ 18184us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:13:58.415964 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:13:58.890682 (+474718us) log.cc:844] Serialized 14 byte log entry
0709 12:13:59.211893 (+321211us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:13:59.286649 (+ 74756us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:59.311294 (+ 24645us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:59.319875 (+  8581us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:59.361462 (+ 41587us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b54003d40d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:59.371734 (+ 10272us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":144254,"prepare.queue_time_us":221916,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":28253,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":222,"spinlock_wait_cycles":41166080,"thread_start_us":683431,"threads_started":2,"wal-append.queue_time_us":394316}
I20260709 12:13:59.567747 12524 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 27. Current term is 28. Ops: []
I20260709 12:13:59.571158 12600 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 26 index: 13. Preceding OpId from leader: term: 28 index: 14. (index mismatch)
I20260709 12:13:59.572660 21099 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:59.574891 21984 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:59.559954 21979 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:13:59.577771 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 133421us
I20260709 12:13:59.580960 13302 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 21 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:59.584966 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:59.589712 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4155) took 1135 ms. Trace:
I20260709 12:13:59.585937 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:59.592247 12605 rpcz_store.cc:276] 0709 12:13:58.446419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.207034 (+760615us) service_pool.cc:224] Handling call
0709 12:13:59.581495 (+374461us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:13:59.554538 21978 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:13:59.597066 12601 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:13:59.578755 21738 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:13:59.584257 22015 consensus_meta.cc:287] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.632s	user 0.004s	sys 0.012s
I20260709 12:13:59.594491 12423 catalog_manager.cc:5697] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 24 to 26. New cstate: current_term: 26 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:59.578886 13337 rpcz_store.cc:276] 0709 12:13:58.310616 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:58.311391 (+   775us) service_pool.cc:224] Handling call
0709 12:13:58.341981 (+ 30590us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:58.342504 (+   523us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:13:58.962721 (+620217us) spinlock_profiling.cc:243] Waited 6.65 ms on lock 0x7b18000ea160. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf 00007f1488a3a038
0709 12:13:59.194248 (+231527us) log.cc:844] Serialized 14 byte log entry
0709 12:13:59.394019 (+199771us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:59.402768 (+  8749us) log.cc:844] Serialized 26 byte log entry
0709 12:13:59.402959 (+   191us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:13:59.403196 (+   237us) log.cc:844] Serialized 14 byte log entry
0709 12:13:59.403474 (+   278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:59.415292 (+ 11818us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:59.540442 (+125150us) raft_consensus.cc:1685] finished
0709 12:13:59.540481 (+    39us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:59.540727 (+   246us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14,"prepare.queue_time_us":1065362,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":95,"raft.queue_time_us":2648,"raft.run_cpu_time_us":522,"raft.run_wall_time_us":44132,"spinlock_wait_cycles":15294848,"thread_start_us":342878,"threads_started":2,"wal-append.queue_time_us":327250}
I20260709 12:13:59.580065 12606 rpcz_store.cc:276] 0709 12:13:58.440933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:58.442306 (+  1373us) service_pool.cc:224] Handling call
0709 12:13:58.442978 (+   672us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:58.971063 (+528085us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:58.971170 (+   107us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:59.102551 (+131381us) log.cc:844] Serialized 26 byte log entry
0709 12:13:59.344542 (+241991us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:13:59.379308 (+ 34766us) log.cc:844] Serialized 14 byte log entry
0709 12:13:59.379561 (+   253us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:59.404132 (+ 24571us) spinlock_profiling.cc:243] Waited 514 ms on lock 0x7b540022fbd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:59.404185 (+    53us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:59.541823 (+137638us) raft_consensus.cc:1685] finished
0709 12:13:59.541931 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:59.542090 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16173,"prepare.queue_time_us":41991,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":6108,"raft.queue_time_us":20243,"raft.run_cpu_time_us":328,"raft.run_wall_time_us":2167,"spinlock_wait_cycles":1181990272,"thread_start_us":241427,"threads_started":1,"wal-append.queue_time_us":436625}
I20260709 12:13:59.598196 21978 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:13:59.630865 22043 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:13:59.631500 12680 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 20.
I20260709 12:13:59.603391 21358 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:13:59.601397 21979 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:13:59.601260 21640 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader election lost for term 21. Reason: could not achieve majority
W20260709 12:13:59.654350 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 211317us
I20260709 12:13:59.649694 21971 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:13:59.658986 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 99176.5us
I20260709 12:13:59.647065 22043 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:13:59.682438 22043 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:13:59.688321 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:59.613344 22015 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:59.659912 21099 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:13:59.690683 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:59.693696 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 146887us
W20260709 12:13:59.661661 21465 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.104s	user 0.017s	sys 0.010s
I20260709 12:13:59.658929 21971 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:13:59.698520 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 52998.4us
W20260709 12:13:59.700646 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 198420us
I20260709 12:13:59.656243 12423 catalog_manager.cc:5697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 22 to 26. New cstate: current_term: 26 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:59.705863 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:59.702069 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:59.697613 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 99462.1us
I20260709 12:13:59.708101 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 20 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:59.686273 21979 raft_consensus.cc:697] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:59.711995 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:59.714202 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:59.716922 12523 raft_consensus.cc:2410] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 29 because replica has last-logged OpId of term: 28 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 16.
I20260709 12:13:59.723045 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:13:59.723800 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:59.724596 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:13:59.725178 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 281397us
W20260709 12:13:59.728410 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.729972 21997 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:13:59.730969 12523 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 3537) took 1892 ms (client timeout 1924 ms). Trace:
W20260709 12:13:59.731426 12523 rpcz_store.cc:269] 0709 12:13:57.838437 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:57.838594 (+   157us) service_pool.cc:224] Handling call
0709 12:13:59.730646 (+1892052us) spinlock_profiling.cc:243] Waited 1.79 s on lock 0x7b5400052b54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:13:59.730887 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4106448256}
I20260709 12:13:59.732270 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:59.733649 12496 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:13:59.740374 12675 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
I20260709 12:13:59.743638 21465 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:13:59.744096 21465 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:13:59.744488 21465 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:59.727008 21740 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:13:59.747478 21740 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:59.748554 12599 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:13:59.749027 21358 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:13:59.750917 22061 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.090s	user 0.001s	sys 0.000s
I20260709 12:13:59.747745 12496 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:13:59.751969 12572 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:59.738736 12423 catalog_manager.cc:5697] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:59.755860 12679 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:13:59.751083 21935 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:59.742094 21099 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:59.761529 21935 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:59.765600 12423 catalog_manager.cc:5697] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 19 to 24. New cstate: current_term: 24 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:13:59.763293 12494 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:59.716352 21738 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:59.771845 12423 catalog_manager.cc:5697] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 19 to 23. New cstate: current_term: 23 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:13:59.776911 12427 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59006 (request call id 179) took 76048 ms (client timeout 15000 ms). Trace:
I20260709 12:13:59.771006 21977 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:13:59.774231 21935 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:13:59.775784 21358 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:13:59.774536 21984 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:13:59.792068 12600 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 33. Current term is 34. Ops: [33.22-33.22]
I20260709 12:13:59.793309 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:13:59.769263 21640 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 26.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.028s
I20260709 12:13:59.809113 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:13:59.816812 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:13:59.818150 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 80802.4us
I20260709 12:13:59.821935 12679 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:13:59.821475 21099 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:59.816514 13305 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:13:59.849524 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:13:59.835765 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 115906us
I20260709 12:13:59.854643 12680 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: []
I20260709 12:13:59.856757 21551 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:13:59.857230 21551 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:13:59.857599 21551 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:59.860255 22043 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:59.822471 22019 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:13:59.862360 22019 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:13:59.864569 22043 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:13:59.865496 22043 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:13:59.854676 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:59.811151 21984 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:13:59.865927 22043 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:13:59.812050 22015 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:59.877168 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:13:59.878389 12600 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 23 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
W20260709 12:13:59.889577 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:13:59.889797 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4164) took 1149 ms. Trace:
I20260709 12:13:59.868474 21738 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:13:59.895236 12675 raft_consensus.cc:2410] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 28 because replica has last-logged OpId of term: 27 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 17.
I20260709 12:13:59.898792 12672 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:13:59.908836 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:13:59.902479 12423 catalog_manager.cc:5697] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 18 to 20, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 20 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:13:59.890311 12603 rpcz_store.cc:276] 0709 12:13:58.728213 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.451771 (+723558us) service_pool.cc:224] Handling call
0709 12:13:59.452193 (+   422us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:59.452744 (+   551us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:59.452777 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:59.453362 (+   585us) log.cc:844] Serialized 26 byte log entry
0709 12:13:59.696905 (+243543us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:13:59.696952 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:59.697027 (+    75us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:59.877726 (+180699us) raft_consensus.cc:1685] finished
0709 12:13:59.877750 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:59.877969 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":5723,"raft.run_cpu_time_us":116,"raft.run_wall_time_us":116,"thread_start_us":220213,"threads_started":1,"wal-append.queue_time_us":396272}
W20260709 12:13:59.778683 12427 rpcz_store.cc:269] 0709 12:12:43.636943 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:43.637695 (+   752us) service_pool.cc:224] Handling call
0709 12:12:43.853056 (+215361us) spinlock_profiling.cc:243] Waited 163 ms on lock 0x7b54000104d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148898a090 00007f1488a28f3a 00007f148e7dee9c 00007f148e8dd04e 00007f148cd9e2a4 00007f148cd9e232 00007f148cd9e161 00007f148cd9e0dc 00007f148cd9d392 00007f1486d5b48c 00007f1486d5a8c6
0709 12:12:44.235271 (+382215us) spinlock_profiling.cc:243] Waited 2.71 ms on lock 0x7b5400280190. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fc640 00007f148e7d3846 00007f148e8dd90e 00007f148cd9e2a4 00007f148cd9e232 00007f148cd9e161 00007f148cd9e0dc 00007f148cd9d392 00007f1486d5b48c 00007f1486d5a8c6
0709 12:13:59.668916 (+75433645us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:13:50.581879 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:13:50.582870 (+   991us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:13:50.582926 (+    56us) write_op.cc:435] Acquired schema lock
0709 12:13:50.582963 (+    37us) tablet.cc:662] Decoding operations
0709 12:13:50.720612 (+137649us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:13:50.720721 (+   109us) write_op.cc:641] Partition lock acquired for write op
0709 12:13:50.720750 (+    29us) tablet.cc:685] Acquiring locks for 21 operations
0709 12:13:50.722479 (+  1729us) tablet.cc:701] Row locks acquired
0709 12:13:50.722533 (+    54us) write_op.cc:260] PREPARE: finished
0709 12:13:50.722723 (+   190us) write_op.cc:270] Start()
0709 12:13:50.722906 (+   183us) write_op.cc:276] Timestamp: P: 1783599230722650 usec, L: 0
0709 12:13:50.722949 (+    43us) op_driver.cc:348] REPLICATION: starting
0709 12:13:50.723831 (+   882us) log.cc:844] Serialized 8294 byte log entry
0709 12:13:52.822372 (+2098541us) spinlock_profiling.cc:243] Waited 9.09 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:13:53.351693 (+529321us) op_driver.cc:464] REPLICATION: finished
0709 12:13:53.618415 (+266722us) write_op.cc:301] APPLY: starting
0709 12:13:53.618547 (+   132us) tablet.cc:1367] starting BulkCheckPresence
0709 12:13:53.908678 (+290131us) tablet.cc:1370] finished BulkCheckPresence
0709 12:13:53.908746 (+    68us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:13:54.292452 (+383706us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:13:54.388146 (+ 95694us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=42,key_file_lookups=42,delta_file_lookups=0,mrs_lookups=0
0709 12:13:54.400081 (+ 11935us) write_op.cc:312] APPLY: finished
0709 12:13:54.400805 (+   724us) log.cc:844] Serialized 188 byte log entry
0709 12:13:54.633561 (+232756us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:54.697939 (+ 64378us) write_op.cc:454] Released schema lock
0709 12:13:54.749891 (+ 51952us) write_op.cc:341] FINISH: Updating metrics
0709 12:13:54.809061 (+ 59170us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:13:54.811905 (+  2844us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":2024276,"reported_tablets":53,"spinlock_wait_cycles":380394752,"child_traces":[["op",{"apply.queue_time_us":219112,"apply.run_cpu_time_us":60125,"apply.run_wall_time_us":1204751,"cfile_cache_hit":86,"cfile_cache_hit_bytes":91781,"cfile_init":1,"lbm_read_time_us":516,"lbm_reads_lt_1ms":4,"num_ops":21,"prepare.queue_time_us":216724,"prepare.run_cpu_time_us":35007,"prepare.run_wall_time_us":514185,"raft.queue_time_us":280492,"raft.run_cpu_time_us":33192,"raft.run_wall_time_us":161353,"replication_time_us":2628497,"spinlock_wait_cycles":20907264,"thread_start_us":898630,"threads_started":5,"wal-append.queue_time_us":661388,"wal-append.run_cpu_time_us":64859,"wal-append.run_wall_time_us":4013615}]]}
I20260709 12:13:59.925195 21971 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:59.909643 22043 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.15, Last appended by leader: 15, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:59.931039 12679 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:13:59.937482 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:13:59.938350 12680 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:13:59.918361 21740 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:13:59.959476 13302 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:13:59.960805 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:59.869150 21979 consensus_queue.cc:237] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:13:59.970808 22016 raft_consensus.cc:3055] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:13:59.971324 22016 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:13:59.972388 22016 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:13:59.974197 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:13:59.978205 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4157) took 1531 ms. Trace:
I20260709 12:13:59.978863 12597 rpcz_store.cc:276] 0709 12:13:58.446932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.388596 (+941664us) service_pool.cc:224] Handling call
0709 12:13:59.465209 (+ 76613us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:59.465910 (+   701us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:13:59.465945 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:59.492737 (+ 26792us) log.cc:844] Serialized 26 byte log entry
0709 12:13:59.762563 (+269826us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:59.762601 (+    38us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:59.762664 (+    63us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:59.977830 (+215166us) raft_consensus.cc:1685] finished
0709 12:13:59.977865 (+    35us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:59.978071 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":304,"raft.run_cpu_time_us":139,"raft.run_wall_time_us":140,"thread_start_us":227822,"threads_started":1,"wal-append.queue_time_us":395685}
I20260709 12:13:59.984381 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3630) took 1271 ms. Trace:
I20260709 12:13:59.984895 12528 rpcz_store.cc:276] 0709 12:13:58.712375 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.460293 (+747918us) service_pool.cc:224] Handling call
0709 12:13:59.460691 (+   398us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:59.461045 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:13:59.461076 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:59.461706 (+   630us) log.cc:844] Serialized 26 byte log entry
0709 12:13:59.777757 (+316051us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:59.777817 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:59.777914 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:13:59.983942 (+206028us) raft_consensus.cc:1685] finished
0709 12:13:59.983967 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:13:59.984154 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":315551,"threads_started":1,"wal-append.queue_time_us":502420}
I20260709 12:13:59.991679 21465 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:00.003958 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 135779us
I20260709 12:13:59.937415 21971 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:14:00.010718 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 322583us
I20260709 12:14:00.011946 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:00.016283 21358 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.017045 21358 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:00.019675 21640 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:00.020717 21753 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:00.021426 21753 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:00.023958 21753 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:00.013892 12643 leader_election.cc:400] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
I20260709 12:14:00.026665 12643 leader_election.cc:403] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:00.032192 12599 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:14:00.036530 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 91733us
I20260709 12:13:59.963953 21815 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:00.005968 21997 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:00.048130 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 153625us
I20260709 12:14:00.059742 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:13:59.973929 22016 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:14:00.076614 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.080010 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:00.084761 12572 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:00.028091 21984 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:00.099121 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 65106.2us
I20260709 12:14:00.056427 21997 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:00.104445 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 90814.9us
I20260709 12:14:00.104863 12671 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 24: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:14:00.099185 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 181882us
I20260709 12:14:00.074720 21815 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:00.093613 12672 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
W20260709 12:14:00.103838 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 379650us
I20260709 12:14:00.123371 12599 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 31.
I20260709 12:14:00.105573 21465 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:00.107192 21971 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:00.133038 22015 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 32.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
W20260709 12:14:00.133936 12427 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59006 (request call id 181) took 17295 ms (client timeout 15000 ms). Trace:
I20260709 12:14:00.133687 21978 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:00.134076 12646 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:00.126865 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:14:00.140750 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 66757.4us
W20260709 12:14:00.143190 12427 rpcz_store.cc:269] 0709 12:13:42.826709 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:00.114033 (+17287324us) service_pool.cc:211] Skipping call since client already timed out
0709 12:14:00.121865 (+  7832us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:14:00.097084 21551 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:00.134456 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:00.149631 12678 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:00.150835 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:00.151230 21551 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:00.139150 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:00.152395 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:14:00.158557 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.159004 13337 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:00.159976 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:00.176991 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.119899 22071 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:14:00.177505 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:00.179248 12601 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.577s	user 0.010s	sys 0.002s
I20260709 12:14:00.165473 21358 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:00.154922 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:00.157320 22063 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.095s	user 0.000s	sys 0.000s
I20260709 12:14:00.180428 21640 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:14:00.180231 21935 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.181391 21972 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:00.162938 22043 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:14:00.182332 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.195217 12601 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:14:00.136907 21971 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:00.180212 22070 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.173s	user 0.000s	sys 0.002s
I20260709 12:14:00.203153 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:00.208981 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4165) took 1480 ms. Trace:
I20260709 12:14:00.199286 22020 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:14:00.227568 12671 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:14:00.211557 12601 rpcz_store.cc:276] 0709 12:13:58.728575 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.326560 (+597985us) service_pool.cc:224] Handling call
0709 12:14:00.208594 (+882034us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:00.227795 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:00.178583 21551 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 15, Time since last communication: 0.032s
I20260709 12:14:00.204434 21972 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:00.203900 21997 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:00.267817 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.207362 22071 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:00.210492 21971 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:00.210618 21640 raft_consensus.cc:697] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 31 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:00.302889 21640 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:00.306931 22069 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.212s	user 0.001s	sys 0.000s
I20260709 12:14:00.307597 22019 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:00.308020 22019 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:00.308386 22019 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:00.308813 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3564) took 1195 ms. Trace:
W20260709 12:14:00.287142 22067 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.066s	user 0.000s	sys 0.001s
I20260709 12:14:00.309238 12529 rpcz_store.cc:276] 0709 12:13:59.113795 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.504356 (+390561us) service_pool.cc:224] Handling call
0709 12:13:59.504916 (+   560us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:59.534461 (+ 29545us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:59.534505 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:59.535118 (+   613us) log.cc:844] Serialized 26 byte log entry
0709 12:13:59.780200 (+245082us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:13:59.813355 (+ 33155us) log.cc:844] Serialized 14 byte log entry
0709 12:13:59.956173 (+142818us) spinlock_profiling.cc:243] Waited 3.52 ms on lock 0x7b18004aa520. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:13:59.956485 (+   312us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:59.956596 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:00.308427 (+351831us) raft_consensus.cc:1685] finished
0709 12:14:00.308551 (+   124us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:00.308739 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81195,"prepare.queue_time_us":33627,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":54,"raft.queue_time_us":173551,"raft.run_cpu_time_us":255,"raft.run_wall_time_us":6100,"spinlock_wait_cycles":8098176,"thread_start_us":244688,"threads_started":1,"wal-append.queue_time_us":524294}
I20260709 12:14:00.297241 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:00.313608 12677 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:14:00.314760 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:00.315611 12677 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 24
W20260709 12:14:00.300261 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:00.316363 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.312418 21753 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.316370 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3561) took 1434 ms. Trace:
I20260709 12:14:00.311869 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4159) took 1836 ms. Trace:
W20260709 12:14:00.320222 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 513575us
I20260709 12:14:00.296777 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:00.324093 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:00.320552 12598 rpcz_store.cc:276] 0709 12:13:58.476043 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.492642 (+1016599us) service_pool.cc:224] Handling call
0709 12:13:59.506300 (+ 13658us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:59.531102 (+ 24802us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:13:59.531139 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:59.531758 (+   619us) log.cc:844] Serialized 26 byte log entry
0709 12:13:59.917872 (+386114us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:13:59.917912 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:59.917996 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:00.311628 (+393632us) raft_consensus.cc:1685] finished
0709 12:14:00.311649 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:00.311804 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24162,"raft.queue_time_us":25589,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"thread_start_us":385423,"threads_started":1,"wal-append.queue_time_us":562514}
I20260709 12:14:00.327625 12673 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:14:00.327334 21753 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:00.328521 21815 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:00.328814 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:00.329295 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3631) took 1616 ms. Trace:
I20260709 12:14:00.329358 22071 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:00.256590 22020 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:00.331216 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:00.289927 21972 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:00.212185 21935 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:00.340427 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:00.341468 12672 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
W20260709 12:14:00.335024 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 213164us
I20260709 12:14:00.320182 12526 rpcz_store.cc:276] 0709 12:13:58.882181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.499540 (+617359us) service_pool.cc:224] Handling call
0709 12:13:59.500057 (+   517us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:13:59.542833 (+ 42776us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:13:59.542868 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:13:59.543390 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 12:13:59.764066 (+220676us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:13:59.764124 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:13:59.764233 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:00.316128 (+551895us) raft_consensus.cc:1685] finished
0709 12:14:00.316149 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:00.316301 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":42282,"raft.queue_time_us":49498,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":16892,"thread_start_us":220246,"threads_started":1,"wal-append.queue_time_us":358387}
I20260709 12:14:00.329931 12522 rpcz_store.cc:276] 0709 12:13:58.712848 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:59.467434 (+754586us) service_pool.cc:224] Handling call
0709 12:13:59.467866 (+   432us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:13:59.507070 (+ 39204us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:13:59.507455 (+   385us) log.cc:844] Serialized 14 byte log entry
0709 12:14:00.123028 (+615573us) log.cc:844] Serialized 14 byte log entry
0709 12:14:00.123341 (+   313us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:00.134579 (+ 11238us) log.cc:844] Serialized 52 byte log entry
0709 12:14:00.134825 (+   246us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:00.135066 (+   241us) log.cc:844] Serialized 14 byte log entry
0709 12:14:00.158385 (+ 23319us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:00.158504 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:00.329038 (+170534us) raft_consensus.cc:1685] finished
0709 12:14:00.329061 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:00.329214 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":58633,"prepare.queue_time_us":85425,"prepare.run_cpu_time_us":143,"prepare.run_wall_time_us":148,"raft.queue_time_us":71107,"raft.run_cpu_time_us":542,"raft.run_wall_time_us":18083,"thread_start_us":584844,"threads_started":2,"wal-append.queue_time_us":426905}
W20260709 12:14:00.362991 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 223954us
I20260709 12:14:00.329936 21753 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:00.374207 12496 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:00.336197 22020 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:14:00.371487 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 190527us
I20260709 12:14:00.379839 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:00.383278 12674 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:00.386523 12494 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:00.402034 12494 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:00.396067 12571 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:00.403312 12672 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:00.374408 22071 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:14:00.408401 12494 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:00.410406 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 86397.5us
I20260709 12:14:00.410584 21099 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:00.413434 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:00.414012 21465 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:14:00.392778 21997 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.025s
I20260709 12:14:00.418555 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:14:00.419701 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.420022 12678 raft_consensus.cc:2410] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 29 because replica has last-logged OpId of term: 28 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 14.
I20260709 12:14:00.382778 21358 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:00.437711 13334 raft_consensus.cc:2410] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 29 because replica has last-logged OpId of term: 28 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 19.
I20260709 12:14:00.438862 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:00.440405 12674 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:00.444460 12494 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:00.449807 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 207261us
I20260709 12:14:00.437858 21358 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:00.451777 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 317835us
I20260709 12:14:00.444104 21978 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:00.486048 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:00.481571 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:14:00.487978 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 219265us
I20260709 12:14:00.495682 12672 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:00.497252 12572 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:00.463615 21358 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:00.498957 12526 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:00.502490 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:00.506018 12522 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 22. Current term is 23. Ops: []
I20260709 12:14:00.509799 12529 raft_consensus.cc:2368] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 27. Current term is 28.
I20260709 12:14:00.521215 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:00.525774 12528 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:00.536095 22081 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:14:00.537246 22081 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:14:00.538996 22091 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:14:00.539652 22091 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:00.540068 22091 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:14:00.548445 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 94286.4us
I20260709 12:14:00.550745 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:00.553519 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:00.554934 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:00.560340 12678 raft_consensus.cc:2410] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 22 because replica has last-logged OpId of term: 21 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 20 index: 13.
I20260709 12:14:00.526947 21984 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:14:00.562258 13305 leader_election.cc:400] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 27. Current term is 28.
I20260709 12:14:00.569784 13305 leader_election.cc:403] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 election: Cancelling election due to peer responding with higher term
I20260709 12:14:00.563805 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:00.570545 12529 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:14:00.572109 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 200827us
W20260709 12:14:00.575579 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 180978us
I20260709 12:14:00.561462 12646 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:00.584844 21738 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:14:00.585629 21738 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:00.587219 21738 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 33.22, Last appended by leader: 22, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:00.589305 21738 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:00.599750 21465 raft_consensus.cc:697] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:00.610697 12678 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:14:00.618323 12672 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 22 index: 13. (index mismatch)
I20260709 12:14:00.559384 22075 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
I20260709 12:14:00.577572 21935 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:00.620011 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:00.619521 12675 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
I20260709 12:14:00.604213 21979 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Stepping down as leader of term 27
W20260709 12:14:00.668699 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 171598us
W20260709 12:14:00.627269 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 182904us
W20260709 12:14:00.652098 13304 leader_election.cc:336] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.631s (SENT)
W20260709 12:14:00.625465 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 159842us
I20260709 12:14:00.671934 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:00.676990 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.677989 12572 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:00.678282 22085 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:00.679559 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:00.680876 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:00.681847 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 123252us
W20260709 12:14:00.682272 22072 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.207s	user 0.000s	sys 0.001s
I20260709 12:14:00.683049 12678 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:14:00.683518 12646 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:00.685583 12676 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 28 index: 14. (term mismatch)
I20260709 12:14:00.686009 12676 pending_rounds.cc:85] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 13
I20260709 12:14:00.686319 12676 pending_rounds.cc:107] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 27.14
I20260709 12:14:00.686620 12676 raft_consensus.cc:2889] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:00.688804 12676 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26. Ops: []
I20260709 12:14:00.671428 22043 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:14:00.694415 22079 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 12:14:00.698752 22087 consensus_queue.cc:1059] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 24.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:14:00.697948 22091 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:00.700353 22094 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:14:00.700989 21997 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:14:00.701159 22100 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:00.701510 22091 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:00.702023 22100 raft_consensus.cc:3055] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:14:00.702224 21935 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:00.702476 22100 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:00.703461 22100 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:00.704774 22100 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:00.608460 21640 consensus_queue.cc:1059] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.103s
I20260709 12:14:00.699301 22097 consensus_queue.cc:1059] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:14:00.669991 21979 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:00.708632 21979 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:00.709750 21979 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:00.601292 21465 consensus_queue.cc:237] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:00.715261 22094 raft_consensus.cc:697] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:00.728605 12671 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:14:00.726500 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:14:00.727946 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 125847us
W20260709 12:14:00.742311 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 123643us
I20260709 12:14:00.744016 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:00.744438 12674 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 34.
I20260709 12:14:00.758737 12677 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:14:00.762849 12643 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:00.767016 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 114978us
I20260709 12:14:00.770174 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:00.770993 22015 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:14:00.771196 13337 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:14:00.771971 22015 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:00.772595 22015 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:00.757172 12496 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:14:00.782354 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 55995.7us
I20260709 12:14:00.790548 22074 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:00.790831 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:00.791369 22074 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:00.791548 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:00.791927 22074 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:00.793061 22088 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:00.794252 22088 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:00.794992 22088 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:00.798416 22071 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:14:00.799183 22071 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:00.799569 21935 raft_consensus.cc:3055] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:14:00.799999 21935 raft_consensus.cc:740] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:00.800944 21935 consensus_queue.cc:260] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:00.765281 22043 consensus_queue.cc:1059] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 23.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.041s
I20260709 12:14:00.807606 22043 raft_consensus.cc:3055] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:14:00.808079 22043 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:00.809051 22043 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:00.810463 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:00.810791 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.801963 21935 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:14:00.817899 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 60794.4us
I20260709 12:14:00.819608 12529 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 28.
I20260709 12:14:00.819875 12526 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 31.
I20260709 12:14:00.821514 13305 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:14:00.835100 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 113655us
W20260709 12:14:00.837805 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 228736us
I20260709 12:14:00.840276 13334 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:00.794289 22081 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:14:00.841707 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:00.846302 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.849408 22102 consensus_queue.cc:1059] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:14:00.850754 21099 raft_consensus.cc:3055] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:14:00.851239 21358 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:00.851186 21099 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:00.852105 22102 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260709 12:14:00.851641 12496 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:00.853135 21099 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:00.855232 21099 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:00.854911 22113 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:14:00.858779 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:00.823568 22105 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:00.862778 12672 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 15. (term mismatch)
I20260709 12:14:00.863029 13337 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:14:00.866045 12672 pending_rounds.cc:85] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 14
I20260709 12:14:00.866592 12672 pending_rounds.cc:107] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 25.15
I20260709 12:14:00.866981 12672 raft_consensus.cc:2889] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:00.868039 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:00.872184 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:00.873167 12676 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
W20260709 12:14:00.876330 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:00.865381 21972 consensus_meta.cc:287] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.512s	user 0.011s	sys 0.000s
I20260709 12:14:00.858471 21640 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:00.886735 12528 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 31.
I20260709 12:14:00.886989 21740 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:14:00.853572 21358 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:00.891739 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:00.878376 22102 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
W20260709 12:14:00.895305 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52800.2us
I20260709 12:14:00.881317 22043 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:00.857286 22113 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.030s
I20260709 12:14:00.879859 22105 raft_consensus.cc:697] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:00.904506 22015 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:00.905372 22015 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:00.905889 22086 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:14:00.906714 22086 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 31 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:00.886721 22016 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:00.910430 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 76090.2us
I20260709 12:14:00.910602 22016 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:00.900414 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:00.901571 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 222105us
I20260709 12:14:00.905927 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:00.895376 21740 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:00.914322 12571 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:00.890213 21713 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 12:14:00.923118 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:00.897928 22102 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:00.924358 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:00.891310 21972 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:00.900210 22112 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:00.894801 22081 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 25.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.024s
I20260709 12:14:00.921097 21740 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:00.929104 12569 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:00.925294 21640 raft_consensus.cc:697] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:00.919111 21722 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:00.911737 22071 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.020s
I20260709 12:14:00.948735 12494 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:00.930263 22091 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:14:00.906040 22015 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:00.943585 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 53611.6us
I20260709 12:14:00.931937 22102 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:14:00.969408 22019 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.660s	user 0.014s	sys 0.004s
I20260709 12:14:00.974033 22115 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 12:14:00.974967 22016 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:00.975652 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:00.976518 22119 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 25.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.005s
I20260709 12:14:00.970376 21977 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:00.905368 22107 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.077s
I20260709 12:14:00.968385 21722 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:00.892064 22080 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.056s
I20260709 12:14:00.885305 22094 consensus_queue.cc:237] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:00.968137 22081 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:14:00.989450 21753 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:01.004369 21935 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.006290 21972 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:01.001124 22019 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:01.030294 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 87699.1us
I20260709 12:14:01.007210 21358 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:14:01.008481 22115 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:00.973088 22091 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:00.904683 22105 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 26.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:00.907859 22086 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:01.041481 22019 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:01.043021 22107 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election won for term 22
W20260709 12:14:01.010423 21971 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.677s	user 0.012s	sys 0.001s
I20260709 12:14:01.012527 21713 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:01.071805 21713 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.066076 22085 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader election lost for term 27. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 27. Current term is 28.
I20260709 12:14:01.023555 21722 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:00.968969 22088 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:01.038347 22091 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:14:01.049775 22020 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.617s	user 0.008s	sys 0.005s
W20260709 12:14:01.055609 22114 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.076s	user 0.001s	sys 0.000s
I20260709 12:14:01.044171 21753 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:01.023756 22081 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:01.061975 22115 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:01.043687 22107 raft_consensus.cc:697] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:01.062740 22015 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:01.068470 22074 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:14:01.034345 21935 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.011339 21977 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:01.062734 12494 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:14:01.085062 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 94369.9us
I20260709 12:14:01.085736 22112 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:14:01.089563 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 119229us
W20260709 12:14:01.092118 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 180763us
W20260709 12:14:01.090546 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 156470us
I20260709 12:14:01.107179 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:14:01.099627 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 208019us
I20260709 12:14:01.099752 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:01.108485 12673 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:14:01.073454 21971 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:01.115406 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52810.6us
I20260709 12:14:01.094548 22115 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:01.100310 21977 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:01.096195 22087 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:01.138912 21753 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.093539 22020 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.142753 21740 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:01.149698 12678 raft_consensus.cc:1240] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
I20260709 12:14:01.150733 22120 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:14:01.151399 22120 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:01.151948 22118 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.073s
I20260709 12:14:01.152505 22120 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 32.21, Last appended by leader: 21, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:01.153421 22019 consensus_queue.cc:1059] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:14:01.153906 22081 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.126008 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:01.155810 12529 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 30.
I20260709 12:14:01.122601 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:01.157532 12528 raft_consensus.cc:2410] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 27 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:14:01.158367 12526 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:14:01.125510 12523 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 22. Current term is 23. Ops: []
I20260709 12:14:01.092893 21640 consensus_queue.cc:237] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:01.128160 21971 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:01.128612 22102 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.128916 22122 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:01.129720 22107 consensus_queue.cc:237] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 20.13, Last appended by leader: 13, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:01.134737 22074 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:14:01.136359 22054 raft_consensus.cc:3055] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:14:01.114197 22071 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 12:14:01.120743 12571 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:01.167840 12673 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:14:01.166441 13305 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:01.123971 22115 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:14:01.242347 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 76030us
I20260709 12:14:01.243057 22054 raft_consensus.cc:740] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:01.172768 21713 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:01.245172 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 92727.8us
I20260709 12:14:01.249168 12494 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:01.154563 22126 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 21.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.028s
W20260709 12:14:01.246096 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 321822us
I20260709 12:14:01.181746 21358 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.245879 22086 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:01.258663 22086 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:01.259084 22086 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:01.258781 21358 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.259366 22129 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:14:01.259967 22129 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:01.260833 22129 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.261227 21358 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:01.256162 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 135531us
I20260709 12:14:01.179911 22043 consensus_queue.cc:1059] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 21.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.075s
I20260709 12:14:01.168709 21465 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:14:01.272981 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:14:01.278805 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.279942 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:01.281139 12673 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 20 FOLLOWER]: Advancing to term 22
I20260709 12:14:01.289278 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:14:01.291859 13304 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.537s (SENT) [suppressed 21 similar messages]
I20260709 12:14:01.295010 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:01.296258 12525 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:01.302927 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:01.255537 22081 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:01.254077 21740 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:01.306252 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.256148 22020 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:01.277118 21713 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:01.308102 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:01.315979 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:01.317430 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:01.318861 12572 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:01.315838 12679 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 28. Current term is 29. Ops: [28.15-28.15]
I20260709 12:14:01.317289 12530 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:14:01.320094 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:01.283571 22071 raft_consensus.cc:3055] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:14:01.322528 12674 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:14:01.321552 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:01.343293 12522 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 21.
I20260709 12:14:01.324236 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:01.344493 12527 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:01.322013 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:14:01.338023 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 151033us
I20260709 12:14:01.324534 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:01.307241 22087 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.239205 22119 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.067s
I20260709 12:14:01.347652 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:01.304207 22085 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.348599 12603 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:14:01.244859 22122 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.311800 21465 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:01.360785 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.358754 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:01.367623 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:01.368407 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 83600us
I20260709 12:14:01.368906 12600 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 30.
I20260709 12:14:01.338999 22121 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.363019 22043 raft_consensus.cc:3055] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:14:01.337874 21713 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:01.382182 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 63635.5us
W20260709 12:14:01.362569 22100 consensus_meta.cc:287] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.657s	user 0.013s	sys 0.004s
I20260709 12:14:01.350083 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:14:01.395390 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 103562us
I20260709 12:14:01.342337 22071 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:01.339054 12673 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:14:01.413682 12528 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 31: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:14:01.409929 22131 raft_consensus.cc:3055] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:14:01.394137 22128 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.410547 21740 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:01.309638 22112 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.407635 22043 raft_consensus.cc:740] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:14:01.439121 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.444800 12527 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29.
I20260709 12:14:01.365118 22085 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:01.416507 21935 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:01.361939 22087 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.456390 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:01.431442 22131 raft_consensus.cc:740] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:01.434525 22119 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Stepping down as leader of term 27
W20260709 12:14:01.472898 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 66935.8us
I20260709 12:14:01.454324 21740 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:01.405171 22121 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:01.462833 22102 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:01.516837 12598 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 28.
I20260709 12:14:01.471148 21935 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:01.435571 22128 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.492477 21713 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.14, Last appended by leader: 14, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.529830 22128 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:01.521881 22122 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:01.535167 21722 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.465167 22119 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:01.526409 22131 consensus_queue.cc:260] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.405478 22054 consensus_queue.cc:260] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:01.502949 22113 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:14:01.490367 22111 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:01.479856 22132 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:01.555476 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:01.556957 12527 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: [22.14-22.14]
I20260709 12:14:01.558144 12572 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:01.567137 12522 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:14:01.569046 22131 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:14:01.455828 12528 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:14:01.585341 22125 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.175s
I20260709 12:14:01.558463 22088 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:01.554177 22113 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:01.644487 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:01.579643 22133 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.157s	user 0.000s	sys 0.001s
W20260709 12:14:01.612711 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.614753 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:01.650985 12522 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 24: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:14:01.616062 22138 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
W20260709 12:14:01.616396 22135 log.cc:927] Time spent T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.168s	user 0.000s	sys 0.001s
W20260709 12:14:01.616729 22130 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.187s	user 0.000s	sys 0.000s
I20260709 12:14:01.569068 21740 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:01.638249 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:01.639087 21971 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.539826 22071 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.574285 22110 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.673696 22110 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.615626 22124 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
W20260709 12:14:01.678265 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 82386.1us
I20260709 12:14:01.678829 22113 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.680065 12643 leader_election.cc:400] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23.
I20260709 12:14:01.682138 22019 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:01.682364 12527 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25. Ops: []
W20260709 12:14:01.683658 21977 consensus_meta.cc:287] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.510s	user 0.011s	sys 0.000s
I20260709 12:14:01.638991 22119 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.643638 12525 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29.
I20260709 12:14:01.645783 12680 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:14:01.684134 21977 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:01.684721 22119 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:01.685287 12598 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:01.686929 22121 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:01.587368 22132 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:01.693555 22091 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.592s	user 0.010s	sys 0.000s
I20260709 12:14:01.515799 22043 consensus_queue.cc:260] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.651165 12528 raft_consensus.cc:606] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:01.652613 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:01.592161 22136 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:01.695374 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.695645 12528 raft_consensus.cc:614] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:01.700062 13334 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 33 index: 22. Preceding OpId from leader: term: 34 index: 22. (term mismatch)
I20260709 12:14:01.700459 13334 pending_rounds.cc:85] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 21
I20260709 12:14:01.697540 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:01.701094 13334 pending_rounds.cc:107] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 33.22
I20260709 12:14:01.701448 13334 raft_consensus.cc:2889] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:01.701551 12525 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 29.
I20260709 12:14:01.593487 22102 raft_consensus.cc:2764] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election decision vote started in defunct term 27: won
I20260709 12:14:01.656148 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:14:01.669185 22137 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.240s	user 0.000s	sys 0.001s
W20260709 12:14:01.669797 22123 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.345s	user 0.000s	sys 0.004s
I20260709 12:14:01.670459 21971 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.647234 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:01.465349 21465 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.696028 12673 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
W20260709 12:14:01.694725 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 51121us
I20260709 12:14:01.698859 22087 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:01.706220 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 166935us
W20260709 12:14:01.708662 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52798.4us
I20260709 12:14:01.694088 22091 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.694782 22043 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:14:01.714911 12529 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:14:01.680099 22113 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:01.726536 12531 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 17. (term mismatch)
I20260709 12:14:01.727205 12531 pending_rounds.cc:85] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 16
I20260709 12:14:01.724490 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:01.718154 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:01.699532 22141 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 27.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.065s
I20260709 12:14:01.729527 12531 pending_rounds.cc:107] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 24.17
I20260709 12:14:01.731482 12531 raft_consensus.cc:2889] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:01.712940 21935 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:01.730293 12571 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:01.706725 22136 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:01.738739 22122 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:01.722432 22086 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:01.739269 21935 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:01.748066 21978 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:01.745091 22144 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:14:01.775924 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:01.762598 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:01.744709 22141 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:14:01.767091 22117 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:14:01.767706 22125 consensus_queue.cc:1059] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:14:01.769400 22121 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:14:01.769961 21740 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.776686 12677 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:14:01.765350 22132 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:01.742734 22087 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.019s
I20260709 12:14:01.754870 22122 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:01.806137 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:01.803679 12677 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:14:01.776508 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:01.773455 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:01.782325 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:01.777473 12599 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 29.
I20260709 12:14:01.792018 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:01.795320 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:01.796157 12525 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 26. Current term is 27. Ops: [26.15-26.15]
I20260709 12:14:01.778049 21935 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:01.798318 22081 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:14:01.812077 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:01.814426 12524 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 29: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:01.798606 12671 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:14:01.799037 21740 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.786478 12496 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:01.801178 22075 raft_consensus.cc:3055] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:14:01.801752 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:14:01.802827 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.772104 12646 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:01.824586 12674 raft_consensus.cc:2368] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29.
W20260709 12:14:01.810324 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 72702.5us
W20260709 12:14:01.817016 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 43678us
I20260709 12:14:01.781312 22144 raft_consensus.cc:697] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:01.777248 21978 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:01.780342 22141 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:01.834873 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:01.835798 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 63803us
I20260709 12:14:01.823097 12676 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:14:01.835855 12525 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:14:01.823442 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:01.832132 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:01.838660 12524 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:01.842577 12527 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 23: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:01.839280 12494 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:14:01.695775 22074 consensus_queue.cc:1175] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" responder_term: 23 status { last_received { term: 21 index: 13 } last_committed_idx: 13 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:14:01.846184 22100 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:01.846450 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:01.846845 22100 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.847407 12530 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 29: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:01.847553 22086 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:01.849888 12523 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 28. Current term is 29. Ops: []
W20260709 12:14:01.851495 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.851733 22119 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:14:01.829718 22125 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:01.852684 22125 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:01.853209 22125 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:01.854575 22100 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:01.860157 21740 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:01.820644 22107 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:01.862888 22020 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:14:01.862782 22107 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.863831 22020 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:01.863931 21984 consensus_queue.cc:1059] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:14:01.865428 21984 raft_consensus.cc:3055] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:14:01.823982 22075 raft_consensus.cc:740] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:01.813922 22122 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:01.866667 12494 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:01.866930 22075 consensus_queue.cc:260] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.868327 22075 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:01.779633 22088 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.042s
I20260709 12:14:01.864965 22020 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:01.870981 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:01.865868 21984 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:01.879522 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:01.829901 21753 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:01.885501 22088 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:14:01.881067 12646 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:01.837730 21978 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:01.862771 21740 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:01.852257 22119 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:01.918550 12646 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:14:01.924208 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 54955.2us
I20260709 12:14:01.925125 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:01.889941 22127 raft_consensus.cc:2764] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election decision vote started in defunct term 30: won
I20260709 12:14:01.937674 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:01.862696 22136 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:01.896098 22088 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:01.956949 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:01.922077 22019 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:01.961652 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:01.957484 12525 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 27. Current term is 28. Ops: [27.17-27.17]
I20260709 12:14:01.924517 22117 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:01.897078 21977 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:01.964949 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 187924us
I20260709 12:14:01.925182 22119 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:14:01.927448 22126 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election won for term 29
W20260709 12:14:01.960992 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 88084.4us
I20260709 12:14:01.969489 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:01.976358 22107 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:01.978835 22167 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.092s	user 0.001s	sys 0.000s
W20260709 12:14:01.983094 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 45518.3us
I20260709 12:14:01.984196 22172 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:14:01.967636 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:14:01.978740 22115 consensus_meta.cc:287] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.725s	user 0.013s	sys 0.000s
W20260709 12:14:02.016845 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:02.019364 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.016419 12496 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:01.993841 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:02.023236 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:02.025066 12527 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:14:01.968107 22091 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:02.029621 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4398) took 1851 ms. Trace:
I20260709 12:14:02.028288 22054 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:14:01.990808 21971 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:02.033030 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.019816 22126 raft_consensus.cc:697] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:01.972839 22019 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:02.038269 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:02.034873 12678 rpcz_store.cc:276] 0709 12:14:00.177714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:00.684012 (+506298us) service_pool.cc:224] Handling call
0709 12:14:00.684345 (+   333us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:01.150423 (+466078us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:02.029343 (+878920us) spinlock_profiling.cc:243] Waited 465 ms on lock 0x7b54002c5954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:02.029551 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1069560064}
I20260709 12:14:02.049494 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:01.938189 22141 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.15, Last appended by leader: 15, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:14:02.055606 12530 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:14:02.023346 22131 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
I20260709 12:14:02.056001 22141 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:02.052578 12646 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:01.959074 22088 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:02.018957 21753 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:02.062438 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.063265 12525 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 29: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:14:02.066756 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.022332 22115 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:02.071331 12679 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 28: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:02.072335 22115 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:02.072325 22126 consensus_queue.cc:237] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 27.14, Last appended by leader: 14, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:02.074298 22136 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:14:02.074965 22136 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:02.075941 22136 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:02.076001 22119 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:02.078449 22159 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:14:02.079991 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:02.080647 12671 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 18. (term mismatch)
I20260709 12:14:02.081029 12671 pending_rounds.cc:85] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 17
I20260709 12:14:02.081277 12671 pending_rounds.cc:107] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 27.18
I20260709 12:14:02.081051 22173 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:14:02.082966 12524 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:14:02.076267 22125 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:02.039361 22127 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:14:02.081676 12671 raft_consensus.cc:2889] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:02.086380 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:02.094168 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:01.986539 22117 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:02.105971 22125 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:02.107048 22117 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:02.107714 21753 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:01.971707 22144 consensus_queue.cc:237] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:02.113616 22174 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.114229 22174 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:02.118166 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:01.987226 21984 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:02.133039 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:02.133991 12524 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 23: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:02.119094 22088 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:02.113677 22127 raft_consensus.cc:697] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:02.145368 12672 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 14. (term mismatch)
I20260709 12:14:02.144176 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:02.127836 22115 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:02.146669 12525 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:14:02.154981 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.145967 12672 pending_rounds.cc:85] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 13
I20260709 12:14:02.167501 12672 pending_rounds.cc:107] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 22.14
I20260709 12:14:02.167835 12672 raft_consensus.cc:2889] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:02.168411 12496 leader_election.cc:400] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29.
I20260709 12:14:02.168968 12496 leader_election.cc:403] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Cancelling election due to peer responding with higher term
W20260709 12:14:02.174994 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.175355 12496 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:02.179559 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:02.180303 12600 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:02.208096 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:02.208796 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:02.222867 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:02.169449 22136 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:02.239620 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 135267us
I20260709 12:14:02.234030 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:02.242277 12603 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:02.243391 12674 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
W20260709 12:14:02.244784 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.174808 22131 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.178460 21713 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election lost for term 28. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29.
I20260709 12:14:02.264117 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:02.171489 22141 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.080s
W20260709 12:14:02.265123 22163 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
I20260709 12:14:02.265731 22127 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:02.181617 22118 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:02.267985 12601 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 27.
W20260709 12:14:02.281529 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.251592 22119 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:02.288995 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:02.290460 12524 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:14:02.162573 22019 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:14:02.308423 22177 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:14:02.287369 22118 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:02.310990 22164 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.282s	user 0.002s	sys 0.000s
I20260709 12:14:02.313365 21978 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:02.209024 22172 consensus_queue.cc:1059] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 12, Time since last communication: 0.088s
I20260709 12:14:02.319669 22118 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:02.323877 22122 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:02.324740 21972 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.236032 22105 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.062s
I20260709 12:14:02.334913 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:02.341307 12523 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:02.255213 22136 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:02.343994 21972 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:02.347988 22178 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:14:02.270411 22131 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:02.351596 12523 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:14:02.360203 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:02.407665 12525 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
W20260709 12:14:02.409179 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 94813.3us
I20260709 12:14:02.379410 12496 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:02.414466 22105 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:14:02.415004 22105 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:02.415454 22105 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:02.415542 21972 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:02.412806 12646 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:14:02.426375 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.426626 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:02.431716 12572 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:02.433295 12572 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:02.436000 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:02.446550 12572 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:02.412887 22113 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.682s	user 0.013s	sys 0.000s
I20260709 12:14:02.454375 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:02.458607 12522 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:14:02.447005 21972 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:02.461766 22132 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:14:02.463891 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48476 (request call id 92) took 1046 ms. Trace:
I20260709 12:14:02.464383 12528 rpcz_store.cc:276] 0709 12:14:01.426683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.431173 (+  4490us) service_pool.cc:224] Handling call
0709 12:14:02.463608 (+1032435us) spinlock_profiling.cc:243] Waited 78.7 ms on lock 0x7b5400210cd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1a853 00007f148d9e860a 00007f1486f6e7d4 00007f1486f6e762 00007f1486f6e691 00007f1486f6e60c 00007f1486f6d8c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:02.463773 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":180960000}
I20260709 12:14:02.473867 22136 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:02.475539 21978 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:02.478144 12600 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 31.
I20260709 12:14:02.478479 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:02.427299 22145 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:14:02.445050 22119 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:02.483551 12528 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:14:02.447481 22141 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:02.495271 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:02.496582 22131 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:02.497260 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.498191 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:02.482765 22075 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.614s	user 0.010s	sys 0.005s
I20260709 12:14:02.500780 12646 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:02.503767 12680 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:14:02.507339 22131 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:14:02.508111 22131 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:02.508602 22131 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:14:02.515157 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:02.521675 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:02.522437 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.535452 13337 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:14:02.498409 22119 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:02.494513 22094 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:14:02.537863 21935 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.712s	user 0.009s	sys 0.004s
I20260709 12:14:02.494095 22145 raft_consensus.cc:697] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:14:02.543593 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 129888us
W20260709 12:14:02.549254 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.556061 22145 consensus_queue.cc:237] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:02.560058 22119 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:02.562469 22155 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.818s	user 0.009s	sys 0.012s
I20260709 12:14:02.565019 22132 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:14:02.565362 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:02.570520 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:02.570760 12606 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
W20260709 12:14:02.572009 22181 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.309s	user 0.006s	sys 0.010s
I20260709 12:14:02.572697 22171 raft_consensus.cc:3055] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:14:02.573184 22171 raft_consensus.cc:740] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:02.574813 12600 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:02.575858 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:02.575506 12601 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 30. Current term is 31. Ops: []
W20260709 12:14:02.589392 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.608412 12530 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25. Ops: []
I20260709 12:14:02.609093 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:02.609930 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:02.610597 12679 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:02.612848 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:02.550333 22192 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:02.629745 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:02.640911 12601 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:14:02.519697 22118 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:02.637903 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:02.638787 12523 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3af899b7fab042d0ac433430f162d106"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:14:02.578508 22122 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:02.648394 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.648588 12523 raft_consensus.cc:606] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:02.649551 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 22 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:02.635900 12603 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 30.
W20260709 12:14:02.651104 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.608697 22097 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.619191 21978 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.614084 22119 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.656497 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:14:02.667551 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.554020 21935 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:02.576668 22131 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:02.670396 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:02.673664 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:02.674154 12525 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:02.652684 22107 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:02.676949 12606 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:02.677596 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:02.672077 22097 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:02.684209 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:02.669373 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:02.687732 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.681850 12599 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
W20260709 12:14:02.683022 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.679387 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:02.698342 12600 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:14:02.574067 22171 consensus_queue.cc:260] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:02.704914 12679 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:02.706614 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:02.713290 12600 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:14:02.673509 21978 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:02.716809 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:02.718554 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:02.720047 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:02.720665 12605 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:02.719448 12676 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 23: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:02.722323 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:02.723124 12676 raft_consensus.cc:2410] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 30 because replica has last-logged OpId of term: 28 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 14.
I20260709 12:14:02.723883 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:02.724452 12676 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:14:02.724658 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:02.725275 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:02.725668 12604 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 27: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:02.726604 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:02.727310 12676 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 28
I20260709 12:14:02.728866 22071 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:02.726986 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:02.652637 22192 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:02.745654 12676 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:02.733278 22110 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:02.747131 22192 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:02.748382 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:02.739092 21935 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:02.750151 12604 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:14:02.737095 22118 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:02.744849 22097 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:02.718623 21978 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:02.766067 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:02.676796 22119 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:02.740810 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:02.743088 22105 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:02.750806 12676 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:14:02.778484 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:02.779372 12676 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 31: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:14:02.752179 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 297926us
I20260709 12:14:02.730238 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:02.781421 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:02.755306 12599 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:14:02.756982 21984 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:02.763110 22110 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:14:02.735175 22131 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:02.769220 12600 raft_consensus.cc:2368] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29.
I20260709 12:14:02.786597 12679 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:02.776671 12603 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:14:02.784334 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:02.788089 12676 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:02.785257 12572 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:02.786494 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:02.789204 12604 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:14:02.791857 12606 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:14:02.795886 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:02.816373 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3666) took 1117 ms. Trace:
I20260709 12:14:02.816906 12531 rpcz_store.cc:276] 0709 12:14:01.699249 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.699388 (+   139us) service_pool.cc:224] Handling call
0709 12:14:01.721539 (+ 22151us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:02.815624 (+1094085us) spinlock_profiling.cc:243] Waited 3.84 ms on lock 0x7b240011b388. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14857b99c4 00007f14857b7acb 00007f1485972dc9 00007f1485972a55 00007f1485971db5 00007f14885e82a5 00007f14885e8f12 00007f14885f04c4 00007f14889dca86 00007f1488997a75
0709 12:14:02.816043 (+   419us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:02.816295 (+   252us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":8822144}
I20260709 12:14:02.786245 22110 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:02.819139 22110 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:02.820315 22110 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:02.843779 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:02.845417 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:02.847743 12676 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:02.851523 13334 raft_consensus.cc:2368] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22.
I20260709 12:14:02.851472 22131 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:14:02.852774 22131 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:14:02.853180 22131 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:02.855216 21815 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.006s
I20260709 12:14:02.855571 22131 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:02.859074 12526 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b65110412db1492aa47f4a845ec89242"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:14:02.873888 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:02.874246 12526 raft_consensus.cc:606] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:02.874895 12526 raft_consensus.cc:614] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:02.873795 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:02.878681 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:02.874069 12605 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:14:02.849736 22054 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:14:02.881026 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 97175us
I20260709 12:14:02.799094 22129 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:14:02.856491 22145 raft_consensus.cc:3055] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:14:02.897392 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:02.899039 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:02.899106 12605 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27. Ops: []
I20260709 12:14:02.903177 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:02.900666 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:02.850265 21640 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:02.909588 21640 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:02.911019 12605 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 29 index: 15. (index mismatch)
I20260709 12:14:02.912053 21640 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:02.912484 12605 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:14:02.903630 12572 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:02.913930 22200 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:14:02.915084 12604 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:14:02.909590 22119 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:02.908346 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:02.845695 22019 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:02.928071 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:02.916574 12572 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:02.930928 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:02.932260 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:02.906912 12679 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 31: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:02.933305 13330 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:14:02.918555 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:02.892421 21815 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:02.924615 13304 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:02.903589 13334 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:14:02.939052 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:02.920352 13338 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:02.905244 22145 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:02.934513 13331 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29.
I20260709 12:14:02.936215 22124 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:02.940323 12531 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:14:02.940953 22097 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:14:02.942255 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.937155 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:02.959885 12679 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:14:02.942550 22085 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:02.944700 12572 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:02.919450 22105 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:02.953853 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:02.929373 22131 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:02.952284 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:02.934957 22200 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:14:02.902083 22054 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:02.981328 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:02.954720 22192 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:14:02.991065 13302 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:02.994890 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:02.995671 12526 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:02.997593 12679 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:14:03.001190 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:02.998987 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:02.962708 22097 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:03.003540 12496 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:14:03.008373 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.016368 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:03.036868 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.018587 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:03.064934 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:03.060000 12526 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:03.065876 13331 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:14:03.068979 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:02.994690 22192 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:02.993372 22129 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:14:03.073839 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:02.952862 22019 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:02.956184 21815 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:03.080752 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:03.103530 13331 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:14:03.016072 22097 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:03.136843 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:03.137708 13331 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:14:03.019639 22105 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:14:03.077983 22192 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:14:03.155531 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.156441 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:14:03.142352 22208 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.135s	user 0.000s	sys 0.001s
I20260709 12:14:03.157227 13334 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:03.135541 22129 raft_consensus.cc:697] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:03.145007 22119 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:03.147773 22210 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:03.159000 22105 raft_consensus.cc:697] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:03.177058 21099 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.017445 22145 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:03.172031 22203 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.171s	user 0.000s	sys 0.001s
I20260709 12:14:03.182721 21099 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:03.181392 22127 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.181895 22169 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.185091 13330 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 30. Current term is 31. Ops: []
I20260709 12:14:03.175097 22210 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:03.199972 13336 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:03.200580 13338 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
W20260709 12:14:03.204396 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:03.220979 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:03.221789 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.225250 13331 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:14:03.229439 21099 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:03.230408 21099 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.231292 21099 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:14:03.174784 22119 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:03.194882 22054 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:03.236341 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 237461us
I20260709 12:14:03.244061 13302 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:03.226231 22111 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.225328 21640 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:03.225656 22210 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:03.228245 22212 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:14:03.250892 21099 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:03.212771 22127 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:03.253729 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.265484 12643 leader_election.cc:400] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22.
I20260709 12:14:03.271184 12643 leader_election.cc:403] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 21 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:14:03.271319 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.249080 22075 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.290724 13336 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:14:03.296689 22216 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 21. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 21. Current term is 22.
I20260709 12:14:03.297549 22216 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:03.375828 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.274443 22212 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 31 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:03.384789 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:03.298202 22198 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.286s	user 0.001s	sys 0.000s
W20260709 12:14:03.299599 22207 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.225s	user 0.001s	sys 0.000s
I20260709 12:14:03.382252 21815 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:03.294268 22075 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:03.401108 12679 raft_consensus.cc:2410] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 28 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 12:14:03.401219 22075 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:03.404933 22071 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.658s	user 0.008s	sys 0.003s
W20260709 12:14:03.299451 22209 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 12:14:03.406390 12675 raft_consensus.cc:2410] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 26 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
W20260709 12:14:03.408751 22204 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
I20260709 12:14:03.414017 13334 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:14:03.416709 22124 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:14:03.419694 22205 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.417s	user 0.003s	sys 0.001s
W20260709 12:14:03.373883 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 126845us
I20260709 12:14:03.377419 22220 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:03.285178 22105 consensus_queue.cc:237] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:03.420490 22097 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:03.429786 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 164449us
I20260709 12:14:03.437034 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:03.260214 22129 consensus_queue.cc:237] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 22.12, Last appended by leader: 12, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:03.444536 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:03.451969 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:03.434448 22220 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 30 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:03.454061 22229 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:03.456099 22071 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.457216 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:03.459173 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:03.460366 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:03.260676 22111 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:03.298092 22216 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:03.471464 12599 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:14:03.225198 22169 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:03.402240 22212 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:03.462716 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:03.472944 12604 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 29: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:03.463060 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:03.438203 22019 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:03.465070 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:03.469864 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4422) took 1385 ms. Trace:
I20260709 12:14:03.476117 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 21 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:03.474103 12600 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 31: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:14:03.454102 22218 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.064s	user 0.000s	sys 0.001s
I20260709 12:14:03.483242 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:03.483376 12605 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:03.477196 12671 rpcz_store.cc:276] 0709 12:14:02.084691 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.103304 (+ 18613us) service_pool.cc:224] Handling call
0709 12:14:02.119885 (+ 16581us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:02.128322 (+  8437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:02.128374 (+    52us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:02.135703 (+  7329us) log.cc:844] Serialized 52 byte log entry
0709 12:14:02.820040 (+684337us) spinlock_profiling.cc:243] Waited 21.3 ms on lock 0x7b18000e9320. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:14:02.900447 (+ 80407us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:14:03.133682 (+233235us) log.cc:844] Serialized 14 byte log entry
0709 12:14:03.467611 (+333929us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:03.467694 (+    83us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:03.467739 (+    45us) raft_consensus.cc:1685] finished
0709 12:14:03.469600 (+  1861us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:03.469793 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8894,"prepare.queue_time_us":278918,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":21724,"raft.run_cpu_time_us":302,"raft.run_wall_time_us":12200,"spinlock_wait_cycles":49063168,"thread_start_us":646462,"threads_started":3,"wal-append.queue_time_us":801127}
I20260709 12:14:03.552372 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:03.554343 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:03.555478 12599 raft_consensus.cc:2410] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 29 because replica has last-logged OpId of term: 28 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 14.
I20260709 12:14:03.556640 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:03.557407 12599 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
W20260709 12:14:03.563092 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 77148.1us
I20260709 12:14:03.564265 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:03.567911 12600 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:14:03.566802 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:03.569782 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:03.572202 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:03.573941 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:03.577174 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:03.579914 12601 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26. Ops: [25.13-25.13]
I20260709 12:14:03.489456 22075 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:14:03.585541 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:03.587800 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:03.594501 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.472079 22071 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:03.601845 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.620472 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:03.621693 12671 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 21.
W20260709 12:14:03.624517 22219 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.175s	user 0.000s	sys 0.001s
W20260709 12:14:03.642208 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.647125 22043 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:03.647653 22043 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:03.648037 22043 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:03.650672 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3724) took 2007 ms. Trace:
W20260709 12:14:03.656225 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.659606 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3729) took 1178 ms. Trace:
I20260709 12:14:03.660303 12675 rpcz_store.cc:276] 0709 12:14:02.481512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.673919 (+192407us) service_pool.cc:224] Handling call
0709 12:14:03.659280 (+985361us) spinlock_profiling.cc:243] Waited 714 ms on lock 0x7b540016f3d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:03.659513 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1642292480}
I20260709 12:14:03.664717 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:03.668892 13337 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38472 (request call id 1067) took 1523 ms (client timeout 1977 ms). Trace:
W20260709 12:14:03.669598 13337 rpcz_store.cc:269] 0709 12:14:02.145277 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.167533 (+ 22256us) service_pool.cc:224] Handling call
0709 12:14:03.668541 (+1501008us) spinlock_profiling.cc:243] Waited 326 ms on lock 0x7b54006f8454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:03.668755 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":749555456}
I20260709 12:14:03.651273 12672 rpcz_store.cc:276] 0709 12:14:01.642906 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.643064 (+   158us) service_pool.cc:224] Handling call
0709 12:14:01.643444 (+   380us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:02.173288 (+529844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:03.650249 (+1476961us) spinlock_profiling.cc:243] Waited 501 ms on lock 0x7b54000ac8d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:03.650570 (+   321us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1153050880}
I20260709 12:14:03.671689 21722 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:03.672219 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:03.673203 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.675009 22136 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:03.675612 22136 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:03.678423 22111 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:03.609588 22220 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:03.681905 12602 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:14:03.683800 22136 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:03.581957 22228 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:03.587652 22019 raft_consensus.cc:2764] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:14:03.697268 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:03.478077 22229 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:03.705943 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:03.707304 12672 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
W20260709 12:14:03.707549 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:03.709133 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:03.710631 12672 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 31: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:03.698657 22223 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:03.714646 22127 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:03.725073 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:03.726183 12672 raft_consensus.cc:2410] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 29 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 15.
I20260709 12:14:03.727182 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:03.727823 12672 raft_consensus.cc:2368] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 27. Current term is 28.
I20260709 12:14:03.614372 22119 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:03.734519 12603 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 12:14:03.736176 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:03.737072 12603 raft_consensus.cc:2393] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 26: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:03.738533 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:03.739562 12604 raft_consensus.cc:2410] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 29 because replica has last-logged OpId of term: 28 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 14.
I20260709 12:14:03.740586 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:03.741889 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:03.742245 22229 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:03.743276 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:03.743902 12600 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29.
I20260709 12:14:03.743981 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 35 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:03.738051 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:03.747038 12675 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 24: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:14:03.746095 13304 leader_election.cc:336] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.972s (SENT)
I20260709 12:14:03.748256 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:03.729674 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:03.749771 12602 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 29: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:03.738196 22223 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:03.746181 12603 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:03.752405 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:03.700142 22216 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:03.759508 12599 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 23. Current term is 24. Ops: []
I20260709 12:14:03.760999 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:03.761111 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:03.761812 12605 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 12:14:03.764861 12604 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:03.728610 22196 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:03.766024 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:03.732376 22193 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:03.775255 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1076) took 2234 ms. Trace:
I20260709 12:14:03.775884 12598 rpcz_store.cc:276] 0709 12:14:01.531415 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.531587 (+   172us) service_pool.cc:224] Handling call
0709 12:14:01.531960 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:01.685700 (+153740us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:03.765611 (+2079911us) spinlock_profiling.cc:243] Waited 153 ms on lock 0x7b5400231254. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:03.765869 (+   258us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":352225664}
I20260709 12:14:03.766148 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:03.786229 12672 raft_consensus.cc:2393] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 31: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:14:03.802783 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:03.803090 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:03.763386 22110 consensus_meta.cc:287] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.938s	user 0.013s	sys 0.004s
W20260709 12:14:03.765276 21984 consensus_meta.cc:287] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.977s	user 0.013s	sys 0.000s
I20260709 12:14:03.756579 22221 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:14:03.839133 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 251407us
I20260709 12:14:03.983575 12598 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 29 index: 15. (index mismatch)
I20260709 12:14:03.901702 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:04.027159 12643 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:03.901261 22097 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:04.028505 22216 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:14:03.768841 22174 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:03.808425 22196 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:03.845497 12605 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27. Ops: [26.16-26.16]
I20260709 12:14:03.852522 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4421) took 1772 ms. Trace:
I20260709 12:14:04.024580 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:04.025111 12604 raft_consensus.cc:3055] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:14:04.050199 12604 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:14:03.854250 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.082515 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:04.083374 12529 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:14:04.092516 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 65482.8us
I20260709 12:14:04.031620 12673 rpcz_store.cc:276] 0709 12:14:02.079695 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.083533 (+  3838us) service_pool.cc:224] Handling call
0709 12:14:02.083876 (+   343us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:02.084269 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:14:02.084543 (+   274us) log.cc:844] Serialized 14 byte log entry
0709 12:14:03.419956 (+1335413us) spinlock_profiling.cc:243] Waited 3.62 ms on lock 0x7b18003707e0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a
0709 12:14:03.420213 (+   257us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:03.420763 (+   550us) log.cc:844] Serialized 26 byte log entry
0709 12:14:03.757165 (+336402us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:03.757209 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:03.757282 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:03.841698 (+ 84416us) raft_consensus.cc:1685] finished
0709 12:14:03.852146 (+ 10448us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:03.852416 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":24695,"prepare.queue_time_us":5545,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":3496,"raft.queue_time_us":5994,"raft.run_cpu_time_us":458,"raft.run_wall_time_us":13159,"spinlock_wait_cycles":8322816,"thread_start_us":350409,"threads_started":2,"wal-append.queue_time_us":465841,"wal-append.run_cpu_time_us":2972,"wal-append.run_wall_time_us":1225480}
W20260709 12:14:04.094192 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 348067us
W20260709 12:14:04.094868 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.095429 12600 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 29 index: 15. (term mismatch)
I20260709 12:14:04.095908 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.025385 22223 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:04.099103 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:03.854202 22169 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:04.101114 22110 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:14:04.101557 22210 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.826s	user 0.011s	sys 0.001s
I20260709 12:14:04.101750 22110 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:04.103060 22110 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:04.104498 22192 consensus_meta.cc:287] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.936s	user 0.007s	sys 0.004s
I20260709 12:14:04.104974 22192 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:04.106842 22228 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:04.112867 22192 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:04.114956 12606 raft_consensus.cc:2368] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 23. Current term is 24.
I20260709 12:14:04.118170 12673 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:04.095883 12600 pending_rounds.cc:85] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 14
I20260709 12:14:04.098554 12604 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 21.14, Last appended by leader: 14, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:04.119555 12600 pending_rounds.cc:107] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 28.15
I20260709 12:14:04.119974 12600 raft_consensus.cc:2889] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:03.843406 21753 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.120030 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1093) took 1657 ms. Trace:
I20260709 12:14:04.121018 12673 rpcz_store.cc:276] 0709 12:14:02.462377 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.117546 (+1655169us) service_pool.cc:224] Handling call
0709 12:14:04.117936 (+   390us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.118821 (+   885us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.119970 (+  1149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.125319 12673 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 22. Current term is 23. Ops: []
W20260709 12:14:04.130074 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.126942 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1095) took 1663 ms. Trace:
I20260709 12:14:04.128939 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:04.133131 12673 rpcz_store.cc:276] 0709 12:14:02.463121 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.124632 (+1661511us) service_pool.cc:224] Handling call
0709 12:14:04.125037 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.126379 (+  1342us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.126820 (+   441us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:03.835741 22193 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:04.135799 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.127770 12529 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:14:04.136355 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:04.138425 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:04.118747 22221 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.146044 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:04.127594 22129 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.159534 22196 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:04.160950 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:04.162271 13337 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:04.164301 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:04.165163 22227 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.522s	user 0.010s	sys 0.004s
I20260709 12:14:04.145282 22221 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:04.176491 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:04.176951 22221 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:04.177438 12602 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 31.
I20260709 12:14:04.177639 22221 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.126268 22210 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:04.185570 22210 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:04.186391 13330 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:04.142570 22111 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:14:04.191655 12529 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: []
I20260709 12:14:04.140110 22233 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:14:04.141424 22228 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.052s
I20260709 12:14:04.134907 21753 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:04.192404 22229 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.210996 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:04.193027 22226 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:04.200525 22020 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:04.219421 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.201586 22111 raft_consensus.cc:697] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:04.161314 22129 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:04.210570 22233 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:04.207265 22216 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.205322 22075 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:04.224224 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:04.206354 22043 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.558s	user 0.010s	sys 0.003s
I20260709 12:14:04.207152 22144 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.195057 22221 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:04.212626 22119 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.249449 22119 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:04.249599 22111 consensus_queue.cc:237] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:04.252131 22119 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:04.236750 13304 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:04.214890 22229 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:04.253600 22054 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Leader pre-election won for term 22
I20260709 12:14:04.254076 22054 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:04.254189 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:04.254558 22054 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:14:04.255661 22229 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:04.256498 13304 leader_election.cc:400] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 27. Current term is 28.
W20260709 12:14:04.259291 22234 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.051s	user 0.000s	sys 0.000s
I20260709 12:14:04.259479 21640 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:04.259883 21640 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:04.260295 21640 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:04.234083 22020 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:04.262998 22193 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:04.262907 22233 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 24.18, Last appended by leader: 18, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:04.263764 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4427) took 1884 ms. Trace:
I20260709 12:14:04.233803 22019 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.224174 22226 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:04.265095 22019 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:04.241910 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3665) took 2561 ms. Trace:
I20260709 12:14:04.266167 12522 rpcz_store.cc:276] 0709 12:14:01.679035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:01.683371 (+  4336us) service_pool.cc:224] Handling call
0709 12:14:01.683701 (+   330us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:02.461789 (+778088us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.240257 (+1778468us) spinlock_profiling.cc:243] Waited 771 ms on lock 0x7b54003c1c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:04.240445 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1774079232}
I20260709 12:14:04.266862 22226 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:04.264308 12672 rpcz_store.cc:276] 0709 12:14:02.379174 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.808341 (+1429167us) service_pool.cc:224] Handling call
0709 12:14:03.841024 (+ 32683us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:04.025405 (+184381us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.025445 (+    40us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:04.034162 (+  8717us) log.cc:844] Serialized 26 byte log entry
0709 12:14:04.189660 (+155498us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:14:04.189704 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.189785 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:04.263483 (+ 73698us) raft_consensus.cc:1685] finished
0709 12:14:04.263507 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.263659 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":183718,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":122,"thread_start_us":62079,"threads_started":1,"wal-append.queue_time_us":175873}
I20260709 12:14:04.267053 22144 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:04.267583 22019 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:04.267808 13338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 1070) took 1003 ms. Trace:
I20260709 12:14:04.268461 12672 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:14:04.269261 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1101) took 1804 ms. Trace:
I20260709 12:14:04.268270 13338 rpcz_store.cc:276] 0709 12:14:03.264338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.266638 (+  2300us) service_pool.cc:224] Handling call
0709 12:14:03.267100 (+   462us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:03.267590 (+   490us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:14:03.536621 (+269031us) log.cc:844] Serialized 14 byte log entry
0709 12:14:03.834888 (+298267us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:04.025198 (+190310us) log.cc:844] Serialized 26 byte log entry
0709 12:14:04.025389 (+   191us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:14:04.025425 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.025502 (+    77us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:04.267568 (+242066us) raft_consensus.cc:1685] finished
0709 12:14:04.267591 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.267747 (+   156us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":19863,"prepare.queue_time_us":142380,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":32,"raft.queue_time_us":131601,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":16714,"thread_start_us":339370,"threads_started":2,"wal-append.queue_time_us":568383}
I20260709 12:14:04.269614 12672 rpcz_store.cc:276] 0709 12:14:02.464931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.267758 (+1802827us) service_pool.cc:224] Handling call
0709 12:14:04.268111 (+   353us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.269006 (+   895us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.269208 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.242828 22043 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:04.271117 12672 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:14:04.271834 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3737) took 1778 ms. Trace:
I20260709 12:14:04.272161 12672 rpcz_store.cc:276] 0709 12:14:02.493308 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.270439 (+1777131us) service_pool.cc:224] Handling call
0709 12:14:04.270795 (+   356us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.271561 (+   766us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.271779 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.272891 22043 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:04.273878 12677 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 30: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:04.239758 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1090) took 1888 ms. Trace:
I20260709 12:14:04.241870 22075 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:04.275517 12600 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:04.274201 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3738) took 1779 ms. Trace:
I20260709 12:14:04.278100 12672 rpcz_store.cc:276] 0709 12:14:02.494872 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.273019 (+1778147us) service_pool.cc:224] Handling call
0709 12:14:04.273362 (+   343us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.273588 (+   226us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.273623 (+    35us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:04.273678 (+    55us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:04.273719 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.273802 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.273929 (+   127us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.275617 12671 rpcz_store.cc:276] 0709 12:14:02.351731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.758986 (+1407255us) service_pool.cc:224] Handling call
0709 12:14:03.781404 (+ 22418us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:03.834798 (+ 53394us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:14:04.123865 (+289067us) log.cc:844] Serialized 14 byte log entry
0709 12:14:04.239348 (+115483us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:04.239424 (+    76us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:04.239488 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.239565 (+    77us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.239697 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":325481,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":13515,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":109,"thread_start_us":336465,"threads_started":2}
I20260709 12:14:04.228226 22071 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:04.283465 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.241235 22242 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:04.242587 22216 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:04.264529 22020 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:04.277709 22144 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:04.303828 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3741) took 1755 ms. Trace:
I20260709 12:14:04.304607 22112 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:04.305220 22112 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:04.305894 12597 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 30. Current term is 31. Ops: []
I20260709 12:14:04.306733 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4205) took 1585 ms. Trace:
I20260709 12:14:04.307562 22075 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:04.307137 12597 rpcz_store.cc:276] 0709 12:14:02.721360 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.305202 (+1583842us) service_pool.cc:224] Handling call
0709 12:14:04.305638 (+   436us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.306414 (+   776us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.306654 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.304399 12679 rpcz_store.cc:276] 0709 12:14:02.548443 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.000918 (+452475us) service_pool.cc:224] Handling call
0709 12:14:04.303501 (+1302583us) spinlock_profiling.cc:243] Waited 396 ms on lock 0x7b54001414d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:04.303715 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":911032192}
I20260709 12:14:04.309482 12679 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:14:04.307571 22112 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:04.310060 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1108) took 1783 ms. Trace:
I20260709 12:14:04.311743 12679 rpcz_store.cc:276] 0709 12:14:02.526585 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.308796 (+1782211us) service_pool.cc:224] Handling call
0709 12:14:04.309145 (+   349us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.309826 (+   681us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.310003 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.292403 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3739) took 1790 ms. Trace:
I20260709 12:14:04.294561 22129 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:04.295728 12604 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
W20260709 12:14:04.297464 12680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 4420) took 2218 ms (client timeout 1812 ms). Trace:
W20260709 12:14:04.315582 12680 rpcz_store.cc:269] 0709 12:14:02.079417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.079564 (+   147us) service_pool.cc:224] Handling call
0709 12:14:04.297174 (+2217610us) spinlock_profiling.cc:243] Waited 423 ms on lock 0x7b540015e0d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:04.297326 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":972515712}
I20260709 12:14:04.314710 12671 rpcz_store.cc:276] 0709 12:14:02.499217 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.281374 (+1782157us) service_pool.cc:224] Handling call
0709 12:14:04.287361 (+  5987us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.287570 (+   209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.288800 (+  1230us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:04.288897 (+    97us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:14:04.288968 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.289061 (+    93us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.289264 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.297796 22221 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:04.298595 12678 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 33. Current term is 34. Ops: [32.21-33.22]
W20260709 12:14:04.316696 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:04.317292 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.324565 12680 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 29 index: 15. (term mismatch)
I20260709 12:14:04.325002 12680 pending_rounds.cc:85] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 14
I20260709 12:14:04.325238 12680 pending_rounds.cc:107] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 28.15
I20260709 12:14:04.325477 12680 raft_consensus.cc:2889] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:04.336519 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4428) took 1783 ms. Trace:
I20260709 12:14:04.341868 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4429) took 1786 ms. Trace:
I20260709 12:14:04.342787 12679 rpcz_store.cc:276] 0709 12:14:02.553086 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.312645 (+1759559us) service_pool.cc:224] Handling call
0709 12:14:04.335743 (+ 23098us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.336030 (+   287us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.336070 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:04.336134 (+    64us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:14:04.336195 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.336283 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.336435 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.342885 12672 rpcz_store.cc:276] 0709 12:14:02.555057 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.315978 (+1760921us) service_pool.cc:224] Handling call
0709 12:14:04.341313 (+ 25335us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.341531 (+   218us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.341555 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:04.341600 (+    45us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:04.341645 (+    45us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.341699 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.341815 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.341796 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4431) took 1786 ms. Trace:
I20260709 12:14:04.356989 12680 rpcz_store.cc:276] 0709 12:14:02.555715 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.318184 (+1762469us) service_pool.cc:224] Handling call
0709 12:14:04.324122 (+  5938us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.325972 (+  1850us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.341723 (+ 15751us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.358958 12604 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:14:04.361635 12531 raft_consensus.cc:606] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:04.362378 12531 raft_consensus.cc:614] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:04.364213 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:04.375389 22075 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:04.376994 12679 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:04.377856 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4432) took 1771 ms. Trace:
I20260709 12:14:04.378387 12679 rpcz_store.cc:276] 0709 12:14:02.606502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.364061 (+1757559us) service_pool.cc:224] Handling call
0709 12:14:04.376771 (+ 12710us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.377540 (+   769us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.377789 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.381325 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4433) took 1774 ms. Trace:
I20260709 12:14:04.381763 12679 rpcz_store.cc:276] 0709 12:14:02.607007 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.380258 (+1773251us) service_pool.cc:224] Handling call
0709 12:14:04.380671 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.380918 (+   247us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.380950 (+    32us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:04.381001 (+    51us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:04.381047 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.381109 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.381261 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.387306 12679 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: []
I20260709 12:14:04.388012 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3743) took 1734 ms. Trace:
I20260709 12:14:04.388346 12679 rpcz_store.cc:276] 0709 12:14:02.653205 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.386712 (+1733507us) service_pool.cc:224] Handling call
0709 12:14:04.387091 (+   379us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.387769 (+   678us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.387955 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.389992 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1089) took 2073 ms. Trace:
I20260709 12:14:04.391286 12678 rpcz_store.cc:276] 0709 12:14:02.333569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.337510 (+  3941us) service_pool.cc:224] Handling call
0709 12:14:02.337995 (+   485us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:04.316963 (+1978968us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.389457 (+ 72494us) spinlock_profiling.cc:243] Waited 1.96 s on lock 0x7b5400092ed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:04.389878 (+   421us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4508253056}
I20260709 12:14:04.402715 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3744) took 1720 ms. Trace:
I20260709 12:14:04.403259 12679 rpcz_store.cc:276] 0709 12:14:02.682469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.389087 (+1706618us) service_pool.cc:224] Handling call
0709 12:14:04.402051 (+ 12964us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.402309 (+   258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.402346 (+    37us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:04.402398 (+    52us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:14:04.402440 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.402503 (+    63us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.402644 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.404582 12678 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:14:04.405310 22247 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:04.405319 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4434) took 1721 ms. Trace:
I20260709 12:14:04.406013 22247 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:04.406284 12678 rpcz_store.cc:276] 0709 12:14:02.684125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.394958 (+1710833us) service_pool.cc:224] Handling call
0709 12:14:04.399718 (+  4760us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.405056 (+  5338us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.405266 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.408696 12678 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:14:04.412636 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:04.405496 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3745) took 1559 ms. Trace:
I20260709 12:14:04.409466 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4436) took 1555 ms. Trace:
I20260709 12:14:04.416280 12678 rpcz_store.cc:276] 0709 12:14:02.854572 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.407852 (+1553280us) service_pool.cc:224] Handling call
0709 12:14:04.408357 (+   505us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.409199 (+   842us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.409407 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.415552 12679 rpcz_store.cc:276] 0709 12:14:02.846101 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.404470 (+1558369us) service_pool.cc:224] Handling call
0709 12:14:04.404919 (+   449us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.405136 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.405167 (+    31us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:04.405217 (+    50us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:04.405266 (+    49us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.405331 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.405442 (+   111us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.418926 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4438) took 1351 ms. Trace:
I20260709 12:14:04.423861 12678 rpcz_store.cc:276] 0709 12:14:03.067797 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.417325 (+1349528us) service_pool.cc:224] Handling call
0709 12:14:04.417634 (+   309us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.417928 (+   294us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.417957 (+    29us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:04.418000 (+    43us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:04.418032 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.418088 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.418229 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.323099 22242 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.098s
I20260709 12:14:04.408731 22247 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:04.425073 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4439) took 1212 ms. Trace:
I20260709 12:14:04.418620 22088 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:04.430138 12679 rpcz_store.cc:276] 0709 12:14:03.212341 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.418040 (+1205699us) service_pool.cc:224] Handling call
0709 12:14:04.424559 (+  6519us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.424751 (+   192us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.424776 (+    25us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:04.424819 (+    43us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:14:04.424856 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.424907 (+    51us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.425014 (+   107us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.427990 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4440) took 1202 ms. Trace:
I20260709 12:14:04.435961 12679 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:14:04.436754 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4443) took 1002 ms. Trace:
I20260709 12:14:04.437433 22054 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:04.434032 12678 rpcz_store.cc:276] 0709 12:14:03.225817 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.427079 (+1201262us) service_pool.cc:224] Handling call
0709 12:14:04.427377 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.427553 (+   176us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:04.427607 (+    54us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:04.427653 (+    46us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:14:04.427785 (+   132us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.427839 (+    54us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:04.427933 (+    94us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.437134 12679 rpcz_store.cc:276] 0709 12:14:03.434232 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.435172 (+1000940us) service_pool.cc:224] Handling call
0709 12:14:04.435553 (+   381us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.436459 (+   906us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.436681 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.444595 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:04.444736 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:04.446311 12679 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:14:04.432529 22216 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:04.455277 21753 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:04.445427 22118 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.458711 22118 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:04.445678 12678 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:04.462169 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:04.462850 12496 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:04.457486 13331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 1115) took 1294 ms. Trace:
I20260709 12:14:04.463713 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:04.463642 13331 rpcz_store.cc:276] 0709 12:14:03.163245 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.163518 (+   273us) service_pool.cc:224] Handling call
0709 12:14:03.223085 (+ 59567us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:03.225925 (+  2840us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.457274 (+1231349us) spinlock_profiling.cc:243] Waited 59 ms on lock 0x7b540061e9d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:04.457423 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":135727360}
I20260709 12:14:04.464522 12679 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:14:04.469216 22118 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:04.456617 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:04.470356 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.471257 12676 raft_consensus.cc:2410] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 28 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 15.
I20260709 12:14:04.471872 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 4435) took 1733 ms. Trace:
I20260709 12:14:04.471975 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.472306 12676 rpcz_store.cc:276] 0709 12:14:02.738241 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.456334 (+1718093us) service_pool.cc:224] Handling call
0709 12:14:04.471799 (+ 15465us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.473456 12523 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
W20260709 12:14:04.459663 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 47155.2us
I20260709 12:14:04.469331 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:04.477692 12522 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:04.457329 21640 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:04.483932 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.483556 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:04.496526 12643 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:04.498517 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.499820 12530 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:14:04.501137 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:04.503657 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:04.506249 12525 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:14:04.512938 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50170.2us
I20260709 12:14:04.519292 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:04.520090 13331 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
W20260709 12:14:04.520905 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:04.522203 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 91194.4us
I20260709 12:14:04.524185 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:14:04.521523 12569 connection.cc:256] RPC call timeout handler was delayed by -1.0531799279999632s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1744s for a response
W20260709 12:14:04.526538 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.537097 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 21 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.536324 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:04.542291 13338 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 20 index: 13. Preceding OpId from leader: term: 22 index: 14. (index mismatch)
I20260709 12:14:04.523342 22111 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.520244 22088 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:04.544214 22233 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:04.538566 22196 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.028s
I20260709 12:14:04.538959 22129 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.524907 21753 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:04.567668 21753 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:04.567507 22129 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:04.568250 22196 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
W20260709 12:14:04.563333 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.568697 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:04.571568 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:04.571513 22129 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:04.572274 22088 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:04.573238 22216 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:04.573653 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:04.573678 22216 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:04.574541 22216 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:04.576778 12569 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:04.579001 22172 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:14:04.579464 22172 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:04.579854 22112 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:04.580276 22112 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:04.580668 22112 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:04.589112 12522 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 30.
I20260709 12:14:04.590588 12523 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
W20260709 12:14:04.590895 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47020.8us
I20260709 12:14:04.569396 21753 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:04.593832 13330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38484 (request call id 1118) took 1348 ms. Trace:
I20260709 12:14:04.596417 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:04.596637 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:04.597318 12678 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:04.598320 12529 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 29 index: 15. (index mismatch)
I20260709 12:14:04.600327 12679 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:14:04.594421 13330 rpcz_store.cc:276] 0709 12:14:03.244964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.245127 (+   163us) service_pool.cc:224] Handling call
0709 12:14:03.260432 (+ 15305us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.188823 (+928391us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.593548 (+404725us) spinlock_profiling.cc:243] Waited 926 ms on lock 0x7b5400469954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:04.593746 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2129040256}
I20260709 12:14:04.587138 22131 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:04.599004 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:04.603902 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:04.605029 12676 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 34 index: 22. (index mismatch)
I20260709 12:14:04.606308 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:04.607216 12676 raft_consensus.cc:2410] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 27 because replica has last-logged OpId of term: 25 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 12.
I20260709 12:14:04.607787 12679 raft_consensus.cc:2410] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 29 because replica has last-logged OpId of term: 28 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 13.
I20260709 12:14:04.610682 12522 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 22 index: 15. (index mismatch)
I20260709 12:14:04.612324 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:04.615199 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.617007 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:04.618930 22102 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.974s	user 0.009s	sys 0.018s
W20260709 12:14:04.619400 22102 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.975s	user 0.009s	sys 0.018s
I20260709 12:14:04.623534 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:04.624626 12600 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:04.626576 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:04.628013 12601 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 24: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:04.628707 21640 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:04.635371 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.648363 21640 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:14:04.648573 22120 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:14:04.655799 22054 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:04.667321 22129 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.668203 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 35 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:04.668248 22129 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:04.672643 13336 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 21. Current term is 22. Ops: []
I20260709 12:14:04.671968 22129 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:04.667651 21935 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.674384 22120 consensus_queue.cc:1059] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:14:04.674651 21935 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:14:04.677771 22129 raft_consensus.cc:3055] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Stepping down as leader of term 21
I20260709 12:14:04.678833 21935 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:14:04.678473 22129 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:04.681624 22054 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.681702 22129 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 21, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:04.566913 22111 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:04.695362 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 1105) took 2229 ms. Trace:
I20260709 12:14:04.696249 12671 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 29. Current term is 30. Ops: []
W20260709 12:14:04.706044 12675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 1110) took 1545 ms (client timeout 2038 ms). Trace:
W20260709 12:14:04.710011 12675 rpcz_store.cc:269] 0709 12:14:03.161052 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.661785 (+500733us) service_pool.cc:224] Handling call
0709 12:14:04.705797 (+1044012us) spinlock_profiling.cc:243] Waited 33.2 ms on lock 0x7b54003d45d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:04.705981 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":76317440}
I20260709 12:14:04.705888 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.690526 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:14:04.707448 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 90483.1us
I20260709 12:14:04.713035 12527 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:14:04.706447 12673 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 22. Current term is 24. Ops: []
I20260709 12:14:04.712107 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:04.716550 12675 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:04.728102 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:04.730885 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:04.731457 12679 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 24: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:04.708712 12677 rpcz_store.cc:276] 0709 12:14:02.483619 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:03.664313 (+1180694us) service_pool.cc:224] Handling call
0709 12:14:04.694959 (+1030646us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b54003d4fd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:04.695251 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1398297344}
I20260709 12:14:04.724565 22081 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
I20260709 12:14:04.735404 22233 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:04.684906 22129 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 21 FOLLOWER]: Advancing to term 22
I20260709 12:14:04.736330 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:04.737905 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:04.738107 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.739686 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.741322 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:04.728569 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:04.743978 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.745290 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:04.580276 22172 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:04.611637 22088 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:14:04.743479 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48476 (request call id 95) took 1782 ms. Trace:
W20260709 12:14:04.746819 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 129966us
I20260709 12:14:04.751180 12531 rpcz_store.cc:276] 0709 12:14:02.928495 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:02.928714 (+   219us) service_pool.cc:224] Handling call
0709 12:14:04.710665 (+1781951us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54000eef54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1a853 00007f148d9e860a 00007f1486f6e7d4 00007f1486f6e762 00007f1486f6e691 00007f1486f6e60c 00007f1486f6d8c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:04.711077 (+   412us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3230681984}
I20260709 12:14:04.752735 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:04.754541 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:04.756786 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:04.757706 13337 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:14:04.758575 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:04.758777 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.760547 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:04.761746 12494 connection.cc:256] RPC call timeout handler was delayed by -1.2565960950000772s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15380000000000002s for a response
W20260709 12:14:04.763326 12494 connection.cc:256] RPC call timeout handler was delayed by -1.2565960950000772s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15380000000000002s for a response
W20260709 12:14:04.764467 12494 connection.cc:256] RPC call timeout handler was delayed by -1.2196960950000175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1579s for a response
I20260709 12:14:04.765928 12522 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:14:04.766395 22233 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:14:04.766621 13334 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:14:04.767074 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.767491 22247 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:04.768043 22247 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:04.768344 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.769665 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.770509 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:04.771376 12674 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 30: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:14:04.771821 12674 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 1094) took 2309 ms (client timeout 2972 ms). Trace:
W20260709 12:14:04.772193 12674 rpcz_store.cc:269] 0709 12:14:02.462790 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.724835 (+2262045us) service_pool.cc:224] Handling call
0709 12:14:04.771758 (+ 46923us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:04.739399 22118 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.035s
I20260709 12:14:04.775621 22105 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.778908 12496 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:04.780326 12496 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:14:04.784235 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:04.786584 12494 connection.cc:256] RPC call timeout handler was delayed by -1.2196960950000175s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1579s for a response
W20260709 12:14:04.788359 12494 connection.cc:256] RPC call timeout handler was delayed by -1.2151960950000102s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1584s for a response
W20260709 12:14:04.788911 12494 connection.cc:256] RPC call timeout handler was delayed by -1.1989960949999841s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1602s for a response
W20260709 12:14:04.791673 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:04.793259 12494 connection.cc:256] RPC call timeout handler was delayed by -1.1755960949999462s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1628s for a response
I20260709 12:14:04.794521 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:04.794946 12494 connection.cc:256] RPC call timeout handler was delayed by -1.0099960950001332s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18120000000000003s for a response
I20260709 12:14:04.797597 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.707558 22131 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:04.802620 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.776285 22105 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:04.804298 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.826382 12527 raft_consensus.cc:2393] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 28: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:04.803866 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.807260 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1097) took 2343 ms. Trace:
I20260709 12:14:04.807637 12606 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:14:04.807322 12528 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 29 index: 15. (index mismatch)
I20260709 12:14:04.827209 12524 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
I20260709 12:14:04.806386 12522 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:14:04.827133 12673 rpcz_store.cc:276] 0709 12:14:02.463697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.134816 (+1671119us) service_pool.cc:224] Handling call
0709 12:14:04.705992 (+571176us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.715265 (+  9273us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:04.807025 (+ 91760us) spinlock_profiling.cc:243] Waited 571 ms on lock 0x7b54003d45d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:04.807193 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1312536832}
W20260709 12:14:04.832607 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.810415 12600 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:04.777899 22145 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:04.821487 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.847585 12530 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:14:04.822414 12672 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:14:04.823100 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:04.823716 12601 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 25. Current term is 26. Ops: []
I20260709 12:14:04.800501 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:04.850291 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 47230.6us
W20260709 12:14:04.851094 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 92589.5us
I20260709 12:14:04.821133 22169 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.033s
I20260709 12:14:04.907764 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:04.853834 22255 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.237s	user 0.000s	sys 0.000s
I20260709 12:14:04.857969 22088 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:14:04.774009 22216 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:04.932425 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:04.932242 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:04.937166 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:04.801808 22247 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:04.684038 22054 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:04.846763 21713 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:04.867633 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:14:04.898854 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:04.899524 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:04.904624 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 90687.7us
I20260709 12:14:04.865610 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:04.926908 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.946897 12528 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:14:04.927852 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:04.939469 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:04.949174 12605 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:14:04.939705 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:14:04.929854 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 78609.6us
I20260709 12:14:04.950541 12606 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:04.951689 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:04.867020 22118 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:14:04.955658 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 90150.3us
I20260709 12:14:04.956518 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:04.953608 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.957736 12531 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:14:04.958824 12522 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:14:04.959493 12522 raft_consensus.cc:606] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:04.960017 12522 raft_consensus.cc:614] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:04.917918 12678 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:14:04.963078 12494 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:04.922689 22172 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:04.965106 12571 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:04.965335 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.938619 22233 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:04.973690 12494 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:04.906975 22175 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.057s
I20260709 12:14:04.952369 22141 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:04.988384 22141 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:04.988905 22141 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:04.987602 22223 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:14:04.991048 12531 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 20 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:14:04.940532 22129 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:04.994771 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:04.994560 22129 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:04.995855 12527 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:04.847275 22131 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:04.997850 13305 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:04.997735 22210 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:14:04.998528 22216 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:04.999536 22131 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:04.957796 22081 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.042s
I20260709 12:14:04.955071 22118 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:05.000020 22210 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:04.964567 22087 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:14:05.000415 22118 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:05.000149 22111 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:05.000455 22210 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:05.001771 22260 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:05.002142 22210 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:05.004117 12494 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:04.969066 22074 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:05.011592 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4430) took 2456 ms. Trace:
I20260709 12:14:05.012023 12671 rpcz_store.cc:276] 0709 12:14:02.555417 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.317109 (+1761692us) service_pool.cc:224] Handling call
0709 12:14:04.695834 (+378725us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:04.729282 (+ 33448us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.011319 (+282037us) spinlock_profiling.cc:243] Waited 336 ms on lock 0x7b54003d4fd0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:05.011491 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":773540224}
I20260709 12:14:05.012192 12494 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:05.014844 22267 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.087s	user 0.000s	sys 0.000s
I20260709 12:14:05.017804 22220 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:14:05.018497 22220 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:04.978549 22200 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:05.018811 22196 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:05.018832 22168 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:14:05.020123 22168 raft_consensus.cc:697] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:05.021224 22168 consensus_queue.cc:237] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 21.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:05.008231 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:04.969332 22259 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:05.030773 22107 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.031508 22107 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:05.034365 22196 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:05.036635 21935 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.036768 22171 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:14:04.952008 21713 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:05.040364 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 75337us
I20260709 12:14:05.041483 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:05.042357 12522 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:05.009312 22141 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:14:05.037328 21935 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:05.047348 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:05.047969 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 78952.5us
I20260709 12:14:05.051774 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:05.045958 12606 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 28.
I20260709 12:14:05.062525 12530 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:14:05.030815 22259 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:05.077203 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:05.046698 21713 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:05.049723 22111 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:14:05.078394 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:05.084748 12643 leader_election.cc:400] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 23. Current term is 24.
I20260709 12:14:05.085191 12643 leader_election.cc:403] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:05.088091 22269 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.088845 22269 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:05.089339 21935 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:05.090652 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:05.091487 22269 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:05.093626 22131 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:14:05.095163 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:05.097401 22141 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:14:05.078454 21815 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:14:05.098738 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260709 12:14:05.101263 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:14:05.102370 22020 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:05.102981 22020 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:05.103500 22020 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:05.059425 22120 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:05.056021 22260 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.051s
I20260709 12:14:05.105991 22120 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:05.113400 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:05.046624 22088 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 12:14:05.123754 21935 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:05.124065 22107 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:05.124620 21935 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:05.125301 12423 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47524 (request call id 104) took 26797 ms (client timeout 15000 ms). Trace:
I20260709 12:14:05.082213 22259 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:05.134899 22120 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:05.137759 22105 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:05.138427 22105 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.138929 22105 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:05.139732 22129 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:05.141135 22105 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:05.142752 21935 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:05.151037 12597 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22.
I20260709 12:14:05.152267 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:05.040688 22210 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:05.058344 22233 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 23.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.090s
W20260709 12:14:05.125886 12423 rpcz_store.cc:269] 0709 12:13:38.327669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:48.687659 (+10359990us) service_pool.cc:224] Handling call
0709 12:13:50.166178 (+1478519us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b54000104d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148898a090 00007f1488a28f3a 00007f148e7dee9c 00007f148e8dd04e 00007f148cd9e2a4 00007f148cd9e232 00007f148cd9e161 00007f148cd9e0dc 00007f148cd9d392 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:05.125103 (+14958925us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:00.594392 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:00.696163 (+101771us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:00.696226 (+    63us) write_op.cc:435] Acquired schema lock
0709 12:14:00.696248 (+    22us) tablet.cc:662] Decoding operations
0709 12:14:00.697484 (+  1236us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:00.697612 (+   128us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:00.697641 (+    29us) tablet.cc:685] Acquiring locks for 11 operations
0709 12:14:00.767526 (+ 69885us) tablet.cc:701] Row locks acquired
0709 12:14:00.767585 (+    59us) write_op.cc:260] PREPARE: finished
0709 12:14:00.767809 (+   224us) write_op.cc:270] Start()
0709 12:14:00.768106 (+   297us) write_op.cc:276] Timestamp: P: 1783599240767735 usec, L: 0
0709 12:14:00.768164 (+    58us) op_driver.cc:348] REPLICATION: starting
0709 12:14:00.769178 (+  1014us) log.cc:844] Serialized 4367 byte log entry
0709 12:14:01.582295 (+813117us) op_driver.cc:464] REPLICATION: finished
0709 12:14:01.774293 (+191998us) write_op.cc:301] APPLY: starting
0709 12:14:01.785904 (+ 11611us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:01.980893 (+194989us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:01.981072 (+   179us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:03.728759 (+1747687us) spinlock_profiling.cc:243] Waited 6.45 ms on lock 0x7b180045f8e0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f148875f0ec 00007f1488761af7 00007f1488762fb0 00007f148c4771bb 00007f148c4be5d4 00007f148c36a8da 00007f148c36d3cb 00007f148c36cfd3 00007f148c46be34
0709 12:14:03.901838 (+173079us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:03.943846 (+ 42008us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=22,key_file_lookups=22,delta_file_lookups=0,mrs_lookups=0
0709 12:14:03.943919 (+    73us) write_op.cc:312] APPLY: finished
0709 12:14:03.944452 (+   533us) log.cc:844] Serialized 105 byte log entry
0709 12:14:04.124458 (+180006us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:04.124648 (+   190us) write_op.cc:454] Released schema lock
0709 12:14:04.124726 (+    78us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:04.158971 (+ 34245us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:04.159022 (+    51us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1482590,"reported_tablets":21,"spinlock_wait_cycles":65066368,"child_traces":[["op",{"apply.queue_time_us":160517,"apply.run_cpu_time_us":53207,"apply.run_wall_time_us":2405592,"cfile_cache_hit":46,"cfile_cache_hit_bytes":50041,"num_ops":11,"prepare.queue_time_us":351588,"prepare.run_cpu_time_us":25278,"prepare.run_wall_time_us":341781,"raft.queue_time_us":203311,"raft.run_cpu_time_us":40157,"raft.run_wall_time_us":202070,"replication_time_us":784947,"spinlock_wait_cycles":14827904,"thread_start_us":695159,"threads_started":5,"wal-append.queue_time_us":358322,"wal-append.run_cpu_time_us":15820,"wal-append.run_wall_time_us":1527085}]]}
I20260709 12:14:05.068205 22216 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.093s
I20260709 12:14:05.093773 22269 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:14:05.163460 22110 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s	user 0.012s	sys 0.043s
W20260709 12:14:05.213107 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 128424us
I20260709 12:14:05.218169 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:05.019541 22220 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:05.073642 22174 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.113s
I20260709 12:14:05.170333 22088 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 23. Current term is 24.
I20260709 12:14:05.083616 22144 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.106s
W20260709 12:14:05.231411 22110 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.651s	user 0.012s	sys 0.043s
W20260709 12:14:05.232390 22102 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.610s	user 0.065s	sys 0.126s
I20260709 12:14:05.252249 22131 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:05.260447 22102 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.640s	user 0.066s	sys 0.126s
I20260709 12:14:05.256755 22220 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:05.358036 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:05.358781 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:05.310312 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:05.359221 12605 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
W20260709 12:14:05.360328 12496 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.001s	sys 0.024s
W20260709 12:14:05.360710 12496 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.575s	user 0.001s	sys 0.024s
W20260709 12:14:05.361008 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 580752us
I20260709 12:14:05.357761 12523 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
W20260709 12:14:05.316339 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 336244us
I20260709 12:14:05.319653 12678 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 27. Current term is 28. Ops: []
W20260709 12:14:05.320518 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 321146us
I20260709 12:14:05.363634 13338 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 25 index: 19. (index mismatch)
I20260709 12:14:05.363809 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:05.364862 12603 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 27: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:05.326542 12643 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:05.329519 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:05.334494 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:05.310895 22074 raft_consensus.cc:3055] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:14:05.333752 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:05.338384 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:05.252130 22221 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.047s
W20260709 12:14:05.332512 22112 consensus_meta.cc:287] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.741s	user 0.007s	sys 0.006s
W20260709 12:14:05.337277 21753 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.730s	user 0.011s	sys 0.002s
W20260709 12:14:05.346758 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 299512us
I20260709 12:14:05.386507 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:05.387560 12602 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:14:05.357668 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:05.388767 12531 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:05.388314 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:05.367305 12606 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:14:05.361951 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:05.255143 21815 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:05.391281 12571 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:05.379052 12601 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:14:05.392875 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:05.393944 12604 raft_consensus.cc:2410] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 29 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 15.
I20260709 12:14:05.394016 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:05.394984 12601 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:05.378829 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:05.370074 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:05.397034 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:05.370541 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:05.400254 12597 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 30.
I20260709 12:14:05.331941 22212 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.034s
I20260709 12:14:05.375809 12679 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:14:05.386963 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:05.408250 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:05.408142 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46254us
I20260709 12:14:05.386122 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:05.369138 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:05.371853 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:05.410799 12599 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:14:05.392097 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:05.379586 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:05.371368 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:05.410787 13337 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:05.412125 12530 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:14:05.402431 12600 raft_consensus.cc:2393] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 23: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:05.404903 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:05.397743 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:05.420905 12605 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 30: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:05.422897 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:05.442008 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:05.443584 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:05.426987 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:05.447248 12600 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:05.374886 22074 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:14:05.449189 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 410025us
I20260709 12:14:05.452533 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:05.456092 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:05.457623 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:05.459522 12600 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:05.464390 12494 leader_election.cc:400] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29.
I20260709 12:14:05.392488 21815 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:05.466957 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:05.414777 22120 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:14:05.442106 22266 log.cc:927] Time spent T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.061s	user 0.001s	sys 0.000s
I20260709 12:14:05.476912 12523 raft_consensus.cc:2393] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 26: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:05.473162 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:05.453073 22261 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.249s	user 0.001s	sys 0.000s
I20260709 12:14:05.477444 12601 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
W20260709 12:14:05.478655 22264 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.173s	user 0.001s	sys 0.000s
I20260709 12:14:05.454792 22102 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 29.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.029s
I20260709 12:14:05.422909 22269 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:05.422199 22107 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:05.369632 22131 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:05.479944 21815 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:05.491300 13337 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 30.
W20260709 12:14:05.496034 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 169579us
I20260709 12:14:05.392360 22110 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:14:05.441636 22141 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.037s
I20260709 12:14:05.391965 22112 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:05.498908 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:05.400132 21753 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:05.403088 22247 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:05.424350 22280 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:05.505661 22081 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:14:05.512341 22269 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:05.513733 12597 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:14:05.514258 12600 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 25 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:14:05.399058 22200 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:05.482878 12599 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:14:05.513708 22107 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:05.520756 12531 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
W20260709 12:14:05.518770 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 154812us
W20260709 12:14:05.548272 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:05.425475 22118 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:05.524324 12604 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
W20260709 12:14:05.551687 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 93845.7us
W20260709 12:14:05.552593 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 120108us
I20260709 12:14:05.499085 22210 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:05.527508 12671 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22.
W20260709 12:14:05.510823 22270 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.208s	user 0.001s	sys 0.000s
I20260709 12:14:05.531353 12679 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 15. (index mismatch)
I20260709 12:14:05.493254 22278 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:05.568419 12675 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:14:05.533636 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:05.416639 22129 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 12:14:05.558813 12601 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:14:05.524228 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:05.572924 12526 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 26: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:05.499554 21935 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.033s
W20260709 12:14:05.585877 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:05.531977 22280 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:05.587006 13337 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: []
I20260709 12:14:05.588085 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:05.589452 12680 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 29. Current term is 30. Ops: [29.15-29.15]
I20260709 12:14:05.589867 12524 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: []
I20260709 12:14:05.593295 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:14:05.598883 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 233983us
W20260709 12:14:05.567291 22263 log.cc:927] Time spent T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.187s	user 0.000s	sys 0.001s
I20260709 12:14:05.559594 22107 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:05.569718 22278 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:05.573390 22081 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:05.547206 22102 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 12:14:05.559470 22273 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.072s
I20260709 12:14:05.668584 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:05.607609 12646 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:05.590158 22283 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:14:05.632679 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:05.561928 22168 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.070s
I20260709 12:14:05.558235 22274 consensus_queue.cc:1059] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.116s
I20260709 12:14:05.594319 22280 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:05.737390 12680 raft_consensus.cc:2410] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 29 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 15.
W20260709 12:14:05.590670 22271 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.264s	user 0.001s	sys 0.001s
I20260709 12:14:05.778437 12606 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 29. Current term is 30. Ops: [29.15-29.15]
I20260709 12:14:05.732366 22283 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:05.786257 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 178759us
I20260709 12:14:05.740963 22168 consensus_queue.cc:1243] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 27.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:14:05.645045 22285 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.136s
I20260709 12:14:05.765410 22273 consensus_queue.cc:1243] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 25.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:14:05.762027 22141 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:05.696770 22289 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:05.774660 22087 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.003s	sys 0.032s
I20260709 12:14:05.677695 22278 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:05.664767 22269 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:05.789642 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 121145us
I20260709 12:14:05.848771 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:05.837404 12497 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.551s	user 0.012s	sys 0.005s
W20260709 12:14:05.849611 12497 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.793s	user 0.013s	sys 0.005s
I20260709 12:14:05.852566 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:05.855053 12527 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:14:05.855597 12527 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:05.856648 12527 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.22, Last appended by leader: 22, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:05.774780 22281 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.066s
I20260709 12:14:05.795851 22129 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:05.866369 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:05.867779 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:05.875698 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:05.857949 12527 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:14:05.870209 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:05.769447 22120 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.069s
W20260709 12:14:05.844055 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 250981us
W20260709 12:14:05.841149 22087 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.004s	sys 0.032s
I20260709 12:14:05.893415 12572 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:05.893567 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:05.774199 22293 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.084s
I20260709 12:14:05.880609 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:05.881886 12571 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:05.794973 22223 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.070s
I20260709 12:14:05.895370 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:05.892376 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 104471us
W20260709 12:14:05.882428 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 241528us
I20260709 12:14:05.894254 12523 raft_consensus.cc:2376] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 30. Re-sending same reply.
I20260709 12:14:05.803030 22292 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.108s
I20260709 12:14:05.879956 21935 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:05.934242 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:06.011075 12530 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
W20260709 12:14:05.933221 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 51639.6us
I20260709 12:14:06.012861 12571 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:05.970748 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:05.767096 22200 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:05.892433 22141 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:05.895052 22277 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:05.893906 22196 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:05.997331 22307 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:14:06.003293 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 107991us
I20260709 12:14:05.843366 22287 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.155s
I20260709 12:14:05.891512 22272 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.116s
I20260709 12:14:05.969209 22247 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:06.022927 21935 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:06.023504 22288 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:06.016695 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:14:05.994982 22020 consensus_meta.cc:287] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.891s	user 0.012s	sys 0.008s
I20260709 12:14:05.962592 22112 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:05.848706 22081 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.16, Last appended by leader: 16, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:05.872277 22269 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:05.992401 22129 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:14:05.863186 22295 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 27.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.153s
I20260709 12:14:05.971841 22298 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:05.960848 22131 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:06.022668 22284 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.255s	user 0.000s	sys 0.001s
I20260709 12:14:05.848003 22289 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:05.995777 21753 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:06.019225 22118 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:06.099678 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 83138.1us
I20260709 12:14:06.135042 12571 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:06.027555 22273 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:14:06.038319 22294 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:14:06.103323 12530 pending_rounds.cc:85] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 14
I20260709 12:14:06.143955 12530 pending_rounds.cc:107] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 25.15
I20260709 12:14:06.144340 12530 raft_consensus.cc:2889] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:06.066344 22275 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:14:06.100476 22171 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.063s	user 0.022s	sys 0.009s
I20260709 12:14:06.099675 22302 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:06.039253 22196 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:06.138501 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 139858us
I20260709 12:14:06.164978 12646 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:14:06.140129 12643 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.002s	sys 0.006s
I20260709 12:14:06.103488 22020 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:06.120707 22293 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:14:06.118402 22129 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:06.213272 22129 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:05.998436 22283 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:06.163686 22259 consensus_meta.cc:287] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.028s	user 0.016s	sys 0.005s
I20260709 12:14:06.167090 12646 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:06.167683 12643 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.003s	sys 0.006s
W20260709 12:14:06.232105 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 708022us
I20260709 12:14:06.150435 12527 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 30 index: 22. Preceding OpId from leader: term: 31 index: 22. (term mismatch)
I20260709 12:14:06.233460 12527 pending_rounds.cc:85] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 21
I20260709 12:14:06.233781 12527 pending_rounds.cc:107] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 30.22
I20260709 12:14:06.234028 12527 raft_consensus.cc:2889] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:06.235276 12643 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:06.140462 12494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.661s	user 0.000s	sys 0.009s
W20260709 12:14:06.236181 12494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.757s	user 0.000s	sys 0.010s
W20260709 12:14:06.236490 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 757342us
W20260709 12:14:06.175508 22312 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:14:06.133049 21935 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:14:06.183348 22301 log.cc:927] Time spent T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.215s	user 0.000s	sys 0.000s
W20260709 12:14:06.187084 22309 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
I20260709 12:14:06.163079 12525 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
W20260709 12:14:06.287111 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 120248us
W20260709 12:14:06.148772 22145 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.245s	user 0.003s	sys 0.015s
I20260709 12:14:06.136041 22298 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:06.171433 22302 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:06.222757 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3691) took 1485 ms. Trace:
I20260709 12:14:06.294679 12526 rpcz_store.cc:276] 0709 12:14:04.737365 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.740964 (+  3599us) service_pool.cc:224] Handling call
0709 12:14:06.214470 (+1473506us) spinlock_profiling.cc:243] Waited 819 ms on lock 0x7b54003c0854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:06.222690 (+  8220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1884846336}
I20260709 12:14:06.143638 22294 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:06.131711 22247 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 12:14:06.152968 22141 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:14:06.170500 22200 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:06.122917 22306 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.099s
I20260709 12:14:06.234309 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:06.240198 22074 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:06.221271 22054 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:06.101610 22288 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:06.241099 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:06.237900 12643 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:06.227177 22107 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.582s	user 0.004s	sys 0.012s
I20260709 12:14:06.255985 12679 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27. Ops: []
W20260709 12:14:06.355252 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 117391us
I20260709 12:14:06.256809 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:06.227599 22287 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:06.295732 12675 raft_consensus.cc:2410] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 26 because replica has last-logged OpId of term: 24 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 12.
I20260709 12:14:06.231228 22293 raft_consensus.cc:697] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:06.362875 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:06.300945 12569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.003s	sys 0.002s
I20260709 12:14:06.349170 12603 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:14:06.348096 12673 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
W20260709 12:14:06.364627 12569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.770s	user 0.004s	sys 0.003s
I20260709 12:14:06.289047 22193 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:06.326325 12677 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.587s	user 0.000s	sys 0.011s
W20260709 12:14:06.370847 12677 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.006s	sys 0.012s
W20260709 12:14:06.348381 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 45908.7us
I20260709 12:14:06.295054 22233 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:14:06.348578 22283 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:06.356163 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:06.300505 12604 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.671s	user 0.005s	sys 0.001s
W20260709 12:14:06.297900 22172 consensus_meta.cc:287] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.323s	user 0.005s	sys 0.011s
I20260709 12:14:06.366369 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:06.383139 12604 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.754s	user 0.005s	sys 0.001s
I20260709 12:14:06.321292 12601 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:06.348392 22087 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.386818 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:14:06.365268 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 771069us
W20260709 12:14:06.300191 22111 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.166s	user 0.019s	sys 0.002s
I20260709 12:14:06.253016 22307 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:06.321259 22141 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:06.366251 22318 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:06.388576 12643 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:06.389998 12673 raft_consensus.cc:2393] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 23: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:06.436895 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:06.412519 22145 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.393508 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:06.366732 22075 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.431993 22141 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:06.438246 22145 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:06.439038 22075 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:06.410377 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:06.440584 12526 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:06.441527 22145 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:06.442389 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:06.443601 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:06.444769 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:06.445642 12679 raft_consensus.cc:2410] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 24 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:14:06.394273 22054 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.058s
I20260709 12:14:06.399889 22287 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:06.446846 22020 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:06.447683 22111 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:14:06.448475 22141 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:06.449429 22323 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.101s	user 0.000s	sys 0.000s
I20260709 12:14:06.352001 22200 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:06.454622 22141 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:06.456650 22200 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:06.461122 22129 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:06.401429 22087 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:06.464828 22087 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:06.469064 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 80598.6us
W20260709 12:14:06.469343 12672 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.522s	user 0.000s	sys 0.022s
W20260709 12:14:06.469794 12672 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.534s	user 0.000s	sys 0.022s
I20260709 12:14:06.470191 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:06.379462 22233 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:06.469906 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:06.416775 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:06.476168 12606 raft_consensus.cc:2376] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 28. Re-sending same reply.
I20260709 12:14:06.386327 22074 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:14:06.394430 22274 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.004s	sys 0.013s
I20260709 12:14:06.471436 12676 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
I20260709 12:14:06.472709 12673 raft_consensus.cc:2410] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 25 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
W20260709 12:14:06.473438 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:06.413172 22277 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.253315 22259 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:06.363451 22107 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:06.493384 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:06.494340 12679 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:06.495028 12673 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:14:06.412952 22292 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.409209 22272 consensus_queue.cc:1059] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 15, Time since last communication: 0.082s
I20260709 12:14:06.409677 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:06.427698 22307 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:06.416335 22269 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:06.487819 22020 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:06.476020 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:06.382995 22193 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:06.500437 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:14:06.489317 22274 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.697s	user 0.004s	sys 0.013s
W20260709 12:14:06.539390 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 63439.3us
I20260709 12:14:06.540428 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:06.541291 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:06.435859 22318 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:06.493456 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:06.486979 22233 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:06.393051 22316 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.104s
I20260709 12:14:06.446866 22287 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:06.535018 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:06.545500 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:06.510816 12571 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:06.545961 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:06.382416 21935 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 15, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:14:06.490613 21713 consensus_meta.cc:287] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.330s	user 0.015s	sys 0.000s
I20260709 12:14:06.409614 22242 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.081s
I20260709 12:14:06.543157 12676 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:14:06.546623 12604 raft_consensus.cc:2376] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 22. Re-sending same reply.
I20260709 12:14:06.508901 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:06.548585 12602 raft_consensus.cc:2393] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 29: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:06.537050 12679 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
W20260709 12:14:06.563788 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 54974.5us
I20260709 12:14:06.551797 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:06.565900 12571 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:06.502769 22247 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
W20260709 12:14:06.554481 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 144925us
I20260709 12:14:06.561194 22306 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.016s
I20260709 12:14:06.550191 22131 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:06.551860 22318 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:06.581809 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:06.576220 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:06.580996 12673 raft_consensus.cc:2410] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 28 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:14:06.575695 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:06.582566 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:06.584507 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 35 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:06.570180 22112 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:14:06.587872 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:06.534735 22289 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:06.588549 12604 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:14:06.586112 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:06.589845 12679 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:14:06.590355 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:06.586269 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:06.591151 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:06.573799 21640 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:14:06.574178 22326 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:06.585604 22087 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:06.586097 22200 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:14:06.592020 12679 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:06.401257 22294 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:06.538719 22307 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:06.545212 22020 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:06.551378 22287 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:06.552796 22273 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:06.550093 22317 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:14:06.561510 22272 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:06.595938 22112 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:06.559211 22332 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:14:06.598534 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:06.594760 13304 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:06.598424 22141 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.012s
W20260709 12:14:06.611078 22325 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.076s	user 0.000s	sys 0.000s
I20260709 12:14:06.467344 22298 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.20, Last appended by leader: 20, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:06.612042 22302 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:06.581676 22145 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 27.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.047s
I20260709 12:14:06.670570 22216 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:14:06.670979 22259 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:06.670984 22216 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:06.647055 22242 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.025s
I20260709 12:14:06.672098 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3615) took 1909 ms. Trace:
I20260709 12:14:06.672245 22216 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 29.15, Last appended by leader: 15, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:14:06.672523 12524 rpcz_store.cc:276] 0709 12:14:04.763156 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.858672 (+ 95516us) service_pool.cc:224] Handling call
0709 12:14:04.859100 (+   428us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:05.598279 (+739179us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.671836 (+1073557us) spinlock_profiling.cc:243] Waited 730 ms on lock 0x7b54003c1754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:06.672032 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1680130944}
I20260709 12:14:06.673374 22216 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:06.502996 22277 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:06.544682 22292 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:06.675529 22129 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:06.675992 22277 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:06.622341 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 3631) took 1265 ms. Trace:
I20260709 12:14:06.623558 22020 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:06.678640 12525 rpcz_store.cc:276] 0709 12:14:05.356419 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.358359 (+  1940us) service_pool.cc:224] Handling call
0709 12:14:06.622109 (+1263750us) spinlock_profiling.cc:243] Waited 742 ms on lock 0x7b540038f9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:06.622268 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1706870656}
I20260709 12:14:06.623287 22288 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:06.682500 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 35 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:06.682726 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:06.683401 12598 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:06.683892 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:06.684535 12604 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 31.
W20260709 12:14:06.685105 22305 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.430s	user 0.001s	sys 0.000s
I20260709 12:14:06.685444 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:06.624150 22294 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:06.686376 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3666) took 1319 ms. Trace:
I20260709 12:14:06.686770 12599 rpcz_store.cc:276] 0709 12:14:05.366597 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.535490 (+168893us) service_pool.cc:224] Handling call
0709 12:14:05.567979 (+ 32489us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:05.585237 (+ 17258us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:05.585268 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:05.595189 (+  9921us) log.cc:844] Serialized 26 byte log entry
0709 12:14:05.901188 (+305999us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:05.901243 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:05.901344 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:06.684478 (+783134us) spinlock_profiling.cc:243] Waited 30.9 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:14:06.686047 (+  1569us) raft_consensus.cc:1685] finished
0709 12:14:06.686069 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.686319 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":44489,"raft.queue_time_us":17475,"raft.run_cpu_time_us":225,"raft.run_wall_time_us":39589,"spinlock_wait_cycles":71166848,"thread_start_us":255362,"threads_started":1,"wal-append.queue_time_us":527022}
I20260709 12:14:06.688833 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:06.629355 22280 consensus_meta.cc:287] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.862s	user 0.009s	sys 0.001s
I20260709 12:14:06.571898 22131 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:06.571493 22107 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:06.594008 21640 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:06.689867 22280 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:06.626926 22318 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:06.581781 22278 consensus_meta.cc:287] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.693s	user 0.011s	sys 0.002s
I20260709 12:14:06.593343 22196 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:06.744534 12531 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 26: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:06.600155 22087 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:06.577837 22306 consensus_queue.cc:1059] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.060s
I20260709 12:14:06.597451 22326 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:06.641599 22317 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:06.648283 13302 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:06.752168 22317 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:06.758685 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 110509us
I20260709 12:14:06.759590 13302 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 22 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:06.761322 13302 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:06.762360 22242 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:06.763286 22280 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:06.764360 22306 raft_consensus.cc:3055] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:14:06.764899 22306 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:06.766023 22306 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:06.568383 21713 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:06.509943 22293 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.19, Last appended by leader: 19, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:06.653630 22088 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:06.636930 22329 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 11, Time since last communication: 0.057s
I20260709 12:14:06.655517 22272 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:14:06.647197 22332 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:06.658675 22169 raft_consensus.cc:3055] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:14:06.662263 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4459) took 1104 ms. Trace:
I20260709 12:14:06.665676 12420 catalog_manager.cc:5697] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 10 to 16, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 16 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:06.676051 22292 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:06.690013 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:06.818056 22088 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:06.692703 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 20 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:06.819640 22332 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:06.828390 12599 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:06.831193 22088 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:06.691500 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3662) took 1328 ms. Trace:
I20260709 12:14:06.832319 12597 rpcz_store.cc:276] 0709 12:14:05.365470 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.414904 (+ 49434us) service_pool.cc:224] Handling call
0709 12:14:06.674069 (+1259165us) spinlock_profiling.cc:243] Waited 67.5 ms on lock 0x7b54003a0cd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:06.691430 (+ 17361us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":155143040}
I20260709 12:14:06.832226 12420 catalog_manager.cc:5697] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 12 to 16, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 16 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:14:06.701814 22322 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.264s	user 0.003s	sys 0.000s
I20260709 12:14:06.707113 12598 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 30.
I20260709 12:14:06.838604 12599 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 32.
I20260709 12:14:06.841636 22332 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:06.702160 21640 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:14:06.847143 22318 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:06.702401 22131 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:06.750633 22307 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:06.852572 22307 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:06.742926 21815 consensus_meta.cc:287] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.215s	user 0.009s	sys 0.013s
I20260709 12:14:06.854513 12671 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 26: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:06.854619 21815 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:06.856837 21713 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:06.857427 21815 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:06.769165 13302 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:06.733361 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:06.858239 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 89189.4us
I20260709 12:14:06.792243 22306 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:06.749915 22020 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:06.860697 22020 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:06.742771 22278 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:06.801982 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:06.861440 12673 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:06.617444 22233 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:06.862712 22278 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:06.767468 22326 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:06.864288 12674 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:06.820087 12529 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:06.815779 12677 rpcz_store.cc:276] 0709 12:14:05.558090 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.558497 (+   407us) service_pool.cc:224] Handling call
0709 12:14:05.558870 (+   373us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:05.738689 (+179819us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:05.738724 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:05.739413 (+   689us) log.cc:844] Serialized 26 byte log entry
0709 12:14:06.371823 (+632410us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:06.371873 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.371951 (+    78us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:06.662009 (+290058us) raft_consensus.cc:1685] finished
0709 12:14:06.662031 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.662197 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":35968,"raft.queue_time_us":286068,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":40409,"thread_start_us":811177,"threads_started":2,"wal-append.queue_time_us":689319}
I20260709 12:14:06.863759 22233 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:06.782269 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:06.865517 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:06.865809 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 83687.4us
I20260709 12:14:06.866447 12677 raft_consensus.cc:2410] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 24 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
I20260709 12:14:06.866546 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 35 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:06.868172 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:06.751298 22087 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:06.828083 22272 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Leader election won for term 22
I20260709 12:14:06.844400 22287 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:06.852310 22332 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:06.819576 22169 raft_consensus.cc:740] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:06.825493 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:06.868964 13337 raft_consensus.cc:2393] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 26: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:14:06.869611 12569 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:06.874354 22345 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:14:06.874464 12606 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26.
W20260709 12:14:06.802529 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:06.913272 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:06.881081 22278 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:06.913909 13337 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:06.826697 12679 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26.
I20260709 12:14:06.889465 12673 raft_consensus.cc:2410] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 25 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:14:06.883944 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:06.891371 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:06.893263 22242 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:06.894367 22169 consensus_queue.cc:260] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:06.892074 22260 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:14:06.908068 21640 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:06.910259 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:06.913800 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:06.914709 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:06.881078 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:06.917938 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:14:06.971273 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 87407.1us
I20260709 12:14:06.974102 12603 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:06.918754 13331 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
W20260709 12:14:06.919727 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 50203.6us
I20260709 12:14:06.975364 13337 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29.
I20260709 12:14:06.971683 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:06.976317 12605 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:06.912164 22293 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.917824 22278 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:06.973153 12597 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:14:06.975240 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:06.928766 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 334063us
I20260709 12:14:06.973502 12601 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 22: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:14:06.982803 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:06.917728 22296 raft_consensus.cc:3055] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:14:06.928659 22329 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:14:06.985296 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:06.982477 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:06.934837 22247 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:06.988817 13330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38482 (request call id 866) took 1639 ms. Trace:
I20260709 12:14:06.990617 13330 rpcz_store.cc:276] 0709 12:14:05.349236 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.358711 (+  9475us) service_pool.cc:224] Handling call
0709 12:14:05.387688 (+ 28977us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:06.690697 (+1303009us) spinlock_profiling.cc:243] Waited 8.23 ms on lock 0x7b1800299980. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f1488a18060 00007f1488a18bfb 00007f1488a2364b
0709 12:14:06.690913 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:06.690954 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:06.691566 (+   612us) log.cc:844] Serialized 26 byte log entry
0709 12:14:06.876176 (+184610us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:06.876244 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.876338 (+    94us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:06.984140 (+107802us) raft_consensus.cc:1685] finished
0709 12:14:06.984161 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:06.988735 (+  4574us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10927,"raft.queue_time_us":159913,"raft.run_cpu_time_us":174,"raft.run_wall_time_us":12218,"spinlock_wait_cycles":18920192,"thread_start_us":347429,"threads_started":2,"wal-append.queue_time_us":183037}
W20260709 12:14:06.993903 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 705587us
W20260709 12:14:06.994820 22341 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:14:06.997022 13334 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23. Ops: []
I20260709 12:14:06.972330 22260 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:07.008364 12646 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:06.988307 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:07.014700 12597 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:14:07.013435 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:06.984536 22278 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:07.024854 22349 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:06.986739 22296 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:07.024755 12603 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:14:06.996398 22247 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:06.969806 22272 raft_consensus.cc:697] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:07.078501 12676 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 12:14:06.997043 22332 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:14:06.998682 22348 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:14:07.031793 12646 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:07.033625 12420 catalog_manager.cc:5697] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 13 to 17, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 17 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:07.038323 22339 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:14:07.039971 21640 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:07.041121 22331 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.588s	user 0.006s	sys 0.005s
I20260709 12:14:07.042279 22242 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:07.045408 22296 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 25.19, Last appended by leader: 19, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:07.007055 22129 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.039s
I20260709 12:14:07.012161 22169 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:14:07.096375 22120 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:07.097002 22120 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.026383 22318 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:07.030051 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:07.101760 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 71793.3us
I20260709 12:14:07.102969 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:07.046815 22260 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.16, Last appended by leader: 16, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.049010 12605 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:14:07.048142 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:07.108155 12603 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:07.050503 22144 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:07.051092 22088 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:14:07.053202 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:14:07.054679 12602 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4218) took 1692 ms (client timeout 1638 ms). Trace:
I20260709 12:14:07.018111 22307 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 12:14:06.987561 22293 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:07.059011 22330 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.630s	user 0.013s	sys 0.011s
I20260709 12:14:07.060153 13302 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:14:07.115018 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 54947us
I20260709 12:14:06.998679 21815 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.019s
I20260709 12:14:07.063369 22193 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:07.067463 22310 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.638s	user 0.000s	sys 0.009s
I20260709 12:14:07.069118 22287 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:07.072293 12671 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 25. Current term is 26. Ops: [23.18-25.19]
W20260709 12:14:07.089578 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 57890.3us
W20260709 12:14:07.131729 12572 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.788s (SENT) [suppressed 17 similar messages]
I20260709 12:14:07.132762 12646 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:07.080257 22233 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:07.093114 22216 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:07.148677 22233 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.087358 22348 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:07.119567 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:07.107960 12643 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:07.107321 22296 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:14:07.129165 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:07.162577 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:07.120527 22306 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.112613 22088 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:07.124084 22294 consensus_queue.cc:1059] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.015s
I20260709 12:14:07.126801 22288 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:07.140100 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 902992us
I20260709 12:14:07.101327 22318 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:07.141968 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:07.142347 22144 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:07.205962 22144 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 15, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.144757 12606 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 25. Current term is 26. Ops: []
I20260709 12:14:07.210835 12601 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:07.125191 22332 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:07.124221 22302 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:07.159911 22348 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:07.147783 12646 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:07.113206 12602 rpcz_store.cc:269] 0709 12:14:05.361914 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.403970 (+ 42056us) service_pool.cc:224] Handling call
0709 12:14:07.054405 (+1650435us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400231c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:07.054578 (+   173us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2518460800}
I20260709 12:14:07.157032 22272 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 22, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:07.157827 22200 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.158424 22233 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:07.120535 22317 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:14:07.158996 22145 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:14:07.078299 22349 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:07.112661 22339 raft_consensus.cc:697] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:07.175659 12603 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 31.
I20260709 12:14:07.176878 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1209) took 1308 ms. Trace:
W20260709 12:14:07.156271 22351 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:14:07.098402 22326 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.151089 22118 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:14:07.126786 22340 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:14:07.151849 22131 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.203821 12420 catalog_manager.cc:5697] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 12 to 16, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 16 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:07.208689 22278 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.168190 22307 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:07.147102 21640 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:14:07.178727 22088 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:14:07.223176 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 75515.4us
I20260709 12:14:07.186697 22260 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:07.209990 22318 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:14:07.233392 22353 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.138s	user 0.001s	sys 0.000s
I20260709 12:14:07.236258 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:60222 (request call id 1208) took 1368 ms. Trace:
I20260709 12:14:07.178527 22306 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:07.231256 22200 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.241432 22174 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 26.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:14:07.242225 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3667) took 1875 ms. Trace:
I20260709 12:14:07.243166 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3621) took 2310 ms. Trace:
I20260709 12:14:07.231665 22145 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:07.247476 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 63800.1us
I20260709 12:14:07.248966 12522 rpcz_store.cc:276] 0709 12:14:05.868693 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.888152 (+ 19459us) service_pool.cc:224] Handling call
0709 12:14:05.888523 (+   371us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:05.888755 (+   232us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:14:06.015098 (+126343us) log.cc:844] Serialized 14 byte log entry
0709 12:14:07.108659 (+1093561us) spinlock_profiling.cc:243] Waited 3.33 ms on lock 0x7b18002c3bc0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:14:07.176358 (+ 67699us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:07.176447 (+    89us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:14:07.176512 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.176598 (+    86us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:07.176780 (+   182us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":100911,"prepare.queue_time_us":231586,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":43,"raft.queue_time_us":991130,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":103,"spinlock_wait_cycles":7661440,"thread_start_us":266607,"threads_started":3,"wal-append.queue_time_us":239024}
I20260709 12:14:07.210709 22288 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.288401 12603 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 28. Current term is 29. Ops: [28.18-28.18]
I20260709 12:14:07.292583 12646 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:07.290645 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:07.297703 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 189813us
I20260709 12:14:07.299242 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 4463) took 1711 ms. Trace:
I20260709 12:14:07.396529 12675 rpcz_store.cc:276] 0709 12:14:05.587653 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.587835 (+   182us) service_pool.cc:224] Handling call
0709 12:14:07.299000 (+1711165us) spinlock_profiling.cc:243] Waited 707 ms on lock 0x7b5400091fd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:07.299176 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1624983296}
I20260709 12:14:07.397826 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:07.300310 12672 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 13. (term mismatch)
I20260709 12:14:07.398589 12672 pending_rounds.cc:85] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 12
I20260709 12:14:07.398881 12672 pending_rounds.cc:107] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 24.13
I20260709 12:14:07.399132 12672 raft_consensus.cc:2889] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:07.266134 22120 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:07.282238 22118 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:07.353457 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:07.281600 22327 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.225262 22302 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:07.346189 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 214439us
W20260709 12:14:07.446642 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52195.6us
I20260709 12:14:07.248253 22317 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:07.458110 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:07.390477 22269 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.295274 22340 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:07.284807 22168 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.072s
I20260709 12:14:07.311748 22307 raft_consensus.cc:697] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:07.391283 12529 rpcz_store.cc:276] 0709 12:14:04.932779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:04.933181 (+   402us) service_pool.cc:224] Handling call
0709 12:14:04.955340 (+ 22159us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:06.864862 (+1909522us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.242917 (+378055us) spinlock_profiling.cc:243] Waited 1.86 s on lock 0x7b5400407c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:07.243101 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4287951744}
I20260709 12:14:07.443578 22081 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:14:07.391180 12600 rpcz_store.cc:276] 0709 12:14:05.367021 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.536841 (+169820us) service_pool.cc:224] Handling call
0709 12:14:05.550461 (+ 13620us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:05.550908 (+   447us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:05.550954 (+    46us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:05.563435 (+ 12481us) log.cc:844] Serialized 26 byte log entry
0709 12:14:06.619785 (+1056350us) spinlock_profiling.cc:243] Waited 4.39 ms on lock 0x7b18000646c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:14:06.793689 (+173904us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:14:06.793736 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:06.793812 (+    76us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:06.870309 (+ 76497us) raft_consensus.cc:1685] finished
0709 12:14:06.870331 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:07.242031 (+371700us) spinlock_profiling.cc:243] Waited 13.2 ms on lock 0x7b5400374950. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:07.242160 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":23956,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":39011,"spinlock_wait_cycles":40393344,"thread_start_us":85843,"threads_started":1,"wal-append.queue_time_us":1201473}
I20260709 12:14:07.293550 22294 consensus_queue.cc:1059] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.085s
I20260709 12:14:07.367552 22169 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.443715 12676 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:14:07.444823 12675 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:14:07.448246 12522 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 25 index: 13. (index mismatch)
I20260709 12:14:07.448266 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:07.375753 12531 rpcz_store.cc:276] 0709 12:14:05.868302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.880333 (+ 12031us) service_pool.cc:224] Handling call
0709 12:14:07.235839 (+1355506us) spinlock_profiling.cc:243] Waited 848 ms on lock 0x7b5400105ed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:07.236025 (+   186us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1950621440}
W20260709 12:14:07.458406 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 58063.6us
I20260709 12:14:07.407380 22145 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:07.466787 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:07.535022 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 21 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:07.536273 12524 raft_consensus.cc:2410] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 24 because replica has last-logged OpId of term: 22 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 12.
W20260709 12:14:07.466737 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 173935us
I20260709 12:14:07.461046 22172 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.413677 22298 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:07.469627 12494 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:14:07.548444 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:07.552173 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 82619us
I20260709 12:14:07.491618 22118 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 29.15, Last appended by leader: 15, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.510571 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:14:07.511643 22354 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.145s	user 0.005s	sys 0.000s
I20260709 12:14:07.470086 22081 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:07.512702 22196 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:14:07.513759 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:07.515009 22306 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:07.508145 22317 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:07.466936 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 69578.7us
I20260709 12:14:07.527848 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:07.463892 22290 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.543840 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:07.585448 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:07.586982 12671 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 26: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:14:07.590112 12494 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:07.595450 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:07.557164 22356 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.190s	user 0.001s	sys 0.000s
I20260709 12:14:07.468210 22269 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.523438 22295 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:14:07.592558 22359 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:14:07.461200 22327 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.592433 22279 raft_consensus.cc:3055] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:14:07.523125 22360 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:14:07.615087 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:07.624434 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:07.587740 22285 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:07.586095 22287 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
W20260709 12:14:07.619776 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:07.633095 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:07.592191 22293 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:07.621948 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 224228us
I20260709 12:14:07.623701 12604 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:14:07.629289 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:07.541280 22169 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.451913 22339 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:07.631421 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:07.609493 22332 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:07.636972 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:07.646697 12602 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:14:07.611904 22075 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:07.629244 22087 consensus_meta.cc:287] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.746s	user 0.010s	sys 0.001s
I20260709 12:14:07.651065 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3783) took 1001 ms. Trace:
I20260709 12:14:07.560213 22172 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:07.621523 22279 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:07.655710 12603 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:07.584620 22298 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.599031 22259 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.033s
I20260709 12:14:07.664917 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:07.637912 22118 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:14:07.669312 12673 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.504s	user 0.004s	sys 0.011s
W20260709 12:14:07.693751 12673 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.005s	sys 0.011s
I20260709 12:14:07.671125 13330 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 22. Current term is 23. Ops: []
W20260709 12:14:07.670920 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 378413us
I20260709 12:14:07.665838 12680 rpcz_store.cc:276] 0709 12:14:06.649789 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.095050 (+445261us) service_pool.cc:224] Handling call
0709 12:14:07.095388 (+   338us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:07.095771 (+   383us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:14:07.095799 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:07.096243 (+   444us) log.cc:844] Serialized 26 byte log entry
0709 12:14:07.180716 (+ 84473us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:07.181107 (+   391us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.181203 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:07.650817 (+469614us) raft_consensus.cc:1685] finished
0709 12:14:07.650841 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:07.650998 (+   157us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7639,"raft.queue_time_us":109,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":8635,"thread_start_us":84108,"threads_started":1,"wal-append.queue_time_us":187428}
I20260709 12:14:07.699060 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:07.646670 22292 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.595908 22290 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:07.639209 22168 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:07.685078 12526 raft_consensus.cc:2410] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 28 because replica has last-logged OpId of term: 26 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 15.
I20260709 12:14:07.648710 22285 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:07.689344 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 99422.8us
I20260709 12:14:07.620500 22295 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:07.725412 12603 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:14:07.726820 22295 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:07.727348 22327 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:07.728837 22168 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.731401 22168 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:07.733917 22088 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.735394 12677 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: []
I20260709 12:14:07.737298 22378 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:14:07.737641 22088 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:07.705127 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:07.711401 12494 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:07.739591 22168 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:07.740236 12494 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:07.740463 22168 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.673240 22087 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.741480 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:07.693010 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3766) took 1381 ms. Trace:
I20260709 12:14:07.717221 22172 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:07.716674 22279 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.742264 12672 rpcz_store.cc:276] 0709 12:14:06.311223 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.564695 (+253472us) service_pool.cc:224] Handling call
0709 12:14:07.299942 (+735247us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:07.399622 (+ 99680us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.692712 (+293090us) spinlock_profiling.cc:243] Waited 701 ms on lock 0x7b5400091fd0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:07.692920 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1612749184}
I20260709 12:14:07.743036 22087 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:07.743330 22279 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:07.743700 22087 consensus_queue.cc:1059] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
W20260709 12:14:07.716771 22358 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.117s	user 0.000s	sys 0.001s
I20260709 12:14:07.719264 22298 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:07.693974 22196 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:14:07.723441 22364 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:14:07.724408 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3785) took 1048 ms. Trace:
W20260709 12:14:07.717880 22369 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:14:07.742295 22259 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:07.747000 22259 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:07.747426 22259 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:07.741045 22274 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:07.747685 22290 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:07.664899 22326 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:07.724531 22269 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:07.748873 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:07.750206 22274 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:07.738193 22378 consensus_queue.cc:1243] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 27.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:14:07.745914 12678 rpcz_store.cc:276] 0709 12:14:06.675901 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.165020 (+489119us) service_pool.cc:224] Handling call
0709 12:14:07.165384 (+   364us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:07.165763 (+   379us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:07.165791 (+    28us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:07.178852 (+ 13061us) log.cc:844] Serialized 26 byte log entry
0709 12:14:07.319387 (+140535us) spinlock_profiling.cc:243] Waited 8.45 ms on lock 0x7b18002c1220. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:14:07.572256 (+252869us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:07.572323 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.572444 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:07.724110 (+151666us) raft_consensus.cc:1685] finished
0709 12:14:07.724132 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:07.724346 (+   214us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":127,"raft.run_cpu_time_us":156,"raft.run_wall_time_us":12718,"spinlock_wait_cycles":19425664,"thread_start_us":202359,"threads_started":1,"wal-append.queue_time_us":462763}
I20260709 12:14:07.750713 12494 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:07.750790 22274 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:07.756983 22196 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election lost for term 23. Reason: could not achieve majority
I20260709 12:14:07.757169 22274 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:14:07.761737 22348 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.530s	user 0.006s	sys 0.002s
I20260709 12:14:07.712450 22292 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.763634 22168 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:07.764619 12494 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:07.769014 22259 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:07.773056 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:07.828662 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 64128.7us
I20260709 12:14:07.650628 22081 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.797940 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3639) took 1388 ms. Trace:
I20260709 12:14:07.836580 12528 rpcz_store.cc:276] 0709 12:14:06.409551 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.411527 (+  1976us) service_pool.cc:224] Handling call
0709 12:14:06.433088 (+ 21561us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:07.164297 (+731209us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:07.164331 (+    34us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:07.180430 (+ 16099us) log.cc:844] Serialized 26 byte log entry
0709 12:14:07.383455 (+203025us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:07.383508 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.383590 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:07.797694 (+414104us) raft_consensus.cc:1685] finished
0709 12:14:07.797719 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:07.797874 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":748598,"raft.queue_time_us":734659,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":34520,"spinlock_wait_cycles":70912,"thread_start_us":164407,"threads_started":1,"wal-append.queue_time_us":420917}
I20260709 12:14:07.799415 22269 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:07.803438 22374 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:07.629437 22307 consensus_queue.cc:237] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:07.755286 22285 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:07.823964 22247 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.754s	user 0.013s	sys 0.001s
I20260709 12:14:07.817991 22329 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:07.632120 22340 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:07.828652 22200 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:07.822345 22302 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:07.869108 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:07.843884 22327 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:07.883081 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3637) took 1844 ms. Trace:
I20260709 12:14:07.887547 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:07.889406 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:07.889940 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:07.891821 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:07.893710 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:07.886230 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:07.895517 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:07.835707 22348 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:07.902693 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 246612us
I20260709 12:14:07.883738 12530 rpcz_store.cc:276] 0709 12:14:06.038652 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.102566 (+ 63914us) service_pool.cc:224] Handling call
0709 12:14:06.103040 (+   474us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:07.127280 (+1024240us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:07.127334 (+    54us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:07.139319 (+ 11985us) log.cc:844] Serialized 52 byte log entry
0709 12:14:07.256528 (+117209us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:07.256595 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.256702 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:07.882748 (+626046us) raft_consensus.cc:1685] finished
0709 12:14:07.882780 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:07.882998 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1063228,"raft.queue_time_us":976123,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":99,"thread_start_us":116693,"threads_started":1,"wal-append.queue_time_us":324850}
I20260709 12:14:07.903743 12494 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:07.912437 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:07.913468 12675 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:14:07.916190 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 82329.5us
I20260709 12:14:07.887400 22081 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:07.883383 22327 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:07.852914 22269 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:07.867411 22247 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.868006 22279 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.051s
I20260709 12:14:07.864567 22329 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:07.908766 22283 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:14:07.911127 22313 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.004s	sys 0.015s
I20260709 12:14:07.941298 12572 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:07.860486 22374 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:07.961158 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:07.896521 22172 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 27.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.039s
I20260709 12:14:07.900688 22200 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.035s
I20260709 12:14:07.932659 22327 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:07.981742 12572 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:07.979894 22355 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.585s	user 0.007s	sys 0.023s
I20260709 12:14:07.828442 12420 catalog_manager.cc:5697] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 11 to 16. New cstate: current_term: 16 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:07.962252 22169 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:08.002835 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3782) took 1402 ms. Trace:
W20260709 12:14:07.951231 22313 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.004s	sys 0.016s
I20260709 12:14:08.080256 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:08.019413 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3788) took 1160 ms. Trace:
I20260709 12:14:08.041908 22292 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:08.042524 22259 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:08.042884 22382 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.059s	user 0.001s	sys 0.005s
I20260709 12:14:07.982937 22368 consensus_queue.cc:1059] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.162s
I20260709 12:14:08.043529 22020 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:14:08.044286 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3775) took 1668 ms. Trace:
I20260709 12:14:08.073520 22300 raft_consensus.cc:3055] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Stepping down as leader of term 22
W20260709 12:14:08.046425 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 146558us
I20260709 12:14:08.048398 22131 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:08.073077 22279 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:08.154831 12597 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 12. Preceding OpId from leader: term: 24 index: 13. (index mismatch)
I20260709 12:14:08.156337 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:07.915082 22293 consensus_queue.cc:1059] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.036s
I20260709 12:14:07.980386 22303 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:08.057992 12525 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27. Ops: []
W20260709 12:14:08.060940 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.063648 12522 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 29. Current term is 30. Ops: [29.15-29.15]
I20260709 12:14:08.064967 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:14:08.073047 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:08.079433 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:08.079960 12677 rpcz_store.cc:276] 0709 12:14:06.600320 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.082990 (+482670us) service_pool.cc:224] Handling call
0709 12:14:07.083252 (+   262us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:07.735789 (+652537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.002433 (+266644us) spinlock_profiling.cc:243] Waited 652 ms on lock 0x7b54003be554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:08.002742 (+   309us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1499386624}
I20260709 12:14:08.180763 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:08.181690 12677 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 28.
I20260709 12:14:08.184280 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:08.087998 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 106476us
I20260709 12:14:08.045872 21935 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:08.088649 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:08.089174 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:08.078256 22216 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.929s	user 0.009s	sys 0.013s
I20260709 12:14:08.049970 22326 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:14:08.106235 12528 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:08.045715 22374 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:08.085161 22145 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.526s	user 0.012s	sys 0.000s
I20260709 12:14:08.116829 12673 rpcz_store.cc:276] 0709 12:14:06.376188 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.926462 (+550274us) service_pool.cc:224] Handling call
0709 12:14:06.926822 (+   360us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:06.971736 (+ 44914us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:06.971792 (+    56us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:06.980750 (+  8958us) log.cc:844] Serialized 52 byte log entry
0709 12:14:07.099622 (+118872us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:14:07.694176 (+594554us) log.cc:844] Serialized 14 byte log entry
0709 12:14:07.694689 (+   513us) log.cc:844] Serialized 14 byte log entry
0709 12:14:08.001430 (+306741us) spinlock_profiling.cc:243] Waited 35.1 ms on lock 0x7b54003f55a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f148899b733 00007f1488a20dbf 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c
0709 12:14:08.001493 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.001640 (+   147us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:08.044008 (+ 42368us) raft_consensus.cc:1685] finished
0709 12:14:08.044032 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.044220 (+   188us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":47914,"prepare.queue_time_us":378380,"prepare.run_cpu_time_us":3710,"prepare.run_wall_time_us":22971,"raft.queue_time_us":65445,"raft.run_cpu_time_us":299,"raft.run_wall_time_us":12681,"spinlock_wait_cycles":140260736,"thread_start_us":603754,"threads_started":2,"wal-append.queue_time_us":234831}
I20260709 12:14:08.127641 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:08.136708 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:08.138715 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:08.140408 12604 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26. Ops: []
I20260709 12:14:08.049130 22348 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:08.119071 22392 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.122s	user 0.001s	sys 0.001s
I20260709 12:14:08.161355 12599 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 31. Current term is 32. Ops: [29.17-31.18]
I20260709 12:14:08.162611 12606 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 29. Current term is 30. Ops: [29.15-29.15]
I20260709 12:14:08.163275 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:08.337925 12602 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 26: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:14:08.121627 22390 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.072s	user 0.001s	sys 0.000s
I20260709 12:14:08.174937 12605 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27. Ops: [26.16-26.16]
I20260709 12:14:08.174721 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:08.070927 22375 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:14:07.953222 22349 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:08.086043 12676 rpcz_store.cc:276] 0709 12:14:06.858547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.244889 (+386342us) service_pool.cc:224] Handling call
0709 12:14:07.245262 (+   373us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:07.526302 (+281040us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.010102 (+483800us) spinlock_profiling.cc:243] Waited 196 ms on lock 0x7b54002c5e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:08.019332 (+  9230us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":449762688}
I20260709 12:14:08.185312 12677 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26.
I20260709 12:14:08.237818 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:08.094528 22294 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.078s
I20260709 12:14:08.123664 22300 raft_consensus.cc:740] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:08.105705 22274 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:08.286659 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 480486us
I20260709 12:14:08.288944 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:08.290472 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:08.291589 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1127) took 1714 ms. Trace:
I20260709 12:14:08.308578 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1211) took 1061 ms. Trace:
I20260709 12:14:08.275224 22216 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:08.155150 21713 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.132s
I20260709 12:14:08.234534 22384 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:14:08.291424 22345 raft_consensus.cc:3055] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:14:08.176002 22279 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:08.340286 12604 raft_consensus.cc:1217] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Deduplicated request from leader. Original: 24.14->[26.15-30.17]   Dedup: 26.15->[28.16-30.17]
I20260709 12:14:08.341626 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:08.343451 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:08.237134 21935 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:08.352694 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:08.360385 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:08.360678 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:08.362042 12531 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:08.363785 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:08.340814 22145 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:08.346017 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 132043us
I20260709 12:14:08.346414 12597 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 22.
W20260709 12:14:08.345908 22360 consensus_meta.cc:287] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.705s	user 0.003s	sys 0.010s
I20260709 12:14:08.368571 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:08.346063 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 160244us
I20260709 12:14:08.317435 22278 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:08.516594 12675 consensus_meta.cc:287] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.602s	user 0.011s	sys 0.002s
I20260709 12:14:08.600880 12675 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
W20260709 12:14:08.476804 22388 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.202s	user 0.000s	sys 0.001s
W20260709 12:14:08.545259 13305 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.922s	user 0.010s	sys 0.010s
W20260709 12:14:08.606935 13305 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.983s	user 0.012s	sys 0.011s
I20260709 12:14:08.415033 12680 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:08.427070 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:08.386270 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:08.600885 12606 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:14:08.609660 12604 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 30. Current term is 31. Ops: [26.15-30.17]
I20260709 12:14:08.346176 22292 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:08.605180 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:08.601305 12527 rpcz_store.cc:276] 0709 12:14:07.247690 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.248751 (+  1061us) service_pool.cc:224] Handling call
0709 12:14:07.286829 (+ 38078us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:07.356945 (+ 70116us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:07.356978 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:07.373218 (+ 16240us) log.cc:844] Serialized 52 byte log entry
0709 12:14:07.824422 (+451204us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:14:07.824510 (+    88us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.824614 (+   104us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:08.308243 (+483629us) raft_consensus.cc:1685] finished
0709 12:14:08.308274 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.308489 (+   215us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":69624,"raft.queue_time_us":70342,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":177,"thread_start_us":362310,"threads_started":1,"wal-append.queue_time_us":479312}
I20260709 12:14:08.600713 12677 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:08.381016 12605 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 25. Ops: []
I20260709 12:14:08.609107 12599 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:14:08.607524 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 984127us
I20260709 12:14:08.620335 22409 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.621747 22145 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:08.612124 22129 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.622305 22292 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:08.358568 22329 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:08.622013 22409 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:08.477540 22383 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.185s	user 0.001s	sys 0.000s
W20260709 12:14:08.483242 22391 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
W20260709 12:14:08.529353 22318 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.176s	user 0.008s	sys 0.008s
I20260709 12:14:08.619077 22278 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:08.623878 22279 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:08.362563 22348 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:14:08.466946 22260 consensus_meta.cc:287] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.156s	user 0.008s	sys 0.007s
I20260709 12:14:08.359714 22303 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.265s
I20260709 12:14:08.624960 22274 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:08.624848 12671 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 12:14:08.606665 12674 rpcz_store.cc:276] 0709 12:14:06.576794 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.078533 (+501739us) service_pool.cc:224] Handling call
0709 12:14:07.083890 (+  5357us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:07.084327 (+   437us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:07.084362 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:07.084900 (+   538us) log.cc:844] Serialized 26 byte log entry
0709 12:14:07.195045 (+110145us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:14:07.383173 (+188128us) log.cc:844] Serialized 14 byte log entry
0709 12:14:07.383705 (+   532us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:07.384066 (+   361us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:07.765384 (+381318us) spinlock_profiling.cc:243] Waited 7.15 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:14:08.278426 (+513042us) raft_consensus.cc:1685] finished
0709 12:14:08.291268 (+ 12842us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:08.291484 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18271,"prepare.queue_time_us":319098,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":7317,"raft.run_cpu_time_us":294,"raft.run_wall_time_us":25455,"spinlock_wait_cycles":16433536,"thread_start_us":264565,"threads_started":2,"wal-append.queue_time_us":140959}
I20260709 12:14:08.330250 22326 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:14:08.413601 22118 consensus_meta.cc:287] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.706s	user 0.013s	sys 0.000s
I20260709 12:14:08.585187 22131 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:08.639289 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:08.640408 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:08.641412 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:08.467223 22296 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.303s	user 0.012s	sys 0.005s
W20260709 12:14:08.550431 22395 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.169s	user 0.001s	sys 0.000s
I20260709 12:14:08.451622 22172 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.412737 22169 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.539230 22307 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.645983 12600 raft_consensus.cc:2410] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 27 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 14.
I20260709 12:14:08.611595 22345 raft_consensus.cc:740] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:08.647657 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:08.648746 13338 raft_consensus.cc:1240] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: [28.20-28.20]
I20260709 12:14:08.649667 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:08.649142 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:08.651957 22129 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:08.402639 22144 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:08.613191 22360 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:14:08.659643 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:08.660933 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:08.663496 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:08.686571 12602 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27. Ops: [25.13-26.14]
I20260709 12:14:08.413162 22389 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.245s
I20260709 12:14:08.572929 22302 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.085s
I20260709 12:14:08.652714 22410 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:08.692881 12523 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 31.
I20260709 12:14:08.653038 22375 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.713244 12599 raft_consensus.cc:2410] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 30 because replica has last-logged OpId of term: 29 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 14.
I20260709 12:14:08.890786 12675 raft_consensus.cc:2410] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 28 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 14.
I20260709 12:14:08.473495 22300 consensus_queue.cc:260] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:08.798102 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:08.840171 12598 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 30. Current term is 31. Ops: []
I20260709 12:14:08.892660 12673 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:14:08.889442 12676 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:14:08.713985 12677 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
W20260709 12:14:08.893138 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 409438us
I20260709 12:14:08.623988 12420 catalog_manager.cc:5697] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 12 to 14. New cstate: current_term: 14 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:08.896320 13331 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
W20260709 12:14:08.896492 22408 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.239s	user 0.000s	sys 0.000s
I20260709 12:14:08.670056 22292 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:08.897403 22402 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.333s	user 0.001s	sys 0.000s
I20260709 12:14:08.895761 12530 raft_consensus.cc:2410] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 30 because replica has last-logged OpId of term: 29 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 14.
I20260709 12:14:08.898239 22326 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:08.899417 22326 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:08.899628 12420 catalog_manager.cc:5697] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 13 to 16. New cstate: current_term: 16 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
W20260709 12:14:08.774966 22391 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:14:08.895510 22385 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:08.919967 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:08.688040 22280 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.505s	user 0.003s	sys 0.008s
W20260709 12:14:08.902765 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 255210us
I20260709 12:14:08.924767 12572 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:08.691543 22278 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:08.680846 22318 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:08.766945 12494 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:08.901484 22389 raft_consensus.cc:3055] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:14:08.907819 12677 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27. Ops: []
I20260709 12:14:08.903405 22349 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.670240 22411 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.231s
W20260709 12:14:08.910821 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 261737us
W20260709 12:14:08.932008 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 165196us
W20260709 12:14:08.800141 22383 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.118s	user 0.000s	sys 0.001s
I20260709 12:14:08.919672 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:08.955106 12676 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 21.
I20260709 12:14:08.919728 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:08.900722 12599 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:14:08.923942 12672 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 22. Current term is 23. Ops: []
W20260709 12:14:08.910111 22081 consensus_meta.cc:287] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.978s	user 0.008s	sys 0.005s
I20260709 12:14:08.911774 22145 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:08.926162 12572 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:08.816798 22290 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.020s	sys 0.040s
I20260709 12:14:08.927421 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:08.929077 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:08.891089 22196 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:08.892967 13330 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:14:08.945149 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4481) took 1564 ms. Trace:
I20260709 12:14:08.901942 22131 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:08.912487 22410 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:14:08.692850 22348 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:08.915792 22087 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:08.932327 22302 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:09.049747 22292 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:14:08.909144 22375 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:08.909997 22144 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:08.930282 22416 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:08.909025 22307 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:08.950963 22295 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:14:08.928388 22360 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.063594 22144 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:09.063943 22360 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:09.064772 22302 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:09.066898 22302 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:09.067637 22278 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:09.069808 22348 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:09.072800 22131 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:08.930428 22385 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:08.894968 22172 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:09.001758 12675 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26. Ops: []
I20260709 12:14:08.994843 22409 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:08.945683 22300 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:14:08.958606 22349 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:08.908593 22169 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:08.989840 22400 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.364s	user 0.001s	sys 0.000s
W20260709 12:14:08.989283 22403 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.403s	user 0.000s	sys 0.001s
I20260709 12:14:08.991168 22196 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:08.952545 22345 consensus_queue.cc:260] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:08.986363 22283 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.033s	user 0.013s	sys 0.001s
I20260709 12:14:09.110458 22283 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:08.991190 21935 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:09.111933 22169 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:09.054699 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:08.944615 22389 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:09.055511 13334 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28. Ops: []
W20260709 12:14:09.052215 22419 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.160s	user 0.001s	sys 0.000s
I20260709 12:14:09.050189 22145 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:09.114181 22389 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:09.116202 22389 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:09.116721 22145 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:09.117877 22168 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:09.118875 22247 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:09.092434 22087 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:09.129012 22087 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:09.099268 22327 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.002s	user 0.012s	sys 0.002s
I20260709 12:14:09.133047 22327 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:09.135515 22327 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:09.124112 22172 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:09.125762 22318 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:09.123646 22375 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:09.139433 22387 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 1.093s	user 0.005s	sys 0.000s
I20260709 12:14:09.139585 22307 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:09.150053 22269 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:09.150862 22360 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:09.153425 22385 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:09.006969 12526 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:14:09.047539 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:09.205044 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3717) took 1136 ms. Trace:
I20260709 12:14:08.950208 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:09.208670 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3709) took 1142 ms. Trace:
I20260709 12:14:09.205747 12531 rpcz_store.cc:276] 0709 12:14:08.067859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.047119 (+979260us) service_pool.cc:224] Handling call
0709 12:14:09.204644 (+157525us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:08.990427 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:09.214867 12678 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 31. Current term is 32. Ops: []
I20260709 12:14:09.214710 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:09.052731 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:08.997493 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:09.219106 12602 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:09.048974 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:09.014760 12643 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:09.223932 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 209189us
I20260709 12:14:09.224906 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:09.057749 12677 rpcz_store.cc:276] 0709 12:14:07.380939 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.894672 (+1513733us) service_pool.cc:224] Handling call
0709 12:14:08.894998 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:08.944815 (+ 49817us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.945074 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.216627 12531 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:14:09.230579 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3710) took 1164 ms. Trace:
I20260709 12:14:09.218088 12599 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 26: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:09.231237 12531 rpcz_store.cc:276] 0709 12:14:08.065989 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.213763 (+1147774us) service_pool.cc:224] Handling call
0709 12:14:09.230466 (+ 16703us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:09.209681 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 259446us
I20260709 12:14:09.225723 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:09.244921 12676 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 24: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:14:09.246114 22269 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:09.247123 22339 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:09.247936 22269 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:09.248950 22339 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:09.209319 12524 rpcz_store.cc:276] 0709 12:14:08.065684 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.065835 (+   151us) service_pool.cc:224] Handling call
0709 12:14:08.066301 (+   466us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:08.089564 (+ 23263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:08.089597 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:08.096651 (+  7054us) log.cc:844] Serialized 26 byte log entry
0709 12:14:08.314146 (+217495us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:08.314211 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:08.314323 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:09.207858 (+893535us) raft_consensus.cc:1685] finished
0709 12:14:09.207936 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:09.208358 (+   422us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26968,"raft.queue_time_us":30436,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":16394,"thread_start_us":213987,"threads_started":1,"wal-append.queue_time_us":451594}
I20260709 12:14:09.262724 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:09.251569 22385 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.279681 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:09.287599 22172 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:09.288489 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4239) took 1100 ms. Trace:
I20260709 12:14:09.280874 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3720) took 1212 ms. Trace:
I20260709 12:14:09.290328 12531 rpcz_store.cc:276] 0709 12:14:08.068761 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.258899 (+1190138us) service_pool.cc:224] Handling call
0709 12:14:09.280791 (+ 21892us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.294411 22269 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:09.294858 12494 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:09.297619 12599 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 22 index: 17. (index mismatch)
I20260709 12:14:09.299371 22360 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:09.300060 22360 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:09.310705 12524 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:14:09.323813 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3723) took 1250 ms. Trace:
I20260709 12:14:09.326282 12524 rpcz_store.cc:276] 0709 12:14:08.069464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.279255 (+1209791us) service_pool.cc:224] Handling call
0709 12:14:09.320260 (+ 41005us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.331522 22385 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:09.334497 22172 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:14:09.337347 22279 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:14:09.337899 22279 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:09.337994 22385 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:09.339545 22279 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:09.342068 22279 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:09.342423 22327 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:14:09.343470 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 45631.9us
I20260709 12:14:09.344242 12643 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:09.287676 22317 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:14:09.337930 12676 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 30. Current term is 31. Ops: []
I20260709 12:14:09.345657 22317 raft_consensus.cc:697] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:14:09.346474 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.301594 22360 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:09.348699 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4484) took 1901 ms. Trace:
I20260709 12:14:09.352861 12676 rpcz_store.cc:276] 0709 12:14:07.444500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.272954 (+1828454us) service_pool.cc:224] Handling call
0709 12:14:09.337490 (+ 64536us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.345836 (+  8346us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.346095 (+   259us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.362560 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4238) took 1218 ms. Trace:
I20260709 12:14:09.364794 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:14:09.368158 12530 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3703) took 1311 ms (client timeout 1521 ms). Trace:
I20260709 12:14:09.373611 22287 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election won for term 31
W20260709 12:14:09.368722 12530 rpcz_store.cc:269] 0709 12:14:08.056832 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.056953 (+   121us) service_pool.cc:224] Handling call
0709 12:14:09.367606 (+1310653us) spinlock_profiling.cc:243] Waited 714 ms on lock 0x7b54001072d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:09.367878 (+   272us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1641421568}
I20260709 12:14:09.361946 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:09.375072 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3716) took 1305 ms. Trace:
I20260709 12:14:09.289696 12597 rpcz_store.cc:276] 0709 12:14:08.144350 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.840609 (+696259us) service_pool.cc:224] Handling call
0709 12:14:08.841032 (+   423us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:08.891649 (+ 50617us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:08.891696 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:08.926192 (+ 34496us) log.cc:844] Serialized 52 byte log entry
0709 12:14:09.210709 (+284517us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:09.211646 (+   937us) log.cc:844] Serialized 14 byte log entry
0709 12:14:09.212537 (+   891us) log.cc:844] Serialized 14 byte log entry
0709 12:14:09.212826 (+   289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.213008 (+   182us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:09.213124 (+   116us) raft_consensus.cc:1685] finished
0709 12:14:09.213135 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:09.244545 (+ 31410us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32975,"prepare.queue_time_us":33479,"prepare.run_cpu_time_us":91,"prepare.run_wall_time_us":2745,"raft.queue_time_us":103812,"raft.run_cpu_time_us":890,"raft.run_wall_time_us":34297,"spinlock_wait_cycles":2113280,"thread_start_us":228699,"threads_started":1,"wal-append.queue_time_us":198349}
I20260709 12:14:09.379482 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:09.383282 12523 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3707) took 1319 ms (client timeout 1510 ms). Trace:
I20260709 12:14:09.383556 22339 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:09.383780 12523 rpcz_store.cc:269] 0709 12:14:08.064253 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.064426 (+   173us) service_pool.cc:224] Handling call
0709 12:14:09.382891 (+1318465us) spinlock_profiling.cc:243] Waited 481 ms on lock 0x7b5400054454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:09.383100 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1105995008}
I20260709 12:14:09.385637 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:09.376812 22317 consensus_queue.cc:237] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:09.378748 12527 rpcz_store.cc:276] 0709 12:14:08.067615 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.364170 (+1296555us) service_pool.cc:224] Handling call
0709 12:14:09.373004 (+  8834us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.394034 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:09.385630 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3726) took 1299 ms. Trace:
I20260709 12:14:09.397701 12531 rpcz_store.cc:276] 0709 12:14:08.086851 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.319719 (+1232868us) service_pool.cc:224] Handling call
0709 12:14:09.385537 (+ 65818us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.386579 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:09.403107 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:09.403183 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3774) took 3044 ms. Trace:
I20260709 12:14:09.405514 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4479) took 2108 ms. Trace:
I20260709 12:14:09.410244 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:09.405993 12680 rpcz_store.cc:276] 0709 12:14:07.296689 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.752487 (+455798us) service_pool.cc:224] Handling call
0709 12:14:07.752788 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:08.608420 (+855632us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.404961 (+796541us) spinlock_profiling.cc:243] Waited 652 ms on lock 0x7b54002c4f54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:09.405324 (+   363us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1500552960}
I20260709 12:14:09.413272 12599 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:14:09.411537 12523 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:09.412014 22287 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:09.412678 22332 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:09.419482 22287 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:09.404798 12679 rpcz_store.cc:276] 0709 12:14:06.360074 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.926406 (+566332us) service_pool.cc:224] Handling call
0709 12:14:07.090769 (+164363us) spinlock_profiling.cc:243] Waited 1.13 ms on lock 0x7b58001f2c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:14:07.090989 (+   220us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:07.095112 (+  4123us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:07.095143 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:07.103032 (+  7889us) log.cc:844] Serialized 26 byte log entry
0709 12:14:07.352788 (+249756us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:07.579359 (+226571us) log.cc:844] Serialized 14 byte log entry
0709 12:14:07.579862 (+   503us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.402504 (+1822642us) spinlock_profiling.cc:243] Waited 3.73 ms on lock 0x7b54001acf54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:09.402612 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:09.402690 (+    78us) raft_consensus.cc:1685] finished
0709 12:14:09.402704 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:09.402972 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5915,"prepare.queue_time_us":161585,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":6825,"raft.run_cpu_time_us":401,"raft.run_wall_time_us":16579,"spinlock_wait_cycles":11173376,"thread_start_us":391353,"threads_started":2,"wal-append.queue_time_us":316171,"wal-append.run_cpu_time_us":3294,"wal-append.run_wall_time_us":1190612}
I20260709 12:14:09.428040 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4487) took 1725 ms. Trace:
I20260709 12:14:09.424198 22332 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:09.429677 22332 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:09.430364 12571 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:09.433259 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3789) took 2560 ms. Trace:
I20260709 12:14:09.428610 12680 rpcz_store.cc:276] 0709 12:14:07.702916 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.426420 (+1723504us) service_pool.cc:224] Handling call
0709 12:14:09.427067 (+   647us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.427576 (+   509us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:09.427615 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:09.427685 (+    70us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:14:09.427748 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.427812 (+    64us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:09.427953 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.439656 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3730) took 1286 ms. Trace:
W20260709 12:14:09.440917 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.451200 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:09.451632 22216 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:14:09.453354 12525 rpcz_store.cc:276] 0709 12:14:08.153547 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.386076 (+1232529us) service_pool.cc:224] Handling call
0709 12:14:09.439563 (+ 53487us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.457345 22277 raft_consensus.cc:3055] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:14:09.457932 22277 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:09.458706 22288 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:09.459487 22277 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:09.378365 22327 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 25.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.066s
I20260709 12:14:09.383630 22424 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.060s
I20260709 12:14:09.434649 22087 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:14:09.469240 22087 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.469823 22087 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:14:09.458964 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3792) took 1672 ms. Trace:
I20260709 12:14:09.445561 22129 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:09.433851 12671 rpcz_store.cc:276] 0709 12:14:06.872421 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.794508 (+922087us) service_pool.cc:224] Handling call
0709 12:14:07.839255 (+ 44747us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:08.702411 (+863156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.432418 (+730007us) spinlock_profiling.cc:243] Waited 785 ms on lock 0x7b5400013454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:09.433009 (+   591us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1806085376}
I20260709 12:14:09.483266 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:09.483590 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:09.484187 12530 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:09.484671 12671 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:09.428638 12679 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 21 index: 17. Preceding OpId from leader: term: 26 index: 20. (index mismatch)
I20260709 12:14:09.486846 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3791) took 1703 ms. Trace:
I20260709 12:14:09.477541 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:09.477885 22172 raft_consensus.cc:3055] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:14:09.490372 22172 raft_consensus.cc:740] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:09.492792 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:09.492511 22172 consensus_queue.cc:260] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.12, Last appended by leader: 12, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:09.493841 12527 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:14:09.479007 12674 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27. Ops: []
I20260709 12:14:09.494639 12528 raft_consensus.cc:2393] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 28: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:09.495049 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3725) took 1413 ms. Trace:
I20260709 12:14:09.472824 12680 rpcz_store.cc:276] 0709 12:14:07.786092 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.443951 (+1657859us) service_pool.cc:224] Handling call
0709 12:14:09.458070 (+ 14119us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.458550 (+   480us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:09.458577 (+    27us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:09.458643 (+    66us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:09.458689 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.458745 (+    56us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:09.458876 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.479764 12531 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27. Ops: [26.17-26.17]
I20260709 12:14:09.478349 22174 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:14:09.496990 22174 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:09.482096 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 107066us
W20260709 12:14:09.489995 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 79392.3us
I20260709 12:14:09.495314 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3715) took 1428 ms. Trace:
I20260709 12:14:09.499188 12527 rpcz_store.cc:276] 0709 12:14:08.081469 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.456076 (+1374607us) service_pool.cc:224] Handling call
0709 12:14:09.494866 (+ 38790us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.495352 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4486) took 1835 ms. Trace:
I20260709 12:14:09.504736 12525 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 29. Current term is 30. Ops: [29.15-29.15]
I20260709 12:14:09.487354 12679 rpcz_store.cc:276] 0709 12:14:07.783196 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.427595 (+1644399us) service_pool.cc:224] Handling call
0709 12:14:09.428075 (+   480us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.486489 (+ 58414us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.486759 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.498621 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3711) took 1432 ms. Trace:
I20260709 12:14:09.500372 12528 rpcz_store.cc:276] 0709 12:14:08.067359 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.361339 (+1293980us) service_pool.cc:224] Handling call
0709 12:14:09.495236 (+133897us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.507624 12531 rpcz_store.cc:276] 0709 12:14:08.066278 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.453195 (+1386917us) service_pool.cc:224] Handling call
0709 12:14:09.479344 (+ 26149us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:09.496362 (+ 17018us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.498479 (+  2117us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.502475 12674 rpcz_store.cc:276] 0709 12:14:07.660039 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.416874 (+1756835us) service_pool.cc:224] Handling call
0709 12:14:09.478432 (+ 61558us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.494872 (+ 16440us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.495266 (+   394us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.516505 12602 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
I20260709 12:14:09.455544 22216 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:09.518532 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3721) took 1447 ms. Trace:
I20260709 12:14:09.519110 12525 rpcz_store.cc:276] 0709 12:14:08.068996 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.468608 (+1399612us) service_pool.cc:224] Handling call
0709 12:14:09.482443 (+ 13835us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:09.514854 (+ 32411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.516250 (+  1396us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.521364 12523 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:14:09.528151 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:09.529490 22288 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:14:09.532380 22288 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:14:09.533236 22288 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:09.536633 22290 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.536314 22174 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:09.537457 22290 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:09.522361 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3728) took 1405 ms. Trace:
I20260709 12:14:09.539785 12523 rpcz_store.cc:276] 0709 12:14:08.118050 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.384782 (+1266732us) service_pool.cc:224] Handling call
0709 12:14:09.522279 (+137497us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.538086 22174 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:09.570736 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:09.574031 22168 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:14:09.576599 12528 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:14:09.577557 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3724) took 1501 ms. Trace:
I20260709 12:14:09.579821 22307 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:09.584015 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4494) took 1217 ms. Trace:
W20260709 12:14:09.585374 22406 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.981s	user 0.002s	sys 0.003s
I20260709 12:14:09.503453 22339 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.082s
I20260709 12:14:09.586831 22288 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:09.595458 13304 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:09.597033 22172 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:09.598902 22424 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:09.599294 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3794) took 1812 ms. Trace:
I20260709 12:14:09.599869 22424 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:09.539048 12524 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 26: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:09.600843 22424 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:14:09.601305 12524 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3714) took 1534 ms (client timeout 1973 ms). Trace:
W20260709 12:14:09.601784 12524 rpcz_store.cc:269] 0709 12:14:08.067014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.351886 (+1284872us) service_pool.cc:224] Handling call
0709 12:14:09.601220 (+249334us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.603050 22118 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:14:09.605535 22020 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:09.606518 22020 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:09.608474 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:09.584730 12679 rpcz_store.cc:276] 0709 12:14:08.366631 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.514707 (+1148076us) service_pool.cc:224] Handling call
0709 12:14:09.583024 (+ 68317us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.583512 (+   488us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:09.583555 (+    43us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:09.583639 (+    84us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:09.583699 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.583779 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:09.583925 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.599898 12671 rpcz_store.cc:276] 0709 12:14:07.786765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.485885 (+1699120us) service_pool.cc:224] Handling call
0709 12:14:09.554387 (+ 68502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.598759 (+ 44372us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:09.598823 (+    64us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:09.598916 (+    93us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:14:09.598993 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.599050 (+    57us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:09.599218 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.577998 12528 rpcz_store.cc:276] 0709 12:14:08.076502 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.508169 (+1431667us) service_pool.cc:224] Handling call
0709 12:14:09.575945 (+ 67776us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.577186 (+  1241us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.577482 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.573354 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:09.615721 12531 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 29 index: 15. Preceding OpId from leader: term: 30 index: 15. (term mismatch)
I20260709 12:14:09.611572 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4482) took 2223 ms. Trace:
I20260709 12:14:09.619519 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:09.643134 12531 pending_rounds.cc:85] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 14
I20260709 12:14:09.609632 12673 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:14:09.651280 12675 rpcz_store.cc:276] 0709 12:14:07.388603 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.916469 (+1527866us) service_pool.cc:224] Handling call
0709 12:14:08.944081 (+ 27612us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.098084 (+154003us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.611016 (+512932us) spinlock_profiling.cc:243] Waited 57.4 ms on lock 0x7b540016f3d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:09.611378 (+   362us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":131947264}
I20260709 12:14:09.656039 12530 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 32.
I20260709 12:14:09.656912 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3731) took 1159 ms. Trace:
W20260709 12:14:09.610946 12600 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4230) took 1469 ms (client timeout 1725 ms). Trace:
I20260709 12:14:09.657287 12530 rpcz_store.cc:276] 0709 12:14:08.497984 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.423596 (+925612us) service_pool.cc:224] Handling call
0709 12:14:09.656839 (+233243us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.660625 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:09.662683 12605 raft_consensus.cc:1240] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 28. Current term is 29. Ops: [28.18-28.18]
I20260709 12:14:09.663647 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:09.609694 22020 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:09.671739 22118 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:14:09.611960 22277 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:09.363034 12601 rpcz_store.cc:276] 0709 12:14:08.144053 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.850313 (+706260us) service_pool.cc:224] Handling call
0709 12:14:08.850879 (+   566us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:08.875380 (+ 24501us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:08.875434 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:08.892741 (+ 17307us) log.cc:844] Serialized 26 byte log entry
0709 12:14:09.219018 (+326277us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:14:09.260535 (+ 41517us) log.cc:844] Serialized 14 byte log entry
0709 12:14:09.322738 (+ 62203us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.328028 (+  5290us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:09.348935 (+ 20907us) raft_consensus.cc:1685] finished
0709 12:14:09.348952 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:09.362336 (+ 13384us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23543,"prepare.queue_time_us":52426,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":102,"raft.queue_time_us":60826,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":35523,"spinlock_wait_cycles":26640640,"thread_start_us":315361,"threads_started":2,"wal-append.queue_time_us":246813}
W20260709 12:14:09.657892 12600 rpcz_store.cc:269] 0709 12:14:08.141590 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.288567 (+146977us) service_pool.cc:224] Handling call
0709 12:14:09.610336 (+1321769us) spinlock_profiling.cc:243] Waited 35.8 ms on lock 0x7b540023fd54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:09.610683 (+   347us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":82322304}
I20260709 12:14:09.674738 12531 pending_rounds.cc:107] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 29.15
I20260709 12:14:09.689257 12531 raft_consensus.cc:2889] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:09.688169 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3798) took 1606 ms. Trace:
I20260709 12:14:09.695421 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4485) took 2142 ms. Trace:
I20260709 12:14:09.702369 22294 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 22.13, Next index: 14, Last known committed idx: 11, Time since last communication: 0.001s
I20260709 12:14:09.703994 22398 raft_consensus.cc:3055] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:14:09.704595 22398 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:09.706182 22398 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:09.707877 22398 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:14:09.708082 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.708668 22349 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:09.655864 22087 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:09.750831 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:09.761447 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.765918 12602 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 22. Current term is 23. Ops: [22.17-22.17]
I20260709 12:14:09.713559 21713 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:09.768846 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4483) took 2364 ms. Trace:
I20260709 12:14:09.769198 21713 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:09.771528 22349 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.772330 22349 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:09.773087 21713 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:09.696027 12676 rpcz_store.cc:276] 0709 12:14:07.547749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.357938 (+1810189us) service_pool.cc:224] Handling call
0709 12:14:09.358491 (+   553us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.359107 (+   616us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:14:09.376404 (+ 17297us) log.cc:844] Serialized 14 byte log entry
0709 12:14:09.688620 (+312216us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:09.688686 (+    66us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:09.688744 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.688819 (+    75us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:09.689523 (+   704us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1783,"prepare.queue_time_us":42848,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":64,"raft.queue_time_us":15615,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":17653,"thread_start_us":236052,"threads_started":2}
W20260709 12:14:09.778110 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 68195us
I20260709 12:14:09.778403 22332 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:09.779958 12675 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:09.780189 22378 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:09.780665 12571 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:09.781423 22332 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:09.779011 22349 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:09.777168 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3729) took 1620 ms. Trace:
W20260709 12:14:09.784644 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 211551us
I20260709 12:14:09.784435 12531 rpcz_store.cc:276] 0709 12:14:08.148339 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.518462 (+1370123us) service_pool.cc:224] Handling call
0709 12:14:09.615174 (+ 96712us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.765038 (+149864us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.768310 (+  3272us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:09.790181 22290 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:09.791333 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.792019 12673 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:14:09.769407 12677 rpcz_store.cc:276] 0709 12:14:07.388936 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.228165 (+1839229us) service_pool.cc:224] Handling call
0709 12:14:09.228930 (+   765us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.229638 (+   708us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:14:09.371116 (+141478us) log.cc:844] Serialized 14 byte log entry
0709 12:14:09.748553 (+377437us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:09.748641 (+    88us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:14:09.748897 (+   256us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.749028 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:09.753781 (+  4753us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":163920,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":92,"raft.queue_time_us":32891,"raft.run_cpu_time_us":217,"raft.run_wall_time_us":5299,"thread_start_us":392440,"threads_started":2}
I20260709 12:14:09.795061 12677 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 29 index: 15. Preceding OpId from leader: term: 30 index: 15. (term mismatch)
I20260709 12:14:09.795401 12677 pending_rounds.cc:85] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 14
I20260709 12:14:09.795702 12677 pending_rounds.cc:107] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 29.15
I20260709 12:14:09.795953 12677 raft_consensus.cc:2889] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:09.797240 21713 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:09.797987 21713 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:09.798477 12526 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3701) took 1719 ms (client timeout 1789 ms). Trace:
I20260709 12:14:09.798996 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:09.800372 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4243) took 1646 ms. Trace:
I20260709 12:14:09.800915 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:09.801081 13304 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:09.801827 12677 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 29: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:09.802646 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4480) took 2458 ms. Trace:
I20260709 12:14:09.802923 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:09.803297 12678 rpcz_store.cc:276] 0709 12:14:07.376713 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.754791 (+378078us) service_pool.cc:224] Handling call
0709 12:14:07.755084 (+   293us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.216070 (+1460986us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.802189 (+586119us) spinlock_profiling.cc:243] Waited 1.46 s on lock 0x7b54003bd654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:09.802561 (+   372us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3354201984}
I20260709 12:14:09.781807 22376 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 17, Time since last communication: 0.024s
I20260709 12:14:09.801088 12603 rpcz_store.cc:276] 0709 12:14:08.169124 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.645141 (+476017us) service_pool.cc:224] Handling call
0709 12:14:08.645539 (+   398us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:08.649907 (+  4368us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:08.649937 (+    30us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:08.649980 (+    43us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:08.650016 (+    36us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.799858 (+1149842us) spinlock_profiling.cc:243] Waited 4.16 ms on lock 0x7b54001617d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:09.799931 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:09.800141 (+   210us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9570048}
I20260709 12:14:09.800616 21713 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:09.805454 12678 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:14:09.811210 12677 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 26. Ops: []
I20260709 12:14:09.810271 12678 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26. Ops: []
I20260709 12:14:09.694947 12680 rpcz_store.cc:276] 0709 12:14:08.072859 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.496390 (+1423531us) service_pool.cc:224] Handling call
0709 12:14:09.496979 (+   589us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.497737 (+   758us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:14:09.498365 (+   628us) log.cc:844] Serialized 14 byte log entry
0709 12:14:09.678994 (+180629us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:09.679073 (+    79us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:14:09.679151 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.679242 (+    91us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:09.679393 (+   151us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":5771,"prepare.run_cpu_time_us":95,"prepare.run_wall_time_us":3141,"raft.queue_time_us":14248,"raft.run_cpu_time_us":200,"raft.run_wall_time_us":3960,"thread_start_us":176062,"threads_started":1}
I20260709 12:14:09.814065 22087 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:09.825086 22196 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:09.831477 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4495) took 1246 ms. Trace:
I20260709 12:14:09.832011 12675 rpcz_store.cc:276] 0709 12:14:08.584860 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.656469 (+1071609us) service_pool.cc:224] Handling call
0709 12:14:09.749969 (+ 93500us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.780364 (+ 30395us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.831156 (+ 50792us) spinlock_profiling.cc:243] Waited 29.8 ms on lock 0x7b54003bc754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:09.831398 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":68425472}
W20260709 12:14:09.833230 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.834653 22216 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:09.835065 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:09.837199 22216 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:09.838729 22376 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:14:09.842376 22087 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:14:09.848361 12674 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: [24.14-24.14]
I20260709 12:14:09.848694 12678 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:14:09.849743 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:09.850615 12674 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 21 FOLLOWER]: Advancing to term 23
I20260709 12:14:09.850996 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4471) took 2609 ms. Trace:
I20260709 12:14:09.851428 12672 rpcz_store.cc:276] 0709 12:14:07.241209 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.743455 (+502246us) service_pool.cc:224] Handling call
0709 12:14:07.743771 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:08.958832 (+1215061us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.850551 (+891719us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b5400013e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:09.850836 (+   285us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2704468352}
W20260709 12:14:09.798976 12526 rpcz_store.cc:269] 0709 12:14:08.056131 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:08.079928 (+ 23797us) service_pool.cc:224] Handling call
0709 12:14:09.747978 (+1668050us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b54003f7d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:09.775065 (+ 27087us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2046948224}
I20260709 12:14:09.849800 13331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 1097) took 2115 ms. Trace:
W20260709 12:14:09.868429 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 80141.6us
W20260709 12:14:09.869580 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 114467us
I20260709 12:14:09.780668 22378 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:09.868564 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 65325.7us
I20260709 12:14:09.863070 13331 rpcz_store.cc:276] 0709 12:14:07.734055 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:07.735645 (+  1590us) service_pool.cc:224] Handling call
0709 12:14:07.735931 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:08.896754 (+1160823us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:09.849286 (+952532us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400690354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:09.849551 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2667756288}
I20260709 12:14:09.896831 13304 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:09.899295 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.878640 22378 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:09.913481 22345 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:09.918435 12494 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:09.920588 12494 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:09.928942 12646 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:09.931510 22307 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:14:09.931969 22307 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:09.932327 22307 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:14:09.934491 13336 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 873) took 3043 ms (client timeout 1616 ms). Trace:
W20260709 12:14:09.934953 13336 rpcz_store.cc:269] 0709 12:14:06.890822 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:06.891006 (+   184us) service_pool.cc:224] Handling call
0709 12:14:09.876450 (+2985444us) spinlock_profiling.cc:243] Waited 933 ms on lock 0x7b540063ecd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:09.934251 (+ 57801us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2147006848}
I20260709 12:14:09.936350 22054 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:14:09.936962 22054 raft_consensus.cc:697] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 24 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:09.938635 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:09.943706 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 373502us
I20260709 12:14:09.938242 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:09.931985 12646 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:14:09.907011 12496 connection.cc:256] RPC call timeout handler was delayed by -1.0437289219999002s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1674s for a response
I20260709 12:14:09.893708 22339 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:09.967639 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:09.969698 12680 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:14:09.973762 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:09.946244 22327 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:14:09.975530 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:09.973634 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:09.977571 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:09.977113 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 22 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:09.987286 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:09.989511 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:09.937992 22054 consensus_queue.cc:237] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 24, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:09.990600 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:09.991176 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:09.991878 13336 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:14:09.991958 12530 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:09.990226 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:09.950210 22339 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:09.994307 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:14:09.990912 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.977792 22327 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:10.001660 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:14:10.005357 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:09.948213 22360 consensus_queue.cc:1059] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 27.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.048s
I20260709 12:14:10.013718 12524 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:10.023223 22313 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:10.023881 22313 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:10.024287 22313 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:10.030514 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:09.991884 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:09.950461 22424 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:09.999354 22339 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:09.974053 22020 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 22.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.037s
I20260709 12:14:10.038537 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:10.007001 22327 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:10.031298 12529 raft_consensus.cc:2376] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 32. Re-sending same reply.
W20260709 12:14:10.035491 22285 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.426s	user 0.010s	sys 0.018s
I20260709 12:14:10.044004 12523 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:14:10.045742 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:10.005005 22398 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.048843 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.054544 22329 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.055625 13336 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 30 index: 21. (index mismatch)
I20260709 12:14:10.055519 22329 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:10.058272 22168 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:14:10.062882 12526 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: []
I20260709 12:14:10.072517 12531 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 30. Current term is 31. Ops: []
I20260709 12:14:10.015235 22172 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.059s
W20260709 12:14:10.075997 22447 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.082s	user 0.001s	sys 0.000s
I20260709 12:14:10.075759 22242 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:14:10.076121 12526 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 30 index: 15. (index mismatch)
I20260709 12:14:10.077656 22375 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.078460 22375 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:10.078990 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.048744 22087 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:10.083043 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.086515 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:10.048259 12530 raft_consensus.cc:2410] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 25 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
W20260709 12:14:10.089772 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 57831.5us
I20260709 12:14:10.100160 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.088907 12529 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
W20260709 12:14:10.045941 22285 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.481s	user 0.011s	sys 0.018s
I20260709 12:14:10.106408 12523 raft_consensus.cc:3055] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:14:10.106985 12523 raft_consensus.cc:740] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:10.112329 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.118637 12524 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:10.120180 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:10.120682 22293 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.121372 22293 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:10.120676 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:10.126240 22293 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:10.127063 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:10.126585 22277 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.129890 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.131520 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.132637 12530 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 31.
I20260709 12:14:10.130570 22277 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:10.134274 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:10.134644 22421 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.135378 22421 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:10.136355 22196 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 30.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:14:10.137945 22260 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:14:10.138619 22421 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:10.139640 12522 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:14:10.137051 12530 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:10.140964 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.141824 12522 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 30: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:10.142264 22087 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:10.144745 22087 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:10.114699 22398 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:10.147933 22398 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:10.103336 22290 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.046s
W20260709 12:14:10.151479 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.135035 22247 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.153499 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:10.128363 12523 consensus_queue.cc:260] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:10.181460 12525 raft_consensus.cc:2393] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 29: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:10.126893 22168 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:14:10.150826 22332 raft_consensus.cc:3055] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:14:10.149608 22277 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:10.200444 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 85025.2us
I20260709 12:14:10.205374 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.224551 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.227473 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.138545 22260 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:10.157683 22424 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:10.135514 22172 consensus_queue.cc:1243] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 27.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:14:10.226469 12524 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:10.233843 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:10.239017 12523 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:10.241349 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:10.242020 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.242317 12525 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:14:10.243556 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.244716 12525 raft_consensus.cc:2410] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 28 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 14.
I20260709 12:14:10.249436 22288 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:10.250290 22288 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:10.250780 12522 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 28: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:10.252310 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.253432 22288 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:10.254935 22339 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:10.258874 22260 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.16, Last appended by leader: 16, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:10.259603 22398 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 30.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:14:10.260210 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:10.260761 22260 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:10.260986 22398 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:14:10.262356 22376 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:14:10.262645 22398 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:10.262825 22376 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:10.263463 22307 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:10.263937 22376 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:10.265475 22376 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:10.266999 13334 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:14:10.267961 22279 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:14:10.268554 22279 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:10.274545 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.276248 22145 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:10.277271 22145 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:10.280191 22145 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:10.281066 22318 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:14:10.282315 22247 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:10.285264 22247 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:10.288239 22375 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:10.229655 22329 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:10.290616 13330 raft_consensus.cc:2410] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 27 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 17.
I20260709 12:14:10.203341 22168 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:10.282311 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 60483.6us
W20260709 12:14:10.292712 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.302340 12680 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
W20260709 12:14:10.306363 12671 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s	user 0.015s	sys 0.023s
W20260709 12:14:10.306893 12671 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.015s	sys 0.023s
I20260709 12:14:10.260273 12496 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:10.298074 12529 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:14:10.204830 22332 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:10.299060 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:14:10.313871 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.315968 12528 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 30.
W20260709 12:14:10.299512 12599 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.720s	user 0.052s	sys 0.050s
W20260709 12:14:10.319036 12599 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.740s	user 0.053s	sys 0.050s
I20260709 12:14:10.245977 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.323380 12530 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
I20260709 12:14:10.281842 22318 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:10.324774 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.325838 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:10.329528 12674 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:14:10.331229 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:10.333920 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:10.317827 12597 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.541s	user 0.000s	sys 0.037s
W20260709 12:14:10.336206 12597 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.562s	user 0.000s	sys 0.038s
I20260709 12:14:10.338150 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.339123 12528 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:14:10.335558 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3800) took 1724 ms. Trace:
I20260709 12:14:10.326694 12522 raft_consensus.cc:2376] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27. Re-sending same reply.
I20260709 12:14:10.341355 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1114) took 1132 ms. Trace:
I20260709 12:14:10.354303 12677 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 22. Current term is 23. Ops: []
I20260709 12:14:10.341267 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:10.359042 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.359885 12530 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:14:10.230443 22196 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 24.12, Next index: 13, Last known committed idx: 12, Time since last communication: 0.093s
I20260709 12:14:10.367923 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:10.368882 12603 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 26: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:10.363461 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:10.375705 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:10.376772 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.269766 22279 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:10.378058 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:10.379204 12528 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: []
I20260709 12:14:10.380386 22174 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:14:10.380796 22087 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.381603 22087 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:10.385074 12572 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:10.387490 12572 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:10.389631 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:10.395056 22279 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:10.357856 22375 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election won for term 32
W20260709 12:14:10.395932 22450 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.153s	user 0.003s	sys 0.000s
I20260709 12:14:10.322546 22307 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:10.334918 22339 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:10.361291 22216 consensus_meta.cc:287] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.517s	user 0.008s	sys 0.011s
I20260709 12:14:10.397889 12529 raft_consensus.cc:2410] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 27 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 12:14:10.397938 12571 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:10.340898 12671 rpcz_store.cc:276] 0709 12:14:08.610886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.662046 (+1051160us) service_pool.cc:224] Handling call
0709 12:14:09.662511 (+   465us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.662962 (+   451us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:14:09.672393 (+  9431us) log.cc:844] Serialized 14 byte log entry
0709 12:14:10.334858 (+662465us) log.cc:844] Serialized 14 byte log entry
0709 12:14:10.335135 (+   277us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:10.335197 (+    62us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:10.335273 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:10.335358 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:10.335490 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":19570,"prepare.run_cpu_time_us":73,"prepare.run_wall_time_us":76,"raft.queue_time_us":15001,"thread_start_us":634425,"threads_started":1}
I20260709 12:14:10.353652 22294 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:10.399060 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 22 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:10.412009 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:10.453635 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 75162.1us
I20260709 12:14:10.353519 12599 rpcz_store.cc:276] 0709 12:14:09.209303 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.446803 (+237500us) service_pool.cc:224] Handling call
0709 12:14:09.467678 (+ 20875us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.578450 (+110772us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:14:09.579022 (+   572us) log.cc:844] Serialized 14 byte log entry
0709 12:14:10.340727 (+761705us) log.cc:844] Serialized 14 byte log entry
0709 12:14:10.340969 (+   242us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:10.341020 (+    51us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:14:10.341083 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:10.341150 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:10.341295 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":6130,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":20642,"spinlock_wait_cycles":35050368,"thread_start_us":740043,"threads_started":1}
W20260709 12:14:10.409680 12676 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.599s	user 0.002s	sys 0.042s
I20260709 12:14:10.418883 22398 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:14:10.420363 22118 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:14:10.421887 12530 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 28. Current term is 29. Ops: [28.17-28.17]
I20260709 12:14:10.457890 12523 raft_consensus.cc:1275] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:14:10.433283 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:10.436053 13334 raft_consensus.cc:2393] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 23: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:14:10.438098 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:10.421334 22329 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:14:10.451609 12672 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.027s	sys 0.025s
W20260709 12:14:10.469507 12672 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.561s	user 0.027s	sys 0.025s
W20260709 12:14:10.456964 12676 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.646s	user 0.003s	sys 0.043s
W20260709 12:14:10.471637 12679 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.500s	user 0.028s	sys 0.032s
W20260709 12:14:10.472096 12679 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.502s	user 0.028s	sys 0.032s
W20260709 12:14:10.477144 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 545222us
I20260709 12:14:10.444051 22375 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:10.478773 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:10.484133 12523 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:14:10.479535 12673 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.656s	user 0.013s	sys 0.026s
I20260709 12:14:10.480746 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.472854 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3802) took 1853 ms. Trace:
I20260709 12:14:10.485208 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.485621 12679 rpcz_store.cc:276] 0709 12:14:08.619765 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.668866 (+1049101us) service_pool.cc:224] Handling call
0709 12:14:09.669596 (+   730us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:09.670051 (+   455us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:14:09.671744 (+  1693us) log.cc:844] Serialized 14 byte log entry
0709 12:14:10.472473 (+800729us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:10.472529 (+    56us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:14:10.472579 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:10.472664 (+    85us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:10.472793 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7466,"prepare.queue_time_us":1073,"prepare.run_cpu_time_us":65,"prepare.run_wall_time_us":65,"raft.queue_time_us":37986,"raft.run_cpu_time_us":351,"raft.run_wall_time_us":99277,"thread_start_us":774389,"threads_started":2,"wal-append.queue_time_us":567326}
W20260709 12:14:10.488422 12673 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.665s	user 0.014s	sys 0.027s
I20260709 12:14:10.498198 12528 raft_consensus.cc:2393] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 31: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:10.500037 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:10.501471 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:10.503069 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 57611.6us
W20260709 12:14:10.504509 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 244324us
W20260709 12:14:10.507313 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:10.508006 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.512549 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:10.467216 22118 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:10.511499 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:10.513307 13330 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:14:10.518211 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:10.504021 12531 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 30.
I20260709 12:14:10.478137 22216 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election lost for term 22. Reason: could not achieve majority
I20260709 12:14:10.441335 22196 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.062s
W20260709 12:14:10.523104 12675 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.673s	user 0.006s	sys 0.030s
W20260709 12:14:10.524479 12675 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.006s	sys 0.031s
W20260709 12:14:10.526299 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 136881us
I20260709 12:14:10.495945 22375 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:10.529451 12524 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 27: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:10.530579 12671 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 28: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:10.492825 22398 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:14:10.535826 12572 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:10.541370 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:10.543869 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:10.546396 12524 raft_consensus.cc:2410] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 30 because replica has last-logged OpId of term: 28 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 19.
I20260709 12:14:10.473757 22168 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:10.552552 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:10.554960 12678 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.702s	user 0.038s	sys 0.057s
I20260709 12:14:10.556202 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:10.557754 12678 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.705s	user 0.039s	sys 0.057s
W20260709 12:14:10.558774 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 196848us
W20260709 12:14:10.554080 22378 consensus_meta.cc:287] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.578s	user 0.010s	sys 0.003s
I20260709 12:14:10.566974 12671 raft_consensus.cc:2410] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 25 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:14:10.483258 22332 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:10.571806 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:10.594520 22242 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:10.595779 22242 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:10.602737 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:10.617311 12496 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:10.619580 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 58856.7us
I20260709 12:14:10.611115 22294 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:10.517025 22318 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:10.636096 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 26 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:10.637965 12531 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:14:10.630729 22294 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:14:10.643000 22454 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.106s	user 0.000s	sys 0.001s
I20260709 12:14:10.646306 22296 consensus_queue.cc:1059] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.165s
I20260709 12:14:10.647488 22296 raft_consensus.cc:3055] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:14:10.647894 22296 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:10.649246 22296 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 22.17, Last appended by leader: 17, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:10.646492 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:10.653117 22294 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:10.659257 22294 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:10.659436 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:10.661536 22294 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:14:10.661537 12680 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:14:10.667026 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:10.629549 22452 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.298s	user 0.001s	sys 0.002s
I20260709 12:14:10.696885 12680 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:14:10.697592 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:10.698887 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 63780.2us
I20260709 12:14:10.712706 12522 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:14:10.633253 22247 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:14:10.572495 22378 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:10.717343 12677 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 29: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:10.717914 12572 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:10.721094 12527 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.868s	user 0.064s	sys 0.166s
W20260709 12:14:10.721565 12527 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.869s	user 0.064s	sys 0.166s
I20260709 12:14:10.721629 12680 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 26. Current term is 27. Ops: [26.18-26.18]
I20260709 12:14:10.724509 12523 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
I20260709 12:14:10.730017 12522 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
W20260709 12:14:10.731518 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 96095.6us
I20260709 12:14:10.726408 12494 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:10.709713 22196 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
W20260709 12:14:10.769135 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 164718us
W20260709 12:14:10.740439 22287 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.029s	sys 0.055s
I20260709 12:14:10.719522 22247 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:10.778226 12569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.016s	sys 0.010s
W20260709 12:14:10.778774 12569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.658s	user 0.017s	sys 0.010s
W20260709 12:14:10.780812 22345 consensus_meta.cc:287] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.866s	user 0.014s	sys 0.005s
I20260709 12:14:10.781142 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:10.754804 12571 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:10.775130 22287 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.634s	user 0.029s	sys 0.055s
I20260709 12:14:10.606482 22118 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:10.746440 22172 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:10.775452 22296 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:14:10.784148 22293 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.002s	sys 0.027s
W20260709 12:14:10.777077 22129 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.655s	user 0.000s	sys 0.026s
I20260709 12:14:10.766489 22294 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:10.783684 22290 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:10.790365 22054 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.715s	user 0.011s	sys 0.034s
I20260709 12:14:10.810034 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:10.783597 22247 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:10.789855 22329 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:14:10.793401 22360 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.712s	user 0.012s	sys 0.052s
I20260709 12:14:10.820861 12496 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:10.794471 22332 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:14:10.809588 22293 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.003s	sys 0.027s
I20260709 12:14:10.769095 22216 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.039s
W20260709 12:14:10.812808 22129 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.690s	user 0.000s	sys 0.027s
I20260709 12:14:10.822243 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:14:10.827723 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:10.818784 22054 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.748s	user 0.011s	sys 0.034s
W20260709 12:14:10.885999 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:10.824007 22360 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.771s	user 0.012s	sys 0.052s
I20260709 12:14:10.793453 22307 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 19, Time since last communication: 0.044s
W20260709 12:14:10.897989 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 266907us
W20260709 12:14:10.900197 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 97032.9us
W20260709 12:14:10.908049 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 91945.9us
W20260709 12:14:10.835640 22438 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.749s	user 0.013s	sys 0.031s
I20260709 12:14:10.835812 22172 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:10.821748 22290 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:10.897670 22294 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:10.918567 22438 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.098s	user 0.013s	sys 0.032s
I20260709 12:14:10.913055 22465 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.916435 22360 raft_consensus.cc:3055] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:14:10.933374 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:14:10.928012 13304 leader_election.cc:336] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.523s (SENT)
I20260709 12:14:10.904910 22196 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:10.915357 22287 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:10.921414 22118 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:14:10.945732 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 434001us
I20260709 12:14:10.950330 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:10.941344 22360 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:14:10.955928 13305 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.660s	user 0.015s	sys 0.039s
W20260709 12:14:10.968221 13305 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.819s	user 0.016s	sys 0.040s
W20260709 12:14:10.968744 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 819928us
I20260709 12:14:10.947824 22287 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:10.977423 12602 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.643s	user 0.040s	sys 0.070s
W20260709 12:14:10.977926 12602 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.643s	user 0.040s	sys 0.070s
W20260709 12:14:10.972697 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46426.1us
W20260709 12:14:11.021370 22460 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.124s	user 0.000s	sys 0.001s
W20260709 12:14:11.021956 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 238705us
I20260709 12:14:10.963790 22398 raft_consensus.cc:697] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:10.965166 22378 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:10.989342 22277 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:10.952402 22294 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:10.952697 22465 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:11.023978 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 212061us
I20260709 12:14:11.022905 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:11.032141 12496 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:11.023516 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3817) took 1176 ms. Trace:
W20260709 12:14:11.022853 22459 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.115s	user 0.000s	sys 0.000s
I20260709 12:14:10.982313 22054 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader election won for term 23
W20260709 12:14:11.046908 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:10.985821 22287 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:11.053066 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:11.029798 22327 consensus_meta.cc:287] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.933s	user 0.012s	sys 0.004s
I20260709 12:14:11.023634 22129 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:11.059973 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 259348us
W20260709 12:14:11.063618 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.053412 12675 rpcz_store.cc:276] 0709 12:14:09.847399 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.847600 (+   201us) service_pool.cc:224] Handling call
0709 12:14:09.848055 (+   455us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:14:09.848626 (+   571us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:09.848656 (+    30us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:14:09.849513 (+   857us) log.cc:844] Serialized 78 byte log entry
0709 12:14:10.524975 (+675462us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:10.525033 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:10.525141 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:11.023217 (+498076us) raft_consensus.cc:1685] finished
0709 12:14:11.023239 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:11.023448 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1716,"raft.run_cpu_time_us":102,"raft.run_wall_time_us":102,"thread_start_us":674861,"threads_started":1,"wal-append.queue_time_us":1043484}
I20260709 12:14:11.014849 22172 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:11.073395 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:11.076066 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 132562us
I20260709 12:14:11.077324 13304 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:10.994483 12416 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 70b8c9b89e3d43fbb70422110d76702c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica a2fdbe7eef6748a9b4611f6c9bc9b201 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 24 committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:11.052626 22329 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:14:11.085534 12416 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.ChangeConfig -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 136203us
I20260709 12:14:10.984387 22196 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:11.101643 12569 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:11.080124 22345 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:11.080319 22464 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:11.102099 22449 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.276s	user 0.001s	sys 0.001s
I20260709 12:14:11.104383 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:11.123111 12671 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:11.080246 22054 raft_consensus.cc:697] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 23 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:11.110687 13305 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:11.122022 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 99227.8us
W20260709 12:14:11.143069 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 65809.8us
I20260709 12:14:11.143992 13304 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:11.145361 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 69808.6us
I20260709 12:14:11.147280 12646 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:11.067729 22327 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:11.150559 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:11.124048 22260 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.863s	user 0.016s	sys 0.000s
I20260709 12:14:11.114395 22293 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:14:11.135341 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 111856us
I20260709 12:14:11.080134 22378 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.041s
I20260709 12:14:11.112466 22468 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.084899 22172 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:11.164513 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3811) took 1571 ms. Trace:
I20260709 12:14:11.173300 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:11.176571 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 100120us
I20260709 12:14:11.179369 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:11.181219 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:11.192330 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:11.103077 22129 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:11.151341 22485 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:11.197337 12526 raft_consensus.cc:2368] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24.
I20260709 12:14:11.196070 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:11.199977 12531 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
W20260709 12:14:11.203258 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 101738us
I20260709 12:14:11.194401 12530 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 31.
I20260709 12:14:11.136650 22360 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 29.15, Last appended by leader: 15, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:11.184578 22424 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:11.173133 12673 rpcz_store.cc:276] 0709 12:14:09.592776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.793260 (+200484us) service_pool.cc:224] Handling call
0709 12:14:09.793869 (+   609us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:09.821882 (+ 28013us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:09.821926 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:09.822596 (+   670us) log.cc:844] Serialized 26 byte log entry
0709 12:14:10.489463 (+666867us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:14:10.489548 (+    85us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:10.489660 (+   112us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:11.164268 (+674608us) raft_consensus.cc:1685] finished
0709 12:14:11.164292 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:11.164454 (+   162us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":904,"raft.run_cpu_time_us":108,"raft.run_wall_time_us":110,"thread_start_us":665929,"threads_started":1,"wal-append.queue_time_us":977428}
I20260709 12:14:11.212625 22490 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:11.222379 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 117505us
I20260709 12:14:11.206786 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:11.228286 12600 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:14:11.228987 22196 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:11.208835 12603 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28. Ops: []
W20260709 12:14:11.229755 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 56555.9us
I20260709 12:14:11.222766 22339 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:14:11.231788 12572 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:11.233254 22488 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:11.233671 22327 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:11.211156 22485 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:14:11.234145 22488 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:14:11.214998 22463 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.250s	user 0.000s	sys 0.001s
I20260709 12:14:11.216192 22216 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:14:11.238870 22216 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:14:11.239923 22461 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.118s	user 0.000s	sys 0.000s
I20260709 12:14:11.240064 22216 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.16, Last appended by leader: 16, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:11.216576 22458 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.131s	user 0.000s	sys 0.001s
I20260709 12:14:11.244784 13336 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:11.139600 22464 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:11.246170 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3820) took 1397 ms. Trace:
I20260709 12:14:11.251139 13305 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:11.246619 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3722) took 3177 ms. Trace:
W20260709 12:14:11.251888 12643 leader_election.cc:336] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.682s (SENT)
I20260709 12:14:11.248049 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:11.253887 12643 leader_election.cc:336] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.720s (SENT)
I20260709 12:14:11.258888 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:14:11.256561 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 112655us
I20260709 12:14:11.211874 22424 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:11.208303 22360 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:14:11.264253 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 118128us
I20260709 12:14:11.252158 12527 rpcz_store.cc:276] 0709 12:14:08.069235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.517094 (+1447859us) service_pool.cc:224] Handling call
0709 12:14:09.686595 (+169501us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:09.774066 (+ 87471us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:09.774097 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:09.852425 (+ 78328us) log.cc:844] Serialized 26 byte log entry
0709 12:14:10.727692 (+875267us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:10.727770 (+    78us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:10.727871 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:11.246419 (+518548us) raft_consensus.cc:1685] finished
0709 12:14:11.246440 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:11.246563 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":6404,"raft.run_cpu_time_us":358,"raft.run_wall_time_us":69499,"thread_start_us":869107,"threads_started":1,"wal-append.queue_time_us":1207944}
I20260709 12:14:11.274032 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:11.207417 22478 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:14:11.275125 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:11.212255 22375 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.670s	user 0.012s	sys 0.000s
I20260709 12:14:11.278636 12675 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:14:11.278697 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:11.279817 12674 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:14:11.274101 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3670) took 1145 ms. Trace:
I20260709 12:14:11.249716 12678 rpcz_store.cc:276] 0709 12:14:09.848351 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.851474 (+  3123us) service_pool.cc:224] Handling call
0709 12:14:09.851813 (+   339us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:09.852216 (+   403us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:09.852249 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:09.852791 (+   542us) log.cc:844] Serialized 26 byte log entry
0709 12:14:10.558927 (+706136us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:10.558983 (+    56us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:10.559103 (+   120us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:11.245938 (+686835us) raft_consensus.cc:1685] finished
0709 12:14:11.245960 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:11.246092 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":134,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":155,"thread_start_us":705657,"threads_started":1,"wal-append.queue_time_us":1067298}
I20260709 12:14:11.275942 12673 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:14:11.278059 12677 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:11.151859 22398 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 21.16, Last appended by leader: 16, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:11.214471 22129 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:11.280766 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:11.269663 22455 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.305s	user 0.000s	sys 0.001s
I20260709 12:14:11.284026 22482 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:14:11.284595 22471 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.093s	user 0.001s	sys 0.000s
W20260709 12:14:11.295826 22376 consensus_meta.cc:287] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.030s	user 0.017s	sys 0.000s
W20260709 12:14:11.296490 22457 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.269s	user 0.000s	sys 0.001s
I20260709 12:14:11.297693 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3809) took 1781 ms. Trace:
I20260709 12:14:11.301698 12676 rpcz_store.cc:276] 0709 12:14:09.524498 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.782310 (+257812us) service_pool.cc:224] Handling call
0709 12:14:09.808987 (+ 26677us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:09.809528 (+   541us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:09.809559 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:09.810366 (+   807us) log.cc:844] Serialized 26 byte log entry
0709 12:14:10.471127 (+660761us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:14:10.471170 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:10.471238 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:11.297466 (+826228us) raft_consensus.cc:1685] finished
0709 12:14:11.297488 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:11.297632 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":11574,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":135,"thread_start_us":660214,"threads_started":1,"wal-append.queue_time_us":1097255}
I20260709 12:14:11.283298 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3671) took 1153 ms. Trace:
I20260709 12:14:11.294497 22465 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:11.306565 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:11.304939 12528 rpcz_store.cc:276] 0709 12:14:10.129524 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.562768 (+433244us) service_pool.cc:224] Handling call
0709 12:14:10.575200 (+ 12432us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:10.618468 (+ 43268us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:14:10.954446 (+335978us) log.cc:844] Serialized 14 byte log entry
0709 12:14:11.277923 (+323477us) log.cc:844] Serialized 14 byte log entry
0709 12:14:11.282933 (+  5010us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:11.282991 (+    58us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:11.283043 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:11.283111 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:11.283239 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9970,"prepare.queue_time_us":477524,"prepare.run_cpu_time_us":80,"prepare.run_wall_time_us":82,"raft.queue_time_us":91641,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":5393,"spinlock_wait_cycles":25595648,"thread_start_us":469747,"threads_started":3,"wal-append.queue_time_us":190931}
I20260709 12:14:11.279830 22478 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:11.315719 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:11.312479 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:11.275760 22485 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.244333 22504 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.060s
I20260709 12:14:11.283733 22488 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.320101 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:11.320524 22285 raft_consensus.cc:3055] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:14:11.321624 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:11.215816 22489 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.083s
I20260709 12:14:11.290506 22294 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:11.322176 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1144) took 1098 ms. Trace:
I20260709 12:14:11.206110 22468 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:11.281116 12525 rpcz_store.cc:276] 0709 12:14:10.129189 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.375342 (+246153us) service_pool.cc:224] Handling call
0709 12:14:10.422384 (+ 47042us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:10.452163 (+ 29779us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:14:10.989210 (+537047us) log.cc:844] Serialized 14 byte log entry
0709 12:14:11.273689 (+284479us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:11.273750 (+    61us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:14:11.273827 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:11.273900 (+    73us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:11.274031 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":605538,"prepare.run_cpu_time_us":4251,"prepare.run_wall_time_us":7204,"raft.queue_time_us":38743,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":14581,"thread_start_us":676311,"threads_started":3}
I20260709 12:14:11.228798 22494 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 13, Time since last communication: 0.076s
W20260709 12:14:11.340998 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 53158.1us
I20260709 12:14:11.339097 12598 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:14:11.338701 12679 rpcz_store.cc:276] 0709 12:14:10.223505 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.700494 (+476989us) service_pool.cc:224] Handling call
0709 12:14:10.709053 (+  8559us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:10.746355 (+ 37302us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:10.746403 (+    48us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:10.811779 (+ 65376us) log.cc:844] Serialized 26 byte log entry
0709 12:14:11.094337 (+282558us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:11.094396 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:11.094519 (+   123us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:11.321908 (+227389us) raft_consensus.cc:1685] finished
0709 12:14:11.321930 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:11.322085 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11495,"raft.queue_time_us":29636,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":9132,"thread_start_us":192987,"threads_started":1,"wal-append.queue_time_us":362249}
I20260709 12:14:11.263823 22490 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:11.290490 22375 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:11.317327 22313 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.293s	user 0.016s	sys 0.001s
I20260709 12:14:11.366410 12646 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:11.361096 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:11.266906 22329 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:11.317198 22478 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:11.388685 12604 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:14:11.362283 12601 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:14:11.362821 22497 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:11.313192 22129 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.313246 22318 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:11.340765 22466 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.321s	user 0.002s	sys 0.000s
I20260709 12:14:11.361860 12525 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 29. Current term is 30. Ops: [29.15-29.15]
I20260709 12:14:11.323167 22087 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:11.392968 12528 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
W20260709 12:14:11.401309 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.403525 12600 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:14:11.356745 22168 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:14:11.420243 22296 consensus_meta.cc:287] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.607s	user 0.010s	sys 0.001s
I20260709 12:14:11.425288 22172 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:11.426072 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1119) took 1776 ms. Trace:
I20260709 12:14:11.426712 12597 rpcz_store.cc:276] 0709 12:14:09.650025 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.650245 (+   220us) service_pool.cc:224] Handling call
0709 12:14:09.675328 (+ 25083us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:09.711588 (+ 36260us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:09.711643 (+    55us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:09.745665 (+ 34022us) log.cc:844] Serialized 26 byte log entry
0709 12:14:10.336831 (+591166us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:14:10.336877 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:10.336949 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:11.425844 (+1088895us) raft_consensus.cc:1685] finished
0709 12:14:11.425868 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:11.426016 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17067,"raft.queue_time_us":20530,"raft.run_cpu_time_us":120,"raft.run_wall_time_us":119,"thread_start_us":562357,"threads_started":1,"wal-append.queue_time_us":1164368}
I20260709 12:14:11.428853 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:11.429666 12597 raft_consensus.cc:2393] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 29: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:11.430886 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:11.432116 13333 raft_consensus.cc:2410] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 30 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
I20260709 12:14:11.433771 22490 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:11.339476 22285 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:11.289901 22054 consensus_queue.cc:237] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 22.14, Last appended by leader: 14, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:11.436141 12494 connection.cc:256] RPC call timeout handler was delayed by -1.4733197170000949s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1506s for a response
W20260709 12:14:11.436606 12494 connection.cc:256] RPC call timeout handler was delayed by -1.2258197169999221s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1781s for a response
W20260709 12:14:11.436913 12494 connection.cc:256] RPC call timeout handler was delayed by -1.2159197169999061s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17920000000000003s for a response
I20260709 12:14:11.438488 22174 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.411477 22506 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:11.439138 22506 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:11.439033 22174 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:14:11.439633 22506 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:11.439906 22087 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.411978 12599 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 31. Current term is 32. Ops: []
I20260709 12:14:11.414429 22482 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:14:11.443014 22464 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:11.403993 22488 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:11.386422 22485 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:11.417963 22375 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:11.340564 22491 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:11.422950 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:11.423589 22468 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:11.423772 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:11.424510 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:11.442147 22129 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:11.443893 12494 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:11.324321 22465 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.082s
W20260709 12:14:11.455422 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 51011.7us
I20260709 12:14:11.456014 12601 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:14:11.454866 12605 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:11.440469 22087 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:11.478715 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:11.474848 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:11.481377 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:11.483217 12602 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 22.
I20260709 12:14:11.479887 12597 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29.
W20260709 12:14:11.485926 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.489255 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:11.492323 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:11.489887 12601 raft_consensus.cc:2376] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26. Re-sending same reply.
I20260709 12:14:11.498064 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:11.499547 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 61502us
I20260709 12:14:11.499858 22502 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:14:11.500044 22247 consensus_meta.cc:287] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.667s	user 0.029s	sys 0.006s
I20260709 12:14:11.500620 12494 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:11.501475 12646 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:11.502964 22465 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:11.503257 12646 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:11.493417 12605 raft_consensus.cc:2376] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24. Re-sending same reply.
I20260709 12:14:11.504076 22464 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:11.504462 22464 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:11.504843 12598 raft_consensus.cc:2410] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 25 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 12:14:11.504915 22464 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:11.500634 22247 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:11.479012 22174 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:11.507036 22174 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:14:11.507946 22247 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:11.503400 22465 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:11.508872 22465 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:11.509238 22087 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:11.501977 12494 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:11.504490 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:11.511807 12605 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:11.512939 22174 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:11.513406 22174 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.514123 22174 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:11.514719 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:11.404549 22313 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:11.518627 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:11.518898 22313 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:11.521555 22488 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:11.522207 22172 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
W20260709 12:14:11.523450 22279 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.080s	user 0.014s	sys 0.006s
W20260709 12:14:11.532060 22118 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.578s	user 0.011s	sys 0.000s
I20260709 12:14:11.539672 22293 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:11.542678 12571 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:11.544356 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.544975 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:11.545405 22345 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:11.545657 12675 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:11.451546 22318 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:11.543040 22332 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.715s	user 0.013s	sys 0.000s
I20260709 12:14:11.547053 22118 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.547626 22118 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:11.548472 22318 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:11.550463 22129 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:11.550874 22376 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:11.551591 22376 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.552091 22376 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:11.544198 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:11.553745 12679 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:11.558542 12671 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
I20260709 12:14:11.574260 12674 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:11.584425 12494 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:11.591778 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:11.593176 12597 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:14:11.600639 12605 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:14:11.489802 22485 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:11.605050 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 331090us
I20260709 12:14:11.608759 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:11.608698 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:11.611624 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.608672 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:11.612185 12597 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:11.614411 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:11.614945 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 111754us
I20260709 12:14:11.615986 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:11.616506 12601 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:11.616653 12602 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 24
I20260709 12:14:11.620353 12605 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 32.
I20260709 12:14:11.621142 12646 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:11.615315 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:11.610687 22287 consensus_meta.cc:287] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.521s	user 0.016s	sys 0.012s
I20260709 12:14:11.602303 22424 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:11.629009 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:11.622314 12678 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:14:11.605564 22172 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.611097 22293 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:11.553517 22345 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.072s
I20260709 12:14:11.633761 12494 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:11.629940 12679 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:14:11.472781 22491 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:11.639638 12597 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29.
I20260709 12:14:11.648710 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:11.650539 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:11.652045 12605 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:11.639256 22293 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:11.650910 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 154603us
I20260709 12:14:11.628657 22313 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:14:11.650089 22318 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:14:11.724570 22504 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:14:11.727799 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 94108.1us
I20260709 12:14:11.641150 22172 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:11.631080 22287 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:11.731452 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:11.729967 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:11.714350 22293 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:11.649986 22174 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:11.734879 12494 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:11.750481 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:11.714056 22313 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:11.746945 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 125901us
I20260709 12:14:11.739480 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:11.721832 22118 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:11.750690 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:11.751806 12646 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:11.751461 13333 raft_consensus.cc:2410] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 27 because replica has last-logged OpId of term: 26 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:14:11.751358 12602 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:14:11.760610 12597 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:11.762883 12680 raft_consensus.cc:2368] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 26. Current term is 27.
I20260709 12:14:11.735708 22494 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.765347 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:11.740674 22485 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:11.772871 12605 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 32.
I20260709 12:14:11.775041 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 23 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:11.775710 12602 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:14:11.760618 22279 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:11.777390 12531 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 31: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:11.753116 22313 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:11.795826 12646 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:11.803696 12597 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:11.807668 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:11.822178 22501 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.099s
W20260709 12:14:11.822628 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 43608.4us
I20260709 12:14:11.774912 22327 raft_consensus.cc:2764] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election decision vote started in defunct term 31: won
I20260709 12:14:11.782012 22465 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:11.786548 22485 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:11.788479 22172 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:11.825489 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:11.778719 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:11.790544 22216 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:14:11.831483 22174 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:11.833920 12646 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:11.835198 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:11.836961 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:11.839232 22491 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:11.824872 22360 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.548s	user 0.011s	sys 0.000s
I20260709 12:14:11.825066 22478 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:11.777024 22494 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:11.847841 22287 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:11.825163 22332 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.027s
I20260709 12:14:11.852605 22247 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:14:11.853708 22498 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.639s	user 0.000s	sys 0.012s
I20260709 12:14:11.907464 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:11.908618 12605 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:11.903475 22495 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:14:11.910877 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 75733us
I20260709 12:14:11.910802 22495 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:11.912012 12494 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:14:11.854911 22512 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
I20260709 12:14:11.930917 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:14:11.855435 22493 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.643s	user 0.003s	sys 0.000s
I20260709 12:14:11.932250 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:11.856853 22196 raft_consensus.cc:3055] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:14:11.933434 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:11.858875 22285 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:11.862067 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1127) took 1233 ms. Trace:
I20260709 12:14:11.934520 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:11.935189 12599 rpcz_store.cc:276] 0709 12:14:10.628614 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.635755 (+  7141us) service_pool.cc:224] Handling call
0709 12:14:10.650550 (+ 14795us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:11.441842 (+791292us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:11.861765 (+419923us) spinlock_profiling.cc:243] Waited 758 ms on lock 0x7b540007b554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:11.861994 (+   229us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1743518976}
I20260709 12:14:11.935953 13330 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 31.
I20260709 12:14:11.934077 12529 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
I20260709 12:14:11.936815 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:11.937702 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:11.791461 22279 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:11.938333 22279 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:11.934538 22174 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:14:11.938854 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:11.934641 22491 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:14:11.939682 22491 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:11.940061 22491 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:11.940019 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:11.941370 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:11.882730 12602 raft_consensus.cc:2410] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 29 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 15.
I20260709 12:14:11.941926 12529 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:14:11.942595 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:11.942804 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:11.943264 12606 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:14:11.943596 12529 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 28: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:11.943718 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:11.944162 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:11.944448 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:11.944761 12606 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 23.
I20260709 12:14:11.944847 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:11.945724 12527 raft_consensus.cc:2410] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 30 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
I20260709 12:14:11.945593 22290 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:11.971324 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:11.897998 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 285648us
I20260709 12:14:11.909776 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:11.927030 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:11.966313 22468 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:11.973981 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 47092us
I20260709 12:14:11.927933 22277 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:11.928468 22485 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:11.929728 22481 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.723s	user 0.004s	sys 0.003s
I20260709 12:14:11.931726 22478 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:11.889256 22339 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:11.947937 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:11.948231 22465 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:11.949404 22247 raft_consensus.cc:697] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:11.933687 22196 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:12.003392 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3675) took 1872 ms. Trace:
I20260709 12:14:11.934973 22285 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:11.970844 22293 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.005108 12598 raft_consensus.cc:2393] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 24: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:12.006137 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:12.006716 22293 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:12.007145 22242 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:11.874099 12601 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:14:11.935091 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:11.942808 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:12.008507 12602 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 31: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:12.009099 12597 raft_consensus.cc:2393] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 30: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:12.009649 12602 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:11.962164 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:11.954422 22522 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 31.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.005s
I20260709 12:14:11.945739 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:12.009032 12604 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 30 index: 15. (index mismatch)
W20260709 12:14:11.954497 22459 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
I20260709 12:14:11.978933 12527 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 29.
I20260709 12:14:11.981248 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:11.983103 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:11.988121 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:12.017984 13333 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:11.988785 22468 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.020897 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:11.982978 22277 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:11.980010 22332 raft_consensus.cc:2764] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election decision vote started in defunct term 22: won
I20260709 12:14:12.025050 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:12.025277 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:12.026054 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:11.997692 12677 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:12.026739 12605 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:14:12.000602 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:11.963110 22495 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:12.003026 22196 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.026475 13305 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:12.004485 22507 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:14:12.007401 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:11.970834 22464 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:11.962657 12572 connection.cc:256] RPC call timeout handler was delayed by -1.2385696349997488s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17880000000000001s for a response
W20260709 12:14:11.955672 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 121852us
I20260709 12:14:12.011169 12606 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:14:11.957815 22494 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:12.016696 12569 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:12.003801 12524 rpcz_store.cc:276] 0709 12:14:10.130784 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.567501 (+436717us) service_pool.cc:224] Handling call
0709 12:14:10.605401 (+ 37900us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:10.640725 (+ 35324us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:10.640782 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:10.710961 (+ 70179us) log.cc:844] Serialized 26 byte log entry
0709 12:14:11.079529 (+368568us) raft_consensus.cc:1645] Marking committed up to 12
0709 12:14:11.080693 (+  1164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:11.080762 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:11.928883 (+848121us) spinlock_profiling.cc:243] Waited 14.4 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:14:12.003087 (+ 74204us) raft_consensus.cc:1685] finished
0709 12:14:12.003110 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:12.003332 (+   222us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":12413,"raft.queue_time_us":15527,"raft.run_cpu_time_us":154,"raft.run_wall_time_us":39259,"spinlock_wait_cycles":33155840,"thread_start_us":265048,"threads_started":1,"wal-append.queue_time_us":418383}
I20260709 12:14:12.037147 12602 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 23. Current term is 24. Ops: []
I20260709 12:14:12.037462 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:11.968899 22376 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.023834 12494 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:11.998862 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:12.028568 12599 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26. Ops: []
W20260709 12:14:12.083289 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 59579.4us
I20260709 12:14:12.029268 12598 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:14:12.008332 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 21 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:12.026499 22327 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:12.032110 22507 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:12.089278 22525 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:14:12.089816 22525 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.090289 22525 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:12.089977 22507 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.090929 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:12.091388 22507 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:12.091676 12522 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:14:12.092803 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:12.093469 12522 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
W20260709 12:14:12.094501 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 61299.1us
I20260709 12:14:12.094710 22485 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.095479 12646 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:12.095892 22480 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.066s	user 0.000s	sys 0.005s
I20260709 12:14:12.026309 22478 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:12.097551 22478 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:12.032107 22260 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.016s
I20260709 12:14:12.098515 22478 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:12.098397 22490 consensus_queue.cc:1048] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:14:12.025218 22174 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.023s
I20260709 12:14:12.099979 22174 raft_consensus.cc:3055] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:14:12.100430 22174 raft_consensus.cc:740] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:12.101545 22174 consensus_queue.cc:260] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.102233 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:12.084833 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:12.103055 12530 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
I20260709 12:14:12.084365 12494 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:12.086402 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 49009.2us
I20260709 12:14:12.106014 12572 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:12.103155 12524 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 31.
I20260709 12:14:12.109946 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:12.108579 22490 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.003s
I20260709 12:14:12.112463 22174 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:12.096112 22495 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:14:12.096599 22287 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:12.002139 22247 consensus_queue.cc:237] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.12, Last appended by leader: 12, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.086918 22476 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:12.114470 22476 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.115025 22476 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:12.115782 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:12.113765 22287 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:12.117606 22287 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:12.019294 22488 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.042s
I20260709 12:14:12.095402 22485 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.118564 22495 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:14:12.118948 22495 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:12.119805 22495 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:12.120309 22485 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:12.121070 22495 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:12.121208 12494 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:12.122328 12598 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 29. Current term is 30. Ops: []
I20260709 12:14:12.122785 22494 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:14:12.123358 22494 raft_consensus.cc:697] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:12.113773 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 24 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:12.124274 22494 consensus_queue.cc:237] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.125195 12523 raft_consensus.cc:2410] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 28 because replica has last-logged OpId of term: 26 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 12.
I20260709 12:14:12.125828 22172 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:12.126977 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:12.127554 12522 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:12.088016 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:12.088594 13302 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:12.110859 12572 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:12.128445 12646 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:12.134810 12529 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:14:12.135316 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:12.136842 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:12.133965 22503 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:12.133641 22504 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 28.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 12:14:12.138978 22503 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:12.141700 22462 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.230s	user 0.002s	sys 0.000s
I20260709 12:14:12.136859 22360 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:12.144816 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:12.145732 12679 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
I20260709 12:14:12.123389 22293 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:12.136531 22118 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.146512 22293 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.146983 22293 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:12.146683 22118 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.149593 22118 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:12.086079 22196 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:12.151554 22087 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:12.126438 22172 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:12.162576 22522 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:14:12.163013 22522 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:12.163317 12600 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:14:12.163924 22522 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.22, Last appended by leader: 22, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:12.146265 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:12.166277 22501 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.166380 12604 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 29.
I20260709 12:14:12.167177 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 1148) took 1246 ms. Trace:
I20260709 12:14:12.166955 22501 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:12.168025 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 141665us
I20260709 12:14:12.168603 22424 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:12.169090 22424 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.169492 22424 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:12.167811 12604 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:14:12.171263 12604 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 24 index: 12. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:14:12.161420 22172 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.172454 22504 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:14:12.172892 22504 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:12.165266 22522 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:12.167567 12680 rpcz_store.cc:276] 0709 12:14:10.920447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.920588 (+   141us) service_pool.cc:224] Handling call
0709 12:14:12.166948 (+1246360us) spinlock_profiling.cc:243] Waited 823 ms on lock 0x7b54000c3ad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:12.167083 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1893998976}
W20260709 12:14:12.182430 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47179us
I20260709 12:14:12.160076 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:12.185665 12572 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:12.187484 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 59113.9us
I20260709 12:14:12.192531 12675 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:14:12.193518 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:12.193763 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 55552.7us
I20260709 12:14:12.194080 13333 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:14:12.195377 12571 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:12.195403 12572 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:12.199892 12600 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
W20260709 12:14:12.203420 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 114915us
I20260709 12:14:12.219369 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:12.204051 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 116121us
I20260709 12:14:12.224493 12643 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:12.197322 22360 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:12.231696 12604 raft_consensus.cc:2410] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 26 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:14:12.201347 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:12.178750 22503 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:12.234536 12646 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:12.197664 22490 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:14:12.248425 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.249248 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:12.197813 22329 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:12.197860 22482 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:12.205649 22527 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:14:12.207036 22488 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:12.203263 22485 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:12.206053 22260 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.203687 22464 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:12.230322 22506 consensus_meta.cc:287] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.790s	user 0.015s	sys 0.000s
I20260709 12:14:12.229363 22168 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.228791 22508 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:14:12.236325 22360 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.239049 22285 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.238569 22118 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:12.270548 22488 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.279745 22504 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.14, Last appended by leader: 14, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:12.284508 22529 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.176s	user 0.000s	sys 0.001s
I20260709 12:14:12.286870 22500 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.043s
I20260709 12:14:12.291298 22279 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.296276 22313 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.250672 22490 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:12.298584 22087 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:12.300195 22478 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:12.304466 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.306417 22307 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:12.294325 22287 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.253526 22329 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:12.320779 12600 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 26. Current term is 27. Ops: []
I20260709 12:14:12.256336 22527 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:12.325966 12522 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
I20260709 12:14:12.332279 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3821) took 2483 ms. Trace:
W20260709 12:14:12.333133 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 137772us
I20260709 12:14:12.366226 12572 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:12.328327 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 127066us
W20260709 12:14:12.282567 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 60564.2us
I20260709 12:14:12.282347 22485 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:12.321813 22360 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:14:12.333052 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 94512us
I20260709 12:14:12.324528 22376 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:12.257939 22375 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:12.327320 22488 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:12.368948 12643 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:12.318224 22506 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:12.377556 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:12.376549 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 101378us
I20260709 12:14:12.353628 22490 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:12.380468 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:12.319234 22260 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.382421 12530 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 30. Current term is 31. Ops: []
I20260709 12:14:12.354465 22087 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.327683 22285 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.365592 12672 rpcz_store.cc:276] 0709 12:14:09.848600 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:09.852369 (+  3769us) service_pool.cc:224] Handling call
0709 12:14:09.852706 (+   337us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:09.853189 (+   483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:09.853233 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:09.889142 (+ 35909us) log.cc:844] Serialized 26 byte log entry
0709 12:14:10.470353 (+581211us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:10.470423 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:10.470541 (+   118us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:12.140783 (+1670242us) spinlock_profiling.cc:243] Waited 1.87 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:14:12.319516 (+178733us) raft_consensus.cc:1685] finished
0709 12:14:12.319550 (+    34us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:12.332153 (+ 12603us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1215,"raft.run_cpu_time_us":329,"raft.run_wall_time_us":24919,"spinlock_wait_cycles":4295040,"thread_start_us":561896,"threads_started":1,"wal-append.queue_time_us":922538}
I20260709 12:14:12.327931 22168 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.362675 22307 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:12.338521 22118 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.374037 22329 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:12.386659 22502 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:12.456219 22525 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.459219 12606 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:14:12.409417 22087 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:12.402709 22491 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.464938 22424 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:12.465325 22491 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:12.466415 22527 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:12.411748 22376 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:12.467617 22527 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:12.377821 22468 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:12.397907 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:12.400471 22504 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:12.482687 12675 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
I20260709 12:14:12.431674 12597 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: [25.15-28.16]
I20260709 12:14:12.439718 22547 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:14:12.442353 22478 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:12.443322 22540 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
I20260709 12:14:12.411468 22307 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:12.450553 22506 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:12.450783 22277 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:12.452185 22287 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:12.452771 22545 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.375816 22501 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:12.474056 13333 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 22 index: 17. Preceding OpId from leader: term: 23 index: 17. (term mismatch)
I20260709 12:14:12.475551 22118 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:12.430836 22470 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.413s	user 0.000s	sys 0.003s
I20260709 12:14:12.443677 22285 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:12.504794 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4506) took 1212 ms. Trace:
I20260709 12:14:12.498163 22216 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:14:12.515918 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 118105us
I20260709 12:14:12.524303 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:12.524534 12599 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:12.516932 13333 pending_rounds.cc:85] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 16
I20260709 12:14:12.453564 22502 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:12.497433 22547 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.520282 12674 rpcz_store.cc:276] 0709 12:14:11.291975 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:11.304094 (+ 12119us) service_pool.cc:224] Handling call
0709 12:14:11.573966 (+269872us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:11.574860 (+   894us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:12.504488 (+929628us) spinlock_profiling.cc:243] Waited 251 ms on lock 0x7b54000aafd0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:12.504694 (+   206us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":576680192}
I20260709 12:14:12.470664 22398 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:12.526816 22476 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:12.527184 22247 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:14:12.536424 22525 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:12.536584 22476 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:12.538589 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:12.539296 13338 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:12.540446 22502 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:12.525070 13333 pending_rounds.cc:107] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 22.17
I20260709 12:14:12.543869 13333 raft_consensus.cc:2889] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:12.515254 22478 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:14:12.611377 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 84651.3us
I20260709 12:14:12.545053 22287 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:14:12.543136 22547 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:14:12.616312 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:12.600445 22458 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.063s	user 0.001s	sys 0.000s
W20260709 12:14:12.620168 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 244775us
W20260709 12:14:12.607435 22507 consensus_meta.cc:287] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.516s	user 0.016s	sys 0.001s
I20260709 12:14:12.620591 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:12.624675 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:12.623240 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:12.609988 22279 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:12.631902 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 251477us
I20260709 12:14:12.633607 12674 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 26. Current term is 27. Ops: []
I20260709 12:14:12.617309 22506 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.639042 12675 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 23 index: 15. (index mismatch)
I20260709 12:14:12.521502 22545 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.616873 22539 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.019s
I20260709 12:14:12.647938 12672 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 26. Current term is 27. Ops: [26.18-26.18]
I20260709 12:14:12.627163 22168 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:12.635025 22468 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:14:12.659301 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.551503 22485 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.14, Last appended by leader: 14, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.651124 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:12.659322 12572 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:14:12.677492 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.678349 22467 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.062s
I20260709 12:14:12.681440 13305 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:12.683025 22549 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:14:12.683884 22549 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:12.685357 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:12.685091 22549 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.19, Last appended by leader: 19, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:12.688831 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:12.689754 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:12.691145 22552 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
W20260709 12:14:12.692620 12526 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 3657) took 2624 ms (client timeout 1788 ms). Trace:
W20260709 12:14:12.693055 12526 rpcz_store.cc:269] 0709 12:14:10.068042 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:10.078561 (+ 10519us) service_pool.cc:224] Handling call
0709 12:14:12.692334 (+2613773us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540038f9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:12.692506 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2557227264}
I20260709 12:14:12.694691 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:12.694928 22482 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:12.626044 22478 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:12.697479 22478 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:12.698367 12600 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:12.699128 22277 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:12.701314 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:12.702525 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:12.703910 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:12.705221 22293 consensus_meta.cc:287] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.558s	user 0.013s	sys 0.000s
I20260709 12:14:12.705189 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:12.707947 12606 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:12.689626 12527 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:14:12.709014 12597 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 26: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:12.636726 22313 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:12.711927 13338 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 32.
W20260709 12:14:12.712426 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:12.714859 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:12.715680 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.719957 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4250) took 1596 ms. Trace:
I20260709 12:14:12.723169 12676 raft_consensus.cc:2393] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 24: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:12.727937 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:12.639060 22424 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:12.734521 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:12.720542 12603 rpcz_store.cc:276] 0709 12:14:11.123515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:11.123720 (+   205us) service_pool.cc:224] Handling call
0709 12:14:11.208628 (+ 84908us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:11.229664 (+ 21036us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:12.719713 (+1490049us) spinlock_profiling.cc:243] Waited 84.5 ms on lock 0x7b54002e59d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:12.719879 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":194405120}
W20260709 12:14:12.737169 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.739419 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:12.741948 12676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 4508) took 1426 ms (client timeout 1552 ms). Trace:
W20260709 12:14:12.742894 12676 rpcz_store.cc:269] 0709 12:14:11.315482 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.694310 (+1378828us) service_pool.cc:224] Handling call
0709 12:14:12.741855 (+ 47545us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:12.745693 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:12.741009 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:12.663961 22468 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.705761 22293 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:12.737054 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 86019.2us
I20260709 12:14:12.646003 22488 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:12.757969 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 98732.9us
I20260709 12:14:12.755971 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:12.707751 22260 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:12.759428 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:12.649278 22506 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:12.762771 12675 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:14:12.772765 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:12.773488 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:12.739145 22478 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:12.733924 22491 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:12.764232 12572 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:12.744728 22277 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:12.765551 22508 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:12.773954 12671 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: []
I20260709 12:14:12.763914 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:12.802683 12680 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:12.749737 22172 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:12.780529 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:12.750676 22468 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:12.788957 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:12.788990 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:12.778599 12494 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:12.808965 12597 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:12.789776 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:12.795614 12572 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:12.800499 22491 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.810420 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:12.811924 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:12.812597 12600 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:12.812763 22491 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:12.773800 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3709) took 1122 ms. Trace:
W20260709 12:14:12.814015 22544 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.279s	user 0.000s	sys 0.001s
I20260709 12:14:12.814302 12598 rpcz_store.cc:276] 0709 12:14:11.651907 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.098472 (+446565us) service_pool.cc:224] Handling call
0709 12:14:12.098776 (+   304us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:12.122732 (+ 23956us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:12.773554 (+650822us) spinlock_profiling.cc:243] Waited 23.3 ms on lock 0x7b5400374954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:12.773741 (+   187us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":53702016}
I20260709 12:14:12.815420 22546 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.815541 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:12.816541 12598 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:12.816244 22546 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:12.818279 22548 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.150s	user 0.001s	sys 0.000s
I20260709 12:14:12.819113 22279 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:14:12.820230 22279 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:14:12.820576 22279 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:12.826287 22260 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:12.826823 22260 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.827190 22260 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:12.828387 22559 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:12.770613 22398 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:12.807785 12678 raft_consensus.cc:2393] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 30: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:12.829283 22398 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.829707 22398 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:12.830211 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:12.830890 12678 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:14:12.799576 12679 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 25. Ops: []
I20260709 12:14:12.835556 12679 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:12.839023 22279 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.839813 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:12.840787 22279 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:12.842228 22172 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:12.810420 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:12.843851 12675 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 29: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:12.833719 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:12.840782 12605 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:12.853209 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:12.828399 22329 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.771616 22087 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.859651 22172 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:12.860339 22329 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:12.860641 22087 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:12.798079 22196 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:12.862464 22196 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:12.863013 22196 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:12.796316 22478 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:12.868907 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:12.876430 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 117894us
I20260709 12:14:12.797555 22522 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.078s
I20260709 12:14:12.797092 22378 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.072s
I20260709 12:14:12.882594 12680 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:12.881489 12598 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 30.
I20260709 12:14:12.884485 13302 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:12.860364 22469 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:12.904778 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:12.905316 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:12.906986 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.916805 12677 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 28: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:12.910748 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:12.918439 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:12.919647 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:12.924659 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:12.927943 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:12.928658 12676 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:14:12.933954 12496 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:12.935273 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:12.894429 22497 raft_consensus.cc:2804] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:12.936635 12680 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 26: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:12.937330 12597 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:14:12.903882 22313 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:12.939723 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 144237us
I20260709 12:14:12.941012 12496 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:12.941097 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:12.929163 22327 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:12.944494 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:12.956184 12496 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:12.957573 12496 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:12.939455 22168 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:12.932632 22054 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 28.21, Next index: 22, Last known committed idx: 20, Time since last communication: 0.022s
I20260709 12:14:12.943753 22497 raft_consensus.cc:697] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 30 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:12.944017 22313 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:12.948414 22558 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:12.945006 22277 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:12.959571 22174 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:14:12.955327 22507 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:14:12.957121 22556 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:12.961606 22550 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:12.977517 12572 leader_election.cc:400] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24.
I20260709 12:14:12.963369 22502 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election won for term 26
W20260709 12:14:12.982620 22551 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.212s	user 0.000s	sys 0.001s
I20260709 12:14:12.975835 22504 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
I20260709 12:14:13.040966 22504 raft_consensus.cc:3055] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:14:13.041539 22504 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:12.962436 22547 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.042584 22504 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:13.043730 22478 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:13.044266 22547 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:12.975394 22360 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.579s	user 0.011s	sys 0.000s
I20260709 12:14:13.045142 22360 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:13.045598 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:12.977344 22293 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:13.047679 22360 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:12.989689 22556 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:12.971917 22545 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:13.051064 22556 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:13.051467 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1132) took 1015 ms. Trace:
I20260709 12:14:13.052835 12679 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 28: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:12.995476 12496 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:13.054740 22550 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:13.059547 22277 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.004911 22502 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:13.062755 22502 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:13.063361 12606 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
W20260709 12:14:13.063751 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 68355.2us
I20260709 12:14:13.066969 12675 raft_consensus.cc:2393] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 24: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:13.067775 12496 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:13.088635 12496 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:13.101012 12572 leader_election.cc:403] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:13.044579 22504 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:13.052183 12604 rpcz_store.cc:276] 0709 12:14:12.035968 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.256876 (+220908us) service_pool.cc:224] Handling call
0709 12:14:12.257242 (+   366us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:12.257548 (+   306us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:12.257587 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:12.272939 (+ 15352us) log.cc:844] Serialized 26 byte log entry
0709 12:14:12.504438 (+231499us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:12.504475 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:12.504546 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:13.051246 (+546700us) raft_consensus.cc:1685] finished
0709 12:14:13.051266 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:13.051401 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":141041,"threads_started":1,"wal-append.queue_time_us":281845}
I20260709 12:14:13.117923 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:13.137624 22562 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.167s	user 0.001s	sys 0.000s
W20260709 12:14:13.032575 22527 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.549s	user 0.012s	sys 0.000s
I20260709 12:14:13.147812 13334 raft_consensus.cc:2410] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 28 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
W20260709 12:14:13.148562 22490 consensus_meta.cc:287] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.739s	user 0.007s	sys 0.004s
I20260709 12:14:13.137864 22478 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:13.151845 22545 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:12.971961 22546 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:13.155033 12600 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 33.
I20260709 12:14:13.117847 12496 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:13.176471 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58718.2us
I20260709 12:14:13.155544 22487 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election lost for term 23. Reason: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24.
I20260709 12:14:13.201514 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:13.209326 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:13.194918 22174 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:13.179200 22553 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:13.171154 22478 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:13.174335 22547 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:14:13.215202 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 242297us
I20260709 12:14:13.227178 12572 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:13.178455 22490 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.195119 22506 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:13.238394 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:13.238953 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:13.239557 12600 raft_consensus.cc:2368] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26.
I20260709 12:14:13.239712 12604 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:13.221166 22478 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:13.245308 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:13.249339 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:13.253883 12673 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:13.249027 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:13.255756 12602 raft_consensus.cc:2393] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 28: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:13.202468 22497 consensus_queue.cc:237] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:13.265754 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:13.266211 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:13.245469 22487 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:14:13.269340 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:13.270016 12602 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:13.269829 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:13.222091 22553 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.273897 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:13.274485 12606 raft_consensus.cc:2376] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26. Re-sending same reply.
I20260709 12:14:13.311216 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:14:13.322302 22568 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:14:13.321552 22260 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:13.325893 22508 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.520s	user 0.008s	sys 0.001s
I20260709 12:14:13.268962 22487 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:13.243726 22547 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:14:13.332391 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:13.327822 22279 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:13.338876 22487 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:13.338948 22279 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.350565 12606 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:13.329655 22313 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:13.345139 22398 consensus_meta.cc:287] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.515s	user 0.010s	sys 0.006s
I20260709 12:14:13.355432 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:13.358542 12597 raft_consensus.cc:2376] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29. Re-sending same reply.
I20260709 12:14:13.335398 22508 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:13.352761 22507 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:13.358036 22279 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:13.362771 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:13.363056 22507 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:13.364208 12602 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:14:13.347024 22494 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:13.363752 22553 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:13.374293 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:14:13.377853 22196 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.514s	user 0.006s	sys 0.007s
I20260709 12:14:13.378401 22196 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.381230 22196 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:13.361996 22550 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:13.363897 22260 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:13.386986 12678 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26. Ops: []
I20260709 12:14:13.365633 22398 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.396261 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:13.371590 22494 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:13.409653 12600 raft_consensus.cc:2368] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26.
I20260709 12:14:13.409732 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:13.416431 12597 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
W20260709 12:14:13.409665 22468 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.584s	user 0.008s	sys 0.004s
W20260709 12:14:13.421373 13304 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.106s (SENT) [suppressed 33 similar messages]
I20260709 12:14:13.414795 22490 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:13.427372 22398 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:13.420264 22468 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.430790 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3710) took 1699 ms. Trace:
I20260709 12:14:13.431638 22563 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:14:13.432252 22563 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:13.432775 22468 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:13.433462 22494 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:13.435603 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:13.431368 12601 rpcz_store.cc:276] 0709 12:14:11.731414 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.094377 (+362963us) service_pool.cc:224] Handling call
0709 12:14:12.094847 (+   470us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:12.259567 (+164720us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:14:12.351171 (+ 91604us) log.cc:844] Serialized 14 byte log entry
0709 12:14:12.622658 (+271487us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:12.631269 (+  8611us) log.cc:844] Serialized 52 byte log entry
0709 12:14:12.631455 (+   186us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:12.631709 (+   254us) log.cc:844] Serialized 14 byte log entry
0709 12:14:12.647326 (+ 15617us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:12.647426 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:12.827422 (+179996us) spinlock_profiling.cc:243] Waited 10.2 ms on lock 0x7b180034be80. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485933522 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:14:13.427336 (+599914us) spinlock_profiling.cc:243] Waited 1.91 ms on lock 0x7b180034be80. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:14:13.430402 (+  3066us) raft_consensus.cc:1685] finished
0709 12:14:13.430426 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:13.430601 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28472,"prepare.queue_time_us":96232,"prepare.run_cpu_time_us":104,"prepare.run_wall_time_us":9320,"raft.queue_time_us":19075,"raft.run_cpu_time_us":406,"raft.run_wall_time_us":6562,"spinlock_wait_cycles":35217280,"thread_start_us":378246,"threads_started":2,"wal-append.queue_time_us":260950}
I20260709 12:14:13.443665 12602 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 30. Current term is 31. Ops: [30.15-30.15]
I20260709 12:14:13.433736 22563 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 28.21, Last appended by leader: 21, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:13.452642 22307 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.929s	user 0.013s	sys 0.001s
I20260709 12:14:13.453002 22563 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:13.453140 22307 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.451973 12601 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:14:13.457450 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:13.452013 22494 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.459317 12494 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:13.471848 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:13.473320 12606 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:14:13.473933 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:13.475589 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:13.475852 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:13.476248 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:13.477151 22547 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:14:13.477485 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:13.499125 12606 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:14:13.500677 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:13.501940 12601 raft_consensus.cc:2393] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 24: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:13.502883 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:13.503993 12602 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 29: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:13.506243 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:13.506973 12598 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:13.463649 22504 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
W20260709 12:14:13.510247 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 88866.1us
I20260709 12:14:13.510488 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:13.512269 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:13.512554 12606 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 28: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:13.513578 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:13.507190 22572 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.099s	user 0.001s	sys 0.003s
I20260709 12:14:13.500631 22260 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:13.501075 22497 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.516988 22545 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:13.524049 22545 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:13.523953 22497 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:13.524488 22545 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:13.526238 22497 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:13.530822 12526 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24. Ops: []
I20260709 12:14:13.517447 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:13.506461 22553 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.521322 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:13.523209 22527 consensus_queue.cc:1059] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 26.18, Next index: 19, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:14:13.518796 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:13.537508 12598 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:14:13.585718 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:13.586606 12598 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 29.
I20260709 12:14:13.540022 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:13.509217 22196 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.545482 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:13.547236 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:13.520023 22174 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:13.547904 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:13.549887 22507 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:13.550961 22260 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:14:13.552717 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.544521 22504 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:13.553835 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:14:13.565014 22502 consensus_meta.cc:287] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.502s	user 0.009s	sys 0.002s
I20260709 12:14:13.514222 22313 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.485368 22487 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.566682 12606 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:13.478885 22494 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.583487 12569 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:13.628135 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 44741.3us
I20260709 12:14:13.590317 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:13.611328 12526 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:14:13.603588 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.612604 12528 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:13.613106 12531 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:13.613968 22129 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:13.627036 22507 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.631917 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:13.636889 12522 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:14:13.640388 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.647709 12604 raft_consensus.cc:2368] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
I20260709 12:14:13.543665 22054 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.093s
I20260709 12:14:13.659266 12522 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:14:13.634557 22468 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:14:13.631748 22307 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:13.631399 22547 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:13.589532 22553 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.640720 22488 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:13.626417 22502 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.625531 22196 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.671963 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:13.628242 22313 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.676821 22502 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:13.638896 22129 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.677961 12525 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 26: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:14:13.669198 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 155688us
W20260709 12:14:13.681370 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 160207us
I20260709 12:14:13.641008 22478 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.644027 22507 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.663782 22550 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:13.683226 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1236) took 1692 ms. Trace:
I20260709 12:14:13.700717 12530 rpcz_store.cc:276] 0709 12:14:11.990664 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.190611 (+199947us) service_pool.cc:224] Handling call
0709 12:14:12.230833 (+ 40222us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:12.397082 (+166249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:13.682961 (+1285879us) spinlock_profiling.cc:243] Waited 151 ms on lock 0x7b5400407754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:13.683159 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":347994368}
I20260709 12:14:13.701262 12529 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
I20260709 12:14:13.689723 22487 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:13.690464 12525 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:14:13.702979 12525 raft_consensus.cc:606] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:13.703394 12525 raft_consensus.cc:614] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:13.695058 12680 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 26 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
I20260709 12:14:13.696475 22503 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.697482 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:13.665921 22172 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:13.668021 22485 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:13.708213 22485 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.709064 22196 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:13.707149 22172 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.713691 12524 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: []
I20260709 12:14:13.710706 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:13.670655 22468 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:13.716079 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:13.716698 12522 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 31.
I20260709 12:14:13.716952 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:13.711743 13330 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:14:13.684782 12494 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:13.718667 12494 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:13.715524 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:13.719944 12671 raft_consensus.cc:2393] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 27: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:13.720641 12494 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:13.665465 22547 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:13.725181 12676 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 26: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:13.723369 12672 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:14:13.726976 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:13.729208 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:13.729808 12680 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:13.735796 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:13.687727 22054 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 12:14:13.738198 12494 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:13.723809 22196 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:14:13.746740 12527 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:14:13.743571 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:13.745209 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:13.745316 22279 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:14:13.746033 22575 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:14:13.750275 22507 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:13.750967 22487 consensus_queue.cc:1059] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:14:13.754136 22468 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:13.747920 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:13.762650 12676 raft_consensus.cc:2410] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 31 because replica has last-logged OpId of term: 29 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 14.
I20260709 12:14:13.725723 22174 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:13.765385 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3766) took 1027 ms. Trace:
I20260709 12:14:13.798820 12527 rpcz_store.cc:276] 0709 12:14:12.745621 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.730695 (+985074us) service_pool.cc:224] Handling call
0709 12:14:13.746428 (+ 15733us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:13.759344 (+ 12916us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:13.765328 (+  5984us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:13.777725 12526 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 32.
I20260709 12:14:13.712252 22550 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:13.712918 22503 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.818625 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:13.826078 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:13.811880 22575 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:13.842811 22577 log.cc:927] Time spent T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.168s	user 0.000s	sys 0.000s
I20260709 12:14:13.826309 22279 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.017s
W20260709 12:14:13.844601 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:13.845453 22172 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:13.845678 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:13.846849 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:13.848718 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4515) took 1079 ms. Trace:
I20260709 12:14:13.850095 12597 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:14:13.850566 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:13.852916 22129 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:13.855118 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 122172us
I20260709 12:14:13.874397 12525 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:14:13.876365 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:13.815493 22482 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.857693 12531 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
I20260709 12:14:13.883533 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:13.884652 12531 raft_consensus.cc:2368] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27.
I20260709 12:14:13.867509 12527 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29.
I20260709 12:14:13.819550 22174 raft_consensus.cc:697] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:13.877266 12525 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:13.872640 12673 rpcz_store.cc:276] 0709 12:14:12.769839 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.769954 (+   115us) service_pool.cc:224] Handling call
0709 12:14:12.795111 (+ 25157us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:13.254787 (+459676us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:13.848435 (+593648us) spinlock_profiling.cc:243] Waited 458 ms on lock 0x7b54002c4a54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:13.848648 (+   213us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1054400256}
I20260709 12:14:13.872637 12677 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:13.825544 22485 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:13.888477 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:13.873484 13332 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:14:13.904233 12680 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:14:13.905020 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:13.912492 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:13.913777 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:13.914606 12528 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:13.919092 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:13.919094 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:13.919664 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 181636us
I20260709 12:14:13.920418 12531 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:13.920624 12494 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:13.921319 12527 raft_consensus.cc:2410] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 25 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:14:13.919760 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:13.920306 12678 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:14:13.929998 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:13.930228 12494 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:13.869738 22494 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:13.916673 22562 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.080s	user 0.001s	sys 0.000s
I20260709 12:14:13.880419 22054 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.907707 22172 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election lost for term 23. Reason: could not achieve majority
I20260709 12:14:13.911783 22247 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:13.949496 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:13.909271 22307 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:13.908020 22332 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:13.908715 22506 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:13.964162 12531 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:14:13.965514 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:13.966257 12531 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:14:13.911132 22482 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.962889 22054 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:13.973063 22482 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:13.966717 22307 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:13.980808 22307 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:13.981392 22332 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:13.982645 22547 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:13.982314 22506 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:13.983270 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:13.947693 22313 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:13.991849 12525 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:14:14.012279 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:13.930893 22196 raft_consensus.cc:697] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:13.964460 22478 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:14.015383 12676 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:13.984009 22503 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:14.017433 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 87323.6us
I20260709 12:14:13.973991 22054 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:14.015664 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 92991.5us
I20260709 12:14:14.022961 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:14.026978 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:14.028083 12530 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 26: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:14.029131 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:14.029753 12530 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:14:14.023576 12494 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:14.031868 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:14.036384 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:13.963913 22247 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:14.039940 22550 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:14.045336 22508 raft_consensus.cc:3055] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:14:14.046005 22508 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:14.047010 22508 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:14.047941 22553 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:14.048067 22508 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:14.021349 22587 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:14.021796 22279 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:14:14.057217 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 179981us
I20260709 12:14:14.035126 12528 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 31.
I20260709 12:14:14.039839 22586 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:14:14.041775 22545 consensus_meta.cc:287] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.517s	user 0.013s	sys 0.000s
I20260709 12:14:14.038708 22547 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:14.067605 22332 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:14.074705 22277 raft_consensus.cc:3055] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:14:14.075191 22277 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:14.075341 22196 consensus_queue.cc:237] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 22.13, Last appended by leader: 13, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:14.076278 22277 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:14.077565 22277 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:14.079178 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:14.079836 12679 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:14:14.082038 22571 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.743s	user 0.006s	sys 0.000s
I20260709 12:14:14.082437 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:14.083106 12673 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:14.084059 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:14.084724 12673 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:14:14.086675 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:14.088212 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:14.088951 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:14.069908 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:14.090888 12672 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:14.074354 22591 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.010s
I20260709 12:14:13.997081 22575 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:14.095018 22545 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:14.100764 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 4531) took 1212 ms. Trace:
I20260709 12:14:14.101264 12675 rpcz_store.cc:276] 0709 12:14:12.888617 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:12.921760 (+ 33143us) service_pool.cc:224] Handling call
0709 12:14:14.100543 (+1178783us) spinlock_profiling.cc:243] Waited 141 ms on lock 0x7b5400013e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:14.100692 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":324214144}
I20260709 12:14:14.104290 22247 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:14.062649 22587 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:14.106458 22587 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:14.113653 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:14.113739 12600 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 32 index: 17. (index mismatch)
I20260709 12:14:14.117816 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:14.118180 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:14.116607 12527 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 29: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:14.118939 12606 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 31: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:14.118817 12600 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 26: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:14.118708 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:14.120461 12598 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:14:14.129644 12494 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:14.132148 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:14.133289 12528 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 22.
I20260709 12:14:14.115288 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:14.139294 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:14.140069 12606 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 32.
W20260709 12:14:14.141602 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:14.143541 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.144805 12604 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 29: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:14.148190 12572 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:14.151110 12572 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:14.154508 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:14.021184 22174 consensus_queue.cc:237] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:14.155284 12530 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 30.
I20260709 12:14:14.155683 12494 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:14.157140 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:14.158041 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 48283.5us
I20260709 12:14:14.160044 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:14.161513 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:14.169257 12602 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 31: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:14.069952 22506 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:14.166558 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:14:14.114991 22563 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.661s	user 0.012s	sys 0.002s
I20260709 12:14:14.164919 12605 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:14.166334 12673 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:14:14.163111 12572 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:14.174853 12530 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:14.137240 22476 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.173897 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:14.175530 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:14:14.180001 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.166894 12603 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:14.179739 12572 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:14.182910 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:14.184445 12494 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:14.185842 13336 raft_consensus.cc:2410] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 29 because replica has last-logged OpId of term: 28 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 20.
I20260709 12:14:14.187283 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:14.188695 12643 leader_election.cc:400] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26.
I20260709 12:14:14.189091 12643 leader_election.cc:403] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Cancelling election due to peer responding with higher term
I20260709 12:14:14.191699 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:14.092679 22553 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.082s
I20260709 12:14:14.194936 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:14.199000 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:14.199759 12523 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:14.196488 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:14.179067 22555 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:14.203986 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:14.155936 22490 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.038s
I20260709 12:14:14.160946 22594 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:14.208516 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:14.184057 22495 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:14:14.177894 22592 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:14:14.181541 22398 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:14:14.207828 12602 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
W20260709 12:14:14.260893 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 64467.1us
I20260709 12:14:14.188133 22279 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:14:14.352916 12572 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:14.336660 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 213279us
I20260709 12:14:14.181742 22550 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.031s
I20260709 12:14:14.188537 22478 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:14.189510 22488 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:14:14.346822 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:14:14.350629 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 164902us
I20260709 12:14:14.389847 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:14.188383 22588 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:14.357580 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:14.399009 12602 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:14.360136 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:14.400017 12643 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:14.198549 22539 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:14.202018 22332 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:14.373418 12599 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:14:14.162551 22482 consensus_queue.cc:1059] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 23.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.040s
I20260709 12:14:14.375068 12530 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:14:14.376786 12572 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:14.180768 22307 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 14, Time since last communication: 0.154s
I20260709 12:14:14.209708 22506 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:14.206589 22555 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:14.386570 12523 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:14:14.487461 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:14.488425 12523 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:14:14.173748 22247 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 12, Time since last communication: 0.033s
I20260709 12:14:14.346619 22509 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:14.385619 22495 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:14.189863 22476 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:14.414012 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 53979.1us
I20260709 12:14:14.535358 12494 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:14.380179 22547 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.375686 22279 raft_consensus.cc:697] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:14.387416 22468 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:14:14.480042 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:14.484252 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:14.542049 12528 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29.
I20260709 12:14:14.542543 12530 raft_consensus.cc:2410] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 25 because replica has last-logged OpId of term: 23 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:14:14.507246 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:14.479777 22307 consensus_queue.cc:1243] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 27.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:14:14.537813 12523 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:14:14.479131 22478 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:14.410312 22332 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:14.553751 12530 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 30. Current term is 31. Ops: [30.21-30.21]
I20260709 12:14:14.478394 22539 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:14.509486 22469 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Leader pre-election won for term 23
I20260709 12:14:14.544423 12525 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:14:14.568029 12523 raft_consensus.cc:606] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Received request to transfer leadership to TS 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:14.568557 12523 raft_consensus.cc:614] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:14.568624 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:14.569504 12525 raft_consensus.cc:3060] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:14.401960 22597 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:14:14.569540 22595 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.154s	user 0.000s	sys 0.000s
I20260709 12:14:14.559037 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:14.574950 22196 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:14:14.575517 22196 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:14.575989 22196 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:14.583276 12494 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:14.535450 22490 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:14.567890 22469 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:14.555719 12530 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 30. Current term is 31. Ops: [30.15-30.15]
I20260709 12:14:14.590973 22469 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:14:14.592168 12530 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:14:14.595163 22279 consensus_queue.cc:237] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:14.599237 22593 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:14.599840 22593 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:14.600281 22593 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:14.561129 22332 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:14.607595 12494 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:14.511072 22555 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:14.553016 22563 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.560148 22553 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:14.620786 12525 raft_consensus.cc:2468] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 30.
I20260709 12:14:14.531128 12679 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:14:14.624471 12528 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: []
I20260709 12:14:14.665398 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3712) took 1123 ms. Trace:
I20260709 12:14:14.665859 12528 rpcz_store.cc:276] 0709 12:14:13.541666 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:14.543131 (+1001465us) service_pool.cc:224] Handling call
0709 12:14:14.624284 (+ 81153us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:14.665130 (+ 40846us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.665339 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:14.625190 12646 leader_election.cc:336] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Timed out: RequestConsensusVote RPC to 127.11.13.129:38811 timed out after 1.973s (SENT)
I20260709 12:14:14.536046 22509 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:14.411664 22588 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:14.556757 22478 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:14.635708 12602 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:14.635483 22504 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:14.549850 22547 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:14.684219 22504 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:14.633744 22490 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.671754 22509 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:14:14.696426 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 296460us
I20260709 12:14:14.634217 22592 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:14:14.563108 22468 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:14:14.713136 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:14.756665 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:14.717949 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 92774.2us
I20260709 12:14:14.724076 22469 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:14.752266 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:14.683537 22478 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:14.616386 22593 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:14.634420 22502 consensus_queue.cc:1059] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.071s
I20260709 12:14:14.753330 22592 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:14.672158 22563 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:14.765400 12646 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:14.772681 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:14.779345 22547 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:14.779130 22468 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.14, Last appended by leader: 14, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:14.782582 22591 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:14.783151 22591 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:14.783326 22476 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:14.783725 22591 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:14.784073 22476 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:14.784662 22476 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:14.771636 22539 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.21, Last appended by leader: 21, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:14.786927 22522 raft_consensus.cc:3055] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:14:14.770217 22592 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:14.788915 22522 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:14.789997 22522 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:14.791097 22522 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:14.704360 22490 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:14.795238 22593 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:14.796952 22503 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:14:14.703044 22495 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:14.801616 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:14.820806 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:14.776325 12646 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:14:14.821959 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 45727.3us
I20260709 12:14:14.778028 22247 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.026s
W20260709 12:14:14.816329 22600 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.052s	user 0.001s	sys 0.000s
I20260709 12:14:14.800487 22487 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.778334 22307 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:14:14.827735 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 55130.3us
I20260709 12:14:14.854342 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:14.819814 22503 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:14.827001 22547 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:14:14.830032 22597 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:14.884073 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:14.879135 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 55409us
I20260709 12:14:14.899389 22196 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:14.901789 12530 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 32. Current term is 33. Ops: []
I20260709 12:14:14.905023 22196 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:14.882241 22575 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:14.926380 12526 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:14.856527 22602 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 29.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.067s
I20260709 12:14:14.883901 22597 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:14:14.940881 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 56895.5us
I20260709 12:14:14.940996 22503 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:14.942369 22616 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:14.942934 22247 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:14:14.862322 22487 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:14.903908 22504 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:14.950946 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:14.940270 22587 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.833s	user 0.010s	sys 0.006s
I20260709 12:14:14.939527 22469 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:14.950764 22508 consensus_meta.cc:287] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.902s	user 0.016s	sys 0.002s
I20260709 12:14:14.904132 22307 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 27.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 12:14:14.946844 22506 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election lost for term 25. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26.
I20260709 12:14:14.948869 22597 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:14.963438 22279 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:14.972258 22332 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:14.972851 22279 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:14.976855 22307 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:14:14.978437 22476 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:14.957320 22588 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:14.974658 22279 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:14.937270 22575 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:14.960093 22616 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:14.990494 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3780) took 1238 ms. Trace:
I20260709 12:14:14.959448 22490 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:15.000962 12528 rpcz_store.cc:276] 0709 12:14:13.752182 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:14.666693 (+914511us) service_pool.cc:224] Handling call
0709 12:14:14.690981 (+ 24288us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:14.815483 (+124502us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:14.815537 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:14.821180 (+  5643us) log.cc:844] Serialized 26 byte log entry
0709 12:14:14.878454 (+ 57274us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:14:14.895702 (+ 17248us) log.cc:844] Serialized 14 byte log entry
0709 12:14:14.896012 (+   310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:14.896133 (+   121us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:14.990167 (+ 94034us) raft_consensus.cc:1685] finished
0709 12:14:14.990200 (+    33us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:14.990412 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8415,"prepare.queue_time_us":110678,"raft.queue_time_us":101460,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":24227,"thread_start_us":156064,"threads_started":3,"wal-append.queue_time_us":82946}
I20260709 12:14:14.970459 22602 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 LEADER]: Stepping down as leader of term 30
W20260709 12:14:15.012318 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 65204.1us
I20260709 12:14:14.970934 22617 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:14.962463 22587 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:15.018567 12643 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:14.999544 22622 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.023221 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:15.006889 22588 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:15.048035 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:15.048883 12528 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:14:15.009915 22563 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:15.059188 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:15.013458 22602 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:15.058826 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:15.014400 22547 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:15.074790 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:15.083688 12528 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 31.
I20260709 12:14:15.085071 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:15.019186 22490 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:14:15.085863 12528 raft_consensus.cc:2376] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29. Re-sending same reply.
W20260709 12:14:15.073953 22277 consensus_meta.cc:287] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.996s	user 0.008s	sys 0.010s
I20260709 12:14:15.170028 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:15.201170 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:14:15.201570 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.201988 12528 raft_consensus.cc:2376] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25. Re-sending same reply.
I20260709 12:14:15.203265 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:15.088876 22490 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:14:15.235841 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.073884 22509 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:15.075917 22588 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:15.262960 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:15.242856 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 219711us
I20260709 12:14:15.310298 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:15.171136 22247 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 27.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.068s
I20260709 12:14:15.056643 22622 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:15.239008 22623 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
W20260709 12:14:15.349402 22615 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.129s	user 0.000s	sys 0.000s
W20260709 12:14:15.356907 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.358599 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:15.360895 12679 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 29: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:15.364298 12672 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:14:15.366539 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:15.366992 22575 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:15.365761 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:15.344992 22547 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:15.372711 22555 consensus_meta.cc:287] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.704s	user 0.010s	sys 0.002s
I20260709 12:14:15.373296 22555 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:15.376263 22555 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:15.379341 22622 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:15.220074 22616 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:15.228857 22627 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.163s
W20260709 12:14:15.387653 22594 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.016s	user 0.009s	sys 0.004s
I20260709 12:14:15.388686 22587 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:15.389573 22594 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:14:15.389827 22522 consensus_meta.cc:287] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.598s	user 0.011s	sys 0.005s
I20260709 12:14:15.392400 22588 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:15.261842 22487 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:15.412447 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:15.415385 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:15.243528 22625 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:14:15.343695 22509 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:15.436189 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:15.446370 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 387585us
I20260709 12:14:15.379793 22597 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:15.456925 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:15.458981 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:15.462031 12675 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
W20260709 12:14:15.462504 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 344269us
I20260709 12:14:15.462248 12673 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 31.
I20260709 12:14:15.460464 13330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 1117) took 2316 ms. Trace:
W20260709 12:14:15.464768 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 163824us
I20260709 12:14:15.417069 22621 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:15.465914 12643 leader_election.cc:400] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
I20260709 12:14:15.473183 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3700) took 1957 ms. Trace:
I20260709 12:14:15.415019 22617 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:15.463970 13330 rpcz_store.cc:276] 0709 12:14:13.144275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.144481 (+   206us) service_pool.cc:224] Handling call
0709 12:14:13.699479 (+554998us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:13.718080 (+ 18601us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:15.455698 (+1737618us) spinlock_profiling.cc:243] Waited 555 ms on lock 0x7b54004781d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:15.460351 (+  4653us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1275483520}
I20260709 12:14:15.474720 12524 rpcz_store.cc:276] 0709 12:14:13.515438 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.515619 (+   181us) service_pool.cc:224] Handling call
0709 12:14:13.546527 (+ 30908us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:13.718396 (+171869us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:15.472182 (+1753786us) spinlock_profiling.cc:243] Waited 167 ms on lock 0x7b54003f7854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:15.473104 (+   922us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":383636352}
I20260709 12:14:15.475769 12643 leader_election.cc:403] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:14:15.475091 22468 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.531s	user 0.013s	sys 0.020s
I20260709 12:14:15.463052 22594 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.488109 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:15.491071 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:15.473510 22621 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:15.464944 22588 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:15.495365 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:15.491643 12524 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
W20260709 12:14:15.492594 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.476316 22633 raft_consensus.cc:3055] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Stepping down as leader of term 26
W20260709 12:14:15.508195 12526 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3779) took 1756 ms (client timeout 1729 ms). Trace:
I20260709 12:14:15.527057 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3778) took 1775 ms. Trace:
W20260709 12:14:15.527215 12526 rpcz_store.cc:269] 0709 12:14:13.751932 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.808509 (+ 56577us) service_pool.cc:224] Handling call
0709 12:14:15.507995 (+1699486us) spinlock_profiling.cc:243] Waited 1.08 s on lock 0x7b54001068d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:15.508113 (+   118us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2477202304}
I20260709 12:14:15.508355 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:15.529071 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:15.528559 12602 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 32: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:15.509011 22604 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:15.509881 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:15.527869 12525 rpcz_store.cc:276] 0709 12:14:13.751667 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:14.663587 (+911920us) service_pool.cc:224] Handling call
0709 12:14:14.663961 (+   374us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:14.899962 (+236001us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:14.900214 (+   252us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:14.944985 (+ 44771us) log.cc:844] Serialized 26 byte log entry
0709 12:14:15.075622 (+130637us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:15.075663 (+    41us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:15.075747 (+    84us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:15.526443 (+450696us) raft_consensus.cc:1685] finished
0709 12:14:15.526474 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:15.526934 (+   460us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":128189,"raft.run_cpu_time_us":80,"raft.run_wall_time_us":79,"thread_start_us":353598,"threads_started":2,"wal-append.queue_time_us":387070}
I20260709 12:14:15.530846 12597 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:15.479516 22625 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:15.533353 12603 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:14:15.523413 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:15.537181 12605 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:15.526347 12528 raft_consensus.cc:2376] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27. Re-sending same reply.
I20260709 12:14:15.545176 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:15.545936 12525 raft_consensus.cc:2376] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25. Re-sending same reply.
I20260709 12:14:15.546263 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:15.489748 22468 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.550s	user 0.013s	sys 0.021s
I20260709 12:14:15.546854 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:15.546890 12601 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 32.
I20260709 12:14:15.547816 12525 raft_consensus.cc:2410] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 25 because replica has last-logged OpId of term: 24 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:14:15.548908 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:15.550776 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:15.551375 12525 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 22 FOLLOWER]: Advancing to term 23
W20260709 12:14:15.537230 22495 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.557s	user 0.001s	sys 0.018s
I20260709 12:14:15.568077 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:15.455252 22602 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:15.531701 22610 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:15.554669 22591 consensus_meta.cc:287] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.770s	user 0.010s	sys 0.004s
I20260709 12:14:15.499686 22621 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:15.558713 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:15.576572 12671 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:14:15.581955 12597 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 33.
W20260709 12:14:15.568037 22495 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s	user 0.002s	sys 0.018s
I20260709 12:14:15.585799 12605 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:15.526854 22633 raft_consensus.cc:740] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:15.570226 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:15.618748 12569 connection.cc:256] RPC call timeout handler was delayed by -1.2404320279999865s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 1.5s for a response
W20260709 12:14:15.592552 22279 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.536s	user 0.029s	sys 0.026s
I20260709 12:14:15.610467 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:15.610162 22617 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 27.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:15.610486 22277 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.629086 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:15.629266 22277 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:15.630461 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:15.499297 22594 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:15.633262 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.633822 22594 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:15.618795 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:15.635154 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:15.636317 12524 raft_consensus.cc:2368] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
I20260709 12:14:15.637547 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:15.550972 22604 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:15.639169 22633 consensus_queue.cc:260] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 26.13, Last appended by leader: 13, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:15.640146 22604 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:15.641119 22633 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:15.605235 22610 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:15.643325 22277 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:15.645380 22610 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:15.604143 22527 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.646610 22527 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:15.647482 12529 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:14:15.649245 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:15.621282 12525 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:14:15.611380 22539 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:15.531530 22588 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:15.652189 22539 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:15.653294 22545 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:15.653986 22588 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:15.654507 22539 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:15.655381 22597 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:15.621245 22612 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.655741 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:15.656574 12525 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:14:15.656466 22612 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:15.657490 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:15.658489 12529 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:15.659693 12680 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: []
W20260709 12:14:15.661212 22553 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.992s	user 0.015s	sys 0.001s
I20260709 12:14:15.661646 22553 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:15.658069 12525 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 31.
I20260709 12:14:15.667158 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:15.667713 22592 consensus_meta.cc:287] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.878s	user 0.014s	sys 0.000s
I20260709 12:14:15.668393 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:15.668994 12525 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 30.
I20260709 12:14:15.669796 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:15.670611 12525 raft_consensus.cc:2410] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 31 because replica has last-logged OpId of term: 28 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
I20260709 12:14:15.671636 12525 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:14:15.672209 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3781) took 1919 ms. Trace:
I20260709 12:14:15.672523 12525 rpcz_store.cc:276] 0709 12:14:13.752449 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.671184 (+1918735us) service_pool.cc:224] Handling call
0709 12:14:15.671460 (+   276us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:15.671961 (+   501us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:15.672157 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:15.670639 13336 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38484 (request call id 1138) took 2199 ms (client timeout 2217 ms). Trace:
I20260709 12:14:15.624919 12671 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:14:15.674878 13336 rpcz_store.cc:269] 0709 12:14:13.471403 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.471539 (+   136us) service_pool.cc:224] Handling call
0709 12:14:15.670405 (+2198866us) spinlock_profiling.cc:243] Waited 713 ms on lock 0x7b5400469954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:15.670570 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1639707904}
W20260709 12:14:15.674382 22279 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s	user 0.030s	sys 0.026s
I20260709 12:14:15.609490 22591 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:14:15.722388 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 215516us
I20260709 12:14:15.676606 12524 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:15.677685 12680 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 28 index: 14. Preceding OpId from leader: term: 29 index: 15. (index mismatch)
I20260709 12:14:15.668179 22592 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:15.655547 12646 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:15.606832 22627 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 25
W20260709 12:14:15.679373 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 59684.9us
I20260709 12:14:15.683104 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 3709) took 2143 ms. Trace:
I20260709 12:14:15.673986 12525 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 32. Current term is 33. Ops: [32.19-32.19]
I20260709 12:14:15.741432 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3782) took 1988 ms. Trace:
I20260709 12:14:15.741866 12525 rpcz_store.cc:276] 0709 12:14:13.752677 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.673262 (+1920585us) service_pool.cc:224] Handling call
0709 12:14:15.673611 (+   349us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:15.741137 (+ 67526us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:15.741372 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:15.711858 12529 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
W20260709 12:14:15.725395 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:15.742962 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:15.730373 12671 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 31.
I20260709 12:14:15.602286 22625 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.649780 22527 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:15.674487 22478 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.906s	user 0.005s	sys 0.012s
W20260709 12:14:15.735186 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 79852.9us
I20260709 12:14:15.749058 12646 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:15.651407 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:15.749549 12676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 4549) took 1863 ms (client timeout 1518 ms). Trace:
I20260709 12:14:15.738184 12571 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:15.749958 12676 rpcz_store.cc:269] 0709 12:14:13.885831 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.886411 (+   580us) service_pool.cc:224] Handling call
0709 12:14:15.749371 (+1862960us) spinlock_profiling.cc:243] Waited 92 ms on lock 0x7b54000c2bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:15.749484 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":211694336}
I20260709 12:14:15.740991 12522 rpcz_store.cc:276] 0709 12:14:13.539878 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.661070 (+121192us) service_pool.cc:224] Handling call
0709 12:14:15.682839 (+2021769us) spinlock_profiling.cc:243] Waited 44.1 ms on lock 0x7b54003f7354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:15.683017 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":101319936}
I20260709 12:14:15.736351 22547 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:15.627167 22602 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:15.746922 12496 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:15.745415 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 279637us
I20260709 12:14:15.753489 12525 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 31. Current term is 32. Ops: [31.18-31.18]
I20260709 12:14:15.754416 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:15.764991 12643 leader_election.cc:400] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26.
I20260709 12:14:15.756098 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:15.758534 12646 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:15.760083 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:15.766278 12676 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:14:15.766623 12646 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:15.768051 12571 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:15.789642 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:15.790323 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:15.790477 12680 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:15.768463 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:15.772737 12496 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:15.770921 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3788) took 1862 ms. Trace:
I20260709 12:14:15.790962 12671 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 23.
I20260709 12:14:15.792346 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:15.792184 12525 rpcz_store.cc:276] 0709 12:14:13.908338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.747767 (+1839429us) service_pool.cc:224] Handling call
0709 12:14:15.753115 (+  5348us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:15.762933 (+  9818us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:15.770837 (+  7904us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:15.736919 22468 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.791846 22129 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:15.744850 22545 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:15.801075 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:15.793942 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:15.796437 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 23 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:15.792964 12674 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:14:15.802904 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:15.750627 22611 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:15.798915 12522 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:14:15.810139 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3789) took 1868 ms. Trace:
I20260709 12:14:15.810539 12522 rpcz_store.cc:276] 0709 12:14:13.941368 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.794826 (+1853458us) service_pool.cc:224] Handling call
0709 12:14:15.798723 (+  3897us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:15.809861 (+ 11138us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:15.810064 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:15.811362 12526 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:14:15.799889 12646 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:15.808706 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:15.787436 22597 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:15.812434 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:15.812198 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3791) took 1841 ms. Trace:
I20260709 12:14:15.797863 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:15.870038 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:15.918349 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 106590us
I20260709 12:14:15.803936 22129 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:15.878949 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 77953.9us
I20260709 12:14:15.807958 22277 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:15.814023 22632 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.065s
I20260709 12:14:15.794317 22594 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:14:15.796303 22555 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:14:15.884255 12522 raft_consensus.cc:2410] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 27 because replica has last-logged OpId of term: 24 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
I20260709 12:14:15.815500 12526 rpcz_store.cc:276] 0709 12:14:13.970900 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.796404 (+1825504us) service_pool.cc:224] Handling call
0709 12:14:15.811112 (+ 14708us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:15.811910 (+   798us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:15.812130 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:15.797369 22506 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:14:15.886569 22640 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:14:15.887274 22547 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:14:15.888516 22597 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:15.804019 22539 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 25.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.085s
I20260709 12:14:15.808871 22468 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:15.799412 22586 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:15.795346 22639 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:14:15.748378 22625 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:15.761740 22478 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:15.811923 13304 leader_election.cc:336] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.106s (SENT)
I20260709 12:14:15.942279 12525 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 28 index: 14. Preceding OpId from leader: term: 29 index: 15. (index mismatch)
I20260709 12:14:15.943311 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:15.944427 13304 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:15.945832 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 133928us
I20260709 12:14:15.919528 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:15.920423 12676 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 29. Current term is 30. Ops: []
I20260709 12:14:15.948601 12676 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: []
W20260709 12:14:15.949240 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 149421us
I20260709 12:14:15.950297 12676 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 30. Current term is 31. Ops: []
I20260709 12:14:15.920915 12496 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:15.953226 12646 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:15.809212 22545 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:15.956902 22592 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:15.957470 22545 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:15.957762 22592 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:14:15.815056 22611 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:15.901077 22553 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:15.922304 22129 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:15.958560 22509 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:15.939857 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:15.961828 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 153199us
I20260709 12:14:15.962333 12522 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:14:15.815237 22375 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.123s
I20260709 12:14:15.960530 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:48476 (request call id 98) took 1930 ms. Trace:
I20260709 12:14:15.965648 12523 rpcz_store.cc:276] 0709 12:14:14.028931 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:14.489575 (+460644us) service_pool.cc:224] Handling call
0709 12:14:15.959247 (+1469672us) spinlock_profiling.cc:243] Waited 17.1 ms on lock 0x7b5400015c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1a853 00007f148d9e860a 00007f1486f6e7d4 00007f1486f6e762 00007f1486f6e691 00007f1486f6e60c 00007f1486f6d8c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:15.959363 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":39417728}
I20260709 12:14:15.963629 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:15.922828 22612 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:15.961377 22545 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.011s
W20260709 12:14:15.969822 22638 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.165s	user 0.000s	sys 0.000s
I20260709 12:14:15.970402 22488 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:15.970827 22488 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:15.971257 22488 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:14:15.971467 22634 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:14:15.972774 22592 raft_consensus.cc:697] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:15.973873 22592 consensus_queue.cc:237] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:15.940946 22594 raft_consensus.cc:697] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:15.978571 22594 consensus_queue.cc:237] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:15.978996 22591 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:14:15.979970 22591 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:15.980923 22625 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:15.983193 22478 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:15.984527 22597 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:15.986240 12679 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 32 because replica has last-logged OpId of term: 30 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 20.
I20260709 12:14:15.948341 22586 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:15.988184 22586 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:15.939857 22610 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:14:15.988926 22610 raft_consensus.cc:697] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:15.989862 22610 consensus_queue.cc:237] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 29.15, Last appended by leader: 15, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:15.959307 22277 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:15.977388 22632 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:14:15.960940 22527 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:14:15.966624 22547 raft_consensus.cc:697] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:15.995190 22277 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:15.979103 12646 leader_election.cc:400] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
I20260709 12:14:15.996945 12646 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:15.998756 12646 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:15.968883 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:16.014063 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 45248.2us
W20260709 12:14:16.020347 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.015515 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:16.020917 13305 connection.cc:256] RPC call timeout handler was delayed by -1.5252158480000162s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2189s for a response
I20260709 12:14:16.083011 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:16.082864 22482 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 29.16, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:14:16.083726 13336 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 32.
I20260709 12:14:16.083967 22469 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:14:16.084354 22469 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:16.083421 22490 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 30.15, Last appended by leader: 15, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.085054 22490 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:16.085287 22469 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.021966 12526 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:14:15.965279 22553 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:16.086993 22553 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:16.087188 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3795) took 2043 ms. Trace:
I20260709 12:14:16.087554 12526 rpcz_store.cc:276] 0709 12:14:14.044067 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.999751 (+1955684us) service_pool.cc:224] Handling call
0709 12:14:16.021793 (+ 22042us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:16.086934 (+ 65141us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.087134 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:16.088536 22621 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:16.045195 12523 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 28. Current term is 29. Ops: [28.21-28.21]
I20260709 12:14:15.977474 22631 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.066s
I20260709 12:14:16.052215 22611 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:16.018167 22277 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:14:16.090085 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1237) took 1687 ms. Trace:
I20260709 12:14:16.090612 22277 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:16.090680 12523 rpcz_store.cc:276] 0709 12:14:14.402447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.019049 (+1616602us) service_pool.cc:224] Handling call
0709 12:14:16.044922 (+ 25873us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:16.089747 (+ 44825us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.090014 (+   267us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:16.091126 22277 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:16.092093 12523 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:16.092694 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1238) took 1582 ms. Trace:
I20260709 12:14:16.093004 12523 rpcz_store.cc:276] 0709 12:14:14.510187 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.091612 (+1581425us) service_pool.cc:224] Handling call
0709 12:14:16.091923 (+   311us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:16.092465 (+   542us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.092641 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:16.054278 22547 consensus_queue.cc:237] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.097486 13305 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:16.055665 22650 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:16.104969 22650 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:16.057241 22503 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:14:16.108566 12531 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3768) took 2382 ms (client timeout 2513 ms). Trace:
W20260709 12:14:16.109030 12531 rpcz_store.cc:269] 0709 12:14:13.726181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.970615 (+244434us) service_pool.cc:224] Handling call
0709 12:14:16.108350 (+2137735us) spinlock_profiling.cc:243] Waited 554 ms on lock 0x7b54003f8754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:16.108503 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1275104000}
I20260709 12:14:16.057948 22485 raft_consensus.cc:3055] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:14:16.110327 22485 raft_consensus.cc:740] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:16.111833 22621 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:16.112517 22588 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:14:16.113076 22588 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:16.113502 22588 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:14:16.120261 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.094324 22644 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:16.122973 22644 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:16.123252 12599 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4290) took 2650 ms (client timeout 1714 ms). Trace:
W20260709 12:14:16.123792 12599 rpcz_store.cc:269] 0709 12:14:13.472897 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.473020 (+   123us) service_pool.cc:224] Handling call
0709 12:14:16.122957 (+2649937us) spinlock_profiling.cc:243] Waited 875 ms on lock 0x7b5400116cd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:16.123132 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2011882752}
I20260709 12:14:16.126003 12423 catalog_manager.cc:5697] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 23 to 25, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 25 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:16.128645 22627 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 24. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 24. Current term is 25.
I20260709 12:14:16.128963 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3776) took 2377 ms. Trace:
I20260709 12:14:16.129446 12530 rpcz_store.cc:276] 0709 12:14:13.751130 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:14.593028 (+841898us) service_pool.cc:224] Handling call
0709 12:14:14.593348 (+   320us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:14.902277 (+308929us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.128690 (+1226413us) spinlock_profiling.cc:243] Waited 308 ms on lock 0x7b5400054454. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:16.128902 (+   212us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":708770048}
I20260709 12:14:16.138826 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:16.139220 12531 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 26. Current term is 27. Ops: []
I20260709 12:14:16.139644 12530 raft_consensus.cc:2393] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:16.000313 22509 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.069803 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.073962 22555 raft_consensus.cc:3055] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:14:16.073740 22586 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:16.021591 12527 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3771) took 2294 ms (client timeout 2208 ms). Trace:
I20260709 12:14:16.085906 22527 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 30.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.000s
I20260709 12:14:16.089078 12526 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:14:16.162540 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3722) took 1744 ms. Trace:
I20260709 12:14:16.163017 12526 rpcz_store.cc:276] 0709 12:14:14.417958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.088457 (+1670499us) service_pool.cc:224] Handling call
0709 12:14:16.088779 (+   322us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:16.162159 (+ 73380us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.162451 (+   292us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:16.164242 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.095525 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1239) took 1541 ms. Trace:
I20260709 12:14:16.164717 12523 rpcz_store.cc:276] 0709 12:14:14.554168 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.093724 (+1539556us) service_pool.cc:224] Handling call
0709 12:14:16.094002 (+   278us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:16.095082 (+  1080us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:16.095124 (+    42us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:16.095186 (+    62us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:14:16.095232 (+    46us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.095310 (+    78us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:16.095459 (+   149us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:16.085922 22129 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.100652 12524 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
I20260709 12:14:16.167397 12529 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 28. Current term is 29. Ops: [27.16-28.17]
I20260709 12:14:16.168377 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1243) took 1500 ms. Trace:
I20260709 12:14:16.168891 12524 rpcz_store.cc:276] 0709 12:14:14.667869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.167502 (+1499633us) service_pool.cc:224] Handling call
0709 12:14:16.167803 (+   301us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:16.168006 (+   203us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:14:16.168032 (+    26us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:16.168085 (+    53us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:16.168127 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.168176 (+    49us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:16.168307 (+   131us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:16.017015 22625 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:14:16.106477 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:16.179598 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.111318 22485 consensus_queue.cc:260] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.059998 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3804) took 1365 ms. Trace:
I20260709 12:14:16.134802 22332 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:16.135319 22478 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.103817 22640 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:16.137531 22279 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.063941 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.139892 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1241) took 1516 ms. Trace:
I20260709 12:14:16.021250 12646 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:16.138533 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.147480 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:16.150640 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 22 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.105446 22650 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:16.128269 12423 catalog_manager.cc:5697] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 18 to 26, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 26 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:14:16.153362 12527 rpcz_store.cc:269] 0709 12:14:13.727127 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:13.929454 (+202327us) service_pool.cc:224] Handling call
0709 12:14:16.021370 (+2091916us) spinlock_profiling.cc:243] Waited 177 ms on lock 0x7b54003f6e54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:16.021518 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":406568832}
I20260709 12:14:16.093940 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4303) took 1941 ms. Trace:
I20260709 12:14:16.145368 22468 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:16.192747 12528 rpcz_store.cc:276] 0709 12:14:14.694981 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.202961 (+507980us) service_pool.cc:224] Handling call
0709 12:14:16.059789 (+856828us) spinlock_profiling.cc:243] Waited 322 ms on lock 0x7b5400105ed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:16.059924 (+   135us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":741303168}
I20260709 12:14:16.184902 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.180621 12524 raft_consensus.cc:1275] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:14:16.149616 22555 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:16.150147 22506 raft_consensus.cc:697] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:16.199206 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:16.201756 12531 rpcz_store.cc:276] 0709 12:14:14.623380 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.109823 (+1486443us) service_pool.cc:224] Handling call
0709 12:14:16.139027 (+ 29204us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:16.139649 (+   622us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.139840 (+   191us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:16.211151 12526 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 32.
W20260709 12:14:16.201921 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 180744us
I20260709 12:14:16.202970 12530 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 27.
I20260709 12:14:16.213105 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:14:16.214438 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 260274us
I20260709 12:14:16.202283 12671 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 23.
I20260709 12:14:16.215948 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.216446 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.216579 12671 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 27.
I20260709 12:14:16.217430 12675 raft_consensus.cc:2410] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 29 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 14.
I20260709 12:14:16.217794 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.208904 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1244) took 1534 ms. Trace:
I20260709 12:14:16.220674 12524 rpcz_store.cc:276] 0709 12:14:14.673802 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.171219 (+1497417us) service_pool.cc:224] Handling call
0709 12:14:16.171620 (+   401us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:16.208247 (+ 36627us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.208470 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:16.222498 12524 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: []
I20260709 12:14:16.223359 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3724) took 1195 ms. Trace:
I20260709 12:14:16.223848 12524 rpcz_store.cc:276] 0709 12:14:15.028300 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.221850 (+1193550us) service_pool.cc:224] Handling call
0709 12:14:16.222255 (+   405us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:16.223046 (+   791us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.223287 (+   241us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:16.225567 12524 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:16.198674 22625 raft_consensus.cc:697] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:16.232774 22625 consensus_queue.cc:237] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.204437 12606 rpcz_store.cc:276] 0709 12:14:14.152587 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:14.152744 (+   157us) service_pool.cc:224] Handling call
0709 12:14:14.153131 (+   387us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:14.155075 (+  1944us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:14.155234 (+   159us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:14.167404 (+ 12170us) log.cc:844] Serialized 52 byte log entry
0709 12:14:14.509101 (+341697us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:14.760711 (+251610us) log.cc:844] Serialized 14 byte log entry
0709 12:14:15.506802 (+746091us) spinlock_profiling.cc:243] Waited 5.76 ms on lock 0x7b18004ac500. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f148c8f458f 00007f1485933571 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd
0709 12:14:16.093435 (+586633us) spinlock_profiling.cc:243] Waited 16.4 ms on lock 0x7b18004ac500. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287
0709 12:14:16.093587 (+   152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.093682 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:16.093738 (+    56us) raft_consensus.cc:1685] finished
0709 12:14:16.093751 (+    13us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:16.093879 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8580,"prepare.queue_time_us":386597,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":48,"raft.queue_time_us":105002,"raft.run_cpu_time_us":288,"raft.run_wall_time_us":16443,"spinlock_wait_cycles":51048960,"thread_start_us":564873,"threads_started":3,"wal-append.queue_time_us":506890,"wal-append.run_cpu_time_us":12503,"wal-append.run_wall_time_us":1182316}
I20260709 12:14:16.198222 22640 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:16.182096 22611 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:16.202907 12678 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:14:16.179241 22376 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 30.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.041s
W20260709 12:14:16.229342 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.167342 22652 consensus_queue.cc:1059] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.082s
I20260709 12:14:16.166891 22488 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:16.207083 22279 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.199668 22478 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.246838 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.252355 13304 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:16.265944 12643 connection.cc:256] RPC call timeout handler was delayed by -1.010998501000131s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1537s for a response
I20260709 12:14:16.210148 22485 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
W20260709 12:14:16.316073 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 63824.9us
I20260709 12:14:16.203769 22627 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:16.286489 22653 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.296296 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.327747 12675 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 30: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:16.280529 22469 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:16.276705 22494 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 12:14:16.283329 22555 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 25.13, Last appended by leader: 13, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.297799 22640 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:16.299057 22620 raft_consensus.cc:3055] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:14:16.336704 22620 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:16.307655 22654 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.294240 12524 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:14:16.325371 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.345194 12673 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:14:16.299197 22277 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:16.298237 22376 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:14:16.283018 22586 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:16.330312 12676 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:16.281728 22644 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:16.333297 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.331661 22666 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.228215 22602 consensus_queue.cc:1059] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 31.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.095s
I20260709 12:14:16.363324 22602 consensus_queue.cc:1243] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 33.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:14:16.363440 22665 raft_consensus.cc:3055] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:14:16.346899 22376 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:16.316898 13304 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:16.340356 12496 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:16.323565 22627 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:16.340847 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 158079us
I20260709 12:14:16.345715 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:14:16.346413 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.367453 22376 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 30.15, Last appended by leader: 15, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:14:16.372371 22129 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:16.373091 22376 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:16.373728 12677 raft_consensus.cc:2410] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 31 because replica has last-logged OpId of term: 30 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 14.
I20260709 12:14:16.374611 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.375984 22495 raft_consensus.cc:3055] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:14:16.376336 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:16.377012 12674 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:16.349309 12680 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:14:16.377815 12524 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:16.301477 22611 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.354450 22666 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:16.356281 22488 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:16.310958 22617 consensus_queue.cc:1059] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.032s
I20260709 12:14:16.311120 22628 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.033s
I20260709 12:14:16.281224 22506 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:16.358728 22509 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:16.359378 22277 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:16.310487 22247 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.034s
I20260709 12:14:16.363994 22602 raft_consensus.cc:3055] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:14:16.407141 22602 raft_consensus.cc:740] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:16.408322 22602 consensus_queue.cc:260] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 32.19, Last appended by leader: 19, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.409144 22611 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:16.409638 22602 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:16.297372 22656 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 30.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.048s
I20260709 12:14:16.414435 22656 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:14:16.414831 22656 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:16.415763 22656 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 16, Last appended: 31.18, Last appended by leader: 18, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.417106 22656 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:16.321141 22621 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:16.375926 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.376472 22495 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:16.418576 12673 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:14:16.418440 22621 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:16.419786 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.421020 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:16.421136 22621 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:16.279224 22588 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:16.227061 22332 consensus_queue.cc:1059] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.100s
I20260709 12:14:16.379213 22527 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.363844 22478 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:16.391582 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:16.454411 22527 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:16.338474 22653 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.444589 12674 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:14:16.456689 22527 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:16.456974 22653 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:16.457353 22527 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:16.363924 22665 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:16.457794 22527 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:16.339449 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.464514 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:16.476593 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:16.409713 22666 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:16.381450 22594 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:16.410418 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3815) took 1470 ms. Trace:
I20260709 12:14:16.378575 22620 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.411319 22503 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:16.346009 22654 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.412623 22247 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:14:16.411273 22628 raft_consensus.cc:3055] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:14:16.426644 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:16.428164 12606 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 12. Preceding OpId from leader: term: 24 index: 14. (index mismatch)
I20260709 12:14:16.499425 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:16.428589 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:16.428617 12601 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:14:16.500684 12605 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:16.430521 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:16.431880 12673 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:16.506060 12604 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:16.435453 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:16.507367 12597 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:14:16.436172 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:16.509303 13336 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 32 because replica has last-logged OpId of term: 30 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 20.
W20260709 12:14:16.434942 12494 connection.cc:256] RPC call timeout handler was delayed by -1.4102056579999953s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1714s for a response
I20260709 12:14:16.437426 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3715) took 2425 ms. Trace:
I20260709 12:14:16.442250 12524 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:16.445534 12599 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:14:16.523109 22174 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.447765 22588 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:16.523698 22174 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.450667 22279 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:16.456614 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:14:16.524989 12572 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.009s	sys 0.017s
W20260709 12:14:16.525350 12572 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.603s	user 0.009s	sys 0.017s
W20260709 12:14:16.525604 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 616956us
I20260709 12:14:16.454866 12680 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:16.458575 22665 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:16.530162 12605 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 32.
I20260709 12:14:16.398819 12423 catalog_manager.cc:5697] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 21 to 25. New cstate: current_term: 25 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:16.460017 22495 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 25.17, Last appended by leader: 17, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.402025 22664 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:14:16.462496 22332 consensus_queue.cc:1243] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 27.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:14:16.462901 22482 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.405480 22650 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.465001 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:16.467422 22640 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.468338 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:16.469725 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:16.549798 12528 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Advancing to term 33
W20260709 12:14:16.549945 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 80313.7us
W20260709 12:14:16.552115 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.471553 22553 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:16.472364 12600 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 33 index: 19. (index mismatch)
I20260709 12:14:16.474431 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:16.477782 12602 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 32: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:16.555229 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:16.479624 12646 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:16.477819 12675 raft_consensus.cc:2393] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 24: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:16.464494 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:16.486440 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:16.556974 12531 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 31.
I20260709 12:14:16.483080 12527 rpcz_store.cc:276] 0709 12:14:14.940235 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.218691 (+1278456us) service_pool.cc:224] Handling call
0709 12:14:16.287543 (+ 68852us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:16.298398 (+ 10855us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:16.298430 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:16.298898 (+   468us) log.cc:844] Serialized 26 byte log entry
0709 12:14:16.379480 (+ 80582us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:14:16.379532 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.379635 (+   103us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:16.410201 (+ 30566us) raft_consensus.cc:1685] finished
0709 12:14:16.410221 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:16.410355 (+   134us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":10493,"raft.queue_time_us":38729,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":9018,"thread_start_us":80224,"threads_started":1,"wal-append.queue_time_us":108122}
W20260709 12:14:16.496522 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 179687us
I20260709 12:14:16.517091 12606 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:14:16.518651 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 4563) took 1116 ms. Trace:
I20260709 12:14:16.558882 12679 rpcz_store.cc:276] 0709 12:14:15.402773 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.403083 (+   310us) service_pool.cc:224] Handling call
0709 12:14:16.518364 (+1115281us) spinlock_profiling.cc:243] Waited 570 ms on lock 0x7b54003d4fd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:16.518573 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1310275328}
I20260709 12:14:16.523921 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:16.525018 12674 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:14:16.370843 22627 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:16.527078 12572 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:16.571357 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 44370.6us
I20260709 12:14:16.572264 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.573184 12572 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:16.522789 12522 rpcz_store.cc:276] 0709 12:14:14.011546 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.963319 (+1951773us) service_pool.cc:224] Handling call
0709 12:14:15.963732 (+   413us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:15.964175 (+   443us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:14:16.230554 (+266379us) log.cc:844] Serialized 14 byte log entry
0709 12:14:16.436850 (+206296us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:16.436928 (+    78us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:16.437003 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.437091 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:16.437355 (+   264us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":301217,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":42,"raft.queue_time_us":24802,"raft.run_cpu_time_us":88,"raft.run_wall_time_us":88,"thread_start_us":435115,"threads_started":3,"wal-append.queue_time_us":150327}
I20260709 12:14:16.574362 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:16.575006 12522 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 29.
I20260709 12:14:16.497707 22628 raft_consensus.cc:740] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:16.483047 22527 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:16.609534 12523 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 31. Current term is 32. Ops: []
I20260709 12:14:16.510730 22503 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.483144 22478 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:16.538012 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:16.497843 22247 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:16.553881 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:16.563098 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.568642 12528 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 33.
I20260709 12:14:16.556907 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:16.534435 22582 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.576079 12572 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 23 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:16.578955 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3865) took 1028 ms. Trace:
I20260709 12:14:16.579728 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3856) took 1123 ms. Trace:
I20260709 12:14:16.582916 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.583462 12680 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:14:16.584093 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3787) took 2830 ms. Trace:
I20260709 12:14:16.536624 22664 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:16.576795 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3714) took 2565 ms. Trace:
I20260709 12:14:16.590399 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:16.593538 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 114077us
I20260709 12:14:16.550783 22620 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:16.610802 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.606885 13305 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:16.575302 22588 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.087s
I20260709 12:14:16.608115 22509 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:14:16.664911 22509 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:16.665407 22509 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:16.650453 22478 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:16.533456 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:16.613147 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.669605 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.617784 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:16.665900 22664 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:16.613567 22174 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:16.669543 22488 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Leader election won for term 23
I20260709 12:14:16.669727 22588 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.658562 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:16.670188 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 63512.4us
I20260709 12:14:16.617677 22591 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:14:16.646296 22495 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:16.671262 22582 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.664271 22650 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:16.668514 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 114706us
I20260709 12:14:16.668633 12677 rpcz_store.cc:276] 0709 12:14:15.456528 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.456668 (+   140us) service_pool.cc:224] Handling call
0709 12:14:16.579553 (+1122885us) spinlock_profiling.cc:243] Waited 911 ms on lock 0x7b54003bcc54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:16.579666 (+   113us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2096412032}
W20260709 12:14:16.671489 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 95323.5us
I20260709 12:14:16.712980 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.675472 12678 rpcz_store.cc:276] 0709 12:14:15.550756 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.630180 (+ 79424us) service_pool.cc:224] Handling call
0709 12:14:16.578805 (+948625us) spinlock_profiling.cc:243] Waited 498 ms on lock 0x7b54003d40d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:16.578903 (+    98us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1146348416}
I20260709 12:14:16.676182 12529 rpcz_store.cc:276] 0709 12:14:13.753869 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.758804 (+2004935us) service_pool.cc:224] Handling call
0709 12:14:16.167130 (+408326us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:16.167855 (+   725us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.583872 (+416017us) spinlock_profiling.cc:243] Waited 408 ms on lock 0x7b54003cc3d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:16.584037 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":938084608}
I20260709 12:14:16.677385 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:16.713143 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:16.678043 12676 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 31.
I20260709 12:14:16.671030 12675 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:16.602631 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:16.678874 22247 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.549278 22482 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.617128 22670 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.672652 12671 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:16.588891 22665 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:16.592120 22611 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:14:16.679877 12605 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:16.672053 12525 rpcz_store.cc:276] 0709 12:14:14.011275 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:15.796946 (+1785671us) service_pool.cc:224] Handling call
0709 12:14:15.799924 (+  2978us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:15.944487 (+144563us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:16.576577 (+632090us) spinlock_profiling.cc:243] Waited 142 ms on lock 0x7b540042a4d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:16.576745 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":326206848}
I20260709 12:14:16.685155 12674 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
I20260709 12:14:16.681497 12679 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:14:16.693497 12604 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
W20260709 12:14:16.696143 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.674243 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:16.672300 22640 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:16.575875 22666 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:14:16.746738 22488 raft_consensus.cc:697] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:16.762617 22666 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:16.763101 22666 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:16.763484 22488 consensus_queue.cc:237] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 23, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.767180 12679 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 22 index: 13. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:14:16.670842 22654 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:16.753204 13305 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:16.770962 22553 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:16.767180 12525 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 24 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:14:16.772858 22503 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:16.718503 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.717767 22174 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:16.723140 12676 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24. Ops: [23.15-23.15]
I20260709 12:14:16.723248 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:16.743368 12671 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 27. Current term is 28. Ops: []
W20260709 12:14:16.743892 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 141358us
I20260709 12:14:16.708966 22494 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.750787 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:16.755580 12522 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 30. Current term is 31. Ops: [30.15-30.15]
I20260709 12:14:16.792557 12531 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:16.758246 22673 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:14:16.796120 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 72951.1us
I20260709 12:14:16.797747 12572 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:16.750443 22624 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:14:16.801666 12571 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:16.776937 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:16.783159 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 109004us
I20260709 12:14:16.793475 12522 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:16.794647 12676 raft_consensus.cc:3055] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:14:16.749383 22592 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.748306 22129 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:14:16.740507 22478 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:16.803553 12572 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:16.655759 22674 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.823556 12676 raft_consensus.cc:740] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:16.824144 12671 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 30 index: 15. Preceding OpId from leader: term: 31 index: 15. (term mismatch)
W20260709 12:14:16.842401 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.847374 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:16.849246 12671 pending_rounds.cc:85] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 14
I20260709 12:14:16.716208 22588 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:16.851970 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.794467 22632 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:16.849783 12671 pending_rounds.cc:107] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 30.15
W20260709 12:14:16.893429 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.791836 22611 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:16.850725 12676 consensus_queue.cc:260] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.803186 22652 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:14:16.826965 22469 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:16.799845 22673 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:16.865236 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 63651.5us
I20260709 12:14:16.827234 22129 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:16.745512 22628 consensus_queue.cc:260] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 32.17, Last appended by leader: 17, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.750752 22670 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:16.798967 22494 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:16.802974 22610 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.108s
I20260709 12:14:16.959597 12671 raft_consensus.cc:2889] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:16.922108 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:16.935895 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:16.905628 12531 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:14:16.969892 12675 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:14:16.955246 12673 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 32.
I20260709 12:14:16.856027 22485 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:16.855759 22674 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:16.984848 22478 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:16.992569 22527 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:16.994767 22482 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:16.998248 22672 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.101s	user 0.001s	sys 0.000s
I20260709 12:14:17.002938 12529 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:14:17.006672 22652 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:16.986289 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:16.850237 22592 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:17.057071 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 70966.6us
I20260709 12:14:16.913815 22277 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:17.013128 22129 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:17.030263 12674 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 29. Current term is 30. Ops: [29.15-29.15]
I20260709 12:14:17.049115 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:17.033675 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:17.003590 22673 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:16.911645 22644 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:17.055904 12529 raft_consensus.cc:606] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:17.055768 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 315071us
I20260709 12:14:17.031757 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:17.060189 12673 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
W20260709 12:14:17.061101 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.063913 12529 raft_consensus.cc:614] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:17.036114 22485 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:17.072201 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 538774us
I20260709 12:14:17.071041 12496 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:17.074842 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:17.076484 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:17.073088 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:17.055292 22610 consensus_queue.cc:1243] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 29.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:14:17.048532 22506 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.047827 22582 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:17.075927 12496 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:17.108465 12494 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:14:17.111941 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 358821us
I20260709 12:14:17.098995 12671 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:14:17.061712 22469 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election won for term 30
W20260709 12:14:17.110562 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 43492.1us
I20260709 12:14:17.112951 13305 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:17.115635 12496 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:17.109175 22591 raft_consensus.cc:697] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:17.058975 22678 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:17.063360 22644 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:17.080397 22485 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:17.133198 12529 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 25 index: 14. (index mismatch)
I20260709 12:14:17.134146 12494 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:17.115502 22553 raft_consensus.cc:2764] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Leader pre-election decision vote started in defunct term 31: won
W20260709 12:14:17.143683 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:17.147977 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.138677 12496 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:14:17.163519 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 316271us
I20260709 12:14:17.057106 22478 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:17.112744 22503 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:14:17.187163 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 53122.3us
I20260709 12:14:17.188580 12494 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:14:17.182199 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 287688us
I20260709 12:14:17.107744 22277 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.066s
I20260709 12:14:17.195668 12494 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:17.163558 22668 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:14:17.145517 22469 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:17.144761 22680 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:14:17.139055 22678 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:17.196149 22627 consensus_meta.cc:287] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.584s	user 0.006s	sys 0.006s
I20260709 12:14:17.223446 12525 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:14:17.138109 22654 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:17.145563 22644 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:17.179970 22624 raft_consensus.cc:697] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:17.078289 22611 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:17.241921 12530 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:14:17.115744 22506 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:17.244660 12527 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:14:17.194751 22586 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:17.293970 12423 catalog_manager.cc:5697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 30 to 33. New cstate: current_term: 33 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:14:17.228466 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 161511us
I20260709 12:14:17.197479 22174 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:14:17.205224 22503 raft_consensus.cc:697] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:17.252964 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3873) took 1580 ms. Trace:
I20260709 12:14:17.304312 12678 rpcz_store.cc:276] 0709 12:14:15.672519 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.718387 (+1045868us) service_pool.cc:224] Handling call
0709 12:14:16.718769 (+   382us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:16.718986 (+   217us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:16.719008 (+    22us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:16.740965 (+ 21957us) log.cc:844] Serialized 26 byte log entry
0709 12:14:17.148145 (+407180us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:17.148213 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:17.148301 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:17.252701 (+104400us) raft_consensus.cc:1685] finished
0709 12:14:17.252724 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:17.252891 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":9472,"thread_start_us":295732,"threads_started":1,"wal-append.queue_time_us":455731}
W20260709 12:14:17.230295 22509 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.564s	user 0.013s	sys 0.000s
I20260709 12:14:17.231771 22588 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:17.163990 22664 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:17.257526 22666 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:17.267701 22277 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:14:17.268661 22469 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:17.221189 22550 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:17.163800 22610 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.063s
I20260709 12:14:17.275733 12525 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:14:17.297730 13302 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:17.196389 22632 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:17.233407 22680 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.316398 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4577) took 1033 ms. Trace:
I20260709 12:14:17.367002 12679 rpcz_store.cc:276] 0709 12:14:16.282779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.770458 (+487679us) service_pool.cc:224] Handling call
0709 12:14:16.770738 (+   280us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:16.770938 (+   200us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:14:17.024482 (+253544us) log.cc:844] Serialized 14 byte log entry
0709 12:14:17.315990 (+291508us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:17.316050 (+    60us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:14:17.316113 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:17.316187 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:17.316331 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16760,"prepare.queue_time_us":433838,"prepare.run_cpu_time_us":64,"prepare.run_wall_time_us":66,"thread_start_us":517971,"threads_started":2}
I20260709 12:14:17.317112 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1160) took 1348 ms. Trace:
I20260709 12:14:17.226018 22686 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:17.247429 22681 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:17.231732 22678 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:17.334793 12675 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
W20260709 12:14:17.306948 22665 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.531s	user 0.007s	sys 0.005s
W20260709 12:14:17.380942 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 215870us
I20260709 12:14:17.270184 22684 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:17.198171 22652 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:17.357553 12525 raft_consensus.cc:606] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:17.364457 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 66817.8us
W20260709 12:14:17.365243 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 172271us
W20260709 12:14:17.337126 22594 consensus_meta.cc:287] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.849s	user 0.009s	sys 0.004s
I20260709 12:14:17.368937 12677 rpcz_store.cc:276] 0709 12:14:15.968821 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.716028 (+747207us) service_pool.cc:224] Handling call
0709 12:14:16.716341 (+   313us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:16.716557 (+   216us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:14:17.020245 (+303688us) log.cc:844] Serialized 14 byte log entry
0709 12:14:17.316088 (+295843us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:17.316307 (+   219us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:17.316449 (+   142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:17.316646 (+   197us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:17.316942 (+   296us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":451935,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":10482,"thread_start_us":570085,"threads_started":2}
W20260709 12:14:17.373154 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 246435us
I20260709 12:14:17.239495 22627 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:17.350761 22277 consensus_queue.cc:1243] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 27.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:14:17.381891 12572 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:17.385490 13302 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:17.350734 22174 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:17.377031 22680 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:17.326139 22509 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:17.382100 22682 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:14:17.402297 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:17.384018 22684 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.404320 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:17.401454 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:17.454851 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.382723 22668 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:17.384186 22494 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:17.402747 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:17.416555 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:17.456256 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:17.324975 22591 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:17.462539 13338 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 24: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:14:17.389947 22594 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:17.427893 12680 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 30. Current term is 31. Ops: []
I20260709 12:14:17.424448 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:17.476673 13334 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:17.393682 22527 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:17.473248 13339 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:14:17.478505 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:17.479000 22679 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.479211 22682 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 33 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:14:17.482470 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 100683us
W20260709 12:14:17.463800 22620 consensus_meta.cc:287] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.794s	user 0.014s	sys 0.000s
I20260709 12:14:17.547155 12602 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 24. Current term is 25. Ops: []
W20260709 12:14:17.547220 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 138918us
I20260709 12:14:17.548354 12525 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed"
dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:48476
I20260709 12:14:17.547353 13331 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24. Ops: [23.17-23.17]
I20260709 12:14:17.457464 22588 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:14:17.457122 22690 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.561900 12524 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 30.
W20260709 12:14:17.578505 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 350652us
W20260709 12:14:17.563841 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 360069us
W20260709 12:14:17.564742 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 451839us
I20260709 12:14:17.581912 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:17.582270 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:17.458859 22674 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:17.582897 12522 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 28.
I20260709 12:14:17.582994 12530 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:17.583164 22674 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:17.571698 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4584) took 1112 ms. Trace:
I20260709 12:14:17.574752 12525 raft_consensus.cc:606] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:17.401225 22670 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:17.570749 22586 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:17.459692 22684 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:17.584390 12525 raft_consensus.cc:614] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:17.584201 12671 rpcz_store.cc:276] 0709 12:14:16.459557 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.117072 (+657515us) service_pool.cc:224] Handling call
0709 12:14:17.118945 (+  1873us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:17.119508 (+   563us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:14:17.309487 (+189979us) log.cc:844] Serialized 14 byte log entry
0709 12:14:17.571268 (+261781us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:17.571329 (+    61us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:17.571390 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:17.571460 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:17.571605 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18672,"prepare.queue_time_us":129700,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"thread_start_us":431987,"threads_started":3}
I20260709 12:14:17.382305 22686 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:17.590253 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:17.564205 22527 consensus_queue.cc:1048] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:14:17.406262 22665 consensus_queue.cc:1059] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.153s
I20260709 12:14:17.607903 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:17.608562 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:17.593703 13339 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 32. Current term is 33. Ops: []
I20260709 12:14:17.614449 12525 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 28: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:17.616035 12530 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 28.
W20260709 12:14:17.609720 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 217820us
I20260709 12:14:17.619436 13336 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 34.
I20260709 12:14:17.620646 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:17.621006 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:17.621483 22627 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:17.607234 22690 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:17.622537 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:17.623749 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:17.624257 22701 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.030s
I20260709 12:14:17.625000 22690 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:17.625286 22701 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:14:17.625767 22701 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:17.626937 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:17.626786 22701 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:17.400898 22503 consensus_queue.cc:237] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 23.15, Last appended by leader: 15, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:17.628191 22701 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:14:17.629611 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 49583.1us
I20260709 12:14:17.630465 22714 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.001s
I20260709 12:14:17.632068 12646 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:17.632499 22610 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.016s
I20260709 12:14:17.633203 22686 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:17.634351 22620 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:14:17.634641 22610 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 25.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:14:17.635527 22692 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:14:17.636549 22692 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:14:17.636287 22682 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.22, Last appended by leader: 22, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:17.637120 22692 raft_consensus.cc:697] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:17.466110 22632 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:17.638234 22692 consensus_queue.cc:237] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:17.639933 22717 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:14:17.634060 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:17.630757 22696 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.178s	user 0.000s	sys 0.000s
I20260709 12:14:17.643644 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:17.650588 12522 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:17.644198 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:17.651728 12530 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:17.576483 22550 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:17.651728 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:17.658862 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50383.4us
I20260709 12:14:17.661239 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:17.625662 22690 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:17.669023 12494 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:17.635251 22620 consensus_queue.cc:1243] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 24.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:14:17.410279 22624 consensus_queue.cc:237] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:17.702576 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:17.705394 13333 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 31.
I20260709 12:14:17.658691 22277 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.673710 22714 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.063s
I20260709 12:14:17.663472 22686 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:14:17.727072 12530 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 33.
I20260709 12:14:17.721765 12522 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 31.
I20260709 12:14:17.662876 22670 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:17.668054 22588 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:17.577431 22679 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:17.749807 22485 consensus_meta.cc:287] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.565s	user 0.008s	sys 0.004s
I20260709 12:14:17.750751 22679 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:17.751078 22495 consensus_meta.cc:287] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.034s	user 0.009s	sys 0.004s
I20260709 12:14:17.609031 22174 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.16, Last appended by leader: 16, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:17.741504 22277 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:17.699307 22708 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:17.750407 22485 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:17.771615 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 212438us
W20260709 12:14:17.728358 22703 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.133s	user 0.000s	sys 0.001s
I20260709 12:14:17.730602 22664 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:17.805851 12525 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 29 index: 22. (index mismatch)
I20260709 12:14:17.730573 22690 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:17.754889 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4580) took 1415 ms. Trace:
I20260709 12:14:17.715904 22627 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:17.807559 22627 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:17.807880 22690 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:17.810432 22627 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:17.744680 22670 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:17.822973 22490 raft_consensus.cc:3055] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Stepping down as leader of term 22
I20260709 12:14:17.823487 22490 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:17.824316 22490 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 22, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:17.825609 22490 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 22 FOLLOWER]: Advancing to term 23
I20260709 12:14:17.755927 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:17.752028 22495 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:17.744962 22673 consensus_meta.cc:287] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.638s	user 0.009s	sys 0.005s
I20260709 12:14:17.658329 22668 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.16, Last appended by leader: 16, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:17.853420 12522 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:14:17.813047 22670 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:17.811134 12673 rpcz_store.cc:276] 0709 12:14:16.339776 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.077525 (+737749us) service_pool.cc:224] Handling call
0709 12:14:17.077919 (+   394us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:17.081231 (+  3312us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:17.081388 (+   157us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:17.081910 (+   522us) log.cc:844] Serialized 26 byte log entry
0709 12:14:17.334647 (+252737us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:17.334694 (+    47us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:17.334762 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:17.754613 (+419851us) raft_consensus.cc:1685] finished
0709 12:14:17.754634 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:17.754805 (+   171us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":173246,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":96,"thread_start_us":196998,"threads_started":1,"wal-append.queue_time_us":490132}
I20260709 12:14:17.742461 22686 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:14:17.910579 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 79009.2us
I20260709 12:14:17.844071 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:17.847435 22495 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:17.962743 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1253) took 1265 ms. Trace:
I20260709 12:14:17.818698 22679 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:17.961076 13305 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:17.730841 22586 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 28.14, Last appended by leader: 14, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:17.963218 12531 rpcz_store.cc:276] 0709 12:14:16.697167 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.026785 (+329618us) service_pool.cc:224] Handling call
0709 12:14:17.057052 (+ 30267us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:17.061984 (+  4932us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:17.062019 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:17.108335 (+ 46316us) log.cc:844] Serialized 26 byte log entry
0709 12:14:17.325709 (+217374us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:14:17.325865 (+   156us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:17.326227 (+   362us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:17.962502 (+636275us) raft_consensus.cc:1685] finished
0709 12:14:17.962521 (+    19us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:17.962667 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2470,"raft.queue_time_us":4897,"raft.run_cpu_time_us":187,"raft.run_wall_time_us":16263,"thread_start_us":187340,"threads_started":1,"wal-append.queue_time_us":309839}
I20260709 12:14:17.965489 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:17.983049 12525 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 31: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:14:17.840811 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 171860us
I20260709 12:14:17.969409 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:17.975328 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:17.830091 22553 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:14:17.791141 22588 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:17.980108 12527 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 31 index: 15. (index mismatch)
I20260709 12:14:17.975549 13305 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:17.841007 22503 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 13, Time since last communication: 0.121s
I20260709 12:14:17.986407 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:17.988099 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:17.987251 12494 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:17.990087 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:17.995590 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:17.994187 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:17.999140 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:17.797570 22708 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:18.000221 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:18.005059 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:18.005309 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:17.996609 12531 raft_consensus.cc:2376] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27. Re-sending same reply.
W20260709 12:14:17.997476 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.006163 12525 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 27: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:18.007788 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:18.010905 12531 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 31 index: 16. (index mismatch)
I20260709 12:14:18.011097 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:17.986558 22664 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:17.830302 22550 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:17.991842 22721 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.029s
I20260709 12:14:17.998823 22527 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:18.041357 12522 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 31.
I20260709 12:14:18.030711 13334 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:18.036159 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:18.037467 12525 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 30. Current term is 31. Ops: [28.17-30.18]
I20260709 12:14:17.912599 22673 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:18.003662 22553 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:18.044010 12646 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:18.044306 22720 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:14:18.026257 22664 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:18.021610 22714 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:17.996438 22712 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.091s
W20260709 12:14:18.077713 12672 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 1165) took 2031 ms (client timeout 1538 ms). Trace:
I20260709 12:14:18.101354 22712 consensus_queue.cc:1059] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 21.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.002s
I20260709 12:14:18.104702 22469 raft_consensus.cc:3055] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:14:18.065400 22555 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:18.069830 22527 raft_consensus.cc:697] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:18.065521 13330 raft_consensus.cc:2368] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 29. Current term is 30.
I20260709 12:14:18.106945 22527 consensus_queue.cc:237] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:18.105423 22469 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:18.061419 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:18.065069 22721 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.112365 22721 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.064155 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:18.110944 22469 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:18.103186 12672 rpcz_store.cc:269] 0709 12:14:16.045950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.063596 (+ 17646us) service_pool.cc:224] Handling call
0709 12:14:18.071439 (+2007843us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b54003d45d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:18.077587 (+  6148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1264792320}
I20260709 12:14:18.116430 22721 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:18.118036 22469 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:18.057955 22692 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:14:18.120090 22692 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:18.123205 22485 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:18.016036 22686 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.19, Last appended by leader: 19, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.130611 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:18.041798 22710 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.069s
I20260709 12:14:18.062180 22664 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:18.060359 22690 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.133245 22690 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:18.081969 12569 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:18.135500 22690 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:18.135878 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 54040.5us
I20260709 12:14:18.136106 22690 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:18.054493 22705 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.100s
I20260709 12:14:18.063140 22711 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.143329 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:18.051874 22644 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.752s	user 0.010s	sys 0.005s
I20260709 12:14:18.147607 13333 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:14:18.154449 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:18.068872 22714 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:18.142361 22707 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.310s	user 0.001s	sys 0.000s
I20260709 12:14:18.192673 22727 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:18.193738 22727 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:18.194411 22727 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:18.203747 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:18.160241 22644 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.209355 22644 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:18.140194 22652 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.067s
W20260709 12:14:18.158424 22129 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.089s	user 0.013s	sys 0.001s
I20260709 12:14:18.054301 22718 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.125s
I20260709 12:14:18.214277 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:18.216210 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:18.159129 22503 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.158373 22277 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:18.230765 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:18.317027 12604 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 30.
I20260709 12:14:18.231930 12603 raft_consensus.cc:1240] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 32. Current term is 33. Ops: [29.17-32.19]
I20260709 12:14:18.273833 12599 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 28. Current term is 29. Ops: [28.17-28.17]
I20260709 12:14:18.211127 22712 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.251900 13302 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:18.319048 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 67260.4us
W20260709 12:14:18.210304 22709 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.323s	user 0.000s	sys 0.001s
I20260709 12:14:18.232151 22586 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.049345 22632 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.130s
I20260709 12:14:18.252800 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:18.158910 22711 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.252336 22129 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:18.180331 22714 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:18.121420 22692 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.264649 22588 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:18.253525 12598 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:14:18.287524 22728 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.056s	user 0.000s	sys 0.000s
I20260709 12:14:18.284348 13333 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7bd864b75c34027b05582a55b152877"
dest_uuid: "e56d487365394ac5ac5f1002926a5645"
mode: GRACEFUL
new_leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
 from {username='slave'} at 127.0.0.1:34880
I20260709 12:14:18.328115 12598 raft_consensus.cc:1217] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Deduplicated request from leader. Original: 24.14->[26.15-31.18]   Dedup: 26.15->[28.16-31.18]
I20260709 12:14:18.328455 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:18.311800 22718 consensus_queue.cc:1243] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 28.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:14:18.210525 22485 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.272346 22495 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:18.322669 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:18.067314 22553 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.328758 12598 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 31. Current term is 32. Ops: [26.15-31.18]
W20260709 12:14:18.291190 22704 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.307s	user 0.000s	sys 0.001s
I20260709 12:14:18.315143 12600 raft_consensus.cc:2410] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:14:18.329456 12604 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 29: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:18.332427 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:18.319084 22722 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.304s	user 0.001s	sys 0.000s
I20260709 12:14:18.310865 22720 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 33 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:18.340968 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:14:18.342101 22735 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.072s	user 0.000s	sys 0.000s
I20260709 12:14:18.343430 22664 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:18.347154 22678 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.954s	user 0.012s	sys 0.001s
I20260709 12:14:18.352564 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:18.353600 12599 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 27: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:18.355480 12603 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 30.
I20260709 12:14:18.356670 12605 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 33: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:18.361400 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:18.371531 12604 raft_consensus.cc:2410] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 25 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 13.
I20260709 12:14:18.371781 12603 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 30. Current term is 31. Ops: [30.15-30.15]
I20260709 12:14:18.372200 22720 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:18.329332 22712 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.372918 12604 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 32. Current term is 33. Ops: [28.15-32.17]
I20260709 12:14:18.374289 12599 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27. Ops: [24.14-26.15]
I20260709 12:14:18.374643 22712 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:18.375402 22129 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:18.375507 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:18.353899 22485 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.377259 12603 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:14:18.378388 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:18.366912 12598 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 31.
I20260709 12:14:18.379633 12603 raft_consensus.cc:2410] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 27 because replica has last-logged OpId of term: 24 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:14:18.376384 12599 raft_consensus.cc:2393] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 27: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:14:18.386806 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:18.387305 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.353904 22682 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:18.388592 12604 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 24 index: 18. Preceding OpId from leader: term: 27 index: 20. (index mismatch)
I20260709 12:14:18.390547 13339 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 29: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:14:18.372721 22711 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:18.398510 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.329052 22586 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.400429 12572 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:18.401346 22690 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 11, Time since last communication: 0.000s
I20260709 12:14:18.402017 22708 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:18.402807 13336 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 899) took 1001 ms. Trace:
I20260709 12:14:18.449788 13336 rpcz_store.cc:276] 0709 12:14:17.400973 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.401164 (+   191us) service_pool.cc:224] Handling call
0709 12:14:18.402563 (+1001399us) spinlock_profiling.cc:243] Waited 164 ms on lock 0x7b54006b0154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:18.402723 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":377462016}
I20260709 12:14:18.451457 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:18.408836 22482 raft_consensus.cc:3055] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:14:18.409682 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3886) took 2243 ms. Trace:
I20260709 12:14:18.452829 12674 rpcz_store.cc:276] 0709 12:14:16.166184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.767238 (+601054us) service_pool.cc:224] Handling call
0709 12:14:17.030000 (+262762us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:17.059301 (+ 29301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:18.409458 (+1350157us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b54000aafd0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:18.409613 (+   155us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":603471616}
I20260709 12:14:18.415052 22592 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:18.416693 22681 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:18.416854 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:18.368430 22673 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:18.419114 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:18.413432 22509 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:18.400252 22129 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:14:18.436908 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 24 candidate_status { last_received { term: 23 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:18.321590 22503 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.439927 12572 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:18.464380 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:18.472215 13336 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 31.
I20260709 12:14:18.465920 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:18.474344 12572 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:18.380170 22678 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:18.445493 22711 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:18.477823 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:18.480983 13334 raft_consensus.cc:2410] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 28 because replica has last-logged OpId of term: 25 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 18.
I20260709 12:14:18.446123 22485 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:18.477496 12569 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:18.466173 22482 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:18.466833 22681 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:18.448745 22610 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.034s
I20260709 12:14:18.490972 13332 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 30. Current term is 32. Ops: []
W20260709 12:14:18.482734 22490 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.643s	user 0.012s	sys 0.000s
I20260709 12:14:18.480389 22509 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:18.483863 22624 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
W20260709 12:14:18.493765 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:18.492676 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 93696.5us
I20260709 12:14:18.496268 22482 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.487138 22712 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election won for term 31
W20260709 12:14:18.496057 12496 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.734s (SENT) [suppressed 29 similar messages]
I20260709 12:14:18.497040 22586 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:18.528483 12572 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:18.532050 22711 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election won for term 27
W20260709 12:14:18.532454 22745 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:14:18.532632 22711 raft_consensus.cc:697] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:18.534509 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:18.533826 22711 consensus_queue.cc:237] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 25.12, Last appended by leader: 12, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.535122 22666 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:18.536128 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:18.536962 12531 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:18.533869 22506 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:18.539430 22678 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:18.540927 22624 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:14:18.538851 22478 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:18.500645 22712 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:18.547466 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:18.550949 12522 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:18.554482 12531 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 29.
I20260709 12:14:18.541128 12572 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:18.556289 13305 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:14:18.568043 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 69326.7us
I20260709 12:14:18.572540 12522 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 32.
I20260709 12:14:18.573681 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:18.574185 13305 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:18.576025 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:18.577320 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:18.579176 12525 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:14:18.579121 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:18.546096 22701 consensus_meta.cc:287] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.917s	user 0.010s	sys 0.003s
I20260709 12:14:18.546478 22712 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:18.581219 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:18.546657 22628 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:18.577430 22129 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:14:18.579347 22720 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:18.581063 22611 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:18.582659 22706 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:14:18.601022 22485 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:18.612510 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1242) took 3975 ms. Trace:
I20260709 12:14:18.614923 12523 rpcz_store.cc:276] 0709 12:14:14.634464 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.166269 (+1531805us) service_pool.cc:224] Handling call
0709 12:14:16.166775 (+   506us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:16.610211 (+443436us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:18.592540 (+1982329us) spinlock_profiling.cc:243] Waited 443 ms on lock 0x7b540027f6d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:18.609306 (+ 16766us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1017831296}
W20260709 12:14:18.606055 22680 consensus_meta.cc:287] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.154s	user 0.014s	sys 0.000s
I20260709 12:14:18.622416 22527 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 31.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:14:18.624140 22666 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:14:18.624576 22666 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:18.616919 22485 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:18.625155 22720 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 29 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:18.625268 22485 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:18.625583 22666 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:18.627082 22666 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:18.617259 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:14:18.637832 22670 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.678s	user 0.011s	sys 0.000s
I20260709 12:14:18.633580 22720 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:18.637895 12523 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 33: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:14:18.642691 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:48492 (request call id 1149) took 1240 ms. Trace:
I20260709 12:14:18.680346 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:18.643563 12524 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53864 (request call id 3820) took 2337 ms (client timeout 2192 ms). Trace:
I20260709 12:14:18.681177 12522 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 31.
W20260709 12:14:18.681361 12524 rpcz_store.cc:269] 0709 12:14:16.305825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.523655 (+217830us) service_pool.cc:224] Handling call
0709 12:14:18.643377 (+2119722us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b54003cafd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:18.643493 (+   116us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2280036736}
I20260709 12:14:18.643749 22485 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:18.646466 12528 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 29. Current term is 30. Ops: []
W20260709 12:14:18.650794 13330 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:38482 (request call id 900) took 1249 ms (client timeout 1565 ms). Trace:
W20260709 12:14:18.656170 22738 log.cc:927] Time spent T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.305s	user 0.002s	sys 0.000s
W20260709 12:14:18.660678 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:18.683543 13330 rpcz_store.cc:269] 0709 12:14:17.401338 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.404082 (+  2744us) service_pool.cc:224] Handling call
0709 12:14:18.650530 (+1246448us) spinlock_profiling.cc:243] Waited 609 ms on lock 0x7b540061f654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:18.650698 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1400639360}
I20260709 12:14:18.607944 22664 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:18.663600 22676 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.720s	user 0.005s	sys 0.033s
I20260709 12:14:18.664528 22592 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:14:18.670214 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3874) took 2997 ms. Trace:
I20260709 12:14:18.690055 12680 rpcz_store.cc:276] 0709 12:14:15.672824 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.720205 (+1047381us) service_pool.cc:224] Handling call
0709 12:14:16.757249 (+ 37044us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:17.476212 (+718963us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:18.669743 (+1193531us) spinlock_profiling.cc:243] Waited 670 ms on lock 0x7b54000ac3d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:18.670074 (+   331us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1541783424}
W20260709 12:14:18.683945 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.691215 22716 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:18.691586 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.691840 22716 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.692318 22656 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.692982 22656 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.679523 22680 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.660547 22714 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.686810 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:18.679996 12605 rpcz_store.cc:276] 0709 12:14:17.402355 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.402494 (+   139us) service_pool.cc:224] Handling call
0709 12:14:18.642462 (+1239968us) spinlock_profiling.cc:243] Waited 895 ms on lock 0x7b540007b554. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:18.642620 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2059521792}
I20260709 12:14:18.592599 22509 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.025s
I20260709 12:14:18.708060 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:18.711444 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 34 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:18.718919 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:18.695642 22656 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:18.745556 12530 raft_consensus.cc:1240] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 25. Ops: []
I20260709 12:14:18.698103 22653 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.694639 22332 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.695824 22628 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.755343 22628 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.701184 22692 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.756608 22692 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:18.757689 22628 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:18.699407 22725 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:18.713232 12646 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:18.688249 22611 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.701450 22620 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.736795 12523 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
I20260709 12:14:18.685138 22706 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 32 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:18.700613 22718 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.702179 22506 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.702498 22503 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:18.749399 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:18.750378 12529 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 29. Current term is 30. Ops: [27.16-29.17]
I20260709 12:14:18.730914 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:18.701314 22701 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:18.694079 22716 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:18.754467 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:18.758823 22692 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:18.698410 22547 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.797710 12672 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:14:18.684952 22670 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.779290 22628 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:18.747625 22509 consensus_queue.cc:1243] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 27.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:14:18.767160 22653 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.767400 22332 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:18.767936 22650 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.945s	user 0.008s	sys 0.017s
I20260709 12:14:18.787994 12522 raft_consensus.cc:2376] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 30. Re-sending same reply.
I20260709 12:14:18.766521 22656 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:18.821390 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:18.767800 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 57691.6us
I20260709 12:14:18.777030 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:18.800209 12601 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 22. Current term is 23. Ops: []
I20260709 12:14:18.811302 12680 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:14:18.812891 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:18.814899 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:18.815912 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:18.784384 22506 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.796991 22247 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:18.821316 12527 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27. Ops: [23.17-26.18]
I20260709 12:14:18.828105 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:18.783187 22620 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.812767 22692 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:18.808414 22628 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:18.827059 12677 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:18.825688 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:18.830199 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:18.830713 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:18.779040 22611 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:18.820017 22650 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.998s	user 0.008s	sys 0.018s
I20260709 12:14:18.830421 12646 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:18.778275 22725 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.798348 22701 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.787002 22718 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:18.803381 22547 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:19.182173 22739 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.852s	user 0.002s	sys 0.000s
W20260709 12:14:19.071395 22727 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.874s	user 0.009s	sys 0.005s
I20260709 12:14:19.206895 22763 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:19.210857 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:19.211272 22763 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:18.861869 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:18.833470 12671 raft_consensus.cc:2410] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 28 because replica has last-logged OpId of term: 26 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 12.
I20260709 12:14:18.854763 12600 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
W20260709 12:14:19.030534 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.884013 22680 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:18.946729 22503 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.215356 22503 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:18.835987 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:19.216676 12680 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:14:19.217084 22547 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:18.852624 22694 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.029s	user 0.002s	sys 0.000s
I20260709 12:14:19.218021 12680 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 33 index: 19. (term mismatch)
I20260709 12:14:18.853240 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:18.853785 13339 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:14:19.187119 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.828639 22656 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:18.905952 12572 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:19.220523 22509 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:19.220932 22509 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:19.221278 22509 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:18.832568 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:19.221903 22656 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:19.222234 12679 raft_consensus.cc:2410] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 27 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
W20260709 12:14:18.912603 22684 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.314s	user 0.012s	sys 0.004s
I20260709 12:14:19.223011 22684 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:19.224087 13333 raft_consensus.cc:606] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Received request to transfer leadership to TS f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:19.224503 13333 raft_consensus.cc:614] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:14:19.225018 13331 raft_consensus.cc:1240] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 29. Current term is 30. Ops: []
W20260709 12:14:18.886147 22755 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:14:19.225863 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:18.902181 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3733) took 3095 ms. Trace:
I20260709 12:14:19.227427 12528 rpcz_store.cc:276] 0709 12:14:15.807132 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.664996 (+857864us) service_pool.cc:224] Handling call
0709 12:14:18.646281 (+1981285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:18.683210 (+ 36929us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:18.901834 (+218624us) spinlock_profiling.cc:243] Waited 1.98 s on lock 0x7b54003cafd0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:18.902052 (+   218us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":4555686144}
I20260709 12:14:19.229764 12599 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 14. (term mismatch)
I20260709 12:14:19.232303 12599 pending_rounds.cc:85] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 13
I20260709 12:14:19.212718 22763 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 29.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:19.085317 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.168227 22653 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:19.209383 22611 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:18.852262 12605 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:14:18.851307 22129 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:14:19.230063 12526 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
W20260709 12:14:19.230247 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 324392us
I20260709 12:14:18.970971 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1151) took 1565 ms. Trace:
I20260709 12:14:19.230842 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:19.214040 12677 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:14:19.254741 12572 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:19.233048 22701 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:19.254519 12602 rpcz_store.cc:276] 0709 12:14:17.405825 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.405947 (+   122us) service_pool.cc:224] Handling call
0709 12:14:17.456321 (+ 50374us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:17.574466 (+118145us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:18.970581 (+1396115us) spinlock_profiling.cc:243] Waited 90.5 ms on lock 0x7b540023f854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:18.970880 (+   299us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":208226176}
I20260709 12:14:19.258757 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:19.259375 12677 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:19.233882 12599 pending_rounds.cc:107] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 24.14
I20260709 12:14:19.235260 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:19.091426 12524 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
I20260709 12:14:19.261322 12599 raft_consensus.cc:2889] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:18.914805 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:19.262318 12531 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 29: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:14:19.164517 13332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38482 (request call id 903) took 1699 ms. Trace:
I20260709 12:14:18.885003 12606 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:14:19.078493 22485 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:19.100613 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3837) took 1084 ms. Trace:
I20260709 12:14:18.951048 22478 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.265122 12527 rpcz_store.cc:276] 0709 12:14:18.015886 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:18.345610 (+329724us) service_pool.cc:224] Handling call
0709 12:14:18.345998 (+   388us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:18.828176 (+482178us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.099947 (+271771us) spinlock_profiling.cc:243] Waited 475 ms on lock 0x7b54003f8c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:19.100538 (+   591us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1092392192}
I20260709 12:14:18.857623 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:19.237385 12603 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 26 index: 18. (index mismatch)
I20260709 12:14:19.262759 13332 rpcz_store.cc:276] 0709 12:14:17.423552 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.456011 (+ 32459us) service_pool.cc:224] Handling call
0709 12:14:17.478026 (+ 22015us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:18.491437 (+1013411us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.122895 (+631458us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54001b49d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:19.123088 (+   193us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2329138432}
I20260709 12:14:19.238711 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:19.211124 22727 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:19.218503 12680 pending_rounds.cc:85] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 18
I20260709 12:14:19.270171 12680 pending_rounds.cc:107] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 32.19
I20260709 12:14:19.270503 12680 raft_consensus.cc:2889] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:19.191335 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:18.862906 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:19.097649 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:18.946887 22666 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:18.886277 22706 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:19.207918 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 377550us
W20260709 12:14:19.279578 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.209693 22692 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:19.249495 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:19.251300 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 182677us
I20260709 12:14:19.065982 22716 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:19.260818 12675 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 4585) took 2459 ms (client timeout 1789 ms). Trace:
W20260709 12:14:19.284296 12675 rpcz_store.cc:269] 0709 12:14:16.792692 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.031480 (+238788us) service_pool.cc:224] Handling call
0709 12:14:19.251507 (+2220027us) spinlock_profiling.cc:243] Waited 269 ms on lock 0x7b54003bcc54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:19.251619 (+   112us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":618041216}
W20260709 12:14:19.285205 22759 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.373s	user 0.001s	sys 0.000s
I20260709 12:14:19.285605 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:19.252141 22721 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.251909 22654 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.240824 22708 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:19.286532 22503 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:19.286937 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:19.287550 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:19.277483 22711 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.271077 22620 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:19.258044 22733 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:14:19.253386 22129 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:19.294586 12522 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 32. Current term is 33. Ops: [32.21-32.21]
I20260709 12:14:19.292037 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:19.295240 13339 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:14:19.299914 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:19.325269 12678 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 23.
I20260709 12:14:19.326265 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:19.208545 22628 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:19.310856 12675 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:19.275471 22488 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.282163 22670 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:19.275920 22495 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:19.298384 22721 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:19.327899 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:19.291276 12677 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 32.
I20260709 12:14:19.341714 12680 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
W20260709 12:14:19.332525 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.316100 22332 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:19.282773 22478 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:19.345304 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:19.357955 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.336509 12675 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:14:19.331386 12572 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:19.290796 22666 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:19.352453 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:19.362669 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 89648.2us
I20260709 12:14:19.364496 12677 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:19.366093 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:19.367552 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:19.368278 12677 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:19.292039 22716 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:19.337371 22506 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:19.377672 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:19.317238 22509 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:19.304050 22654 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:19.382063 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:19.383210 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:19.382524 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:19.383890 12603 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:14:19.384483 12522 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 32: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:14:19.315564 22708 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:14:19.384172 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:19.347239 22736 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:14:19.384035 12600 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:14:19.387621 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:19.388795 12680 raft_consensus.cc:2410] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 27 because replica has last-logged OpId of term: 25 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 12:14:19.389729 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:19.390352 12680 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 25
W20260709 12:14:19.399530 13304 leader_election.cc:336] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.538s (SENT)
I20260709 12:14:19.400288 13304 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:19.325358 22711 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:19.379245 22485 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:19.406507 12677 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
I20260709 12:14:19.348428 22495 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:19.414445 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:19.415340 12522 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 31.
I20260709 12:14:19.411770 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 34 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:19.344693 22488 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:19.417889 12677 raft_consensus.cc:2410] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 34 because replica has last-logged OpId of term: 31 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 16.
I20260709 12:14:19.413009 22740 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:19.419394 22721 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:19.392246 22736 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:19.370373 22611 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 28.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.038s
I20260709 12:14:19.422853 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:19.423110 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:19.423808 12522 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:14:19.423632 12677 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
I20260709 12:14:19.392189 22620 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.029s
I20260709 12:14:19.411914 22485 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:19.392349 22506 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.028s
I20260709 12:14:19.432143 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:19.420811 22718 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:19.436069 13339 raft_consensus.cc:3055] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:14:19.436170 13334 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:19.439841 13336 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:19.439821 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:19.456712 13339 raft_consensus.cc:740] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 23 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:19.459309 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:19.427884 22553 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.079s	user 0.007s	sys 0.010s
I20260709 12:14:19.462605 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:19.462980 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:19.413825 22733 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 30.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.018s
I20260709 12:14:19.458861 13339 consensus_queue.cc:260] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.15, Last appended by leader: 15, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:19.474505 12680 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:14:19.391947 22129 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.18, Last appended by leader: 18, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:19.429361 22485 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:19.469202 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:19.476711 13335 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 33 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 32. Current term is 33. Ops: [32.21-32.21]
I20260709 12:14:19.461866 22721 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:14:19.461834 22620 consensus_queue.cc:1243] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 24.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:14:19.507246 22721 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:19.508450 22721 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 29.15, Last appended by leader: 15, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:19.509250 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 22 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:19.510828 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:19.510962 22733 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.000s
I20260709 12:14:19.511672 22733 consensus_queue.cc:1243] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 33.21 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 20
W20260709 12:14:19.493475 22767 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:14:19.511682 12679 raft_consensus.cc:2410] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 26 because replica has last-logged OpId of term: 22 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 14.
I20260709 12:14:19.519297 22488 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:19.519951 12679 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 32 index: 17. Preceding OpId from leader: term: 33 index: 17. (term mismatch)
I20260709 12:14:19.520551 12679 pending_rounds.cc:85] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 16
I20260709 12:14:19.521051 22495 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:19.470779 22553 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.122s	user 0.008s	sys 0.011s
W20260709 12:14:19.493686 22712 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.884s	user 0.013s	sys 0.000s
I20260709 12:14:19.522192 22712 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:19.522840 22711 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:19.524310 22708 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:19.524884 22712 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:19.499231 22692 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:19.500970 22478 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:19.504451 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3841) took 1066 ms. Trace:
I20260709 12:14:19.526597 12523 rpcz_store.cc:276] 0709 12:14:18.438366 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:18.967473 (+529107us) service_pool.cc:224] Handling call
0709 12:14:18.967971 (+   498us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:19.190073 (+222102us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:19.190137 (+    64us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:19.240484 (+ 50347us) log.cc:844] Serialized 26 byte log entry
0709 12:14:19.408319 (+167835us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:14:19.408359 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.408431 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:19.504190 (+ 95759us) raft_consensus.cc:1685] finished
0709 12:14:19.504214 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:19.504374 (+   160us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":97942,"raft.queue_time_us":224656,"raft.run_cpu_time_us":151,"raft.run_wall_time_us":13482,"thread_start_us":167300,"threads_started":1,"wal-append.queue_time_us":261633}
I20260709 12:14:19.528554 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:19.529218 12523 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29.
I20260709 12:14:19.530818 12523 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 29. Current term is 30. Ops: [29.15-29.15]
I20260709 12:14:19.532567 12523 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:19.532439 22712 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 28.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:14:19.533174 22712 consensus_queue.cc:1243] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 30.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:14:19.489315 12522 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:14:19.496236 22129 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:19.525524 12606 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:14:19.534475 12523 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:14:19.520871 12679 pending_rounds.cc:107] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 32.17
I20260709 12:14:19.456920 22725 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:19.537192 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:19.537599 12603 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:19.557212 12679 raft_consensus.cc:2889] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:19.538484 22509 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:19.539417 22681 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.049s	user 0.014s	sys 0.000s
W20260709 12:14:19.533473 22247 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.702s	user 0.010s	sys 0.000s
I20260709 12:14:19.488497 22727 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:19.546554 22712 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:14:19.547207 22654 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:19.571261 12679 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:14:19.494007 22666 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:19.586621 22485 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:19.575059 22490 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:19.603152 22490 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:19.575634 22681 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:19.591840 22775 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:14:19.562090 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 293784us
I20260709 12:14:19.840896 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1264) took 1299 ms. Trace:
I20260709 12:14:19.694969 12423 catalog_manager.cc:5697] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 22 to 24, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 24 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:19.863415 12527 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:14:19.863897 12423 catalog_manager.cc:5697] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 21 to 28. New cstate: current_term: 28 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:19.865309 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:19.866395 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3737) took 2069 ms. Trace:
I20260709 12:14:19.866952 12423 catalog_manager.cc:5697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:19.869064 12605 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 21 index: 14. Preceding OpId from leader: term: 25 index: 16. (index mismatch)
I20260709 12:14:19.869180 12530 rpcz_store.cc:276] 0709 12:14:17.796361 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.796563 (+   202us) service_pool.cc:224] Handling call
0709 12:14:17.796971 (+   408us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:18.746423 (+949452us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.865746 (+1119323us) spinlock_profiling.cc:243] Waited 948 ms on lock 0x7b5400408154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:19.866193 (+   447us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2180968320}
I20260709 12:14:19.871107 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:19.872387 12598 raft_consensus.cc:2393] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 32: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:19.862536 12528 rpcz_store.cc:276] 0709 12:14:18.541475 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:19.228922 (+687447us) service_pool.cc:224] Handling call
0709 12:14:19.229222 (+   300us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:19.258852 (+ 29630us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:14:19.453756 (+194904us) log.cc:844] Serialized 14 byte log entry
0709 12:14:19.839629 (+385873us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:19.839854 (+   225us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:14:19.840210 (+   356us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.840382 (+   172us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:19.840642 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":31691,"prepare.queue_time_us":245614,"prepare.run_cpu_time_us":43,"prepare.run_wall_time_us":45,"raft.queue_time_us":53919,"raft.run_cpu_time_us":502,"raft.run_wall_time_us":8632,"thread_start_us":519361,"threads_started":2}
I20260709 12:14:19.875918 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:19.875557 12423 catalog_manager.cc:5697] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 18 to 22, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 22 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:19.879522 12423 catalog_manager.cc:5697] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 26 to 30, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 30 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:19.846951 22485 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:19.859370 12526 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:19.841727 12523 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:14:19.849957 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3846) took 1225 ms. Trace:
I20260709 12:14:19.873752 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:19.881076 12676 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:14:19.798671 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 301940us
W20260709 12:14:19.855671 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.614101 22681 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:19.891323 12605 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:19.802356 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3839) took 1400 ms. Trace:
I20260709 12:14:19.888440 12524 rpcz_store.cc:276] 0709 12:14:18.624671 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:19.848452 (+1223781us) service_pool.cc:224] Handling call
0709 12:14:19.848999 (+   547us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:19.849390 (+   391us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:19.849414 (+    24us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:19.849481 (+    67us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:14:19.849525 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.849583 (+    58us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:19.849704 (+   121us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:19.619405 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:19.926607 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:19.900236 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3845) took 1305 ms. Trace:
I20260709 12:14:19.928619 12523 rpcz_store.cc:276] 0709 12:14:18.585480 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:19.840134 (+1254654us) service_pool.cc:224] Handling call
0709 12:14:19.840962 (+   828us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:19.888043 (+ 47081us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.890596 (+  2553us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:19.892983 12529 rpcz_store.cc:276] 0709 12:14:18.401958 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:18.410549 (+  8591us) service_pool.cc:224] Handling call
0709 12:14:18.410968 (+   419us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:18.788033 (+377065us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.801224 (+1013191us) spinlock_profiling.cc:243] Waited 339 ms on lock 0x7b54003caad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:19.802029 (+   805us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":779815808}
I20260709 12:14:19.895310 12423 catalog_manager.cc:5697] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 18 to 21, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 21 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:14:19.950217 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:19.955528 12420 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58980 (request call id 186) took 95980 ms (client timeout 15000 ms). Trace:
W20260709 12:14:19.956657 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 91804.4us
I20260709 12:14:19.957000 12672 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 29 index: 15. Preceding OpId from leader: term: 31 index: 16. (index mismatch)
I20260709 12:14:19.799968 22788 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.166s
I20260709 12:14:19.973311 22482 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:19.966481 12423 catalog_manager.cc:5697] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 24 to 27, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 27 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:19.976171 12423 catalog_manager.cc:5697] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 22 to 27, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 27 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:19.973168 12672 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
I20260709 12:14:19.978863 12423 catalog_manager.cc:5697] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 17 to 22, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 22 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:19.981312 12672 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 17. (term mismatch)
I20260709 12:14:19.982466 13331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 1125) took 2501 ms. Trace:
I20260709 12:14:19.982990 13331 rpcz_store.cc:276] 0709 12:14:17.480491 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:17.599283 (+118792us) service_pool.cc:224] Handling call
0709 12:14:17.644361 (+ 45078us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:19.225392 (+1581031us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.981733 (+756341us) spinlock_profiling.cc:243] Waited 1.58 s on lock 0x7b5400469954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:19.982199 (+   466us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3634779392}
I20260709 12:14:19.992136 12598 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 30: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:20.001250 22789 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:14:19.964068 22706 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.002743 22706 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:20.004781 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:19.982297 12672 pending_rounds.cc:85] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 16
I20260709 12:14:20.006361 12672 pending_rounds.cc:107] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 28.17
I20260709 12:14:20.006726 12672 raft_consensus.cc:2889] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:20.009533 12672 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:14:19.615111 22775 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:20.017011 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:20.015132 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:20.019908 12672 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:14:19.957995 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:20.040324 12601 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:20.013988 22775 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:20.043889 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.046523 12676 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 15. (term mismatch)
I20260709 12:14:20.046824 12672 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:20.047040 12676 pending_rounds.cc:85] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 14
I20260709 12:14:20.047407 12676 pending_rounds.cc:107] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 26.15
I20260709 12:14:20.047650 12676 raft_consensus.cc:2889] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:20.048794 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4578) took 3765 ms. Trace:
I20260709 12:14:20.049146 12676 rpcz_store.cc:276] 0709 12:14:16.283022 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:16.794223 (+511201us) service_pool.cc:224] Handling call
0709 12:14:16.794509 (+   286us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:19.879640 (+3085131us) spinlock_profiling.cc:243] Waited 1.67 s on lock 0x7b580053c3d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f148897c0aa 00007f14889fe447 00007f1488a16fe7 00007f1488a187de 00007f1488a22b49 00007f1488a23416 00007f1488a1fd8f 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4
0709 12:14:20.048372 (+168732us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:20.048708 (+   336us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3850378624}
W20260709 12:14:20.050442 22621 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 27 current term: 27 is higher.
I20260709 12:14:20.053941 22690 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.054711 22789 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.055079 22690 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:20.055585 22789 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:20.055859 22625 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:20.054270 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.069448 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:20.073065 12599 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:20.078068 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:20.079412 22625 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:20.080554 22690 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:20.081784 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3898) took 1376 ms. Trace:
I20260709 12:14:20.082300 12674 rpcz_store.cc:276] 0709 12:14:18.731714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:18.742345 (+ 10631us) service_pool.cc:224] Handling call
0709 12:14:18.765489 (+ 23144us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:18.765767 (+   278us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:14:18.835537 (+ 69770us) log.cc:844] Serialized 14 byte log entry
0709 12:14:20.068492 (+1232955us) spinlock_profiling.cc:243] Waited 3.75 ms on lock 0x7b1800242680. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf 00007f1488a3a038
0709 12:14:20.076056 (+  7564us) log.cc:844] Serialized 14 byte log entry
0709 12:14:20.081037 (+  4981us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:20.081128 (+    91us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:14:20.081241 (+   113us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:20.081343 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:20.081562 (+   219us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25475,"prepare.queue_time_us":561741,"raft.queue_time_us":102,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":3696,"spinlock_wait_cycles":52047744,"thread_start_us":547221,"threads_started":2,"wal-append.queue_time_us":566750}
I20260709 12:14:20.084903 22129 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:20.085400 22129 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.085767 22129 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:20.079417 12606 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 31: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:19.643978 22628 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:20.075289 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 137188us
I20260709 12:14:20.106069 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:14:19.956264 12420 rpcz_store.cc:269] 0709 12:12:43.849883 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:43.851673 (+  1790us) service_pool.cc:224] Handling call
0709 12:14:19.829806 (+95978133us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:09.535935 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:09.536843 (+   908us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:09.536939 (+    96us) write_op.cc:435] Acquired schema lock
0709 12:14:09.536970 (+    31us) tablet.cc:662] Decoding operations
0709 12:14:09.673863 (+136893us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:09.689624 (+ 15761us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:09.693171 (+  3547us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:14:09.772428 (+ 79257us) tablet.cc:701] Row locks acquired
0709 12:14:09.772481 (+    53us) write_op.cc:260] PREPARE: finished
0709 12:14:09.772688 (+   207us) write_op.cc:270] Start()
0709 12:14:09.772873 (+   185us) write_op.cc:276] Timestamp: P: 1783599249772609 usec, L: 0
0709 12:14:09.772912 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 12:14:09.819601 (+ 46689us) log.cc:844] Serialized 2870 byte log entry
0709 12:14:11.745255 (+1925654us) op_driver.cc:464] REPLICATION: finished
0709 12:14:11.927994 (+182739us) write_op.cc:301] APPLY: starting
0709 12:14:11.928115 (+   121us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:11.958470 (+ 30355us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:11.958598 (+   128us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:12.198517 (+239919us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:12.327194 (+128677us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=0,mrs_lookups=0
0709 12:14:12.339445 (+ 12251us) write_op.cc:312] APPLY: finished
0709 12:14:12.500376 (+160931us) log.cc:844] Serialized 73 byte log entry
0709 12:14:12.518065 (+ 17689us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:12.525978 (+  7913us) write_op.cc:454] Released schema lock
0709 12:14:12.547455 (+ 21477us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:12.647758 (+100303us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:12.652131 (+  4373us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":15473,"reported_tablets":52,"spinlock_wait_cycles":1280,"child_traces":[["op",{"apply.queue_time_us":173837,"apply.run_cpu_time_us":17649,"apply.run_wall_time_us":704850,"cfile_cache_hit":30,"cfile_cache_hit_bytes":33345,"num_ops":7,"prepare.queue_time_us":214040,"prepare.run_cpu_time_us":53290,"prepare.run_wall_time_us":1411812,"raft.queue_time_us":389523,"raft.run_cpu_time_us":20818,"raft.run_wall_time_us":245562,"replication_time_us":1879217,"thread_start_us":1514405,"threads_started":4,"wal-append.queue_time_us":1171532,"wal-append.run_cpu_time_us":26058,"wal-append.run_wall_time_us":2638333}]]}
I20260709 12:14:20.049927 22736 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.21, Last appended by leader: 21, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:20.127992 22789 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:20.148931 12675 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 31. Current term is 32. Ops: [31.15-31.15]
I20260709 12:14:20.150684 22656 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:14:20.151434 22656 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:20.150484 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3842) took 1487 ms. Trace:
I20260709 12:14:20.147871 12605 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:14:20.159654 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 21 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:20.168220 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 210492us
I20260709 12:14:20.170562 22706 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:20.177618 22714 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:20.170722 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:20.157001 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:20.153139 22656 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:20.197149 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:20.209791 12606 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:14:20.190943 22490 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:20.212436 12528 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:20.216267 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:20.217851 13338 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
W20260709 12:14:20.223639 12420 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58992 (request call id 177) took 19456 ms (client timeout 15000 ms). Trace:
W20260709 12:14:20.224164 12420 rpcz_store.cc:269] 0709 12:14:00.767595 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.223332 (+19455737us) service_pool.cc:211] Skipping call since client already timed out
0709 12:14:20.223527 (+   195us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:14:20.221663 22653 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.268512 22740 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
I20260709 12:14:20.268784 22653 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:20.271641 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:20.272847 13332 raft_consensus.cc:2393] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 30: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:14:20.233425 22801 log.cc:927] Time spent T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:14:20.234019 22682 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:20.241539 22714 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.274163 22682 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:20.275233 22714 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:20.243132 12526 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:14:20.280678 12604 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:14:20.277879 22714 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:20.214077 22656 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:20.286175 22682 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:20.244638 22129 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:20.289258 22129 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:20.262830 12671 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 4592) took 1450 ms (client timeout 1835 ms). Trace:
W20260709 12:14:20.294409 12671 rpcz_store.cc:269] 0709 12:14:18.812282 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:18.812492 (+   210us) service_pool.cc:224] Handling call
0709 12:14:20.262302 (+1449810us) spinlock_profiling.cc:243] Waited 6.22 ms on lock 0x7b540012d754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:20.262579 (+   277us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":14309760}
I20260709 12:14:20.295874 22736 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:20.189055 22485 consensus_queue.cc:1059] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.115s
I20260709 12:14:20.303128 12605 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:14:20.152741 12522 rpcz_store.cc:276] 0709 12:14:18.571515 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:19.540925 (+969410us) service_pool.cc:224] Handling call
0709 12:14:19.541263 (+   338us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:19.541539 (+   276us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:14:19.711642 (+170103us) log.cc:844] Serialized 14 byte log entry
0709 12:14:19.897214 (+185572us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:19.937035 (+ 39821us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:20.023119 (+ 86084us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:20.038342 (+ 15223us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:20.058886 (+ 20544us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18091,"prepare.queue_time_us":236639,"prepare.run_cpu_time_us":119,"prepare.run_wall_time_us":121,"raft.queue_time_us":75130,"raft.run_cpu_time_us":393,"raft.run_wall_time_us":65459,"thread_start_us":106200,"threads_started":1,"wal-append.queue_time_us":306794}
I20260709 12:14:20.213673 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:20.305621 22714 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.306515 22624 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:20.306823 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3901) took 1070 ms. Trace:
I20260709 12:14:20.306450 22714 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:20.307564 22503 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.308328 22503 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:20.309094 22714 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:20.311157 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:20.313225 22684 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:20.317134 22686 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:20.318320 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:20.318502 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:20.320353 12522 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:14:20.321746 12525 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:14:20.322639 22628 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:20.291023 13339 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:20.323930 12569 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:20.303031 22485 raft_consensus.cc:3055] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:14:20.326398 22485 raft_consensus.cc:740] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:20.327682 22485 consensus_queue.cc:260] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.22, Last appended by leader: 22, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:20.329444 22485 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:20.291749 22716 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:20.307238 12675 rpcz_store.cc:276] 0709 12:14:19.236000 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:19.884038 (+648038us) service_pool.cc:224] Handling call
0709 12:14:19.884878 (+   840us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:20.149762 (+264884us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:20.306507 (+156745us) spinlock_profiling.cc:243] Waited 257 ms on lock 0x7b54003bcc54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:20.306727 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":591122304}
I20260709 12:14:20.330618 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:20.331982 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:20.332610 12530 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:20.334362 13339 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:14:20.264696 22625 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:20.335222 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:20.326289 22129 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:14:20.370714 12527 raft_consensus.cc:2368] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 31. Current term is 32.
I20260709 12:14:20.370760 22129 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.371573 22129 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:20.371981 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:20.372951 12527 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 27: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:20.337572 13333 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:34880 (request call id 1) took 2122 ms. Trace:
I20260709 12:14:20.374320 13333 rpcz_store.cc:276] 0709 12:14:18.250179 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:18.251568 (+  1389us) service_pool.cc:224] Handling call
0709 12:14:20.337134 (+2085566us) spinlock_profiling.cc:243] Waited 892 ms on lock 0x7b5400469954. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1a853 00007f148d9e860a 00007f1486f6e7d4 00007f1486f6e762 00007f1486f6e691 00007f1486f6e60c 00007f1486f6d8c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:20.337324 (+   190us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2052048256}
I20260709 12:14:20.377075 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260709 12:14:20.377174 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:20.377923 12527 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:20.381295 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
W20260709 12:14:20.339181 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 119774us
I20260709 12:14:20.339725 22721 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:20.382886 12571 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:20.383664 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:20.384150 22624 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:20.384577 22624 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.384950 22624 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:20.387138 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:20.388464 22611 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:14:20.388928 12601 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:14:20.341606 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3900) took 1598 ms. Trace:
I20260709 12:14:20.342547 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:20.344385 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:20.345225 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.256873 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:20.334240 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:20.349494 22674 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 24 current term: 24 is higher.
I20260709 12:14:20.355087 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:20.355358 22789 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:14:20.355978 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:20.357285 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:14:20.405062 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.362453 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:20.363577 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:20.364549 12530 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:14:20.366858 22716 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:14:20.368371 22592 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:14:20.414988 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:20.415903 12530 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 32.
I20260709 12:14:20.416950 12675 raft_consensus.cc:2410] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 28 because replica has last-logged OpId of term: 26 index: 13, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 12.
I20260709 12:14:20.404460 12598 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 27: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:20.418113 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:20.340286 22653 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:20.410226 12531 raft_consensus.cc:2393] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 25: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:20.415076 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:20.416872 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:20.427815 12676 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:14:20.392771 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:20.407033 13337 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 31.
I20260709 12:14:20.424166 12599 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:14:20.425745 12528 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 32.
I20260709 12:14:20.429949 12526 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 31. Current term is 32. Ops: []
I20260709 12:14:20.440214 12672 raft_consensus.cc:2393] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 31: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:14:20.445175 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.421351 22503 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:20.411011 22714 consensus_queue.cc:1059] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.024s
W20260709 12:14:20.450816 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 63852.8us
I20260709 12:14:20.454774 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:20.457042 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:20.457592 12526 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 29 index: 20. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:14:20.394918 12673 rpcz_store.cc:276] 0709 12:14:18.743313 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:18.743435 (+   122us) service_pool.cc:224] Handling call
0709 12:14:18.743867 (+   432us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:18.788772 (+ 44905us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:14:18.951086 (+162314us) log.cc:844] Serialized 14 byte log entry
0709 12:14:19.112281 (+161195us) log.cc:844] Serialized 14 byte log entry
0709 12:14:19.121461 (+  9180us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:19.122379 (+   918us) log.cc:844] Serialized 52 byte log entry
0709 12:14:19.122539 (+   160us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:19.122728 (+   189us) log.cc:844] Serialized 14 byte log entry
0709 12:14:19.122977 (+   249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:19.123048 (+    71us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:20.018340 (+895292us) spinlock_profiling.cc:243] Waited 29.5 ms on lock 0x7b4800060858. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:14:20.134056 (+115716us) spinlock_profiling.cc:243] Waited 15.3 ms on lock 0x7b18004135c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485933522 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:14:20.242635 (+108579us) spinlock_profiling.cc:243] Waited 3.86 ms on lock 0x7b18004135c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:14:20.341143 (+ 98508us) raft_consensus.cc:1685] finished
0709 12:14:20.341174 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:20.341410 (+   236us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":715844,"prepare.queue_time_us":2478006,"prepare.run_cpu_time_us":129,"prepare.run_wall_time_us":132,"raft.queue_time_us":32283,"raft.run_cpu_time_us":4891,"raft.run_wall_time_us":982187,"spinlock_wait_cycles":112762368,"thread_start_us":338809,"threads_started":3,"wal-append.queue_time_us":404941}
I20260709 12:14:20.461585 12569 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:20.466300 12672 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 28 index: 14. Preceding OpId from leader: term: 30 index: 15. (index mismatch)
I20260709 12:14:20.467299 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:20.468119 12675 raft_consensus.cc:2410] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 32 because replica has last-logged OpId of term: 29 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 14.
I20260709 12:14:20.452142 22653 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.474054 12522 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 27. Current term is 28. Ops: []
W20260709 12:14:20.502266 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.477599 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:20.482658 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:20.515116 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.472913 22625 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:20.517666 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:20.523819 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:20.524335 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.518167 12527 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
I20260709 12:14:20.519871 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:14:20.503516 12643 connection.cc:256] RPC call timeout handler was delayed by -1.2051001830000132s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1565s for a response
I20260709 12:14:20.476950 22611 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election won for term 31
W20260709 12:14:20.546262 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 46048.9us
I20260709 12:14:20.546700 12531 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:14:20.548853 12571 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:20.531307 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:20.508047 22628 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:20.532111 12496 connection.cc:256] RPC call timeout handler was delayed by -1.0984269189998486s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.209s for a response
I20260709 12:14:20.532527 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:20.533016 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:20.533922 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.476949 22653 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:20.556262 22625 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:20.556595 22628 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:20.558911 22628 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:20.560062 12496 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:20.561062 22668 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.561712 22668 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:20.562422 22654 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:14:20.563944 22654 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:20.564199 22668 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:20.564519 13305 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:20.566169 22775 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:14:20.567922 22611 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:20.566407 22674 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:14:20.569419 22611 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 30.21, Last appended by leader: 21, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:20.561867 12496 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:20.574826 12646 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:14:20.576465 12643 connection.cc:256] RPC call timeout handler was delayed by -1.2051001830000132s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1565s for a response
I20260709 12:14:20.469318 22482 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:20.584566 12643 connection.cc:256] RPC call timeout handler was delayed by -1.0143001829999321s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1777s for a response
I20260709 12:14:20.558935 22625 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:20.580577 22684 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Leader pre-election won for term 33
W20260709 12:14:20.609618 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.609727 13337 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 30 index: 20. (index mismatch)
W20260709 12:14:20.607216 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 60133us
I20260709 12:14:20.612936 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4608) took 1372 ms. Trace:
I20260709 12:14:20.584410 22714 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:14:20.619977 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58197.5us
I20260709 12:14:20.624315 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:20.624569 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:20.615494 12677 rpcz_store.cc:276] 0709 12:14:19.232893 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:19.578351 (+345458us) service_pool.cc:224] Handling call
0709 12:14:19.595172 (+ 16821us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:19.595391 (+   219us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:14:20.195773 (+600382us) log.cc:844] Serialized 14 byte log entry
0709 12:14:20.579277 (+383504us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:20.583604 (+  4327us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:20.583661 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:20.588752 (+  5091us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:20.605149 (+ 16397us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":480819,"prepare.queue_time_us":483140,"prepare.run_cpu_time_us":108,"prepare.run_wall_time_us":109,"raft.queue_time_us":33919,"raft.run_cpu_time_us":92,"raft.run_wall_time_us":93,"thread_start_us":417842,"threads_started":2}
I20260709 12:14:20.633106 22733 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:20.634020 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:20.635462 22247 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:20.635951 22247 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.636309 22247 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:20.612296 22482 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:20.639292 22503 raft_consensus.cc:3055] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:14:20.639731 22503 raft_consensus.cc:740] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:20.640525 22482 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:20.619851 12646 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:20.637797 12496 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:20.610512 22684 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:20.649752 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 117387us
I20260709 12:14:20.633850 22733 raft_consensus.cc:697] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:14:20.666720 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 118029us
I20260709 12:14:20.667179 12572 leader_election.cc:400] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 election: Vote denied by peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811) with higher term. Message: Invalid argument: T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 31. Current term is 32.
I20260709 12:14:20.667896 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:20.668571 12572 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:20.621680 22625 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.683100 22482 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:14:20.683856 12597 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:14:20.684892 22482 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:20.685475 22482 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:20.686642 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:20.687448 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:14:20.717913 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:20.694010 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 47655.4us
I20260709 12:14:20.618566 22674 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 31.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.077s
W20260709 12:14:20.699796 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 62089.8us
I20260709 12:14:20.683156 22625 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:20.706939 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 101866us
I20260709 12:14:20.707662 22628 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:20.708159 22610 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.013s
I20260709 12:14:20.708909 12602 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 29 index: 15. Preceding OpId from leader: term: 30 index: 15. (term mismatch)
I20260709 12:14:20.714111 22654 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:20.715369 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47671.9us
I20260709 12:14:20.715978 22485 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:20.716174 22624 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:20.688648 12601 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 33 index: 17. (index mismatch)
I20260709 12:14:20.681670 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:20.723474 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:20.688899 22809 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.050s	user 0.001s	sys 0.002s
I20260709 12:14:20.647579 22684 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:20.620313 22775 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.100s
I20260709 12:14:20.719343 22714 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:20.729924 12602 pending_rounds.cc:85] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 14
I20260709 12:14:20.767133 12602 pending_rounds.cc:107] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 29.15
I20260709 12:14:20.767503 12602 raft_consensus.cc:2889] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:20.744766 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 76282.8us
I20260709 12:14:20.729377 22628 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.765184 22721 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:14:20.776625 12597 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:14:20.772315 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:20.785683 12604 raft_consensus.cc:2410] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 34 because replica has last-logged OpId of term: 32 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 20.
I20260709 12:14:20.778592 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:20.774235 12496 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:14:20.793079 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 50426.9us
I20260709 12:14:20.796900 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:20.799309 12598 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:20.640940 22503 consensus_queue.cc:260] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:20.800988 12606 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:14:20.780582 22628 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:20.822831 12605 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 33.
I20260709 12:14:20.825016 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:20.814137 12572 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:20.796985 22721 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:20.753666 22654 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:20.842397 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:20.814754 22668 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:20.818312 22653 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:20.861639 12602 raft_consensus.cc:1240] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27. Ops: [26.14-26.14]
I20260709 12:14:20.814327 22716 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.043s
I20260709 12:14:20.839985 22674 raft_consensus.cc:3055] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 31 LEADER]: Stepping down as leader of term 31
W20260709 12:14:20.876636 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 50690.9us
I20260709 12:14:20.888124 12605 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27. Ops: [26.18-26.18]
W20260709 12:14:20.896998 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 367559us
I20260709 12:14:20.897746 22701 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:20.898391 22681 raft_consensus.cc:2764] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election decision vote started in defunct term 28: won
W20260709 12:14:20.902558 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 148397us
I20260709 12:14:20.903765 12496 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:20.908221 22654 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:20.917179 22668 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:20.918036 22668 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:20.920075 22716 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:14:20.920975 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 239411us
I20260709 12:14:20.921124 22503 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:20.875387 22611 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader election won for term 32
W20260709 12:14:20.923913 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 159422us
W20260709 12:14:20.894515 22656 consensus_meta.cc:287] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.610s	user 0.012s	sys 0.002s
I20260709 12:14:20.896878 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:20.932928 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.861200 22714 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:20.914014 22805 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.128s	user 0.000s	sys 0.000s
W20260709 12:14:20.940128 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 53461.3us
W20260709 12:14:20.911999 22806 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.188s	user 0.000s	sys 0.001s
W20260709 12:14:20.950507 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.885371 22674 raft_consensus.cc:740] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:14:20.949352 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:20.964744 12680 raft_consensus.cc:2410] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 27 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 17.
W20260709 12:14:20.926152 22804 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.307s	user 0.002s	sys 0.000s
I20260709 12:14:20.938779 22668 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:14:20.957726 22686 consensus_meta.cc:287] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.640s	user 0.007s	sys 0.008s
I20260709 12:14:20.977213 22611 raft_consensus.cc:697] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:20.996961 12571 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:20.997938 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 92514.5us
I20260709 12:14:21.002023 12526 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 31. Current term is 32. Ops: []
I20260709 12:14:21.005877 12678 raft_consensus.cc:2410] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 30 because replica has last-logged OpId of term: 28 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 18.
I20260709 12:14:20.928577 22733 consensus_queue.cc:237] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.17, Last appended by leader: 17, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:20.966547 22714 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:20.987811 22482 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:21.006600 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 208479us
I20260709 12:14:21.017069 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:21.022327 22686 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:14:21.012672 22621 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.010062 22610 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:14:21.036033 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:21.025643 22624 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:21.028375 22625 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:21.057447 12572 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:21.056418 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:21.064476 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 79298.6us
W20260709 12:14:21.059638 22816 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.053s	user 0.001s	sys 0.000s
I20260709 12:14:21.075702 22715 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.063401 22482 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:21.032691 22621 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:21.011972 22721 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:21.109596 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:21.109759 12496 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:21.016651 22674 consensus_queue.cc:260] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:21.112794 12646 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:21.103379 22625 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:14:21.115377 22129 consensus_meta.cc:287] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.743s	user 0.013s	sys 0.002s
W20260709 12:14:21.129202 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 53876.3us
I20260709 12:14:21.132459 22485 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:21.134256 22823 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:21.134766 13304 leader_election.cc:336] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.624s (SENT)
I20260709 12:14:21.135836 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 34 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:21.138278 22821 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Leader pre-election won for term 24
I20260709 12:14:21.138842 22821 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:21.141278 22821 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:21.142860 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:21.143909 22654 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:14:21.144435 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:21.146395 22815 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:14:21.152136 22503 raft_consensus.cc:2764] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election decision vote started in defunct term 29: won
W20260709 12:14:21.152954 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 124935us
I20260709 12:14:21.154281 12494 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:21.155265 12494 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:21.116770 22482 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:14:21.157486 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 51261.1us
W20260709 12:14:21.160811 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:21.167836 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 109621us
I20260709 12:14:21.117368 22775 consensus_queue.cc:1059] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 31.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.024s
I20260709 12:14:21.176379 22733 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.177186 22733 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:14:21.177686 22690 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:21.178454 22690 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:21.179869 22733 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:14:21.171679 22656 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:21.177793 22736 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.181874 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3767) took 1132 ms. Trace:
I20260709 12:14:21.186568 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:21.170749 22832 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:14:21.208057 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:21.119223 22715 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:21.209156 12527 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
I20260709 12:14:21.180851 22690 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:21.134953 22823 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:21.133908 22822 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.124s
I20260709 12:14:21.113499 22611 consensus_queue.cc:237] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:21.213789 12646 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:21.133261 22129 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:21.183526 22656 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:21.183670 12529 rpcz_store.cc:276] 0709 12:14:20.049669 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.108708 (+ 59039us) service_pool.cc:224] Handling call
0709 12:14:20.109688 (+   980us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:20.132932 (+ 23244us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:14:20.159665 (+ 26733us) log.cc:844] Serialized 14 byte log entry
0709 12:14:20.736187 (+576522us) log.cc:844] Serialized 14 byte log entry
0709 12:14:20.758949 (+ 22762us) log.cc:844] Serialized 14 byte log entry
0709 12:14:20.759779 (+   830us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:20.810774 (+ 50995us) log.cc:844] Serialized 26 byte log entry
0709 12:14:20.810937 (+   163us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:20.810970 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:20.811044 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:21.181654 (+370610us) raft_consensus.cc:1685] finished
0709 12:14:21.181680 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:21.181812 (+   132us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":30413,"prepare.queue_time_us":48826,"prepare.run_cpu_time_us":153,"prepare.run_wall_time_us":3553,"raft.queue_time_us":5383,"raft.run_cpu_time_us":463,"raft.run_wall_time_us":10505,"spinlock_wait_cycles":29478144,"thread_start_us":575686,"threads_started":2,"wal-append.queue_time_us":487141}
I20260709 12:14:21.218355 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:14:21.230224 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 205306us
I20260709 12:14:21.208122 22733 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:21.242462 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 56033.9us
W20260709 12:14:21.243822 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 99465.9us
I20260709 12:14:21.228603 22656 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:21.250002 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:21.229686 22503 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:21.230399 22832 raft_consensus.cc:697] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:21.239233 22714 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:21.252974 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:21.266287 12526 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:21.216581 22736 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:21.262462 13302 leader_election.cc:336] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.668s (SENT)
I20260709 12:14:21.254245 22503 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:14:21.277894 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 121450us
I20260709 12:14:21.265758 22625 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:21.245296 22733 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:21.281785 12531 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
I20260709 12:14:21.297583 22653 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.011s
I20260709 12:14:21.308794 22653 consensus_queue.cc:1243] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 27.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:14:21.309346 22823 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:21.310072 22653 consensus_queue.cc:1059] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:14:21.310896 22715 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:21.303031 22826 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.169s	user 0.002s	sys 0.000s
W20260709 12:14:21.312072 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 188465us
I20260709 12:14:21.280766 22503 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:14:21.293617 22818 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.274s	user 0.000s	sys 0.003s
I20260709 12:14:21.282351 22625 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:21.292881 22129 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:21.298379 22733 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:21.336159 22247 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.699s	user 0.017s	sys 0.000s
I20260709 12:14:21.284538 22690 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:14:21.361562 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 89447.6us
W20260709 12:14:21.368037 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:21.273885 22482 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:21.352489 22625 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:14:21.375612 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 77933.8us
W20260709 12:14:21.379395 12678 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3914) took 2098 ms (client timeout 1880 ms). Trace:
I20260709 12:14:21.383742 12674 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 19. (term mismatch)
W20260709 12:14:21.384114 12598 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.575s	user 0.004s	sys 0.008s
I20260709 12:14:21.384688 12598 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:21.362942 22654 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:21.387430 22686 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.001s
W20260709 12:14:21.374931 22684 consensus_meta.cc:287] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.595s	user 0.011s	sys 0.004s
W20260709 12:14:21.382917 12678 rpcz_store.cc:269] 0709 12:14:19.280837 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:19.325992 (+ 45155us) service_pool.cc:224] Handling call
0709 12:14:21.367490 (+2041498us) spinlock_profiling.cc:243] Waited 1.66 s on lock 0x7b54003bc754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:21.379310 (+ 11820us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3814049536}
I20260709 12:14:21.384163 12674 pending_rounds.cc:85] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 18
I20260709 12:14:21.396296 12674 pending_rounds.cc:107] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 28.19
I20260709 12:14:21.375274 22129 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:21.396699 12674 raft_consensus.cc:2889] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:21.397044 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:21.367431 22821 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:21.403949 22840 raft_consensus.cc:3055] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:14:21.404203 13339 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 31 index: 22. (index mismatch)
I20260709 12:14:21.401292 22832 consensus_queue.cc:237] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:21.383472 22485 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.371124 22247 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:21.398445 22129 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:21.411664 22840 raft_consensus.cc:740] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:21.422953 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:21.398886 22684 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:21.432515 22838 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:14:21.425544 22485 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:21.433199 13304 leader_election.cc:336] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.904s (SENT)
I20260709 12:14:21.434314 12494 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:21.436058 22837 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:14:21.440374 13332 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:14:21.461360 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 69138.6us
W20260709 12:14:21.471426 22842 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 12:14:21.475813 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:21.439471 22129 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:14:21.482714 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 229840us
I20260709 12:14:21.489955 12494 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:21.493080 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 323338us
I20260709 12:14:21.475943 22838 consensus_queue.cc:1243] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 27.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
W20260709 12:14:21.502923 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 68828.7us
I20260709 12:14:21.502705 22823 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:21.497398 22654 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:21.525620 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:21.532236 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 112772us
I20260709 12:14:21.522172 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:21.512033 22701 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.015s
I20260709 12:14:21.554229 12526 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24. Ops: []
I20260709 12:14:21.555016 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:21.556754 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:21.554468 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 77463.2us
I20260709 12:14:21.545383 22822 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:21.544824 22823 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:21.565402 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 60691.6us
I20260709 12:14:21.547821 22837 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:21.565739 13332 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:14:21.527010 22690 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.028s
I20260709 12:14:21.568506 12527 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:14:21.569135 22854 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.550899 22736 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:21.576947 13302 leader_election.cc:336] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.624s (SENT)
I20260709 12:14:21.579895 13302 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 24 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:21.576063 22837 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:21.582659 22837 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:21.582978 22588 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader election lost for term 24. Reason: could not achieve majority
I20260709 12:14:21.582770 22846 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.081s
I20260709 12:14:21.584977 22847 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.039s
I20260709 12:14:21.586436 22821 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:21.547900 22840 consensus_queue.cc:260] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:21.586500 22852 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:14:21.587430 22821 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:14:21.587666 22840 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:21.588207 22485 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:21.589288 22247 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:21.589658 22684 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:21.592664 22656 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:21.595099 22823 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:21.596522 22736 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:14:21.601223 22129 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:21.581955 22851 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.036s
I20260709 12:14:21.602615 22851 consensus_queue.cc:1243] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer a2fdbe7eef6748a9b4611f6c9bc9b201 log is divergent from this leader: its last log entry 28.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:14:21.603227 22851 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:14:21.603554 22851 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:21.570528 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:21.587643 22852 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:21.615084 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 56842.9us
I20260709 12:14:21.574906 22854 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:21.620494 22485 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 12:14:21.622305 22129 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:21.623384 22852 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:21.623891 22854 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:21.625078 22851 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:21.625991 22628 consensus_meta.cc:287] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.799s	user 0.014s	sys 0.002s
I20260709 12:14:21.626667 22628 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:21.624136 22844 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.105s	user 0.000s	sys 0.001s
W20260709 12:14:21.633952 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 108464us
I20260709 12:14:21.652791 12494 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:21.654534 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:21.661881 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 357952us
I20260709 12:14:21.669252 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:21.669853 12527 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:21.652351 22714 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.012s
I20260709 12:14:21.675729 12643 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:21.652177 22838 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.014s
W20260709 12:14:21.670855 22668 consensus_meta.cc:287] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.687s	user 0.010s	sys 0.005s
I20260709 12:14:21.677688 22850 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:21.683059 22656 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:21.691022 22628 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:21.676465 22714 consensus_queue.cc:1243] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 22.15 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 14
I20260709 12:14:21.663333 22822 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.020s
I20260709 12:14:21.661381 22701 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.022s
I20260709 12:14:21.687531 22850 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:21.677781 22853 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.051s
I20260709 12:14:21.694857 22854 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:14:21.681325 22625 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:21.691977 22668 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:21.756350 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 85041.6us
I20260709 12:14:21.754001 12527 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
W20260709 12:14:21.763406 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:21.764057 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:21.767931 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 78999.7us
I20260709 12:14:21.768143 12572 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:21.759498 22823 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.011s
I20260709 12:14:21.747860 22846 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 27.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.031s
I20260709 12:14:21.794632 12525 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: []
I20260709 12:14:21.765798 22854 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:14:21.795413 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 107021us
I20260709 12:14:21.771116 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:21.755026 22850 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:21.799458 12494 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:21.799430 12527 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 27: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:21.793758 22624 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.826787 12643 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:21.821401 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52363.3us
I20260709 12:14:21.802256 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:21.830381 13304 leader_election.cc:336] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.154s (SENT)
I20260709 12:14:21.796180 22690 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election won for term 30
W20260709 12:14:21.823151 22858 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:14:21.769429 22855 consensus_queue.cc:1059] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.183s
I20260709 12:14:21.842315 12496 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:21.834939 22624 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:21.837642 22862 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:14:21.837765 22856 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.855602 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:21.858799 12494 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:21.838238 22863 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:21.863029 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:21.864310 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:21.843235 22611 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:21.868108 22864 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.068s	user 0.001s	sys 0.000s
I20260709 12:14:21.883791 22865 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:21.883566 22611 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:21.883566 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:21.918246 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 117699us
I20260709 12:14:21.885902 12496 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:21.870767 22863 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:21.837625 22854 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 30.15, Last appended by leader: 15, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:14:21.866451 22846 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:21.934414 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:21.936774 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:21.922343 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 63646.4us
I20260709 12:14:21.883670 22866 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:21.866134 22838 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:14:21.857686 22821 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.064s
I20260709 12:14:21.942454 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3865) took 1499 ms. Trace:
W20260709 12:14:21.947410 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 147573us
I20260709 12:14:21.949481 12494 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:21.950392 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:21.934824 22674 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:21.952096 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:21.944060 22503 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.600s	user 0.011s	sys 0.003s
W20260709 12:14:21.960246 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 85876.2us
I20260709 12:14:21.942294 22838 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:21.961695 22838 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:21.962496 22863 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:21.976033 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:21.942278 22866 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:21.972679 22625 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:21.953002 22848 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:14:21.961609 22869 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:21.950184 22852 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:21.956079 22668 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:21.884984 22856 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:21.983906 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3770) took 1845 ms. Trace:
W20260709 12:14:21.979123 22867 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.094s	user 0.001s	sys 0.001s
I20260709 12:14:21.955876 22714 consensus_queue.cc:1059] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 23.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:14:22.002713 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:22.004957 12496 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:22.010756 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 213439us
I20260709 12:14:21.977538 22866 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:21.984457 22848 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:21.986034 12524 rpcz_store.cc:276] 0709 12:14:20.138620 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.138830 (+   210us) service_pool.cc:224] Handling call
0709 12:14:20.148529 (+  9699us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:20.149353 (+   824us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:14:20.150308 (+   955us) log.cc:844] Serialized 14 byte log entry
0709 12:14:20.419835 (+269527us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:20.443677 (+ 23842us) log.cc:844] Serialized 26 byte log entry
0709 12:14:20.443853 (+   176us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:20.443890 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:20.443980 (+    90us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:20.925044 (+481064us) spinlock_profiling.cc:243] Waited 28.3 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:14:21.934025 (+1008981us) spinlock_profiling.cc:243] Waited 3.42 ms on lock 0x7b240011b418. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14857b99c4 00007f14857b7acb 00007f1485972dc9 00007f1485972a55 00007f1485971fb0 00007f14885e8488 00007f14885e8be8 00007f14885c0398 00007f14885bb76a 00007f14885babb6
0709 12:14:21.983662 (+ 49637us) raft_consensus.cc:1685] finished
0709 12:14:21.983684 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:21.983837 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7684,"prepare.queue_time_us":1368,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":58,"raft.queue_time_us":34751,"raft.run_cpu_time_us":278,"raft.run_wall_time_us":321,"spinlock_wait_cycles":72863488,"thread_start_us":243498,"threads_started":1,"wal-append.queue_time_us":379554}
I20260709 12:14:21.960069 22503 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:21.984011 22654 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:21.991847 22690 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:22.025547 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:22.010762 22668 consensus_queue.cc:1059] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 25.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.022s
I20260709 12:14:22.039706 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:22.046892 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:22.047618 12598 raft_consensus.cc:2368] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 24. Current term is 25.
W20260709 12:14:22.040354 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.047989 12524 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 30: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:22.059803 12494 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:14:22.062460 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.061738 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:22.066906 22868 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.177s	user 0.001s	sys 0.000s
I20260709 12:14:22.067148 22503 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:22.067408 22714 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:22.067935 22714 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:22.068272 12527 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:14:22.067193 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:22.070446 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 96268.4us
I20260709 12:14:22.059752 22611 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:22.088308 12524 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:14:22.093149 12598 raft_consensus.cc:1240] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 31. Current term is 32. Ops: []
I20260709 12:14:22.098733 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
W20260709 12:14:22.091313 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 86661.2us
I20260709 12:14:22.080760 22503 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:22.098860 12524 pending_rounds.cc:85] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting all ops after (but not including) 16
I20260709 12:14:22.104746 12524 pending_rounds.cc:107] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting uncommitted NO_OP operation due to leader change: 27.17
I20260709 12:14:22.105041 12524 raft_consensus.cc:2889] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:22.085638 22822 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:22.104166 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:22.108172 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:22.112985 12603 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 27.
I20260709 12:14:22.068274 22837 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:14:22.122583 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:22.123914 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:22.124678 22880 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.010s
W20260709 12:14:22.125780 22870 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.076s	user 0.000s	sys 0.001s
I20260709 12:14:22.126780 12527 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:22.135742 22851 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:22.136479 13339 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:14:22.089042 22848 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:22.139601 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4617) took 1781 ms. Trace:
I20260709 12:14:22.140062 22848 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:22.140028 12674 rpcz_store.cc:276] 0709 12:14:20.358035 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.358220 (+   185us) service_pool.cc:224] Handling call
0709 12:14:21.372698 (+1014478us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:21.404957 (+ 32259us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.139394 (+734437us) spinlock_profiling.cc:243] Waited 1.01 s on lock 0x7b54003bc750. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:22.139536 (+   142us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2332303616}
I20260709 12:14:22.068602 22690 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:22.142697 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:22.143571 22823 raft_consensus.cc:3055] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:14:22.144215 22823 raft_consensus.cc:740] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:22.144867 22860 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:21.945822 12528 rpcz_store.cc:276] 0709 12:14:20.442863 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.555906 (+113043us) service_pool.cc:224] Handling call
0709 12:14:20.578723 (+ 22817us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:20.618568 (+ 39845us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:20.618634 (+    66us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:20.641405 (+ 22771us) log.cc:844] Serialized 52 byte log entry
0709 12:14:20.876117 (+234712us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:14:21.293107 (+416990us) spinlock_profiling.cc:243] Waited 28.2 ms on lock 0x7b240011b2f8. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14857b99c4 00007f14857b7acb 00007f1485972dc9 00007f1485972a55 00007f148597209d 00007f14885a410e 00007f14885bb88c 00007f14885babb6 00007f14885ba7d8 00007f14885c4221
0709 12:14:21.918190 (+625083us) log.cc:844] Serialized 14 byte log entry
0709 12:14:21.934334 (+ 16144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:21.934423 (+    89us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:21.934465 (+    42us) raft_consensus.cc:1685] finished
0709 12:14:21.942198 (+  7733us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:21.942383 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":986219,"raft.queue_time_us":25872,"raft.run_cpu_time_us":305,"raft.run_wall_time_us":21645,"spinlock_wait_cycles":64771840,"thread_start_us":233984,"threads_started":1,"wal-append.queue_time_us":365213}
I20260709 12:14:22.143579 12603 raft_consensus.cc:2410] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 33 because replica has last-logged OpId of term: 32 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 20.
I20260709 12:14:22.149163 12528 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 29. Current term is 30. Ops: [29.17-29.17]
W20260709 12:14:22.151005 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.145632 22860 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:22.151839 22821 raft_consensus.cc:3055] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:14:22.152359 22821 raft_consensus.cc:740] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:22.145346 22823 consensus_queue.cc:260] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:22.154268 22860 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:22.155269 22823 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:22.090204 22714 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:22.158702 22853 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:14:22.160741 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3784) took 1055 ms. Trace:
I20260709 12:14:22.161177 12525 rpcz_store.cc:276] 0709 12:14:21.105176 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:21.105520 (+   344us) service_pool.cc:224] Handling call
0709 12:14:21.115526 (+ 10006us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:21.799668 (+684142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.160485 (+360817us) spinlock_profiling.cc:243] Waited 677 ms on lock 0x7b54003c0d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:22.160674 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1557315456}
I20260709 12:14:22.163486 13305 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:22.163929 22851 consensus_queue.cc:1059] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:14:22.164420 12643 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:22.165153 22853 raft_consensus.cc:3055] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:14:22.165340 22656 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:14:22.165644 22853 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:22.165956 22656 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:22.166457 22656 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:22.166792 22853 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:22.167438 12529 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
I20260709 12:14:22.168210 22853 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:14:22.176913 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 70835.9us
W20260709 12:14:22.176702 22871 log.cc:927] Time spent T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.180s	user 0.001s	sys 0.000s
I20260709 12:14:22.200843 12525 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:14:22.206066 12527 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:22.194594 22482 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:22.210467 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 44652.5us
I20260709 12:14:22.211508 12643 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:22.214776 22856 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:22.215493 22856 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:22.215893 22856 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:22.216244 22856 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:22.110814 22701 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.083s
I20260709 12:14:22.210515 22482 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:22.220460 22701 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:14:22.210645 12603 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
W20260709 12:14:22.222369 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 98578.1us
I20260709 12:14:22.236303 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:22.213784 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:22.193727 22674 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:22.153460 22821 consensus_queue.cc:260] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:22.251933 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.271508 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1279) took 1314 ms. Trace:
I20260709 12:14:22.275426 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:22.207815 22668 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:14:22.285032 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:22.275012 12522 rpcz_store.cc:276] 0709 12:14:20.956656 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.956816 (+   160us) service_pool.cc:224] Handling call
0709 12:14:20.970293 (+ 13477us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:20.979530 (+  9237us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:20.979577 (+    47us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:20.985589 (+  6012us) log.cc:844] Serialized 52 byte log entry
0709 12:14:21.056904 (+ 71315us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:14:21.302360 (+245456us) spinlock_profiling.cc:243] Waited 6.04 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:14:22.011645 (+709285us) log.cc:844] Serialized 14 byte log entry
0709 12:14:22.246996 (+235351us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.247083 (+    87us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:22.247128 (+    45us) raft_consensus.cc:1685] finished
0709 12:14:22.271149 (+ 24021us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:22.271414 (+   265us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":916627,"prepare.queue_time_us":999586,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":39,"raft.queue_time_us":38663,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":8166,"spinlock_wait_cycles":13901184,"thread_start_us":310446,"threads_started":3,"wal-append.queue_time_us":117728}
W20260709 12:14:22.295822 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 117936us
I20260709 12:14:22.296902 12603 raft_consensus.cc:2410] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 32 because replica has last-logged OpId of term: 30 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 17.
I20260709 12:14:22.307281 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:22.307649 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:22.308079 12598 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:14:22.312160 12597 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 32 index: 21. Preceding OpId from leader: term: 33 index: 21. (term mismatch)
I20260709 12:14:22.312625 12597 pending_rounds.cc:85] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 20
I20260709 12:14:22.312866 12597 pending_rounds.cc:107] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 32.21
I20260709 12:14:22.313097 12597 raft_consensus.cc:2889] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:22.249439 22674 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:22.273044 22880 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:14:22.356058 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:22.372723 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.396945 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:22.402041 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 29 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:22.403244 12674 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 33 because replica has last-logged OpId of term: 30 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 20.
I20260709 12:14:22.404723 12671 raft_consensus.cc:1240] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:22.404625 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:22.406046 12675 raft_consensus.cc:2410] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 30 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 15.
I20260709 12:14:22.406414 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:22.407209 12597 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 33: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:22.408191 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:22.408908 12597 raft_consensus.cc:2368] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 24. Current term is 25.
I20260709 12:14:22.404536 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:22.415124 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:22.316828 22865 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:22.409920 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:22.434453 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:22.361725 22674 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:14:22.419149 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 218712us
W20260709 12:14:22.445113 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 231423us
I20260709 12:14:22.449637 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:22.453286 12606 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:22.455267 12494 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:22.424508 22821 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:22.473059 13337 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 32. Current term is 33. Ops: []
I20260709 12:14:22.477885 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:22.465643 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:22.481549 12671 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:22.476043 22823 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:22.482720 22823 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:22.483122 22823 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:14:22.483904 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.393625 22890 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.148s
I20260709 12:14:22.494159 12643 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:14:22.496109 22852 consensus_meta.cc:287] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.509s	user 0.004s	sys 0.014s
W20260709 12:14:22.516502 22889 log.cc:927] Time spent T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.070s	user 0.000s	sys 0.000s
I20260709 12:14:22.520458 22656 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:22.522497 22482 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:22.522367 12680 pending_rounds.cc:85] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 14
I20260709 12:14:22.527092 12680 pending_rounds.cc:107] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 22.15
I20260709 12:14:22.527453 12680 raft_consensus.cc:2889] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:22.526734 12674 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 25. Ops: []
W20260709 12:14:22.531620 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.532763 12671 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 24.
I20260709 12:14:22.504277 22834 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:22.505517 22852 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:14:22.543736 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.526401 22851 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:22.551599 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:22.552392 12671 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:22.553874 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3785) took 1438 ms. Trace:
I20260709 12:14:22.490942 22841 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.047s
W20260709 12:14:22.578085 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 108189us
W20260709 12:14:22.586724 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.553810 22621 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:22.548013 22837 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:14:22.593165 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:22.572423 22861 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:22.602406 22896 consensus_queue.cc:1059] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.005s
I20260709 12:14:22.603618 22896 raft_consensus.cc:3055] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:14:22.604031 22896 raft_consensus.cc:740] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:22.605037 22896 consensus_queue.cc:260] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:22.615154 22624 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:22.616410 12672 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:14:22.616506 22656 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:22.591643 22880 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:14:22.645403 22674 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:22.573030 22834 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:22.646931 22899 log.cc:927] Time spent T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:14:22.648499 22880 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:22.654803 22851 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.029s
W20260709 12:14:22.759589 22838 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.772s	user 0.022s	sys 0.004s
I20260709 12:14:22.760656 22838 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:22.763175 12530 raft_consensus.cc:1240] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 33. Current term is 34. Ops: []
I20260709 12:14:22.765923 12679 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 30 index: 16. Preceding OpId from leader: term: 32 index: 17. (index mismatch)
W20260709 12:14:22.776311 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.784142 12606 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:14:22.782547 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:22.790448 13337 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
I20260709 12:14:22.794310 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:22.796190 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:22.807930 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.818995 12671 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
W20260709 12:14:22.824764 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:22.825313 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 46489.2us
I20260709 12:14:22.826778 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:22.835103 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:22.843926 13304 leader_election.cc:336] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.818s (SENT)
W20260709 12:14:22.805549 22866 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.753s	user 0.010s	sys 0.002s
I20260709 12:14:22.860404 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:22.861874 12606 raft_consensus.cc:2410] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 30 because replica has last-logged OpId of term: 29 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 21.
I20260709 12:14:22.864771 12679 pending_rounds.cc:85] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 17
I20260709 12:14:22.865176 12679 pending_rounds.cc:107] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 31.18
I20260709 12:14:22.865489 12679 raft_consensus.cc:2889] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:22.866902 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3799) took 1168 ms. Trace:
I20260709 12:14:22.872709 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:22.875114 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 48512.6us
I20260709 12:14:22.875751 22656 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:14:22.890388 22656 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:22.890921 22656 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:14:22.892769 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:22.894448 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50597.5us
I20260709 12:14:22.894291 22903 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.001s
W20260709 12:14:22.863407 22827 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.726s	user 0.012s	sys 0.004s
I20260709 12:14:22.899060 12525 raft_consensus.cc:2393] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 24: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:22.868196 12528 rpcz_store.cc:276] 0709 12:14:21.698779 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.166150 (+467371us) service_pool.cc:224] Handling call
0709 12:14:22.166507 (+   357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:22.166729 (+   222us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:14:22.555240 (+388511us) log.cc:844] Serialized 14 byte log entry
0709 12:14:22.863220 (+307980us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:22.863305 (+    85us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:22.863369 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.866691 (+  3322us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:22.866830 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":388228,"prepare.queue_time_us":388788,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":42,"raft.queue_time_us":231,"raft.run_cpu_time_us":105,"raft.run_wall_time_us":8569,"thread_start_us":307552,"threads_started":1}
I20260709 12:14:22.869520 22866 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:22.888353 22701 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:14:22.894906 12643 leader_election.cc:400] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 29. Current term is 30.
I20260709 12:14:22.883396 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:22.904524 12643 leader_election.cc:403] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Cancelling election due to peer responding with higher term
W20260709 12:14:22.905180 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.919663 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:22.879271 22896 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:14:22.923305 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:22.929929 12522 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 27. Current term is 28. Ops: [27.16-27.16]
I20260709 12:14:22.871202 22851 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.052s
W20260709 12:14:22.922346 22850 consensus_meta.cc:287] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.100s	user 0.013s	sys 0.004s
I20260709 12:14:22.921118 22865 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
W20260709 12:14:22.934883 13304 leader_election.cc:336] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 1.905s (SENT)
I20260709 12:14:22.925012 22903 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.020s
W20260709 12:14:22.979008 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48738.4us
W20260709 12:14:22.963937 22904 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.140s	user 0.002s	sys 0.000s
I20260709 12:14:22.981775 12601 raft_consensus.cc:1217] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Deduplicated request from leader. Original: 24.14->[26.15-32.18]   Dedup: 26.15->[28.16-32.18]
I20260709 12:14:22.944296 22850 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:23.003688 12601 pending_rounds.cc:85] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 15
I20260709 12:14:23.018831 22838 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:23.002796 22856 consensus_meta.cc:287] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.786s	user 0.011s	sys 0.005s
I20260709 12:14:23.073942 22834 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:23.180383 12601 pending_rounds.cc:107] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 27.16
I20260709 12:14:23.074497 22866 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:23.075248 22674 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:22.925000 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:23.080060 22701 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election lost for term 29. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 29. Current term is 30.
I20260709 12:14:23.081142 22686 raft_consensus.cc:3055] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Stepping down as leader of term 29
W20260709 12:14:23.111887 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 232898us
I20260709 12:14:22.961937 22865 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:23.180732 12601 raft_consensus.cc:2889] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:23.082006 22847 raft_consensus.cc:3055] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Stepping down as leader of term 23
I20260709 12:14:23.214962 12526 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:23.190289 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:23.136771 22129 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 LEADER]: Stepping down as leader of term 31
W20260709 12:14:23.181429 22848 consensus_meta.cc:287] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.041s	user 0.007s	sys 0.022s
I20260709 12:14:23.211855 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:23.184115 22909 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.203s	user 0.010s	sys 0.000s
W20260709 12:14:23.183946 22906 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.181s	user 0.004s	sys 0.003s
W20260709 12:14:23.231462 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 336669us
I20260709 12:14:23.188393 22851 raft_consensus.cc:3055] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Stepping down as leader of term 26
W20260709 12:14:23.206391 22891 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.689s	user 0.003s	sys 0.025s
I20260709 12:14:23.233963 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:23.196470 22686 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:23.243204 22686 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:23.244598 22686 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:23.169793 22822 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:14:23.246553 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:23.252192 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:23.254830 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.215440 22847 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:23.218372 22129 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:23.188222 22856 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:23.251178 22822 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:23.236946 22851 raft_consensus.cc:740] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:23.282724 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:23.294762 12531 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:23.285580 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:23.286623 12646 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:22.554729 12523 rpcz_store.cc:276] 0709 12:14:21.115751 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:21.115913 (+   162us) service_pool.cc:224] Handling call
0709 12:14:21.127192 (+ 11279us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:21.127422 (+   230us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:14:22.019252 (+891830us) log.cc:844] Serialized 14 byte log entry
0709 12:14:22.144864 (+125612us) spinlock_profiling.cc:243] Waited 1.95 ms on lock 0x7b180044c4a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:14:22.553017 (+408153us) spinlock_profiling.cc:243] Waited 1.92 ms on lock 0x7b180044c4a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf 00007f1488a3a038
0709 12:14:22.553333 (+   316us) log.cc:844] Serialized 14 byte log entry
0709 12:14:22.553519 (+   186us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:22.553565 (+    46us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:22.553620 (+    55us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.553694 (+    74us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:22.553818 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":1373209,"prepare.queue_time_us":1424688,"prepare.run_cpu_time_us":81,"prepare.run_wall_time_us":85,"raft.queue_time_us":29542,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":109,"spinlock_wait_cycles":368768896,"thread_start_us":96841,"threads_started":2,"wal-append.queue_time_us":132440}
I20260709 12:14:23.251564 22865 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:23.315173 22674 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:14:23.345860 22897 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.830s	user 0.004s	sys 0.011s
I20260709 12:14:23.319173 22850 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:23.319799 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:23.327132 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:23.328111 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.335197 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:23.393999 12527 raft_consensus.cc:2410] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
I20260709 12:14:23.343986 22896 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:14:23.317550 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:23.401443 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:23.350071 22919 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.099s	user 0.001s	sys 0.000s
W20260709 12:14:23.351437 22894 log.cc:927] Time spent T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.825s	user 0.005s	sys 0.006s
I20260709 12:14:23.358491 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:23.403417 12604 raft_consensus.cc:2410] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 30 because replica has last-logged OpId of term: 29 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 21.
W20260709 12:14:23.360137 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.365692 12678 raft_consensus.cc:1240] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27. Ops: [26.18-26.18]
W20260709 12:14:23.317492 22503 consensus_meta.cc:287] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.208s	user 0.012s	sys 0.008s
W20260709 12:14:23.397341 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 110805us
I20260709 12:14:23.214691 22880 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.21, Last appended by leader: 21, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:23.410701 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:14:23.354072 22823 consensus_meta.cc:287] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.870s	user 0.008s	sys 0.004s
I20260709 12:14:23.449754 22823 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:23.452723 12525 raft_consensus.cc:2393] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 31: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:23.453114 22823 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:23.293390 22853 consensus_meta.cc:287] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.125s	user 0.012s	sys 0.005s
W20260709 12:14:23.454608 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 169123us
I20260709 12:14:23.454926 22853 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:14:23.351518 22910 log.cc:927] Time spent T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.105s	user 0.000s	sys 0.000s
I20260709 12:14:23.455618 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:23.457396 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:23.458428 22921 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:23.459276 22921 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:14:23.460493 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1286) took 1449 ms. Trace:
I20260709 12:14:23.462455 22503 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:23.462860 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.463232 12528 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
W20260709 12:14:23.463577 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.464250 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:23.464529 22129 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 31.15, Last appended by leader: 15, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:23.465713 22129 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:23.441571 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:23.442476 22927 raft_consensus.cc:2804] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:23.447376 22847 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 23.16, Last appended by leader: 16, Current term: 23, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:23.493515 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.429324 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:14:23.398970 22914 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.219s	user 0.002s	sys 0.000s
I20260709 12:14:23.466804 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.483546 22865 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:23.460914 22921 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:23.484157 22856 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:23.484380 22853 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.485019 22880 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:23.487474 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1153) took 1430 ms. Trace:
I20260709 12:14:23.467519 22866 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:23.488888 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.519793 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:23.476308 22701 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.461093 12529 rpcz_store.cc:276] 0709 12:14:22.014054 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.238740 (+224686us) service_pool.cc:224] Handling call
0709 12:14:22.277090 (+ 38350us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:22.430988 (+153898us) raft_consensus.cc:1534] Early marking committed up to index 21
0709 12:14:22.556850 (+125862us) log.cc:844] Serialized 14 byte log entry
0709 12:14:22.940889 (+384039us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:22.940964 (+    75us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:14:22.941023 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.460152 (+519129us) spinlock_profiling.cc:243] Waited 139 ms on lock 0x7b54003c1254. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:23.460214 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:23.460412 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125458,"prepare.queue_time_us":153559,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":9732,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":110,"spinlock_wait_cycles":320155520,"thread_start_us":377462,"threads_started":1,"wal-append.queue_time_us":419972}
I20260709 12:14:23.465119 12674 raft_consensus.cc:2410] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 28 because replica has last-logged OpId of term: 27 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 18.
I20260709 12:14:23.507800 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:23.508059 12531 raft_consensus.cc:2468] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26.
I20260709 12:14:23.514060 12522 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3880) took 1449 ms. Trace:
I20260709 12:14:23.525533 12527 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:23.525610 12522 rpcz_store.cc:276] 0709 12:14:22.064251 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.288124 (+223873us) service_pool.cc:224] Handling call
0709 12:14:22.319972 (+ 31848us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:22.931808 (+611836us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.513555 (+581747us) spinlock_profiling.cc:243] Waited 608 ms on lock 0x7b540042a9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:23.513853 (+   298us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1397967232}
I20260709 12:14:23.528023 12522 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 31. Current term is 32. Ops: []
I20260709 12:14:23.520690 12604 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 27.
I20260709 12:14:23.533176 12523 raft_consensus.cc:1275] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
W20260709 12:14:23.517014 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 87823.2us
I20260709 12:14:23.535039 12496 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:23.529942 12522 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24. Ops: [23.16-23.16]
I20260709 12:14:23.500583 22861 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.593222 22721 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.412818 22674 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:23.593683 22861 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:23.513072 12602 rpcz_store.cc:276] 0709 12:14:22.056902 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.057125 (+   223us) service_pool.cc:224] Handling call
0709 12:14:22.081319 (+ 24194us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:22.115024 (+ 33705us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:22.115073 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:22.154841 (+ 39768us) log.cc:844] Serialized 52 byte log entry
0709 12:14:22.248406 (+ 93565us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:14:22.476603 (+228197us) log.cc:844] Serialized 14 byte log entry
0709 12:14:22.502767 (+ 26164us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.502848 (+    81us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:23.350308 (+847460us) spinlock_profiling.cc:243] Waited 1.43 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:14:23.487112 (+136804us) raft_consensus.cc:1685] finished
0709 12:14:23.487220 (+   108us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:23.487381 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":28900,"prepare.queue_time_us":72750,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":36240,"raft.run_cpu_time_us":532,"raft.run_wall_time_us":42662,"spinlock_wait_cycles":3283712,"thread_start_us":250179,"threads_started":2,"wal-append.queue_time_us":307780}
I20260709 12:14:23.591935 12525 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27. Ops: []
I20260709 12:14:23.596267 22674 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:23.608608 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:23.552986 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.436808 22896 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:23.558069 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:23.558295 12523 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:14:23.553067 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:23.615078 12676 raft_consensus.cc:2410] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:14:23.560559 13302 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:23.561697 22857 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:23.562747 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3882) took 1366 ms. Trace:
I20260709 12:14:23.564807 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1282) took 1707 ms. Trace:
I20260709 12:14:23.623199 12530 rpcz_store.cc:276] 0709 12:14:21.857072 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.157150 (+300078us) service_pool.cc:224] Handling call
0709 12:14:22.157442 (+   292us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:22.763690 (+606248us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.564568 (+800878us) spinlock_profiling.cc:243] Waited 605 ms on lock 0x7b54003cb9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:23.564744 (+   176us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1392518528}
I20260709 12:14:23.566541 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 24 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:23.567976 12597 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24. Ops: []
I20260709 12:14:23.624807 12677 raft_consensus.cc:2410] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 24 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
W20260709 12:14:23.576215 22884 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.424s	user 0.000s	sys 0.017s
I20260709 12:14:23.546221 22847 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 23 FOLLOWER]: Advancing to term 24
I20260709 12:14:23.586270 22866 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:23.587630 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 34 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.644079 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 22 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.645568 12679 raft_consensus.cc:2410] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 27 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 16.
W20260709 12:14:23.592059 22898 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.079s	user 0.008s	sys 0.004s
I20260709 12:14:23.646908 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.647955 12674 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 32.
I20260709 12:14:23.608660 22861 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:23.518004 22853 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:23.592979 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:23.513046 22656 consensus_meta.cc:287] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.622s	user 0.005s	sys 0.011s
I20260709 12:14:23.550621 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:23.667845 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 117328us
I20260709 12:14:23.668927 12496 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:23.601457 12494 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:23.670204 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 68838.5us
I20260709 12:14:23.552014 12531 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:23.511165 22927 raft_consensus.cc:697] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:23.619851 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:23.530928 22668 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:23.583977 22856 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:14:23.626380 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1163) took 1164 ms. Trace:
I20260709 12:14:23.622403 12526 rpcz_store.cc:276] 0709 12:14:22.196252 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.649860 (+453608us) service_pool.cc:224] Handling call
0709 12:14:22.650362 (+   502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:23.219362 (+569000us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.562511 (+343149us) spinlock_profiling.cc:243] Waited 564 ms on lock 0x7b54003f7d54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:23.562686 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1298017664}
I20260709 12:14:23.547704 22850 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:23.655455 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.657872 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4627) took 1345 ms. Trace:
I20260709 12:14:23.658530 12604 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 33.
I20260709 12:14:23.658573 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.655095 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3874) took 1986 ms. Trace:
I20260709 12:14:23.663089 12671 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 32.
I20260709 12:14:23.684958 12679 raft_consensus.cc:2410] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 30 because replica has last-logged OpId of term: 29 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 16.
W20260709 12:14:23.642567 13304 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.327s (SENT) [suppressed 29 similar messages]
I20260709 12:14:23.684355 12678 rpcz_store.cc:276] 0709 12:14:22.312852 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.321133 (+  8281us) service_pool.cc:224] Handling call
0709 12:14:22.411948 (+ 90815us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:23.406160 (+994212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.657586 (+251426us) spinlock_profiling.cc:243] Waited 945 ms on lock 0x7b5400012cd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:23.657809 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2173320448}
W20260709 12:14:23.664664 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 104218us
I20260709 12:14:23.677961 12496 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 24 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:23.683280 12525 raft_consensus.cc:2410] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 27 because replica has last-logged OpId of term: 26 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 17.
I20260709 12:14:23.680968 12597 rpcz_store.cc:276] 0709 12:14:22.462364 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.567406 (+1105042us) service_pool.cc:224] Handling call
0709 12:14:23.567725 (+   319us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:23.626072 (+ 58347us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.626322 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:23.690949 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.691192 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4389) took 1436 ms. Trace:
I20260709 12:14:23.694706 12527 raft_consensus.cc:2468] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 32.
I20260709 12:14:23.543008 22701 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:23.684636 12524 rpcz_store.cc:276] 0709 12:14:21.668950 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.098248 (+429298us) service_pool.cc:224] Handling call
0709 12:14:22.098607 (+   359us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:22.171870 (+ 73263us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:22.171898 (+    28us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:22.184780 (+ 12882us) log.cc:844] Serialized 52 byte log entry
0709 12:14:22.392544 (+207764us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:14:22.556246 (+163702us) log.cc:844] Serialized 14 byte log entry
0709 12:14:22.556458 (+   212us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:22.556537 (+    79us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:23.135761 (+579224us) spinlock_profiling.cc:243] Waited 8.9 ms on lock 0x7b1800025840. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:14:23.591421 (+455660us) spinlock_profiling.cc:243] Waited 20.4 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:14:23.654769 (+ 63348us) raft_consensus.cc:1685] finished
0709 12:14:23.654799 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:23.655016 (+   217us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":164170,"prepare.queue_time_us":179172,"prepare.run_cpu_time_us":63,"prepare.run_wall_time_us":5076,"raft.queue_time_us":22372,"raft.run_cpu_time_us":362,"raft.run_wall_time_us":39545,"spinlock_wait_cycles":67377792,"thread_start_us":168683,"threads_started":1,"wal-append.queue_time_us":280832}
I20260709 12:14:23.714004 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:23.543242 22851 consensus_queue.cc:260] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:23.665169 22674 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:14:23.594110 22721 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:23.691674 12606 rpcz_store.cc:276] 0709 12:14:22.254586 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.214423 (+959837us) service_pool.cc:224] Handling call
0709 12:14:23.229589 (+ 15166us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:23.685897 (+456308us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:23.685937 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:23.685994 (+    57us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:23.686031 (+    37us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.686099 (+    68us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:23.691127 (+  5028us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":198198,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":118,"thread_start_us":455342,"threads_started":1}
I20260709 12:14:23.714365 13302 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:23.679814 22839 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:23.720191 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:23.680968 22887 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.680526 22852 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:14:23.720471 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:23.720494 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:23.720825 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:23.729385 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:23.746789 12676 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:23.731966 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.732156 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:14:23.732851 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.748623 12604 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:14:23.752213 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 62138.6us
I20260709 12:14:23.719779 22903 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.754599 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:23.654599 22857 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:23.768388 12678 raft_consensus.cc:2393] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 27: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:14:23.769577 22907 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.766s	user 0.000s	sys 0.003s
I20260709 12:14:23.733439 22681 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader election won for term 24
I20260709 12:14:23.774382 22902 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:23.774750 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.776057 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.723975 22668 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:23.768460 22698 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:23.778879 22823 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:23.780404 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.728575 22686 consensus_queue.cc:1059] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 26.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.061s
I20260709 12:14:23.769661 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:23.782951 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.764635 22860 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:23.764870 22852 raft_consensus.cc:697] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:23.785164 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:23.782881 12525 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26. Ops: []
I20260709 12:14:23.791729 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:14:23.716289 22716 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.503s	user 0.014s	sys 0.010s
I20260709 12:14:23.799211 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:23.681313 22656 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:23.805719 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:23.806394 12678 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:14:23.798576 12569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.567s	user 0.034s	sys 0.025s
I20260709 12:14:23.799741 12674 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26.
I20260709 12:14:23.800819 12599 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:23.804304 12606 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:23.753638 22856 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 27.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.043s
W20260709 12:14:23.811352 12569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.589s	user 0.034s	sys 0.025s
W20260709 12:14:23.835012 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:23.846549 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:23.811717 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 589836us
W20260709 12:14:23.855437 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.855469 12604 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:23.799655 22698 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:23.810352 22668 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:23.874279 22129 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:23.874917 22668 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:23.874888 22129 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:23.875315 22896 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:23.810570 22716 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.622s	user 0.015s	sys 0.010s
I20260709 12:14:23.878404 12678 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:14:23.799192 22681 raft_consensus.cc:697] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:23.879366 22129 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:23.798982 22927 consensus_queue.cc:237] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 26.13, Last appended by leader: 13, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:23.863752 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4640) took 1352 ms. Trace:
I20260709 12:14:23.878767 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.864255 22721 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:23.864832 22625 raft_consensus.cc:3055] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:14:23.856905 22922 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.054s
I20260709 12:14:23.914819 12604 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 16. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:14:23.865226 22656 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:23.866004 22892 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 1.348s	user 0.001s	sys 0.011s
I20260709 12:14:23.867141 22921 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:23.872766 22698 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:23.859071 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.760550 22887 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:23.888223 12673 rpcz_store.cc:276] 0709 12:14:22.511466 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.511723 (+   257us) service_pool.cc:224] Handling call
0709 12:14:22.518214 (+  6491us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:22.518655 (+   441us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:22.518684 (+    29us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:22.519165 (+   481us) log.cc:844] Serialized 26 byte log entry
0709 12:14:22.864745 (+345580us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:23.094103 (+229358us) log.cc:844] Serialized 14 byte log entry
0709 12:14:23.094417 (+   314us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.094526 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:23.723839 (+629313us) spinlock_profiling.cc:243] Waited 7.41 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:14:23.863413 (+139574us) raft_consensus.cc:1685] finished
0709 12:14:23.863527 (+   114us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:23.863680 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15987,"prepare.queue_time_us":395225,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":33,"raft.queue_time_us":126230,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":19684,"spinlock_wait_cycles":17046400,"thread_start_us":511326,"threads_started":2,"wal-append.queue_time_us":434701}
I20260709 12:14:23.900125 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.913154 12676 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 32.
W20260709 12:14:23.914103 13305 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.736s	user 0.009s	sys 0.032s
W20260709 12:14:23.922634 13305 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.745s	user 0.009s	sys 0.032s
I20260709 12:14:23.898766 22625 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:23.926056 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.927006 12678 raft_consensus.cc:2376] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25. Re-sending same reply.
I20260709 12:14:23.928210 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.928956 12678 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:23.930231 12675 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 29.
I20260709 12:14:23.930951 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.931658 12674 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 27.
I20260709 12:14:23.931869 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.932590 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:23.932753 22857 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:23.933166 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:23.933974 22902 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:23.934628 22902 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:23.936451 22851 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:23.936762 22701 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:23.937112 22902 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:23.938688 22866 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:23.922787 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 126480us
I20260709 12:14:23.940852 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.941694 12673 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:14:23.942816 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:23.942833 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:23.943552 22865 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:14:23.943969 22865 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:23.944254 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.945039 12673 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 32.
I20260709 12:14:23.944983 22865 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:23.801694 22823 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:23.947168 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.947909 12677 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 27.
I20260709 12:14:23.948027 22823 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:23.880645 22860 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:23.948935 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.950742 22860 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:23.933809 12675 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:23.781167 22903 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:23.918171 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:23.929947 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.955796 12671 raft_consensus.cc:2410] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 28 because replica has last-logged OpId of term: 27 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 18.
I20260709 12:14:23.955996 12678 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26.
I20260709 12:14:23.925336 22681 consensus_queue.cc:237] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 22.15, Last appended by leader: 15, Current term: 24, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:23.953737 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.909965 12599 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:23.953994 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:23.933734 12674 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 27: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:14:23.964543 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.965797 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:23.966773 12677 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:23.967978 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:14:23.969257 12572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.609s	user 0.012s	sys 0.014s
W20260709 12:14:23.969739 12572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.012s	sys 0.014s
I20260709 12:14:23.969956 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:23.971248 12679 raft_consensus.cc:2410] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 30 because replica has last-logged OpId of term: 29 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 16.
W20260709 12:14:23.929898 22822 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.573s	user 0.008s	sys 0.000s
I20260709 12:14:23.983326 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:23.985814 12673 raft_consensus.cc:2393] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 30: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:14:23.953683 22832 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.335s	user 0.033s	sys 0.080s
I20260709 12:14:23.993935 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:23.994658 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:23.995399 13333 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:14:23.993819 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:23.999888 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:24.000852 12677 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:24.008277 12606 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:14:24.020375 12525 raft_consensus.cc:2393] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 32: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:14:24.022591 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:23.993583 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:23.955186 22896 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.037024 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:23.991204 22832 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.405s	user 0.033s	sys 0.081s
I20260709 12:14:24.036985 13331 raft_consensus.cc:2410] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 25 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:14:24.037206 22656 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:24.037438 22823 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.037437 13337 raft_consensus.cc:1240] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:24.043215 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:24.030159 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:24.048175 12675 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 33.
I20260709 12:14:24.108553 12571 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:24.100468 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:24.111474 12526 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 27. Current term is 28. Ops: [27.16-27.16]
I20260709 12:14:24.094450 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:24.129565 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:24.133018 12675 raft_consensus.cc:2393] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:14:24.118755 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 126887us
I20260709 12:14:24.117357 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:24.130002 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:24.137504 12671 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:14:24.124771 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:24.140874 12528 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.674s	user 0.016s	sys 0.011s
W20260709 12:14:24.141523 12528 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.675s	user 0.016s	sys 0.011s
I20260709 12:14:24.150290 12671 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 31. Current term is 33. Ops: []
W20260709 12:14:24.042974 22624 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.591s	user 0.000s	sys 0.025s
I20260709 12:14:23.998059 22822 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:24.176211 12671 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 23. Current term is 24. Ops: [23.15-23.15]
W20260709 12:14:24.103116 22929 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.203s	user 0.000s	sys 0.000s
I20260709 12:14:24.175995 13339 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26. Ops: [25.16-25.16]
I20260709 12:14:24.175944 12527 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 30. Current term is 31. Ops: []
I20260709 12:14:24.179677 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:24.185986 13331 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:14:24.186244 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.181489 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:24.182737 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:24.195885 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:24.198815 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.100841 22853 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:24.200558 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 92279.6us
I20260709 12:14:24.162228 22865 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:14:24.199846 12529 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.646s	user 0.004s	sys 0.036s
I20260709 12:14:24.202338 12675 raft_consensus.cc:1275] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 19. (term mismatch)
W20260709 12:14:24.219442 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 119043us
I20260709 12:14:24.204113 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:24.219857 12529 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.666s	user 0.005s	sys 0.036s
W20260709 12:14:24.177101 22624 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.725s	user 0.000s	sys 0.028s
I20260709 12:14:24.206393 12679 raft_consensus.cc:2393] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 30: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:14:24.227859 12496 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.507s	user 0.008s	sys 0.016s
I20260709 12:14:24.207439 12571 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:24.203549 13305 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:24.219931 12675 pending_rounds.cc:85] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 18
I20260709 12:14:24.176610 22921 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:14:24.228721 12496 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.508s	user 0.008s	sys 0.016s
I20260709 12:14:24.228842 12572 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:24.201110 22621 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.624s	user 0.006s	sys 0.021s
W20260709 12:14:24.221346 12602 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.570s	user 0.015s	sys 0.013s
I20260709 12:14:24.201467 12674 raft_consensus.cc:2410] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 27 because replica has last-logged OpId of term: 25 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 13.
W20260709 12:14:24.232686 12602 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.582s	user 0.015s	sys 0.013s
W20260709 12:14:24.220983 12672 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.780s	user 0.009s	sys 0.027s
W20260709 12:14:24.233639 12672 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.009s	sys 0.028s
I20260709 12:14:24.235898 12675 pending_rounds.cc:107] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 27.19
I20260709 12:14:24.236420 12675 raft_consensus.cc:2889] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:24.056876 22896 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:24.158799 22823 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:24.161832 22656 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:24.175674 22857 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.054s
I20260709 12:14:24.175335 22860 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.055s
I20260709 12:14:24.249045 12606 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:14:24.267961 12675 raft_consensus.cc:1240] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 24. Current term is 25. Ops: []
I20260709 12:14:24.249882 12671 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:14:24.250494 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:24.027689 22852 consensus_queue.cc:237] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:24.260848 22880 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.276952 22880 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:24.278789 22936 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.074s	user 0.000s	sys 0.000s
I20260709 12:14:24.279548 22880 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:24.280100 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4390) took 1962 ms. Trace:
W20260709 12:14:24.238202 22714 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 2.069s	user 0.015s	sys 0.033s
I20260709 12:14:24.254910 22129 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:24.280537 12605 rpcz_store.cc:276] 0709 12:14:22.317500 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.215892 (+898392us) service_pool.cc:224] Handling call
0709 12:14:23.235901 (+ 20009us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:23.256561 (+ 20660us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:23.256614 (+    53us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:23.257321 (+   707us) log.cc:844] Serialized 26 byte log entry
0709 12:14:23.696127 (+438806us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:23.696167 (+    40us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:23.696236 (+    69us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:24.279885 (+583649us) raft_consensus.cc:1685] finished
0709 12:14:24.279905 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.280029 (+   124us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":200969,"raft.run_cpu_time_us":68,"raft.run_wall_time_us":68,"thread_start_us":422861,"threads_started":1,"wal-append.queue_time_us":816738}
I20260709 12:14:24.281790 22129 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:24.282212 22129 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:24.284842 13332 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
W20260709 12:14:24.232280 22621 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.700s	user 0.006s	sys 0.021s
I20260709 12:14:24.177914 22933 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.084s
I20260709 12:14:24.231498 22903 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:24.269865 12671 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 23. Current term is 24. Ops: []
W20260709 12:14:24.239795 22893 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.690s	user 0.033s	sys 0.010s
I20260709 12:14:24.271154 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:24.293339 12679 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 30 index: 21. Preceding OpId from leader: term: 32 index: 22. (index mismatch)
I20260709 12:14:24.308295 12675 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:14:24.253304 22674 raft_consensus.cc:2804] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:14:24.227895 22853 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.042s
W20260709 12:14:24.317551 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 445319us
W20260709 12:14:24.318760 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.320048 22129 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:24.229580 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 551678us
I20260709 12:14:24.307337 13335 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
W20260709 12:14:24.286654 22714 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 2.124s	user 0.015s	sys 0.034s
W20260709 12:14:24.322978 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 72570.7us
I20260709 12:14:24.344187 12571 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:24.297482 22893 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.748s	user 0.034s	sys 0.010s
W20260709 12:14:24.351374 12494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.631s	user 0.008s	sys 0.019s
W20260709 12:14:24.366855 12494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.647s	user 0.009s	sys 0.020s
I20260709 12:14:24.367444 12675 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 31. Current term is 32. Ops: [31.16-31.16]
W20260709 12:14:24.336580 13304 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.651s	user 0.016s	sys 0.057s
W20260709 12:14:24.369932 13304 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.684s	user 0.017s	sys 0.058s
W20260709 12:14:24.370232 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 727677us
I20260709 12:14:24.371999 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:24.372740 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.377625 12675 raft_consensus.cc:1275] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 29 index: 18. (index mismatch)
W20260709 12:14:24.379727 12600 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.819s	user 0.006s	sys 0.021s
W20260709 12:14:24.380177 12600 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.057s	user 0.006s	sys 0.021s
I20260709 12:14:24.290647 22921 consensus_queue.cc:1059] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.057s
W20260709 12:14:24.383916 12598 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.620s	user 0.022s	sys 0.018s
W20260709 12:14:24.387215 13304 leader_election.cc:336] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.486s (SENT)
I20260709 12:14:24.261654 22930 consensus_queue.cc:1059] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 33.23, Next index: 24, Last known committed idx: 22, Time since last communication: 0.085s
I20260709 12:14:24.389200 12675 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 27 index: 17. (term mismatch)
I20260709 12:14:24.389684 22129 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:24.389209 12598 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.625s	user 0.022s	sys 0.018s
I20260709 12:14:24.385070 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4392) took 1944 ms. Trace:
I20260709 12:14:24.392202 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:24.389838 13304 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:24.318300 22941 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:24.400070 12678 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: [25.15-25.15]
I20260709 12:14:24.343243 22674 raft_consensus.cc:697] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:24.307837 22837 consensus_queue.cc:1059] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.076s
I20260709 12:14:24.415271 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3887) took 1535 ms. Trace:
I20260709 12:14:24.396874 22129 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:24.379501 22860 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:24.405892 12606 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 26: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:24.389724 12675 pending_rounds.cc:85] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 16
I20260709 12:14:24.435452 12675 pending_rounds.cc:107] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 25.17
I20260709 12:14:24.435904 12675 raft_consensus.cc:2889] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:24.407006 12496 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:24.415997 12528 rpcz_store.cc:276] 0709 12:14:22.880004 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.465826 (+585822us) service_pool.cc:224] Handling call
0709 12:14:23.466107 (+   281us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:23.466500 (+   393us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:14:24.142049 (+675549us) log.cc:844] Serialized 14 byte log entry
0709 12:14:24.414665 (+272616us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:24.414756 (+    91us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:14:24.414829 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.414939 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.415179 (+   240us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":640030,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"thread_start_us":947223,"threads_started":2}
I20260709 12:14:24.346992 22829 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:24.408144 22937 raft_consensus.cc:3055] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:14:24.314860 22721 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.075s
W20260709 12:14:24.410280 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.390801 12600 rpcz_store.cc:276] 0709 12:14:22.441302 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.222074 (+780772us) service_pool.cc:224] Handling call
0709 12:14:23.251517 (+ 29443us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:23.274510 (+ 22993us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:14:23.274814 (+   304us) log.cc:844] Serialized 14 byte log entry
0709 12:14:23.323039 (+ 48225us) spinlock_profiling.cc:243] Waited 6.82 ms on lock 0x7b180015b200. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a 00007f1488a2aa2a
0709 12:14:24.384699 (+1061660us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:24.384760 (+    61us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:14:24.384817 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.384887 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.385006 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":4086,"prepare.queue_time_us":134,"prepare.run_cpu_time_us":107,"prepare.run_wall_time_us":3218,"spinlock_wait_cycles":15681920,"thread_start_us":1057287,"threads_started":1}
I20260709 12:14:24.390542 22933 raft_consensus.cc:3055] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Stepping down as leader of term 25
W20260709 12:14:24.392926 12601 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.626s	user 0.021s	sys 0.020s
W20260709 12:14:24.464964 12601 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.710s	user 0.021s	sys 0.020s
W20260709 12:14:24.442603 12644 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.508s	user 0.011s	sys 0.010s
W20260709 12:14:24.465562 12644 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.531s	user 0.011s	sys 0.010s
I20260709 12:14:24.463083 12675 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 31. Current term is 32. Ops: [31.15-31.15]
W20260709 12:14:24.426733 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:24.391681 22839 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.615s	user 0.007s	sys 0.024s
I20260709 12:14:24.444509 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:14:24.444458 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:24.469321 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:24.439869 22924 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.659s	user 0.008s	sys 0.017s
I20260709 12:14:24.396656 12527 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 31. Current term is 32. Ops: []
I20260709 12:14:24.403869 22903 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
W20260709 12:14:24.453054 22698 consensus_meta.cc:287] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.527s	user 0.001s	sys 0.010s
W20260709 12:14:24.456182 12524 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.644s	user 0.007s	sys 0.014s
I20260709 12:14:24.404034 22880 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:24.461418 12678 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:14:24.481220 12606 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:14:24.488276 12675 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 30 index: 17. (term mismatch)
I20260709 12:14:24.495489 12675 pending_rounds.cc:85] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 16
I20260709 12:14:24.485508 12494 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:24.495813 12678 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 34. Current term is 35. Ops: []
W20260709 12:14:24.497256 12524 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.686s	user 0.007s	sys 0.014s
I20260709 12:14:24.495870 12675 pending_rounds.cc:107] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 29.17
I20260709 12:14:24.501215 12675 raft_consensus.cc:2889] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:24.408830 22866 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.058s
I20260709 12:14:24.499979 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:24.480846 22839 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.704s	user 0.007s	sys 0.024s
W20260709 12:14:24.452733 22922 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.518s	user 0.006s	sys 0.015s
I20260709 12:14:24.451485 22887 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:24.457351 22937 raft_consensus.cc:740] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:14:24.494174 22924 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s	user 0.008s	sys 0.018s
I20260709 12:14:24.477766 22933 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:14:24.534651 12603 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 1.315s	user 0.008s	sys 0.037s
I20260709 12:14:24.494432 22681 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election won for term 26
W20260709 12:14:24.526211 22922 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.592s	user 0.007s	sys 0.015s
I20260709 12:14:24.569675 22129 consensus_queue.cc:1048] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
W20260709 12:14:24.576056 12603 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.367s	user 0.009s	sys 0.037s
W20260709 12:14:24.585534 12677 consensus_meta.cc:287] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.557s	user 0.009s	sys 0.000s
I20260709 12:14:24.587492 12677 raft_consensus.cc:2410] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 27 because replica has last-logged OpId of term: 26 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:14:24.588277 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3929) took 1056 ms. Trace:
I20260709 12:14:24.588832 12677 rpcz_store.cc:276] 0709 12:14:23.532010 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.999550 (+467540us) service_pool.cc:224] Handling call
0709 12:14:24.588191 (+588641us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:24.506083 22837 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:24.599536 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.518218 22822 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:24.610539 13331 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 35.
W20260709 12:14:24.623618 22942 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.119s	user 0.000s	sys 0.000s
I20260709 12:14:24.624976 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4648) took 1302 ms. Trace:
I20260709 12:14:24.625357 12672 rpcz_store.cc:276] 0709 12:14:23.322672 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.323691 (+  1019us) service_pool.cc:224] Handling call
0709 12:14:23.324195 (+   504us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:23.354845 (+ 30650us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:23.354899 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:23.355669 (+   770us) log.cc:844] Serialized 26 byte log entry
0709 12:14:24.234108 (+878439us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:14:24.234185 (+    77us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.234284 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:24.624755 (+390471us) raft_consensus.cc:1685] finished
0709 12:14:24.624778 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.624917 (+   139us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64128,"raft.queue_time_us":30823,"raft.run_cpu_time_us":113,"raft.run_wall_time_us":113,"thread_start_us":792845,"threads_started":1,"wal-append.queue_time_us":1108303}
I20260709 12:14:24.587970 22921 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.633106 22921 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:24.635278 22943 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.184s	user 0.000s	sys 0.000s
W20260709 12:14:24.636623 22954 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.069s	user 0.000s	sys 0.000s
I20260709 12:14:24.640097 22822 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:24.641134 22822 raft_consensus.cc:697] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:24.647765 12604 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.605s	user 0.010s	sys 0.029s
I20260709 12:14:24.517257 22880 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:24.496352 22592 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.708s	user 0.015s	sys 0.027s
W20260709 12:14:24.659655 12604 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s	user 0.011s	sys 0.029s
I20260709 12:14:24.507688 22698 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:24.618285 22898 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.071s	user 0.001s	sys 0.000s
I20260709 12:14:24.606585 22896 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:24.465170 22829 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:24.657097 22592 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.879s	user 0.015s	sys 0.028s
W20260709 12:14:24.680066 22851 consensus_meta.cc:287] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.743s	user 0.013s	sys 0.000s
W20260709 12:14:24.683833 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 448925us
W20260709 12:14:24.701680 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 459481us
W20260709 12:14:24.701732 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 456888us
I20260709 12:14:24.649461 22823 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:24.749455 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:24.614398 22837 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:24.752712 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.702083 12529 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3805) took 1805 ms. Trace:
I20260709 12:14:24.603017 22686 consensus_queue.cc:1059] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 24.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.086s
I20260709 12:14:24.749027 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1173) took 1398 ms. Trace:
I20260709 12:14:24.755980 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4386) took 2502 ms. Trace:
I20260709 12:14:24.755997 12572 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:24.756050 12680 raft_consensus.cc:2410] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 27 because replica has last-logged OpId of term: 26 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 13.
I20260709 12:14:24.757993 22129 raft_consensus.cc:2804] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:14:24.759222 22129 raft_consensus.cc:697] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:24.762671 12527 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:14:24.755337 12597 rpcz_store.cc:276] 0709 12:14:23.351125 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.811991 (+460866us) service_pool.cc:224] Handling call
0709 12:14:23.812391 (+   400us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:23.862423 (+ 50032us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:23.862458 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:23.862938 (+   480us) log.cc:844] Serialized 26 byte log entry
0709 12:14:24.231798 (+368860us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:14:24.250394 (+ 18596us) log.cc:844] Serialized 14 byte log entry
0709 12:14:24.748660 (+498266us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.748760 (+   100us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:24.748813 (+    53us) raft_consensus.cc:1685] finished
0709 12:14:24.748827 (+    14us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.748964 (+   137us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":115576,"prepare.queue_time_us":74917,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":54,"raft.queue_time_us":156995,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":134,"thread_start_us":750664,"threads_started":2,"wal-append.queue_time_us":684877}
I20260709 12:14:24.767709 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:24.756886 12598 rpcz_store.cc:276] 0709 12:14:22.253075 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.120236 (+867161us) service_pool.cc:224] Handling call
0709 12:14:23.184906 (+ 64670us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:23.664927 (+480021us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:23.664962 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:23.683671 (+ 18709us) log.cc:844] Serialized 26 byte log entry
0709 12:14:24.390898 (+707227us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:14:24.390958 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.391054 (+    96us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:24.755697 (+364643us) raft_consensus.cc:1685] finished
0709 12:14:24.755724 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.755907 (+   183us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":11258,"raft.queue_time_us":46803,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":11241,"thread_start_us":1094820,"threads_started":2,"wal-append.queue_time_us":738491}
I20260709 12:14:24.742425 22656 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:24.602283 22674 consensus_queue.cc:237] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:24.774471 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:24.778556 12528 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 28: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:24.775209 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:24.775630 12525 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:24.778860 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:24.781275 22129 consensus_queue.cc:237] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 30.15, Last appended by leader: 15, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:14:24.813493 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.754601 12529 rpcz_store.cc:276] 0709 12:14:22.896964 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.524221 (+627257us) service_pool.cc:224] Handling call
0709 12:14:23.553416 (+ 29195us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:23.553693 (+   277us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:14:24.220441 (+666748us) log.cc:844] Serialized 14 byte log entry
0709 12:14:24.701610 (+481169us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:24.701685 (+    75us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:24.701748 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.701831 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.701998 (+   167us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":553137,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":153,"raft.run_cpu_time_us":104,"raft.run_wall_time_us":105,"thread_start_us":1146719,"threads_started":2}
I20260709 12:14:24.848345 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201"
I20260709 12:14:24.849365 12523 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:24.850544 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:24.861716 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:24.867950 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:24.872607 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.875059 12643 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:24.876611 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:24.878194 12677 raft_consensus.cc:2410] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 26 because replica has last-logged OpId of term: 23 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 21 index: 16.
I20260709 12:14:24.880314 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:24.881057 13332 raft_consensus.cc:3060] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:24.764580 22860 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.094s
I20260709 12:14:24.764249 22857 consensus_queue.cc:1059] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.090s
I20260709 12:14:24.894600 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:24.898196 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:24.690900 22937 consensus_queue.cc:260] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 33.23, Last appended by leader: 23, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:24.900914 13330 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 27. Current term is 28. Ops: [27.20-27.20]
I20260709 12:14:24.901621 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:24.902012 22946 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.245s	user 0.000s	sys 0.000s
I20260709 12:14:24.903105 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:24.904019 12598 raft_consensus.cc:2410] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 34 because replica has last-logged OpId of term: 33 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 32 index: 19.
I20260709 12:14:24.903851 22961 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:14:24.905035 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:24.905537 22961 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:14:24.906692 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:24.907313 13337 raft_consensus.cc:2376] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25. Re-sending same reply.
I20260709 12:14:24.910665 22961 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:24.912006 22961 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 34.22, Last appended by leader: 22, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:24.905790 12598 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:14:24.917735 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 432294us
I20260709 12:14:24.920486 13332 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 31.
I20260709 12:14:24.917955 12523 raft_consensus.cc:2468] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 31.
I20260709 12:14:24.919225 12494 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:24.882956 22939 log.cc:927] Time spent T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.400s	user 0.001s	sys 0.000s
I20260709 12:14:24.937058 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:24.918964 12569 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:24.940361 12494 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:24.933178 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:24.941427 12569 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:24.946821 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.952162 12597 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 31 index: 18. (term mismatch)
W20260709 12:14:24.954373 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:24.952744 12597 pending_rounds.cc:85] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 17
I20260709 12:14:24.962831 12597 pending_rounds.cc:107] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 30.18
I20260709 12:14:24.963157 12597 raft_consensus.cc:2889] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:24.964916 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:24.966394 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:24.773656 22822 consensus_queue.cc:237] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:24.760473 22933 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.15, Last appended by leader: 15, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:24.978233 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4398) took 1368 ms. Trace:
I20260709 12:14:24.949433 22937 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:24.943473 22851 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:24.943979 22953 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:14:24.964450 22960 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:24.992103 13304 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.594s	user 0.011s	sys 0.015s
I20260709 12:14:24.950467 22887 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:14:24.953586 22866 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:24.980876 12604 rpcz_store.cc:276] 0709 12:14:23.609853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.949458 (+339605us) service_pool.cc:224] Handling call
0709 12:14:23.949760 (+   302us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:23.954436 (+  4676us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:14:23.983279 (+ 28843us) log.cc:844] Serialized 14 byte log entry
0709 12:14:24.974102 (+990823us) log.cc:844] Serialized 14 byte log entry
0709 12:14:24.977653 (+  3551us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:24.977776 (+   123us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:24.977852 (+    76us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.977957 (+   105us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:24.978151 (+   194us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3045,"prepare.queue_time_us":36305,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":51,"raft.queue_time_us":1376,"raft.run_cpu_time_us":163,"raft.run_wall_time_us":164,"spinlock_wait_cycles":16128,"thread_start_us":930321,"threads_started":2,"wal-append.queue_time_us":801045}
I20260709 12:14:24.958298 12494 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:24.955274 22967 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:24.958240 22857 raft_consensus.cc:2804] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:14:24.959667 22621 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:25.001035 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:25.042448 12597 raft_consensus.cc:2410] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 32 because replica has last-logged OpId of term: 31 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 14.
I20260709 12:14:24.941735 22681 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:24.977084 22927 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.670s	user 0.002s	sys 0.023s
W20260709 12:14:25.019860 13304 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.621s	user 0.011s	sys 0.015s
W20260709 12:14:25.044147 13304 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.041483 12600 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:25.024960 12598 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:24.940093 22902 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.055s
W20260709 12:14:25.033785 12599 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.000s	sys 0.036s
W20260709 12:14:25.035816 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 77656.2us
W20260709 12:14:25.031044 22932 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.000s	sys 0.028s
I20260709 12:14:24.968251 22852 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.069s
W20260709 12:14:25.048923 22927 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.742s	user 0.002s	sys 0.024s
W20260709 12:14:25.583873 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 431162us
I20260709 12:14:25.720134 22721 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.720806 12673 raft_consensus.cc:1240] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:25.720729 22721 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:25.722700 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3930) took 2160 ms. Trace:
I20260709 12:14:25.722836 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:25.723019 22721 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:25.723908 22988 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.174s	user 0.000s	sys 0.000s
I20260709 12:14:25.724227 22721 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:25.724598 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:25.724829 22721 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:25.726418 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:25.727185 22721 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:25.728266 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:25.561987 12678 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.013s	sys 0.008s
I20260709 12:14:25.729281 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:14:25.729969 12678 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.736s	user 0.017s	sys 0.008s
W20260709 12:14:25.354734 12601 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.834s	user 0.000s	sys 0.023s
W20260709 12:14:25.730950 12601 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.219s	user 0.003s	sys 0.029s
I20260709 12:14:25.723074 12673 rpcz_store.cc:276] 0709 12:14:23.562383 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.264355 (+701972us) service_pool.cc:224] Handling call
0709 12:14:24.280402 (+ 16047us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:25.721307 (+1440905us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.722468 (+  1161us) spinlock_profiling.cc:243] Waited 1.44 s on lock 0x7b540012d754. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:25.722642 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3312108800}
W20260709 12:14:25.363616 12571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.991s	user 0.002s	sys 0.022s
W20260709 12:14:25.732276 12571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.360s	user 0.010s	sys 0.022s
W20260709 12:14:25.733057 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 1.38886e+06us
I20260709 12:14:25.266848 13330 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38482 (request call id 916) took 1236 ms. Trace:
I20260709 12:14:25.734027 12571 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:25.491271 12530 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.552s	user 0.000s	sys 0.025s
I20260709 12:14:25.729314 12524 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3892) took 1946 ms. Trace:
W20260709 12:14:25.735356 12530 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.818s	user 0.008s	sys 0.032s
W20260709 12:14:25.205415 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:25.216383 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:25.204325 12496 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.766s	user 0.001s	sys 0.024s
I20260709 12:14:25.735773 12524 rpcz_store.cc:276] 0709 12:14:23.782564 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.783234 (+   670us) service_pool.cc:224] Handling call
0709 12:14:23.783580 (+   346us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:23.811426 (+ 27846us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:14:24.505524 (+694098us) log.cc:844] Serialized 14 byte log entry
0709 12:14:24.537941 (+ 32417us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:24.538009 (+    68us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:14:24.538062 (+    53us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.729021 (+1190959us) spinlock_profiling.cc:243] Waited 27.6 ms on lock 0x7b5400407c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:25.729124 (+   103us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:25.729254 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26057,"prepare.queue_time_us":296032,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":45,"raft.queue_time_us":133126,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":127,"spinlock_wait_cycles":63382784,"thread_start_us":688720,"threads_started":1}
W20260709 12:14:25.045289 13304 leader_election.cc:336] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Timed out: RequestConsensusVote RPC to 127.11.13.131:38613 timed out after 2.709s (SENT)
W20260709 12:14:25.383354 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:25.121017 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:25.747151 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.739497 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:25.762501 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.758497 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3959) took 1841 ms. Trace:
W20260709 12:14:25.736830 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:25.125084 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:25.606642 13338 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.003s	sys 0.020s
W20260709 12:14:25.331271 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:25.734337 13330 rpcz_store.cc:276] 0709 12:14:23.966362 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.966614 (+   252us) service_pool.cc:224] Handling call
0709 12:14:24.014055 (+ 47441us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:24.901510 (+887455us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.182995 (+281485us) spinlock_profiling.cc:243] Waited 733 ms on lock 0x7b54004e84d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:25.202343 (+ 19348us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1685131520}
W20260709 12:14:25.760097 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 714797us
I20260709 12:14:25.771075 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:25.772763 13338 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.796s	user 0.008s	sys 0.021s
I20260709 12:14:25.769304 12597 raft_consensus.cc:2410] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 31 because replica has last-logged OpId of term: 30 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 14.
W20260709 12:14:25.190088 22968 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
W20260709 12:14:25.579074 22975 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.400s	user 0.000s	sys 0.001s
W20260709 12:14:25.774844 22129 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.779s	user 0.012s	sys 0.023s
I20260709 12:14:25.772763 12598 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
W20260709 12:14:25.126459 22947 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.467s	user 0.001s	sys 0.001s
W20260709 12:14:25.344834 22624 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.941s	user 0.006s	sys 0.016s
W20260709 12:14:25.117465 22971 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.117s	user 0.000s	sys 0.000s
I20260709 12:14:25.763334 22933 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:25.056984 22857 raft_consensus.cc:697] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:25.770535 22982 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.770215 22714 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:25.798604 13335 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 34. Current term is 35. Ops: []
W20260709 12:14:25.798923 13332 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.000s	sys 0.022s
W20260709 12:14:25.799371 13332 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.696s	user 0.000s	sys 0.022s
W20260709 12:14:25.120711 22965 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.084s	user 0.000s	sys 0.000s
I20260709 12:14:25.763595 12678 rpcz_store.cc:276] 0709 12:14:23.916782 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.498446 (+581664us) service_pool.cc:224] Handling call
0709 12:14:24.498837 (+   391us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:24.499117 (+   280us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:14:24.965737 (+466620us) log.cc:844] Serialized 14 byte log entry
0709 12:14:25.757983 (+792246us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:25.758074 (+    91us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:25.758176 (+   102us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.758272 (+    96us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:25.758430 (+   158us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":679817,"prepare.run_cpu_time_us":44,"prepare.run_wall_time_us":47,"thread_start_us":1183741,"threads_started":2}
W20260709 12:14:25.119071 22950 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.238s	user 0.000s	sys 0.000s
W20260709 12:14:25.760011 12496 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.322s	user 0.002s	sys 0.037s
W20260709 12:14:25.760221 12599 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.363s	user 0.009s	sys 0.055s
W20260709 12:14:25.737708 12494 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.006s	sys 0.013s
W20260709 12:14:25.803658 12494 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.732s	user 0.008s	sys 0.014s
W20260709 12:14:25.678923 22847 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 2.037s	user 0.012s	sys 0.006s
W20260709 12:14:25.780023 22129 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.792s	user 0.012s	sys 0.024s
I20260709 12:14:25.804620 22847 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:14:25.780360 12674 thread.cc:641] prepare [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.011s	sys 0.031s
I20260709 12:14:25.440454 22686 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:25.776294 12646 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.575s	user 0.014s	sys 0.006s
W20260709 12:14:25.811008 12646 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.610s	user 0.015s	sys 0.006s
I20260709 12:14:25.048338 22887 raft_consensus.cc:697] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:25.016389 22851 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:25.266381 22837 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:25.364949 22981 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.670436 22701 raft_consensus.cc:3055] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:14:25.073539 22823 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:25.433925 22979 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:25.837411 22979 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:25.838654 23007 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.839231 23007 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:25.840354 22979 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:25.841246 23007 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:25.116226 22880 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:25.188880 22829 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:25.166628 22656 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.842283 22656 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:25.775640 12571 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:25.041462 22953 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:25.049090 22866 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:25.103516 22698 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:25.555033 22961 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.638s	user 0.003s	sys 0.027s
I20260709 12:14:25.790973 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:25.791456 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:25.257006 22964 log.cc:927] Time spent T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.296s	user 0.001s	sys 0.001s
I20260709 12:14:25.827831 22714 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:25.770634 22853 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.857841 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:25.858286 22941 raft_consensus.cc:3055] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:14:25.858791 22941 raft_consensus.cc:740] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:25.859405 22714 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:25.859704 22941 consensus_queue.cc:260] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 31.22, Last appended by leader: 22, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:25.119872 22966 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.169s	user 0.000s	sys 0.001s
I20260709 12:14:25.860787 22941 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:14:25.116604 22955 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.250s	user 0.001s	sys 0.000s
W20260709 12:14:25.759999 22932 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 1.371s	user 0.009s	sys 0.043s
I20260709 12:14:25.861960 22866 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:25.785557 22624 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 1.397s	user 0.010s	sys 0.022s
I20260709 12:14:25.037557 22960 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:25.862936 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4410) took 1460 ms. Trace:
I20260709 12:14:25.863358 12606 rpcz_store.cc:276] 0709 12:14:24.402487 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.514165 (+111678us) service_pool.cc:224] Handling call
0709 12:14:24.514491 (+   326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:24.773381 (+258890us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:24.773417 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:24.773460 (+    43us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:24.773492 (+    32us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.862704 (+1089212us) spinlock_profiling.cc:243] Waited 259 ms on lock 0x7b54002e5ed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:25.862750 (+    46us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:25.862878 (+   128us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":594833152}
W20260709 12:14:25.864668 22961 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.936s	user 0.003s	sys 0.031s
I20260709 12:14:25.057305 22621 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:25.859468 22856 raft_consensus.cc:3055] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:14:25.866124 22856 raft_consensus.cc:740] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:25.867028 22621 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:25.867100 22856 consensus_queue.cc:260] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:25.868557 22856 raft_consensus.cc:3060] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:25.819165 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:25.805570 22982 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:25.864645 22960 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:25.870393 12598 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:14:25.835111 22823 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:25.871645 22823 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:25.871791 22982 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:25.874859 22978 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.386s	user 0.001s	sys 0.001s
I20260709 12:14:25.860707 22829 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:25.876511 22829 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:25.876916 22829 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:25.882470 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3944) took 2107 ms. Trace:
I20260709 12:14:25.818570 22686 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:25.883044 12679 rpcz_store.cc:276] 0709 12:14:23.775298 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.318163 (+542865us) service_pool.cc:224] Handling call
0709 12:14:24.366662 (+ 48499us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:24.375570 (+  8908us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:24.375627 (+    57us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:24.401532 (+ 25905us) log.cc:844] Serialized 26 byte log entry
0709 12:14:24.848472 (+446940us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:14:24.848533 (+    61us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.848647 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:25.882149 (+1033502us) raft_consensus.cc:1685] finished
0709 12:14:25.882173 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:25.882396 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":18830,"raft.queue_time_us":19798,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":3544,"thread_start_us":365885,"threads_started":1,"wal-append.queue_time_us":804971}
I20260709 12:14:25.884788 22686 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:25.836479 23010 consensus_queue.cc:1059] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.19, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:14:25.040695 22967 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:25.844447 12530 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3822) took 1064 ms. Trace:
I20260709 12:14:25.886839 12530 rpcz_store.cc:276] 0709 12:14:24.780486 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.781297 (+   811us) service_pool.cc:224] Handling call
0709 12:14:24.781598 (+   301us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:24.781952 (+   354us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:24.781995 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:24.828638 (+ 46643us) log.cc:844] Serialized 26 byte log entry
0709 12:14:25.760231 (+931593us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:25.760270 (+    39us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.760330 (+    60us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:25.844121 (+ 83791us) raft_consensus.cc:1685] finished
0709 12:14:25.844149 (+    28us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:25.844357 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":382,"raft.run_cpu_time_us":165,"raft.run_wall_time_us":48393,"thread_start_us":842552,"threads_started":1,"wal-append.queue_time_us":970944}
I20260709 12:14:25.888165 22967 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:25.888778 22967 raft_consensus.cc:3055] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:14:25.889143 22967 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:25.805326 22847 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.806227 22949 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.467221 22681 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 23.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:25.858428 22853 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:25.803249 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 1.36542e+06us
W20260709 12:14:25.805577 12674 thread.cc:608] prepare [worker] (thread pool) Time spent starting thread: real 0.841s	user 0.012s	sys 0.031s
W20260709 12:14:25.860669 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 69274.6us
I20260709 12:14:25.912335 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:25.867153 22887 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 23.17, Last appended by leader: 17, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:25.844632 22981 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:25.836165 22986 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:14:25.811321 22902 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:25.894726 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3942) took 2120 ms. Trace:
W20260709 12:14:25.782222 22674 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.657s	user 0.011s	sys 0.015s
I20260709 12:14:25.782733 22625 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:25.843907 22921 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:25.828533 22701 raft_consensus.cc:740] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:14:26.269671 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.345695 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:26.026706 12606 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 30. Current term is 31. Ops: []
W20260709 12:14:26.080303 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 159709us
I20260709 12:14:26.208706 12527 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3821) took 1428 ms. Trace:
W20260709 12:14:26.349985 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.360723 12671 rpcz_store.cc:276] 0709 12:14:23.774508 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.319824 (+545316us) service_pool.cc:224] Handling call
0709 12:14:24.320471 (+   647us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:24.321342 (+   871us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:24.321387 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:24.321974 (+   587us) log.cc:844] Serialized 26 byte log entry
0709 12:14:24.786511 (+464537us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:24.786571 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.786670 (+    99us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:25.894399 (+1107729us) raft_consensus.cc:1685] finished
0709 12:14:25.894428 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:25.894639 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":25159,"raft.run_cpu_time_us":191,"raft.run_wall_time_us":18605,"thread_start_us":372016,"threads_started":1,"wal-append.queue_time_us":635563}
W20260709 12:14:26.242363 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 443463us
I20260709 12:14:26.108075 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4403) took 1739 ms. Trace:
W20260709 12:14:26.363819 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 431551us
W20260709 12:14:26.410856 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 498576us
I20260709 12:14:26.000272 13338 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 1169) took 1100 ms. Trace:
I20260709 12:14:26.009717 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:26.412386 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:26.412613 13304 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:26.365097 12604 raft_consensus.cc:1240] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27. Ops: [26.14-26.14]
I20260709 12:14:26.021018 13332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38482 (request call id 921) took 1076 ms. Trace:
W20260709 12:14:26.415211 12680 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3920) took 2884 ms (client timeout 1601 ms). Trace:
W20260709 12:14:26.194665 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.251096 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:25.995303 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 219713us
W20260709 12:14:26.415657 12680 rpcz_store.cc:269] 0709 12:14:23.464460 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.466547 (+  2087us) service_pool.cc:224] Handling call
0709 12:14:26.287758 (+2821211us) spinlock_profiling.cc:243] Waited 1.25 s on lock 0x7b5400091fd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:26.348581 (+ 60823us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2866417408}
I20260709 12:14:26.424047 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:26.132174 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:26.425639 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 174683us
I20260709 12:14:26.427640 12494 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:14:26.187471 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:26.426250 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:26.430828 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.319667 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4387) took 4066 ms. Trace:
I20260709 12:14:26.364367 12599 rpcz_store.cc:276] 0709 12:14:24.368459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.369309 (+   850us) service_pool.cc:224] Handling call
0709 12:14:24.369656 (+   347us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:24.369928 (+   272us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:14:25.803741 (+1433813us) log.cc:844] Serialized 14 byte log entry
0709 12:14:26.107355 (+303614us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:26.107415 (+    60us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:14:26.107483 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.107573 (+    90us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.107979 (+   406us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":6651,"prepare.queue_time_us":744425,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":57,"raft.queue_time_us":1949,"raft.run_cpu_time_us":97,"raft.run_wall_time_us":99,"thread_start_us":1709043,"threads_started":2,"wal-append.queue_time_us":603035}
I20260709 12:14:26.415519 13332 rpcz_store.cc:276] 0709 12:14:24.944583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.959849 (+ 15266us) service_pool.cc:224] Handling call
0709 12:14:24.960217 (+   368us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:24.991518 (+ 31301us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:24.991553 (+    35us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:25.015723 (+ 24170us) log.cc:844] Serialized 26 byte log entry
0709 12:14:25.799751 (+784028us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:14:25.819123 (+ 19372us) log.cc:844] Serialized 14 byte log entry
0709 12:14:25.819401 (+   278us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.819502 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.020745 (+201243us) raft_consensus.cc:1685] finished
0709 12:14:26.020775 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.020943 (+   168us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":44626,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":3297,"raft.queue_time_us":16559,"raft.run_cpu_time_us":275,"raft.run_wall_time_us":15613,"thread_start_us":696506,"threads_started":1,"wal-append.queue_time_us":970345}
I20260709 12:14:26.431675 12603 rpcz_store.cc:276] 0709 12:14:22.253394 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.142865 (+889471us) service_pool.cc:224] Handling call
0709 12:14:23.185604 (+ 42739us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:14:24.653253 (+1467649us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:24.653524 (+   271us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:14:24.686732 (+ 33208us) log.cc:844] Serialized 78 byte log entry
0709 12:14:25.204063 (+517331us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:25.326090 (+122027us) log.cc:844] Serialized 14 byte log entry
0709 12:14:25.741166 (+415076us) log.cc:844] Serialized 14 byte log entry
0709 12:14:25.741653 (+   487us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.741783 (+   130us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.112010 (+370227us) spinlock_profiling.cc:243] Waited 11.5 ms on lock 0x7b480024c258. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:14:26.319378 (+207368us) raft_consensus.cc:1685] finished
0709 12:14:26.319402 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.319574 (+   172us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":23947,"prepare.queue_time_us":153284,"prepare.run_cpu_time_us":115,"prepare.run_wall_time_us":4929,"raft.queue_time_us":50245,"raft.run_cpu_time_us":4640,"raft.run_wall_time_us":794885,"spinlock_wait_cycles":26446208,"thread_start_us":1817692,"threads_started":2,"wal-append.queue_time_us":1393798}
I20260709 12:14:26.436776 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:26.437599 12603 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:26.438791 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:26.439477 12603 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 30.
I20260709 12:14:26.440508 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:26.441193 12603 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:14:26.442291 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:26.443194 12603 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 27: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:14:26.286679 22993 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.297s	user 0.000s	sys 0.001s
I20260709 12:14:26.452536 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:26.311733 22980 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.444s	user 0.001s	sys 0.002s
I20260709 12:14:26.448343 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:26.006810 22950 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.093s	user 0.000s	sys 0.000s
I20260709 12:14:26.448565 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:26.449656 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:26.450603 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:26.457417 12571 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:26.461289 12599 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29.
I20260709 12:14:26.463061 12599 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 29. Current term is 30. Ops: [29.17-29.17]
I20260709 12:14:26.464766 12599 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 31. Current term is 32. Ops: []
W20260709 12:14:26.312983 22992 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.438s	user 0.000s	sys 0.002s
I20260709 12:14:26.455243 12523 raft_consensus.cc:1240] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 31. Current term is 32. Ops: [31.22-31.22]
I20260709 12:14:26.083958 22937 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.412204 13338 rpcz_store.cc:276] 0709 12:14:24.900314 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.900481 (+   167us) service_pool.cc:224] Handling call
0709 12:14:24.942891 (+ 42410us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:24.943172 (+   281us) raft_consensus.cc:1534] Early marking committed up to index 22
0709 12:14:25.778281 (+835109us) log.cc:844] Serialized 14 byte log entry
0709 12:14:25.999767 (+221486us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:25.999842 (+    75us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:14:25.999915 (+    73us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.000004 (+    89us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.000188 (+   184us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":246406,"prepare.run_cpu_time_us":38,"prepare.run_wall_time_us":38,"raft.queue_time_us":124,"raft.run_cpu_time_us":134,"raft.run_wall_time_us":17823,"thread_start_us":1017702,"threads_started":2}
I20260709 12:14:26.309958 22960 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:26.257414 22962 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.386s	user 0.001s	sys 0.003s
W20260709 12:14:26.020541 22865 consensus_meta.cc:287] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.784s	user 0.013s	sys 0.002s
I20260709 12:14:25.988495 22698 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:26.025873 22851 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:26.509265 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 60785.4us
I20260709 12:14:26.470815 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:26.024757 22656 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:26.255198 23014 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:26.175242 22852 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:26.089898 22856 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:14:26.331389 22933 consensus_meta.cc:287] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.533s	user 0.010s	sys 0.000s
I20260709 12:14:26.479795 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:14:26.480193 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.479786 12604 raft_consensus.cc:2410] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 30 because replica has last-logged OpId of term: 29 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 19.
I20260709 12:14:26.031296 22934 raft_consensus.cc:3055] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:14:26.227088 22990 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:26.247227 22999 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:14:26.410380 22674 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.286s	user 0.011s	sys 0.029s
I20260709 12:14:26.410365 22930 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:26.455726 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:26.625205 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:26.496733 23005 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.612s	user 0.007s	sys 0.000s
I20260709 12:14:26.022341 22967 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.20, Last appended by leader: 20, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:26.585651 22993 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.108s	user 0.001s	sys 0.000s
I20260709 12:14:26.587026 22921 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.504119 12673 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 26 index: 18. Preceding OpId from leader: term: 27 index: 18. (term mismatch)
I20260709 12:14:26.639766 12673 pending_rounds.cc:85] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 17
I20260709 12:14:26.640128 12673 pending_rounds.cc:107] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 26.18
I20260709 12:14:26.453517 22714 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:14:25.913081 22847 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:26.513772 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:26.646168 12677 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:25.921995 22982 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.547s
I20260709 12:14:26.459897 22625 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:26.349735 12527 rpcz_store.cc:276] 0709 12:14:24.780208 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.780582 (+   374us) service_pool.cc:224] Handling call
0709 12:14:24.780880 (+   298us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:24.781092 (+   212us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:14:24.829255 (+ 48163us) log.cc:844] Serialized 14 byte log entry
0709 12:14:25.843798 (+1014543us) spinlock_profiling.cc:243] Waited 3.57 ms on lock 0x7b1800218560. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a 00007f1488a2aa2a
0709 12:14:26.208152 (+364354us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:26.208238 (+    86us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:14:26.208312 (+    74us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.208438 (+   126us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.208623 (+   185us) inbound_call.cc:177] Queueing success response
0709 12:14:26.316529 (+107906us) spinlock_profiling.cc:243] Waited 18.5 ms on lock 0x7b4800002958. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
Metrics: {"prepare.queue_time_us":47996,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":4010,"raft.queue_time_us":47304,"raft.run_cpu_time_us":1461,"raft.run_wall_time_us":61283,"spinlock_wait_cycles":50857728,"thread_start_us":364021,"threads_started":1}
I20260709 12:14:26.569275 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:14:26.657678 13305 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.528s	user 0.023s	sys 0.052s
W20260709 12:14:26.658145 13305 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.529s	user 0.023s	sys 0.052s
I20260709 12:14:26.575344 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:26.577507 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:26.578692 12675 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 30 index: 17. (term mismatch)
W20260709 12:14:26.493546 22941 consensus_meta.cc:287] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.632s	user 0.016s	sys 0.001s
I20260709 12:14:26.008416 22823 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:26.142947 22973 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 34.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.369s
I20260709 12:14:26.583603 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1168) took 3325 ms. Trace:
W20260709 12:14:26.620234 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 45639.8us
I20260709 12:14:26.506909 22927 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:25.913972 22949 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:26.448421 22902 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:26.458451 22952 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:25.931310 22857 consensus_queue.cc:237] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 29.22, Last appended by leader: 22, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:26.568351 22880 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:26.665767 22714 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.640412 13338 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26. Ops: [25.16-25.16]
I20260709 12:14:26.640503 12673 raft_consensus.cc:2889] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:26.630662 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:26.614975 22981 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:26.649554 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:26.617292 22832 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.617780 22934 raft_consensus.cc:740] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:26.622936 22592 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:14:26.488094 22995 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.592s	user 0.000s	sys 0.007s
I20260709 12:14:26.505966 22960 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:26.656911 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:26.744920 12677 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:14:26.661607 12675 pending_rounds.cc:85] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 16
I20260709 12:14:26.746266 12675 pending_rounds.cc:107] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 29.17
I20260709 12:14:26.746685 12675 raft_consensus.cc:2889] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:26.661517 12678 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
W20260709 12:14:26.662619 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.506567 22937 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:26.661590 12671 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:14:26.663473 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3948) took 2886 ms. Trace:
I20260709 12:14:26.667725 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4385) took 4415 ms. Trace:
W20260709 12:14:26.652994 22829 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.776s	user 0.014s	sys 0.000s
W20260709 12:14:26.654456 22866 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.789s	user 0.027s	sys 0.054s
I20260709 12:14:26.643324 22933 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:14:26.653882 22681 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s	user 0.024s	sys 0.054s
I20260709 12:14:26.614292 22852 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:26.702039 12602 rpcz_store.cc:276] 0709 12:14:23.257718 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:23.595942 (+338224us) service_pool.cc:224] Handling call
0709 12:14:23.596457 (+   515us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:23.617297 (+ 20840us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:23.617341 (+    44us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:23.618173 (+   832us) log.cc:844] Serialized 26 byte log entry
0709 12:14:24.233266 (+615093us) raft_consensus.cc:1645] Marking committed up to 23
0709 12:14:24.740609 (+507343us) log.cc:844] Serialized 14 byte log entry
0709 12:14:24.850519 (+109910us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:24.850630 (+   111us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:25.835554 (+984924us) spinlock_profiling.cc:243] Waited 1.38 ms on lock 0x7b18002a4960. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:14:26.568620 (+733066us) raft_consensus.cc:1685] finished
0709 12:14:26.583278 (+ 14658us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.583512 (+   234us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":87501,"prepare.queue_time_us":73812,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":174725,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":21524,"spinlock_wait_cycles":3164544,"thread_start_us":1005981,"threads_started":2,"wal-append.queue_time_us":1106687}
I20260709 12:14:26.655045 22674 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:26.656981 22698 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:26.655370 22932 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:26.623208 22856 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:26.655158 23007 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:26.026219 22953 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 29.16, Last appended by leader: 16, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:26.638078 22930 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:26.718909 22993 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.064s	user 0.001s	sys 0.000s
I20260709 12:14:26.702358 22860 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:26.623818 22990 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:26.803591 22674 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:26.650724 22921 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:26.670881 22935 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:26.721419 22952 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:26.783521 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:26.784997 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:26.638496 22701 consensus_queue.cc:260] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 30.17, Last appended by leader: 17, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:26.785317 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:26.870604 12678 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:26.750056 12674 rpcz_store.cc:276] 0709 12:14:23.776630 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.376165 (+599535us) service_pool.cc:224] Handling call
0709 12:14:24.405617 (+ 29452us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:24.438636 (+ 33019us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:24.438683 (+    47us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:24.439298 (+   615us) log.cc:844] Serialized 26 byte log entry
0709 12:14:24.941842 (+502544us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:25.866948 (+925106us) spinlock_profiling.cc:243] Waited 37.2 ms on lock 0x7b180003bf80. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:14:26.259741 (+392793us) spinlock_profiling.cc:243] Waited 5.28 ms on lock 0x7b180003bf80. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f148c4183c7 00007f1488a2a9a7 00007f1488a3a0bf
0709 12:14:26.259963 (+   222us) log.cc:844] Serialized 14 byte log entry
0709 12:14:26.260308 (+   345us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.260414 (+   106us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.310649 (+ 50235us) spinlock_profiling.cc:243] Waited 43.4 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:14:26.663182 (+352533us) raft_consensus.cc:1685] finished
0709 12:14:26.663209 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.663394 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":14777,"prepare.queue_time_us":363881,"prepare.run_cpu_time_us":55,"prepare.run_wall_time_us":56,"raft.queue_time_us":28599,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":6712,"spinlock_wait_cycles":197494144,"thread_start_us":1356535,"threads_started":2,"wal-append.queue_time_us":769968}
I20260709 12:14:26.872205 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:26.750309 12601 rpcz_store.cc:276] 0709 12:14:22.252798 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:22.958616 (+705818us) service_pool.cc:224] Handling call
0709 12:14:22.981525 (+ 22909us) raft_consensus.cc:1471] Updating replica for 4 ops
0709 12:14:23.670757 (+689232us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:14:23.717037 (+ 46280us) log.cc:844] Serialized 14 byte log entry
0709 12:14:24.502768 (+785731us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:14:24.511547 (+  8779us) log.cc:844] Serialized 78 byte log entry
0709 12:14:24.511774 (+   227us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:14:25.731511 (+1219737us) log.cc:844] Serialized 14 byte log entry
0709 12:14:25.773922 (+ 42411us) log.cc:844] Serialized 14 byte log entry
0709 12:14:25.860002 (+ 86080us) spinlock_profiling.cc:243] Waited 5.29 ms on lock 0x7b1800265a20. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:14:26.274130 (+414128us) spinlock_profiling.cc:243] Waited 2.65 ms on lock 0x7b1800265a20. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:14:26.667281 (+393151us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.667386 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.667442 (+    56us) raft_consensus.cc:1685] finished
0709 12:14:26.667459 (+    17us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.667644 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":81501,"prepare.queue_time_us":676388,"prepare.run_cpu_time_us":143,"prepare.run_wall_time_us":131,"raft.queue_time_us":187843,"raft.run_cpu_time_us":584,"raft.run_wall_time_us":117261,"spinlock_wait_cycles":18251904,"thread_start_us":2774458,"threads_started":4,"wal-append.queue_time_us":1017496}
I20260709 12:14:26.793013 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:26.733423 23020 raft_consensus.cc:3055] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Stepping down as leader of term 24
W20260709 12:14:26.782831 22997 log.cc:927] Time spent T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.695s	user 0.004s	sys 0.003s
W20260709 12:14:26.789423 22866 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.924s	user 0.027s	sys 0.054s
I20260709 12:14:26.802019 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:26.805671 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:26.794025 22681 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.761s	user 0.025s	sys 0.055s
I20260709 12:14:26.741187 23016 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:26.791553 22980 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.056s	user 0.000s	sys 0.001s
I20260709 12:14:26.721522 22927 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:26.811491 23021 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.351s
I20260709 12:14:26.815824 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:26.793793 22933 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:26.798237 22981 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:26.820153 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 250150us
I20260709 12:14:26.918022 12601 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 25 index: 13. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:14:26.824426 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:26.827677 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.735610 22714 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:26.888971 23016 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:26.616906 23014 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:26.743177 22832 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:14:26.863970 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.823632 22998 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:26.747054 22592 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:26.795061 22829 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:26.868440 12677 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:14:27.014959 12677 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 24 index: 16. (index mismatch)
I20260709 12:14:26.868238 12673 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
W20260709 12:14:26.788470 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.873445 13337 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
W20260709 12:14:27.016352 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.874097 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:26.875844 12528 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 31.
I20260709 12:14:26.829444 22865 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:26.876626 12671 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
W20260709 12:14:26.881026 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 466919us
I20260709 12:14:26.833212 22860 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:26.883872 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:26.840983 22674 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:14:26.812517 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 355209us
I20260709 12:14:26.889377 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3818) took 2104 ms. Trace:
I20260709 12:14:26.822988 22932 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:26.870496 22952 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:26.875239 23020 raft_consensus.cc:740] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:26.929409 12497 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.691s	user 0.018s	sys 0.011s
W20260709 12:14:27.032325 12497 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.906s	user 0.019s	sys 0.012s
I20260709 12:14:27.025928 12526 rpcz_store.cc:276] 0709 12:14:24.776676 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.776803 (+   127us) service_pool.cc:224] Handling call
0709 12:14:24.777126 (+   323us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:24.831208 (+ 54082us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:24.831245 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:24.871289 (+ 40044us) log.cc:844] Serialized 26 byte log entry
0709 12:14:25.042035 (+170746us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:14:25.042077 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.042165 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.321759 (+1279594us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:14:26.871853 (+550094us) raft_consensus.cc:1685] finished
0709 12:14:26.871875 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.881042 (+  9167us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":138,"raft.queue_time_us":242,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":23333,"spinlock_wait_cycles":30028544,"thread_start_us":148713,"threads_started":1,"wal-append.queue_time_us":993145}
I20260709 12:14:26.894757 23018 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:26.969384 22701 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:26.970386 22902 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:26.971362 23016 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:26.974191 23023 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.361s	user 0.002s	sys 0.000s
I20260709 12:14:27.029174 22860 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:26.975598 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:26.964860 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:26.967289 22930 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:26.968184 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49728.2us
I20260709 12:14:26.977188 12676 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:26.980495 12527 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:26.966600 13339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 1167) took 2134 ms. Trace:
I20260709 12:14:27.059212 13339 rpcz_store.cc:276] 0709 12:14:24.831697 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.831826 (+   129us) service_pool.cc:224] Handling call
0709 12:14:24.881217 (+ 49391us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:24.881604 (+   387us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:24.881637 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:24.882160 (+   523us) log.cc:844] Serialized 26 byte log entry
0709 12:14:25.273870 (+391710us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:14:25.273954 (+    84us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:25.274073 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:26.322314 (+1048241us) spinlock_profiling.cc:243] Waited 13.4 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:14:26.966308 (+643994us) raft_consensus.cc:1685] finished
0709 12:14:26.966331 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:26.966533 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":26572,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":124,"spinlock_wait_cycles":30830464,"thread_start_us":372258,"threads_started":1,"wal-append.queue_time_us":984335}
I20260709 12:14:26.938961 23044 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.032s
W20260709 12:14:26.982924 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:26.985460 12678 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
W20260709 12:14:27.063827 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:27.064257 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:27.066401 23015 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.320s	user 0.000s	sys 0.001s
I20260709 12:14:26.968600 22856 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:26.993844 13333 raft_consensus.cc:2410] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 33 because replica has last-logged OpId of term: 31 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 21.
I20260709 12:14:26.993718 12523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:27.015458 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:27.069476 12601 raft_consensus.cc:2410] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 33 because replica has last-logged OpId of term: 31 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 21.
I20260709 12:14:27.071091 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:27.072595 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:27.073751 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.074345 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:27.032356 22656 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.026168 12571 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:27.017805 22998 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:27.027850 22865 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:27.034793 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:27.039043 12672 raft_consensus.cc:2393] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 25: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:27.042207 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:27.127986 12673 raft_consensus.cc:2468] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
W20260709 12:14:27.035794 23022 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.313s	user 0.000s	sys 0.003s
I20260709 12:14:26.975086 22823 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:27.053252 12522 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 27. Current term is 28. Ops: [27.16-27.16]
I20260709 12:14:27.054518 12529 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 25 index: 12. Preceding OpId from leader: term: 26 index: 13. (index mismatch)
I20260709 12:14:27.043669 22853 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:26.833848 23007 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:26.901767 22933 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:27.024665 12674 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:14:27.142113 12524 raft_consensus.cc:1240] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 25. Current term is 26. Ops: [25.16-25.16]
I20260709 12:14:27.090764 12494 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:27.092728 23033 log.cc:927] Time spent T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
W20260709 12:14:27.103432 23028 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.283s	user 0.000s	sys 0.001s
W20260709 12:14:27.103848 23024 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.259s	user 0.001s	sys 0.000s
I20260709 12:14:27.158010 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:27.107816 12676 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3963) took 3054 ms (client timeout 2216 ms). Trace:
I20260709 12:14:27.115171 22852 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:27.116452 22960 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:26.988013 22981 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:27.123556 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:27.130450 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4416) took 1364 ms. Trace:
I20260709 12:14:26.842378 22934 consensus_queue.cc:260] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 27.13, Last appended by leader: 13, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:27.133106 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 81877us
W20260709 12:14:27.133257 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 107211us
I20260709 12:14:27.142387 23044 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:14:27.179790 12571 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:27.180874 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:27.130376 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 51261.3us
I20260709 12:14:27.150880 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:27.117916 22921 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:27.181907 12524 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 30.
W20260709 12:14:27.160918 12676 rpcz_store.cc:269] 0709 12:14:24.060843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:24.131635 (+ 70792us) service_pool.cc:224] Handling call
0709 12:14:27.107466 (+2975831us) spinlock_profiling.cc:243] Waited 2.79 s on lock 0x7b54000c3fd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:27.107674 (+   208us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":6414962432}
I20260709 12:14:27.130592 22832 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:27.172610 12605 rpcz_store.cc:276] 0709 12:14:25.766376 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:25.767330 (+   954us) service_pool.cc:224] Handling call
0709 12:14:25.767772 (+   442us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:25.768131 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:25.768169 (+    38us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:25.778105 (+  9936us) log.cc:844] Serialized 26 byte log entry
0709 12:14:26.242907 (+464802us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:26.242961 (+    54us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:26.243071 (+   110us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:27.130208 (+887137us) raft_consensus.cc:1685] finished
0709 12:14:27.130232 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:27.130384 (+   152us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":436379,"threads_started":1,"wal-append.queue_time_us":660960}
I20260709 12:14:27.189728 23007 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:27.190260 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:27.191495 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:27.192888 12604 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:14:27.193189 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:27.193514 12604 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:27.176192 12531 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:14:27.194856 22934 raft_consensus.cc:3060] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:27.195132 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:27.196441 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:27.181078 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:27.199303 22935 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:27.199734 22935 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:27.199813 12571 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:27.134862 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:27.094412 22856 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.072s
I20260709 12:14:27.200852 22829 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:27.141847 22714 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:27.162175 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 71566.4us
I20260709 12:14:27.200927 12601 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:14:27.185359 12529 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 28: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:27.200348 22935 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:27.203584 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:27.184031 23044 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:27.204411 12601 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:14:27.202965 23014 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:27.206794 23016 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:14:27.207790 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.169709 22823 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:27.205817 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:27.210512 22823 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.211773 22847 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:27.212240 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:27.213488 12601 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
I20260709 12:14:27.046948 23018 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:27.223618 12571 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:27.213954 12605 raft_consensus.cc:2410] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 24 because replica has last-logged OpId of term: 23 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 15.
I20260709 12:14:27.214232 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:27.203768 22932 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:27.199520 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:27.223915 12604 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 34.22, Last appended by leader: 22, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:27.204779 23044 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:27.221949 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:27.165517 22887 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.125926 22656 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:27.212940 22823 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:27.232918 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:27.272679 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:27.236120 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:27.237071 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:27.237865 12522 raft_consensus.cc:2410] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 29 because replica has last-logged OpId of term: 28 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 18.
I20260709 12:14:27.239809 12571 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:27.229820 22829 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.011s
I20260709 12:14:27.236006 23016 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:27.241386 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
W20260709 12:14:27.287015 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.275243 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:27.287748 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.287882 12529 raft_consensus.cc:2410] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 33 because replica has last-logged OpId of term: 32 index: 22, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 21.
I20260709 12:14:27.245134 22851 raft_consensus.cc:3055] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:14:27.254081 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:27.254928 12672 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
W20260709 12:14:27.255446 23034 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.297s	user 0.002s	sys 0.000s
I20260709 12:14:27.293290 12601 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:14:27.256142 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:27.223138 22930 consensus_queue.cc:1048] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.035s
I20260709 12:14:27.237543 23037 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.033s
I20260709 12:14:27.263144 22865 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:27.264869 23032 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.139s	user 0.000s	sys 0.001s
I20260709 12:14:27.215657 22990 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:27.226547 22949 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:27.266258 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:27.236703 23038 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:14:27.270332 23007 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:14:27.242058 22998 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:27.303725 22921 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:27.291381 12531 raft_consensus.cc:2410] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 30 because replica has last-logged OpId of term: 29 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 16.
I20260709 12:14:27.293103 12530 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:27.296877 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:27.299629 12527 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:27.302654 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:27.360440 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 57879us
I20260709 12:14:27.361239 12643 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:27.217172 22721 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:27.230432 22847 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:27.269093 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:27.332337 13335 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 21 index: 16. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
I20260709 12:14:27.332571 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:27.333662 12604 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:14:27.317355 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:27.318195 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:27.365149 13338 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 28: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:14:27.311978 22927 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:27.337708 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:27.292450 22851 raft_consensus.cc:740] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:27.313797 22853 consensus_queue.cc:1059] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
I20260709 12:14:27.342700 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:27.343166 23044 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:27.343940 12496 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:27.345229 22934 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:27.261845 22933 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:27.295537 23014 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:27.347455 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:27.347573 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:27.350925 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.397816 12673 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:14:27.289750 12572 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.364645 12672 raft_consensus.cc:2410] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 27 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 12:14:27.304081 22857 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.061s
I20260709 12:14:27.151733 23020 consensus_queue.cc:260] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:27.368932 12599 raft_consensus.cc:1275] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 27 index: 13. Preceding OpId from leader: term: 26 index: 13. (term mismatch)
I20260709 12:14:27.358219 22865 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:27.376004 12531 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:27.368436 22960 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
W20260709 12:14:27.380870 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 141136us
I20260709 12:14:27.365345 22847 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:27.388273 12496 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:27.417613 12496 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:27.300881 22856 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:27.394327 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:27.418905 12675 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 32.
I20260709 12:14:27.396916 12679 raft_consensus.cc:2410] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 31 because replica has last-logged OpId of term: 30 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 16.
I20260709 12:14:27.332571 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:27.287812 22887 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:27.420683 12601 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:14:27.359098 22921 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.408296 12599 pending_rounds.cc:85] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 12
I20260709 12:14:27.425127 12599 pending_rounds.cc:107] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 27.13
I20260709 12:14:27.425431 12599 raft_consensus.cc:2889] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:27.372231 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:27.426414 12677 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 32: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:27.409060 12674 raft_consensus.cc:2393] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 32: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:27.318650 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:27.401564 22714 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:27.400005 23014 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:27.424636 22856 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:27.450649 22856 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:27.435045 22973 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.451407 22973 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:27.451685 22887 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:27.455061 22933 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:27.452442 22902 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:27.432256 22979 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:27.420668 22847 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:27.434082 22921 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:27.467794 12529 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:27.408071 22832 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.036s
I20260709 12:14:27.473054 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:27.473978 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 185361us
I20260709 12:14:27.479947 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:27.470834 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3829) took 1444 ms. Trace:
I20260709 12:14:27.483958 12601 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 32.
I20260709 12:14:27.434873 22714 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:27.484336 12604 rpcz_store.cc:276] 0709 12:14:26.026028 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.618642 (+592614us) service_pool.cc:224] Handling call
0709 12:14:26.619077 (+   435us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:27.192742 (+573665us) spinlock_profiling.cc:243] Waited 120 ms on lock 0x7b54003b6da0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488997dc0 00007f1488a221c0 00007f1488a23231 00007f1488a1fd8f 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c
0709 12:14:27.469784 (+277042us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:27.469868 (+    84us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:27.470054 (+   186us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:14:27.470196 (+   142us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:27.470368 (+   172us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:27.470557 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":5754,"raft.queue_time_us":82726,"spinlock_wait_cycles":275623680,"thread_start_us":70301,"threads_started":1}
I20260709 12:14:27.493407 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:27.489781 12673 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:27.492043 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:27.495625 12496 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:27.496110 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:27.448624 23014 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:27.497967 12604 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:27.421455 22865 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:27.466208 23056 log.cc:927] Time spent T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:14:27.491521 22832 consensus_queue.cc:1243] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 23.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
W20260709 12:14:27.483251 22941 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 35 current term: 35 is higher.
I20260709 12:14:27.519244 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:27.520185 12601 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 33.
W20260709 12:14:27.484287 23051 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.141s	user 0.001s	sys 0.000s
I20260709 12:14:27.506033 22714 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:27.531191 22999 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:14:27.531618 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:27.532579 13335 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 31.
I20260709 12:14:27.532781 22934 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
W20260709 12:14:27.526993 23060 log.cc:927] Time spent T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.102s	user 0.000s	sys 0.000s
I20260709 12:14:27.500607 22935 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:27.528945 22973 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:27.462550 22839 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 13, Time since last communication: 0.047s
I20260709 12:14:27.477281 22979 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:27.472345 22902 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:27.535871 12569 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:27.509395 22990 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:27.550434 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:27.541090 12528 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:53862 (request call id 3826) took 1124 ms. Trace:
W20260709 12:14:27.543233 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 143498us
W20260709 12:14:27.546186 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 112093us
I20260709 12:14:27.523697 23040 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:27.537968 22656 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:27.492648 22952 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:27.547858 22839 consensus_queue.cc:1243] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 27.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:14:27.510978 22681 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.038s
I20260709 12:14:27.553690 12528 rpcz_store.cc:276] 0709 12:14:26.411582 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.412104 (+   522us) service_pool.cc:224] Handling call
0709 12:14:27.535779 (+1123675us) spinlock_profiling.cc:243] Waited 463 ms on lock 0x7b54001072d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:27.535932 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1063758080}
I20260709 12:14:27.547673 22979 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:27.563505 12604 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:27.549535 22990 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:27.582715 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:27.560341 22973 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:14:27.557523 23040 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:27.584692 12526 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:27.550598 22902 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:27.570253 23018 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:27.598096 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 102542us
I20260709 12:14:27.560243 22934 consensus_queue.cc:1059] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 29.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.027s
W20260709 12:14:27.601473 22674 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.564s	user 0.011s	sys 0.000s
I20260709 12:14:27.588959 23040 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:27.584645 22990 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:27.585521 22973 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.608004 12601 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 23 index: 16. Preceding OpId from leader: term: 24 index: 16. (term mismatch)
I20260709 12:14:27.613384 12601 pending_rounds.cc:85] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 15
I20260709 12:14:27.613636 12601 pending_rounds.cc:107] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 23.16
I20260709 12:14:27.613871 12601 raft_consensus.cc:2889] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:27.609016 23047 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.121s	user 0.001s	sys 0.000s
I20260709 12:14:27.616024 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:27.589762 23070 consensus_queue.cc:1048] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.030s
I20260709 12:14:27.619838 23071 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.006s
W20260709 12:14:27.621960 23062 log.cc:927] Time spent T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.112s	user 0.000s	sys 0.001s
I20260709 12:14:27.622833 22865 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:27.623756 12572 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:27.629318 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 24 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:27.664458 12597 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 24.
W20260709 12:14:27.665020 12597 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3816) took 1642 ms (client timeout 1643 ms). Trace:
W20260709 12:14:27.665431 12597 rpcz_store.cc:269] 0709 12:14:26.022169 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.027214 (+  5045us) service_pool.cc:224] Handling call
0709 12:14:27.629113 (+1601899us) spinlock_profiling.cc:243] Waited 15 ms on lock 0x7b5800156c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9e60a0 00007f148d9e5984 00007f1486f606d4 00007f1486f60662
0709 12:14:27.664916 (+ 35803us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":34393344}
I20260709 12:14:27.599970 22857 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.018s
I20260709 12:14:27.630894 13330 raft_consensus.cc:1275] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 28 index: 21. Preceding OpId from leader: term: 31 index: 23. (index mismatch)
I20260709 12:14:27.631325 13332 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38482 (request call id 922) took 1178 ms. Trace:
I20260709 12:14:27.612236 23063 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.026s
W20260709 12:14:27.635629 23069 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.106s	user 0.001s	sys 0.000s
W20260709 12:14:27.636893 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:27.638053 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:27.672187 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:27.673460 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:27.643051 12522 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:27.640467 22921 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:27.676864 12597 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: [28.16-28.16]
W20260709 12:14:27.655998 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 67886.8us
I20260709 12:14:27.656400 12496 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:27.606473 23044 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:27.660789 12528 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:27.663664 12524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" is_pre_election: true
I20260709 12:14:27.688006 12524 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
I20260709 12:14:27.687927 12528 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 34.
W20260709 12:14:27.688658 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 64984.9us
W20260709 12:14:27.687939 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 76107.5us
I20260709 12:14:27.615233 22973 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:14:27.693329 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 286669us
I20260709 12:14:27.668192 13332 rpcz_store.cc:276] 0709 12:14:26.452561 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.467928 (+ 15367us) service_pool.cc:224] Handling call
0709 12:14:26.481317 (+ 13389us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:26.643692 (+162375us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:26.643728 (+    36us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:26.643771 (+    43us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:14:26.643813 (+    42us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:27.631042 (+987229us) spinlock_profiling.cc:243] Waited 162 ms on lock 0x7b540061e9d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:27.631107 (+    65us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:27.631253 (+   146us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":372774656}
I20260709 12:14:27.673750 12601 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 15. (term mismatch)
I20260709 12:14:27.694852 12601 pending_rounds.cc:85] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 14
I20260709 12:14:27.695139 12601 pending_rounds.cc:107] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 25.15
I20260709 12:14:27.659113 23075 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:14:27.643066 22952 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:27.610507 22674 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:27.695415 12601 raft_consensus.cc:2889] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:27.689752 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:27.701388 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:27.702955 12494 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:27.705389 12496 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:27.702391 13305 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:27.687023 22853 consensus_queue.cc:1243] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 32.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 15
I20260709 12:14:27.693058 22852 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:27.686803 22701 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:27.686380 22681 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:27.717401 22935 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:27.718652 22592 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:27.696367 23038 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.724650 13339 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 26. Current term is 27. Ops: []
I20260709 12:14:27.725934 12646 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:27.731839 22847 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:27.733348 22921 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:27.711621 22832 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:27.734298 22832 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:27.734830 22847 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:27.735666 22860 consensus_meta.cc:287] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.603s	user 0.010s	sys 0.002s
I20260709 12:14:27.736413 22860 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:27.738080 12598 raft_consensus.cc:2368] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26.
I20260709 12:14:27.739946 22823 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:27.744626 22625 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 1.078s	user 0.011s	sys 0.003s
I20260709 12:14:27.711225 22656 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.037s
I20260709 12:14:27.745577 12675 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 16. (term mismatch)
I20260709 12:14:27.745988 12675 pending_rounds.cc:85] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 15
I20260709 12:14:27.746255 12675 pending_rounds.cc:107] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 27.16
I20260709 12:14:27.724042 22952 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:27.746495 12675 raft_consensus.cc:2889] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:27.709959 23072 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:14:27.746774 22952 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:27.747010 23072 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.747403 23010 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:27.747939 23038 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:27.748138 23010 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:27.750358 23078 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:27.750751 23038 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:27.751075 23078 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:27.751686 23078 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:27.734808 22832 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:27.725255 22829 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:14:27.711117 22880 raft_consensus.cc:2804] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:27.731591 22856 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:27.747315 22952 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Advancing to term 32
/apache/dev/git/kudu/src/kudu/master/auto_leader_rebalancer-test.cc:461: Failure
Failed
Bad status: Illegal state: leader_transfer_task size should be 0, but 6
I20260709 12:14:27.747483 23072 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:27.763859 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 4654) took 1293 ms. Trace:
I20260709 12:14:27.775794 12679 rpcz_store.cc:276] 0709 12:14:26.470717 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.487968 (+ 17251us) service_pool.cc:224] Handling call
0709 12:14:27.763631 (+1275663us) spinlock_profiling.cc:243] Waited 739 ms on lock 0x7b54000c2bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:27.763794 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1698663680}
W20260709 12:14:27.772464 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 46605.4us
I20260709 12:14:27.791610 12646 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:14:27.797242 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 73283.2us
I20260709 12:14:27.798498 12571 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:27.767807 11318 tablet_server.cc:179] TabletServer@127.11.13.129:0 shutting down...
I20260709 12:14:27.726562 22998 consensus_queue.cc:1048] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 21, Time since last communication: 0.072s
I20260709 12:14:27.758384 22829 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.721855 23014 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:27.797370 22701 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:27.761281 22856 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:27.762295 22852 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:27.758639 22880 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:27.768211 22823 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:27.837880 22656 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:27.754070 22902 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:27.865077 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 503871us
W20260709 12:14:27.840109 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 132953us
I20260709 12:14:27.874768 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:27.763875 22714 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:27.876434 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:27.861784 22829 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:14:27.862835 22701 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:27.949415 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 73131.9us
I20260709 12:14:27.950429 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:27.870000 22880 raft_consensus.cc:3060] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:27.955399 12643 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:27.877558 22656 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:27.960764 12606 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:14:27.952950 22701 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:14:27.956977 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 254122us
I20260709 12:14:27.961580 12643 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:27.881556 23038 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:14:27.953639 23069 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.083s	user 0.001s	sys 0.000s
I20260709 12:14:27.951792 22625 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:27.977396 12643 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 24 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:27.973811 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:14:27.974781 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 269435us
I20260709 12:14:27.958156 22856 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:27.958606 22860 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:27.963429 22656 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:27.883471 22990 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:27.985323 23088 log.cc:927] Time spent T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.115s	user 0.000s	sys 0.001s
I20260709 12:14:27.989717 12494 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:27.965328 23094 raft_consensus.cc:2764] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election decision vote started in defunct term 27: won
I20260709 12:14:27.978559 22865 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:14:27.994408 23038 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 24. Reason: could not achieve majority
I20260709 12:14:27.994640 23017 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:14:27.998466 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:27.992581 23091 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:14:28.011773 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.012883 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:28.012949 12606 raft_consensus.cc:2393] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 29: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:27.998979 22823 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:14:28.006289 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.013425 12673 raft_consensus.cc:2376] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 33. Re-sending same reply.
I20260709 12:14:28.014191 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:28.014245 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:28.014451 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:28.015937 12676 raft_consensus.cc:2410] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 26 because replica has last-logged OpId of term: 25 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 15.
I20260709 12:14:28.016556 12678 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:28.015844 12672 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 24.
I20260709 12:14:28.027050 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:28.030933 22714 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:28.032399 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.034075 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.034736 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4417) took 1206 ms. Trace:
I20260709 12:14:28.035533 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.036334 12606 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:14:28.037302 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.027081 23014 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:28.038415 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.039801 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:28.035462 12602 rpcz_store.cc:276] 0709 12:14:26.828288 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.828424 (+   136us) service_pool.cc:224] Handling call
0709 12:14:26.840088 (+ 11664us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:27.054656 (+214568us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:14:27.412972 (+358316us) log.cc:844] Serialized 14 byte log entry
0709 12:14:27.477834 (+ 64862us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:27.477910 (+    76us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:27.478030 (+   120us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.034284 (+556254us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b54001617d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:28.034354 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:28.034552 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":353479,"prepare.run_cpu_time_us":61,"prepare.run_wall_time_us":59,"raft.queue_time_us":44568,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":9159,"spinlock_wait_cycles":492633856,"thread_start_us":351297,"threads_started":2,"wal-append.queue_time_us":177114}
I20260709 12:14:28.001925 22865 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:28.015655 22674 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:28.059903 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.057946 12643 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:28.062173 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:28.011672 23091 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:14:28.065583 23082 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.092s	user 0.000s	sys 0.001s
I20260709 12:14:27.988802 22852 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:28.034101 22952 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:28.054247 22865 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:28.069754 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:28.069536 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:28.082154 12673 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:28.078459 12678 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 28.
I20260709 12:14:28.086386 13304 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:28.079630 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 61627.5us
I20260709 12:14:28.089211 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:14:28.101679 13305 consensus_peers.cc:597] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:28.074352 23016 consensus_meta.cc:287] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.770s	user 0.007s	sys 0.008s
I20260709 12:14:28.068005 23091 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:28.126374 12673 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 27.
W20260709 12:14:28.127586 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.129207 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.131603 12602 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 26: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:28.102569 22674 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:14:28.073055 22839 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.058s
I20260709 12:14:28.131037 23016 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:28.115998 22822 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:14:28.151446 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 131531us
I20260709 12:14:28.073343 22973 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:28.130543 22902 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:28.154832 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:28.159565 12602 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 29: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:28.159891 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:28.107916 23040 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:28.179076 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.204133 23084 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:14:28.204782 23084 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:28.205186 23084 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:28.211277 12531 raft_consensus.cc:1240] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 29. Current term is 30. Ops: []
I20260709 12:14:28.166742 22823 raft_consensus.cc:697] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:28.216630 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.196151 22990 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:28.223582 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 114985us
I20260709 12:14:28.163412 23042 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.096s
W20260709 12:14:28.238730 23103 log.cc:927] Time spent T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:14:28.238950 22952 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:28.243094 23042 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:14:28.243535 23042 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:28.214694 23010 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:28.246559 22701 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:28.250635 22823 consensus_queue.cc:237] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:28.261279 12606 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:14:28.267897 12496 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:28.271624 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:28.276003 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 43494us
I20260709 12:14:28.283171 12496 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:28.292344 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.293291 12602 raft_consensus.cc:2410] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 33 because replica has last-logged OpId of term: 32 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 18.
W20260709 12:14:28.295573 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 135755us
I20260709 12:14:28.301316 13304 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:28.316159 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4674) took 1284 ms. Trace:
W20260709 12:14:28.320533 23072 consensus_meta.cc:287] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.507s	user 0.007s	sys 0.002s
I20260709 12:14:28.321007 23072 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:28.325040 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 526637us
I20260709 12:14:28.326197 12571 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:28.328049 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3828) took 2302 ms. Trace:
I20260709 12:14:28.328711 22932 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:28.328580 12599 rpcz_store.cc:276] 0709 12:14:26.025736 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.465729 (+439993us) service_pool.cc:224] Handling call
0709 12:14:26.466092 (+   363us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:27.447393 (+981301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.327731 (+880338us) spinlock_profiling.cc:243] Waited 882 ms on lock 0x7b54002ff654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:28.327973 (+   242us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2029632768}
I20260709 12:14:28.329932 22932 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:28.296543 22952 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:28.330320 22934 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.330482 22932 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:28.331182 22973 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:28.331774 22973 raft_consensus.cc:2764] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election decision vote started in defunct term 27: won
I20260709 12:14:28.331247 22934 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:28.332803 22932 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:28.334925 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:28.335565 23037 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:28.336109 23037 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:28.338006 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:28.338100 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:28.338402 22839 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:14:28.340109 22990 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:14:28.340632 22851 consensus_queue.cc:260] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 31.16, Last appended by leader: 16, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:14:28.343325 23040 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:28.343812 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:28.344599 12678 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 32.
I20260709 12:14:28.345564 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:28.346915 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 34 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:28.348230 12597 raft_consensus.cc:2468] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:28.349213 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.351339 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 24 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:28.353248 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:28.356272 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:28.356580 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.356736 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 30 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.329416 22829 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:28.295874 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:28.356312 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:28.282066 23105 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:28.244390 23042 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:28.341931 22851 raft_consensus.cc:3060] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:28.359711 12599 raft_consensus.cc:2393] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 33: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:28.349501 23091 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:28.372133 12678 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 17. Preceding OpId from leader: term: 25 index: 18. (index mismatch)
W20260709 12:14:28.395325 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 99544.8us
I20260709 12:14:28.396008 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:28.396733 12678 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:14:28.391211 12646 leader_election.cc:336] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:28.397382 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:28.397850 12646 leader_election.cc:336] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:28.399245 12646 consensus_peers.cc:597] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:28.392172 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:14:28.370342 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.393685 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.385072 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.316772 12671 rpcz_store.cc:276] 0709 12:14:27.031639 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:27.033808 (+  2169us) service_pool.cc:224] Handling call
0709 12:14:27.034144 (+   336us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:27.159978 (+125834us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:14:27.382845 (+222867us) log.cc:844] Serialized 14 byte log entry
0709 12:14:27.839360 (+456515us) log.cc:844] Serialized 14 byte log entry
0709 12:14:27.860710 (+ 21350us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:27.861203 (+   493us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:27.861325 (+   122us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.315806 (+454481us) spinlock_profiling.cc:243] Waited 126 ms on lock 0x7b54003bd154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:28.315876 (+    70us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:28.316074 (+   198us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":284548,"prepare.queue_time_us":84690,"prepare.run_cpu_time_us":102,"prepare.run_wall_time_us":5902,"raft.queue_time_us":285928,"raft.run_cpu_time_us":330,"raft.run_wall_time_us":15366,"spinlock_wait_cycles":288689536,"thread_start_us":346885,"threads_started":3,"wal-append.queue_time_us":146801}
I20260709 12:14:28.391639 12672 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:14:28.397214 12496 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:28.401963 12597 raft_consensus.cc:2376] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28. Re-sending same reply.
I20260709 12:14:28.404757 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:28.404991 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.406245 12597 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:28.406456 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.407112 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.407965 12597 raft_consensus.cc:2410] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 28 because replica has last-logged OpId of term: 27 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 14.
I20260709 12:14:28.362454 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 23 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:28.410454 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:28.411953 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:14:28.364687 22953 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.150s	user 0.009s	sys 0.018s
W20260709 12:14:28.410053 12646 leader_election.cc:336] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:28.431380 12677 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:14:28.429319 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.432118 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:28.432567 12599 raft_consensus.cc:2368] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 32. Current term is 33.
I20260709 12:14:28.433468 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 32 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:28.433758 13305 consensus_peers.cc:597] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:28.430850 12671 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:14:28.431988 12646 leader_election.cc:336] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:28.377342 22674 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 33 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:14:28.436569 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 135372us
W20260709 12:14:28.411134 23078 consensus_meta.cc:287] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.659s	user 0.010s	sys 0.004s
W20260709 12:14:28.439211 12646 consensus_peers.cc:597] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:28.441782 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:28.437476 12571 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:28.437871 12599 raft_consensus.cc:2368] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 32. Current term is 33.
I20260709 12:14:28.443553 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:28.443853 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:28.410946 22990 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:28.444459 12677 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:28.403208 23040 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:14:28.446208 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:28.446992 12571 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:28.448854 12571 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:28.418596 22832 consensus_meta.cc:287] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.664s	user 0.014s	sys 0.000s
W20260709 12:14:28.432937 22953 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.227s	user 0.010s	sys 0.020s
I20260709 12:14:28.453730 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:28.415547 23037 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:28.418584 23042 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:28.460361 12525 raft_consensus.cc:1240] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 30. Current term is 31. Ops: [30.17-30.17]
I20260709 12:14:28.437502 23038 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:28.440640 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.455134 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:28.437723 12672 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 29: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:28.403466 23084 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:28.431098 23108 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:28.460393 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.435177 23074 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:28.454861 12677 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 33.
I20260709 12:14:28.462949 12678 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
I20260709 12:14:28.451370 23040 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:28.464815 22865 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:28.455535 22832 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:28.454483 22934 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:28.469868 23107 raft_consensus.cc:2804] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:14:28.488169 22674 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:28.488812 23084 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:28.489213 22865 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:28.449856 23093 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.027s
I20260709 12:14:28.466799 22937 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:28.470351 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.471537 13339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 1172) took 1210 ms. Trace:
W20260709 12:14:28.471869 13305 consensus_peers.cc:597] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:28.472805 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:28.492214 13339 rpcz_store.cc:276] 0709 12:14:27.260748 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:27.294228 (+ 33480us) service_pool.cc:224] Handling call
0709 12:14:27.319075 (+ 24847us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:27.725209 (+406134us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.471291 (+746082us) spinlock_profiling.cc:243] Waited 405 ms on lock 0x7b5400690354. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:28.471476 (+   185us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":932177152}
W20260709 12:14:28.492602 13305 leader_election.cc:336] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:28.493858 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:28.473671 12646 leader_election.cc:336] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:28.451737 23078 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:28.451891 22857 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:28.475980 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:28.478386 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:28.453816 22721 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:28.483876 12671 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 33.
I20260709 12:14:28.455678 23016 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:14:28.498488 12646 consensus_peers.cc:597] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:28.474839 22979 consensus_meta.cc:287] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.889s	user 0.014s	sys 0.000s
I20260709 12:14:28.505810 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:28.505669 22979 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:28.506419 12677 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:28.507788 22979 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:28.490903 12673 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 25: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:28.418033 22880 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:28.509971 22880 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:28.473546 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.497620 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:28.499635 22934 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:28.511583 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:28.513002 12571 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:28.513676 22998 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:28.514027 12571 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:28.514357 22998 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:14:28.515432 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:28.515875 22998 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:28.516270 22998 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:28.513980 12677 raft_consensus.cc:2468] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:14:28.516708 22998 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:28.517354 23075 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:28.516511 12571 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:28.518033 23075 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:14:28.518308 23092 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:28.518421 23075 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.518879 23092 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.518996 23075 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:28.519309 23092 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:28.520515 23085 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:28.521099 23085 raft_consensus.cc:697] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:28.521944 23085 consensus_queue.cc:237] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:28.522509 22656 consensus_meta.cc:287] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.529s	user 0.011s	sys 0.001s
I20260709 12:14:28.489830 23074 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:28.491755 23108 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:28.525523 23108 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:28.493320 23107 raft_consensus.cc:697] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 27 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:28.488826 23112 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:28.528399 23112 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:28.530997 22981 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:28.534865 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3817) took 2512 ms. Trace:
I20260709 12:14:28.535945 23078 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:28.537299 22967 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:14:28.539511 23113 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.080s	user 0.002s	sys 0.000s
I20260709 12:14:28.542866 23065 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:14:28.535430 12603 rpcz_store.cc:276] 0709 12:14:26.022489 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.443829 (+421340us) service_pool.cc:224] Handling call
0709 12:14:26.444297 (+   468us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:26.731412 (+287115us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:26.731443 (+    31us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:26.737550 (+  6107us) log.cc:844] Serialized 52 byte log entry
0709 12:14:27.027202 (+289652us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:14:27.027246 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.534231 (+1506985us) spinlock_profiling.cc:243] Waited 258 ms on lock 0x7b54001167d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:28.534301 (+    70us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:28.534370 (+    69us) raft_consensus.cc:1685] finished
0709 12:14:28.534391 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:28.534774 (+   383us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":32035,"raft.queue_time_us":39645,"raft.run_cpu_time_us":4704,"raft.run_wall_time_us":47497,"spinlock_wait_cycles":594349312,"thread_start_us":213783,"threads_started":1,"wal-append.queue_time_us":419881}
I20260709 12:14:28.543490 23065 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:28.544520 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.544662 23065 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:28.545770 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:28.477309 23040 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:28.477445 22847 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:28.575671 12603 raft_consensus.cc:2393] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 27: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:28.502291 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.504065 12672 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:28.503878 23072 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:28.494508 12602 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 33.
W20260709 12:14:28.504604 12646 leader_election.cc:336] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:28.579483 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 74909.9us
I20260709 12:14:28.528189 23107 consensus_queue.cc:237] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:28.571276 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:28.573009 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:28.474942 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.599759 13335 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:28.592571 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:28.580261 12646 leader_election.cc:336] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:28.603705 22832 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:28.606215 22656 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:28.601863 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:28.610052 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:28.611173 12671 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:28.612869 23112 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:28.613714 22656 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:28.614043 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:28.615247 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:28.616571 23093 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.516470 23091 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:28.623279 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:14:28.629045 13305 leader_election.cc:336] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 35 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:28.629654 12603 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:28.634595 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:14:28.635236 23087 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.081s	user 0.002s	sys 0.000s
I20260709 12:14:28.635967 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:28.637732 23074 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:28.641593 23007 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:14:28.648504 12646 leader_election.cc:336] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:28.678388 12598 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51782 (request call id 4421) took 1795 ms (client timeout 2225 ms). Trace:
W20260709 12:14:28.678887 12598 rpcz_store.cc:269] 0709 12:14:26.883306 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:26.885571 (+  2265us) service_pool.cc:224] Handling call
0709 12:14:28.678097 (+1792526us) spinlock_profiling.cc:243] Waited 681 ms on lock 0x7b540023f854. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:28.678313 (+   216us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1565502336}
I20260709 12:14:28.680061 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4423) took 1784 ms. Trace:
I20260709 12:14:28.671232 23093 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:28.680630 12604 rpcz_store.cc:276] 0709 12:14:26.895995 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:27.575023 (+679028us) service_pool.cc:224] Handling call
0709 12:14:27.575450 (+   427us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:14:27.592405 (+ 16955us) raft_consensus.cc:1534] Early marking committed up to index 14
0709 12:14:27.648632 (+ 56227us) log.cc:844] Serialized 14 byte log entry
0709 12:14:27.796485 (+147853us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:14:27.812265 (+ 15780us) log.cc:844] Serialized 78 byte log entry
0709 12:14:27.812431 (+   166us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:27.812666 (+   235us) log.cc:844] Serialized 14 byte log entry
0709 12:14:28.320300 (+507634us) spinlock_profiling.cc:243] Waited 1.02 ms on lock 0x7b180041f200. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14858adc0a 00007f148efed219 00007f1485935a40 00007f14859304e4 00007f1485930455 00007f1488a18060 00007f1488a2a716 00007f1488a3b287 00007f1488a3b208
0709 12:14:28.320589 (+   289us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.320698 (+   109us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:28.346514 (+ 25816us) spinlock_profiling.cc:243] Waited 3.35 ms on lock 0x7b180041f200. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485933522 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:14:28.552666 (+206152us) spinlock_profiling.cc:243] Waited 1.83 ms on lock 0x7b180041f200. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:14:28.679561 (+126895us) raft_consensus.cc:1685] finished
0709 12:14:28.679584 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:28.679975 (+   391us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":606529,"prepare.queue_time_us":79741,"prepare.run_cpu_time_us":83,"prepare.run_wall_time_us":88,"raft.queue_time_us":674329,"raft.run_cpu_time_us":375,"raft.run_wall_time_us":5780,"spinlock_wait_cycles":14246912,"thread_start_us":74087,"threads_started":1,"wal-append.queue_time_us":221943}
W20260709 12:14:28.681880 23110 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.196s	user 0.000s	sys 0.001s
I20260709 12:14:28.665447 23007 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:28.685525 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:28.686902 12675 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 30.
I20260709 12:14:28.688357 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:28.685469 22990 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:14:28.699906 23027 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.900s	user 0.002s	sys 0.005s
W20260709 12:14:28.717029 13305 consensus_peers.cc:597] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:28.734927 12646 leader_election.cc:336] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:28.735975 12646 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 58 similar messages]
I20260709 12:14:28.748062 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:28.757383 12602 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 26: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:28.764217 12671 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 33.
I20260709 12:14:28.760720 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:28.766513 13302 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.767779 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:28.768446 13331 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:28.769322 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:28.773712 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:28.775656 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:28.777944 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:28.780169 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:28.780822 23093 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:28.782269 23007 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:28.786439 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.786309 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:28.789083 13339 raft_consensus.cc:2368] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 34. Current term is 35.
I20260709 12:14:28.780836 13338 raft_consensus.cc:3060] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Advancing to term 33
I20260709 12:14:28.793603 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:28.758461 23065 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:28.801018 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:28.808437 13331 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:14:28.767851 23112 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.767917 22656 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:28.816802 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:28.821403 12671 raft_consensus.cc:2368] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27.
I20260709 12:14:28.817706 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:28.818367 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:28.819131 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:28.820381 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:28.797070 23014 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:28.835263 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 35 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.836763 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:28.837268 12602 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:28.768599 22933 consensus_queue.cc:1059] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 26.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.055s
I20260709 12:14:28.848671 22998 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:28.850757 13302 leader_election.cc:400] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 32. Current term is 33.
I20260709 12:14:28.851212 13302 leader_election.cc:403] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:28.853724 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:28.857565 13338 raft_consensus.cc:2468] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 33.
I20260709 12:14:28.821784 23010 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:28.876868 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:28.798292 22625 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:28.883553 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:28.884821 13337 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:28.839140 23093 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:14:28.838377 22960 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.892289 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:28.893584 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:28.894749 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:28.895268 12671 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:28.875144 22967 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:14:28.880726 22930 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Leader pre-election lost for term 32. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for earlier term 32. Current term is 33.
I20260709 12:14:28.884263 22674 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:28.884196 23020 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
W20260709 12:14:28.904644 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 87023.8us
I20260709 12:14:28.912814 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:28.913808 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:28.896250 23093 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.824596 23112 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:28.888687 22625 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:28.923038 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:28.923324 22625 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:14:28.930860 22851 consensus_meta.cc:287] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.535s	user 0.016s	sys 0.002s
I20260709 12:14:28.903649 22960 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:28.933429 12679 raft_consensus.cc:2410] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 28 because replica has last-logged OpId of term: 27 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 19.
I20260709 12:14:28.920780 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:28.934089 13335 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:28.933585 12672 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:14:28.911156 22934 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:28.936200 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:28.935909 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:28.940310 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:28.941349 12672 raft_consensus.cc:2368] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 32. Current term is 33.
W20260709 12:14:28.942716 23117 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.058s	user 0.000s	sys 0.000s
I20260709 12:14:28.941932 12603 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:28.955282 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:28.943241 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:28.955957 12606 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:14:28.905390 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 145481us
I20260709 12:14:28.936825 23021 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:28.947070 12569 leader_election.cc:400] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 pre-election: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 34. Current term is 35.
I20260709 12:14:28.958670 12569 leader_election.cc:403] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:28.947839 22625 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:28.951604 12496 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:28.934269 23093 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:28.944183 22701 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:28.962482 23021 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:14:28.939877 22851 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:14:28.966676 13305 consensus_peers.cc:597] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:28.975530 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:28.973709 12569 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:28.976514 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:28.967024 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:28.963457 22998 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:28.972270 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:28.978608 12672 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:28.952206 23092 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:28.979969 13335 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:28.979774 22949 raft_consensus.cc:2804] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:14:28.981352 12496 leader_election.cc:400] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27.
I20260709 12:14:28.969990 23123 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:14:28.982645 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:28.974287 23042 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
W20260709 12:14:28.995321 11318 thread.cc:527] Waited for 1000ms trying to join with rpc worker (tid 12542)
I20260709 12:14:28.982321 22865 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:28.999626 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:28.984268 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:28.977519 22933 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Advancing to term 35
I20260709 12:14:28.972584 23091 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:28.986641 12496 leader_election.cc:403] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Cancelling election due to peer responding with higher term
I20260709 12:14:28.981976 12674 raft_consensus.cc:1275] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 31 index: 16. Preceding OpId from leader: term: 32 index: 16. (term mismatch)
I20260709 12:14:28.986487 12494 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:28.976557 22592 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:28.989109 22949 raft_consensus.cc:697] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:14:28.989800 23119 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:14:28.990542 12606 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 32.
I20260709 12:14:29.023965 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:29.024858 12606 raft_consensus.cc:2410] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 28 because replica has last-logged OpId of term: 27 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 19.
I20260709 12:14:29.025597 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:29.026548 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:29.027647 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 23 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:28.998061 12671 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:14:29.030171 12671 raft_consensus.cc:1240] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 29. Current term is 30. Ops: []
I20260709 12:14:29.033011 12671 pending_rounds.cc:85] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 16
I20260709 12:14:29.033340 12671 pending_rounds.cc:107] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 23.17
I20260709 12:14:29.033568 12671 raft_consensus.cc:2889] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:28.984184 12602 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 34.
W20260709 12:14:28.990608 23121 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.071s	user 0.000s	sys 0.001s
I20260709 12:14:29.007831 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:29.043382 12600 raft_consensus.cc:2410] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 28 because replica has last-logged OpId of term: 27 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 19.
I20260709 12:14:29.009707 12674 pending_rounds.cc:85] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 15
I20260709 12:14:29.043954 12531 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:60222 (request call id 1304) took 1390 ms. Trace:
I20260709 12:14:29.044024 12674 pending_rounds.cc:107] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 31.16
I20260709 12:14:29.044597 12674 raft_consensus.cc:2889] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:29.023391 22714 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:29.047246 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:29.044446 12531 rpcz_store.cc:276] 0709 12:14:27.653655 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:27.660689 (+  7034us) service_pool.cc:224] Handling call
0709 12:14:27.660973 (+   284us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:28.217931 (+556958us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.043728 (+825797us) spinlock_profiling.cc:243] Waited 550 ms on lock 0x7b54001068d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:29.043898 (+   170us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1265069952}
I20260709 12:14:29.051227 12600 raft_consensus.cc:1240] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 31. Current term is 32. Ops: [31.16-31.16]
I20260709 12:14:29.018692 22592 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:29.018888 22953 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:29.057513 23107 consensus_queue.cc:1059] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 29.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.010s
I20260709 12:14:29.067603 22953 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:29.068194 23104 raft_consensus.cc:3055] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:14:29.068636 23104 raft_consensus.cc:740] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:29.069582 23104 consensus_queue.cc:260] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:29.071417 23104 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:29.071535 23112 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:29.060503 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:29.075040 12675 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:14:29.079197 23126 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.094s	user 0.000s	sys 0.000s
I20260709 12:14:29.056252 12601 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 31. Current term is 32. Ops: [31.18-31.18]
I20260709 12:14:29.080578 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3864) took 1063 ms. Trace:
I20260709 12:14:29.080957 12601 rpcz_store.cc:276] 0709 12:14:28.017014 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.055672 (+1038658us) service_pool.cc:224] Handling call
0709 12:14:29.056014 (+   342us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:29.080263 (+ 24249us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.080513 (+   250us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:29.022939 23123 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
W20260709 12:14:29.097381 12646 leader_election.cc:336] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:29.011584 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:29.012399 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:29.014036 12494 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.018440 12672 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:29.020233 12643 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.105127 12603 raft_consensus.cc:2368] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 32. Current term is 33.
I20260709 12:14:29.039413 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:29.106567 12606 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26. Ops: []
I20260709 12:14:29.035941 22625 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:29.041965 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 21 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:29.107661 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 95360.9us
I20260709 12:14:29.040807 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:29.108501 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 94528.7us
I20260709 12:14:29.114622 12496 leader_election.cc:400] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 32. Current term is 33.
I20260709 12:14:29.114980 12496 leader_election.cc:403] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Cancelling election due to peer responding with higher term
I20260709 12:14:29.114741 12494 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.023685 22934 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:29.069037 22953 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:29.110797 12600 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 32. Current term is 33. Ops: []
I20260709 12:14:29.109378 12525 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3901) took 2075 ms. Trace:
I20260709 12:14:29.122540 12525 rpcz_store.cc:276] 0709 12:14:27.033710 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:27.035757 (+  2047us) service_pool.cc:224] Handling call
0709 12:14:27.036180 (+   423us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:28.460947 (+1424767us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.109106 (+648159us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b54003cafd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:29.109317 (+   211us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":3274693376}
I20260709 12:14:29.111918 23017 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:14:29.017187 23091 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:29.118046 12496 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:14:29.128978 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 108846us
I20260709 12:14:29.119103 12494 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:29.129940 12603 raft_consensus.cc:1240] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:29.130689 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:29.129952 12643 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.120906 22832 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:14:29.134562 23091 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:29.136919 12599 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: [28.16-28.16]
I20260709 12:14:29.139075 23092 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:29.139882 22949 consensus_queue.cc:237] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 31.22, Last appended by leader: 22, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:29.141623 23085 raft_consensus.cc:3055] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:14:29.142073 23085 raft_consensus.cc:740] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:29.117902 23074 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:29.143095 23074 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:29.143276 23085 consensus_queue.cc:260] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 26.20, Last appended by leader: 20, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:29.118168 22887 raft_consensus.cc:3055] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:14:29.146409 22887 raft_consensus.cc:740] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:29.147323 22887 consensus_queue.cc:260] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 31.16, Last appended by leader: 16, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:29.148361 22887 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:29.148736 23041 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:14:29.111727 22852 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:14:29.149937 22832 consensus_queue.cc:1059] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 29.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:14:29.117595 22998 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:29.151298 22998 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:29.153478 12494 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
W20260709 12:14:29.156193 13305 leader_election.cc:336] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:29.159708 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:29.121016 23078 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:29.165839 12494 leader_election.cc:400] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 32. Current term is 33.
I20260709 12:14:29.166929 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:29.177183 12423 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47524 (request call id 105) took 29934 ms (client timeout 15000 ms). Trace:
W20260709 12:14:29.177433 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 47558.7us
I20260709 12:14:29.111809 23127 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:29.181216 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.176877 12675 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:29.185097 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.187474 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:29.189299 12675 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 30: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:29.145912 23074 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:29.205945 22714 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election lost for term 26. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27.
I20260709 12:14:29.196143 22592 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:29.197032 22998 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:29.197347 23093 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:29.200099 22851 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:29.190346 23124 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:14:29.202890 23040 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.596s	user 0.011s	sys 0.009s
I20260709 12:14:29.178984 22625 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:29.221743 23040 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:29.225332 23040 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:29.204353 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:29.226073 23040 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:14:29.206162 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:29.163823 23017 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:29.191533 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.124400 22934 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:29.179562 12423 rpcz_store.cc:269] 0709 12:13:59.230933 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:05.253588 (+6022655us) service_pool.cc:224] Handling call
0709 12:14:29.164600 (+23911012us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:21.023209 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:21.108244 (+ 85035us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:21.116094 (+  7850us) write_op.cc:435] Acquired schema lock
0709 12:14:21.119399 (+  3305us) tablet.cc:662] Decoding operations
0709 12:14:21.180414 (+ 61015us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:21.180500 (+    86us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:21.180525 (+    25us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:14:21.393031 (+212506us) tablet.cc:701] Row locks acquired
0709 12:14:21.395931 (+  2900us) write_op.cc:260] PREPARE: finished
0709 12:14:21.402688 (+  6757us) write_op.cc:270] Start()
0709 12:14:21.439635 (+ 36947us) write_op.cc:276] Timestamp: P: 1783599261396094 usec, L: 0
0709 12:14:21.439679 (+    44us) op_driver.cc:348] REPLICATION: starting
0709 12:14:21.553979 (+114300us) log.cc:844] Serialized 5196 byte log entry
0709 12:14:22.549240 (+995261us) op_driver.cc:464] REPLICATION: finished
0709 12:14:23.580350 (+1031110us) spinlock_profiling.cc:243] Waited 7.93 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14859160ac 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f148c456ef0
0709 12:14:24.682654 (+1102304us) write_op.cc:301] APPLY: starting
0709 12:14:24.741938 (+ 59284us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:25.724131 (+982193us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:25.724175 (+    44us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:26.103593 (+379418us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:26.104123 (+   530us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=0,mrs_lookups=0
0709 12:14:26.104176 (+    53us) write_op.cc:312] APPLY: finished
0709 12:14:26.104717 (+   541us) log.cc:844] Serialized 121 byte log entry
0709 12:14:26.177246 (+ 72529us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:26.177464 (+   218us) write_op.cc:454] Released schema lock
0709 12:14:26.177579 (+   115us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:26.566421 (+388842us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:26.577673 (+ 11252us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":3714131,"reported_tablets":21,"child_traces":[["op",{"apply.queue_time_us":2053882,"apply.run_cpu_time_us":34154,"apply.run_wall_time_us":1832930,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58389,"num_ops":13,"prepare.queue_time_us":786389,"prepare.run_cpu_time_us":13402,"prepare.run_wall_time_us":640466,"raft.queue_time_us":540873,"raft.run_cpu_time_us":49965,"raft.run_wall_time_us":1837925,"replication_time_us":1093313,"spinlock_wait_cycles":18238720,"thread_start_us":2587971,"threads_started":5,"wal-append.queue_time_us":399105,"wal-append.run_cpu_time_us":47592,"wal-append.run_wall_time_us":2645986}]]}
I20260709 12:14:29.226831 12678 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 31: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:29.229249 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.231351 12675 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:29.231093 12571 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:29.235757 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:29.236665 12571 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:29.210464 22656 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:29.260598 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.190203 22701 consensus_queue.cc:1048] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 12, Time since last communication: 0.047s
I20260709 12:14:29.279424 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:29.272908 12675 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
W20260709 12:14:29.226110 23075 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.707s	user 0.002s	sys 0.018s
I20260709 12:14:29.229596 22625 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader election lost for term 32. Reason: could not achieve majority
I20260709 12:14:29.227560 23040 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:29.313863 12678 raft_consensus.cc:1217] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Deduplicated request from leader. Original: 21.13->[23.14-30.17]   Dedup: 25.15->[28.16-30.17]
I20260709 12:14:29.314661 12678 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 30. Current term is 31. Ops: [23.14-30.17]
I20260709 12:14:29.315019 23040 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 33.21, Last appended by leader: 21, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:29.316697 12678 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 27. Current term is 28. Ops: [27.17-27.17]
I20260709 12:14:29.320755 23075 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:29.321979 12606 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:14:29.228061 23017 raft_consensus.cc:697] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:29.323805 23075 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:29.323432 23017 consensus_queue.cc:237] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.18, Last appended by leader: 18, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:29.295303 12603 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 31 index: 17. (index mismatch)
I20260709 12:14:29.253278 22998 raft_consensus.cc:2804] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:14:29.335649 22998 raft_consensus.cc:697] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:29.336973 22998 consensus_queue.cc:237] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.15, Last appended by leader: 15, Current term: 26, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:29.228626 23111 log.cc:927] Time spent T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.764s	user 0.003s	sys 0.000s
I20260709 12:14:29.253118 23030 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:14:29.238799 23123 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:29.280956 22934 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:29.220798 22592 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:14:29.392653 22592 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:29.393955 22592 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:29.394894 22852 raft_consensus.cc:3055] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:14:29.395481 22852 raft_consensus.cc:740] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:29.294636 23124 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:14:29.338260 12603 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:29.296736 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.396602 22852 consensus_queue.cc:260] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:29.397516 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 100857us
I20260709 12:14:29.398113 12603 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 23 index: 13. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:14:29.398401 12643 leader_election.cc:400] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 32. Current term is 33.
I20260709 12:14:29.398697 12643 leader_election.cc:403] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 election: Cancelling election due to peer responding with higher term
I20260709 12:14:29.399699 12643 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.401011 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.276907 23091 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:29.339254 12678 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: [28.16-28.16]
I20260709 12:14:29.404141 22829 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 35 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:29.349579 23042 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:29.252904 23074 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:14:29.405309 23074 raft_consensus.cc:697] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:29.405130 23042 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:29.406603 23074 consensus_queue.cc:237] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 32.17, Last appended by leader: 17, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:29.409921 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:29.410859 23123 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:29.411819 23123 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader election lost for term 32. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 32. Current term is 33.
I20260709 12:14:29.412700 23123 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:29.413486 23123 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:29.358333 23020 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:29.360841 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 1208) took 1349 ms. Trace:
I20260709 12:14:29.414830 12679 rpcz_store.cc:276] 0709 12:14:28.011793 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.017827 (+  6034us) service_pool.cc:224] Handling call
0709 12:14:29.360565 (+1342738us) spinlock_profiling.cc:243] Waited 905 ms on lock 0x7b54000c3ad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:29.360757 (+   192us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2081753344}
W20260709 12:14:29.362882 13302 leader_election.cc:336] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.098s (SENT)
I20260709 12:14:29.416253 12675 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 25. Ops: []
I20260709 12:14:29.416635 13302 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.311300 22851 raft_consensus.cc:2764] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election decision vote started in defunct term 28: won
W20260709 12:14:29.368466 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:29.380710 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:29.391979 23030 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:29.404270 12678 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 30. Current term is 31. Ops: [30.17-30.17]
W20260709 12:14:29.417915 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 55100.6us
W20260709 12:14:29.368052 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:29.410912 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51304 (request call id 3976) took 1049 ms. Trace:
I20260709 12:14:29.419909 12643 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.431272 12673 rpcz_store.cc:276] 0709 12:14:28.361875 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.362695 (+   820us) service_pool.cc:224] Handling call
0709 12:14:29.410587 (+1047892us) spinlock_profiling.cc:243] Waited 91.8 ms on lock 0x7b54000c35d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:29.410807 (+   220us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":211205888}
I20260709 12:14:29.436472 12679 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: [23.15-24.16]
I20260709 12:14:29.446666 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:29.450893 13302 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.428472 23107 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:14:29.462057 13304 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.451012 22829 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:29.454003 23123 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:14:29.454473 23042 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:29.465607 23065 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.627s	user 0.014s	sys 0.000s
I20260709 12:14:29.434552 23020 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:29.485445 23093 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:29.488209 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1213) took 1444 ms. Trace:
I20260709 12:14:29.505849 13332 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:29.515597 12676 rpcz_store.cc:276] 0709 12:14:28.043818 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.043945 (+   127us) service_pool.cc:224] Handling call
0709 12:14:28.045815 (+  1870us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:28.046221 (+   406us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:14:28.065006 (+ 18785us) log.cc:844] Serialized 14 byte log entry
0709 12:14:28.202721 (+137715us) spinlock_profiling.cc:243] Waited 5.39 ms on lock 0x7b4800060858. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:14:28.413872 (+211151us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:28.414495 (+   623us) log.cc:844] Serialized 26 byte log entry
0709 12:14:28.416109 (+  1614us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:14:28.416412 (+   303us) log.cc:844] Serialized 14 byte log entry
0709 12:14:28.433211 (+ 16799us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.433378 (+   167us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:28.551671 (+118293us) spinlock_profiling.cc:243] Waited 4.42 ms on lock 0x7b18002f1220. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485933522 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:14:29.228031 (+676360us) spinlock_profiling.cc:243] Waited 20.9 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:14:29.487936 (+259905us) raft_consensus.cc:1685] finished
0709 12:14:29.487966 (+    30us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:29.488135 (+   169us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":13616,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":99,"raft.queue_time_us":21858,"raft.run_cpu_time_us":1252,"raft.run_wall_time_us":160288,"spinlock_wait_cycles":70608768,"thread_start_us":297302,"threads_started":1,"wal-append.queue_time_us":365421}
I20260709 12:14:29.517803 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.519712 12676 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:29.520932 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3990) took 1067 ms. Trace:
I20260709 12:14:29.521456 12676 rpcz_store.cc:276] 0709 12:14:28.453029 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.519067 (+1066038us) service_pool.cc:224] Handling call
0709 12:14:29.519536 (+   469us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.520578 (+  1042us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.520849 (+   271us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:29.524117 12643 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:29.529914 12676 raft_consensus.cc:1240] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 31. Current term is 32. Ops: [31.16-31.16]
I20260709 12:14:29.529439 23123 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:29.542763 23078 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Leader election lost for term 32. Reason: Vote denied by peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613) with higher term. Message: Invalid argument: T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 32. Current term is 33.
I20260709 12:14:29.516134 22829 raft_consensus.cc:2804] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader election won for term 25
I20260709 12:14:29.519683 22851 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:29.548799 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:29.552309 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 385433us
I20260709 12:14:29.537643 22701 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:14:29.548349 22933 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.540s	user 0.015s	sys 0.000s
W20260709 12:14:29.561524 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 99587.9us
I20260709 12:14:29.565191 12526 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53864 (request call id 3903) took 2527 ms. Trace:
I20260709 12:14:29.568252 12526 rpcz_store.cc:276] 0709 12:14:27.037911 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:27.042858 (+  4947us) service_pool.cc:224] Handling call
0709 12:14:27.045456 (+  2598us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:27.589349 (+543893us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.564801 (+1975452us) spinlock_profiling.cc:243] Waited 539 ms on lock 0x7b5400015c54. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:29.565087 (+   286us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1239589632}
I20260709 12:14:29.553795 22829 raft_consensus.cc:697] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 25 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:29.571071 13304 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.515121 23030 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 31.15, Last appended by leader: 15, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:29.573711 13304 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.579726 13304 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.580773 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3991) took 1127 ms. Trace:
I20260709 12:14:29.552309 22851 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:29.570250 22933 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Leader pre-election lost for term 34. Reason: Vote denied by peer e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) with higher term. Message: Invalid argument: T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 34. Current term is 35.
I20260709 12:14:29.569553 23123 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:29.581499 12676 rpcz_store.cc:276] 0709 12:14:28.453307 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.522612 (+1069305us) service_pool.cc:224] Handling call
0709 12:14:29.529502 (+  6890us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:29.536646 (+  7144us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.580682 (+ 44036us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:29.596728 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.592754 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:29.582147 23104 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:29.572813 23041 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:29.627700 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.637928 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:29.639564 12643 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:14:29.715366 12497 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.519s	user 0.005s	sys 0.013s
W20260709 12:14:29.716116 12497 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.520s	user 0.006s	sys 0.013s
I20260709 12:14:29.634264 22701 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:29.643935 23124 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:29.643935 23014 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:29.629360 23137 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:29.636221 22953 consensus_queue.cc:1059] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 25.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.154s
W20260709 12:14:29.814865 12496 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.666s	user 0.023s	sys 0.024s
W20260709 12:14:29.826663 12496 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.678s	user 0.024s	sys 0.025s
I20260709 12:14:29.764016 23020 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:29.755631 22865 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:29.821622 13337 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 1185) took 1005 ms. Trace:
I20260709 12:14:29.651993 23078 consensus_queue.cc:1059] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 27.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.159s
I20260709 12:14:29.645282 23040 consensus_queue.cc:1059] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 32.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.165s
W20260709 12:14:29.827194 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 678901us
W20260709 12:14:29.814850 13333 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.601s	user 0.002s	sys 0.024s
W20260709 12:14:29.837917 13333 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.628s	user 0.002s	sys 0.024s
I20260709 12:14:29.683998 23155 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:29.829874 13337 rpcz_store.cc:276] 0709 12:14:28.816040 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.816201 (+   161us) service_pool.cc:224] Handling call
0709 12:14:28.816515 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:28.887695 (+ 71180us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.821270 (+933575us) spinlock_profiling.cc:243] Waited 68.1 ms on lock 0x7b54004781d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:29.821539 (+   269us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":156622720}
W20260709 12:14:29.841296 12571 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.603s	user 0.003s	sys 0.024s
W20260709 12:14:29.846840 12571 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.609s	user 0.003s	sys 0.024s
W20260709 12:14:29.847349 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 609555us
I20260709 12:14:29.856366 12496 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.857602 22953 raft_consensus.cc:3055] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:14:29.767936 23017 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.068s
I20260709 12:14:29.859467 22701 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:29.768549 23104 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:14:29.864144 23041 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:29.858108 12571 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:29.864997 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 99796.7us
I20260709 12:14:29.899097 22953 raft_consensus.cc:740] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:29.910864 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 52829.7us
I20260709 12:14:29.913172 12571 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:29.854336 23155 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:29.921386 23078 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.004s
I20260709 12:14:29.925194 22934 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 24.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.004s
W20260709 12:14:29.926245 23135 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.270s	user 0.000s	sys 0.001s
I20260709 12:14:29.926885 23065 raft_consensus.cc:3055] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Stepping down as leader of term 27
W20260709 12:14:29.927105 23118 log.cc:927] Time spent T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.989s	user 0.007s	sys 0.011s
I20260709 12:14:29.927378 23065 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:29.928643 23040 raft_consensus.cc:3055] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:14:29.928367 23065 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:29.928848 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3981) took 1542 ms. Trace:
I20260709 12:14:29.929154 23040 raft_consensus.cc:740] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:29.929700 23065 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:29.930508 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4696) took 1481 ms. Trace:
W20260709 12:14:29.930886 23139 log.cc:927] Time spent T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.163s	user 0.000s	sys 0.000s
I20260709 12:14:29.932274 22852 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:29.931149 12672 rpcz_store.cc:276] 0709 12:14:28.448880 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.132771 (+683891us) service_pool.cc:224] Handling call
0709 12:14:29.177219 (+ 44448us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:29.177851 (+   632us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:29.177893 (+    42us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:29.178781 (+   888us) log.cc:844] Serialized 26 byte log entry
0709 12:14:29.551160 (+372379us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:29.551229 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.551334 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:29.930218 (+378884us) raft_consensus.cc:1685] finished
0709 12:14:29.930242 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:29.930431 (+   189us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":7447,"raft.queue_time_us":1925,"raft.run_cpu_time_us":153,"raft.run_wall_time_us":9317,"thread_start_us":356649,"threads_started":1,"wal-append.queue_time_us":652064}
I20260709 12:14:29.934413 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4452) took 1356 ms. Trace:
I20260709 12:14:29.934585 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4444) took 1373 ms. Trace:
I20260709 12:14:29.934748 22934 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:29.801947 22829 consensus_queue.cc:237] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:29.934957 12606 rpcz_store.cc:276] 0709 12:14:28.577683 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.107422 (+529739us) service_pool.cc:224] Handling call
0709 12:14:29.107752 (+   330us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:29.322408 (+214656us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.933948 (+611540us) spinlock_profiling.cc:243] Waited 214 ms on lock 0x7b5400310bd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:29.934345 (+   397us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":492120960}
I20260709 12:14:29.925110 23074 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:29.938185 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4695) took 1489 ms. Trace:
I20260709 12:14:29.935261 12598 rpcz_store.cc:276] 0709 12:14:28.560843 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.091978 (+531135us) service_pool.cc:224] Handling call
0709 12:14:29.092491 (+   513us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:29.092998 (+   507us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:29.093034 (+    36us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:29.093819 (+   785us) log.cc:844] Serialized 52 byte log entry
0709 12:14:29.462524 (+368705us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:14:29.462574 (+    50us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.462648 (+    74us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:29.934213 (+471565us) raft_consensus.cc:1685] finished
0709 12:14:29.934244 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:29.934493 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":124,"raft.run_cpu_time_us":83,"raft.run_wall_time_us":84,"thread_start_us":357924,"threads_started":1,"wal-append.queue_time_us":478386}
I20260709 12:14:29.942466 23074 raft_consensus.cc:697] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:29.929289 12677 rpcz_store.cc:276] 0709 12:14:28.386049 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.517204 (+131155us) service_pool.cc:224] Handling call
0709 12:14:28.517520 (+   316us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:28.520588 (+  3068us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:28.520620 (+    32us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:28.541222 (+ 20602us) spinlock_profiling.cc:243] Waited 18.8 ms on lock 0x7b2c00037868. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:14:28.541769 (+   547us) log.cc:844] Serialized 26 byte log entry
0709 12:14:28.792252 (+250483us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:14:28.823028 (+ 30776us) log.cc:844] Serialized 14 byte log entry
0709 12:14:28.823293 (+   265us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:28.823355 (+    62us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:28.949889 (+126534us) spinlock_profiling.cc:243] Waited 2.3 ms on lock 0x7b18002c7d60. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:14:29.926444 (+976555us) spinlock_profiling.cc:243] Waited 1.66 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:14:29.928618 (+  2174us) raft_consensus.cc:1685] finished
0709 12:14:29.928640 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:29.928781 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":30604,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":4225,"raft.queue_time_us":2635,"raft.run_cpu_time_us":185,"raft.run_wall_time_us":187,"spinlock_wait_cycles":52258560,"thread_start_us":250036,"threads_started":1,"wal-append.queue_time_us":354453}
I20260709 12:14:29.943004 23040 consensus_queue.cc:260] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 32.17, Last appended by leader: 17, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:29.943764 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:29.943477 23074 consensus_queue.cc:237] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:29.944339 12677 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:29.942469 12671 rpcz_store.cc:276] 0709 12:14:28.448605 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.032175 (+583570us) service_pool.cc:224] Handling call
0709 12:14:29.032569 (+   394us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:29.106052 (+ 73483us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:29.106085 (+    33us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:29.111308 (+  5223us) log.cc:844] Serialized 52 byte log entry
0709 12:14:29.571574 (+460266us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:29.571638 (+    64us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.571746 (+   108us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:29.937889 (+366143us) raft_consensus.cc:1685] finished
0709 12:14:29.937913 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:29.938090 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":64800,"raft.queue_time_us":67551,"raft.run_cpu_time_us":140,"raft.run_wall_time_us":3310,"thread_start_us":413038,"threads_started":1,"wal-append.queue_time_us":465036}
I20260709 12:14:29.939951 22979 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:29.946337 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:29.946708 22979 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:29.947664 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:29.941075 23021 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:14:29.845841 22930 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:29.949635 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:29.845311 22865 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:29.910719 22933 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:29.911051 23042 raft_consensus.cc:3055] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:14:29.933782 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:14:29.950613 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:29.951017 23042 raft_consensus.cc:740] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:29.951545 23155 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:29.951298 12672 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:29.951930 22865 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:29.952116 12671 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:14:29.948340 22979 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:29.955165 23078 consensus_queue.cc:1243] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 27.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
I20260709 12:14:29.950678 22933 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:29.959369 22933 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:29.960088 22933 consensus_queue.cc:1059] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 22.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:14:29.960309 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4457) took 1357 ms. Trace:
I20260709 12:14:29.961067 12600 rpcz_store.cc:276] 0709 12:14:28.603239 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.128868 (+525629us) service_pool.cc:224] Handling call
0709 12:14:29.151485 (+ 22617us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:29.152033 (+   548us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:29.152070 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:29.152560 (+   490us) log.cc:844] Serialized 26 byte log entry
0709 12:14:29.593263 (+440703us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:29.593330 (+    67us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.593585 (+   255us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:29.960006 (+366421us) raft_consensus.cc:1685] finished
0709 12:14:29.960059 (+    53us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:29.960233 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":2901,"raft.run_cpu_time_us":198,"raft.run_wall_time_us":6507,"thread_start_us":440182,"threads_started":1,"wal-append.queue_time_us":759193}
I20260709 12:14:29.962625 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 32 candidate_status { last_received { term: 28 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:29.954066 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:29.944289 23040 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:29.763659 23091 consensus_queue.cc:1059] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 25.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.144s
W20260709 12:14:29.965476 12643 leader_election.cc:336] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.843s (SENT)
I20260709 12:14:29.969761 12643 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.973345 12606 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 33. Current term is 34. Ops: []
I20260709 12:14:29.758608 22625 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.256s
W20260709 12:14:29.974592 12597 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s	user 0.060s	sys 0.049s
W20260709 12:14:29.981410 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:29.984804 12597 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s	user 0.060s	sys 0.049s
I20260709 12:14:29.983500 12677 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 33.
I20260709 12:14:29.888188 22701 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:29.984145 12672 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 34.
I20260709 12:14:29.993707 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 23 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:29.994699 12677 raft_consensus.cc:2410] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 29 because replica has last-logged OpId of term: 26 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 13.
I20260709 12:14:29.995855 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:29.998710 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3971) took 1655 ms. Trace:
I20260709 12:14:29.999228 12674 rpcz_store.cc:276] 0709 12:14:28.343426 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:28.347946 (+  4520us) service_pool.cc:224] Handling call
0709 12:14:28.348262 (+   316us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.045470 (+697208us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.998374 (+952904us) spinlock_profiling.cc:243] Waited 633 ms on lock 0x7b540016eed4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:29.998627 (+   253us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1456516352}
I20260709 12:14:30.004508 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:29.855489 23099 consensus_queue.cc:1059] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.118s
W20260709 12:14:30.005937 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 356804us
I20260709 12:14:29.994531 23021 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.023918 12678 raft_consensus.cc:1240] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: [28.16-28.16]
I20260709 12:14:30.027801 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4713) took 1274 ms. Trace:
I20260709 12:14:30.028319 12678 rpcz_store.cc:276] 0709 12:14:28.753545 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.019545 (+1266000us) service_pool.cc:224] Handling call
0709 12:14:30.023581 (+  4036us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:30.027432 (+  3851us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.027721 (+   289us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:30.011673 22937 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.523s	user 0.014s	sys 0.018s
I20260709 12:14:30.006727 23141 raft_consensus.cc:3055] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:14:30.014213 22625 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:14:30.010854 23014 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.031s
W20260709 12:14:30.083271 23151 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.110s	user 0.000s	sys 0.000s
W20260709 12:14:30.037386 22937 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.565s	user 0.015s	sys 0.018s
I20260709 12:14:30.087419 22960 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:30.041280 22953 consensus_queue.cc:260] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:30.120482 23154 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.067s	user 0.000s	sys 0.000s
W20260709 12:14:30.121196 23114 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 1.633s	user 0.006s	sys 0.010s
I20260709 12:14:30.038872 23141 raft_consensus.cc:740] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:30.035414 23021 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:30.121562 23153 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.083s	user 0.000s	sys 0.001s
I20260709 12:14:30.047796 23099 consensus_queue.cc:1243] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 29.17 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:14:30.125916 23042 consensus_queue.cc:260] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:30.141757 23085 raft_consensus.cc:3060] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:30.172605 23136 raft_consensus.cc:3055] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:14:30.173247 23136 raft_consensus.cc:740] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:30.174450 23021 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:30.181318 23042 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:30.192159 23136 consensus_queue.cc:260] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:30.208464 23155 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:30.191572 23141 consensus_queue.cc:260] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.16, Last appended by leader: 16, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:30.211405 23141 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:30.218205 23155 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:30.255489 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 358348us
I20260709 12:14:30.315795 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4454) took 1737 ms. Trace:
I20260709 12:14:30.319270 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:14:30.322395 12605 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.696s	user 0.003s	sys 0.015s
W20260709 12:14:30.323022 12605 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.733s	user 0.003s	sys 0.016s
I20260709 12:14:30.040329 22851 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:30.093144 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 310516us
I20260709 12:14:30.309777 12598 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Stepping down as leader of term 27
W20260709 12:14:30.395192 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 91702.3us
I20260709 12:14:30.316759 12602 rpcz_store.cc:276] 0709 12:14:28.578294 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.114769 (+536475us) service_pool.cc:224] Handling call
0709 12:14:29.115094 (+   325us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:29.115327 (+   233us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:29.115354 (+    27us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:29.121970 (+  6616us) log.cc:844] Serialized 26 byte log entry
0709 12:14:29.572049 (+450079us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:14:29.572132 (+    83us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:29.572457 (+   325us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:30.314685 (+742228us) raft_consensus.cc:1685] finished
0709 12:14:30.314784 (+    99us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.315370 (+   586us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":407847,"threads_started":1,"wal-append.queue_time_us":919277}
I20260709 12:14:30.387454 12598 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:14:30.401798 12680 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.565s	user 0.006s	sys 0.014s
W20260709 12:14:30.402263 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.406937 13332 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 30. Current term is 31. Ops: []
I20260709 12:14:30.401537 23021 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:30.402345 12680 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.631s	user 0.006s	sys 0.015s
I20260709 12:14:30.422796 22887 raft_consensus.cc:3055] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:14:30.427047 22847 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 28.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:14:30.446560 22990 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:30.446861 22832 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 12:14:30.447821 22990 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:30.450301 12679 raft_consensus.cc:1240] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 32. Current term is 33. Ops: []
I20260709 12:14:30.451499 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4715) took 1697 ms. Trace:
I20260709 12:14:30.451957 12679 rpcz_store.cc:276] 0709 12:14:28.805909 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.424836 (+1618927us) service_pool.cc:224] Handling call
0709 12:14:30.425654 (+   818us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:30.451047 (+ 25393us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.451341 (+   294us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:30.458729 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:30.460697 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4699) took 2010 ms. Trace:
I20260709 12:14:30.434973 22887 raft_consensus.cc:740] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:30.491369 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:30.493153 22847 raft_consensus.cc:3055] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:14:30.460827 22930 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.461287 12675 rpcz_store.cc:276] 0709 12:14:28.449771 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.283473 (+833702us) service_pool.cc:224] Handling call
0709 12:14:29.415975 (+132502us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.416838 (+   863us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.460088 (+1043250us) spinlock_profiling.cc:243] Waited 132 ms on lock 0x7b54000c35d0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1f8ad 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:30.460348 (+   260us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":303702272}
I20260709 12:14:30.493644 22847 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:30.510244 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:30.516652 13302 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:30.517637 22847 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.17, Last appended by leader: 17, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:30.535624 12671 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27. Ops: []
W20260709 12:14:30.553251 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:30.551734 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:30.530556 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.485450 23041 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.524927 12601 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4443) took 2199 ms. Trace:
I20260709 12:14:30.528859 12679 raft_consensus.cc:1275] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 26 index: 16. (term mismatch)
I20260709 12:14:30.544313 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:30.612273 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.545343 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:14:30.613514 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:30.614084 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 208062us
I20260709 12:14:30.542975 12598 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:30.615228 23106 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:30.615995 12598 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:30.617656 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4702) took 1905 ms. Trace:
I20260709 12:14:30.549008 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4445) took 1987 ms. Trace:
I20260709 12:14:30.619285 22847 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:30.522632 22701 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:30.521298 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4718) took 1705 ms. Trace:
I20260709 12:14:30.537449 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:30.623533 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 627677us
I20260709 12:14:30.623242 12675 rpcz_store.cc:276] 0709 12:14:28.815450 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.519251 (+1703801us) service_pool.cc:224] Handling call
0709 12:14:30.520160 (+   909us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:30.520710 (+   550us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:30.520749 (+    39us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:30.520839 (+    90us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:30.520909 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.521003 (+    94us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.521199 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:30.618566 12597 rpcz_store.cc:276] 0709 12:14:28.561221 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.095119 (+533898us) service_pool.cc:224] Handling call
0709 12:14:29.095443 (+   324us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.095686 (+   243us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:14:29.462958 (+367272us) log.cc:844] Serialized 14 byte log entry
0709 12:14:30.513906 (+1050948us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:30.548332 (+ 34426us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:14:30.548441 (+   109us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.548547 (+   106us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.548815 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":324137,"prepare.queue_time_us":496519,"prepare.run_cpu_time_us":51,"prepare.run_wall_time_us":52,"raft.queue_time_us":442063,"raft.run_cpu_time_us":298,"raft.run_wall_time_us":298,"thread_start_us":1011976,"threads_started":3,"wal-append.queue_time_us":609194}
I20260709 12:14:30.526965 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:30.558300 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3992) took 2104 ms. Trace:
I20260709 12:14:30.541977 23021 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:30.622957 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4703) took 1910 ms. Trace:
I20260709 12:14:30.541365 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4455) took 1947 ms. Trace:
I20260709 12:14:30.518023 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:30.624125 12603 raft_consensus.cc:2410] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 28 because replica has last-logged OpId of term: 27 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 14.
I20260709 12:14:30.522324 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:30.557395 12673 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3989) took 2104 ms. Trace:
W20260709 12:14:30.523854 23183 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.057s	user 0.000s	sys 0.000s
W20260709 12:14:30.528169 13305 consensus_peers.cc:597] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:30.539315 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:30.625644 23041 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:30.601022 12679 pending_rounds.cc:85] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 15
I20260709 12:14:30.555224 12421 catalog_manager.cc:5697] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 18 to 20, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 20 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:30.631233 12605 rpcz_store.cc:276] 0709 12:14:28.594183 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.125017 (+530834us) service_pool.cc:224] Handling call
0709 12:14:29.125331 (+   314us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.231852 (+106521us) raft_consensus.cc:1534] Early marking committed up to index 18
0709 12:14:29.590154 (+358302us) log.cc:844] Serialized 14 byte log entry
0709 12:14:30.518625 (+928471us) log.cc:844] Serialized 14 byte log entry
0709 12:14:30.538677 (+ 20052us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:30.538786 (+   109us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:14:30.538938 (+   152us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.540966 (+  2028us) spinlock_profiling.cc:243] Waited 106 ms on lock 0x7b54002e6dd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:30.541033 (+    67us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.541282 (+   249us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":127375,"prepare.queue_time_us":566548,"prepare.run_cpu_time_us":112,"prepare.run_wall_time_us":8606,"raft.queue_time_us":133281,"raft.run_cpu_time_us":276,"raft.run_wall_time_us":276,"spinlock_wait_cycles":244496128,"thread_start_us":1078615,"threads_started":3,"wal-append.queue_time_us":542877}
I20260709 12:14:30.618096 12672 rpcz_store.cc:276] 0709 12:14:28.712560 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.991692 (+1279132us) service_pool.cc:224] Handling call
0709 12:14:30.557766 (+566074us) spinlock_profiling.cc:243] Waited 14.8 ms on lock 0x7b58001f2c98. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fba22 00007f148e8148d9 00007f148d96eee0 00007f148da380ab 00007f148da38131 00007f148d9f9fc0 00007f148d9e4929 00007f148d9e43d2 00007f1486f5bbd4
0709 12:14:30.597739 (+ 39973us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:30.617143 (+ 19404us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:14:30.617176 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:30.617253 (+    77us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:30.617284 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.617346 (+    62us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.617487 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1537,"raft.run_cpu_time_us":175,"raft.run_wall_time_us":176,"spinlock_wait_cycles":34010880}
I20260709 12:14:30.526300 23074 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.633821 12673 rpcz_store.cc:276] 0709 12:14:28.452780 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.460533 (+1007753us) service_pool.cc:224] Handling call
0709 12:14:29.474290 (+ 13757us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.475417 (+  1127us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:14:29.916672 (+441255us) log.cc:844] Serialized 14 byte log entry
0709 12:14:30.488895 (+572223us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:30.507652 (+ 18757us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:14:30.551353 (+ 43701us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.557004 (+  5651us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.557206 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":9475,"prepare.queue_time_us":509960,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":4875,"raft.queue_time_us":54221,"raft.run_cpu_time_us":463,"raft.run_wall_time_us":55920,"thread_start_us":869093,"threads_started":2}
I20260709 12:14:30.627034 12676 rpcz_store.cc:276] 0709 12:14:28.453553 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.595288 (+1141735us) service_pool.cc:224] Handling call
0709 12:14:29.652971 (+ 57683us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.653288 (+   317us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:14:29.909571 (+256283us) log.cc:844] Serialized 14 byte log entry
0709 12:14:30.557532 (+647961us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:30.557648 (+   116us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:14:30.557760 (+   112us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.557862 (+   102us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.558064 (+   202us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":8943,"prepare.queue_time_us":349277,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":35607,"raft.run_cpu_time_us":256,"raft.run_wall_time_us":256,"thread_start_us":623307,"threads_started":2,"wal-append.queue_time_us":292640}
I20260709 12:14:30.558527 23123 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:14:30.642654 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 114695us
I20260709 12:14:30.750769 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:30.781916 12604 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 35.
I20260709 12:14:30.599646 12601 rpcz_store.cc:276] 0709 12:14:28.325100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.081985 (+756885us) service_pool.cc:224] Handling call
0709 12:14:29.091218 (+  9233us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:29.091517 (+   299us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:14:29.482606 (+391089us) log.cc:844] Serialized 14 byte log entry
0709 12:14:30.462286 (+979680us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:30.494434 (+ 32148us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:30.509735 (+ 15301us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.524523 (+ 14788us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.524753 (+   230us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":409768,"prepare.queue_time_us":491827,"prepare.run_cpu_time_us":53,"prepare.run_wall_time_us":55,"raft.queue_time_us":521129,"raft.run_cpu_time_us":338,"raft.run_wall_time_us":339,"spinlock_wait_cycles":237824,"thread_start_us":882021,"threads_started":3,"wal-append.queue_time_us":572914}
W20260709 12:14:30.690322 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 94425.5us
I20260709 12:14:30.683444 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4710) took 1931 ms. Trace:
I20260709 12:14:30.527765 23030 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.701989 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:30.673480 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4719) took 1851 ms. Trace:
I20260709 12:14:30.630862 12671 rpcz_store.cc:276] 0709 12:14:28.712921 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.997570 (+1284649us) service_pool.cc:224] Handling call
0709 12:14:30.534828 (+537258us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:30.595819 (+ 60991us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.622880 (+ 27061us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:30.718629 22953 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:14:30.680948 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 309594us
I20260709 12:14:30.706318 12602 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
W20260709 12:14:30.700181 12674 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.574s	user 0.013s	sys 0.037s
I20260709 12:14:30.518285 22592 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.530881 23072 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:30.810149 12675 rpcz_store.cc:276] 0709 12:14:28.817910 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.625339 (+1807429us) service_pool.cc:224] Handling call
0709 12:14:30.625868 (+   529us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:30.668852 (+ 42984us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:30.668912 (+    60us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:30.669015 (+   103us) raft_consensus.cc:1645] Marking committed up to 18
0709 12:14:30.669072 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.669152 (+    80us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.669313 (+   161us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:30.702718 12523 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:53862 (request call id 3831) took 3426 ms. Trace:
I20260709 12:14:30.804517 12679 pending_rounds.cc:107] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 25.16
I20260709 12:14:30.817091 12679 raft_consensus.cc:2889] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:30.781270 23014 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.715214 22625 raft_consensus.cc:3055] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 LEADER]: Stepping down as leader of term 28
W20260709 12:14:30.823348 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.702216 23021 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:30.813966 22656 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:30.819609 23124 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:30.825162 23017 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:14:30.838189 23017 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:30.825966 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1220) took 2029 ms. Trace:
I20260709 12:14:30.806341 12677 rpcz_store.cc:276] 0709 12:14:28.752108 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.007963 (+1255855us) service_pool.cc:224] Handling call
0709 12:14:30.008374 (+   411us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:30.008677 (+   303us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:14:30.460795 (+452118us) log.cc:844] Serialized 14 byte log entry
0709 12:14:30.682658 (+221863us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:30.682781 (+   123us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:30.682877 (+    96us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.683008 (+   131us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.683208 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13033,"prepare.queue_time_us":31703,"prepare.run_cpu_time_us":96,"prepare.run_wall_time_us":2847,"raft.queue_time_us":31517,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":188,"thread_start_us":519413,"threads_started":2}
I20260709 12:14:30.654241 22701 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.176s
I20260709 12:14:30.814280 23074 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:30.828472 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:30.712929 22990 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:30.680650 23075 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:30.781817 22865 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:30.694286 23136 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:30.721462 22930 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:30.836237 23124 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:30.838980 12679 rpcz_store.cc:276] 0709 12:14:28.796312 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.461791 (+1665479us) service_pool.cc:224] Handling call
0709 12:14:30.528117 (+ 66326us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:30.825654 (+297537us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.825889 (+   235us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:30.809109 22937 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.856727 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:30.816968 12523 rpcz_store.cc:276] 0709 12:14:27.275768 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:27.277905 (+  2137us) service_pool.cc:224] Handling call
0709 12:14:27.278301 (+   396us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:27.332875 (+ 54574us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:27.332912 (+    37us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:29.128961 (+1796049us) spinlock_profiling.cc:243] Waited 5.86 ms on lock 0x7b180040af20. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859012a6 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a
0709 12:14:30.303708 (+1174747us) spinlock_profiling.cc:243] Waited 2.64 ms on lock 0x7b2c00058028. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1487319a11 00007f1487329372 00007f1488a5ba01 00007f1488a2043a 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2
0709 12:14:30.318604 (+ 14896us) log.cc:844] Serialized 26 byte log entry
0709 12:14:30.367626 (+ 49022us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:30.386083 (+ 18457us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.396975 (+ 10892us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:30.702054 (+305079us) raft_consensus.cc:1685] finished
0709 12:14:30.702228 (+   174us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.702437 (+   209us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":37787,"raft.queue_time_us":54795,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":108,"spinlock_wait_cycles":19540608,"thread_start_us":45869,"threads_started":1,"wal-append.queue_time_us":378852}
I20260709 12:14:30.839484 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:30.702934 23190 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:30.518491 23078 consensus_queue.cc:1059] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 22.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.363s
I20260709 12:14:30.846180 23017 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:30.849596 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 25 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:30.825795 22960 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.827961 22625 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:30.863214 13330 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:30.864037 13333 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:30.864876 13339 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:14:30.782526 22887 consensus_queue.cc:260] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:30.861617 12597 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:30.866573 12605 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:14:30.875316 12672 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:14:30.870411 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4707) took 2119 ms. Trace:
I20260709 12:14:30.883383 13332 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 25 LEADER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: []
I20260709 12:14:30.864794 22937 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:30.888036 22937 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:30.881687 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 3995) took 2296 ms. Trace:
I20260709 12:14:30.892769 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4720) took 1897 ms. Trace:
I20260709 12:14:30.893262 12672 rpcz_store.cc:276] 0709 12:14:28.983526 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.823102 (+1839576us) service_pool.cc:224] Handling call
0709 12:14:30.857922 (+ 34820us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:30.875898 (+ 17976us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.880709 (+  4811us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:30.902436 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:30.905447 12679 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
I20260709 12:14:30.856240 23030 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:30.911551 23017 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:14:30.912020 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:30.913822 22930 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:30.914294 22960 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:30.915352 22847 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:30.915584 22934 consensus_meta.cc:287] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.892s	user 0.003s	sys 0.016s
I20260709 12:14:30.906883 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1223) took 1389 ms. Trace:
I20260709 12:14:30.891309 12680 rpcz_store.cc:276] 0709 12:14:28.577184 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.770522 (+1193338us) service_pool.cc:224] Handling call
0709 12:14:29.771012 (+   490us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:30.551511 (+780499us) raft_consensus.cc:1534] Early marking committed up to index 16
0709 12:14:30.558420 (+  6909us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:30.559210 (+   790us) log.cc:844] Serialized 26 byte log entry
0709 12:14:30.802234 (+243024us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:30.802277 (+    43us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.811880 (+  9603us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:30.865147 (+ 53267us) raft_consensus.cc:1685] finished
0709 12:14:30.865176 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.873772 (+  8596us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":47490,"raft.run_cpu_time_us":251,"raft.run_wall_time_us":17419,"thread_start_us":902203,"threads_started":2,"wal-append.queue_time_us":289540}
I20260709 12:14:30.924077 12679 rpcz_store.cc:276] 0709 12:14:29.517409 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.904291 (+1386882us) service_pool.cc:224] Handling call
0709 12:14:30.904897 (+   606us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:30.905906 (+  1009us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.906806 (+   900us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:30.935879 12598 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:14:30.876303 12674 rpcz_store.cc:276] 0709 12:14:28.751240 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.006202 (+1254962us) service_pool.cc:224] Handling call
0709 12:14:30.007449 (+  1247us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:30.014077 (+  6628us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:30.014179 (+   102us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:30.015456 (+  1277us) log.cc:844] Serialized 26 byte log entry
0709 12:14:30.839983 (+824527us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:14:30.851482 (+ 11499us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.865640 (+ 14158us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:30.865685 (+    45us) raft_consensus.cc:1685] finished
0709 12:14:30.865696 (+    11us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.870334 (+  4638us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":3787,"raft.queue_time_us":108959,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":13453,"thread_start_us":795759,"threads_started":1,"wal-append.queue_time_us":680500}
I20260709 12:14:30.939569 12421 catalog_manager.cc:5697] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 16 to 18. New cstate: current_term: 18 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:30.940735 22930 consensus_queue.cc:1048] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:14:30.943873 23123 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:30.944345 23041 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:30.944847 23085 consensus_meta.cc:287] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.771s	user 0.010s	sys 0.012s
I20260709 12:14:30.944643 23123 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:30.946833 23021 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:14:30.947036 22960 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:30.947343 23021 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:30.947687 22865 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:30.948573 23021 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 31.18, Last appended by leader: 18, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:30.949291 23195 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:30.952823 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 52030us
I20260709 12:14:30.964438 23146 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.964993 22847 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:30.965266 23146 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:30.986670 23146 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:30.990393 12677 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 33. Current term is 34. Ops: []
I20260709 12:14:30.990720 22952 raft_consensus.cc:3055] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:14:30.991106 22952 raft_consensus.cc:740] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:30.991226 12677 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1222) took 1474 ms. Trace:
I20260709 12:14:30.991597 12677 rpcz_store.cc:276] 0709 12:14:29.517110 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.858904 (+1341794us) service_pool.cc:224] Handling call
0709 12:14:30.937056 (+ 78152us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:30.990947 (+ 53891us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.991171 (+   224us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:30.996294 23124 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:30.968559 23074 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:30.970173 23075 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:30.972503 23040 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.987s	user 0.017s	sys 0.008s
I20260709 12:14:30.973315 23188 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.973780 23190 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.014595 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1193) took 1047 ms. Trace:
W20260709 12:14:30.974761 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 55436.6us
I20260709 12:14:30.978173 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4722) took 1966 ms. Trace:
I20260709 12:14:30.978447 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4714) took 2224 ms. Trace:
I20260709 12:14:30.881686 23072 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:30.980232 12601 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
W20260709 12:14:30.983531 23042 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.801s	user 0.013s	sys 0.008s
I20260709 12:14:30.966279 23161 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.033s
W20260709 12:14:30.996677 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 59206.5us
I20260709 12:14:30.894605 23147 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 30.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.173s
I20260709 12:14:30.997383 12496 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:30.998258 22714 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.925385 22701 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:30.999486 22625 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:30.968257 22847 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:31.003340 12599 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:31.004273 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4721) took 1992 ms. Trace:
I20260709 12:14:31.037863 12671 rpcz_store.cc:276] 0709 12:14:29.011749 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.906147 (+1894398us) service_pool.cc:224] Handling call
0709 12:14:30.933786 (+ 27639us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:30.963068 (+ 29282us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:30.963108 (+    40us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:31.003922 (+ 40814us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:14:31.003993 (+    71us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:31.004072 (+    79us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:31.004217 (+   145us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:30.872398 23014 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:30.869946 22592 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:14:30.950064 23021 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:31.008388 22934 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:30.939237 22887 raft_consensus.cc:3060] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:31.009987 12675 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
I20260709 12:14:31.015041 12598 rpcz_store.cc:276] 0709 12:14:29.961447 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.961589 (+   142us) service_pool.cc:224] Handling call
0709 12:14:29.973453 (+ 11864us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:30.936535 (+963082us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:31.008789 (+ 72254us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:31.015391 12676 rpcz_store.cc:276] 0709 12:14:29.012214 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.834076 (+1821862us) service_pool.cc:224] Handling call
0709 12:14:30.907880 (+ 73804us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:30.977723 (+ 69843us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:30.977771 (+    48us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:30.977833 (+    62us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:14:30.977864 (+    31us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.977935 (+    71us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.978064 (+   129us) inbound_call.cc:177] Queueing success response
Metrics: {}
W20260709 12:14:31.026484 12572 thread.cc:641] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.532s	user 0.010s	sys 0.026s
I20260709 12:14:31.027030 23124 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:30.963589 22865 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:30.996336 23065 consensus_meta.cc:287] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 1.065s	user 0.009s	sys 0.012s
I20260709 12:14:31.032213 23106 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:31.000829 12421 catalog_manager.cc:5697] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 14 to 19, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 19 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:14:31.041847 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.062052 13333 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:14:30.949888 23195 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:31.039505 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:31.015769 22960 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:31.045691 12572 thread.cc:608] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.552s	user 0.011s	sys 0.026s
I20260709 12:14:31.047127 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1221) took 1531 ms. Trace:
I20260709 12:14:31.015646 12678 rpcz_store.cc:276] 0709 12:14:28.753838 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.029205 (+1275367us) service_pool.cc:224] Handling call
0709 12:14:30.029661 (+   456us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:30.029951 (+   290us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:30.029981 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:30.030787 (+   806us) log.cc:844] Serialized 26 byte log entry
0709 12:14:30.776066 (+745279us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:14:30.812639 (+ 36573us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:30.821921 (+  9282us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:30.941520 (+119599us) spinlock_profiling.cc:243] Waited 54.9 ms on lock 0x7b54003f37a0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488995dbe 00007f14889a0930 00007f14889a08d8 00007f14889a0858 00007f14889a0813 00007f148899fa9c 00007f148e88b2f6 00007f148e885aa8 00007f14889dc7cb 00007f14889df2c0
0709 12:14:30.951482 (+  9962us) raft_consensus.cc:1685] finished
0709 12:14:30.951508 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:30.978089 (+ 26581us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":126268672,"thread_start_us":105365,"threads_started":1,"wal-append.queue_time_us":336610}
W20260709 12:14:31.064829 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 571570us
I20260709 12:14:31.052373 13332 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 33 LEADER]: Stepping down as leader of term 33
W20260709 12:14:31.060303 22852 consensus_meta.cc:287] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.127s	user 0.010s	sys 0.019s
I20260709 12:14:31.065620 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:31.061200 12644 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.065554 13332 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:31.064939 12675 rpcz_store.cc:276] 0709 12:14:29.516731 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.834554 (+1317823us) service_pool.cc:224] Handling call
0709 12:14:30.910577 (+ 76023us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:31.045144 (+134567us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:31.047055 (+  1911us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:31.068511 12680 raft_consensus.cc:1240] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 25. Current term is 26. Ops: []
I20260709 12:14:31.076172 12674 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 32. Current term is 33. Ops: []
I20260709 12:14:31.079214 12597 raft_consensus.cc:2410] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 27 because replica has last-logged OpId of term: 26 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 23 index: 14.
I20260709 12:14:31.081494 12598 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:31.076941 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:31.090971 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:31.091864 12597 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 35: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:14:31.059811 22952 consensus_queue.cc:260] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 24.16, Last appended by leader: 16, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.096534 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:31.092922 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4723) took 1231 ms. Trace:
I20260709 12:14:31.098320 12672 rpcz_store.cc:276] 0709 12:14:29.861509 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.938669 (+1077160us) service_pool.cc:224] Handling call
0709 12:14:31.034026 (+ 95357us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:31.083311 (+ 49285us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:31.083357 (+    46us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:31.092540 (+  9183us) raft_consensus.cc:1645] Marking committed up to 14
0709 12:14:31.092600 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:31.092687 (+    87us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:31.092846 (+   159us) inbound_call.cc:177] Queueing success response
Metrics: {}
I20260709 12:14:31.108968 13339 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 35: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:14:31.097399 12601 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:14:31.074227 22865 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.086751 22990 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.120064 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:31.126096 12598 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 25. Ops: []
I20260709 12:14:31.134006 22953 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:31.135480 22625 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:31.148919 12675 raft_consensus.cc:1240] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:31.150905 12600 raft_consensus.cc:1275] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 26 index: 20. Preceding OpId from leader: term: 27 index: 21. (index mismatch)
I20260709 12:14:31.083068 22714 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:31.156780 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.087055 22656 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:31.111248 23141 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.899s	user 0.012s	sys 0.012s
W20260709 12:14:31.162771 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49576.7us
I20260709 12:14:31.159240 22952 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:31.078652 22701 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.170168 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1196) took 1197 ms. Trace:
I20260709 12:14:31.084131 23124 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.200220 12600 rpcz_store.cc:276] 0709 12:14:29.972216 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.972437 (+   221us) service_pool.cc:224] Handling call
0709 12:14:29.974821 (+  2384us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:31.151404 (+1176583us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:31.153920 (+  2516us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54001171d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:31.169750 (+ 15830us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2579981056}
I20260709 12:14:31.180250 12598 raft_consensus.cc:1240] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27. Ops: []
W20260709 12:14:31.177129 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 179901us
W20260709 12:14:31.176765 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 105854us
W20260709 12:14:31.206269 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.082525 23188 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:31.216524 12646 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.167032 23123 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:31.107780 22960 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:31.095253 23106 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:31.176963 22852 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:31.258013 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 160009us
I20260709 12:14:31.207327 13332 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 33.21, Last appended by leader: 21, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:31.271847 23155 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.105s
I20260709 12:14:31.265589 23091 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.275074 23091 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:31.279124 23203 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.116s	user 0.001s	sys 0.000s
I20260709 12:14:31.280969 22656 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.282428 23123 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:31.283072 23123 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:31.283587 23123 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:31.283782 22656 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:31.241679 12599 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:14:31.266580 12672 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
W20260709 12:14:31.303229 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 43914.5us
I20260709 12:14:31.318650 22656 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.319697 22656 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:31.315428 12496 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:31.218386 22865 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.377364 12678 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 28 index: 17. (index mismatch)
W20260709 12:14:31.380075 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 117094us
W20260709 12:14:31.374325 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 115173us
I20260709 12:14:31.257535 23030 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:31.230945 23040 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:14:31.391566 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 166378us
I20260709 12:14:31.397765 23210 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:31.397974 22656 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:31.398494 23141 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:31.399010 23141 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:31.398661 23210 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.399461 23141 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:14:31.398223 23206 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.090s	user 0.000s	sys 0.000s
I20260709 12:14:31.401403 23210 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:31.401669 23072 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:31.414475 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:31.415288 22701 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:31.416918 22960 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:31.256886 22934 consensus_queue.cc:1048] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.161s
I20260709 12:14:31.419481 22865 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:31.419891 23188 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:31.423483 23124 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:31.428656 12598 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 25 index: 17. Preceding OpId from leader: term: 28 index: 19. (index mismatch)
I20260709 12:14:31.429946 12496 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:31.260618 22990 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.415323 12672 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 29.
I20260709 12:14:31.387036 22714 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:31.446314 12603 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
I20260709 12:14:31.446375 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:31.454557 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:31.387825 22852 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:31.443749 23190 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:31.458967 13331 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:31.466614 12671 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 22 index: 14. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:14:31.446830 23124 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:31.474210 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.461259 23040 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:14:31.463971 22592 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:31.490600 22990 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:31.490600 22852 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:31.492153 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:31.495304 23205 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.099s	user 0.000s	sys 0.001s
I20260709 12:14:31.468972 22865 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:31.482755 23021 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:31.506626 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:31.497546 22865 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.507135 23021 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.508471 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:31.509276 22865 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:31.475124 22960 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.511106 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:31.510883 22960 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.517141 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:31.479983 23014 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:31.527842 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:31.525566 22701 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:31.528946 22701 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:31.531831 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.532195 22701 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:31.546958 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:31.514428 23155 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.034s
I20260709 12:14:31.553591 22714 consensus_queue.cc:1059] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:14:31.555357 23106 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:31.555909 22714 raft_consensus.cc:3055] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Stepping down as leader of term 24
I20260709 12:14:31.555989 23106 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:14:31.556420 22714 raft_consensus.cc:740] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:31.557932 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 308104us
I20260709 12:14:31.524151 22990 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:31.376894 12421 catalog_manager.cc:5697] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 12 to 18, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 18 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:14:31.565245 13305 leader_election.cc:336] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:31.557246 22937 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:31.576797 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.581121 12494 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:31.514056 23124 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:31.583031 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.541742 23210 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.533800 23040 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:31.552095 23217 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:31.566411 22852 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Leader election won for term 34
W20260709 12:14:31.590903 13305 leader_election.cc:336] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:31.607556 22847 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:14:31.608271 22847 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:31.611528 23146 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:31.612269 23146 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:31.562363 23213 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.067s
I20260709 12:14:31.590284 23217 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:31.614804 23217 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:31.615706 22990 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.616318 22990 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.618412 23175 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.618996 23175 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.620358 23075 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:31.621096 23175 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:31.585559 22937 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:31.617871 23136 consensus_meta.cc:287] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.728s	user 0.008s	sys 0.002s
I20260709 12:14:31.623842 22937 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:31.624202 22990 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:31.624467 12599 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51792 (request call id 3877) took 1146 ms. Trace:
I20260709 12:14:31.624891 12599 rpcz_store.cc:276] 0709 12:14:30.478100 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.490906 (+ 12806us) service_pool.cc:224] Handling call
0709 12:14:31.624261 (+1133355us) spinlock_profiling.cc:243] Waited 511 ms on lock 0x7b54001176d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:31.624380 (+   119us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1175566208}
I20260709 12:14:31.609483 22847 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:31.625416 23146 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:31.625794 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4724) took 1764 ms. Trace:
I20260709 12:14:31.627094 12679 rpcz_store.cc:276] 0709 12:14:29.861829 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.968944 (+1107115us) service_pool.cc:224] Handling call
0709 12:14:30.979656 (+ 10712us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:31.254961 (+275305us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:31.255016 (+    55us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:31.265278 (+ 10262us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:14:31.265348 (+    70us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:31.625548 (+360200us) spinlock_profiling.cc:243] Waited 266 ms on lock 0x7b54003bd654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:31.625596 (+    48us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:31.625740 (+   144us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":611060480}
I20260709 12:14:31.630378 23021 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:31.631453 23195 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:31.632319 23124 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:31.635389 12672 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:14:31.640961 22852 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:31.643314 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:31.651273 23208 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 32.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.118s
I20260709 12:14:31.652349 23208 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:14:31.652706 23208 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:31.652820 23146 raft_consensus.cc:3060] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:31.654994 12676 raft_consensus.cc:1217] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Deduplicated request from leader. Original: 21.13->[23.14-30.17]   Dedup: 25.15->[28.16-30.17]
I20260709 12:14:31.655601 12676 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 30. Current term is 31. Ops: [23.14-30.17]
I20260709 12:14:31.657189 12675 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
I20260709 12:14:31.657999 23215 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:31.658646 23215 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:31.660171 22625 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.524s	user 0.008s	sys 0.004s
I20260709 12:14:31.661073 23091 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:31.661283 23215 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:31.648375 22990 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.661791 23143 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.662176 22990 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.662385 23143 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.664404 23143 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:31.665481 23040 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.22, Last appended by leader: 22, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:31.667039 13332 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:31.670631 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4728) took 1807 ms. Trace:
I20260709 12:14:31.671077 12680 rpcz_store.cc:276] 0709 12:14:29.863204 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.950442 (+1087238us) service_pool.cc:224] Handling call
0709 12:14:30.950924 (+   482us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:31.069234 (+118310us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:31.670364 (+601130us) spinlock_profiling.cc:243] Waited 117 ms on lock 0x7b54003bd154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:31.670567 (+   203us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":269781760}
I20260709 12:14:31.566781 22714 consensus_queue.cc:260] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 24, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.681181 12421 catalog_manager.cc:5697] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 15 to 16, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 16 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:31.683166 22960 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:31.598933 23210 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.684048 12421 catalog_manager.cc:5697] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 19 to 21, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 21 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:31.686264 23210 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:31.687369 12679 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 31 index: 18. Preceding OpId from leader: term: 32 index: 19. (index mismatch)
W20260709 12:14:31.688889 12646 leader_election.cc:336] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:31.642025 22852 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.21, Last appended by leader: 21, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:31.699107 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 236483us
I20260709 12:14:31.702414 13332 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 34 index: 22. (index mismatch)
W20260709 12:14:31.710157 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:31.711184 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:31.716507 12671 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 31: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:31.664422 22990 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:31.721580 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:31.726936 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 158291us
I20260709 12:14:31.707679 23210 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.727944 12569 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:31.737483 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:31.653846 23208 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.22, Last appended by leader: 22, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:31.744316 12494 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:31.742403 23017 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.817s	user 0.010s	sys 0.005s
I20260709 12:14:31.755152 22851 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:31.758450 23208 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:31.740953 23190 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election won for term 29
W20260709 12:14:31.761561 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:31.748376 12643 leader_election.cc:336] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.205s (SENT)
I20260709 12:14:31.725927 23123 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:31.768851 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:31.769768 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 257244us
I20260709 12:14:31.754436 22990 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:31.746829 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:31.762904 23190 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:31.761085 22960 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.753875 23210 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.787096 23215 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:14:31.789769 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 206764us
I20260709 12:14:31.792822 23017 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:31.779884 22990 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:31.793402 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46679.3us
I20260709 12:14:31.796686 23222 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:31.785625 23190 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:14:31.793205 23212 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.124s	user 0.000s	sys 0.002s
I20260709 12:14:31.802510 12671 raft_consensus.cc:1275] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:14:31.768224 23141 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.766394 23136 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.031s
I20260709 12:14:31.788039 22960 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:31.798065 23017 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:31.800386 22990 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:31.772701 22851 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:31.784528 22625 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.814236 22953 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:14:31.820305 12679 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 26. Current term is 27. Ops: []
I20260709 12:14:31.828713 22847 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:31.818881 23017 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:31.828860 22960 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:31.843086 22592 consensus_queue.cc:1059] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:14:31.844281 22592 consensus_queue.cc:1059] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:14:31.845081 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:31.846419 12680 raft_consensus.cc:1275] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 24 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:14:31.848096 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:31.859824 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 119621us
I20260709 12:14:31.855932 12679 raft_consensus.cc:1240] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 24. Current term is 25. Ops: [23.15-24.16]
I20260709 12:14:31.860649 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:31.853704 23106 consensus_queue.cc:1048] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.011s
I20260709 12:14:31.866859 23218 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:14:31.835587 23143 raft_consensus.cc:3055] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:14:31.862725 12606 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51792 (request call id 3879) took 1371 ms. Trace:
I20260709 12:14:31.878285 23226 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.878767 22851 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:31.879850 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:31.878890 23226 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.830636 22930 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.031s
I20260709 12:14:31.884501 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:31.885488 12671 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 29: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:31.785198 12421 catalog_manager.cc:5697] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 14 to 19, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 19 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:31.874212 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:31.868350 22952 consensus_meta.cc:287] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.660s	user 0.012s	sys 0.001s
I20260709 12:14:31.894358 22960 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:31.895213 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:31.894441 23226 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:31.823323 23222 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:31.901017 23210 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:31.901819 23219 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.238s	user 0.003s	sys 0.001s
I20260709 12:14:31.903236 23091 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:31.904117 23223 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.068s	user 0.000s	sys 0.000s
I20260709 12:14:31.904584 13332 raft_consensus.cc:1275] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 31 index: 22. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
I20260709 12:14:31.867652 12606 rpcz_store.cc:276] 0709 12:14:30.491458 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:30.673599 (+182141us) service_pool.cc:224] Handling call
0709 12:14:30.680142 (+  6543us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:31.072753 (+392611us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:31.072783 (+    30us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:31.111906 (+ 39123us) log.cc:844] Serialized 26 byte log entry
0709 12:14:31.433257 (+321351us) raft_consensus.cc:1645] Marking committed up to 22
0709 12:14:31.433290 (+    33us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:31.546410 (+113120us) spinlock_profiling.cc:243] Waited 381 ms on lock 0x7b54002e68d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:31.546468 (+    58us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:31.735273 (+188805us) spinlock_profiling.cc:243] Waited 20.8 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:14:31.862484 (+127211us) raft_consensus.cc:1685] finished
0709 12:14:31.862505 (+    21us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:31.862653 (+   148us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":924884608,"thread_start_us":209444,"threads_started":1,"wal-append.queue_time_us":421657}
I20260709 12:14:31.913151 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:31.841250 22625 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.870293 23143 raft_consensus.cc:740] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:31.867336 23218 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:31.930442 12680 raft_consensus.cc:2393] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 27: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:31.933105 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:31.930898 22960 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.935336 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:31.942444 12679 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:31.926864 23215 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.870790 23217 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:31.929838 23161 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.953398 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:31.934254 23210 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.969383 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:31.966929 12671 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 29.
I20260709 12:14:31.972994 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:31.973580 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 90657.8us
I20260709 12:14:31.974030 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:31.975226 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:31.924296 23226 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.031s
I20260709 12:14:31.934193 23123 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:31.927359 22952 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.036s
I20260709 12:14:31.980396 12677 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 33. Current term is 34. Ops: [33.21-33.21]
I20260709 12:14:31.982013 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:31.931851 23091 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.987869 12597 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4484) took 1118 ms. Trace:
I20260709 12:14:31.969584 22934 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.953763 23030 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:31.990594 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:31.979900 12421 catalog_manager.cc:5697] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 14 to 15, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 15 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:31.993077 12597 rpcz_store.cc:276] 0709 12:14:30.869405 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:31.121848 (+252443us) service_pool.cc:224] Handling call
0709 12:14:31.208659 (+ 86811us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:31.285669 (+ 77010us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:31.285713 (+    44us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:31.430679 (+144966us) log.cc:844] Serialized 52 byte log entry
0709 12:14:31.640773 (+210094us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:14:31.640836 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:31.640904 (+    68us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:31.830589 (+189685us) spinlock_profiling.cc:243] Waited 21.6 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:14:31.980779 (+150190us) raft_consensus.cc:1685] finished
0709 12:14:31.980803 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:31.987794 (+  6991us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":13249,"raft.run_cpu_time_us":91,"raft.run_wall_time_us":93,"spinlock_wait_cycles":49755008,"thread_start_us":174214,"threads_started":1,"wal-append.queue_time_us":202248}
W20260709 12:14:31.989116 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:31.996712 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:32.000187 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:32.001824 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:32.000303 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:32.002918 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.004447 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:32.004962 13333 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 31: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:14:32.006735 12599 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 34 index: 22. (index mismatch)
I20260709 12:14:32.003616 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:31.966384 22887 consensus_queue.cc:1048] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.033s
I20260709 12:14:32.014873 12598 raft_consensus.cc:1240] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 24. Current term is 25. Ops: [24.16-24.16]
I20260709 12:14:32.002831 22934 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:32.017727 22714 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:32.019627 22934 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:32.020323 23030 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:32.022683 23030 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:32.023757 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:32.024214 23091 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:32.024425 12603 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:32.027158 12421 catalog_manager.cc:5697] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 18 to 20. New cstate: current_term: 20 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:31.999905 23210 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:32.030615 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.030840 23210 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:32.031948 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.014484 23072 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:32.032821 23072 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.972913 23161 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:32.019161 23143 consensus_queue.cc:260] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:32.036496 23175 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.037142 23143 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:32.038336 12604 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:32.014295 22847 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.014873 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.039660 22847 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:32.040472 12600 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:32.041567 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.042471 23233 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.042750 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.042979 22952 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.043274 23146 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:32.043524 22952 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:31.990751 23136 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 16, Time since last communication: 0.057s
I20260709 12:14:32.045375 23222 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:32.033439 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.047477 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.048857 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.055651 22625 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:32.051155 12672 raft_consensus.cc:1240] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 26. Current term is 27. Ops: []
I20260709 12:14:32.055416 12597 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29.
I20260709 12:14:32.043211 23233 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:32.053256 22960 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:31.995553 23215 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:32.064963 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:32.006443 23141 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:32.064030 23233 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:32.029834 12421 catalog_manager.cc:5697] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 14 to 16. New cstate: current_term: 16 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:32.066474 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:32.065950 23017 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:32.045833 22952 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:32.067838 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 71423.1us
I20260709 12:14:32.068070 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.069345 23017 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:32.069574 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:32.070832 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.070969 23225 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:32.071512 23225 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.071924 23225 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:32.072077 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.072880 12597 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:14:32.073750 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.075297 12421 catalog_manager.cc:5697] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 14 to 16, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 16 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:32.077057 22960 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:14:32.078855 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.036680 23161 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:32.044029 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.083740 12600 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:32.084789 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.084697 22990 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:32.084820 12421 catalog_manager.cc:5697] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 12 to 16, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 16 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:32.069468 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:32.081597 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 210169us
I20260709 12:14:32.088467 22990 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:32.088619 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.082480 12597 raft_consensus.cc:2393] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 25: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:32.093849 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:32.094260 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.094906 13339 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:32.095119 12674 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 25.
I20260709 12:14:32.081669 23075 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.098884 23075 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:32.099494 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.099745 22847 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:32.101169 23075 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:32.101702 23075 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.102463 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.103152 12604 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:14:32.104285 23075 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:32.106885 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.089993 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 22 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.078895 12603 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:14:32.065297 23215 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:32.112396 12606 raft_consensus.cc:2410] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 26 because replica has last-logged OpId of term: 24 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 22 index: 14.
I20260709 12:14:32.121937 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.122958 13304 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:14:32.124257 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:32.127810 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:32.128934 13330 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 34.
I20260709 12:14:32.130821 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.132103 12598 raft_consensus.cc:2410] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 33 because replica has last-logged OpId of term: 31 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 29 index: 15.
I20260709 12:14:32.082695 23030 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.037691 23175 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:32.138582 12646 leader_election.cc:336] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.147384 12421 catalog_manager.cc:5697] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 7 to 17, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 17 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:32.150408 23217 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:32.151824 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:32.154415 23213 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:32.155884 23072 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:32.052937 23218 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 33.21, Last appended by leader: 21, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:32.156611 23030 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:32.169035 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:32.168010 12646 leader_election.cc:336] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.099489 23190 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:32.169797 12569 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:32.180691 23215 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:32.181057 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:32.181393 23215 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:32.183632 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:32.184298 12674 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 31
I20260709 12:14:32.187719 23030 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:32.176898 13339 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29.
I20260709 12:14:32.181756 12602 raft_consensus.cc:2376] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27. Re-sending same reply.
I20260709 12:14:32.164568 23143 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:32.192209 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:32.192235 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:32.177474 12600 raft_consensus.cc:2393] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 28: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:32.169214 23213 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:32.203615 12674 raft_consensus.cc:2468] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 31.
W20260709 12:14:32.192979 12646 leader_election.cc:336] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.198369 12643 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:32.209434 12601 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 29: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:32.189448 23072 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:14:32.211984 13304 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:32.216205 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:32.217262 12602 raft_consensus.cc:2410] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 31 because replica has last-logged OpId of term: 30 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 16.
W20260709 12:14:32.221513 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:32.207453 23213 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:32.212698 22625 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:32.235586 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:32.232070 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.244060 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.215143 23072 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.194736 23143 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:32.265980 23143 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:32.228996 23217 raft_consensus.cc:2804] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:14:32.267760 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.268488 23217 raft_consensus.cc:697] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 31 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:32.269065 12598 raft_consensus.cc:2410] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 29 because replica has last-logged OpId of term: 27 index: 17, which is greater than that of the candidate, which has last-logged OpId of term: 24 index: 16.
I20260709 12:14:32.270233 23217 consensus_queue.cc:237] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:32.271111 23232 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:14:32.271186 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:32.272553 12603 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 29: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:32.274425 22953 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:14:32.274952 23040 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:14:32.276463 23040 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:14:32.208565 22990 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.034s
I20260709 12:14:32.278666 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.273857 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 34 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.244915 12597 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:32.280053 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:14:32.203961 22930 consensus_queue.cc:1175] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "193a40da6e364ead8ab561e14c3cc8d1" responder_term: 34 status { last_received { term: 31 index: 20 } last_committed_idx: 20 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 33. Current term is 34. Ops: [33.21-33.21]" } } last_received_current_leader { term: 0 index: 0 } }
I20260709 12:14:32.252882 23075 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:32.258535 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:32.259298 23190 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:32.263558 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.281911 12598 raft_consensus.cc:2376] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 35. Re-sending same reply.
I20260709 12:14:32.281702 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:32.296656 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:32.229214 23141 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.065s
W20260709 12:14:32.297451 12646 leader_election.cc:336] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.296743 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.297566 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.280652 23072 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:14:32.304006 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:32.300871 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:32.301008 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.249217 22625 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:32.302248 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.302897 12673 raft_consensus.cc:1275] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:14:32.316607 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.306600 13339 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 29: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:32.304029 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.301889 12597 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:14:32.299026 12671 raft_consensus.cc:1275] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 32 index: 17. Preceding OpId from leader: term: 33 index: 18. (index mismatch)
I20260709 12:14:32.302616 12675 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
W20260709 12:14:32.312376 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 137507us
W20260709 12:14:32.319973 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:32.313197 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:32.320967 12601 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:14:32.300493 23208 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.535s	user 0.007s	sys 0.005s
I20260709 12:14:32.317749 12600 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:14:32.306591 12677 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 32.
W20260709 12:14:32.345749 12646 leader_election.cc:336] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.351059 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:32.351753 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.352108 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:32.349459 12646 leader_election.cc:336] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.355656 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:32.357414 12646 leader_election.cc:336] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.358179 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:32.364671 12646 leader_election.cc:336] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.353083 12598 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29.
I20260709 12:14:32.371814 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.372604 12675 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:14:32.332798 22592 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:32.378834 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.333897 22952 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:32.385854 23124 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.399524 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.412652 12677 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 33.
I20260709 12:14:32.414232 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:32.387190 12646 leader_election.cc:336] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.379374 22592 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.430192 12600 raft_consensus.cc:2393] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 26: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:32.431999 12601 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:14:32.365176 23017 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.038s
I20260709 12:14:32.380131 12679 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:32.417218 12646 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:32.407186 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.397014 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.413837 12494 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:32.437294 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.402372 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:32.441016 12599 raft_consensus.cc:2468] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:14:32.439358 12673 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 32.
W20260709 12:14:32.443735 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 56606.1us
I20260709 12:14:32.438519 12671 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
W20260709 12:14:32.444653 12646 leader_election.cc:336] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.348253 22714 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.066s
I20260709 12:14:32.411666 23218 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:32.458735 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.467334 12672 raft_consensus.cc:2468] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
W20260709 12:14:32.472999 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 59242.5us
I20260709 12:14:32.402289 22953 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:32.357220 22852 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.079s
I20260709 12:14:32.480015 12599 raft_consensus.cc:1275] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 25 index: 17. (index mismatch)
I20260709 12:14:32.483757 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.437458 22592 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:32.436787 23141 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:32.485056 12646 leader_election.cc:336] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.484438 12671 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:14:32.488523 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.494751 12673 raft_consensus.cc:2468] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 33.
I20260709 12:14:32.378633 23161 consensus_queue.cc:1048] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.086s
I20260709 12:14:32.438699 23175 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:32.493933 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.497301 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:32.498106 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.490736 12494 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:32.455345 22960 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:14:32.501055 12646 leader_election.cc:336] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.503858 12677 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:32.500543 12603 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:14:32.510264 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:32.515770 12672 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:32.517553 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:32.513686 12646 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.518868 12646 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:32.523151 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:32.518306 12679 raft_consensus.cc:2468] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:32.519095 12494 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:32.437672 23030 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 26.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.063s
I20260709 12:14:32.531692 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.432459 23124 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:32.484668 22953 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.532475 12671 raft_consensus.cc:2468] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:32.532580 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.540412 12673 raft_consensus.cc:2393] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 31: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:14:32.474677 23195 consensus_queue.cc:1048] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 12:14:32.547286 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.550539 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.552078 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:32.552819 12671 raft_consensus.cc:2393] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 31: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:32.553203 23017 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:32.553668 23017 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.554394 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:32.554863 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.555490 12673 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:14:32.555729 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:32.556278 12679 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:32.556406 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.549664 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.561995 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:32.563514 12671 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:14:32.512418 22887 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:14:32.569325 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:32.581887 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58831.1us
I20260709 12:14:32.532383 23136 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:32.583376 12571 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:32.531200 23215 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:32.589221 12675 raft_consensus.cc:2410] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 33 because replica has last-logged OpId of term: 31 index: 18, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 17.
I20260709 12:14:32.560846 12677 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:14:32.554028 23017 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:32.540616 22953 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:32.615182 12606 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:14:32.606523 23074 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:32.618965 23074 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:32.619386 23074 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:14:32.626315 23225 consensus_meta.cc:287] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.533s	user 0.015s	sys 0.000s
W20260709 12:14:32.587656 23223 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:14:32.628496 12671 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 29 index: 19. (index mismatch)
I20260709 12:14:32.588052 23091 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.526708 23141 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:32.528136 23072 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:32.589711 23136 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.642146 12571 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:32.637286 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 304543us
I20260709 12:14:32.647780 22952 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.648290 23143 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:32.637091 12671 pending_rounds.cc:85] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 16
W20260709 12:14:32.655694 13305 leader_election.cc:336] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.657290 13305 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:32.527587 22625 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:32.654090 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:32.670560 12671 pending_rounds.cc:107] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 30.17
I20260709 12:14:32.672873 12671 raft_consensus.cc:2889] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:32.679181 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 165539us
I20260709 12:14:32.649972 12496 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:32.683504 12646 leader_election.cc:336] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:32.698002 12646 leader_election.cc:336] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.656339 23136 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:14:32.701656 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 51825.6us
W20260709 12:14:32.702307 12646 leader_election.cc:336] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.709698 12496 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:32.716755 12646 leader_election.cc:336] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.726065 12496 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:32.733449 22851 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:32.735533 22953 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:32.680950 23190 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election won for term 33
W20260709 12:14:32.697105 23220 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.059s	user 0.000s	sys 0.001s
W20260709 12:14:32.739493 12646 leader_election.cc:336] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.745819 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:32.633658 23225 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:32.724345 23143 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:32.755663 23072 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 35 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:32.757282 23124 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:32.746105 12646 leader_election.cc:336] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.758392 22990 raft_consensus.cc:2804] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:14:32.696944 22937 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:32.759974 12496 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:32.757385 23143 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:32.661603 23195 consensus_queue.cc:1048] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.082s
I20260709 12:14:32.713963 23175 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:14:32.764981 12679 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:14:32.652418 23091 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:32.764375 12646 leader_election.cc:336] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.779785 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:32.738407 23226 raft_consensus.cc:2804] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:32.808209 12496 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:14:32.776727 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 189936us
I20260709 12:14:32.680629 22952 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:32.760905 23190 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.812727 12496 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:32.810684 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 46357.5us
W20260709 12:14:32.815579 12646 leader_election.cc:336] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.811378 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:32.817106 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 298069us
I20260709 12:14:32.763204 22990 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:32.818361 12646 leader_election.cc:336] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:32.777586 23141 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:32.864917 12646 leader_election.cc:336] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:32.942613 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 92168.3us
I20260709 12:14:32.802964 23175 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.806192 22714 raft_consensus.cc:2804] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:32.811383 23226 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:32.814368 23143 raft_consensus.cc:2804] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:32.819775 23190 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:33.083400 23143 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:33.083971 23143 raft_consensus.cc:3060] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:32.816037 23075 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:32.759233 22851 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:33.088233 22851 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:33.089833 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:32.818315 23233 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:32.839305 22990 raft_consensus.cc:3060] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:32.817849 12494 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:32.842070 22852 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:14:33.009091 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 144230us
I20260709 12:14:32.805524 23017 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:33.013221 12672 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:14:33.038767 23228 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:14:33.039430 23240 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:33.039937 23253 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:33.040518 23252 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:14:33.060439 23232 consensus_meta.cc:287] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.788s	user 0.008s	sys 0.003s
W20260709 12:14:33.064628 23236 log.cc:927] Time spent T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.526s	user 0.002s	sys 0.008s
I20260709 12:14:33.067415 22952 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:33.068409 23215 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:33.069361 23040 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:33.069612 22847 consensus_queue.cc:1048] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:14:33.070189 23124 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:14:33.070744 23244 log.cc:927] Time spent T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.390s	user 0.001s	sys 0.000s
I20260709 12:14:33.070901 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:33.152701 13332 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:14:33.071769 23247 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.269s	user 0.000s	sys 0.000s
W20260709 12:14:33.072887 23260 log.cc:927] Time spent T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.080s	user 0.000s	sys 0.000s
W20260709 12:14:33.073493 23234 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.490s	user 0.001s	sys 0.000s
I20260709 12:14:33.047518 22714 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:33.047811 23175 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:33.049949 23226 raft_consensus.cc:3060] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:14:33.079949 23248 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.366s	user 0.000s	sys 0.001s
W20260709 12:14:33.106283 12646 leader_election.cc:336] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:33.109406 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 291634us
I20260709 12:14:33.145030 23232 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
W20260709 12:14:33.132340 12598 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.586s	user 0.013s	sys 0.024s
W20260709 12:14:33.146895 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 471445us
I20260709 12:14:33.188335 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:33.111531 23240 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:33.190616 12571 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:33.194597 12571 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:33.130226 23228 raft_consensus.cc:697] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:33.108038 23141 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:33.188941 12598 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.654s	user 0.020s	sys 0.026s
W20260709 12:14:33.170398 13302 leader_election.cc:336] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 35 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.594s (SENT)
I20260709 12:14:33.202442 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:33.172505 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 66257us
W20260709 12:14:33.205076 12644 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.636s	user 0.002s	sys 0.009s
W20260709 12:14:33.205430 12644 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.641s	user 0.002s	sys 0.009s
W20260709 12:14:33.117820 12602 consensus_meta.cc:287] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.827s	user 0.009s	sys 0.000s
I20260709 12:14:33.206374 12602 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 31.
W20260709 12:14:33.208489 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 398716us
I20260709 12:14:33.188206 12494 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:33.209429 12643 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:33.203900 12571 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:33.168430 22714 raft_consensus.cc:3060] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:14:33.201174 23213 consensus_meta.cc:287] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.951s	user 0.007s	sys 0.007s
W20260709 12:14:33.201757 23074 consensus_meta.cc:287] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.582s	user 0.012s	sys 0.000s
I20260709 12:14:33.197567 23240 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:33.157609 23215 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:33.287384 13333 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 29. Current term is 30. Ops: [29.19-29.19]
I20260709 12:14:33.291162 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:33.293164 12646 leader_election.cc:336] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:33.160880 23040 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:33.277729 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 48734.8us
W20260709 12:14:33.275084 23248 log.cc:927] Time spent T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:14:33.296203 12571 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:14:33.294390 12646 leader_election.cc:336] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:33.300256 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1232) took 1347 ms. Trace:
I20260709 12:14:33.299844 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:33.302387 12678 rpcz_store.cc:276] 0709 12:14:31.952569 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:31.952715 (+   146us) service_pool.cc:224] Handling call
0709 12:14:31.970029 (+ 17314us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:31.984689 (+ 14660us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:31.984734 (+    45us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:32.022378 (+ 37644us) log.cc:844] Serialized 26 byte log entry
0709 12:14:32.321802 (+299424us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:14:32.321861 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:32.321949 (+    88us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:33.300008 (+978059us) raft_consensus.cc:1685] finished
0709 12:14:33.300032 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:33.300197 (+   165us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1540,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":5632,"thread_start_us":298992,"threads_started":1,"wal-append.queue_time_us":463184}
W20260709 12:14:33.280105 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 70750.9us
I20260709 12:14:33.276189 23233 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:14:33.302023 12646 leader_election.cc:336] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:33.378723 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 76744us
I20260709 12:14:33.283345 22937 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:14:33.371923 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:33.375614 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:33.383189 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1235) took 1414 ms. Trace:
W20260709 12:14:33.377291 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 81187.5us
W20260709 12:14:33.379449 12646 leader_election.cc:336] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:33.387861 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:33.200546 23141 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:33.298352 23030 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:33.396982 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:33.400924 12646 leader_election.cc:336] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:33.383839 12680 rpcz_store.cc:276] 0709 12:14:31.968367 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:31.968646 (+   279us) service_pool.cc:224] Handling call
0709 12:14:31.971464 (+  2818us) raft_consensus.cc:1471] Updating replica for 3 ops
0709 12:14:31.989021 (+ 17557us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:31.989258 (+   237us) raft_consensus.cc:1539] Triggering prepare for 3 ops
0709 12:14:31.999788 (+ 10530us) log.cc:844] Serialized 78 byte log entry
0709 12:14:32.172928 (+173140us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:32.466843 (+293915us) log.cc:844] Serialized 14 byte log entry
0709 12:14:32.479815 (+ 12972us) log.cc:844] Serialized 14 byte log entry
0709 12:14:32.480244 (+   429us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:32.480341 (+    97us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:32.733089 (+252748us) spinlock_profiling.cc:243] Waited 5.19 ms on lock 0x7b4800060858. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:14:33.063901 (+330812us) spinlock_profiling.cc:243] Waited 17.5 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:14:33.082857 (+ 18956us) spinlock_profiling.cc:243] Waited 12.4 ms on lock 0x7b1800334900. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485933522 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:14:33.382917 (+300060us) raft_consensus.cc:1685] finished
0709 12:14:33.382941 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:33.383119 (+   178us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":77641,"prepare.queue_time_us":580104,"prepare.run_cpu_time_us":99,"prepare.run_wall_time_us":3390,"raft.queue_time_us":12908,"raft.run_cpu_time_us":3824,"raft.run_wall_time_us":273182,"spinlock_wait_cycles":80732160,"thread_start_us":382160,"threads_started":2,"wal-append.queue_time_us":410380}
I20260709 12:14:33.448211 13332 raft_consensus.cc:2468] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 35.
I20260709 12:14:33.452172 13330 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 34. Current term is 35. Ops: [34.22-34.22]
I20260709 12:14:33.288978 23074 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:33.457618 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 68402.2us
W20260709 12:14:33.460711 12646 leader_election.cc:336] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:33.461347 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 64461.2us
I20260709 12:14:33.462410 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:33.463037 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 292680us
I20260709 12:14:33.284011 23213 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:33.401908 23017 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:33.402925 23238 log.cc:927] Time spent T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.295s	user 0.001s	sys 0.000s
I20260709 12:14:33.470439 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:33.370232 23270 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:33.382598 23155 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:33.379839 22847 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:33.378031 23195 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:14:33.378237 23263 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:33.450212 23253 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:33.449788 23235 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:33.383762 23091 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:33.450378 22852 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.047s
I20260709 12:14:33.490177 23282 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:33.498413 23146 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:14:33.500757 22592 consensus_meta.cc:287] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.985s	user 0.005s	sys 0.008s
W20260709 12:14:33.582968 12569 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.545s	user 0.002s	sys 0.015s
I20260709 12:14:33.494475 23225 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:33.502295 23279 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:33.493454 23254 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:33.492954 22953 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:33.502017 23190 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:33.535684 13339 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 25. Current term is 26. Ops: []
I20260709 12:14:33.403909 23233 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:33.584251 12569 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.567s	user 0.002s	sys 0.016s
W20260709 12:14:33.585561 12646 leader_election.cc:336] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:33.528282 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 57945.7us
I20260709 12:14:33.540206 22960 raft_consensus.cc:2764] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election decision vote started in defunct term 29: won
I20260709 12:14:33.534708 23017 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:33.589202 13305 leader_election.cc:336] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:33.612349 23255 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:14:33.491302 23270 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:33.526842 22847 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:33.525992 23195 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:33.570354 22887 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:14:33.483955 22930 consensus_queue.cc:1048] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.104s
I20260709 12:14:33.620368 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:33.649386 12674 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4760) took 1099 ms. Trace:
W20260709 12:14:33.649996 11318 thread.cc:527] Waited for 1000ms trying to join with rpc worker (tid 12502)
I20260709 12:14:33.652139 23190 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:33.657213 23233 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:33.620569 23091 raft_consensus.cc:3055] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:14:33.657989 23030 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:33.659076 23265 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.371s	user 0.000s	sys 0.001s
W20260709 12:14:33.659474 23246 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.669s	user 0.001s	sys 0.000s
I20260709 12:14:33.578922 23143 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:33.571674 23263 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:33.659981 23215 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:33.660746 23290 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:33.661125 23040 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:33.599911 22592 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:33.592382 23146 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:33.643663 23074 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:33.666385 23269 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.163s	user 0.000s	sys 0.001s
W20260709 12:14:33.667356 23261 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.384s	user 0.000s	sys 0.001s
W20260709 12:14:33.668527 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 85112.8us
W20260709 12:14:33.765690 13305 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 106 similar messages]
W20260709 12:14:33.766644 13305 leader_election.cc:336] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:33.767264 22592 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:33.541718 23218 consensus_meta.cc:287] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 1.062s	user 0.006s	sys 0.007s
I20260709 12:14:33.609848 23235 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:33.768186 13305 consensus_peers.cc:597] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:33.771111 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:33.671227 23270 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:14:33.659070 23283 log.cc:927] Time spent T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.066s	user 0.000s	sys 0.000s
W20260709 12:14:33.680905 12646 leader_election.cc:336] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:33.527233 23155 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:33.618781 23277 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:33.682780 13331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 1204) took 1603 ms. Trace:
I20260709 12:14:33.689453 12674 rpcz_store.cc:276] 0709 12:14:32.549583 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.993081 (+443498us) service_pool.cc:224] Handling call
0709 12:14:33.045836 (+ 52755us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:33.094604 (+ 48768us) raft_consensus.cc:1534] Early marking committed up to index 13
0709 12:14:33.094645 (+    41us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:33.109110 (+ 14465us) log.cc:844] Serialized 26 byte log entry
0709 12:14:33.287894 (+178784us) raft_consensus.cc:1645] Marking committed up to 13
0709 12:14:33.287938 (+    44us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:33.288010 (+    72us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:33.649117 (+361107us) raft_consensus.cc:1685] finished
0709 12:14:33.649144 (+    27us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:33.649321 (+   177us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":2478,"raft.queue_time_us":13866,"raft.run_cpu_time_us":270,"raft.run_wall_time_us":28674,"thread_start_us":119688,"threads_started":1,"wal-append.queue_time_us":445927}
I20260709 12:14:33.701228 12679 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:40246 (request call id 1241) took 1209 ms. Trace:
W20260709 12:14:33.683737 23271 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.472s	user 0.000s	sys 0.001s
W20260709 12:14:33.744598 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 124283us
I20260709 12:14:33.678494 23225 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:33.686832 22847 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:33.680218 23255 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:33.686703 23195 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:33.741811 23030 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:33.541136 23228 consensus_queue.cc:237] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:33.698464 23091 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:33.757841 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 146408us
I20260709 12:14:33.668947 23017 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:33.760462 13339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 1210) took 1132 ms. Trace:
I20260709 12:14:33.669046 12569 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:33.773180 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:33.773823 23256 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:14:33.773846 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 94668us
I20260709 12:14:33.753813 23291 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:33.776125 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:33.777408 12679 rpcz_store.cc:276] 0709 12:14:32.492112 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.812017 (+319905us) service_pool.cc:224] Handling call
0709 12:14:32.812393 (+   376us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:33.505015 (+692622us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:33.505048 (+    33us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:33.505093 (+    45us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:14:33.505127 (+    34us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:33.700743 (+195616us) spinlock_profiling.cc:243] Waited 692 ms on lock 0x7b54000c26d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:33.700853 (+   110us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:33.701121 (+   268us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1592378624}
I20260709 12:14:33.779088 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 34 candidate_status { last_received { term: 31 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:33.779101 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:33.779978 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:33.795522 12601 raft_consensus.cc:2376] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 31. Re-sending same reply.
I20260709 12:14:33.796087 12599 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:14:33.775944 13331 rpcz_store.cc:276] 0709 12:14:32.075309 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.075437 (+   128us) service_pool.cc:224] Handling call
0709 12:14:32.075761 (+   324us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:32.076170 (+   409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:32.076201 (+    31us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:32.076774 (+   573us) log.cc:844] Serialized 26 byte log entry
0709 12:14:32.458177 (+381403us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:14:32.458240 (+    63us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:32.458347 (+   107us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:33.678289 (+1219942us) raft_consensus.cc:1685] finished
0709 12:14:33.678312 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:33.678475 (+   163us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4081,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":15639,"thread_start_us":380817,"threads_started":1,"wal-append.queue_time_us":765955}
I20260709 12:14:33.773904 22937 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:33.771029 23074 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:33.788620 13332 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
I20260709 12:14:33.788715 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4510) took 1770 ms. Trace:
I20260709 12:14:33.790136 13339 rpcz_store.cc:276] 0709 12:14:32.627957 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.628151 (+   194us) service_pool.cc:224] Handling call
0709 12:14:32.645089 (+ 16938us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:33.615679 (+970590us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:33.760112 (+144433us) spinlock_profiling.cc:243] Waited 890 ms on lock 0x7b540063ecd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:33.760382 (+   270us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2047718784}
I20260709 12:14:33.761858 23290 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:33.804411 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:33.777954 12645 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.606s	user 0.015s	sys 0.010s
I20260709 12:14:33.798331 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:33.797371 12604 rpcz_store.cc:276] 0709 12:14:32.018506 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.103700 (+ 85194us) service_pool.cc:224] Handling call
0709 12:14:32.104015 (+   315us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:32.104374 (+   359us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:32.104413 (+    39us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:32.104904 (+   491us) log.cc:844] Serialized 26 byte log entry
0709 12:14:32.565611 (+460707us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:32.565676 (+    65us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:32.565777 (+   101us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:33.658670 (+1092893us) spinlock_profiling.cc:243] Waited 3.38 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:14:33.783827 (+125157us) raft_consensus.cc:1685] finished
0709 12:14:33.783847 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:33.788637 (+  4790us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1107,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":11000,"spinlock_wait_cycles":7766784,"thread_start_us":460110,"threads_started":1,"wal-append.queue_time_us":653020}
W20260709 12:14:33.802764 23286 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.060s	user 0.000s	sys 0.001s
I20260709 12:14:33.803404 23218 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:14:33.806387 12645 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.798s	user 0.016s	sys 0.010s
I20260709 12:14:33.806077 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:33.809906 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1205) took 1531 ms. Trace:
I20260709 12:14:33.806815 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:33.808421 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1203) took 1531 ms. Trace:
I20260709 12:14:33.821949 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:33.823560 12599 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 31
I20260709 12:14:33.823714 12597 raft_consensus.cc:2393] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 33: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
W20260709 12:14:33.799119 12572 consensus_peers.cc:597] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:33.823398 12643 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:33.790325 22990 consensus_meta.cc:287] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.679s	user 0.012s	sys 0.000s
I20260709 12:14:33.807906 23279 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:14:33.825204 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 156241us
I20260709 12:14:33.821717 12605 rpcz_store.cc:276] 0709 12:14:32.278030 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.468747 (+190717us) service_pool.cc:224] Handling call
0709 12:14:32.499123 (+ 30376us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:32.507962 (+  8839us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:32.508014 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:32.527200 (+ 19186us) log.cc:844] Serialized 26 byte log entry
0709 12:14:32.648956 (+121756us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:14:32.649014 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:32.649119 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:33.809620 (+1160501us) raft_consensus.cc:1685] finished
0709 12:14:33.809649 (+    29us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:33.809849 (+   200us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":9301,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":12749,"thread_start_us":90439,"threads_started":1,"wal-append.queue_time_us":611134}
I20260709 12:14:33.831167 12679 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 33 index: 16. (index mismatch)
I20260709 12:14:33.809641 23155 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:33.806948 22960 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:33.851291 22960 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:33.851728 22960 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:33.852993 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:33.854494 23255 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:33.815166 23256 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:33.859925 23256 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:33.839963 23190 raft_consensus.cc:2804] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:14:33.860939 23190 raft_consensus.cc:697] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
W20260709 12:14:33.832422 12572 leader_election.cc:336] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:33.851321 23281 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.210s	user 0.000s	sys 0.001s
I20260709 12:14:33.848948 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:33.863077 12572 leader_election.cc:336] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:33.863354 23226 consensus_meta.cc:287] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.673s	user 0.010s	sys 0.001s
I20260709 12:14:33.863823 23226 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:33.864336 12572 consensus_peers.cc:597] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:33.864961 12572 consensus_peers.cc:597] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:33.865525 12572 consensus_peers.cc:597] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:33.847121 23279 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:33.868526 23279 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:33.869622 23284 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.301s	user 0.001s	sys 0.001s
I20260709 12:14:33.807705 23091 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:33.871326 23091 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:33.838979 22990 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:33.798439 23213 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:33.876856 23293 consensus_queue.cc:1059] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 25.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:14:33.832907 23291 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:33.806173 23290 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:33.832302 23215 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:33.878206 23290 raft_consensus.cc:3055] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Stepping down as leader of term 25
I20260709 12:14:33.878657 23290 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:33.878471 23215 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:33.879194 23188 raft_consensus.cc:3055] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:14:33.879626 23188 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:33.880499 23188 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:33.881276 23215 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:33.881520 23188 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:33.882421 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:33.879599 23290 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 25, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:33.883096 12597 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:14:33.780936 23277 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:33.808888 23074 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:33.887166 23074 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:33.888516 23030 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:33.833030 23218 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:33.892696 23218 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:33.892923 23146 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:33.832149 23253 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:33.808673 23235 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:33.784198 23225 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:33.861997 23190 consensus_queue.cc:237] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 33.21, Last appended by leader: 21, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:33.817970 22937 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:33.866039 12572 leader_election.cc:336] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:33.850651 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:33.922420 12606 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:33.822285 12600 rpcz_store.cc:276] 0709 12:14:32.277474 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.443061 (+165587us) service_pool.cc:224] Handling call
0709 12:14:32.478783 (+ 35722us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:32.490192 (+ 11409us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:32.490244 (+    52us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:32.524773 (+ 34529us) log.cc:844] Serialized 26 byte log entry
0709 12:14:32.813237 (+288464us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:14:32.813352 (+   115us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:32.813512 (+   160us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:33.086999 (+273487us) spinlock_profiling.cc:243] Waited 6.98 ms on lock 0x7b480024c258. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148d949aa1 00007f148d940b37 00007f148d9409ca 00007f148da3826f 00007f148da37f37 00007f148da4c55c 00007f148da4c508 00007f148da4c488 00007f148da4c443 00007f148da4b6ec 00007f148c975a96
0709 12:14:33.794771 (+707772us) raft_consensus.cc:1685] finished
0709 12:14:33.794796 (+    25us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:33.808352 (+ 13556us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":17196,"raft.queue_time_us":11701,"raft.run_cpu_time_us":10363,"raft.run_wall_time_us":585380,"spinlock_wait_cycles":16058240,"thread_start_us":221633,"threads_started":1,"wal-append.queue_time_us":605992}
W20260709 12:14:33.897729 12646 leader_election.cc:336] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:33.902014 12599 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:14:33.846197 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:33.825681 23282 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:14:33.928393 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 82301.1us
I20260709 12:14:33.899137 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:33.929435 13339 raft_consensus.cc:2468] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:33.930433 23278 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:33.929416 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:33.931950 23278 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:33.932425 23278 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:33.932902 23278 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:33.937024 12643 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:33.939795 12643 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:33.893631 23263 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:33.897856 23226 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:33.887207 23143 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:33.956113 12643 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:33.958588 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:33.965157 12679 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:33.850703 23141 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:33.920320 23294 log.cc:927] Time spent T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.088s	user 0.000s	sys 0.001s
I20260709 12:14:33.960752 12672 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4765) took 1409 ms. Trace:
I20260709 12:14:33.954504 23124 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:33.980418 12676 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4764) took 1429 ms. Trace:
I20260709 12:14:33.966578 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:33.920863 23290 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:33.981462 12600 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:14:33.981004 12676 rpcz_store.cc:276] 0709 12:14:32.550840 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:33.081146 (+530306us) service_pool.cc:224] Handling call
0709 12:14:33.081571 (+   425us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:33.081981 (+   410us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:33.082019 (+    38us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:33.110330 (+ 28311us) log.cc:844] Serialized 52 byte log entry
0709 12:14:33.277724 (+167394us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:14:33.277792 (+    68us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:33.277897 (+   105us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:33.980108 (+702211us) raft_consensus.cc:1685] finished
0709 12:14:33.980139 (+    31us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:33.980336 (+   197us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":1499,"raft.run_cpu_time_us":47,"raft.run_wall_time_us":48,"thread_start_us":116044,"threads_started":1,"wal-append.queue_time_us":252033}
I20260709 12:14:33.976934 12672 rpcz_store.cc:276] 0709 12:14:32.551136 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:33.105882 (+554746us) service_pool.cc:224] Handling call
0709 12:14:33.106274 (+   392us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:33.165400 (+ 59126us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:33.165454 (+    54us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:33.173161 (+  7707us) log.cc:844] Serialized 26 byte log entry
0709 12:14:33.452744 (+279583us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:33.452802 (+    58us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:33.452884 (+    82us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:33.960422 (+507538us) raft_consensus.cc:1685] finished
0709 12:14:33.960454 (+    32us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:33.960650 (+   196us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":29282,"raft.queue_time_us":36641,"raft.run_cpu_time_us":159,"raft.run_wall_time_us":8659,"thread_start_us":169810,"threads_started":1,"wal-append.queue_time_us":398938}
I20260709 12:14:33.994148 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:33.957734 23253 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:33.997530 13331 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 34 index: 22. (index mismatch)
I20260709 12:14:33.905654 22847 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:34.003497 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 137471us
W20260709 12:14:34.009775 12572 leader_election.cc:336] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:34.000887 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:33.977860 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:34.001338 12646 leader_election.cc:336] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:33.904100 22960 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:33.987008 23263 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:14:34.002733 23136 consensus_meta.cc:287] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.264s	user 0.014s	sys 0.000s
I20260709 12:14:34.020496 12606 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:34.016732 12643 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:34.020087 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:34.015478 12676 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:14:34.028153 12672 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 26. Current term is 27. Ops: []
I20260709 12:14:34.030160 12601 raft_consensus.cc:1240] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:34.033156 12679 raft_consensus.cc:2468] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 33.
I20260709 12:14:34.033804 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:33.961069 23282 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:34.041663 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:34.042584 12604 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:34.040413 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:34.048267 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 839904us
I20260709 12:14:34.049245 12496 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:14:34.051846 12572 leader_election.cc:336] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:34.024583 23136 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:34.058909 23136 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:34.060626 22960 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:34.069551 12604 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:34.035511 23124 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:34.070551 23136 raft_consensus.cc:3060] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:34.103575 12675 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 33 index: 23. (index mismatch)
W20260709 12:14:34.049858 23274 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.287s	user 0.000s	sys 0.002s
I20260709 12:14:34.075639 23258 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:14:34.080751 23240 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.782s	user 0.012s	sys 0.000s
W20260709 12:14:34.087109 12646 leader_election.cc:336] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:34.097108 23278 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:34.100567 22960 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:14:34.073285 23282 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:34.064951 12496 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:14:34.139192 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 74351.3us
I20260709 12:14:34.140194 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:34.083149 12572 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:34.132118 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:34.132907 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:34.079665 23277 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:34.141916 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:34.153448 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 126633us
I20260709 12:14:34.154503 12643 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:34.160425 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:34.169328 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:34.166334 12494 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:34.147197 23226 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:34.174811 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:34.175717 12604 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:34.169708 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:34.133621 23240 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:34.179101 23282 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
W20260709 12:14:34.186897 23301 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:14:34.191067 23288 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:34.191665 23288 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:34.192040 23288 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:14:34.198377 12569 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:34.170475 23146 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:34.167397 23141 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:34.162623 23277 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:14:34.162070 22960 raft_consensus.cc:697] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:14:34.202553 12646 leader_election.cc:336] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:34.201140 12671 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4752) took 1823 ms. Trace:
W20260709 12:14:34.248386 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 45910.5us
I20260709 12:14:34.181911 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:34.220467 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:34.174772 23263 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:14:34.221905 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 170091us
I20260709 12:14:34.178419 23226 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:34.255956 12604 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
W20260709 12:14:34.257387 23296 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.181s	user 0.001s	sys 0.001s
W20260709 12:14:34.258729 23264 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.875s	user 0.003s	sys 0.003s
I20260709 12:14:34.245460 23146 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:34.209841 23225 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:34.299170 12673 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 34 index: 22. (index mismatch)
W20260709 12:14:34.298695 12572 leader_election.cc:336] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:34.341683 12572 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:34.248875 12671 rpcz_store.cc:276] 0709 12:14:32.377963 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.635989 (+258026us) service_pool.cc:224] Handling call
0709 12:14:32.636461 (+   472us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:32.681001 (+ 44540us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:14:32.681037 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:32.681574 (+   537us) log.cc:844] Serialized 26 byte log entry
0709 12:14:33.109383 (+427809us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:33.109452 (+    69us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:33.109571 (+   119us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:34.200908 (+1091337us) raft_consensus.cc:1685] finished
0709 12:14:34.200928 (+    20us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:34.201058 (+   130us) inbound_call.cc:177] Queueing success response
Metrics: {"thread_start_us":349621,"threads_started":1,"wal-append.queue_time_us":955411}
I20260709 12:14:34.324095 12602 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 26 index: 15. Preceding OpId from leader: term: 27 index: 16. (index mismatch)
W20260709 12:14:34.322504 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 140726us
I20260709 12:14:34.345257 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:34.333818 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 167566us
W20260709 12:14:34.270092 22714 consensus_meta.cc:287] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.974s	user 0.008s	sys 0.002s
W20260709 12:14:34.272087 23175 consensus_meta.cc:287] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 1.073s	user 0.005s	sys 0.007s
I20260709 12:14:34.286434 23030 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:34.356482 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:34.356326 23030 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:34.356992 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:34.357761 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4514) took 2110 ms. Trace:
I20260709 12:14:34.358696 13302 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:34.358892 23175 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:34.359995 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:34.361073 23175 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:34.361826 23225 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:34.274397 22847 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:34.364380 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:34.364806 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:34.365046 12677 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
I20260709 12:14:34.325181 22990 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:34.365643 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:34.317052 23226 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:34.362655 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:34.333262 23270 consensus_meta.cc:287] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.555s	user 0.015s	sys 0.000s
I20260709 12:14:34.340873 23124 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:34.348477 23146 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:34.352974 22714 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:34.353976 23215 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:34.358175 12598 rpcz_store.cc:276] 0709 12:14:32.247568 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:32.429338 (+181770us) service_pool.cc:224] Handling call
0709 12:14:32.460217 (+ 30879us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:32.483583 (+ 23366us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:32.483632 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:32.484297 (+   665us) log.cc:844] Serialized 52 byte log entry
0709 12:14:33.218389 (+734092us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:14:33.218464 (+    75us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:33.218816 (+   352us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:34.258376 (+1039560us) spinlock_profiling.cc:243] Waited 4.66 ms on lock 0x7b240033f848. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14857b99c4 00007f14857b7acb 00007f1485972dc9 00007f1485972a55 00007f148597209d 00007f14885a410e 00007f14885bb88c 00007f14885babb6 00007f14885ba7d8 00007f14885c4221
0709 12:14:34.357455 (+ 99079us) raft_consensus.cc:1685] finished
0709 12:14:34.357478 (+    23us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:34.357701 (+   223us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":15969,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":9343,"spinlock_wait_cycles":10728192,"thread_start_us":683549,"threads_started":1,"wal-append.queue_time_us":678891}
I20260709 12:14:34.368285 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:34.351150 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:14:34.366961 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 190717us
I20260709 12:14:34.347195 23279 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:34.372735 12494 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:34.374732 23208 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:14:34.382499 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:34.362501 23188 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election lost for term 26. Reason: could not achieve majority
I20260709 12:14:34.380164 12571 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:34.358708 23030 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:34.398391 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 47343.5us
W20260709 12:14:34.401882 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 103214us
W20260709 12:14:34.402726 12572 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:34.402951 12494 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:34.400417 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 25 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:34.413863 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:34.402607 23306 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.067s	user 0.000s	sys 0.001s
I20260709 12:14:34.419186 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:34.427093 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:34.435636 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:34.424549 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:34.438930 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:34.406198 23291 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:34.438512 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:34.446981 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:34.442422 23258 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:34.442936 12678 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:14:34.377524 23277 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.037s
I20260709 12:14:34.409916 23289 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:14:34.448123 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:34.448575 12599 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 34. Current term is 35. Ops: [34.22-34.22]
I20260709 12:14:34.406950 23208 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 31 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:34.449261 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:34.425119 22592 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:34.460556 12673 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:34.455876 12494 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:34.379233 23270 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:34.412561 23074 raft_consensus.cc:2804] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:14:34.454659 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:34.456841 12677 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
I20260709 12:14:34.455474 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:34.458439 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:34.441550 23253 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:34.397472 23279 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:34.448169 23215 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:34.467763 13338 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 33.
I20260709 12:14:34.482666 23255 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:34.483315 23255 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:34.486016 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:34.486864 12494 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:34.490557 23313 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election won for term 28
W20260709 12:14:34.491266 23309 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.125s	user 0.001s	sys 0.000s
I20260709 12:14:34.492307 23252 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:34.492852 23252 raft_consensus.cc:697] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:34.495010 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:34.495270 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:34.496290 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:34.496829 12675 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:34.498955 23252 consensus_queue.cc:237] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:34.503386 23255 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:34.474112 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:34.508117 12606 raft_consensus.cc:2393] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 28: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:34.467244 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:34.477375 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 74698.9us
W20260709 12:14:34.510648 12572 leader_election.cc:336] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:34.450873 23268 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:34.512768 12678 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 31.
I20260709 12:14:34.486898 13332 raft_consensus.cc:2393] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 26: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:14:34.447293 23291 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:34.481150 13331 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 25 index: 19. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:14:34.496732 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:34.465180 23240 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:34.456795 22937 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:34.509343 12598 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 35.
I20260709 12:14:34.519485 12674 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 25.
I20260709 12:14:34.522974 13330 raft_consensus.cc:2393] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 30: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:14:34.523082 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:34.529587 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 31 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:34.465233 22592 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:34.505373 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:34.479398 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:34.523808 23291 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:34.579568 23074 raft_consensus.cc:697] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:34.478075 23136 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:14:34.560019 23312 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.080s	user 0.000s	sys 0.001s
I20260709 12:14:34.559751 23253 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:34.561189 23278 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:34.565081 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 54489.8us
I20260709 12:14:34.565757 23124 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:34.421845 22960 consensus_queue.cc:237] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:34.568786 12597 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:34.570672 23146 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:34.540324 23208 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:34.521531 12676 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:34.581380 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:34.526422 12496 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:34.518290 23268 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:34.597452 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 92200.1us
I20260709 12:14:34.598340 12677 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 27: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:34.587706 13302 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:34.583211 23195 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.764s	user 0.010s	sys 0.005s
I20260709 12:14:34.568799 23017 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:34.650305 23124 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:34.581254 12675 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 34.
I20260709 12:14:34.594763 22592 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Advancing to term 26
W20260709 12:14:34.702276 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 175948us
I20260709 12:14:34.583424 23263 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:14:34.581192 23290 consensus_meta.cc:287] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.584s	user 0.008s	sys 0.005s
I20260709 12:14:34.668242 12494 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:14:34.670331 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 82782.3us
W20260709 12:14:34.676023 13305 leader_election.cc:336] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:34.606227 23314 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:14:34.762225 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 94097.5us
I20260709 12:14:34.758733 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:34.765391 23143 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:34.765976 23143 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:34.761562 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 85591.7us
I20260709 12:14:34.651648 22714 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:34.768184 13305 leader_election.cc:336] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:34.594934 23288 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:34.757690 23319 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.152s	user 0.001s	sys 0.000s
I20260709 12:14:34.780993 12672 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:34.701686 23268 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:34.766485 23143 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:34.795452 12673 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:14:34.767108 23136 raft_consensus.cc:3060] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:34.776216 23257 raft_consensus.cc:2804] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:14:34.767906 23235 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 0.637s	user 0.013s	sys 0.009s
I20260709 12:14:34.710222 23195 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:34.810287 12672 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
W20260709 12:14:34.811281 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 43636.2us
I20260709 12:14:34.757642 23146 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.087s
I20260709 12:14:34.793955 23289 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:34.817869 22714 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:14:34.817015 12494 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:34.816365 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 194729us
I20260709 12:14:34.799794 23270 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:34.858830 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 294056us
W20260709 12:14:34.864428 12572 leader_election.cc:336] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:34.667963 23253 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:34.872061 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 93287.7us
I20260709 12:14:34.872267 12496 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:34.810993 23235 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 0.680s	user 0.014s	sys 0.010s
I20260709 12:14:34.869783 12605 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 32. Current term is 33. Ops: []
I20260709 12:14:34.807406 23257 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:34.874389 23257 raft_consensus.cc:3060] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:14:34.876740 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 59785.4us
I20260709 12:14:34.882863 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:34.886360 12678 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:34.821990 23218 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:14:34.882787 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:34.893745 12675 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:34.865283 23313 raft_consensus.cc:697] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:34.819958 22953 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 33.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.087s
I20260709 12:14:34.868856 22714 consensus_queue.cc:1243] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 33.21 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 20
I20260709 12:14:34.916778 12673 raft_consensus.cc:1275] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:14:34.906472 23218 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:34.914464 12601 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:34.925032 12673 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 33. Current term is 34. Ops: [31.15-33.16]
I20260709 12:14:34.930328 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:34.935254 12677 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:34.937037 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:34.938009 12673 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 33.
W20260709 12:14:34.938863 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:34.939961 12571 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:34.904862 12705 heartbeater.cc:646] Failed to heartbeat to 127.11.13.190:46629 (8 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.11.13.190:46629 timed out after 15.000s (SENT)
I20260709 12:14:34.828577 22990 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.066s
I20260709 12:14:34.916628 23279 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:34.819391 23074 consensus_queue.cc:237] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 31.16, Last appended by leader: 16, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:34.951802 12675 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 26.
I20260709 12:14:34.921450 23218 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:14:34.983341 23252 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:14:34.940092 23040 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.070s
I20260709 12:14:34.997916 12571 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:34.927881 23331 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:14:35.000468 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:35.052379 12678 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 30.
I20260709 12:14:35.056234 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:35.056545 12677 raft_consensus.cc:2468] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:35.061102 12571 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:34.947031 23290 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:35.008140 23307 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.061s	user 0.000s	sys 0.000s
I20260709 12:14:35.006711 23124 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:35.007743 23195 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:35.002930 22937 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.063014 23188 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:35.052784 23279 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:34.953698 23235 consensus_queue.cc:1059] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.131s
I20260709 12:14:35.072372 23331 raft_consensus.cc:697] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:35.111970 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:35.071303 23030 consensus_queue.cc:1059] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 27.14, Next index: 15, Last known committed idx: 14, Time since last communication: 0.037s
W20260709 12:14:35.095021 23321 log.cc:927] Time spent T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.083s	user 0.002s	sys 0.000s
I20260709 12:14:35.095932 23161 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:35.123770 12673 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 34.
I20260709 12:14:35.101930 23188 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:35.124529 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:35.123970 23226 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.749s	user 0.007s	sys 0.007s
I20260709 12:14:35.082420 23290 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:35.129348 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 68266.6us
W20260709 12:14:35.137838 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 255692us
I20260709 12:14:35.115206 23270 raft_consensus.cc:493] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:14:35.117115 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 244894us
I20260709 12:14:35.123549 23288 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:35.171417 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 35 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:35.173719 13338 raft_consensus.cc:2410] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 35 because replica has last-logged OpId of term: 33 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 31 index: 20.
I20260709 12:14:35.135279 23188 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:35.150770 23161 raft_consensus.cc:697] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:35.179255 23329 consensus_queue.cc:1048] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:14:35.107508 22937 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:35.186293 23072 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:35.188019 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 323622us
I20260709 12:14:35.177614 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:35.198630 12572 leader_election.cc:336] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:35.199295 12572 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:35.186651 23333 log.cc:927] Time spent T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.065s	user 0.001s	sys 0.000s
I20260709 12:14:35.204358 13304 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:35.123463 23279 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:35.207643 12643 leader_election.cc:336] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 2.120s (SENT)
I20260709 12:14:35.152720 23226 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:35.207044 12496 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:35.212114 12643 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:35.168722 23270 raft_consensus.cc:515] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:35.216459 12680 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51304 (request call id 4033) took 1392 ms. Trace:
I20260709 12:14:35.221185 12680 rpcz_store.cc:276] 0709 12:14:33.824357 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:33.831667 (+  7310us) service_pool.cc:224] Handling call
0709 12:14:33.840911 (+  9244us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:34.361304 (+520393us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:34.361337 (+    33us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:34.361856 (+   519us) log.cc:844] Serialized 26 byte log entry
0709 12:14:34.778697 (+416841us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:34.778756 (+    59us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:34.832525 (+ 53769us) spinlock_profiling.cc:243] Waited 520 ms on lock 0x7b54000ab4d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:34.832598 (+    73us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:35.198902 (+366304us) raft_consensus.cc:1685] finished
0709 12:14:35.198924 (+    22us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:35.216385 (+ 17461us) inbound_call.cc:177] Queueing success response
Metrics: {"raft.queue_time_us":4320,"raft.run_cpu_time_us":118,"raft.run_wall_time_us":1225,"spinlock_wait_cycles":1195786112,"thread_start_us":416387,"threads_started":1,"wal-append.queue_time_us":591326}
I20260709 12:14:35.217761 12496 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:35.200778 23253 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:35.100553 23313 consensus_queue.cc:237] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 25.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:35.237715 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:35.307961 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:35.308986 12601 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:35.239239 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:34.864607 23289 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:35.242239 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:35.215521 23337 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Leader pre-election won for term 34
W20260709 12:14:35.292284 11318 thread.cc:527] Waited for 1000ms trying to join with heartbeat (tid 12556)
I20260709 12:14:35.311030 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:35.311257 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 103628us
I20260709 12:14:35.312183 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:35.322846 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 83694.7us
I20260709 12:14:35.317052 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:35.213606 23030 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:35.189575 23072 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:35.226821 23325 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:14:35.316092 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:35.323141 23314 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:14:35.233421 23339 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 33.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.104s
I20260709 12:14:35.322875 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:35.328574 23339 consensus_queue.cc:1243] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 35.22 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 21
W20260709 12:14:35.329779 23268 consensus_meta.cc:287] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.519s	user 0.004s	sys 0.004s
I20260709 12:14:35.329896 23263 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:14:35.225539 23288 raft_consensus.cc:2804] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:35.331133 12643 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:35.333524 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:35.335852 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 146760us
W20260709 12:14:35.339075 12572 leader_election.cc:336] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:35.340657 12678 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
I20260709 12:14:35.301723 23232 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:14:35.301502 23124 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:14:35.213815 23322 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.107s
W20260709 12:14:35.353935 13305 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:35.347275 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:35.345589 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:35.355311 13305 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:35.344488 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 102384us
I20260709 12:14:35.361116 12643 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:35.363710 12601 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:14:35.189524 23161 consensus_queue.cc:237] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:35.366083 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:35.367031 12606 raft_consensus.cc:2410] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 for term 30 because replica has last-logged OpId of term: 27 index: 20, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 19.
I20260709 12:14:35.369735 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:35.320700 23337 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:35.375001 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 25 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:35.378436 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:35.382638 12496 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:35.333989 23072 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:35.354475 23335 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.299s	user 0.001s	sys 0.001s
I20260709 12:14:35.337090 23263 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:35.378561 13304 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:35.232173 23317 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.128s
I20260709 12:14:35.312518 23143 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:35.416097 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:35.366691 23320 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:35.343572 23288 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:35.423627 23350 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.377122 23326 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:35.224306 23331 consensus_queue.cc:237] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 25.16, Last appended by leader: 16, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:35.387228 23279 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:35.243010 23252 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.117s
I20260709 12:14:35.330606 23253 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.058s
W20260709 12:14:35.387667 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 48642.8us
I20260709 12:14:35.353775 23325 raft_consensus.cc:697] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:35.357785 23329 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:35.386047 23314 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:35.399695 12675 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 26 index: 15. (index mismatch)
I20260709 12:14:35.392233 12673 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:14:35.393747 23337 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:35.419864 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:35.424358 23263 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:35.462934 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:35.452050 23288 raft_consensus.cc:3060] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:35.342485 23268 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:35.387810 22960 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:14:35.479135 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 102059us
W20260709 12:14:35.485802 23291 consensus_meta.cc:287] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.888s	user 0.007s	sys 0.007s
I20260709 12:14:35.430706 22953 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:14:35.495983 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:35.459189 23258 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:35.462430 23357 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.504398 12675 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:35.512161 23317 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:14:35.483610 23314 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:14:35.502329 12572 leader_election.cc:336] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:35.520188 12603 raft_consensus.cc:1240] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 29. Current term is 30. Ops: []
I20260709 12:14:35.521659 12572 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:14:35.510805 23327 log.cc:927] Time spent T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.330s	user 0.000s	sys 0.001s
W20260709 12:14:35.510293 23324 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.325s	user 0.001s	sys 0.001s
I20260709 12:14:35.509578 23320 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:14:35.524372 23314 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:35.517598 23317 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:35.516271 22953 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:35.513793 23358 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:35.464118 23350 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:35.542418 23320 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:35.517138 23258 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:14:35.560420 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 69378.3us
W20260709 12:14:35.561319 12572 consensus_peers.cc:597] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:35.511086 23291 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:35.563037 12675 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:14:35.547008 23226 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:35.561931 12572 consensus_peers.cc:597] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:35.520381 23357 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:35.558653 23332 log.cc:927] Time spent T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.344s	user 0.002s	sys 0.000s
W20260709 12:14:35.570080 12572 leader_election.cc:336] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:35.555485 23358 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:35.562630 23320 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:14:35.577586 12676 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:35.578510 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:35.582743 23362 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.577611 23279 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:35.583530 23362 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:35.583818 23279 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:35.586109 12602 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1210) took 1140 ms. Trace:
I20260709 12:14:35.587684 23268 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:35.588402 23317 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:35.589500 23074 raft_consensus.cc:3055] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:14:35.589968 23074 raft_consensus.cc:740] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:35.554741 23325 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:35.591029 23074 consensus_queue.cc:260] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 33.16, Last appended by leader: 16, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:35.592057 23291 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:35.594386 22953 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 31.15, Last appended by leader: 15, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:35.586607 12602 rpcz_store.cc:276] 0709 12:14:34.445680 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:34.445830 (+   150us) service_pool.cc:224] Handling call
0709 12:14:34.458709 (+ 12879us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:34.459049 (+   340us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:34.459092 (+    43us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:34.478778 (+ 19686us) log.cc:844] Serialized 26 byte log entry
0709 12:14:34.810962 (+332184us) raft_consensus.cc:1645] Marking committed up to 16
0709 12:14:34.811022 (+    60us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:34.811124 (+   102us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:35.585864 (+774740us) raft_consensus.cc:1685] finished
0709 12:14:35.585888 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:35.586042 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":65039,"raft.queue_time_us":4422,"raft.run_cpu_time_us":295,"raft.run_wall_time_us":3747,"spinlock_wait_cycles":66816,"thread_start_us":235320,"threads_started":1,"wal-append.queue_time_us":631421}
I20260709 12:14:35.597522 23350 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:35.597860 23279 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:35.598569 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:35.599159 23357 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:35.599678 12572 consensus_peers.cc:597] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:35.600574 12572 leader_election.cc:336] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:35.566529 23361 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.579125 23329 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 30 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:14:35.590107 22592 consensus_meta.cc:287] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.810s	user 0.009s	sys 0.002s
I20260709 12:14:35.577697 23358 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:35.579846 23253 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:35.592150 23074 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:35.600827 23072 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:35.590021 23317 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:14:35.625583 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 124088us
I20260709 12:14:35.586023 23362 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:35.614799 23361 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:35.636304 13304 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:35.603200 23225 thread.cc:641] wal-append [worker] (thread pool) Time spent creating pthread: real 1.045s	user 0.009s	sys 0.004s
W20260709 12:14:35.644358 12572 leader_election.cc:336] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:35.645130 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:35.634807 22934 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:14:35.668074 23188 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:35.669610 23329 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:35.681416 23218 consensus_meta.cc:287] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Time spent flushing consensus metadata: real 0.675s	user 0.007s	sys 0.002s
I20260709 12:14:35.683493 23270 leader_election.cc:290] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:35.626000 22592 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:35.685985 22592 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:35.688967 23136 consensus_meta.cc:287] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.881s	user 0.010s	sys 0.001s
I20260709 12:14:35.689406 23136 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:35.693300 23290 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:35.647027 23350 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:35.697059 23350 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:35.696043 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 51732.2us
W20260709 12:14:35.700556 12572 leader_election.cc:336] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:35.700986 12572 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:35.646803 23225 thread.cc:608] wal-append [worker] (thread pool) Time spent starting thread: real 1.088s	user 0.009s	sys 0.004s
W20260709 12:14:35.696161 23367 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.082s	user 0.000s	sys 0.001s
I20260709 12:14:35.707448 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:35.634308 23253 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:35.703186 23072 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:35.705684 23030 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:35.702893 23369 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.681870 23218 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:35.719527 12572 leader_election.cc:336] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:35.722465 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:35.722684 12572 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:35.728118 12601 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:14:35.703519 23350 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:35.735767 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:35.737293 12601 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:35.750571 12678 raft_consensus.cc:1240] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 33. Current term is 34. Ops: [31.15-33.16]
I20260709 12:14:35.704228 23263 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:35.754659 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:35.755911 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:35.760957 13339 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 36 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 35. Current term is 36. Ops: []
I20260709 12:14:35.763105 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:35.764032 12676 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:14:35.766027 23368 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:14:35.767031 23257 consensus_meta.cc:287] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.892s	user 0.013s	sys 0.000s
I20260709 12:14:35.767547 23257 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:35.718526 22960 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:35.772246 22960 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:35.774879 23146 raft_consensus.cc:3055] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:14:35.775327 23188 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:35.775560 23364 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:35.776190 23364 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:35.779714 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:35.784144 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:35.786899 12676 raft_consensus.cc:2410] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 28 because replica has last-logged OpId of term: 27 index: 16, which is greater than that of the candidate, which has last-logged OpId of term: 26 index: 15.
I20260709 12:14:35.788800 12494 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:35.797291 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 151476us
I20260709 12:14:35.799937 12494 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:35.801697 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:35.803748 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:35.809585 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:35.810592 12676 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:35.775358 23146 raft_consensus.cc:740] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:35.842542 23363 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:35.842854 23146 consensus_queue.cc:260] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:35.843438 23263 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:35.734447 23369 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:35.844210 23146 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:35.844281 23314 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:35.845798 23369 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:35.791107 23136 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:14:35.861101 12646 consensus_peers.cc:597] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:35.779320 23072 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:35.812855 23175 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:14:35.846607 23314 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:35.847777 23364 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:35.847972 23218 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:35.832796 23361 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:35.855753 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:35.884351 23072 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:35.785784 23257 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:35.838537 23258 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:14:35.889293 23253 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:35.890230 22960 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:35.857788 23288 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:35.892364 23288 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:35.858152 23337 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:35.895946 23337 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:35.896528 12646 leader_election.cc:336] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:35.898337 13332 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 33. Current term is 34. Ops: [33.23-33.23]
I20260709 12:14:35.874909 23320 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:35.899587 23345 log.cc:927] Time spent T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.579s	user 0.000s	sys 0.003s
W20260709 12:14:35.870087 13305 consensus_peers.cc:597] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:35.900413 13305 consensus_peers.cc:597] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:35.900759 23030 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:35.863392 12571 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:35.902328 23372 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:35.903198 23372 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:35.899300 23175 raft_consensus.cc:697] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:35.901192 13305 leader_election.cc:336] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:35.802853 23188 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:35.903661 12571 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:35.907783 13305 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:35.908905 13305 consensus_peers.cc:597] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:35.901427 12599 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:14:35.885387 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 77578.5us
I20260709 12:14:35.883111 22937 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:35.910509 13305 leader_election.cc:336] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:35.901257 23320 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 35 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:35.913371 23175 consensus_queue.cc:237] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 24.16, Last appended by leader: 16, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:35.924713 12601 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
I20260709 12:14:35.935292 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 36 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:35.937394 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:35.906826 12676 raft_consensus.cc:2468] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 32.
W20260709 12:14:35.940814 12646 leader_election.cc:336] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:35.944617 12646 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:35.944861 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:35.950942 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:35.941066 12571 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:35.945371 12494 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:35.954238 12646 leader_election.cc:336] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:35.955408 12599 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 32.
I20260709 12:14:35.923364 23337 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:35.958261 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:35.960469 13330 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 34. Current term is 35. Ops: [34.22-34.22]
I20260709 12:14:35.961887 12601 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:35.910221 23372 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:14:35.955446 12646 consensus_peers.cc:597] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:35.967175 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 246111us
W20260709 12:14:35.977276 12646 leader_election.cc:336] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:35.977854 12646 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:14:35.979116 12646 consensus_peers.cc:597] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:35.982766 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 36 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:35.979681 12646 leader_election.cc:336] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:35.984795 12601 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 35.
I20260709 12:14:35.931499 22960 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:35.987160 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:35.982169 23255 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.006s
I20260709 12:14:35.990276 22960 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:35.991708 23377 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:35.992391 23377 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:35.992834 22960 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:35.993301 12569 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:35.967061 23291 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:14:35.981385 23253 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:35.995568 23371 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:14:35.996400 23343 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.605s	user 0.002s	sys 0.002s
I20260709 12:14:35.998432 23072 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:36.000391 23379 log.cc:927] Time spent T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.077s	user 0.000s	sys 0.000s
I20260709 12:14:35.966404 23381 raft_consensus.cc:2804] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:14:36.015077 23291 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:36.015357 23381 raft_consensus.cc:697] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:36.015756 23291 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:36.016461 23381 consensus_queue.cc:237] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 30.17, Last appended by leader: 17, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:36.018182 23291 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:35.944142 23339 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.075s
I20260709 12:14:36.018476 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:35.967262 23136 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:14:36.023226 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:36.023303 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:36.024128 12680 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:14:36.024451 23389 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:14:36.024907 23389 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:36.025272 23389 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:14:36.025365 23136 raft_consensus.cc:697] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:36.025601 12571 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:36.026449 23136 consensus_queue.cc:237] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 24.13, Last appended by leader: 13, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:36.029793 23074 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:36.030340 23074 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:36.030764 23074 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:14:36.038841 13305 leader_election.cc:336] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:36.041277 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:36.046311 12646 leader_election.cc:336] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:36.066490 13305 leader_election.cc:336] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:36.071053 13305 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:36.075682 13305 leader_election.cc:336] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:36.079826 13305 leader_election.cc:336] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:36.086969 13305 leader_election.cc:336] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 35 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:36.018345 23240 raft_consensus.cc:2749] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:14:36.088186 12646 leader_election.cc:336] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:36.093930 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.095721 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:36.096829 12598 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Advancing to term 36
W20260709 12:14:36.097847 12646 leader_election.cc:336] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:36.058360 23253 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:36.104048 12646 leader_election.cc:336] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:36.084054 23337 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:14:36.086845 23329 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:36.090468 23339 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.090171 23291 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.041273 23358 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:36.108589 12646 leader_election.cc:336] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:36.087260 23232 consensus_queue.cc:1059] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 33.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.030s
I20260709 12:14:36.105886 23218 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 19, Time since last communication: 0.014s
W20260709 12:14:36.139711 23391 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.122s	user 0.000s	sys 0.001s
I20260709 12:14:36.142409 23386 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:36.143121 23386 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:36.122467 23339 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:36.164278 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.164709 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:36.172497 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:36.173297 12606 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 31. Current term is 32. Ops: []
I20260709 12:14:36.171361 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:36.172292 12671 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:36.179924 12672 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:36.174482 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:36.156102 13331 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 34. Current term is 35. Ops: [32.21-34.22]
I20260709 12:14:36.165000 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:36.181531 12675 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:14:36.181581 12598 raft_consensus.cc:2468] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 36.
I20260709 12:14:36.183568 13302 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:36.185041 13338 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 34: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:14:36.190851 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 34 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:36.191692 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 36 candidate_status { last_received { term: 35 index: 23 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:36.192909 12678 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate e56d487365394ac5ac5f1002926a5645 in current term 34: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:14:36.195010 12496 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.114136 23253 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:36.203487 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:36.203781 23253 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:36.204442 12674 raft_consensus.cc:2393] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 28: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:36.158687 23377 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:36.190074 23208 raft_consensus.cc:2804] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:14:36.206375 23208 raft_consensus.cc:697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 36 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:36.206882 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:36.207552 12675 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:36.207413 23208 consensus_queue.cc:237] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 35.23, Last appended by leader: 23, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:36.198498 13305 leader_election.cc:336] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:36.209120 13305 leader_election.cc:336] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:36.191936 22714 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 32.22, Next index: 23, Last known committed idx: 22, Time since last communication: 0.019s
I20260709 12:14:36.211019 22714 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 LEADER]: Stepping down as leader of term 33
I20260709 12:14:36.134670 23143 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:36.212736 12678 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
W20260709 12:14:36.199353 12646 leader_election.cc:336] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:36.211395 22714 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:36.216933 23389 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:36.220883 12676 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:14:36.169667 23074 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:36.223206 13302 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:36.223841 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:36.224514 12673 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 32
I20260709 12:14:36.224583 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 35 candidate_status { last_received { term: 33 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:36.224628 23350 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:36.225427 12678 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:14:36.177706 22953 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:36.146098 23386 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:36.153988 23339 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:36.128454 23291 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:36.241865 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:36.244246 12674 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:14:36.244879 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:36.245114 23331 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.214010 23289 consensus_queue.cc:1059] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.038s
I20260709 12:14:36.261088 23289 consensus_queue.cc:1243] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 33.21 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 20
I20260709 12:14:36.260555 23331 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:36.260819 22714 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.23, Last appended by leader: 23, Current term: 33, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:36.262388 22714 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:36.266189 12671 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 26 index: 19. (index mismatch)
I20260709 12:14:36.269701 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:36.274350 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 36 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:36.275182 12605 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4525) took 2501 ms. Trace:
I20260709 12:14:36.275625 12605 rpcz_store.cc:276] 0709 12:14:33.773688 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:33.896750 (+123062us) service_pool.cc:224] Handling call
0709 12:14:33.921035 (+ 24285us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:34.873879 (+952844us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.274918 (+1401039us) spinlock_profiling.cc:243] Waited 947 ms on lock 0x7b540007a654. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:36.275092 (+   174us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":2177228672}
I20260709 12:14:36.286644 13338 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 36 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 36: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
W20260709 12:14:36.234493 12646 leader_election.cc:336] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:36.292070 12678 raft_consensus.cc:2468] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 35.
I20260709 12:14:36.264303 23331 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:36.307145 23141 raft_consensus.cc:3055] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:14:36.307519 23253 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:14:36.307682 23141 raft_consensus.cc:740] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:36.294960 12673 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 32.
I20260709 12:14:36.309257 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:36.296902 12675 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
W20260709 12:14:36.291863 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 57411.7us
I20260709 12:14:36.320814 12672 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
W20260709 12:14:36.321151 11318 thread.cc:527] Waited for 2000ms trying to join with heartbeat (tid 12556)
I20260709 12:14:36.315224 12496 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:36.325057 12674 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:36.327441 12677 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:14:36.330773 12643 leader_election.cc:336] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.602s (SENT)
I20260709 12:14:36.346554 23141 consensus_queue.cc:260] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 34.22, Last appended by leader: 22, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:36.350001 12572 thread.cc:641] raft [worker] (thread pool) Time spent creating pthread: real 0.618s	user 0.008s	sys 0.005s
W20260709 12:14:36.350540 12572 thread.cc:608] raft [worker] (thread pool) Time spent starting thread: real 0.619s	user 0.009s	sys 0.005s
I20260709 12:14:36.351117 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:14:36.351248 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 631727us
I20260709 12:14:36.351794 12678 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 32
I20260709 12:14:36.354550 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:36.355192 12679 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:14:36.358799 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49634.2us
I20260709 12:14:36.360813 23320 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:14:36.364943 23387 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:14:36.374877 23235 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.048s
I20260709 12:14:36.377771 12597 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 26. Current term is 27. Ops: []
I20260709 12:14:36.257387 22953 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:36.380453 22953 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:36.361932 23320 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.020s
I20260709 12:14:36.384724 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
W20260709 12:14:36.387329 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.391265 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:36.392280 13305 consensus_peers.cc:597] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:36.392527 12601 raft_consensus.cc:2410] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 for term 29 because replica has last-logged OpId of term: 28 index: 14, which is greater than that of the candidate, which has last-logged OpId of term: 25 index: 12.
I20260709 12:14:36.395027 12601 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
W20260709 12:14:36.396410 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 81314.7us
I20260709 12:14:36.397082 23228 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
W20260709 12:14:36.352212 12572 leader_election.cc:336] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:36.441576 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.404062 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:36.409792 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 79073.6us
I20260709 12:14:36.363178 13304 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:36.219566 23372 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:36.441428 12572 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:36.412185 23330 thread.cc:641] apply [worker] (thread pool) Time spent creating pthread: real 1.087s	user 0.005s	sys 0.010s
I20260709 12:14:36.439093 12602 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 31 index: 17. (index mismatch)
I20260709 12:14:36.443766 13330 raft_consensus.cc:2468] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 32.
W20260709 12:14:36.464754 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 101667us
I20260709 12:14:36.467870 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:36.468020 13304 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:36.448488 12678 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 32.
I20260709 12:14:36.453977 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:36.454399 12679 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
W20260709 12:14:36.442935 12643 leader_election.cc:336] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.607s (SENT)
I20260709 12:14:36.473578 12643 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:36.462405 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 36 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.463402 23355 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:14:36.475282 23355 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:36.475744 12603 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4562) took 1236 ms. Trace:
I20260709 12:14:36.476030 23213 raft_consensus.cc:2804] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:14:36.476156 12603 rpcz_store.cc:276] 0709 12:14:35.239008 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.239132 (+   124us) service_pool.cc:224] Handling call
0709 12:14:35.311133 (+ 72001us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:35.522268 (+211135us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.475505 (+953237us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b540022fbd4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:36.475684 (+   179us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":480124800}
I20260709 12:14:36.375697 23074 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:36.477702 12603 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 31. Current term is 32. Ops: []
I20260709 12:14:36.476630 23213 raft_consensus.cc:697] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 35 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:36.477880 12601 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 35.
I20260709 12:14:36.478948 23213 consensus_queue.cc:237] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 33.21, Last appended by leader: 21, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:36.479763 23372 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:36.480661 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 128490us
I20260709 12:14:36.481316 12601 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:36.482101 23346 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election lost for term 25. Reason: could not achieve majority
I20260709 12:14:36.484018 12601 raft_consensus.cc:1240] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 34. Current term is 35. Ops: [34.22-34.22]
I20260709 12:14:36.494406 12602 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27. Ops: []
W20260709 12:14:36.464915 23330 thread.cc:608] apply [worker] (thread pool) Time spent starting thread: real 1.140s	user 0.006s	sys 0.011s
I20260709 12:14:36.400641 23322 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.071s
W20260709 12:14:36.484037 23146 consensus_meta.cc:287] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.639s	user 0.013s	sys 0.001s
I20260709 12:14:36.467360 23329 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:36.485406 23175 raft_consensus.cc:2804] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:14:36.465036 22953 consensus_queue.cc:1059] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 32.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.041s
I20260709 12:14:36.531548 12605 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:14:36.540032 12571 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:36.540395 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:36.541493 12680 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:36.541492 12571 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:36.513091 23175 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:14:36.570345 12646 leader_election.cc:336] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:36.593098 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.595170 12677 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 32.
I20260709 12:14:36.571839 23273 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:14:36.568642 23175 raft_consensus.cc:3060] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:36.575836 23190 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:14:36.663959 12599 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 33 index: 16. Preceding OpId from leader: term: 34 index: 16. (term mismatch)
I20260709 12:14:36.665846 12599 pending_rounds.cc:85] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting all ops after (but not including) 15
I20260709 12:14:36.666230 12599 pending_rounds.cc:107] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Aborting uncommitted NO_OP operation due to leader change: 33.16
I20260709 12:14:36.666699 12599 raft_consensus.cc:2889] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:36.671892 13305 leader_election.cc:336] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:36.574934 23320 consensus_queue.cc:1059] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.086s
I20260709 12:14:36.577802 23350 consensus_queue.cc:1059] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.089s
I20260709 12:14:36.674813 23320 raft_consensus.cc:3055] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:14:36.675390 23320 raft_consensus.cc:740] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:36.676707 23320 consensus_queue.cc:260] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.21, Last appended by leader: 21, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:36.678579 23350 raft_consensus.cc:3055] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:14:36.679138 23350 raft_consensus.cc:740] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:36.680265 23350 consensus_queue.cc:260] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 26.19, Last appended by leader: 19, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:36.689510 12680 raft_consensus.cc:2468] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
W20260709 12:14:36.690225 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.681914 23350 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:14:36.681232 23348 log.cc:927] Time spent T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 1.354s	user 0.001s	sys 0.006s
W20260709 12:14:36.690886 23402 log.cc:927] Time spent T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.181s	user 0.001s	sys 0.000s
I20260709 12:14:36.663122 23273 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:36.715930 23320 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:36.732254 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 36 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:36.734493 12599 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:14:36.724284 23389 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:36.740530 23403 log.cc:927] Time spent T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.234s	user 0.000s	sys 0.001s
I20260709 12:14:36.763134 22953 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.774780 13339 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38482 (request call id 956) took 1646 ms. Trace:
I20260709 12:14:36.775261 13339 rpcz_store.cc:276] 0709 12:14:35.128181 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.128403 (+   222us) service_pool.cc:224] Handling call
0709 12:14:35.142053 (+ 13650us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:35.761582 (+619529us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:36.774438 (+1012856us) spinlock_profiling.cc:243] Waited 619 ms on lock 0x7b54006b0154. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a20e01 00007f1488a1f83e 00007f148d9e4419 00007f1486f5bbd4 00007f1486f5bb62 00007f1486f5ba91 00007f1486f5ba0c 00007f1486f5acc2 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:36.774700 (+   262us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1422718720}
W20260709 12:14:36.778360 22714 consensus_meta.cc:287] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.515s	user 0.007s	sys 0.006s
W20260709 12:14:36.787662 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 316507us
I20260709 12:14:36.791852 12496 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645
W20260709 12:14:36.792960 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 311488us
I20260709 12:14:36.796667 12496 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:36.798298 12496 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:36.798857 12599 raft_consensus.cc:2468] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 36.
I20260709 12:14:36.800415 13302 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:36.797957 23338 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:14:36.743041 23273 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 30.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:36.778607 22953 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:36.802006 23190 raft_consensus.cc:697] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:36.802944 22990 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:14:36.796070 22714 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:36.801920 23226 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:14:36.806836 23337 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:14:36.820201 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.821103 12598 raft_consensus.cc:2393] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 29: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:36.821919 12496 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:36.822916 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.825160 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.820755 22714 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:36.826424 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.826601 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:36.827359 12597 raft_consensus.cc:2393] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 29: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:36.827492 12598 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 27: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:36.825174 12601 raft_consensus.cc:1240] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 36 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 35. Current term is 36. Ops: []
I20260709 12:14:36.828979 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.826825 22714 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:36.830113 23337 raft_consensus.cc:697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 36 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:36.831173 23337 consensus_queue.cc:237] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 33.23, Last appended by leader: 23, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:36.828563 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.833874 23355 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:36.836987 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.828323 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.837702 12599 raft_consensus.cc:2393] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 33: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:36.839584 12496 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:36.798041 23146 consensus_queue.cc:1059] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 27.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.023s
I20260709 12:14:36.840533 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.842497 22953 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:36.823550 23371 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:36.843606 23371 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:36.840632 22990 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:14:36.848560 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:36.799193 23175 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:36.851125 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:36.851768 12603 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:36.851737 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.852677 12601 raft_consensus.cc:2393] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 31: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:36.853420 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.854007 12601 raft_consensus.cc:2393] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 33: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:36.859416 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.858034 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.852082 23226 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:36.866004 12599 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:14:36.869035 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fca0ed05cd34c8e92dc950351b864a2" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.881856 12603 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29.
I20260709 12:14:36.883234 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.859704 23355 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:36.884617 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.859220 22990 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:36.885937 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.887192 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 32 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.865789 23146 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.885004 23040 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.890846 23040 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:36.891474 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.892405 12598 raft_consensus.cc:2393] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 30: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:36.893278 23040 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:36.885378 23190 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 25.18, Last appended by leader: 18, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:36.893306 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.894969 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.877079 23226 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:36.882893 23225 raft_consensus.cc:3055] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Stepping down as leader of term 28
I20260709 12:14:36.901125 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.903942 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 27 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.904886 12598 raft_consensus.cc:2410] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 34 because replica has last-logged OpId of term: 31 index: 15, which is greater than that of the candidate, which has last-logged OpId of term: 27 index: 14.
I20260709 12:14:36.907140 12603 raft_consensus.cc:1240] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:36.910423 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:36.912606 12597 raft_consensus.cc:2368] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27.
I20260709 12:14:36.903865 23225 raft_consensus.cc:740] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:36.897168 23146 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:36.927040 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.938374 23225 consensus_queue.cc:260] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 28.20, Last appended by leader: 20, Current term: 28, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:36.922360 22990 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:36.942474 23175 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:36.925534 23226 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:36.952502 23146 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:36.954566 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:36.963711 12597 raft_consensus.cc:1240] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 28. Current term is 29. Ops: []
I20260709 12:14:36.966037 23397 consensus_queue.cc:1059] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 28.20, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:14:36.991137 23317 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:36.991951 23317 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:36.994899 23317 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:37.003841 22714 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:37.104650 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.115617 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4567) took 1392 ms. Trace:
I20260709 12:14:37.116470 12600 rpcz_store.cc:276] 0709 12:14:35.723472 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:35.723659 (+   187us) service_pool.cc:224] Handling call
0709 12:14:35.724046 (+   387us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:35.724303 (+   257us) raft_consensus.cc:1534] Early marking committed up to index 19
0709 12:14:35.754303 (+ 30000us) log.cc:844] Serialized 14 byte log entry
0709 12:14:36.920597 (+1166294us) spinlock_profiling.cc:243] Waited 4.77 ms on lock 0x7b18000d7200. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a 00007f1488a2aa2a
0709 12:14:37.115037 (+194440us) log.cc:844] Serialized 14 byte log entry
0709 12:14:37.115228 (+   191us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:37.115272 (+    44us) raft_consensus.cc:1645] Marking committed up to 19
0709 12:14:37.115324 (+    52us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.115412 (+    88us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:37.115553 (+   141us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":29962,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":45,"raft.queue_time_us":497,"spinlock_wait_cycles":10965376,"thread_start_us":192814,"threads_started":1}
I20260709 12:14:37.134768 22714 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:37.141433 23371 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:37.143826 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.153959 12494 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:37.155498 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.122283 23317 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.122666 23146 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.162636 23040 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:37.151722 22714 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.166630 23331 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.013s
I20260709 12:14:37.163388 23317 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:37.165313 23146 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:37.168335 23040 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.013s
I20260709 12:14:37.172869 22714 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:37.196173 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:37.197407 23355 consensus_queue.cc:1059] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 29.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:14:37.197789 23338 raft_consensus.cc:2804] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:14:37.204026 23175 raft_consensus.cc:3055] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:14:37.193080 23357 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:37.204123 23338 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:37.204123 23355 consensus_queue.cc:1059] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.010s
I20260709 12:14:37.217491 23338 raft_consensus.cc:3060] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:14:37.218552 23355 consensus_queue.cc:1243] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer f5fc807f7ecc4d1ca0fd6be524f734ce log is divergent from this leader: its last log entry 32.18 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 16
I20260709 12:14:37.219110 23355 raft_consensus.cc:3055] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 LEADER]: Stepping down as leader of term 31
I20260709 12:14:37.219427 23355 raft_consensus.cc:740] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:37.220304 23355 consensus_queue.cc:260] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 31.18, Last appended by leader: 18, Current term: 31, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:37.210592 23175 raft_consensus.cc:740] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:37.221469 23355 raft_consensus.cc:3060] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:14:37.223560 12572 leader_election.cc:336] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:37.224105 12572 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:37.222443 23175 consensus_queue.cc:260] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:37.225893 23364 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:37.226574 23175 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:37.227870 23141 raft_consensus.cc:3060] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:14:37.231923 23291 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:37.236536 23414 log.cc:927] Time spent T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.055s	user 0.000s	sys 0.000s
I20260709 12:14:37.238083 12672 raft_consensus.cc:1240] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27. Ops: [26.15-26.15]
W20260709 12:14:37.221517 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 778586us
I20260709 12:14:37.243170 12680 raft_consensus.cc:3055] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:14:37.243753 12680 raft_consensus.cc:740] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
W20260709 12:14:37.240572 23404 log.cc:927] Time spent T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.501s	user 0.002s	sys 0.003s
I20260709 12:14:37.214178 23357 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:37.250741 12671 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 14. Preceding OpId from leader: term: 34 index: 16. (index mismatch)
I20260709 12:14:37.244849 12680 consensus_queue.cc:260] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 27.15, Last appended by leader: 15, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:37.255697 12680 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:37.257066 23188 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:37.259645 12671 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 24 index: 16. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
W20260709 12:14:37.265606 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 53676.2us
W20260709 12:14:37.262750 12572 consensus_peers.cc:597] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:37.271359 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.319392 12678 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 34. Current term is 35. Ops: [32.21-34.22]
I20260709 12:14:37.269927 12679 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 31 index: 17. (index mismatch)
W20260709 12:14:37.323031 11318 thread.cc:527] Waited for 3000ms trying to join with heartbeat (tid 12556)
I20260709 12:14:37.336243 12672 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
W20260709 12:14:37.335783 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 67861.1us
I20260709 12:14:37.339078 22714 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:37.340692 23419 log.cc:927] Time spent T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.154s	user 0.000s	sys 0.001s
I20260709 12:14:37.254734 23226 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.077s
I20260709 12:14:37.328787 12680 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 28 index: 15. (term mismatch)
W20260709 12:14:37.332830 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 63554us
W20260709 12:14:37.344944 12572 leader_election.cc:336] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:37.345431 12572 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:37.334204 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 81013.2us
W20260709 12:14:37.346798 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:37.347407 12572 leader_election.cc:336] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:37.347810 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:37.348416 12680 pending_rounds.cc:85] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 14
I20260709 12:14:37.348727 12680 pending_rounds.cc:107] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 27.15
I20260709 12:14:37.348899 12680 raft_consensus.cc:2889] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:37.348965 12672 raft_consensus.cc:1240] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 31. Current term is 32. Ops: [30.17-31.18]
I20260709 12:14:37.350322 23411 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:14:37.353600 12572 leader_election.cc:336] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:37.354076 12572 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:37.354898 23188 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:14:37.355562 23415 log.cc:927] Time spent T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.172s	user 0.001s	sys 0.000s
I20260709 12:14:37.355672 23188 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:14:37.359159 23417 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.118s	user 0.001s	sys 0.000s
I20260709 12:14:37.362336 23143 raft_consensus.cc:3055] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 LEADER]: Stepping down as leader of term 34
I20260709 12:14:37.362834 23143 raft_consensus.cc:740] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:37.369767 12572 leader_election.cc:336] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:37.370386 23317 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:37.370985 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:37.371433 12494 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:37.373438 23226 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
I20260709 12:14:37.373509 12676 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:37.374691 23175 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:14:37.363745 23143 consensus_queue.cc:260] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 34.22, Last appended by leader: 22, Current term: 34, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:37.376145 23091 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:14:37.376425 23143 raft_consensus.cc:3060] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:14:37.376451 12572 leader_election.cc:336] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:37.379832 22953 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.000s
I20260709 12:14:37.377389 12572 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:37.381179 23357 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:37.381805 23357 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:37.383353 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:37.384581 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:37.385072 12676 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:37.390836 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:37.392390 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:37.395371 22592 raft_consensus.cc:3055] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:14:37.395999 22592 raft_consensus.cc:740] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:37.396195 23146 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:37.393641 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:37.397022 22592 consensus_queue.cc:260] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 32.18, Last appended by leader: 18, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:37.407183 12675 raft_consensus.cc:1240] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:37.409494 23225 raft_consensus.cc:3060] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:37.410676 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:37.412567 23409 raft_consensus.cc:3055] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:14:37.412949 12675 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:14:37.413046 23409 raft_consensus.cc:740] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:37.415398 23409 consensus_queue.cc:260] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:37.416249 12572 consensus_peers.cc:597] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:37.416651 23409 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:14:37.417053 12572 consensus_peers.cc:597] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:37.417747 12572 leader_election.cc:336] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:37.395500 12676 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:14:37.420926 23355 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:14:37.429000 12572 leader_election.cc:336] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:37.429970 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:37.431044 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:37.431521 13333 raft_consensus.cc:2410] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce for term 32 because replica has last-logged OpId of term: 31 index: 21, which is greater than that of the candidate, which has last-logged OpId of term: 30 index: 20.
W20260709 12:14:37.429908 12572 leader_election.cc:336] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:37.421409 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.432601 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 35 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:37.432906 12572 leader_election.cc:336] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:37.422052 23255 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:14:37.423435 12604 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:48492 (request call id 1225) took 1401 ms. Trace:
I20260709 12:14:37.424361 12571 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:37.427140 23358 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:37.368157 12643 leader_election.cc:336] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Timed out: RequestConsensusVote RPC to 127.11.13.130:39365 timed out after 1.723s (SENT)
I20260709 12:14:37.444639 12673 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 29. Current term is 30. Ops: []
W20260709 12:14:37.433871 12572 leader_election.cc:336] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:37.456758 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:37.465401 12676 raft_consensus.cc:2393] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 29: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:14:37.449204 12572 leader_election.cc:336] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:37.467116 12572 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:37.472409 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:37.454617 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:37.479699 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 111584us
I20260709 12:14:37.448292 12604 rpcz_store.cc:276] 0709 12:14:36.022165 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.322751 (+300586us) service_pool.cc:224] Handling call
0709 12:14:36.432699 (+109948us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:36.433003 (+   304us) raft_consensus.cc:1534] Early marking committed up to index 17
0709 12:14:36.700740 (+267737us) log.cc:844] Serialized 14 byte log entry
0709 12:14:37.422847 (+722107us) spinlock_profiling.cc:243] Waited 9.11 ms on lock 0x7b180038d280. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14885c84bc 00007f148591648b 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a
0709 12:14:37.423060 (+   213us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:37.423113 (+    53us) raft_consensus.cc:1645] Marking committed up to 17
0709 12:14:37.423170 (+    57us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.423253 (+    83us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:37.423376 (+   123us) inbound_call.cc:177] Queueing success response
Metrics: {"prepare.queue_time_us":359872,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":1252,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":96,"spinlock_wait_cycles":20958976,"thread_start_us":345373,"threads_started":2,"wal-append.queue_time_us":183343}
I20260709 12:14:37.457268 23277 raft_consensus.cc:2804] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:14:37.457050 23136 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.030s
I20260709 12:14:37.433665 22960 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:14:37.490690 12572 leader_election.cc:336] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:37.494167 13333 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 26 index: 19. Preceding OpId from leader: term: 27 index: 19. (term mismatch)
I20260709 12:14:37.519572 13333 pending_rounds.cc:85] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 18
I20260709 12:14:37.519850 13333 pending_rounds.cc:107] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 26.19
I20260709 12:14:37.520072 13333 raft_consensus.cc:2889] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260709 12:14:37.495245 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 44909.9us
I20260709 12:14:37.497488 12673 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 30 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:14:37.449164 23291 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:37.442634 22592 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:37.487313 23277 raft_consensus.cc:697] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:37.505273 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:37.532943 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:37.487515 23326 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:37.456207 23355 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.044s
I20260709 12:14:37.461933 23338 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:37.531505 23240 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.007s
I20260709 12:14:37.538753 23338 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 36 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:37.494472 23411 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:37.515362 23357 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 29.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:14:37.517622 12599 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 36 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 35 index: 23. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
I20260709 12:14:37.539721 23279 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:37.498446 23146 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.002s
I20260709 12:14:37.525080 12496 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:37.541296 23277 consensus_queue.cc:237] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:37.536500 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:37.543284 13330 raft_consensus.cc:2468] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
I20260709 12:14:37.549388 12643 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:37.551324 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:37.499693 22960 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:37.566430 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:37.508797 23364 consensus_queue.cc:1059] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 35.23, Next index: 24, Last known committed idx: 22, Time since last communication: 0.037s
I20260709 12:14:37.570650 12496 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:37.543563 23124 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:37.548506 23338 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:37.567763 23279 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:37.535835 23371 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.041s
I20260709 12:14:37.565227 23326 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:37.567700 23253 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:14:37.586782 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 211474us
I20260709 12:14:37.532370 23291 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:37.565748 23218 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.020s
I20260709 12:14:37.581651 23338 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:37.573626 22714 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.019s
I20260709 12:14:37.593843 23143 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.594834 23141 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:14:37.596961 23124 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:37.607204 23338 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:14:37.621733 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 196481us
I20260709 12:14:37.629078 13333 raft_consensus.cc:1275] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 33 index: 21. Preceding OpId from leader: term: 35 index: 22. (index mismatch)
I20260709 12:14:37.657888 12671 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:14:37.645581 23289 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:37.665382 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 99061.6us
I20260709 12:14:37.665159 23289 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:37.666311 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.667832 12643 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: e56d487365394ac5ac5f1002926a5645
I20260709 12:14:37.669265 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:37.670825 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.662726 23175 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:37.672257 23175 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:37.674417 12676 raft_consensus.cc:1275] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 29 index: 15. Preceding OpId from leader: term: 33 index: 17. (index mismatch)
I20260709 12:14:37.675854 13333 raft_consensus.cc:1240] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 35 LEADER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 34. Current term is 35. Ops: [32.21-34.22]
I20260709 12:14:37.676437 12676 raft_consensus.cc:1275] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 28 index: 16. Preceding OpId from leader: term: 32 index: 18. (index mismatch)
I20260709 12:14:37.645691 23226 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.684077 12643 leader_election.cc:304] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:37.685443 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:37.640782 23331 consensus_queue.cc:1059] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 24.14, Next index: 15, Last known committed idx: 13, Time since last communication: 0.029s
I20260709 12:14:37.629853 23143 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:37.656296 23141 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:37.661736 23428 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.690711 12678 raft_consensus.cc:1275] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 23 index: 14. Preceding OpId from leader: term: 28 index: 16. (index mismatch)
I20260709 12:14:37.653821 23357 consensus_queue.cc:1048] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.036s
I20260709 12:14:37.700621 12643 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:37.711648 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:37.699162 12677 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:51308 (request call id 4808) took 1528 ms (client timeout 1939 ms). Trace:
W20260709 12:14:37.719022 12677 rpcz_store.cc:269] 0709 12:14:36.162853 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:36.164427 (+  1574us) service_pool.cc:224] Handling call
0709 12:14:37.691447 (+1527020us) spinlock_profiling.cc:243] Waited 149 ms on lock 0x7b54001414d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:37.691600 (+   153us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":341990272}
I20260709 12:14:37.713078 23428 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:14:37.723611 23143 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:37.724957 23428 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:37.725677 23289 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:37.725807 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 1.14867e+06us
I20260709 12:14:37.686753 23435 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 18, Time since last communication: 0.047s
I20260709 12:14:37.729269 13339 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 36 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:37.730029 13332 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:37.730604 22960 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:37.731247 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:37.731595 23291 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:37.692524 22847 consensus_queue.cc:1048] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.042s
I20260709 12:14:37.683823 23358 consensus_queue.cc:1048] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.050s
I20260709 12:14:37.734028 23141 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 29.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:37.689440 23124 raft_consensus.cc:2804] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election won for term 29
W20260709 12:14:37.752918 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 52438.2us
I20260709 12:14:37.754168 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:37.758265 13305 consensus_peers.cc:597] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:37.755316 12646 leader_election.cc:336] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:37.760002 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:37.760689 13337 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 34.
I20260709 12:14:37.774585 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:37.775377 12604 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:37.759732 23435 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:37.747121 23371 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.718238 23437 raft_consensus.cc:2749] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:37.759968 23291 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.688918 23226 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:37.784891 12643 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:37.788123 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 35 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:37.796486 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:37.797106 12603 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:37.797346 12599 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 25 index: 16. Preceding OpId from leader: term: 27 index: 17. (index mismatch)
I20260709 12:14:37.806435 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:37.790457 12601 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 22 index: 15. Preceding OpId from leader: term: 26 index: 17. (index mismatch)
I20260709 12:14:37.781925 12600 raft_consensus.cc:1240] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 32. Current term is 33. Ops: []
I20260709 12:14:37.797910 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 36 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:37.812428 12569 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:37.815651 12603 raft_consensus.cc:2393] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 36 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 36: Already voted for candidate e56d487365394ac5ac5f1002926a5645 in this term.
I20260709 12:14:37.820057 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:37.827184 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:37.829794 23277 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:37.830579 23277 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:37.833424 23277 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:37.834335 23277 raft_consensus.cc:2764] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election decision vote started in defunct term 28: won
I20260709 12:14:37.780428 23364 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:37.783521 22847 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:37.844285 23450 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:37.783360 22960 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:37.783598 23438 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
W20260709 12:14:37.851666 12572 leader_election.cc:336] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 35 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:37.859869 12572 leader_election.cc:336] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:37.860316 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:37.862166 23124 raft_consensus.cc:697] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:37.864200 23124 consensus_queue.cc:237] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.20, Last appended by leader: 20, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:37.867588 23440 log.cc:927] Time spent T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.172s	user 0.000s	sys 0.001s
W20260709 12:14:37.870419 12572 leader_election.cc:336] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:37.871446 12572 leader_election.cc:336] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:37.871855 12572 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:37.841454 23435 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:37.883001 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:37.885735 12572 leader_election.cc:336] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:37.889003 12598 raft_consensus.cc:2393] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 32: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:37.888144 12572 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:37.849686 23368 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
W20260709 12:14:37.892750 12572 leader_election.cc:336] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:37.811115 23371 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:37.816231 23291 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:37.897091 12572 leader_election.cc:336] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:37.905393 12572 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:37.909354 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 124592us
I20260709 12:14:37.910346 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:37.886720 23277 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:37.911988 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:37.898408 23427 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.255s	user 0.001s	sys 0.000s
I20260709 12:14:37.887650 23447 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:14:37.893833 23357 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:37.922470 12643 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:37.861562 22847 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:37.926934 12643 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:37.854938 23364 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:37.923265 12602 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 26 index: 14. (index mismatch)
I20260709 12:14:37.929809 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:37.869194 22960 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:37.921787 23436 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:37.921576 23435 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:37.885174 23438 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:37.954625 23364 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:37.959560 22960 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:37.974103 23438 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:37.974817 23091 raft_consensus.cc:3055] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:14:37.975647 23091 raft_consensus.cc:740] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:37.975993 23371 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:37.937698 23437 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:37.983273 23326 raft_consensus.cc:3055] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:14:37.983845 23326 raft_consensus.cc:740] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:37.984917 23326 consensus_queue.cc:260] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 23, Committed index: 23, Last appended: 35.23, Last appended by leader: 23, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:37.987843 23291 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:37.991572 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 61852.3us
I20260709 12:14:37.993937 12673 raft_consensus.cc:1240] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 29. Current term is 30. Ops: []
I20260709 12:14:37.965839 12494 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:37.903180 23368 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:38.022382 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:38.025422 12606 raft_consensus.cc:1275] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 36 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 33 index: 23. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
W20260709 12:14:38.026672 12645 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.003121 12494 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:37.932058 23453 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:37.976684 23091 consensus_queue.cc:260] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 26.15, Last appended by leader: 15, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:38.031535 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 219251us
I20260709 12:14:38.032933 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:38.037060 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:37.941757 23357 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:38.045620 12643 leader_election.cc:400] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27.
I20260709 12:14:37.992038 23447 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:38.041720 13333 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 31 index: 21. Preceding OpId from leader: term: 32 index: 21. (term mismatch)
I20260709 12:14:38.049165 13333 pending_rounds.cc:85] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 20
I20260709 12:14:38.049553 13333 pending_rounds.cc:107] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 31.21
I20260709 12:14:38.046983 12643 leader_election.cc:403] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:14:38.022589 23364 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:38.049866 13333 raft_consensus.cc:2889] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:38.030408 23228 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:38.034588 23255 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:38.065553 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:38.069039 12677 raft_consensus.cc:1240] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 30. Current term is 31. Ops: [30.19-30.19]
W20260709 12:14:38.068068 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 170992us
I20260709 12:14:38.046955 23326 raft_consensus.cc:3060] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Advancing to term 36
W20260709 12:14:38.072778 12572 consensus_peers.cc:597] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:38.087910 12572 leader_election.cc:336] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:38.088697 12572 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:38.039954 23450 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:38.084115 12599 raft_consensus.cc:3055] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:14:38.090363 12599 raft_consensus.cc:740] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:38.091944 12679 raft_consensus.cc:1240] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 26. Current term is 27. Ops: [26.19-26.19]
I20260709 12:14:38.034379 23449 consensus_queue.cc:1048] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 14, Time since last communication: 0.039s
I20260709 12:14:38.030052 12494 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:38.093313 12672 raft_consensus.cc:1240] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Rejecting Update request from peer e56d487365394ac5ac5f1002926a5645 for earlier term 31. Current term is 32. Ops: []
W20260709 12:14:38.106603 13305 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.066294 23091 raft_consensus.cc:3060] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:38.091396 12599 consensus_queue.cc:260] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:38.116674 12678 raft_consensus.cc:1275] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:14:38.074828 23291 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: Vote denied by peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365) with higher term. Message: Invalid argument: T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 26. Current term is 27.
W20260709 12:14:38.128090 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 98145.1us
I20260709 12:14:38.129755 12494 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:38.131012 12680 raft_consensus.cc:1240] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Rejecting Update request from peer f5fc807f7ecc4d1ca0fd6be524f734ce for earlier term 30. Current term is 32. Ops: []
W20260709 12:14:38.133767 23279 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.540s	user 0.011s	sys 0.000s
I20260709 12:14:38.103464 23228 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.103026 23364 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.135756 12599 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 32
I20260709 12:14:38.110869 22592 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.055428 23357 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:38.139887 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:38.105779 23226 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:38.082377 23188 consensus_queue.cc:1059] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 29.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.045s
I20260709 12:14:38.134575 23141 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:38.144030 23255 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:14:38.156068 23279 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:14:38.173309 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 101715us
I20260709 12:14:38.162619 23240 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.181615 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:38.168186 23358 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:14:38.186945 12494 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:38.167949 23255 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:38.191166 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:38.193287 12494 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1
I20260709 12:14:38.158001 23364 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:38.161365 22592 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:38.196620 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 124720us
I20260709 12:14:38.199381 23040 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:38.200095 23289 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:14:38.152881 23228 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:38.202320 12599 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:14:38.185009 22847 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:38.219269 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 133355us
I20260709 12:14:38.182307 23368 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:38.195977 23255 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:14:38.228312 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:38.229387 23428 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:14:38.195641 23322 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:14:38.231453 23228 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:38.217515 23040 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:38.237691 23338 consensus_meta.cc:287] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.594s	user 0.010s	sys 0.004s
I20260709 12:14:38.243932 23357 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:38.237998 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:38.256312 12680 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 26.
I20260709 12:14:38.217449 23436 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:38.186769 23240 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:38.254384 12674 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 25 index: 18. Preceding OpId from leader: term: 27 index: 19. (index mismatch)
I20260709 12:14:38.254148 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:38.259627 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:38.263419 12606 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 29.
I20260709 12:14:38.269403 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:38.257787 23040 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:14:38.290565 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 70624.9us
W20260709 12:14:38.291514 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:38.300942 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.292747 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:38.314754 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:38.300565 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:38.318506 13305 consensus_peers.cc:597] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:38.321728 22592 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:38.322199 23460 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.079s	user 0.000s	sys 0.001s
W20260709 12:14:38.322641 23421 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.996s	user 0.005s	sys 0.004s
I20260709 12:14:38.262377 23338 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:38.323891 12676 raft_consensus.cc:1275] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 36 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 35 index: 23. Preceding OpId from leader: term: 36 index: 24. (index mismatch)
W20260709 12:14:38.314389 23452 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.151s	user 0.001s	sys 0.000s
I20260709 12:14:38.323895 12671 raft_consensus.cc:1275] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 27 index: 15. (index mismatch)
I20260709 12:14:38.306486 23136 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.331171 12571 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:38.331369 23136 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:38.323907 23464 consensus_queue.cc:1059] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.111s
I20260709 12:14:38.334973 23364 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:38.335275 23464 raft_consensus.cc:3055] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:14:38.335662 23472 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:38.336170 23472 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:38.337268 11318 thread.cc:527] Waited for 4000ms trying to join with heartbeat (tid 12556)
I20260709 12:14:38.336969 23455 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
W20260709 12:14:38.337666 23457 log.cc:927] Time spent T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.224s	user 0.001s	sys 0.002s
I20260709 12:14:38.321215 23143 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.338491 23143 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:38.335676 23464 raft_consensus.cc:740] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:38.340569 23143 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:38.341295 23464 consensus_queue.cc:260] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.19, Last appended by leader: 19, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:38.342569 23136 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:38.342744 23464 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:38.341848 23040 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:38.345172 23040 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:38.347452 23240 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:38.349118 23458 log.cc:927] Time spent T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.183s	user 0.001s	sys 0.000s
I20260709 12:14:38.325224 23289 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
W20260709 12:14:38.351258 23459 log.cc:927] Time spent T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.134s	user 0.001s	sys 0.000s
I20260709 12:14:38.336545 23472 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:38.356215 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 36 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:38.324265 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 70429.3us
W20260709 12:14:38.354960 12646 leader_election.cc:336] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:38.361045 23289 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 36 FOLLOWER]: Leader election lost for term 36. Reason: could not achieve majority
I20260709 12:14:38.323321 23462 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.137s
I20260709 12:14:38.355774 23326 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.390609 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:38.399232 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 283720us
W20260709 12:14:38.400028 12572 leader_election.cc:336] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:38.406793 13303 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:14:38.406700 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 47000.8us
I20260709 12:14:38.408769 13302 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:38.412142 12569 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:38.412968 23411 raft_consensus.cc:3055] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:14:38.413568 23411 raft_consensus.cc:740] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Becoming Follower/Learner. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:38.414222 23368 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:38.414912 23368 raft_consensus.cc:493] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting leader election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:38.414574 23411 consensus_queue.cc:260] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 29.17, Last appended by leader: 17, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:38.415375 23368 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:38.415894 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:38.417565 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:38.418341 23462 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:38.431077 23338 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:38.385341 23326 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:38.438999 23326 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:38.442727 13331 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:14:38.446234 12671 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 34.
I20260709 12:14:38.455041 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 22 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:38.394052 23468 consensus_queue.cc:1048] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.067s
I20260709 12:14:38.454723 23477 consensus_queue.cc:1048] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.010s
I20260709 12:14:38.360467 12494 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:38.464392 23338 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:38.466681 23411 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:14:38.469779 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 109412us
W20260709 12:14:38.478899 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 151971us
I20260709 12:14:38.399601 23472 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:38.485196 12423 catalog_manager.cc:5697] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 23 to 31. New cstate: current_term: 31 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:38.495071 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:38.496452 13331 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 34.
W20260709 12:14:38.497313 12646 leader_election.cc:336] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:38.498566 12646 leader_election.cc:336] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:38.499265 12646 leader_election.cc:336] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:38.499670 12646 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:14:38.502983 23461 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.390s	user 0.003s	sys 0.001s
W20260709 12:14:38.504953 12646 leader_election.cc:336] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:38.505517 12646 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:38.506053 23472 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:38.506407 23253 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:38.507217 23253 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:14:38.508710 12646 leader_election.cc:336] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:38.514299 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:38.518203 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:38.510298 12646 leader_election.cc:336] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:38.539544 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 266602us
I20260709 12:14:38.545802 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:38.548897 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:38.478874 23255 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:38.442830 23368 raft_consensus.cc:515] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:38.563148 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 24 index: 13 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:38.525985 23040 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.026s
W20260709 12:14:38.569579 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 168652us
W20260709 12:14:38.571012 12572 leader_election.cc:336] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:38.571437 12572 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:38.572760 12572 leader_election.cc:336] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:38.573156 12646 leader_election.cc:336] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:38.586007 12572 leader_election.cc:336] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:38.586512 12572 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:38.555881 23124 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:14:38.591653 13338 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 29. Current term is 30. Ops: []
I20260709 12:14:38.567298 23228 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.587436 23472 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
W20260709 12:14:38.594667 12572 consensus_peers.cc:597] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:38.604506 12572 leader_election.cc:336] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:38.605811 12572 consensus_peers.cc:597] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:38.606599 12572 leader_election.cc:336] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:38.607115 12572 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:38.553375 23476 consensus_queue.cc:1048] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 20, Time since last communication: 0.038s
I20260709 12:14:38.589495 23428 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.557125 23253 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.042s
I20260709 12:14:38.592924 23124 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:38.597388 23277 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:14:38.627970 23368 leader_election.cc:290] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:38.642191 23255 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 35 election: Requested vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:38.644629 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:38.646471 23091 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:14:38.647249 23091 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 14, Last known committed idx: 13, Time since last communication: 0.000s
I20260709 12:14:38.591512 23465 consensus_queue.cc:1048] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 23, Time since last communication: 0.030s
I20260709 12:14:38.683869 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:38.684494 13338 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:14:38.678979 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 72409.1us
W20260709 12:14:38.687613 12572 leader_election.cc:336] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:38.688022 12572 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:38.689095 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:38.690461 12674 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:38.697216 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:38.610024 23228 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:38.656701 23124 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:14:38.702082 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 153873us
I20260709 12:14:38.702033 12674 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 30
W20260709 12:14:38.701924 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 60349.1us
W20260709 12:14:38.702570 12570 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.718930 23450 consensus_queue.cc:1059] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 29.21, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
I20260709 12:14:38.719974 23450 raft_consensus.cc:3055] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Stepping down as leader of term 29
I20260709 12:14:38.720384 23450 raft_consensus.cc:740] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Becoming Follower/Learner. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:38.721143 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:38.721997 12604 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:38.721246 23450 consensus_queue.cc:260] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 29, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:38.723142 23450 raft_consensus.cc:3060] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:38.683149 23447 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election lost for term 30. Reason: could not achieve majority
I20260709 12:14:38.726771 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 35 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:38.621490 23428 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:38.747850 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 60272.2us
W20260709 12:14:38.753141 12572 leader_election.cc:336] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:38.703308 23474 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:38.761912 12604 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 30.
W20260709 12:14:38.762738 12572 consensus_peers.cc:597] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:38.763401 12572 leader_election.cc:336] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:38.767055 12572 consensus_peers.cc:597] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:38.767745 12572 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 130 similar messages]
I20260709 12:14:38.769629 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:38.751992 23371 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:14:38.789537 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 64806.4us
I20260709 12:14:38.791455 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:38.792110 12603 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:38.793004 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:38.791186 13302 leader_election.cc:304] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:38.794868 13305 consensus_peers.cc:597] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:38.784318 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e1ff01ff4ede4fc990b9a07b23102b0d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:38.798666 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:38.795569 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 43831.6us
I20260709 12:14:38.779481 23326 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:38.812134 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:38.813691 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:38.814481 12605 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:38.826711 13338 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 35.
I20260709 12:14:38.841965 12674 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
W20260709 12:14:38.843180 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 153598us
I20260709 12:14:38.846266 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:38.851238 12674 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:14:38.859587 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:38.859941 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:38.846364 23190 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:38.842082 23240 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.773326 23474 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:38.861635 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:38.867142 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:38.869086 23190 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:38.875056 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:38.884570 23487 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.184s	user 0.001s	sys 0.000s
I20260709 12:14:38.885646 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:38.886451 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:38.893251 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:38.911374 23428 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:38.912870 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 23 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:38.914268 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:38.914846 23228 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:38.915328 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 29 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:38.915700 12601 raft_consensus.cc:1240] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Rejecting Update request from peer 193a40da6e364ead8ab561e14c3cc8d1 for earlier term 29. Current term is 30. Ops: [29.21-29.21]
I20260709 12:14:38.916623 12604 raft_consensus.cc:1240] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 26. Current term is 27. Ops: [26.14-26.14]
W20260709 12:14:38.907186 23489 log.cc:927] Time spent T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.161s	user 0.001s	sys 0.000s
I20260709 12:14:38.873975 23240 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:38.919831 23474 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:38.920992 23240 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:38.875288 23368 raft_consensus.cc:2804] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:38.922290 23368 raft_consensus.cc:697] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 30 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
W20260709 12:14:38.922998 23490 log.cc:927] Time spent T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.288s	user 0.003s	sys 0.000s
I20260709 12:14:38.923336 23368 consensus_queue.cc:237] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 29.21, Last appended by leader: 21, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:38.908609 23190 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:38.928439 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
W20260709 12:14:38.931783 12643 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:38.946641 12599 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:38.946959 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:38.918018 12604 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 30 index: 17. Preceding OpId from leader: term: 32 index: 18. (index mismatch)
W20260709 12:14:38.945143 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 47128.1us
I20260709 12:14:38.896378 12605 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
W20260709 12:14:38.886005 23488 log.cc:927] Time spent T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.197s	user 0.001s	sys 0.000s
W20260709 12:14:38.894793 23486 log.cc:927] Time spent T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.139s	user 0.001s	sys 0.000s
I20260709 12:14:38.952875 12677 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:14:38.956785 12597 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:14:38.957967 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:38.953734 12606 raft_consensus.cc:2393] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 27: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:38.963201 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:38.963932 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:38.963992 12676 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 28: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:38.965466 12674 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:14:38.967895 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 25 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:38.968539 12643 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:38.975603 23483 log.cc:927] Time spent T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.193s	user 0.000s	sys 0.002s
I20260709 12:14:38.976790 23436 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:38.977881 12604 raft_consensus.cc:1275] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 28 index: 20. Preceding OpId from leader: term: 29 index: 21. (index mismatch)
I20260709 12:14:38.979068 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:38.979756 23253 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:38.980461 12496 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:38.981127 12675 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51308 (request call id 4856) took 1395 ms. Trace:
I20260709 12:14:38.954061 23435 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:38.982586 23435 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:38.981535 12675 rpcz_store.cc:276] 0709 12:14:37.585998 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:37.586199 (+   201us) service_pool.cc:224] Handling call
0709 12:14:37.586664 (+   465us) raft_consensus.cc:1471] Updating replica for 2 ops
0709 12:14:37.664360 (+ 77696us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:37.664409 (+    49us) raft_consensus.cc:1539] Triggering prepare for 2 ops
0709 12:14:37.665481 (+  1072us) log.cc:844] Serialized 52 byte log entry
0709 12:14:37.702415 (+ 36934us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:37.751264 (+ 48849us) log.cc:844] Serialized 14 byte log entry
0709 12:14:37.843899 (+ 92635us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:37.844013 (+   114us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:38.419368 (+575355us) spinlock_profiling.cc:243] Waited 8.08 ms on lock 0x7b18003dd0e0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:14:38.973670 (+554302us) spinlock_profiling.cc:243] Waited 15.7 ms on lock 0x7b18003dd0e0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:14:38.980879 (+  7209us) raft_consensus.cc:1685] finished
0709 12:14:38.980903 (+    24us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:38.981057 (+   154us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":26830,"prepare.queue_time_us":49541,"prepare.run_cpu_time_us":74,"prepare.run_wall_time_us":4520,"raft.queue_time_us":43835,"raft.run_cpu_time_us":300,"raft.run_wall_time_us":10504,"spinlock_wait_cycles":54703744,"thread_start_us":178896,"threads_started":3,"wal-append.queue_time_us":127849}
I20260709 12:14:38.984709 23435 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:38.990909 12599 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
W20260709 12:14:38.988442 12572 leader_election.cc:336] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:38.992465 12572 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:38.966517 23326 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:38.998934 23358 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:14:38.999594 13302 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 203037us
W20260709 12:14:38.977917 23482 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.337s	user 0.001s	sys 0.000s
W20260709 12:14:39.011248 12646 leader_election.cc:336] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:38.997851 23124 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:38.987128 23364 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:39.026173 12673 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 35: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:38.989858 23473 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:39.001847 23409 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:39.025569 23040 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:38.989588 22990 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.045s
W20260709 12:14:39.070400 12572 leader_election.cc:336] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:39.047309 23124 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:39.097575 12646 leader_election.cc:336] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:39.156101 12572 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:39.166452 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 68979.7us
W20260709 12:14:39.171952 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 101571us
I20260709 12:14:39.190250 12598 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4622) took 1132 ms. Trace:
I20260709 12:14:39.190980 12598 rpcz_store.cc:276] 0709 12:14:38.057454 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:38.074279 (+ 16825us) service_pool.cc:224] Handling call
0709 12:14:38.103431 (+ 29152us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:38.220271 (+116840us) raft_consensus.cc:1534] Early marking committed up to index 15
0709 12:14:38.268963 (+ 48692us) log.cc:844] Serialized 14 byte log entry
0709 12:14:38.380546 (+111583us) spinlock_profiling.cc:243] Waited 9.39 ms on lock 0x7b1800342280. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c023a 00007f1488a2aa2a
0709 12:14:38.534030 (+153484us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:38.534670 (+   640us) log.cc:844] Serialized 26 byte log entry
0709 12:14:38.534859 (+   189us) raft_consensus.cc:1645] Marking committed up to 15
0709 12:14:38.534907 (+    48us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:38.535002 (+    95us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:38.977314 (+442312us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
0709 12:14:39.189986 (+212672us) raft_consensus.cc:1685] finished
0709 12:14:39.190012 (+    26us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:39.190187 (+   175us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":13899,"prepare.queue_time_us":49851,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":38,"raft.queue_time_us":56648,"raft.run_cpu_time_us":114,"raft.run_wall_time_us":2064,"spinlock_wait_cycles":53781632,"thread_start_us":269412,"threads_started":2,"wal-append.queue_time_us":311153}
I20260709 12:14:39.070178 23040 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:39.184968 23268 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.186358 23411 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:39.189498 23124 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:39.295126 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 313449us
W20260709 12:14:39.305748 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 337409us
I20260709 12:14:39.306895 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:39.308211 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:39.274348 23268 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:39.314038 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:39.302104 12496 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:39.314900 23436 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:39.293637 23124 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:39.311640 23326 raft_consensus.cc:2749] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:39.331614 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:39.337705 11318 thread.cc:527] Waited for 5000ms trying to join with heartbeat (tid 12556)
I20260709 12:14:39.315517 23040 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:39.347863 23124 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:39.349206 23463 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:39.349969 23463 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:39.349301 23436 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:39.363867 23040 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.364544 23040 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:39.366528 23175 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.367386 23428 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.367097 23175 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:39.367924 23428 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:39.369591 23175 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:39.373688 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:39.374788 12605 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:14:39.376055 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:39.380169 13338 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 30 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:14:39.380071 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:39.377868 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 46352.9us
I20260709 12:14:39.324998 12496 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:39.382612 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:39.374053 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:39.400239 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:39.401803 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:39.405436 23450 consensus_meta.cc:287] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: Time spent flushing consensus metadata: real 0.682s	user 0.012s	sys 0.000s
W20260709 12:14:39.410831 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 85947.5us
I20260709 12:14:39.370525 23428 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:39.400398 23480 log.cc:927] Time spent T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.761s	user 0.012s	sys 0.000s
I20260709 12:14:39.415962 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 35 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:39.366966 23040 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:39.328222 23228 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:39.423478 23253 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:39.400820 23124 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:39.435449 13330 raft_consensus.cc:2393] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 35: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:39.405921 23517 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:39.447235 12496 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:39.417189 23364 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:14:39.417789 23435 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.425935 23338 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:39.423472 23355 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.433846 23228 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:39.464708 12646 leader_election.cc:336] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:39.409652 23436 consensus_queue.cc:1048] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.033s
I20260709 12:14:39.464594 12643 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
W20260709 12:14:39.476469 13305 consensus_peers.cc:597] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:39.506924 23463 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:39.478561 23514 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.083s	user 0.001s	sys 0.001s
I20260709 12:14:39.506975 23289 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.003s
I20260709 12:14:39.488538 23326 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Leader election lost for term 35. Reason: could not achieve majority
I20260709 12:14:39.474802 23228 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:39.481187 23355 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:39.513736 23355 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:39.440615 23253 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:39.464262 23435 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:39.515982 23435 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:39.516001 23253 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:39.465310 23175 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:39.526435 12678 raft_consensus.cc:2468] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 30.
I20260709 12:14:39.526815 23268 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:39.614313 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 196541us
I20260709 12:14:39.623251 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:39.623600 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:39.624208 12679 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 32.
I20260709 12:14:39.644776 12598 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 27 index: 20. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:14:39.648125 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:39.665702 13305 consensus_peers.cc:597] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:39.643244 23533 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.670465 23268 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:14:39.676815 23268 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:39.677313 23268 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:14:39.673991 23528 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.073s	user 0.001s	sys 0.000s
I20260709 12:14:39.676296 23533 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:39.704903 13305 consensus_peers.cc:597] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:39.717659 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 268249us
I20260709 12:14:39.726728 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:39.712020 23253 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:39.734303 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 273593us
I20260709 12:14:39.736958 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:39.735101 12496 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
W20260709 12:14:39.738462 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 361035us
I20260709 12:14:39.729863 23124 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.718147 23495 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 18, Time since last communication: 0.017s
I20260709 12:14:39.736078 23255 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:39.703084 23338 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:39.758733 12572 leader_election.cc:336] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:39.759677 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:39.760586 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:39.754921 13331 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:38472 (request call id 1248) took 1212 ms. Trace:
I20260709 12:14:39.743928 23409 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:39.744299 23357 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:39.804311 23357 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:39.779103 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:39.800169 12646 leader_election.cc:336] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:39.779917 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:39.734138 23253 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:39.755316 23322 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.032s
W20260709 12:14:39.811856 12646 leader_election.cc:336] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:39.743842 23477 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:39.788529 23228 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:39.789839 23338 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:39.805533 23528 log.cc:927] Time spent T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.054s	user 0.001s	sys 0.000s
I20260709 12:14:39.748102 23355 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:39.815177 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:39.753575 23124 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:39.828497 23228 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:39.797544 13331 rpcz_store.cc:276] 0709 12:14:38.542976 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:38.543210 (+   234us) service_pool.cc:224] Handling call
0709 12:14:38.543560 (+   350us) raft_consensus.cc:1471] Updating replica for 1 ops
0709 12:14:38.551596 (+  8036us) raft_consensus.cc:1534] Early marking committed up to index 0
0709 12:14:38.551632 (+    36us) raft_consensus.cc:1539] Triggering prepare for 1 ops
0709 12:14:38.552173 (+   541us) log.cc:844] Serialized 26 byte log entry
0709 12:14:39.349683 (+797510us) spinlock_profiling.cc:243] Waited 1.1 ms on lock 0x7b18002cc5c0. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f1485915e07 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b 00007f14885bfd23 00007f14885c00d1 00007f14889dba0f
0709 12:14:39.535273 (+185590us) raft_consensus.cc:1645] Marking committed up to 21
0709 12:14:39.740267 (+204994us) log.cc:844] Serialized 14 byte log entry
0709 12:14:39.743709 (+  3442us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:39.744977 (+  1268us) raft_consensus.cc:1668] Waiting on the replicates to finish logging
0709 12:14:39.746833 (+  1856us) raft_consensus.cc:1685] finished
0709 12:14:39.754599 (+  7766us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:39.754832 (+   233us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":25484,"prepare.queue_time_us":244075,"prepare.run_cpu_time_us":35,"prepare.run_wall_time_us":37,"raft.queue_time_us":11995,"raft.run_cpu_time_us":239,"raft.run_wall_time_us":6712,"spinlock_wait_cycles":2540800,"thread_start_us":377416,"threads_started":2,"wal-append.queue_time_us":1032926}
W20260709 12:14:39.771981 12572 leader_election.cc:336] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:39.768971 23411 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:39.829677 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:39.830706 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 58897.5us
W20260709 12:14:39.831436 12572 consensus_peers.cc:597] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:39.831995 12572 leader_election.cc:336] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:39.832144 23357 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:39.832427 12572 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:39.832866 23355 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:39.833632 12572 leader_election.cc:336] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:39.833798 23355 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:39.809669 23526 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.054s	user 0.000s	sys 0.000s
I20260709 12:14:39.753024 23175 consensus_queue.cc:1059] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 24.13, Next index: 14, Last known committed idx: 13, Time since last communication: 0.057s
W20260709 12:14:39.838168 12646 leader_election.cc:336] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:39.834105 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:39.834297 23491 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:39.829885 23411 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:39.769248 23536 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.054s
I20260709 12:14:39.843796 12678 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:40246 (request call id 1280) took 1097 ms. Trace:
I20260709 12:14:39.858704 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:39.845578 12672 raft_consensus.cc:1275] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 29 index: 21. Preceding OpId from leader: term: 30 index: 22. (index mismatch)
I20260709 12:14:39.860307 23175 raft_consensus.cc:3055] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Stepping down as leader of term 26
I20260709 12:14:39.824234 23477 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:39.861135 23253 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:39.862313 23533 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:39.864849 23124 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:39.859460 23411 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:39.860762 23175 raft_consensus.cc:740] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:39.859175 12678 rpcz_store.cc:276] 0709 12:14:38.746791 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:38.761120 (+ 14329us) service_pool.cc:224] Handling call
0709 12:14:39.843551 (+1082431us) spinlock_profiling.cc:243] Waited 632 ms on lock 0x7b54000c3ad4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a242fa 00007f148d9e5b17 00007f1486f606d4 00007f1486f60662 00007f1486f60591 00007f1486f6050c 00007f1486f5f7c2 00007f1486d5b48c 00007f1486d5a8c6 00007f1486d5a26f
0709 12:14:39.843717 (+   166us) inbound_call.cc:177] Queueing success response
Metrics: {"spinlock_wait_cycles":1453516544}
I20260709 12:14:39.872638 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:39.879357 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:39.825664 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 77481.4us
I20260709 12:14:39.884184 12494 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:39.831600 23358 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:39.888473 23364 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:39.871148 23188 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election lost for term 27. Reason: could not achieve majority
I20260709 12:14:39.869762 23477 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:39.899621 23544 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:39.890295 12423 catalog_manager.cc:5697] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 25 to 27. New cstate: current_term: 27 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:39.907780 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:39.911279 12678 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
W20260709 12:14:39.911440 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 54708.9us
I20260709 12:14:39.913733 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:39.915562 23357 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:39.916759 23357 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:39.917766 23357 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:39.923024 23544 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:39.887137 23253 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:39.910349 13302 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:39.892758 23326 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.906828 23364 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:39.969519 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 59288.9us
I20260709 12:14:39.931548 23175 consensus_queue.cc:260] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.14, Last appended by leader: 14, Current term: 26, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:39.882289 23510 consensus_queue.cc:1048] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.051s
I20260709 12:14:39.934706 23368 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:39.886408 23124 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:39.915346 12572 leader_election.cc:336] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:39.981693 12572 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:39.982767 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 67852.1us
W20260709 12:14:39.946336 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 199529us
W20260709 12:14:39.947067 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 204386us
I20260709 12:14:39.886143 23436 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:39.989167 12572 leader_election.cc:336] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:39.992287 12572 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:39.970502 23495 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:39.995090 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 28 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:39.997018 12494 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645; no voters: 
I20260709 12:14:39.966446 23544 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:39.972582 23516 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:40.013458 23358 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:40.014741 12423 catalog_manager.cc:5697] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 26 to 29, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 29 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:40.016207 12423 catalog_manager.cc:5697] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 25 to 26, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 26 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:40.020265 12569 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:39.984933 23517 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:39.993513 23240 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:39.970394 23253 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:40.026757 23091 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:14:40.027325 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:40.054246 12646 leader_election.cc:336] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:40.027844 23322 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:14:40.029412 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:40.030341 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:14:40.030725 23547 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.060s	user 0.000s	sys 0.000s
I20260709 12:14:40.030472 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:40.032756 23411 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:40.017673 12423 catalog_manager.cc:5697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 33 to 34, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 34 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:40.037711 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:40.081382 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:40.082000 12675 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 33.
I20260709 12:14:40.007216 23175 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:39.972479 23326 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:40.062676 12679 raft_consensus.cc:2376] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 32. Re-sending same reply.
I20260709 12:14:40.071183 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:40.063179 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:40.087955 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:40.074070 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:40.087919 12600 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 33.
I20260709 12:14:40.091595 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:39.991027 23368 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:40.095566 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:40.097141 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:40.102442 12599 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:39.991281 23124 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:39.996505 23436 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:40.076081 23322 consensus_queue.cc:1243] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer e56d487365394ac5ac5f1002926a5645 log is divergent from this leader: its last log entry 30.19 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 18
I20260709 12:14:40.072851 23473 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:40.115594 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:40.115939 12646 consensus_peers.cc:597] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:40.103147 12680 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 32 index: 18. (index mismatch)
I20260709 12:14:40.157274 23545 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.148994 12674 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
W20260709 12:14:40.170907 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 51955.2us
W20260709 12:14:40.175868 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 69896.1us
I20260709 12:14:40.155849 12423 catalog_manager.cc:5697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 26 to 33. New cstate: current_term: 33 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:14:40.171898 12421 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58992 (request call id 175) took 105311 ms (client timeout 15000 ms). Trace:
I20260709 12:14:40.022018 23516 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:40.118347 23091 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:40.178594 12494 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:40.050191 23517 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:40.198958 23549 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.120s	user 0.000s	sys 0.000s
I20260709 12:14:40.199379 12423 catalog_manager.cc:5697] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 21 to 30. New cstate: current_term: 30 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:40.201300 12423 catalog_manager.cc:5697] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:40.203472 12423 catalog_manager.cc:5697] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 30 to 33, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 33 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:14:40.188916 23550 log.cc:927] Time spent T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.079s	user 0.001s	sys 0.000s
I20260709 12:14:40.196273 23355 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:14:40.213743 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:40.217199 12599 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:14:40.232399 23091 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 33.23, Last appended by leader: 23, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:40.236467 23364 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:40.271085 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 57481.5us
I20260709 12:14:40.271987 12643 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:40.273123 12494 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:40.273258 12643 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:40.239666 13331 pending_rounds.cc:85] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Aborting all ops after (but not including) 18
I20260709 12:14:40.274834 13331 pending_rounds.cc:107] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Aborting uncommitted NO_OP operation due to leader change: 30.19
I20260709 12:14:40.275205 13331 raft_consensus.cc:2889] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:40.245504 23368 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:40.247486 23548 log.cc:927] Time spent T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.086s	user 0.000s	sys 0.000s
I20260709 12:14:40.297063 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:40.248728 23268 consensus_meta.cc:287] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: Time spent flushing consensus metadata: real 0.571s	user 0.008s	sys 0.004s
W20260709 12:14:40.178588 12421 rpcz_store.cc:269] 0709 12:12:54.857471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:54.860732 (+  3261us) service_pool.cc:224] Handling call
0709 12:14:40.168616 (+105307884us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:32.777028 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:33.054669 (+277641us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:33.054731 (+    62us) write_op.cc:435] Acquired schema lock
0709 12:14:33.054764 (+    33us) tablet.cc:662] Decoding operations
0709 12:14:33.650466 (+595702us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:33.650548 (+    82us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:33.650574 (+    26us) tablet.cc:685] Acquiring locks for 13 operations
0709 12:14:33.651828 (+  1254us) tablet.cc:701] Row locks acquired
0709 12:14:33.651880 (+    52us) write_op.cc:260] PREPARE: finished
0709 12:14:33.652059 (+   179us) write_op.cc:270] Start()
0709 12:14:33.652228 (+   169us) write_op.cc:276] Timestamp: P: 1783599273651989 usec, L: 0
0709 12:14:33.652247 (+    19us) op_driver.cc:348] REPLICATION: starting
0709 12:14:33.679529 (+ 27282us) log.cc:844] Serialized 5198 byte log entry
0709 12:14:35.310880 (+1631351us) op_driver.cc:464] REPLICATION: finished
0709 12:14:36.264192 (+953312us) spinlock_profiling.cc:243] Waited 8.77 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14859160ac 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f148c456ef0
0709 12:14:36.674958 (+410766us) write_op.cc:301] APPLY: starting
0709 12:14:36.675072 (+   114us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:36.844615 (+169543us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:36.844659 (+    44us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:37.139447 (+294788us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:37.139955 (+   508us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=26,key_file_lookups=26,delta_file_lookups=0,mrs_lookups=0
0709 12:14:37.140035 (+    80us) write_op.cc:312] APPLY: finished
0709 12:14:37.140698 (+   663us) log.cc:844] Serialized 121 byte log entry
0709 12:14:37.223381 (+ 82683us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:37.264304 (+ 40923us) write_op.cc:454] Released schema lock
0709 12:14:37.316320 (+ 52016us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:37.350295 (+ 33975us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:37.350329 (+    34us) write_op.cc:454] Released schema lock
0709 12:14:38.165079 (+814750us) spinlock_profiling.cc:243] Waited 21.2 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14885bb86e 00007f14885babb6 00007f14885ba7d8 00007f14885c4221 00007f14885c41d9 00007f14885c4169
Metrics: {"mutex_wait_us":2508,"reported_tablets":52,"child_traces":[["op",{"apply.queue_time_us":1350048,"apply.run_cpu_time_us":43814,"apply.run_wall_time_us":675320,"cfile_cache_hit":54,"cfile_cache_hit_bytes":58389,"num_ops":13,"prepare.queue_time_us":258361,"prepare.run_cpu_time_us":74702,"prepare.run_wall_time_us":1259902,"raft.queue_time_us":684958,"raft.run_cpu_time_us":18572,"raft.run_wall_time_us":1288870,"replication_time_us":1576156,"spinlock_wait_cycles":68979968,"thread_start_us":1813220,"threads_started":5,"wal-append.queue_time_us":368893,"wal-append.run_cpu_time_us":67227,"wal-append.run_wall_time_us":3949328}]]}
I20260709 12:14:40.182400 23545 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:40.340553 12421 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58980 (request call id 189) took 15852 ms (client timeout 15000 ms). Trace:
W20260709 12:14:40.340961 12421 rpcz_store.cc:269] 0709 12:14:24.504866 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:40.340285 (+15835419us) service_pool.cc:211] Skipping call since client already timed out
0709 12:14:40.340442 (+   157us) inbound_call.cc:177] Queueing failure response
Metrics: {}
I20260709 12:14:40.275463 12423 catalog_manager.cc:5697] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 23 to 28. New cstate: current_term: 28 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:40.343043 12423 catalog_manager.cc:5697] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 22 to 32. New cstate: current_term: 32 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:40.294201 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:40.344403 12680 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 30
I20260709 12:14:40.348078 12496 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:40.351202 12423 catalog_manager.cc:5697] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 27 to 29. New cstate: current_term: 29 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:40.263561 23556 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.271066 23355 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:14:40.371241 11318 thread.cc:527] Waited for 6000ms trying to join with heartbeat (tid 12556)
I20260709 12:14:40.331702 23557 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:40.353168 23240 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:40.352284 23322 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:40.352979 23190 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:40.355904 22990 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.358902 23228 raft_consensus.cc:493] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.365715 23364 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:40.274058 23357 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:40.369654 23473 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:40.392998 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 45003.6us
I20260709 12:14:40.398408 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:40.371338 23355 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:40.376573 23429 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.352058 23268 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:40.394061 23435 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:40.475234 12602 raft_consensus.cc:1240] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 30. Current term is 31. Ops: []
I20260709 12:14:40.467358 12680 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
W20260709 12:14:40.468389 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 70087.1us
I20260709 12:14:40.402290 23535 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:40.499667 12496 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:40.393051 23190 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:40.394196 22990 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:40.397496 23228 raft_consensus.cc:515] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:40.404335 23450 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:14:40.518298 23512 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.062s	user 0.001s	sys 0.000s
I20260709 12:14:40.475103 23322 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:40.404732 23473 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:40.535342 23289 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:40.526925 23225 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:40.536255 23473 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:40.536700 23225 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:40.537947 23572 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.053s	user 0.000s	sys 0.000s
I20260709 12:14:40.539037 23225 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:40.375222 23556 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:40.524081 23527 log.cc:927] Time spent T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.051s	user 0.000s	sys 0.001s
I20260709 12:14:40.517874 23464 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:40.488250 23518 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:14:40.530802 23545 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:40.595842 23558 log.cc:927] Time spent T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.239s	user 0.001s	sys 0.000s
I20260709 12:14:40.460515 23429 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:40.594875 23562 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.319s	user 0.002s	sys 0.000s
I20260709 12:14:40.533712 23322 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 28.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:40.567940 23464 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:40.636199 23175 consensus_meta.cc:287] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.534s	user 0.012s	sys 0.001s
I20260709 12:14:40.645599 23357 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:40.668949 23517 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:40.670786 23516 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:40.675401 23436 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:40.685192 23567 log.cc:927] Time spent T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.394s	user 0.005s	sys 0.000s
I20260709 12:14:40.688476 23464 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:40.692559 23571 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.149s	user 0.000s	sys 0.000s
I20260709 12:14:40.693594 23556 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:40.693794 23573 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.056s	user 0.001s	sys 0.000s
W20260709 12:14:40.698427 23544 consensus_meta.cc:287] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.677s	user 0.006s	sys 0.008s
I20260709 12:14:40.699033 23544 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:40.701951 23544 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:40.754630 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 419346us
W20260709 12:14:40.756958 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 257245us
I20260709 12:14:40.753911 12601 raft_consensus.cc:1240] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 32. Current term is 33. Ops: [32.18-32.18]
I20260709 12:14:40.766634 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:40.767707 12496 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:40.787422 22990 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:40.788390 23190 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:40.791814 23124 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:40.794423 23190 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:40.795742 23190 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:40.799211 23228 leader_election.cc:290] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:40.800273 23228 raft_consensus.cc:493] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.801255 23228 raft_consensus.cc:515] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:40.649958 23268 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:40.802528 23124 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:40.815186 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:40.815732 23326 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:40.817230 23338 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:40.820048 23225 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:40.821504 23225 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:40.824879 23225 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:40.828131 23175 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.829219 23175 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:40.833133 23175 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:40.811133 23253 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:40.833963 23289 consensus_queue.cc:1059] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 30.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:14:40.804648 23228 leader_election.cc:290] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:40.813547 23124 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:40.844208 23429 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:40.847045 23473 consensus_queue.cc:1059] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 30.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:14:40.849874 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:40.785694 12496 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:40.852025 13331 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:40.959576 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:14:40.961195 23587 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:40.962143 23587 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:40.967118 11318 tablet_replica.cc:333] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:40.967002 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:40.967900 23253 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:40.970067 23253 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:40.973191 23253 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:40.975473 23124 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:40.975832 13330 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:40.976828 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 191282us
I20260709 12:14:40.978535 23190 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:40.987267 23289 raft_consensus.cc:3055] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:14:40.987990 23289 raft_consensus.cc:740] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:40.988032 11318 raft_consensus.cc:2243] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:14:40.975217 23473 raft_consensus.cc:3055] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:14:40.990607 23473 raft_consensus.cc:740] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:40.980386 23587 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:40.976368 23429 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:14:41.032079 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 95252.9us
I20260709 12:14:41.040434 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.043552 23124 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:41.050042 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 32 candidate_status { last_received { term: 31 index: 23 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.020007 23473 consensus_queue.cc:260] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.052125 23473 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:41.058846 23587 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.059746 23587 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.049165 23124 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.041556 23289 consensus_queue.cc:260] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 32.18, Last appended by leader: 18, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.075326 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 34 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.066929 23253 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:41.086485 23124 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:41.087263 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.088173 12604 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:41.089648 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.079838 23253 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.070349 23289 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:41.094662 12597 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 30.
I20260709 12:14:41.103224 11318 pending_rounds.cc:70] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Trying to abort 3 pending ops.
I20260709 12:14:41.108603 23253 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:41.112401 23428 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.113392 23428 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.118775 23595 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:41.119926 23595 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:41.122778 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 81349.2us
I20260709 12:14:41.135917 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.136037 11318 pending_rounds.cc:77] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting op as it isn't in flight: id { term: 30 index: 19 } timestamp: 7305622650209030144 op_type: NO_OP noop_request { }
I20260709 12:14:41.170557 11318 raft_consensus.cc:2889] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:41.170977 11318 pending_rounds.cc:77] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting op as it isn't in flight: id { term: 28 index: 18 } timestamp: 7305622610089644032 op_type: NO_OP noop_request { }
I20260709 12:14:41.171391 11318 raft_consensus.cc:2889] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:41.171705 11318 pending_rounds.cc:77] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting op as it isn't in flight: id { term: 27 index: 17 } timestamp: 7305622574219407360 op_type: NO_OP noop_request { }
I20260709 12:14:41.172102 11318 raft_consensus.cc:2889] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:41.172492 11318 raft_consensus.cc:2272] T bcef55b2ef044bcb9d9a9ffd855c20ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:41.146647 23595 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:41.176182 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:41.177040 12671 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:41.177644 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:41.178732 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.196552 23587 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:41.197285 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:41.198503 12494 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.198654 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.203074 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.200471 12598 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
W20260709 12:14:41.126093 23355 consensus_meta.cc:287] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.720s	user 0.019s	sys 0.006s
I20260709 12:14:41.211339 12599 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:14:41.211373 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.219205 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:41.219518 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.219931 12598 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:14:41.224823 23575 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: Append to log took a long time: real 0.087s	user 0.001s	sys 0.000s
I20260709 12:14:41.225592 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:41.225955 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:41.226018 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:41.227502 23587 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:41.213869 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.247763 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.260555 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:41.263877 23463 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:41.270828 23595 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:41.264660 23463 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.235200 23355 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:41.275223 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 49821.4us
I20260709 12:14:41.272011 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.274770 12605 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:14:41.272756 23557 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:41.286199 23322 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:41.287729 23557 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.288318 23322 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:41.288808 23322 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:41.272956 23595 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.291997 23557 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:41.292541 23595 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:41.293161 12602 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:14:41.282473 23587 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.298992 23593 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:41.299813 23593 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.301322 23587 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:41.307325 23355 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:41.294266 12600 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:51782 (request call id 4665) took 1116 ms. Trace:
I20260709 12:14:41.348928 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.347714 23593 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:41.362322 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.362351 23364 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:41.363430 12605 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 32.
I20260709 12:14:41.363430 23364 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.364670 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.365449 12605 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 30.
I20260709 12:14:41.366659 23364 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:41.368933 23463 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:41.378682 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.380877 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.314946 23428 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:41.381989 12604 raft_consensus.cc:2393] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 27: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:41.388209 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:41.400278 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 147935us
I20260709 12:14:41.409060 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.411741 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.415887 12671 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 33.
I20260709 12:14:41.418104 23556 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:41.418960 23428 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:14:41.426138 23556 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.429560 23556 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:41.432602 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.412587 12606 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 33.
I20260709 12:14:41.431560 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.440353 23368 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.441277 23368 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:41.444275 23368 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:41.445215 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:41.434269 12646 leader_election.cc:336] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:41.450455 12646 consensus_peers.cc:597] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:41.340384 12600 rpcz_store.cc:276] 0709 12:14:40.085089 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:40.253918 (+168829us) service_pool.cc:224] Handling call
0709 12:14:40.257577 (+  3659us) raft_consensus.cc:1471] Updating replica for 0 ops
0709 12:14:40.257816 (+   239us) raft_consensus.cc:1534] Early marking committed up to index 20
0709 12:14:40.567494 (+309678us) log.cc:844] Serialized 14 byte log entry
0709 12:14:40.948832 (+381338us) raft_consensus.cc:1539] Triggering prepare for 0 ops
0709 12:14:41.007403 (+ 58571us) raft_consensus.cc:1645] Marking committed up to 20
0709 12:14:41.085813 (+ 78410us) raft_consensus.cc:1697] Filling consensus response to leader.
0709 12:14:41.095446 (+  9633us) raft_consensus.cc:1691] UpdateReplicas() finished
0709 12:14:41.201539 (+106093us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":16947,"prepare.queue_time_us":144914,"prepare.run_cpu_time_us":84,"prepare.run_wall_time_us":8802,"raft.queue_time_us":51608,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":188,"thread_start_us":584549,"threads_started":3,"wal-append.queue_time_us":99440}
I20260709 12:14:41.455487 12598 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:14:41.453773 12597 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 30.
I20260709 12:14:41.467295 23040 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.467411 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:41.473237 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:41.474314 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:41.475912 12674 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 33.
I20260709 12:14:41.481468 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:41.491464 23364 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:41.491585 23428 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:14:41.492121 23364 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:41.492566 23364 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:41.494053 23368 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:41.494886 23368 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:41.495880 12494 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:41.498442 23429 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:14:41.499104 23429 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:41.500396 23429 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 29.19, Last appended by leader: 19, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.468381 23040 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.488605 12571 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:41.514545 23040 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:41.518890 12598 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:41.520792 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.524933 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 35 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.523237 12599 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 27.
W20260709 12:14:41.529902 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 51551.6us
I20260709 12:14:41.531404 11318 tablet_replica.cc:333] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:41.532533 12598 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:41.543517 23595 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:14:41.547147 23595 raft_consensus.cc:697] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:41.548511 23595 consensus_queue.cc:237] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.544147 12571 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:41.559041 23368 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:41.564178 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8c7e0eddcc2b4449b7ed8821854646e2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 27 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:41.569218 12680 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:14:41.574869 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:41.579025 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bdfb480850c24efa8a38a916d53bdb1f" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 31 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:41.580770 12677 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:41.581398 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:41.583871 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 121731us
I20260709 12:14:41.587520 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 36 candidate_status { last_received { term: 35 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.585911 23533 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:14:41.600660 23533 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:41.602563 23533 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:41.585919 23546 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:41.617316 12497 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:41.622316 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee30d7cd60b04485b7e19c8d119d67bd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:41.635842 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:41.627697 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:41.644757 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 212238us
I20260709 12:14:41.654147 23577 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:14:41.657011 12646 leader_election.cc:336] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:41.657301 23322 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:41.660399 12646 leader_election.cc:336] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:41.661056 23606 log.cc:927] Time spent T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.214s	user 0.003s	sys 0.000s
I20260709 12:14:41.641582 23557 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.663100 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:41.665614 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 131676us
I20260709 12:14:41.669168 12676 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:14:41.673010 12598 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 34.
I20260709 12:14:41.652401 23587 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:41.646781 23546 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.687733 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:41.679816 12646 leader_election.cc:336] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:41.687536 23557 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.709215 23557 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:41.705346 23577 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:41.713474 23577 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:41.713941 23577 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:41.716699 23175 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.717506 23175 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.725232 23609 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:41.726856 23607 log.cc:927] Time spent T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.264s	user 0.004s	sys 0.000s
W20260709 12:14:41.730371 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 67418.4us
I20260709 12:14:41.731441 12643 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:41.733083 23600 log.cc:927] Time spent T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: Append to log took a long time: real 0.281s	user 0.004s	sys 0.000s
I20260709 12:14:41.733739 12671 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 35 index: 24. (index mismatch)
I20260709 12:14:41.734575 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:41.735229 12680 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:41.736014 23322 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:41.736464 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6fa9b4ceed3443f2a6992343ab6ad877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 35 candidate_status { last_received { term: 32 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:41.737993 11318 raft_consensus.cc:2243] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:14:41.739233 11318 raft_consensus.cc:2272] T 3a68955866d543408acca9e959ad1b26 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:41.733184 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:41.743255 23556 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:41.744061 23556 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:41.745005 12494 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:41.746027 23228 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:41.746482 23228 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:41.746975 23228 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:41.750864 12494 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:41.751868 23338 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:14:41.752393 23338 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:41.691948 23587 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.754084 23338 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 32.18, Last appended by leader: 18, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.756101 23587 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:41.759822 12671 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:41.759963 23546 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:41.760704 12671 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 26.
W20260709 12:14:41.702790 12646 leader_election.cc:336] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:41.765403 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 71698.4us
I20260709 12:14:41.745038 12643 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:41.774042 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 69646.1us
W20260709 12:14:41.775460 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 177297us
W20260709 12:14:41.771831 12646 leader_election.cc:336] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:41.720381 23175 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:41.783674 12643 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:41.789677 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:41.798708 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:41.773300 23364 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.837857 23364 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:41.840571 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 74278.4us
W20260709 12:14:41.841434 12646 leader_election.cc:336] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:41.799217 23587 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:14:41.792445 23557 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:41.804106 23556 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:41.725977 23609 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:41.873636 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:41.843942 12674 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 29: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:41.859992 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 36 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:41.842095 12646 leader_election.cc:336] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:41.873852 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 77233.7us
W20260709 12:14:41.850267 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 148179us
I20260709 12:14:41.887765 12677 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:14:41.888540 12676 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 35.
I20260709 12:14:41.842892 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:41.891806 12606 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:41.889249 12646 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:41.897449 12680 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 34.
W20260709 12:14:41.892956 13305 leader_election.cc:336] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:41.902709 13305 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:41.866365 11318 tablet_replica.cc:333] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:41.868440 23556 raft_consensus.cc:493] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:41.868096 23557 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:41.859025 23428 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:41.887566 23364 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:41.917529 23556 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:41.920830 23364 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.923199 23364 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:41.926505 23495 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:41.867617 23533 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:41.931097 12599 raft_consensus.cc:1240] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 33. Current term is 34. Ops: []
I20260709 12:14:41.932476 23533 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:41.935202 11318 raft_consensus.cc:2243] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:41.935851 11318 raft_consensus.cc:2272] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:41.867309 23587 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:41.938216 23587 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:14:41.942283 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 260129us
W20260709 12:14:41.964622 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 71694.7us
I20260709 12:14:41.919706 23557 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:41.972622 12606 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:41.976048 12494 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:41.946642 23595 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:41.974886 12572 leader_election.cc:336] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:41.984961 12676 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:41.957849 23473 raft_consensus.cc:493] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:41.987052 12494 raft_consensus.cc:2686] T 33b7d46307bc4c33acd6c9a58055c7af P a2fdbe7eef6748a9b4611f6c9bc9b201: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:14:41.988370 12572 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:41.967106 23428 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:42.021150 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:42.022475 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 180394us
W20260709 12:14:42.023196 12646 leader_election.cc:336] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:42.023780 12646 leader_election.cc:336] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:42.024781 12646 leader_election.cc:336] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:42.025346 12646 leader_election.cc:336] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:42.026218 12646 leader_election.cc:336] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:42.026402 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: 
W20260709 12:14:42.026681 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 65528.5us
W20260709 12:14:42.026903 12646 leader_election.cc:336] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:42.027266 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 61920.6us
I20260709 12:14:42.027182 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:42.028716 12677 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in current term 29: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:14:42.027391 12572 leader_election.cc:336] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:42.031543 23624 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Leader pre-election won for term 36
I20260709 12:14:42.034979 23624 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:42.035390 23624 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:14:42.035934 23609 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:42.027680 12646 leader_election.cc:336] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:41.989415 23577 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:42.014731 23463 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:41.992592 23473 raft_consensus.cc:515] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:42.006806 23595 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:42.039366 23364 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:14:42.046558 12646 leader_election.cc:336] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:42.080469 12674 raft_consensus.cc:1240] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 30. Current term is 31. Ops: []
W20260709 12:14:42.077068 12646 leader_election.cc:336] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:42.032027 23228 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:42.090250 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 124633us
I20260709 12:14:42.095155 12496 leader_election.cc:304] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:42.102633 23428 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:42.103987 11318 tablet_replica.cc:333] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:42.100968 23556 raft_consensus.cc:515] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:42.123561 23175 raft_consensus.cc:2749] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:14:42.126716 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 58263.8us
W20260709 12:14:42.126897 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49872.9us
W20260709 12:14:42.128106 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 100747us
I20260709 12:14:42.133358 23473 leader_election.cc:290] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:42.139497 12496 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.141003 12496 leader_election.cc:304] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:42.141527 23587 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:42.142899 23609 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:42.143702 23609 raft_consensus.cc:2749] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:42.144531 23587 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:42.147342 23595 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:42.155251 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:42.156040 12600 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:42.164636 12496 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.175765 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:42.178858 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:42.173177 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:42.187136 12606 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:14:42.193714 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 37 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:42.192616 23595 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.206189 12496 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.200556 11318 raft_consensus.cc:2243] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:14:42.212838 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:42.220706 23228 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:42.227725 12496 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:42.226615 13305 leader_election.cc:336] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:42.232543 13305 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.210273 23473 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:42.235132 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.248039 12643 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.261883 23577 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:42.220724 22990 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:42.261794 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ee47f7c4414b4ae98233a31696e99cc0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 37 candidate_status { last_received { term: 33 index: 23 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:42.268185 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.213852 23609 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.041s
I20260709 12:14:42.215588 23595 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:42.275595 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:42.253562 23289 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:14:42.278796 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.286180 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.280499 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:42.284122 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:42.295312 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 68720.7us
I20260709 12:14:42.295969 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:42.298964 12599 raft_consensus.cc:2393] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 34: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
W20260709 12:14:42.296401 13305 leader_election.cc:336] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:42.303656 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "317be0a176ee473b9b3631bdc1030bcc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:42.303815 12602 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:42.270833 23587 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:42.303581 13305 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.307937 12602 raft_consensus.cc:2393] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 29: Already voted for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in this term.
I20260709 12:14:42.307796 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:42.308983 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:42.264755 23228 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.035s
I20260709 12:14:42.281886 23577 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:14:42.320663 12495 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:14:42.322546 12603 raft_consensus.cc:2393] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 31: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:42.298740 23368 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:14:42.321216 12646 leader_election.cc:336] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:42.308463 23618 log.cc:927] Time spent T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: Append to log took a long time: real 0.143s	user 0.000s	sys 0.002s
I20260709 12:14:42.327221 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:42.329124 13305 leader_election.cc:336] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:42.329999 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.333673 23629 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:42.334767 23621 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:42.313459 23536 consensus_queue.cc:1059] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 33.17, Next index: 18, Last known committed idx: 17, Time since last communication: 0.033s
I20260709 12:14:42.366511 23629 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:42.366917 23556 leader_election.cc:290] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:42.367403 23629 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:14:42.367678 23556 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:42.367856 23629 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:42.368353 23629 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:42.400576 12600 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
W20260709 12:14:42.404563 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 161809us
I20260709 12:14:42.406081 23641 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.313697 23557 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:42.407893 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 76991us
I20260709 12:14:42.327967 23624 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 36 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:42.412447 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 83359.1us
I20260709 12:14:42.411480 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:42.413462 13305 consensus_peers.cc:597] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:42.402856 23595 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:42.417636 13305 leader_election.cc:336] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:42.427862 13305 leader_election.cc:336] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:42.428303 13305 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.430702 13337 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 36 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 35. Current term is 36. Ops: []
I20260709 12:14:42.436426 23629 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:42.430552 12679 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
W20260709 12:14:42.490492 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 79648.9us
I20260709 12:14:42.442327 23635 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:14:42.487190 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 59356.8us
I20260709 12:14:42.415787 23368 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:42.497010 23638 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:14:42.520696 23587 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:14:42.502450 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 72747.7us
I20260709 12:14:42.521976 12571 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:42.502578 23616 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:14:42.525105 12494 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:42.525972 23616 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:42.526342 12494 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.526588 23616 raft_consensus.cc:697] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:42.510751 23624 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:42.516050 23629 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:42.532048 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 204893us
W20260709 12:14:42.529460 12646 leader_election.cc:336] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:42.533043 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.534608 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:42.406970 23641 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:42.556366 12494 leader_election.cc:304] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:42.555382 12646 leader_election.cc:336] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:42.568185 23612 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:42.568882 23612 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:42.569271 23612 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:42.574061 23595 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:42.574846 23595 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:42.578042 23595 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:42.578665 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:42.527881 23616 consensus_queue.cc:237] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 13, Committed index: 13, Last appended: 26.14, Last appended by leader: 14, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:42.564854 12646 leader_election.cc:336] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:42.532064 23473 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
W20260709 12:14:42.591104 12646 leader_election.cc:336] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:42.588999 13305 leader_election.cc:336] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:42.593458 12606 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:14:42.563968 23364 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:42.613821 23364 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:42.614295 23364 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:42.600841 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:42.623576 12569 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:14:42.625756 23637 raft_consensus.cc:932] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 LEADER]: Unable to handle notification of new term (34): Illegal state: RaftConsensus is not running: State = Stopping
W20260709 12:14:42.595957 12646 leader_election.cc:336] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:42.610898 12604 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 32 index: 18. Preceding OpId from leader: term: 34 index: 19. (index mismatch)
I20260709 12:14:42.592089 22990 consensus_queue.cc:1059] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 24, Last known committed idx: 22, Time since last communication: 0.020s
I20260709 12:14:42.611817 23612 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:42.598098 23536 raft_consensus.cc:2749] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:14:42.603048 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.627048 11318 raft_consensus.cc:2272] T 111842005e864eab9966b3a00ab9ae8a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:42.627664 23644 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:14:42.629426 12646 leader_election.cc:336] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:42.632789 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 140699us
I20260709 12:14:42.642956 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.652419 12606 raft_consensus.cc:1240] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 30. Current term is 31. Ops: [30.18-30.18]
I20260709 12:14:42.649653 12643 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.658788 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.643935 23595 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:42.657496 23644 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:42.650389 23587 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:42.650962 23629 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:42.664731 23622 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election lost for term 29. Reason: could not achieve majority
I20260709 12:14:42.670243 12643 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.675719 23641 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:42.651104 23040 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.019s
I20260709 12:14:42.680140 12643 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.680536 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.684369 23338 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:42.688825 11318 tablet_replica.cc:333] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:42.688982 23636 raft_consensus.cc:3055] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 LEADER]: Stepping down as leader of term 35
I20260709 12:14:42.689352 23557 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:42.689494 23636 raft_consensus.cc:740] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:42.690022 11318 raft_consensus.cc:2243] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:14:42.690994 23636 consensus_queue.cc:260] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 35.24, Last appended by leader: 24, Current term: 35, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:42.692620 23636 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:14:42.695446 23612 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:42.679566 23629 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader election lost for term 33. Reason: could not achieve majority
I20260709 12:14:42.700232 23364 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:42.706430 23091 consensus_queue.cc:1059] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 30.15, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:14:42.710489 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 26 candidate_status { last_received { term: 24 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:42.711016 23091 raft_consensus.cc:3055] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:14:42.711434 23091 raft_consensus.cc:740] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:42.712452 23091 consensus_queue.cc:260] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 30.15, Last appended by leader: 15, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:42.708933 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:42.717653 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 25 index: 12 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:42.723829 12675 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
I20260709 12:14:42.726016 23641 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.727029 23175 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:14:42.727649 23355 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.726898 23641 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:42.730401 23641 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:42.723894 23587 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:42.731299 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:42.732070 12675 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:42.732184 23355 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:42.737560 23040 consensus_queue.cc:1059] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: INVALID_TERM, Last received: 28.17, Next index: 18, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:14:42.738629 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.738899 23040 raft_consensus.cc:3055] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:14:42.739336 23040 raft_consensus.cc:740] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:42.739192 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:42.740212 23355 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:42.740355 23040 consensus_queue.cc:260] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.18, Last appended by leader: 18, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:42.741647 23587 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:14:42.721977 12646 leader_election.cc:336] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:42.719998 23091 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
W20260709 12:14:42.749526 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 142437us
I20260709 12:14:42.750689 12672 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 29 index: 19. Preceding OpId from leader: term: 31 index: 20. (index mismatch)
I20260709 12:14:42.752036 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:42.755553 12679 raft_consensus.cc:1240] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 32. Current term is 33. Ops: []
I20260709 12:14:42.752899 12604 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:14:42.725955 23364 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:42.741658 23040 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:42.762254 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 36 candidate_status { last_received { term: 32 index: 22 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:42.763160 12678 raft_consensus.cc:3060] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Advancing to term 36
I20260709 12:14:42.765064 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcef55b2ef044bcb9d9a9ffd855c20ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:42.765831 12603 raft_consensus.cc:3060] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:42.756995 11318 raft_consensus.cc:2272] T 19eaf42a0f664c919842f3c2f5cb868f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:42.769742 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:42.774727 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ef80db1de7994402abd14cc926713fb0" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 31 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:42.768316 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:42.781124 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:42.781960 12674 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 32 index: 18. Preceding OpId from leader: term: 34 index: 19. (index mismatch)
I20260709 12:14:42.783557 12675 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:42.787693 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:42.787734 12494 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:42.792092 12572 leader_election.cc:336] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:42.792671 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:42.794292 22990 raft_consensus.cc:493] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.794989 23289 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.794937 22990 raft_consensus.cc:515] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:42.795759 23289 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:42.797509 12496 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.799482 23609 raft_consensus.cc:493] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.797597 22990 leader_election.cc:290] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:42.801824 12676 raft_consensus.cc:1240] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 30. Current term is 31. Ops: [30.19-30.19]
I20260709 12:14:42.804507 12671 raft_consensus.cc:1275] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 33 index: 18. (index mismatch)
I20260709 12:14:42.804606 23605 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:42.807709 12605 raft_consensus.cc:2393] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 34: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:42.811408 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:42.814211 12678 raft_consensus.cc:2468] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 36 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 36.
I20260709 12:14:42.806483 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:42.794705 11318 tablet_replica.cc:333] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:42.815599 23429 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:42.820174 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:42.822520 12674 raft_consensus.cc:2468] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 33.
I20260709 12:14:42.816252 12599 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7bd864b75c34027b05582a55b152877" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 29 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:42.797978 23289 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:42.832530 12672 raft_consensus.cc:1240] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 36 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 35. Current term is 36. Ops: [33.23-35.24]
I20260709 12:14:42.833678 23609 raft_consensus.cc:515] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:42.837750 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 45727.3us
W20260709 12:14:42.839205 12572 leader_election.cc:336] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:42.843196 23175 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:14:42.845132 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:42.852409 23636 consensus_queue.cc:1059] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 32.19, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:14:42.852967 12496 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.854425 23429 raft_consensus.cc:3055] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: Stepping down as leader of term 32
I20260709 12:14:42.849275 12679 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 28 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:42.855886 12679 raft_consensus.cc:3060] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:42.854601 23338 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:42.855193 23429 raft_consensus.cc:740] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:42.858426 23429 consensus_queue.cc:260] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 32, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:42.865007 23429 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:42.862799 13304 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:42.867215 23225 raft_consensus.cc:2804] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 36 FOLLOWER]: Leader election won for term 36
I20260709 12:14:42.868002 23225 raft_consensus.cc:697] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 36 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:42.862485 23637 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:14:42.864888 23593 consensus_queue.cc:1048] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:14:42.877650 23593 consensus_queue.cc:1243] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Peer 193a40da6e364ead8ab561e14c3cc8d1 log is divergent from this leader: its last log entry 27.16 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 13
W20260709 12:14:42.868928 12572 leader_election.cc:336] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:42.879822 12572 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:42.879009 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 72011.3us
I20260709 12:14:42.870368 12494 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.886536 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:42.883814 12646 leader_election.cc:336] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:42.893112 12646 leader_election.cc:336] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:42.892452 23587 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:42.895833 12680 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:14:42.898149 12496 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:42.899787 12673 raft_consensus.cc:1217] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Deduplicated request from leader. Original: 21.13->[23.14-33.18]   Dedup: 25.15->[28.16-33.18]
I20260709 12:14:42.898633 12679 raft_consensus.cc:2468] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 31.
I20260709 12:14:42.902505 12671 raft_consensus.cc:1240] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Rejecting Update request from peer a2fdbe7eef6748a9b4611f6c9bc9b201 for earlier term 27. Current term is 28. Ops: []
I20260709 12:14:42.901949 12571 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:42.818377 23355 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:42.900877 12673 pending_rounds.cc:85] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting all ops after (but not including) 15
I20260709 12:14:42.909183 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "952cdab9198b44e4bdf1b472109715c3" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 30 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:42.907284 12673 pending_rounds.cc:107] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1: Aborting uncommitted NO_OP operation due to leader change: 27.16
I20260709 12:14:42.911068 12673 raft_consensus.cc:2889] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:14:42.906526 23355 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:42.910936 23338 raft_consensus.cc:2804] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Leader pre-election won for term 34
W20260709 12:14:42.924077 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 55183.2us
W20260709 12:14:42.927156 12572 leader_election.cc:336] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:42.912002 23636 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:42.925034 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b7cccd56fa854cbc84a922238ad4add9" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:42.912950 23647 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:14:42.918510 23595 raft_consensus.cc:2804] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election won for term 31
W20260709 12:14:42.936894 12572 leader_election.cc:336] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:42.937476 12572 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:42.913594 23641 consensus_queue.cc:1059] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 28.18, Next index: 19, Last known committed idx: 18, Time since last communication: 0.019s
I20260709 12:14:42.927073 23355 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:42.926789 23338 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:42.928349 22990 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:42.964718 12496 leader_election.cc:304] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:42.970755 23605 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:42.972316 23609 leader_election.cc:290] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:42.973001 22990 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:42.974712 23595 raft_consensus.cc:697] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:42.975740 22990 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:42.976506 22990 raft_consensus.cc:2749] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:14:42.976979 23616 consensus_queue.cc:1059] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: INVALID_TERM, Last received: 27.16, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:14:42.976358 12496 leader_election.cc:304] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:42.978627 23338 raft_consensus.cc:3060] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:42.975886 23595 consensus_queue.cc:237] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 28.17, Last appended by leader: 17, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:42.982183 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:42.991999 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 55116.3us
W20260709 12:14:42.993363 12572 leader_election.cc:336] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:42.993814 12572 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:42.996657 22990 raft_consensus.cc:2749] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:43.004066 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33a0b4d6cfba4ab2b3d54f591439b179" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:42.936312 23225 consensus_queue.cc:237] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 22, Committed index: 22, Last appended: 32.22, Last appended by leader: 22, Current term: 36, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:43.025157 23428 raft_consensus.cc:493] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:43.031878 23636 raft_consensus.cc:3055] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Stepping down as leader of term 30
I20260709 12:14:43.032502 23636 raft_consensus.cc:740] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:43.036279 23473 raft_consensus.cc:3055] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Stepping down as leader of term 27
I20260709 12:14:43.037176 23473 raft_consensus.cc:740] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Becoming Follower/Learner. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
W20260709 12:14:43.039587 13305 consensus_peers.cc:597] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:43.055346 12494 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:43.061120 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 67780.8us
W20260709 12:14:43.061877 12572 leader_election.cc:336] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:43.062366 12572 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:43.063562 12572 leader_election.cc:336] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:43.064006 12572 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:43.033641 23636 consensus_queue.cc:260] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 30, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:43.067615 23636 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:43.068117 23596 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:43.068563 23616 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:43.025843 23428 raft_consensus.cc:515] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:43.070801 23612 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:43.070812 23473 consensus_queue.cc:260] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 27, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:43.072750 23428 leader_election.cc:290] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:43.073082 23473 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:43.076332 12680 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 26 index: 14. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
W20260709 12:14:43.078558 12572 leader_election.cc:336] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:43.079329 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:43.080400 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:43.059010 12603 raft_consensus.cc:2468] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 31.
I20260709 12:14:43.076275 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c00651839ae4132bb331bc6c2602641" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:43.085784 12572 leader_election.cc:336] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:43.094812 12572 leader_election.cc:336] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:43.095294 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:43.093806 23593 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:43.081022 23647 raft_consensus.cc:2749] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:43.122326 23637 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.001s
I20260709 12:14:43.125197 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:43.126633 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:43.131163 12643 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:43.097842 23091 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.136119 12494 leader_election.cc:304] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:43.144603 23364 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:14:43.134323 23629 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:14:43.137305 23612 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:43.132856 23587 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election lost for term 31. Reason: could not achieve majority
I20260709 12:14:43.156839 23609 raft_consensus.cc:2749] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:43.155586 23091 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:43.163615 23091 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:43.168275 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:43.191267 12601 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 24 index: 13. Preceding OpId from leader: term: 28 index: 15. (index mismatch)
I20260709 12:14:43.193403 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e40144c1c60847ee81b8252b107df67d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 36 candidate_status { last_received { term: 34 index: 22 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:43.202430 12494 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:43.205960 23609 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:14:43.215634 23616 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.216418 23616 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:43.228921 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:43.236466 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:43.238272 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 143481us
I20260709 12:14:43.239867 12643 leader_election.cc:304] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:43.225188 23616 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:43.253039 12572 leader_election.cc:336] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:43.253646 12572 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:43.257507 13305 consensus_peers.cc:597] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:43.239691 23587 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
W20260709 12:14:43.258932 12572 consensus_peers.cc:597] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:43.266911 23587 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:43.267503 23587 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:43.269654 23587 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:43.255277 23557 raft_consensus.cc:2804] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:14:43.270810 23557 raft_consensus.cc:697] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:43.271761 23557 consensus_queue.cc:237] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 28.18, Last appended by leader: 18, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:43.276959 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43d4539f52e24ea8adc368a4c9529e66" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:43.261724 23593 raft_consensus.cc:2749] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
W20260709 12:14:43.280576 23040 consensus_meta.cc:287] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Time spent flushing consensus metadata: real 0.514s	user 0.007s	sys 0.006s
I20260709 12:14:43.238168 23637 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 13, Time since last communication: 0.021s
I20260709 12:14:43.287388 23355 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:43.289433 12646 leader_election.cc:336] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:43.294023 12496 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:43.294538 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:43.295358 12673 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:43.296867 11318 raft_consensus.cc:2243] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.299208 11318 raft_consensus.cc:2272] T 0d1b28b57b3d4a9184fbb79b00227f3a P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.295207 23429 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:43.318244 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:43.319104 12603 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:14:43.321841 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 59340.1us
I20260709 12:14:43.289989 23355 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:43.331158 12572 leader_election.cc:336] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:43.348685 12572 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:43.351930 12673 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 32.
I20260709 12:14:43.263339 23338 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:43.352967 12496 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:43.355154 23463 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
I20260709 12:14:43.359324 12603 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 32.
I20260709 12:14:43.358587 23637 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:14:43.396472 12420 catalog_manager.cc:5697] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 24 to 27, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 27 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:43.398597 23338 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:43.398952 12423 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47524 (request call id 106) took 22373 ms (client timeout 15000 ms). Trace:
I20260709 12:14:43.399833 23637 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:43.401243 23637 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:43.399490 12423 rpcz_store.cc:269] 0709 12:14:21.017512 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:29.294028 (+8276516us) service_pool.cc:224] Handling call
0709 12:14:43.390497 (+14096469us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:41.366468 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:41.367416 (+   948us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:41.367489 (+    73us) write_op.cc:435] Acquired schema lock
0709 12:14:41.367521 (+    32us) tablet.cc:662] Decoding operations
0709 12:14:41.369196 (+  1675us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:41.369279 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:41.369316 (+    37us) tablet.cc:685] Acquiring locks for 12 operations
0709 12:14:41.370857 (+  1541us) tablet.cc:701] Row locks acquired
0709 12:14:41.370931 (+    74us) write_op.cc:260] PREPARE: finished
0709 12:14:41.371181 (+   250us) write_op.cc:270] Start()
0709 12:14:41.371387 (+   206us) write_op.cc:276] Timestamp: P: 1783599281371102 usec, L: 0
0709 12:14:41.371414 (+    27us) op_driver.cc:348] REPLICATION: starting
0709 12:14:41.377622 (+  6208us) log.cc:844] Serialized 4813 byte log entry
0709 12:14:41.744689 (+367067us) spinlock_profiling.cc:243] Waited 2.06 ms on lock 0x7b4400022b4c. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f1485859c85 00007f148585a6a2 00007f14885cc1db 00007f14885c7aaf 00007f14859160ac 00007f148e7709a9 00007f148593269b 00007f148593093f 00007f1485932c02 00007f14885ba20b
0709 12:14:42.635141 (+890452us) op_driver.cc:464] REPLICATION: finished
0709 12:14:42.951218 (+316077us) write_op.cc:301] APPLY: starting
0709 12:14:42.951370 (+   152us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:42.960694 (+  9324us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:42.960732 (+    38us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:42.987847 (+ 27115us) spinlock_profiling.cc:243] Waited 31.7 ms on lock 0x7b54000104d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1d32a 00007f14889a2f14 00007f14889a2ec3 00007f14889a2e38 00007f14889a2df3 00007f14889a20dc 00007f14889cbe16 00007f14889b3cbb 00007f148899e39d 00007f14889a1d75
0709 12:14:43.016791 (+ 28944us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:43.017189 (+   398us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=24,key_file_lookups=24,delta_file_lookups=0,mrs_lookups=0
0709 12:14:43.017252 (+    63us) write_op.cc:312] APPLY: finished
0709 12:14:43.017668 (+   416us) log.cc:844] Serialized 113 byte log entry
0709 12:14:43.017872 (+   204us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:43.018017 (+   145us) write_op.cc:454] Released schema lock
0709 12:14:43.018087 (+    70us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:43.019690 (+  1603us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:43.019731 (+    41us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":1592341,"reported_tablets":21,"child_traces":[["op",{"apply.queue_time_us":298588,"apply.run_cpu_time_us":19658,"apply.run_wall_time_us":72160,"cfile_cache_hit":50,"cfile_cache_hit_bytes":54215,"num_ops":12,"prepare.queue_time_us":811280,"prepare.run_cpu_time_us":63423,"prepare.run_wall_time_us":550951,"raft.queue_time_us":257757,"raft.run_cpu_time_us":106508,"raft.run_wall_time_us":395909,"replication_time_us":1263509,"spinlock_wait_cycles":77561984,"thread_start_us":1342147,"threads_started":4,"wal-append.queue_time_us":565356}]]}
I20260709 12:14:43.409627 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
W20260709 12:14:43.413053 13305 consensus_peers.cc:597] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:14:43.419145 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "05ff02a879ef4c94966348127fec27f7" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 34 candidate_status { last_received { term: 32 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:43.424340 12494 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 34 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:43.430845 11318 tablet_replica.cc:333] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:43.423519 12420 catalog_manager.cc:5697] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 17 to 27, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 27 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:43.434078 23616 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Leader election lost for term 34. Reason: could not achieve majority
I20260709 12:14:43.454644 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:43.455482 12606 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
W20260709 12:14:43.470146 12572 leader_election.cc:336] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:43.472355 12572 leader_election.cc:336] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:43.487023 13305 consensus_peers.cc:597] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:14:43.470314 12420 catalog_manager.cc:5697] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 18 to 29. New cstate: current_term: 29 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:43.494587 23355 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.495316 23355 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
W20260709 12:14:43.503057 13305 consensus_peers.cc:597] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:43.503330 11318 raft_consensus.cc:2243] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.504736 11318 pending_rounds.cc:70] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Trying to abort 1 pending ops.
I20260709 12:14:43.504403 23355 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:43.505079 11318 pending_rounds.cc:77] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting op as it isn't in flight: id { term: 32 index: 18 } timestamp: 7305622665053560832 op_type: NO_OP noop_request { }
I20260709 12:14:43.505119 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:43.505657 11318 raft_consensus.cc:2889] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:43.506086 11318 raft_consensus.cc:2272] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.502748 12420 catalog_manager.cc:5697] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 19 to 26, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 26 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:43.507865 12420 catalog_manager.cc:5697] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 16 to 28. New cstate: current_term: 28 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:43.511190 12420 catalog_manager.cc:5697] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 16 to 26, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 26 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:43.513011 12420 catalog_manager.cc:5697] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 9 to 27. New cstate: current_term: 27 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:43.524004 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:43.525552 23557 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:43.526021 23557 raft_consensus.cc:493] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:43.526463 23557 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:43.530653 12420 catalog_manager.cc:5697] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 27 to 28, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 28 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:43.532500 12420 catalog_manager.cc:5697] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 16 to 26, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 26 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:43.533435 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b65110412db1492aa47f4a845ec89242" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:43.537343 12420 catalog_manager.cc:5697] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 16 to 26, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 26 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:43.543740 12494 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:43.546581 23637 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:43.548269 12420 catalog_manager.cc:5697] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 26 to 31. New cstate: current_term: 31 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:14:43.550766 23668 consensus_queue.cc:1175] T 43ba56f9f94346338994fdfb09969ba8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" status { last_received { term: 32 index: 18 } last_committed_idx: 17 last_received_current_leader { term: 32 index: 18 } }
W20260709 12:14:43.550796 12572 consensus_peers.cc:597] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:43.552762 12420 catalog_manager.cc:5697] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 20 to 31, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 31 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:43.556341 11318 tablet_replica.cc:333] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:43.558506 11318 raft_consensus.cc:2243] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.559830 11318 raft_consensus.cc:2272] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.559695 23557 raft_consensus.cc:515] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:43.563437 11318 tablet_replica.cc:333] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:43.564692 12420 catalog_manager.cc:5697] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 16 to 23. New cstate: current_term: 23 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:43.569267 12420 catalog_manager.cc:5697] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 17 to 25, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 25 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:14:43.573118 12646 leader_election.cc:336] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:43.580525 12672 raft_consensus.cc:1275] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 28 index: 17. Preceding OpId from leader: term: 31 index: 18. (index mismatch)
I20260709 12:14:43.574664 11318 raft_consensus.cc:2243] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.581750 11318 pending_rounds.cc:70] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Trying to abort 2 pending ops.
I20260709 12:14:43.583432 23612 consensus_queue.cc:1048] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
I20260709 12:14:43.584187 11318 pending_rounds.cc:77] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting op as it isn't in flight: id { term: 31 index: 18 } timestamp: 7305622625718870016 op_type: NO_OP noop_request { }
I20260709 12:14:43.586320 11318 raft_consensus.cc:2889] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:43.587085 11318 pending_rounds.cc:77] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting op as it isn't in flight: id { term: 30 index: 17 } timestamp: 7305622577740783616 op_type: NO_OP noop_request { }
I20260709 12:14:43.587543 11318 raft_consensus.cc:2889] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:43.587972 11318 raft_consensus.cc:2272] T ef80db1de7994402abd14cc926713fb0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.585297 12420 catalog_manager.cc:5697] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 21 to 23. New cstate: current_term: 23 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:43.597427 12420 catalog_manager.cc:5697] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 12 to 22, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 22 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:43.600903 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3a68955866d543408acca9e959ad1b26" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:43.601804 12606 raft_consensus.cc:3060] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:43.602106 12420 catalog_manager.cc:5697] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 19 to 30, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 30 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } }
I20260709 12:14:43.604000 12420 catalog_manager.cc:5697] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 15 to 24. New cstate: current_term: 24 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:43.603194 11318 tablet_replica.cc:333] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:43.607666 11318 raft_consensus.cc:2243] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.612936 23612 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.620469 23612 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:43.624114 23612 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:43.615061 11318 raft_consensus.cc:2272] T 3c00651839ae4132bb331bc6c2602641 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.624699 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:43.625532 12672 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
I20260709 12:14:43.626947 12571 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:43.627818 23612 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:43.628178 11318 tablet_replica.cc:333] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:43.629498 11318 raft_consensus.cc:2243] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.628217 23612 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:43.630520 23612 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:43.633584 12672 raft_consensus.cc:1275] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 36 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 32 index: 22. Preceding OpId from leader: term: 36 index: 23. (index mismatch)
I20260709 12:14:43.612507 23557 leader_election.cc:290] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:43.636921 23624 consensus_queue.cc:1048] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 23, Last known committed idx: 22, Time since last communication: 0.000s
I20260709 12:14:43.645113 11318 raft_consensus.cc:2272] T 5771c48bd1d24152ab96cfc24da505ed P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.615342 12420 catalog_manager.cc:5697] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 16 to 23, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 23 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:43.637306 12606 raft_consensus.cc:2468] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29.
I20260709 12:14:43.658416 12420 catalog_manager.cc:5697] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 17 to 24. New cstate: current_term: 24 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:43.664404 12643 leader_election.cc:304] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:43.665470 23557 raft_consensus.cc:2804] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:43.666316 23557 raft_consensus.cc:697] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:43.667390 23557 consensus_queue.cc:237] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:43.676973 23593 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:43.681645 12572 leader_election.cc:336] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:43.686846 23593 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:43.689764 12572 consensus_peers.cc:597] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:43.690671 23593 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:43.694756 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:43.673987 11318 tablet_replica.cc:333] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
W20260709 12:14:43.702315 12572 leader_election.cc:336] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:43.702857 12572 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:43.703730 23593 raft_consensus.cc:2749] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:43.715112 11318 raft_consensus.cc:2243] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.715920 11318 raft_consensus.cc:2272] T 952cdab9198b44e4bdf1b472109715c3 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.723218 23612 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:43.724604 12646 consensus_peers.cc:597] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:43.727156 12606 raft_consensus.cc:1275] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
I20260709 12:14:43.728744 23664 consensus_queue.cc:1048] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 17, Time since last communication: 0.000s
I20260709 12:14:43.735492 11318 tablet_replica.cc:333] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:43.737838 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "33b7d46307bc4c33acd6c9a58055c7af" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:43.746089 11318 raft_consensus.cc:2243] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.746559 12674 raft_consensus.cc:3060] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:43.752605 11318 pending_rounds.cc:70] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Trying to abort 1 pending ops.
I20260709 12:14:43.764854 11318 pending_rounds.cc:77] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting op as it isn't in flight: id { term: 29 index: 18 } timestamp: 7305622573150720000 op_type: NO_OP noop_request { }
I20260709 12:14:43.758715 23641 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:14:43.762451 12646 consensus_peers.cc:597] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:14:43.744005 23612 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:43.744874 12572 leader_election.cc:336] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:43.776526 11318 raft_consensus.cc:2889] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:43.783244 23641 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:43.785772 23641 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:43.789454 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:43.786834 12572 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 170 similar messages]
I20260709 12:14:43.786926 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cdb0bb14b2b745b398e14d8c1a7139a2" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 33 candidate_status { last_received { term: 30 index: 21 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:43.793171 12494 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:43.797339 11318 raft_consensus.cc:2272] T 317be0a176ee473b9b3631bdc1030bcc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.801386 12674 raft_consensus.cc:2468] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
I20260709 12:14:43.797041 23637 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:43.822185 12571 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:43.823956 23595 raft_consensus.cc:2804] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:43.824523 23595 raft_consensus.cc:697] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:43.825449 23595 consensus_queue.cc:237] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 27.18, Last appended by leader: 18, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:43.857863 11318 tablet_replica.cc:333] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:43.859262 11318 raft_consensus.cc:2243] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.860251 11318 raft_consensus.cc:2272] T 8c7e0eddcc2b4449b7ed8821854646e2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.870940 11318 tablet_replica.cc:333] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:43.876279 11318 raft_consensus.cc:2243] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.877040 11318 raft_consensus.cc:2272] T 852ac65e35c7462ba906c02b9422dc5b P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.888564 11318 tablet_replica.cc:333] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:43.903488 11318 raft_consensus.cc:2243] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:14:43.904757 11318 raft_consensus.cc:2272] T 502d264440884ecda3d4641918b20d0c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:43.916050 12572 consensus_peers.cc:597] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:43.922194 23428 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.923040 23428 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:43.927114 23428 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:43.928838 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:43.929171 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:43.929590 12674 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 31.
I20260709 12:14:43.929749 12605 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 30.
I20260709 12:14:43.923880 11318 tablet_replica.cc:333] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:43.931229 12496 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:43.934617 23641 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:43.935088 23641 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:43.935441 23641 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:43.939874 11318 raft_consensus.cc:2243] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:43.942464 23428 raft_consensus.cc:493] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 36 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:43.958210 23641 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:43.959100 23428 raft_consensus.cc:515] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:43.961439 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:43.962108 12605 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 32
I20260709 12:14:43.962237 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 37 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:43.962782 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:43.963390 12674 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:43.970305 12605 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 32.
I20260709 12:14:43.971658 12494 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:43.972570 23637 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:14:43.973101 23637 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:43.974002 23637 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 30.15, Last appended by leader: 15, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:43.978868 11318 pending_rounds.cc:70] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Trying to abort 1 pending ops.
I20260709 12:14:43.978768 12674 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 32.
I20260709 12:14:43.979391 11318 pending_rounds.cc:77] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting op as it isn't in flight: id { term: 33 index: 23 } timestamp: 7305622559660163072 op_type: NO_OP noop_request { }
I20260709 12:14:43.980007 11318 raft_consensus.cc:2889] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 36 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:43.981312 11318 raft_consensus.cc:2272] T ee47f7c4414b4ae98233a31696e99cc0 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:43.985441 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a92d1d91cce04cec98366aa6021356df" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 37 candidate_status { last_received { term: 35 index: 24 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:43.980829 23641 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:43.990072 12494 leader_election.cc:304] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:43.991060 23355 raft_consensus.cc:2749] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
I20260709 12:14:43.962216 23428 leader_election.cc:290] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:44.043699 11318 tablet_replica.cc:333] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.044763 11318 raft_consensus.cc:2243] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:44.045579 11318 raft_consensus.cc:2272] T 43d4539f52e24ea8adc368a4c9529e66 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.049150 11318 tablet_replica.cc:333] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.059592 11318 raft_consensus.cc:2243] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:44.067523 11318 raft_consensus.cc:2272] T 70b8c9b89e3d43fbb70422110d76702c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.075838 23228 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.087555 11318 tablet_replica.cc:333] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
W20260709 12:14:44.088666 12413 catalog_manager.cc:5227] ChangeConfig:REMOVE_PEER RPC for tablet 70b8c9b89e3d43fbb70422110d76702c with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER failed with leader e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica e56d487365394ac5ac5f1002926a5645 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 26 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:44.090723 23641 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.091014 11318 raft_consensus.cc:2243] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:44.091498 23641 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.092103 11318 raft_consensus.cc:2272] T b65110412db1492aa47f4a845ec89242 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.077657 23228 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.096669 23228 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:44.097438 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:44.098094 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "47b417a13f2d44ef8922eb717c8ab20d" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 28 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:44.098634 12646 consensus_peers.cc:597] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:14:44.100114 12496 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:44.103843 23428 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:14:44.104773 12572 consensus_peers.cc:597] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:44.105085 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:44.105742 12572 consensus_peers.cc:597] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:44.107263 11318 tablet_replica.cc:333] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.108016 23641 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:44.114784 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3af899b7fab042d0ac433430f162d106" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 26 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:44.118827 11318 raft_consensus.cc:2243] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 LEADER]: Raft consensus shutting down.
W20260709 12:14:44.120795 13305 consensus_peers.cc:597] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:14:44.127074 12496 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:44.128917 23428 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:14:44.130630 12572 consensus_peers.cc:597] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:44.138410 11318 raft_consensus.cc:2272] T 53977095735a4e34bc297c91e8c6f17f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.147047 23593 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:44.147827 23593 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.152491 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:44.153805 12677 raft_consensus.cc:2393] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 31: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:44.156384 23593 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:44.162168 12572 leader_election.cc:336] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:44.162696 12572 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:44.163506 23593 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:44.172395 23593 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:44.142030 11318 tablet_replica.cc:333] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.182754 11318 raft_consensus.cc:2243] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 LEADER]: Raft consensus shutting down.
I20260709 12:14:44.183943 11318 raft_consensus.cc:2272] T 7b825dff40dc4290a25d71534b598330 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 24 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.207286 11318 tablet_replica.cc:333] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.208206 11318 raft_consensus.cc:2243] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 LEADER]: Raft consensus shutting down.
W20260709 12:14:44.208559 12646 consensus_peers.cc:597] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:44.209437 11318 raft_consensus.cc:2272] T 15456f360e78465994eb4d0d39e90b87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.211306 12606 raft_consensus.cc:1275] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:14:44.212922 23664 consensus_queue.cc:1048] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:14:44.255342 11318 tablet_replica.cc:333] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.259732 11318 raft_consensus.cc:2243] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:44.273551 11318 pending_rounds.cc:70] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Trying to abort 2 pending ops.
I20260709 12:14:44.287770 11318 pending_rounds.cc:77] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting op as it isn't in flight: id { term: 29 index: 22 } timestamp: 7305622548000071680 op_type: NO_OP noop_request { }
I20260709 12:14:44.288450 11318 raft_consensus.cc:2889] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:44.290551 11318 pending_rounds.cc:77] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting op as it isn't in flight: id { term: 28 index: 21 } timestamp: 7305622520759541760 op_type: NO_OP noop_request { }
I20260709 12:14:44.299855 23664 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:44.300577 23664 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.303241 23664 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:44.308033 12646 leader_election.cc:336] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:44.311558 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:44.312542 12605 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 33.
I20260709 12:14:44.314038 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:44.317056 23664 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:14:44.316955 23641 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.317760 23664 raft_consensus.cc:493] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:44.318348 23664 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:44.317940 23641 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.323745 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:44.325888 23641 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:44.325714 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d29f84bdbc4c49278fb44e72ddb1a33c" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:44.329767 23593 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:44.330752 12494 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:44.330504 23593 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.332332 23641 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:44.333595 23593 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:44.335340 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:44.324506 12423 catalog_manager.cc:5697] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 10 to 33. New cstate: current_term: 33 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:44.336241 12677 raft_consensus.cc:2393] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 28: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:44.336830 12423 catalog_manager.cc:5697] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 16 to 28. New cstate: current_term: 28 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:44.337757 23593 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:44.291095 11318 raft_consensus.cc:2889] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
W20260709 12:14:44.329912 12572 consensus_peers.cc:597] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:44.338523 11318 raft_consensus.cc:2272] T d7bd864b75c34027b05582a55b152877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:44.339190 12572 consensus_peers.cc:597] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:44.338409 12423 catalog_manager.cc:5697] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 27 to 29. New cstate: current_term: 29 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:44.338340 23593 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.340651 12423 catalog_manager.cc:5697] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 27 to 28, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 28 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:44.342289 12423 catalog_manager.cc:5697] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 20 to 25, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 25 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:44.343739 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:44.344530 12677 raft_consensus.cc:2393] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 33: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
I20260709 12:14:44.343978 12423 catalog_manager.cc:5697] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 16 to 29, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 29 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:14:44.344717 12572 leader_election.cc:336] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:44.345275 12572 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:44.346201 23647 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:14:44.348912 12572 leader_election.cc:336] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:44.349398 12572 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:44.345827 23593 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:44.355676 23647 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:14:44.323879 12420 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59006 (request call id 182) took 25440 ms (client timeout 15000 ms). Trace:
W20260709 12:14:44.356616 12420 rpcz_store.cc:269] 0709 12:14:18.883699 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:20.225169 (+1341470us) service_pool.cc:224] Handling call
0709 12:14:44.323751 (+24098582us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:43.843024 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:43.843688 (+   664us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:43.843734 (+    46us) write_op.cc:435] Acquired schema lock
0709 12:14:43.843753 (+    19us) tablet.cc:662] Decoding operations
0709 12:14:43.845579 (+  1826us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:43.845678 (+    99us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:43.845709 (+    31us) tablet.cc:685] Acquiring locks for 20 operations
0709 12:14:43.867471 (+ 21762us) tablet.cc:701] Row locks acquired
0709 12:14:43.880615 (+ 13144us) write_op.cc:260] PREPARE: finished
0709 12:14:43.882635 (+  2020us) write_op.cc:270] Start()
0709 12:14:43.882778 (+   143us) write_op.cc:276] Timestamp: P: 1783599283882573 usec, L: 0
0709 12:14:43.882801 (+    23us) op_driver.cc:348] REPLICATION: starting
0709 12:14:43.892799 (+  9998us) log.cc:844] Serialized 7890 byte log entry
0709 12:14:43.948212 (+ 55413us) op_driver.cc:464] REPLICATION: finished
0709 12:14:44.100411 (+152199us) write_op.cc:301] APPLY: starting
0709 12:14:44.100528 (+   117us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:44.153291 (+ 52763us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:44.153554 (+   263us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:44.222303 (+ 68749us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:44.222638 (+   335us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=40,key_file_lookups=40,delta_file_lookups=0,mrs_lookups=0
0709 12:14:44.222685 (+    47us) write_op.cc:312] APPLY: finished
0709 12:14:44.223136 (+   451us) log.cc:844] Serialized 180 byte log entry
0709 12:14:44.238332 (+ 15196us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:44.238615 (+   283us) write_op.cc:454] Released schema lock
0709 12:14:44.238684 (+    69us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:44.239209 (+   525us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:44.239242 (+    33us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":53,"spinlock_wait_cycles":14976,"child_traces":[["op",{"apply.queue_time_us":151926,"apply.run_cpu_time_us":29127,"apply.run_wall_time_us":138736,"cfile_cache_hit":82,"cfile_cache_hit_bytes":87607,"num_ops":20,"prepare.queue_time_us":131951,"prepare.run_cpu_time_us":6525,"prepare.run_wall_time_us":52204,"replication_time_us":65249,"thread_start_us":197465,"threads_started":2}]]}
I20260709 12:14:44.364547 11318 tablet_replica.cc:333] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.365828 11318 raft_consensus.cc:2243] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:44.370370 23612 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:44.373780 12677 raft_consensus.cc:1275] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 27 index: 18. Preceding OpId from leader: term: 30 index: 19. (index mismatch)
I20260709 12:14:44.345990 12423 catalog_manager.cc:5697] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 26 to 28. New cstate: current_term: 28 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:44.395243 12423 catalog_manager.cc:5697] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 26 to 28, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 28 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:44.396880 12423 catalog_manager.cc:5697] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 34 to 35, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 35 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:44.399266 12423 catalog_manager.cc:5697] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 26 to 27, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 27 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:44.400914 12423 catalog_manager.cc:5697] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 31 to 32, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 32 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:44.402947 12423 catalog_manager.cc:5697] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 25 to 28, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 28 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:44.408915 12423 catalog_manager.cc:5697] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 18 to 27. New cstate: current_term: 27 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:44.405966 12606 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 30 index: 15. Preceding OpId from leader: term: 32 index: 16. (index mismatch)
I20260709 12:14:44.399310 23647 consensus_queue.cc:1048] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.001s
I20260709 12:14:44.381072 23664 raft_consensus.cc:515] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.413622 23612 raft_consensus.cc:2749] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:44.414541 11318 raft_consensus.cc:2272] T 3af899b7fab042d0ac433430f162d106 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.418613 12423 catalog_manager.cc:5697] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 17 to 27, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 27 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:44.422420 23637 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
I20260709 12:14:44.424386 12423 catalog_manager.cc:5697] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 20 to 32, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 32 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:44.427613 12423 catalog_manager.cc:5697] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 16 to 29, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 29 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:44.438938 12423 catalog_manager.cc:5697] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 22 to 26. New cstate: current_term: 26 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:14:44.432086 12572 consensus_peers.cc:597] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:44.446576 23664 leader_election.cc:290] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:44.444512 13305 consensus_peers.cc:597] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:44.446915 12423 catalog_manager.cc:5697] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 21 to 31. New cstate: current_term: 31 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:14:44.448216 12646 leader_election.cc:336] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:44.448608 12423 catalog_manager.cc:5697] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 11 to 24. New cstate: current_term: 24 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:44.449844 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "19eaf42a0f664c919842f3c2f5cb868f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 33 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:44.450577 12605 raft_consensus.cc:3060] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:44.449923 12423 catalog_manager.cc:5697] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 29 to 31. New cstate: current_term: 31 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:44.451546 12423 catalog_manager.cc:5697] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 16 to 29, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 29 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:44.460214 23428 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.461097 23428 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.463148 11318 tablet_replica.cc:333] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.464464 23428 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:44.463654 12423 catalog_manager.cc:5697] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 17 to 28. New cstate: current_term: 28 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:44.465487 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:44.467088 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f0d97a077cac41709974a6cde7865fcb" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 32 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:44.472963 12496 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:44.474009 23428 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:44.473238 12423 catalog_manager.cc:5697] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 24 to 26, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 26 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:44.475627 12423 catalog_manager.cc:5697] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 30 to 32. New cstate: current_term: 32 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } }
I20260709 12:14:44.468854 12605 raft_consensus.cc:2468] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 34.
I20260709 12:14:44.480516 12423 catalog_manager.cc:5697] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 19 to 33, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 33 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:44.489745 12675 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 30 index: 15. Preceding OpId from leader: term: 32 index: 16. (index mismatch)
I20260709 12:14:44.491045 12423 catalog_manager.cc:5697] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 17 to 33. New cstate: current_term: 33 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:44.492754 23637 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:14:44.493495 12423 catalog_manager.cc:5697] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 17 to 29. New cstate: current_term: 29 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:44.497223 12423 catalog_manager.cc:5697] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 15 to 30. New cstate: current_term: 30 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:44.498837 12423 catalog_manager.cc:5697] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 23 to 25, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 25 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:44.483492 12643 leader_election.cc:304] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:44.510126 23664 raft_consensus.cc:2804] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:14:44.511139 23664 raft_consensus.cc:697] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:44.512495 23664 consensus_queue.cc:237] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 33.19, Last appended by leader: 19, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.517154 12423 catalog_manager.cc:5697] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 24 to 27, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 27 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
W20260709 12:14:44.520228 12572 consensus_peers.cc:597] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:44.484730 11318 raft_consensus.cc:2243] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:14:44.521688 11318 raft_consensus.cc:2272] T 52c63945e7484332bf4c454ec7423a21 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.529852 11318 tablet_replica.cc:333] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
W20260709 12:14:44.532153 12572 consensus_peers.cc:597] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:44.537881 11318 raft_consensus.cc:2243] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: Raft consensus shutting down.
W20260709 12:14:44.541067 12572 consensus_peers.cc:597] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:44.540256 11318 raft_consensus.cc:2272] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:44.555738 12572 consensus_peers.cc:597] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:44.557097 11318 tablet_replica.cc:333] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.558339 11318 raft_consensus.cc:2243] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:44.568146 11318 pending_rounds.cc:70] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Trying to abort 1 pending ops.
I20260709 12:14:44.568531 11318 pending_rounds.cc:77] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting op as it isn't in flight: id { term: 34 index: 22 } timestamp: 7305622607125524480 op_type: NO_OP noop_request { }
I20260709 12:14:44.572957 11318 raft_consensus.cc:2889] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:44.573463 11318 raft_consensus.cc:2272] T e40144c1c60847ee81b8252b107df67d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.577718 23664 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:44.578776 23664 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.581364 23664 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:44.585024 12646 leader_election.cc:336] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:44.587206 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:44.587458 12572 consensus_peers.cc:597] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:44.588110 23428 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.588636 23428 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.590195 11318 tablet_replica.cc:333] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.591346 11318 raft_consensus.cc:2243] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:14:44.592221 12643 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:44.592536 11318 raft_consensus.cc:2272] T ade78efb80994e1b835a9aeec05fd8cd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.593402 23664 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:44.594672 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:44.595372 12675 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:44.597595 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:44.598345 12605 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 28.
I20260709 12:14:44.599902 12494 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:44.600533 12572 consensus_peers.cc:597] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:44.600739 23641 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:44.601217 23641 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:44.601605 23641 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:44.602828 23428 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:44.617830 23428 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.618445 23428 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.619699 11318 tablet_replica.cc:333] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.620672 11318 raft_consensus.cc:2243] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:44.627642 11318 raft_consensus.cc:2272] T 6fa9b4ceed3443f2a6992343ab6ad877 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.632437 23664 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:44.633322 23664 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.635967 23664 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:44.639528 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 35 candidate_status { last_received { term: 34 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:44.641433 23428 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:44.641594 12646 leader_election.cc:336] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:44.642954 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:44.651360 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:44.653241 23664 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:14:44.644405 23641 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.644848 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf6806e0707d42a0b80b9b9a65444604" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 31 candidate_status { last_received { term: 28 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:44.646394 11318 tablet_replica.cc:333] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.655990 11318 raft_consensus.cc:2243] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:44.656774 11318 raft_consensus.cc:2272] T ee30d7cd60b04485b7e19c8d119d67bd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.660339 23664 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.660717 12496 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:44.660975 23664 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.661832 23355 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:44.660375 11318 tablet_replica.cc:333] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.664214 23664 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:44.667129 23641 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:44.667871 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:44.668547 12646 leader_election.cc:336] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:44.670863 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:44.672314 12605 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 33.
I20260709 12:14:44.672850 12675 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:44.673864 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" candidate_term: 29 candidate_status { last_received { term: 27 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:44.674094 12643 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:44.672302 11318 raft_consensus.cc:2243] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:14:44.675587 23664 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election won for term 34
I20260709 12:14:44.675800 11318 raft_consensus.cc:2272] T 6530c20e384d4ff0915a09d0641c82f8 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.676071 23664 raft_consensus.cc:493] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:44.676546 23664 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:44.677343 12606 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:44.695396 11318 tablet_replica.cc:333] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.696427 11318 raft_consensus.cc:2243] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:44.697237 11318 raft_consensus.cc:2272] T cdb0bb14b2b745b398e14d8c1a7139a2 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.700589 11318 tablet_replica.cc:333] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.703720 12675 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:44.719344 12496 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201; no voters: 
I20260709 12:14:44.720459 23641 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:44.721012 23641 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Becoming Leader. State: Replica: a2fdbe7eef6748a9b4611f6c9bc9b201, State: Running, Role: LEADER
I20260709 12:14:44.722012 23641 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 27.16, Last appended by leader: 16, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.703114 12606 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a2fdbe7eef6748a9b4611f6c9bc9b201 in term 29.
I20260709 12:14:44.704305 11318 raft_consensus.cc:2243] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:14:44.733414 11318 raft_consensus.cc:2272] T 9b55aace8c0343b9808746fb1275f5cc P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.706804 23664 raft_consensus.cc:515] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:44.740474 12646 leader_election.cc:336] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:44.742882 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "852ac65e35c7462ba906c02b9422dc5b" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 34 candidate_status { last_received { term: 32 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:44.743705 12606 raft_consensus.cc:3060] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Advancing to term 34
I20260709 12:14:44.749151 23664 leader_election.cc:290] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:44.768203 11318 tablet_replica.cc:333] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.769306 11318 raft_consensus.cc:2243] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:44.795609 12606 raft_consensus.cc:2468] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 34.
I20260709 12:14:44.797379 12643 leader_election.cc:304] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:44.802944 23664 raft_consensus.cc:2804] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader election won for term 34
I20260709 12:14:44.803566 23664 raft_consensus.cc:697] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:44.804490 23664 consensus_queue.cc:237] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 32.19, Last appended by leader: 19, Current term: 34, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:44.813035 12572 consensus_peers.cc:597] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:44.785329 11318 raft_consensus.cc:2272] T bdfb480850c24efa8a38a916d53bdb1f P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.824364 11318 tablet_replica.cc:333] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.825361 11318 raft_consensus.cc:2243] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:14:44.826610 11318 raft_consensus.cc:2272] T 57b456cf0a3b4f0f9f6cae5a1862caca P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.858795 23664 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:44.859396 23664 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.862192 11318 tablet_replica.cc:333] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.863303 11318 raft_consensus.cc:2243] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 LEADER]: Raft consensus shutting down.
W20260709 12:14:44.865162 12646 leader_election.cc:336] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:44.866837 11318 raft_consensus.cc:2272] T 8f2a3bfdbbd6440ba9cf687448fb678c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.867585 23664 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:44.869302 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 35 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:44.870167 12606 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 34.
I20260709 12:14:44.870321 11318 tablet_replica.cc:333] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.871430 11318 raft_consensus.cc:2243] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:44.871734 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:44.872123 11318 pending_rounds.cc:70] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Trying to abort 1 pending ops.
I20260709 12:14:44.872494 11318 pending_rounds.cc:77] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting op as it isn't in flight: id { term: 32 index: 18 } timestamp: 7305622642074140672 op_type: NO_OP noop_request { }
I20260709 12:14:44.872912 11318 raft_consensus.cc:2889] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:44.873061 23587 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:14:44.873346 11318 raft_consensus.cc:2272] T 05ff02a879ef4c94966348127fec27f7 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.873728 23587 raft_consensus.cc:493] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:44.874248 23587 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:14:44.878633 12420 catalog_manager.cc:5697] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 31 to 32, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 32 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:44.880425 12420 catalog_manager.cc:5697] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 27 to 29, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 29 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:44.881582 11318 tablet_replica.cc:333] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.882562 11318 raft_consensus.cc:2243] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:44.883153 11318 pending_rounds.cc:70] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Trying to abort 1 pending ops.
I20260709 12:14:44.883378 11318 pending_rounds.cc:77] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting op as it isn't in flight: id { term: 28 index: 16 } timestamp: 7305622591930392576 op_type: NO_OP noop_request { }
I20260709 12:14:44.883699 11318 raft_consensus.cc:2889] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:44.884035 11318 raft_consensus.cc:2272] T d29f84bdbc4c49278fb44e72ddb1a33c P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.884610 12420 catalog_manager.cc:5697] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 35 to 36, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 36 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:44.887636 11318 tablet_replica.cc:333] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.887246 23587 raft_consensus.cc:515] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.901532 11318 raft_consensus.cc:2243] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:44.902186 11318 pending_rounds.cc:70] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Trying to abort 1 pending ops.
I20260709 12:14:44.902510 11318 pending_rounds.cc:77] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting op as it isn't in flight: id { term: 30 index: 18 } timestamp: 7305622654203133952 op_type: NO_OP noop_request { }
I20260709 12:14:44.901985 12420 catalog_manager.cc:5697] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 28 to 33, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 33 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:44.902877 11318 raft_consensus.cc:2889] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:44.902946 23587 leader_election.cc:290] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:44.903407 11318 raft_consensus.cc:2272] T f0d97a077cac41709974a6cde7865fcb P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.903900 12420 catalog_manager.cc:5697] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 31 to 35. New cstate: current_term: 35 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:44.906467 11318 tablet_replica.cc:333] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.907421 11318 raft_consensus.cc:2243] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:14:44.908527 11318 raft_consensus.cc:2272] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.908864 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "111842005e864eab9966b3a00ab9ae8a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 35 candidate_status { last_received { term: 33 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:44.909592 12606 raft_consensus.cc:3060] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Advancing to term 35
W20260709 12:14:44.912509 12646 leader_election.cc:336] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:44.905619 12420 catalog_manager.cc:5697] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 33 to 36. New cstate: current_term: 36 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:44.921655 12606 raft_consensus.cc:2468] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 35.
I20260709 12:14:44.924280 12643 leader_election.cc:304] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:44.924716 11318 tablet_replica.cc:333] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.925674 11318 raft_consensus.cc:2243] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:44.925896 23593 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:44.926347 11318 pending_rounds.cc:70] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Trying to abort 2 pending ops.
I20260709 12:14:44.926638 11318 pending_rounds.cc:77] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting op as it isn't in flight: id { term: 34 index: 22 } timestamp: 7305622617104384000 op_type: NO_OP noop_request { }
I20260709 12:14:44.926604 23593 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.927227 11318 raft_consensus.cc:2889] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:44.927608 11318 pending_rounds.cc:77] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting op as it isn't in flight: id { term: 32 index: 21 } timestamp: 7305622575805976576 op_type: NO_OP noop_request { }
I20260709 12:14:44.928038 11318 raft_consensus.cc:2889] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:44.927524 12420 catalog_manager.cc:5697] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 27 to 31, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 31 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:44.930428 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 35 candidate_status { last_received { term: 34 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:44.931099 12675 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 34.
I20260709 12:14:44.928360 11318 raft_consensus.cc:2272] T b7cccd56fa854cbc84a922238ad4add9 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.932866 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:44.933794 23612 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:14:44.934293 23612 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:44.934754 23612 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:14:44.936725 11318 tablet_replica.cc:333] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.937678 11318 raft_consensus.cc:2243] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:44.938303 11318 pending_rounds.cc:70] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Trying to abort 1 pending ops.
I20260709 12:14:44.938524 11318 pending_rounds.cc:77] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting op as it isn't in flight: id { term: 26 index: 15 } timestamp: 7305622625915240448 op_type: NO_OP noop_request { }
I20260709 12:14:44.938964 11318 raft_consensus.cc:2889] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:44.939307 11318 raft_consensus.cc:2272] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.944291 23587 raft_consensus.cc:2804] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:14:44.944718 12423 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59006 (request call id 183) took 7604 ms. Trace:
I20260709 12:14:44.945353 23587 raft_consensus.cc:697] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:44.946503 23587 consensus_queue.cc:237] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 33.17, Last appended by leader: 17, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.945076 12423 rpcz_store.cc:276] 0709 12:14:37.340134 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:43.425760 (+6085626us) service_pool.cc:224] Handling call
0709 12:14:43.510364 (+ 84604us) spinlock_profiling.cc:243] Waited 1.7 ms on lock 0x7b5400280190. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fc0b2 00007f148e875759 00007f148e80da76 00007f148e7ded58 00007f148e8dd04e 00007f148cd9e2a4 00007f148cd9e232 00007f148cd9e161 00007f148cd9e0dc
0709 12:14:44.944624 (+1434260us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:44.559588 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:44.569312 (+  9724us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:44.569350 (+    38us) write_op.cc:435] Acquired schema lock
0709 12:14:44.569368 (+    18us) tablet.cc:662] Decoding operations
0709 12:14:44.571720 (+  2352us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:44.571803 (+    83us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:44.571838 (+    35us) tablet.cc:685] Acquiring locks for 30 operations
0709 12:14:44.589799 (+ 17961us) tablet.cc:701] Row locks acquired
0709 12:14:44.589821 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:14:44.589990 (+   169us) write_op.cc:270] Start()
0709 12:14:44.602165 (+ 12175us) write_op.cc:276] Timestamp: P: 1783599284589926 usec, L: 0
0709 12:14:44.602218 (+    53us) op_driver.cc:348] REPLICATION: starting
0709 12:14:44.603830 (+  1612us) log.cc:844] Serialized 11814 byte log entry
0709 12:14:44.616609 (+ 12779us) op_driver.cc:464] REPLICATION: finished
0709 12:14:44.616981 (+   372us) write_op.cc:301] APPLY: starting
0709 12:14:44.617081 (+   100us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:44.654820 (+ 37739us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:44.654937 (+   117us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:44.841794 (+186857us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:44.842197 (+   403us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=60,key_file_lookups=60,delta_file_lookups=0,mrs_lookups=0
0709 12:14:44.842249 (+    52us) write_op.cc:312] APPLY: finished
0709 12:14:44.842778 (+   529us) log.cc:844] Serialized 260 byte log entry
0709 12:14:44.850936 (+  8158us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:44.851268 (+   332us) write_op.cc:454] Released schema lock
0709 12:14:44.851351 (+    83us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:44.851741 (+   390us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:44.851771 (+    30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":20228,"reported_tablets":53,"spinlock_wait_cycles":3916672,"child_traces":[["op",{"apply.queue_time_us":107,"apply.run_cpu_time_us":37844,"apply.run_wall_time_us":234728,"cfile_cache_hit":120,"cfile_cache_hit_bytes":125220,"num_ops":30,"prepare.queue_time_us":13278,"prepare.run_cpu_time_us":7303,"prepare.run_wall_time_us":51663,"replication_time_us":14226,"thread_start_us":30497,"threads_started":1}]]}
W20260709 12:14:44.951846 12572 leader_election.cc:336] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:44.960034 11318 tablet_replica.cc:333] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:44.973489 11318 raft_consensus.cc:2243] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:44.974248 11318 pending_rounds.cc:70] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Trying to abort 1 pending ops.
I20260709 12:14:44.974567 11318 pending_rounds.cc:77] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting op as it isn't in flight: id { term: 32 index: 21 } timestamp: 7305622574352089088 op_type: NO_OP noop_request { }
I20260709 12:14:44.974977 11318 raft_consensus.cc:2889] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:44.975350 11318 raft_consensus.cc:2272] T 33a0b4d6cfba4ab2b3d54f591439b179 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:44.979732 23595 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:44.980410 23595 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:44.948024 23593 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:44.960110 23612 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:44.984910 12646 consensus_peers.cc:597] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:44.992600 23595 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:45.061800 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:45.062865 12672 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
W20260709 12:14:45.119747 12569 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 129866us
I20260709 12:14:45.114711 23612 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 35 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:45.140174 11318 tablet_replica.cc:333] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:45.141016 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6cb14d97e16d424a82a91a8ce1b6cc8d" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 35 candidate_status { last_received { term: 34 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:45.141685 11318 raft_consensus.cc:2243] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:45.141708 12677 raft_consensus.cc:3060] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:14:45.144184 23647 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.144745 23647 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.151559 23647 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:45.155077 12677 raft_consensus.cc:2468] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 35.
I20260709 12:14:45.156486 12571 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:45.158984 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 164371us
I20260709 12:14:45.167392 23557 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:45.171080 23557 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.173583 23557 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:45.173592 23595 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:45.186340 23595 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.186956 23595 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:45.152144 23647 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.191097 23647 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.193634 23647 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:45.194706 11318 raft_consensus.cc:2272] T cf6806e0707d42a0b80b9b9a65444604 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 30 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:45.198525 12646 leader_election.cc:336] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:45.176376 12571 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:45.214298 12420 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:47524 (request call id 107) took 5700 ms. Trace:
I20260709 12:14:45.225808 12420 rpcz_store.cc:276] 0709 12:14:39.508708 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:44.361054 (+4852346us) service_pool.cc:224] Handling call
0709 12:14:45.209134 (+848080us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:44.948684 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:44.949240 (+   556us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:44.949283 (+    43us) write_op.cc:435] Acquired schema lock
0709 12:14:44.949296 (+    13us) tablet.cc:662] Decoding operations
0709 12:14:44.950081 (+   785us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:44.950165 (+    84us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:44.950190 (+    25us) tablet.cc:685] Acquiring locks for 7 operations
0709 12:14:44.950950 (+   760us) tablet.cc:701] Row locks acquired
0709 12:14:44.950967 (+    17us) write_op.cc:260] PREPARE: finished
0709 12:14:44.959506 (+  8539us) write_op.cc:270] Start()
0709 12:14:44.959664 (+   158us) write_op.cc:276] Timestamp: P: 1783599284959451 usec, L: 0
0709 12:14:44.959684 (+    20us) op_driver.cc:348] REPLICATION: starting
0709 12:14:44.984656 (+ 24972us) log.cc:844] Serialized 2835 byte log entry
0709 12:14:44.987093 (+  2437us) op_driver.cc:464] REPLICATION: finished
0709 12:14:44.988093 (+  1000us) spinlock_profiling.cc:243] Waited 21.5 ms on lock 0x7b54000104d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f1488989571 00007f1488a1be28 00007f148c456581 00007f148c455d58 00007f148c45a9f1 00007f148c45a9a9 00007f148c45a939 00007f148c45a901 00007f148c459b2d 0000000000388634 0000000000388479
0709 12:14:45.135067 (+146974us) write_op.cc:301] APPLY: starting
0709 12:14:45.135190 (+   123us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:45.139673 (+  4483us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:45.139702 (+    29us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:45.144178 (+  4476us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:45.144378 (+   200us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=14,key_file_lookups=14,delta_file_lookups=0,mrs_lookups=0
0709 12:14:45.144423 (+    45us) write_op.cc:312] APPLY: finished
0709 12:14:45.144705 (+   282us) log.cc:844] Serialized 73 byte log entry
0709 12:14:45.146411 (+  1706us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:45.146867 (+   456us) write_op.cc:454] Released schema lock
0709 12:14:45.146916 (+    49us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:45.173850 (+ 26934us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:45.173887 (+    37us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":16465,"reported_tablets":21,"child_traces":[["op",{"apply.queue_time_us":120,"apply.run_cpu_time_us":10112,"apply.run_wall_time_us":38739,"cfile_cache_hit":28,"cfile_cache_hit_bytes":29218,"num_ops":7,"prepare.queue_time_us":15412,"prepare.run_cpu_time_us":14947,"prepare.run_wall_time_us":39595,"replication_time_us":27235,"spinlock_wait_cycles":49381760}]]}
I20260709 12:14:45.236673 11318 tablet_replica.cc:333] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:45.237762 11318 raft_consensus.cc:2243] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Raft consensus shutting down.
W20260709 12:14:45.238341 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 62062.8us
I20260709 12:14:45.238824 23612 raft_consensus.cc:2804] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:14:45.239409 23612 raft_consensus.cc:697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:45.240273 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:45.243232 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:45.244954 23595 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.245317 23612 consensus_queue.cc:237] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 34.16, Last appended by leader: 16, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.246800 11318 raft_consensus.cc:2272] T 569d6ab756014cd88db2198d5d25dccd P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:45.259678 23587 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.260295 23587 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.264139 12427 catalog_manager.cc:5697] T 618bbf28983a42beaaf337df12784ee1 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 22 to 24. New cstate: current_term: 24 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:45.277374 11318 tablet_replica.cc:333] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:45.279579 11318 raft_consensus.cc:2243] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:45.280334 11318 pending_rounds.cc:70] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Trying to abort 2 pending ops.
I20260709 12:14:45.280588 11318 pending_rounds.cc:77] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting op as it isn't in flight: id { term: 35 index: 24 } timestamp: 7305622653518606336 op_type: NO_OP noop_request { }
W20260709 12:14:45.280671 12494 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 84955.3us
I20260709 12:14:45.279371 12427 catalog_manager.cc:5697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 16 to 27, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 27 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:45.323912 11318 raft_consensus.cc:2889] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 36 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:45.324440 11318 pending_rounds.cc:77] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting op as it isn't in flight: id { term: 33 index: 23 } timestamp: 7305622616753078272 op_type: NO_OP noop_request { }
I20260709 12:14:45.324855 11318 raft_consensus.cc:2889] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 36 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:45.325286 11318 raft_consensus.cc:2272] T a92d1d91cce04cec98366aa6021356df P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:45.277339 23664 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.310020 23587 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:45.326812 23587 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.327338 23587 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.329653 23587 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:45.326040 23664 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.332649 23664 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:45.314438 23595 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:45.341908 23595 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.343375 23595 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:45.347483 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 201019us
I20260709 12:14:45.343803 11318 tablet_replica.cc:333] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:45.356855 11318 raft_consensus.cc:2243] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:14:45.383273 12427 catalog_manager.cc:5697] T aeb1937b72c34b78909bb9d4840f1f06 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 11 to 26, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 26 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:45.392230 12427 catalog_manager.cc:5697] T 47b417a13f2d44ef8922eb717c8ab20d P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 16 to 21, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 21 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:45.402760 12672 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
I20260709 12:14:45.403179 23595 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:45.417627 12496 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 49591.6us
I20260709 12:14:45.436100 23697 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
I20260709 12:14:45.436862 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:45.438722 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5771c48bd1d24152ab96cfc24da505ed" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:45.440790 12606 raft_consensus.cc:1275] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 33 index: 19. Preceding OpId from leader: term: 34 index: 20. (index mismatch)
I20260709 12:14:45.439502 12677 raft_consensus.cc:3060] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:45.413105 12427 catalog_manager.cc:5697] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 reported cstate change: term changed from 18 to 21, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 21 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:45.444547 11318 raft_consensus.cc:2272] T 29bbd70aa24b4e81b3d2764c3584ad60 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:45.444576 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 28 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:45.446323 12605 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Refusing update from remote peer a2fdbe7eef6748a9b4611f6c9bc9b201: Log matching property violated. Preceding OpId in replica: term: 27 index: 16. Preceding OpId from leader: term: 29 index: 17. (index mismatch)
W20260709 12:14:45.444041 12646 leader_election.cc:336] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:45.446949 12606 raft_consensus.cc:2410] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 for term 32 because replica has last-logged OpId of term: 30 index: 19, which is greater than that of the candidate, which has last-logged OpId of term: 28 index: 18.
I20260709 12:14:45.448589 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:45.451862 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:45.449869 12605 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:45.456105 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:45.456764 12605 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:14:45.450992 23697 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.011s
I20260709 12:14:45.474462 12677 raft_consensus.cc:2468] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
W20260709 12:14:45.479203 12646 consensus_peers.cc:597] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:45.480262 12646 leader_election.cc:336] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:45.483606 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:45.484951 12643 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:45.486202 23557 consensus_queue.cc:1048] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:14:45.487437 23702 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:45.487703 23557 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:45.488095 23557 raft_consensus.cc:493] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:45.488229 12572 leader_election.cc:336] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:45.488493 23557 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:45.477419 12571 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:45.496160 23595 raft_consensus.cc:2804] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election won for term 29
W20260709 12:14:45.493317 12646 leader_election.cc:336] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:45.496488 11318 tablet_replica.cc:333] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:45.496687 23595 raft_consensus.cc:697] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:45.497356 11318 raft_consensus.cc:2243] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:14:45.497691 23595 consensus_queue.cc:237] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 28.19, Last appended by leader: 19, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.498778 12643 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:45.499737 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:45.500617 23587 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election won for term 27
I20260709 12:14:45.501032 23587 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.501353 23702 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election lost for term 25. Reason: could not achieve majority
I20260709 12:14:45.501443 23587 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Advancing to term 27
W20260709 12:14:45.515866 12646 consensus_peers.cc:597] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:45.524537 11318 raft_consensus.cc:2272] T 3585ecc88ce149c5b13cb3e294485b62 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:45.526881 23699 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.550105 12606 raft_consensus.cc:1275] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 32 index: 19. Preceding OpId from leader: term: 34 index: 20. (index mismatch)
I20260709 12:14:45.560438 23693 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.561257 23693 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.565366 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:45.566063 23693 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:45.566463 13305 consensus_peers.cc:597] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
I20260709 12:14:45.568853 11318 tablet_replica.cc:333] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:45.570791 23664 consensus_queue.cc:1048] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:14:45.572458 23587 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.569944 11318 raft_consensus.cc:2243] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:14:45.574368 11318 pending_rounds.cc:70] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Trying to abort 1 pending ops.
I20260709 12:14:45.574700 11318 pending_rounds.cc:77] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201: Aborting op as it isn't in flight: id { term: 29 index: 17 } timestamp: 7305622670250672128 op_type: NO_OP noop_request { }
I20260709 12:14:45.575057 11318 raft_consensus.cc:2889] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:14:45.575397 11318 raft_consensus.cc:2272] T 3c9b5315396948b2af3fa5480c9bee87 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:45.549587 23699 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.584872 23587 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:45.593715 23699 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:45.595608 23557 raft_consensus.cc:515] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:45.596961 12646 leader_election.cc:336] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:45.598021 23557 leader_election.cc:290] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:45.587309 11318 tablet_replica.cc:333] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
I20260709 12:14:45.600020 11318 raft_consensus.cc:2243] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:14:45.604185 23557 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.604861 23557 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.612216 23693 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.612787 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:45.613191 23693 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.617477 12603 raft_consensus.cc:3060] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Advancing to term 27
I20260709 12:14:45.618500 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:45.615388 23557 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:45.619242 12673 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 24.
I20260709 12:14:45.622859 12571 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:45.623797 23595 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Leader pre-election won for term 25
I20260709 12:14:45.624202 23595 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.624563 23595 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:45.625654 12603 raft_consensus.cc:2468] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
I20260709 12:14:45.629914 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "502d264440884ecda3d4641918b20d0c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:45.630062 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:45.630770 12604 raft_consensus.cc:3060] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:45.631078 12601 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 31.
W20260709 12:14:45.633572 12572 leader_election.cc:336] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 35 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:45.636067 12572 leader_election.cc:336] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:45.634683 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
W20260709 12:14:45.636562 12646 leader_election.cc:336] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:45.637629 12572 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:45.640748 12572 leader_election.cc:336] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:45.645403 12572 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:45.646639 23691 raft_consensus.cc:2749] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:45.642915 11318 raft_consensus.cc:2272] T e479222164574b9ea8944c530fbeb919 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:45.650660 23693 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:45.651468 12572 leader_election.cc:336] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:45.655882 12572 leader_election.cc:336] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:45.656651 12572 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:45.660108 12604 raft_consensus.cc:2468] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29.
I20260709 12:14:45.660979 23700 raft_consensus.cc:2749] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:45.664055 23700 raft_consensus.cc:2749] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:14:45.675463 13305 consensus_peers.cc:597] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:45.675777 12646 leader_election.cc:336] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:45.652544 11318 tablet_replica.cc:333] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201: stopping tablet replica
W20260709 12:14:45.676194 13305 consensus_peers.cc:597] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 12:14:45.677132 11318 raft_consensus.cc:2243] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:14:45.678373 11318 raft_consensus.cc:2272] T 10395d6e75b84c3982521323f92e3309 P a2fdbe7eef6748a9b4611f6c9bc9b201 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:45.679379 23595 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:45.682762 12646 leader_election.cc:336] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:45.690268 23700 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.690850 23700 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.693156 23595 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:45.694612 23700 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:45.697256 12423 catalog_manager.cc:5697] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 24 to 31. New cstate: current_term: 31 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:45.697849 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7b825dff40dc4290a25d71534b598330" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 25 candidate_status { last_received { term: 24 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:45.704797 12673 raft_consensus.cc:3060] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 24 FOLLOWER]: Advancing to term 25
I20260709 12:14:45.706103 12423 catalog_manager.cc:5697] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 29 to 30, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 30 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:45.708820 12423 catalog_manager.cc:5697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 28 to 32. New cstate: current_term: 32 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
W20260709 12:14:45.714051 13305 consensus_peers.cc:597] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 12:14:45.699184 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:45.715451 12423 catalog_manager.cc:5697] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 23 to 27. New cstate: current_term: 27 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:45.717769 12643 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:45.726858 23557 raft_consensus.cc:2804] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader election won for term 27
I20260709 12:14:45.727428 23557 raft_consensus.cc:697] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:45.727492 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:45.728586 23587 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:45.728484 23557 consensus_queue.cc:237] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 26.17, Last appended by leader: 17, Current term: 27, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.729212 23587 raft_consensus.cc:493] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.729766 23587 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:45.730176 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:45.730989 23699 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:45.743693 12643 leader_election.cc:304] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:45.744896 23699 raft_consensus.cc:2804] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:45.745448 23699 raft_consensus.cc:697] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:45.746675 23699 consensus_queue.cc:237] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.747665 23693 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.748319 23693 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.750633 23693 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:45.755846 23587 raft_consensus.cc:515] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.724994 12673 raft_consensus.cc:2468] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 25.
I20260709 12:14:45.759768 12571 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:45.760411 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:45.764158 23595 raft_consensus.cc:2804] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader election won for term 25
W20260709 12:14:45.761409 12646 leader_election.cc:336] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:45.764734 23595 raft_consensus.cc:697] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:45.765655 23595 consensus_queue.cc:237] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 24.14, Last appended by leader: 14, Current term: 25, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.763078 12675 raft_consensus.cc:1275] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 34 index: 16. Preceding OpId from leader: term: 35 index: 17. (index mismatch)
I20260709 12:14:45.772696 23691 consensus_queue.cc:1048] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:14:45.783594 23587 leader_election.cc:290] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:45.795312 12423 catalog_manager.cc:5697] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 33 to 35, leader changed from e56d487365394ac5ac5f1002926a5645 (127.11.13.132) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 35 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:45.797446 12423 catalog_manager.cc:5697] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 32 to 36. New cstate: current_term: 36 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260709 12:14:45.798892 12605 raft_consensus.cc:1275] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 33 index: 17. Preceding OpId from leader: term: 35 index: 18. (index mismatch)
I20260709 12:14:45.834998 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15456f360e78465994eb4d0d39e90b87" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:45.835778 12605 raft_consensus.cc:3060] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:45.850265 12605 raft_consensus.cc:2468] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 32.
I20260709 12:14:45.856844 23664 consensus_queue.cc:1048] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.001s
W20260709 12:14:45.874472 23712 log.cc:927] Time spent T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.089s	user 0.000s	sys 0.005s
W20260709 12:14:45.881119 12572 leader_election.cc:336] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:45.881759 23595 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.891981 23700 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.892522 23595 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.896148 23595 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:45.899289 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:45.910756 12673 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 32.
I20260709 12:14:45.892882 23700 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.912410 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:45.913515 23693 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:14:45.913955 23693 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.914216 23700 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:45.914397 23693 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:45.918931 23624 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.919602 23624 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:45.922014 23624 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:45.928211 13305 leader_election.cc:336] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:45.930028 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:45.937574 12425 catalog_manager.cc:5697] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 19 to 22. New cstate: current_term: 22 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:45.940508 23699 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.941135 23699 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.944126 23699 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:45.948562 12646 leader_election.cc:336] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:45.928509 12427 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58980 (request call id 188) took 55337 ms (client timeout 15000 ms). Trace:
I20260709 12:14:45.950989 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:45.949873 12427 rpcz_store.cc:269] 0709 12:13:50.591471 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:00.208196 (+9616725us) service_pool.cc:224] Handling call
0709 12:14:00.477709 (+269513us) spinlock_profiling.cc:243] Waited 50.1 ms on lock 0x7b54000104d4. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148898a090 00007f1488a28f3a 00007f148e7dee9c 00007f148e8dd04e 00007f148cd9e2a4 00007f148cd9e232 00007f148cd9e161 00007f148cd9e0dc 00007f148cd9d392 00007f1486d5b48c 00007f1486d5a8c6
0709 12:14:45.928385 (+45450676us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:45.531309 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:45.582615 (+ 51306us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:45.582654 (+    39us) write_op.cc:435] Acquired schema lock
0709 12:14:45.582668 (+    14us) tablet.cc:662] Decoding operations
0709 12:14:45.583355 (+   687us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:45.583416 (+    61us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:45.583448 (+    32us) tablet.cc:685] Acquiring locks for 5 operations
0709 12:14:45.584008 (+   560us) tablet.cc:701] Row locks acquired
0709 12:14:45.584033 (+    25us) write_op.cc:260] PREPARE: finished
0709 12:14:45.584185 (+   152us) write_op.cc:270] Start()
0709 12:14:45.584322 (+   137us) write_op.cc:276] Timestamp: P: 1783599285584141 usec, L: 0
0709 12:14:45.584347 (+    25us) op_driver.cc:348] REPLICATION: starting
0709 12:14:45.585030 (+   683us) log.cc:844] Serialized 2089 byte log entry
0709 12:14:45.587340 (+  2310us) op_driver.cc:464] REPLICATION: finished
0709 12:14:45.587725 (+   385us) write_op.cc:301] APPLY: starting
0709 12:14:45.587839 (+   114us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:45.593699 (+  5860us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:45.593816 (+   117us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:45.597259 (+  3443us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:45.597441 (+   182us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=10,key_file_lookups=10,delta_file_lookups=0,mrs_lookups=0
0709 12:14:45.597486 (+    45us) write_op.cc:312] APPLY: finished
0709 12:14:45.597723 (+   237us) log.cc:844] Serialized 57 byte log entry
0709 12:14:45.597898 (+   175us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:45.598006 (+   108us) write_op.cc:454] Released schema lock
0709 12:14:45.598051 (+    45us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:45.661388 (+ 63337us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:45.661419 (+    31us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":52,"spinlock_wait_cycles":115180416,"child_traces":[["op",{"apply.queue_time_us":130,"apply.run_cpu_time_us":8539,"apply.run_wall_time_us":73643,"cfile_cache_hit":20,"cfile_cache_hit_bytes":20870,"num_ops":5,"prepare.queue_time_us":9532,"prepare.run_cpu_time_us":3761,"prepare.run_wall_time_us":57653,"replication_time_us":2833}]]}
I20260709 12:14:45.955729 23699 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:45.892076 12572 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:45.959190 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:45.962711 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 81620.6us
I20260709 12:14:45.964428 23694 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:45.965464 23699 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.967036 23595 raft_consensus.cc:2749] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Leader pre-election lost for term 35. Reason: could not achieve majority
I20260709 12:14:45.968729 23699 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:45.973107 12646 leader_election.cc:336] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:45.973246 23693 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:45.977872 12673 raft_consensus.cc:1275] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 28 index: 19. Preceding OpId from leader: term: 29 index: 20. (index mismatch)
I20260709 12:14:45.979492 23595 consensus_queue.cc:1048] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.000s
I20260709 12:14:45.980624 12643 leader_election.cc:304] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:45.981774 23699 raft_consensus.cc:2804] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:14:45.982622 23699 raft_consensus.cc:697] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:45.963356 12425 catalog_manager.cc:5697] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 19 to 22, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 22 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:45.983686 23699 consensus_queue.cc:237] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 31.17, Last appended by leader: 17, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.002257 23700 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:46.002874 23700 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.005196 23700 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:46.005760 12646 consensus_peers.cc:597] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
I20260709 12:14:46.007331 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:46.009358 23693 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:46.011251 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aeb1937b72c34b78909bb9d4840f1f06" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 32 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:46.012010 12673 raft_consensus.cc:3060] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:46.025352 23664 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:46.025550 23593 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.026031 23664 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.026289 23593 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.028831 23593 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:46.031589 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:46.032389 12672 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 32.
W20260709 12:14:46.032755 12646 leader_election.cc:336] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:46.033416 23664 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:46.033965 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:46.035741 23691 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:14:46.035969 12673 raft_consensus.cc:2468] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 33.
I20260709 12:14:46.036190 23691 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:46.036660 23691 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:46.038832 12571 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:46.043336 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:46.043854 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:46.043977 12603 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 31.
I20260709 12:14:46.045828 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:46.046514 12603 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29.
I20260709 12:14:46.044597 12601 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 34.
I20260709 12:14:46.054895 23700 raft_consensus.cc:2804] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:14:46.055507 23700 raft_consensus.cc:697] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:46.056521 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:46.072201 13305 consensus_peers.cc:597] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:46.074334 23664 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:46.074924 23664 raft_consensus.cc:493] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:46.075392 23664 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:46.079135 23700 consensus_queue.cc:237] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 32.20, Last appended by leader: 20, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.084343 23664 raft_consensus.cc:515] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.087152 23691 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:46.074288 12572 leader_election.cc:336] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:46.090811 12646 leader_election.cc:336] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:46.102455 12572 consensus_peers.cc:597] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:46.103699 12572 leader_election.cc:336] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 25 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:46.104861 12572 leader_election.cc:336] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:46.105360 12572 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:46.106276 23693 raft_consensus.cc:2749] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:46.106928 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "43ba56f9f94346338994fdfb09969ba8" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 30 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:46.107685 12673 raft_consensus.cc:3060] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Advancing to term 33
W20260709 12:14:46.108014 12572 consensus_peers.cc:597] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:46.108846 12572 leader_election.cc:336] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:46.109265 12572 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:46.111189 23693 raft_consensus.cc:2749] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:14:46.114713 12572 consensus_peers.cc:597] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:46.120319 23557 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:46.120977 23557 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:46.125401 23587 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:46.126065 23587 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.128258 23587 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:46.130515 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:46.131242 13333 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 31.
I20260709 12:14:46.132845 12643 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:46.133996 23702 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election won for term 30
I20260709 12:14:46.134532 23702 raft_consensus.cc:493] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:46.134939 23702 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:46.103471 23691 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:46.138792 12643 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:46.131227 23701 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:46.146612 23701 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.149902 23701 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:46.146667 23699 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Leader pre-election won for term 35
I20260709 12:14:46.151648 23699 raft_consensus.cc:493] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:46.152240 23699 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:14:46.157162 23557 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:46.161641 12425 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59006 (request call id 180) took 87438 ms (client timeout 15000 ms). Trace:
I20260709 12:14:46.169289 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ade78efb80994e1b835a9aeec05fd8cd" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 31 index: 17 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:46.170063 12601 raft_consensus.cc:3060] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:46.172676 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:46.173408 12603 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 30.
W20260709 12:14:46.172508 12572 leader_election.cc:336] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:46.176378 12572 leader_election.cc:336] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:46.176828 12572 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:46.179888 12572 leader_election.cc:336] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:46.180562 12572 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:46.186139 23595 raft_consensus.cc:2749] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:46.186215 23691 raft_consensus.cc:2749] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:14:46.162289 12425 rpcz_store.cc:269] 0709 12:13:18.722714 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:18.722913 (+   199us) service_pool.cc:224] Handling call
0709 12:14:46.161484 (+87438571us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:45.997527 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:45.998199 (+   672us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:45.998245 (+    46us) write_op.cc:435] Acquired schema lock
0709 12:14:45.998267 (+    22us) tablet.cc:662] Decoding operations
0709 12:14:45.998738 (+   471us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:45.998805 (+    67us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:45.998837 (+    32us) tablet.cc:685] Acquiring locks for 2 operations
0709 12:14:45.999238 (+   401us) tablet.cc:701] Row locks acquired
0709 12:14:45.999260 (+    22us) write_op.cc:260] PREPARE: finished
0709 12:14:45.999435 (+   175us) write_op.cc:270] Start()
0709 12:14:45.999570 (+   135us) write_op.cc:276] Timestamp: P: 1783599285999382 usec, L: 0
0709 12:14:45.999611 (+    41us) op_driver.cc:348] REPLICATION: starting
0709 12:14:46.000333 (+   722us) log.cc:844] Serialized 929 byte log entry
0709 12:14:46.002844 (+  2511us) op_driver.cc:464] REPLICATION: finished
0709 12:14:46.003288 (+   444us) write_op.cc:301] APPLY: starting
0709 12:14:46.003382 (+    94us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:46.004691 (+  1309us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:46.004794 (+   103us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:46.006169 (+  1375us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:46.006382 (+   213us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=4,key_file_lookups=4,delta_file_lookups=0,mrs_lookups=0
0709 12:14:46.006455 (+    73us) write_op.cc:312] APPLY: finished
0709 12:14:46.006689 (+   234us) log.cc:844] Serialized 33 byte log entry
0709 12:14:46.006866 (+   177us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:46.006955 (+    89us) write_op.cc:454] Released schema lock
0709 12:14:46.006999 (+    44us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:46.007385 (+   386us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:46.007420 (+    35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":927801,"reported_tablets":53,"child_traces":[["op",{"apply.queue_time_us":242,"apply.run_cpu_time_us":4042,"apply.run_wall_time_us":4041,"cfile_cache_hit":8,"cfile_cache_hit_bytes":8348,"num_ops":2,"prepare.queue_time_us":8227,"prepare.run_cpu_time_us":3155,"prepare.run_wall_time_us":3480,"replication_time_us":2995}]]}
I20260709 12:14:46.192756 23593 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:46.193471 23593 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.198204 23701 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:46.198537 23593 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:46.200171 23701 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.202932 23701 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:46.203713 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:46.204456 12675 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
W20260709 12:14:46.210687 12572 leader_election.cc:336] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:46.211879 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:46.212180 12572 leader_election.cc:336] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:46.212882 23691 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:46.213364 23691 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:46.213924 23691 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:14:46.213418 12572 consensus_peers.cc:597] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:46.223989 12673 raft_consensus.cc:2468] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 33.
I20260709 12:14:46.225459 12571 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:46.234798 23593 raft_consensus.cc:2804] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:14:46.235426 23593 raft_consensus.cc:697] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:46.236476 23593 consensus_queue.cc:237] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 30.17, Last appended by leader: 17, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:46.238732 12572 leader_election.cc:336] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:46.252671 12646 leader_election.cc:336] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:46.254594 12572 consensus_peers.cc:597] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:46.256006 12572 leader_election.cc:336] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:46.258065 12601 raft_consensus.cc:2468] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 32.
W20260709 12:14:46.275275 12572 consensus_peers.cc:597] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:46.106254 23664 leader_election.cc:290] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:46.293280 12646 leader_election.cc:336] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:46.294505 12646 leader_election.cc:336] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:46.296675 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 156690us
I20260709 12:14:46.301235 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:46.302140 12604 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 25.
I20260709 12:14:46.303320 12601 raft_consensus.cc:1275] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:14:46.311269 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:46.327520 12643 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:46.340090 12646 leader_election.cc:336] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:46.341661 12426 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59006 (request call id 177) took 179205 ms (client timeout 15000 ms). Trace:
W20260709 12:14:46.342212 12426 rpcz_store.cc:269] 0709 12:11:47.123702 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:11:47.130295 (+  6593us) service_pool.cc:224] Handling call
0709 12:14:46.326174 (+179195879us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":208068,"reported_tablets":53}
I20260709 12:14:46.317850 23691 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.356405 12601 raft_consensus.cc:2393] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 29: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:46.360185 12673 raft_consensus.cc:1275] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 24 index: 14. Preceding OpId from leader: term: 25 index: 15. (index mismatch)
I20260709 12:14:46.381080 23724 consensus_queue.cc:1048] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 15, Last known committed idx: 14, Time since last communication: 0.001s
W20260709 12:14:46.396313 12646 consensus_peers.cc:597] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:46.374754 23701 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:46.407101 23701 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:46.422662 23701 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:46.423908 23691 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:46.439024 12646 consensus_peers.cc:597] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:46.442062 12646 consensus_peers.cc:597] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:46.441715 23699 raft_consensus.cc:515] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.444648 23699 leader_election.cc:290] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:46.375605 23702 raft_consensus.cc:515] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.447314 23702 leader_election.cc:290] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:46.457561 12572 consensus_peers.cc:597] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:46.458352 12572 leader_election.cc:336] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:46.478459 12646 leader_election.cc:336] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:46.481827 12646 leader_election.cc:336] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:46.481271 23691 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:46.490808 23691 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.493031 23691 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:46.505367 23719 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:46.517432 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 147277us
I20260709 12:14:46.523476 23701 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:46.525350 23719 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.525954 23701 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
W20260709 12:14:46.527575 12572 leader_election.cc:336] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:46.527769 23719 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:46.534787 12646 leader_election.cc:336] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:46.535430 12572 consensus_peers.cc:597] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:46.538355 12646 leader_election.cc:336] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:46.542681 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:45801, user_credentials={real_user=slave}} blocked reactor thread for 215256us
W20260709 12:14:46.551954 12428 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58980 (request call id 185) took 155962 ms (client timeout 15000 ms). Trace:
W20260709 12:14:46.553404 12646 consensus_peers.cc:597] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:46.544096 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:46.560511 12428 rpcz_store.cc:269] 0709 12:12:10.589459 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:10.589782 (+   323us) service_pool.cc:224] Handling call
0709 12:14:46.551717 (+155961935us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":470045,"reported_tablets":52}
I20260709 12:14:46.565065 12643 leader_election.cc:304] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:46.564342 23719 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election won for term 31
I20260709 12:14:46.566048 23720 raft_consensus.cc:2804] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:14:46.566653 23719 raft_consensus.cc:493] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:46.567095 23719 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:46.567238 23720 raft_consensus.cc:697] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:46.568411 23720 consensus_queue.cc:237] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 31.17, Last appended by leader: 17, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.570554 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:46.571171 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:46.571841 12675 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 31.
I20260709 12:14:46.572881 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8f2a3bfdbbd6440ba9cf687448fb678c" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:46.573395 23701 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election won for term 26
I20260709 12:14:46.573501 12673 raft_consensus.cc:3060] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
I20260709 12:14:46.574513 23731 consensus_queue.cc:1048] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:14:46.573947 23701 raft_consensus.cc:493] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:46.575820 23701 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:46.576431 12643 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:46.600330 12673 raft_consensus.cc:2468] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
I20260709 12:14:46.604322 12604 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "52c63945e7484332bf4c454ec7423a21" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 35 candidate_status { last_received { term: 34 index: 19 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:46.604322 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57b456cf0a3b4f0f9f6cae5a1862caca" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:46.605204 12604 raft_consensus.cc:3060] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Advancing to term 35
I20260709 12:14:46.605424 12603 raft_consensus.cc:3060] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:46.612666 13333 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "618bbf28983a42beaaf337df12784ee1" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:46.613436 13333 raft_consensus.cc:3060] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:46.618202 12597 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:46.597213 23701 raft_consensus.cc:515] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.635406 12605 raft_consensus.cc:1275] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 26 index: 17. Preceding OpId from leader: term: 27 index: 18. (index mismatch)
I20260709 12:14:46.616935 23731 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:46.636082 12604 raft_consensus.cc:2468] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 35.
I20260709 12:14:46.647464 12603 raft_consensus.cc:2468] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 30.
I20260709 12:14:46.642784 23557 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:46.651571 12424 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58992 (request call id 174) took 135793 ms (client timeout 15000 ms). Trace:
I20260709 12:14:46.651451 12421 catalog_manager.cc:5697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 27 to 33. New cstate: current_term: 33 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
W20260709 12:14:46.652168 12424 rpcz_store.cc:269] 0709 12:12:30.858608 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:12:30.859519 (+   911us) service_pool.cc:224] Handling call
0709 12:14:46.651343 (+135791824us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":15100,"reported_tablets":52}
I20260709 12:14:46.652179 23557 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.670203 23701 leader_election.cc:290] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:46.690804 13305 consensus_peers.cc:597] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 12:14:46.694945 12646 leader_election.cc:336] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:46.653785 12421 catalog_manager.cc:5697] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 26 to 27, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 27 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:46.702212 12421 catalog_manager.cc:5697] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 29 to 30, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 30 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:46.708953 13333 raft_consensus.cc:2468] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 32.
I20260709 12:14:46.696769 12571 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:46.685153 23557 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:46.674080 23719 raft_consensus.cc:515] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.744582 23719 leader_election.cc:290] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:46.741992 12643 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:46.749996 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:46.756695 12421 catalog_manager.cc:5697] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 22 to 29, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 29 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:46.760426 12421 catalog_manager.cc:5697] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 26 to 30. New cstate: current_term: 30 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
W20260709 12:14:46.762619 12571 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 65939.9us
I20260709 12:14:46.763386 12571 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:46.750509 23719 raft_consensus.cc:2804] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:14:46.751159 23664 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:46.768227 23721 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:46.769758 23721 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.756821 23699 consensus_queue.cc:1048] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.014s
I20260709 12:14:46.772081 23721 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:46.774683 23691 raft_consensus.cc:2804] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:46.775270 23691 raft_consensus.cc:697] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:46.776292 23691 consensus_queue.cc:237] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 28.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.777094 23729 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:46.777552 23729 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:46.777972 23729 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:46.762578 12643 leader_election.cc:304] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 35 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:46.786273 12646 leader_election.cc:336] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:46.790320 12646 leader_election.cc:336] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:46.791698 12643 leader_election.cc:304] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:46.788569 12421 catalog_manager.cc:5697] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 28 to 29, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 29 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:46.795903 12421 catalog_manager.cc:5697] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 13 to 30, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 30 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:46.808882 23727 raft_consensus.cc:2804] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:46.809794 23727 raft_consensus.cc:697] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:46.811051 23727 consensus_queue.cc:237] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.16, Last appended by leader: 16, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:46.814942 23733 log.cc:927] Time spent T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.164s	user 0.005s	sys 0.000s
I20260709 12:14:46.796819 23731 raft_consensus.cc:2804] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Leader election won for term 35
I20260709 12:14:46.823117 23731 raft_consensus.cc:697] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:46.824164 23731 consensus_queue.cc:237] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 34.19, Last appended by leader: 19, Current term: 35, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:46.830221 12646 consensus_peers.cc:597] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:46.840502 12601 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6530c20e384d4ff0915a09d0641c82f8" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 26 candidate_status { last_received { term: 25 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:46.841272 12601 raft_consensus.cc:3060] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Advancing to term 26
I20260709 12:14:46.847508 12606 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9b55aace8c0343b9808746fb1275f5cc" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 31 candidate_status { last_received { term: 30 index: 18 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:46.847981 12674 raft_consensus.cc:1275] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 32 index: 20. Preceding OpId from leader: term: 33 index: 21. (index mismatch)
I20260709 12:14:46.848219 12606 raft_consensus.cc:3060] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Advancing to term 31
I20260709 12:14:46.806092 12421 catalog_manager.cc:5697] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 21 to 24, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 24 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:46.853101 12603 raft_consensus.cc:1275] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 31 index: 17. Preceding OpId from leader: term: 32 index: 18. (index mismatch)
I20260709 12:14:46.865953 12601 raft_consensus.cc:2468] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:14:46.867022 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:46.869438 12605 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
W20260709 12:14:46.870846 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 126089us
I20260709 12:14:46.835218 23719 raft_consensus.cc:697] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:46.882076 12606 raft_consensus.cc:2468] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 31.
I20260709 12:14:46.911870 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:46.912923 12673 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:14:46.883948 23721 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:46.951803 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:46.956192 12643 leader_election.cc:304] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:46.918659 23720 consensus_queue.cc:1048] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.039s
I20260709 12:14:46.979863 12675 raft_consensus.cc:1275] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 30 index: 17. Preceding OpId from leader: term: 33 index: 18. (index mismatch)
I20260709 12:14:46.959743 23587 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:46.930374 23721 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:46.989399 23587 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:46.986892 23719 consensus_queue.cc:237] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:46.994972 23587 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:47.010371 23724 consensus_queue.cc:1048] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.001s
I20260709 12:14:47.010365 12571 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:46.966768 23664 raft_consensus.cc:2804] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader election won for term 26
I20260709 12:14:47.007804 23587 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:47.016022 23730 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:47.042317 23693 consensus_queue.cc:1048] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.026s
I20260709 12:14:47.053952 23664 raft_consensus.cc:697] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:47.056282 23730 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:47.082191 23587 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:47.085573 23730 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:47.088438 23595 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:47.089020 23595 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:47.065196 23691 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
W20260709 12:14:47.114070 12572 leader_election.cc:336] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:47.116127 12572 consensus_peers.cc:597] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:47.120203 12646 consensus_peers.cc:597] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:47.121472 12646 consensus_peers.cc:597] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 21: this message will repeat every 5th retry.
W20260709 12:14:47.122241 12646 leader_election.cc:336] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:47.125528 13305 consensus_peers.cc:597] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:47.131529 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 235158us
I20260709 12:14:47.138361 23747 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:47.139117 23747 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:47.145696 23747 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:47.147540 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:47.147989 23595 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:47.148239 12674 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 27.
I20260709 12:14:47.151067 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:47.152457 23742 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:47.152717 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:47.153946 12674 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 29.
W20260709 12:14:47.157014 23745 log.cc:927] Time spent T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.066s	user 0.001s	sys 0.000s
I20260709 12:14:47.160496 12571 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:47.071135 23729 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:47.152948 23742 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:47.169893 23595 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election won for term 30
W20260709 12:14:47.186189 12643 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:39365, user_credentials={real_user=slave}} blocked reactor thread for 228701us
W20260709 12:14:47.187702 12572 consensus_peers.cc:597] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:47.132685 23691 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:47.188244 12643 leader_election.cc:304] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:47.183493 23742 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:47.208099 12603 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:47.215966 23730 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:47.218716 23721 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:47.222713 23729 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:47.224777 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:47.226015 23691 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:47.226477 12678 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0d1b28b57b3d4a9184fbb79b00227f3a" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 30 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:47.225857 12673 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 31.
I20260709 12:14:47.227200 12678 raft_consensus.cc:3060] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:47.191010 23664 consensus_queue.cc:237] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 25.15, Last appended by leader: 15, Current term: 26, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:47.240221 23699 raft_consensus.cc:2804] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader election won for term 31
I20260709 12:14:47.240875 23699 raft_consensus.cc:697] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:47.241761 23699 consensus_queue.cc:237] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 18, Committed index: 18, Last appended: 30.18, Last appended by leader: 18, Current term: 31, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:47.251459 12603 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate 193a40da6e364ead8ab561e14c3cc8d1 in current term 28: Already voted for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in this term.
I20260709 12:14:47.188477 23595 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:47.262624 23595 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Advancing to term 30
I20260709 12:14:47.269068 12678 raft_consensus.cc:2468] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 32.
I20260709 12:14:47.271353 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 32 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:47.272382 12672 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 32.
I20260709 12:14:47.272789 12597 raft_consensus.cc:1275] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 31 index: 17. Preceding OpId from leader: term: 32 index: 18. (index mismatch)
I20260709 12:14:47.278440 23730 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:47.269343 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:47.306001 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:47.315743 12571 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:47.322870 23725 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:14:47.323428 23725 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:47.323804 23725 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:47.311219 23740 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Leader pre-election won for term 32
I20260709 12:14:47.332801 23740 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Starting leader election (detected failure of leader a2fdbe7eef6748a9b4611f6c9bc9b201)
I20260709 12:14:47.333317 23740 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Advancing to term 32
W20260709 12:14:47.336400 12572 consensus_peers.cc:597] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:47.342713 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:47.338779 12572 leader_election.cc:336] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:14:47.346262 12572 leader_election.cc:336] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:47.350735 23595 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:47.358893 23595 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:47.347859 23729 raft_consensus.cc:2804] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:14:47.367702 23729 raft_consensus.cc:697] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:47.368702 23729 consensus_queue.cc:237] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 30.19, Last appended by leader: 19, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:47.344259 23720 consensus_queue.cc:1048] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.005s
I20260709 12:14:47.343811 23748 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
I20260709 12:14:47.380276 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3c9b5315396948b2af3fa5480c9bee87" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 29 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:47.387270 12673 raft_consensus.cc:3060] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Advancing to term 30
W20260709 12:14:47.375599 12646 consensus_peers.cc:597] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:47.381356 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:47.386555 12680 raft_consensus.cc:1275] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 28 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
W20260709 12:14:47.379329 23761 log.cc:927] Time spent T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
I20260709 12:14:47.397759 12673 raft_consensus.cc:2468] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 30.
I20260709 12:14:47.417549 12672 raft_consensus.cc:2393] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Denying vote to candidate f5fc807f7ecc4d1ca0fd6be524f734ce in current term 28: Already voted for candidate 193a40da6e364ead8ab561e14c3cc8d1 in this term.
W20260709 12:14:47.421500 23760 log.cc:927] Time spent T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: Append to log took a long time: real 0.103s	user 0.001s	sys 0.000s
I20260709 12:14:47.374091 23742 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:47.431128 12646 consensus_peers.cc:597] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:47.436148 23730 consensus_queue.cc:1048] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.001s
I20260709 12:14:47.431432 12571 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
W20260709 12:14:47.446384 12572 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 99238.9us
W20260709 12:14:47.450872 12572 consensus_peers.cc:597] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:47.451263 23741 raft_consensus.cc:2804] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Leader election won for term 30
I20260709 12:14:47.471293 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e479222164574b9ea8944c530fbeb919" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:47.476727 12675 raft_consensus.cc:3060] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:14:47.475227 12572 consensus_peers.cc:597] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:47.479969 12572 leader_election.cc:336] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:47.459164 23742 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:47.458045 23725 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:47.491146 23725 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:47.475186 23741 raft_consensus.cc:697] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:47.497576 23741 consensus_queue.cc:237] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 29.17, Last appended by leader: 17, Current term: 30, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:47.502365 12572 connection.cc:769] client connection to 127.11.13.129:38811 send error: Network error: sendmsg error: Broken pipe (error 32)
I20260709 12:14:47.509783 12680 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "10395d6e75b84c3982521323f92e3309" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 33 candidate_status { last_received { term: 32 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:47.510774 12680 raft_consensus.cc:3060] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:47.503223 23740 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:47.536033 12675 raft_consensus.cc:2468] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 28.
I20260709 12:14:47.554916 12680 raft_consensus.cc:2468] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 33.
I20260709 12:14:47.557586 12571 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:47.563874 23740 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:47.580690 12571 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:47.579464 23595 raft_consensus.cc:2804] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:47.583384 23595 raft_consensus.cc:697] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:47.584677 23595 consensus_queue.cc:237] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 17, Committed index: 17, Last appended: 27.17, Last appended by leader: 17, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:47.589748 13333 raft_consensus.cc:1275] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
I20260709 12:14:47.595369 12672 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "29bbd70aa24b4e81b3d2764c3584ad60" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 32 candidate_status { last_received { term: 31 index: 20 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:47.596186 12672 raft_consensus.cc:3060] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Advancing to term 32
I20260709 12:14:47.591853 23727 consensus_queue.cc:1048] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.003s
I20260709 12:14:47.594214 23729 raft_consensus.cc:2804] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Leader election won for term 33
W20260709 12:14:47.627285 12646 consensus_peers.cc:597] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:47.612529 23729 raft_consensus.cc:697] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:47.629499 12424 catalog_manager.cc:5697] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 31 to 32, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 32 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } } }
I20260709 12:14:47.635872 23729 consensus_queue.cc:237] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 16, Committed index: 16, Last appended: 32.16, Last appended by leader: 16, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:47.644583 12672 raft_consensus.cc:2468] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5fc807f7ecc4d1ca0fd6be524f734ce in term 32.
I20260709 12:14:47.649307 12571 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:47.650897 23693 raft_consensus.cc:2804] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Leader election won for term 32
I20260709 12:14:47.651702 23693 raft_consensus.cc:697] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Becoming Leader. State: Replica: f5fc807f7ecc4d1ca0fd6be524f734ce, State: Running, Role: LEADER
I20260709 12:14:47.652822 23693 consensus_queue.cc:237] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 20, Committed index: 20, Last appended: 31.20, Last appended by leader: 20, Current term: 32, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:47.684865 12601 raft_consensus.cc:1275] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 29 index: 16. Preceding OpId from leader: term: 30 index: 17. (index mismatch)
I20260709 12:14:47.695892 12600 raft_consensus.cc:1275] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 34 index: 19. Preceding OpId from leader: term: 35 index: 20. (index mismatch)
W20260709 12:14:47.714583 12646 consensus_peers.cc:597] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:47.733665 23727 consensus_queue.cc:1048] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:14:47.750705 12572 consensus_peers.cc:597] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:47.751632 12572 consensus_peers.cc:597] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:47.756150 12572 leader_election.cc:336] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:47.758798 12572 leader_election.cc:336] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:47.763120 12572 leader_election.cc:336] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:47.765957 12572 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:47.756135 23748 consensus_queue.cc:1048] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.013s
I20260709 12:14:47.783069 23740 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader election lost for term 28. Reason: could not achieve majority
W20260709 12:14:47.790411 12572 leader_election.cc:336] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:47.791306 12572 consensus_peers.cc:597] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:47.791977 12572 leader_election.cc:336] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:47.792717 12572 leader_election.cc:336] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:47.793447 12572 leader_election.cc:336] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 32 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:47.795172 12572 consensus_peers.cc:597] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:47.795778 12572 consensus_peers.cc:597] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:47.798841 12598 raft_consensus.cc:1275] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 25 index: 15. Preceding OpId from leader: term: 26 index: 16. (index mismatch)
I20260709 12:14:47.799954 12601 raft_consensus.cc:1275] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 30 index: 18. Preceding OpId from leader: term: 31 index: 19. (index mismatch)
W20260709 12:14:47.804409 12572 consensus_peers.cc:597] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:47.804913 12572 consensus_peers.cc:597] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:47.805322 12572 consensus_peers.cc:597] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:47.810460 12572 consensus_peers.cc:597] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:47.823387 23699 consensus_queue.cc:1048] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 19, Last known committed idx: 18, Time since last communication: 0.002s
I20260709 12:14:47.821216 23748 consensus_queue.cc:1048] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.007s
W20260709 12:14:47.863595 12572 consensus_peers.cc:597] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:47.917063 12420 catalog_manager.cc:5697] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 33 to 34, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 34 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:47.919404 12420 catalog_manager.cc:5697] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 28 to 31, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 31 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:47.920490 12680 raft_consensus.cc:1275] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 29 index: 17. Preceding OpId from leader: term: 30 index: 18. (index mismatch)
W20260709 12:14:47.921516 12646 consensus_peers.cc:597] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:47.922191 12646 consensus_peers.cc:597] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:47.930845 12420 catalog_manager.cc:5697] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 30 to 33, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 33 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:47.931922 23693 consensus_queue.cc:1048] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 16, Time since last communication: 0.000s
I20260709 12:14:47.933249 12420 catalog_manager.cc:5697] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 27 to 30, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 30 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:47.935236 12420 catalog_manager.cc:5697] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 22 to 33, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 33 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:14:47.950663 13305 consensus_peers.cc:597] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:14:47.961403 12675 raft_consensus.cc:1275] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 28 index: 18. Preceding OpId from leader: term: 32 index: 20. (index mismatch)
I20260709 12:14:47.993589 23595 consensus_queue.cc:1048] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 18, Time since last communication: 0.000s
I20260709 12:14:47.990684 12420 catalog_manager.cc:5697] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 29 to 30, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 30 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:14:47.989212 12421 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58992 (request call id 178) took 13090 ms (client timeout 15000 ms). Trace:
I20260709 12:14:48.011687 12420 catalog_manager.cc:5697] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 30 to 32. New cstate: current_term: 32 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:14:48.016273 12646 consensus_peers.cc:597] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:48.023437 12420 catalog_manager.cc:5697] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 21 to 27, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 27 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:48.032928 12420 catalog_manager.cc:5697] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 29 to 31. New cstate: current_term: 31 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:48.038431 12420 catalog_manager.cc:5697] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 32 to 34. New cstate: current_term: 34 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:14:48.006197 12421 rpcz_store.cc:269] 0709 12:14:34.898820 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:40.341725 (+5442905us) service_pool.cc:224] Handling call
0709 12:14:41.110082 (+768357us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b5400280190. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fc0b2 00007f148e875759 00007f148e80da76 00007f148e7ded58 00007f148e8dd04e 00007f148cd9e2a4 00007f148cd9e232 00007f148cd9e161 00007f148cd9e0dc
0709 12:14:47.927405 (+6817323us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:47.125590 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:47.126294 (+   704us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:47.126341 (+    47us) write_op.cc:435] Acquired schema lock
0709 12:14:47.126366 (+    25us) tablet.cc:662] Decoding operations
0709 12:14:47.127362 (+   996us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:47.127434 (+    72us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:47.127468 (+    34us) tablet.cc:685] Acquiring locks for 8 operations
0709 12:14:47.128288 (+   820us) tablet.cc:701] Row locks acquired
0709 12:14:47.128306 (+    18us) write_op.cc:260] PREPARE: finished
0709 12:14:47.128483 (+   177us) write_op.cc:270] Start()
0709 12:14:47.128623 (+   140us) write_op.cc:276] Timestamp: P: 1783599287128427 usec, L: 0
0709 12:14:47.128656 (+    33us) op_driver.cc:348] REPLICATION: starting
0709 12:14:47.154067 (+ 25411us) log.cc:844] Serialized 3258 byte log entry
0709 12:14:47.494018 (+339951us) op_driver.cc:464] REPLICATION: finished
0709 12:14:47.583309 (+ 89291us) spinlock_profiling.cc:243] Waited 18.4 ms on lock 0x7b1800203500. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f14859333f3 00007f14859357c1 00007f1485935779 00007f1485935709 00007f14859356d1 00007f14859349cd 0000000000388634 0000000000388479 00007f148591683a
0709 12:14:47.583480 (+   171us) write_op.cc:301] APPLY: starting
0709 12:14:47.583572 (+    92us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:47.621738 (+ 38166us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:47.621761 (+    23us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:47.796168 (+174407us) spinlock_profiling.cc:243] Waited 4.18 ms on lock 0x7b1800203500. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f148e7fbab1 00007f148e808281 00007f148875f0ec 00007f1488761af7 00007f1488762fb0 00007f148c4771bb 00007f148c4be5d4 00007f148c36a8da 00007f148c36d3cb 00007f148c36cfd3 00007f148c46be34
0709 12:14:47.798204 (+  2036us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:47.841705 (+ 43501us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=16,key_file_lookups=16,delta_file_lookups=0,mrs_lookups=0
0709 12:14:47.841766 (+    61us) write_op.cc:312] APPLY: finished
0709 12:14:47.842161 (+   395us) log.cc:844] Serialized 81 byte log entry
0709 12:14:47.843082 (+   921us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:47.843238 (+   156us) write_op.cc:454] Released schema lock
0709 12:14:47.843296 (+    58us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:47.855939 (+ 12643us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:47.863182 (+  7243us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":52,"spinlock_wait_cycles":25533056,"child_traces":[["op",{"apply.queue_time_us":67040,"apply.run_cpu_time_us":18924,"apply.run_wall_time_us":266674,"cfile_cache_hit":34,"cfile_cache_hit_bytes":37519,"num_ops":8,"prepare.queue_time_us":122630,"prepare.run_cpu_time_us":15671,"prepare.run_wall_time_us":117390,"raft.queue_time_us":206214,"raft.run_cpu_time_us":62257,"raft.run_wall_time_us":137435,"replication_time_us":365154,"spinlock_wait_cycles":52002944,"thread_start_us":329045,"threads_started":4,"wal-append.queue_time_us":40062}]]}
I20260709 12:14:48.050336 12420 catalog_manager.cc:5697] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 30 to 32, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 32 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:48.062865 12420 catalog_manager.cc:5697] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 28 to 31, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 31 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:48.070915 12420 catalog_manager.cc:5697] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 24 to 27, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 27 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:48.082803 12420 catalog_manager.cc:5697] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 24 to 28. New cstate: current_term: 28 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:48.087688 12420 catalog_manager.cc:5697] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 26 to 29, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 29 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:48.094892 12420 catalog_manager.cc:5697] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 30 to 32. New cstate: current_term: 32 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:48.106874 12420 catalog_manager.cc:5697] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 33 to 35, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 35 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:48.115051 12420 catalog_manager.cc:5697] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 29 to 31. New cstate: current_term: 31 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:14:48.101310 12572 consensus_peers.cc:597] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:48.132361 12420 catalog_manager.cc:5697] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 22 to 26, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129). New cstate: current_term: 26 leader_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:48.134101 12420 catalog_manager.cc:5697] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 23 to 28, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 28 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:48.137879 12673 raft_consensus.cc:1275] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 27 index: 17. Preceding OpId from leader: term: 28 index: 18. (index mismatch)
I20260709 12:14:48.149905 12420 catalog_manager.cc:5697] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 27 to 29. New cstate: current_term: 29 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:48.150908 23595 consensus_queue.cc:1048] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 18, Last known committed idx: 17, Time since last communication: 0.000s
W20260709 12:14:48.138202 12572 consensus_peers.cc:597] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:48.153916 12420 catalog_manager.cc:5697] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 32 to 34, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 34 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
W20260709 12:14:48.127697 12646 consensus_peers.cc:597] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:48.161093 12675 raft_consensus.cc:1275] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 32 index: 16. Preceding OpId from leader: term: 33 index: 17. (index mismatch)
I20260709 12:14:48.169850 23595 consensus_queue.cc:1048] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 17, Last known committed idx: 16, Time since last communication: 0.001s
W20260709 12:14:48.179009 12572 consensus_peers.cc:597] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:48.197757 12680 raft_consensus.cc:1275] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Refusing update from remote peer f5fc807f7ecc4d1ca0fd6be524f734ce: Log matching property violated. Preceding OpId in replica: term: 31 index: 20. Preceding OpId from leader: term: 32 index: 21. (index mismatch)
W20260709 12:14:48.212976 13305 consensus_peers.cc:597] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:14:48.209110 23693 consensus_queue.cc:1048] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 21, Last known committed idx: 20, Time since last communication: 0.008s
W20260709 12:14:48.300755 12572 consensus_peers.cc:597] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:48.301226 12572 consensus_peers.cc:597] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:48.310494 13305 consensus_peers.cc:597] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:14:48.397159 13305 consensus_peers.cc:597] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:48.460382 11318 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 12552)
W20260709 12:14:48.499871 13305 consensus_peers.cc:597] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:14:48.524401 12572 consensus_peers.cc:597] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:48.558204 12421 catalog_manager.cc:5697] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 27 to 28, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 28 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:48.560441 12421 catalog_manager.cc:5697] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 33 to 35. New cstate: current_term: 35 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:48.568866 12420 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59006 (request call id 184) took 3187 ms. Trace:
I20260709 12:14:48.569371 12420 rpcz_store.cc:276] 0709 12:14:45.369737 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:14:45.369927 (+   190us) service_pool.cc:224] Handling call
0709 12:14:48.556550 (+3186623us) inbound_call.cc:177] Queueing success response
Related trace 'op':
0709 12:14:48.226349 (+     0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0709 12:14:48.226964 (+   615us) write_op.cc:432] Acquiring schema lock in shared mode
0709 12:14:48.227003 (+    39us) write_op.cc:435] Acquired schema lock
0709 12:14:48.227023 (+    20us) tablet.cc:662] Decoding operations
0709 12:14:48.228816 (+  1793us) write_op.cc:620] Acquiring the partition lock for write op
0709 12:14:48.228916 (+   100us) write_op.cc:641] Partition lock acquired for write op
0709 12:14:48.228947 (+    31us) tablet.cc:685] Acquiring locks for 22 operations
0709 12:14:48.283661 (+ 54714us) tablet.cc:701] Row locks acquired
0709 12:14:48.283685 (+    24us) write_op.cc:260] PREPARE: finished
0709 12:14:48.283851 (+   166us) write_op.cc:270] Start()
0709 12:14:48.285238 (+  1387us) write_op.cc:276] Timestamp: P: 1783599288283792 usec, L: 0
0709 12:14:48.285277 (+    39us) op_driver.cc:348] REPLICATION: starting
0709 12:14:48.298791 (+ 13514us) log.cc:844] Serialized 8686 byte log entry
0709 12:14:48.360925 (+ 62134us) op_driver.cc:464] REPLICATION: finished
0709 12:14:48.367012 (+  6087us) write_op.cc:301] APPLY: starting
0709 12:14:48.367117 (+   105us) tablet.cc:1367] starting BulkCheckPresence
0709 12:14:48.385267 (+ 18150us) tablet.cc:1370] finished BulkCheckPresence
0709 12:14:48.385365 (+    98us) tablet.cc:1372] starting ApplyRowOperation cycle
0709 12:14:48.487255 (+101890us) tablet.cc:1383] finished ApplyRowOperation cycle
0709 12:14:48.487694 (+   439us) tablet_metrics.cc:581] ProbeStats: bloom_lookups=44,key_file_lookups=44,delta_file_lookups=0,mrs_lookups=0
0709 12:14:48.487746 (+    52us) write_op.cc:312] APPLY: finished
0709 12:14:48.488580 (+   834us) log.cc:844] Serialized 196 byte log entry
0709 12:14:48.510303 (+ 21723us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:48.510551 (+   248us) write_op.cc:454] Released schema lock
0709 12:14:48.510627 (+    76us) write_op.cc:341] FINISH: Updating metrics
0709 12:14:48.566273 (+ 55646us) write_op.cc:489] Releasing partition, row and schema locks
0709 12:14:48.566320 (+    47us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":16342,"reported_tablets":33,"spinlock_wait_cycles":57472,"child_traces":[["op",{"apply.queue_time_us":5795,"apply.run_cpu_time_us":29205,"apply.run_wall_time_us":190373,"cfile_cache_hit":88,"cfile_cache_hit_bytes":91828,"num_ops":22,"prepare.queue_time_us":40180,"prepare.run_cpu_time_us":6613,"prepare.run_wall_time_us":72835,"replication_time_us":75452,"thread_start_us":19465,"threads_started":1}]]}
I20260709 12:14:48.573410 12421 catalog_manager.cc:5697] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 31 to 32, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 32 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:48.580039 12421 catalog_manager.cc:5697] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 28 to 32, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 32 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:48.581782 12421 catalog_manager.cc:5697] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 27 to 30, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 30 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:48.583006 12421 catalog_manager.cc:5697] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 32 to 33, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 33 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:48.584308 12421 catalog_manager.cc:5697] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 25 to 26, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 26 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:48.585631 12421 catalog_manager.cc:5697] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 27 to 33, leader changed from 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 33 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:48.588627 12421 catalog_manager.cc:5697] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 26 to 27, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 27 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:48.591022 12421 catalog_manager.cc:5697] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 24 to 25, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 25 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
W20260709 12:14:48.597658 12572 consensus_peers.cc:597] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:48.597222 12421 catalog_manager.cc:5697] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 31 to 32, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 32 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:48.599298 12421 catalog_manager.cc:5697] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 28 to 29, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 29 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:48.633744 12421 catalog_manager.cc:5697] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 34 to 35, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 35 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
W20260709 12:14:48.648056 12646 consensus_peers.cc:597] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:14:48.649510 12421 catalog_manager.cc:5697] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 32 to 33, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 33 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:48.655333 12421 catalog_manager.cc:5697] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 29 to 30, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 30 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:48.656847 12421 catalog_manager.cc:5697] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 30 to 31, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 31 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:48.658644 12421 catalog_manager.cc:5697] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 28 to 29, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 29 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
I20260709 12:14:48.660660 12421 catalog_manager.cc:5697] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 31 to 32, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 32 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } }
I20260709 12:14:48.664275 12421 catalog_manager.cc:5697] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce reported cstate change: term changed from 28 to 29, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130). New cstate: current_term: 29 leader_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
W20260709 12:14:48.880697 12646 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111) [suppressed 420 similar messages]
W20260709 12:14:48.910748 12572 consensus_peers.cc:597] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:48.958911 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 44756.7us
W20260709 12:14:48.965094 12572 consensus_peers.cc:597] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:48.971415 12646 consensus_peers.cc:597] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:49.050901 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 89267.5us
I20260709 12:14:49.088299 23702 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:14:49.088878 23702 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:49.101707 23702 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:49.129061 12646 leader_election.cc:336] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:49.132498 12572 consensus_peers.cc:597] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:14:49.134256 12605 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" is_pre_election: true
I20260709 12:14:49.135093 12605 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:49.138643 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:49.139978 23702 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election won for term 29
I20260709 12:14:49.140431 23702 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:14:49.140843 23702 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:14:49.129719 12646 consensus_peers.cc:597] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:49.154453 13305 consensus_peers.cc:597] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:14:49.154529 12572 consensus_peers.cc:597] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:14:49.161897 23702 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:49.171948 23702 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Requested vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:49.172322 12600 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 29 candidate_status { last_received { term: 27 index: 15 } } ignore_live_leader: false dest_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce"
I20260709 12:14:49.173017 12600 raft_consensus.cc:3060] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Advancing to term 29
W20260709 12:14:49.160436 12646 consensus_peers.cc:597] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:49.175442 12646 consensus_peers.cc:597] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:49.182736 12600 raft_consensus.cc:2468] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 29.
I20260709 12:14:49.192860 12643 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 
I20260709 12:14:49.194406 23702 raft_consensus.cc:2804] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader election won for term 29
I20260709 12:14:49.195286 23702 raft_consensus.cc:697] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:49.196399 23702 consensus_queue.cc:237] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 15, Committed index: 15, Last appended: 27.15, Last appended by leader: 15, Current term: 29, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:49.198803 12572 consensus_peers.cc:597] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:49.208983 12646 leader_election.cc:336] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:49.298861 12646 consensus_peers.cc:597] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:49.416366 12646 consensus_peers.cc:597] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:14:49.459030 12572 consensus_peers.cc:597] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:49.460903 11318 thread.cc:527] Waited for 2000ms trying to join with diag-logger (tid 12552)
W20260709 12:14:49.495564 12572 consensus_peers.cc:597] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:49.509395 12422 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58992 (request call id 176) took 81461 ms (client timeout 15000 ms). Trace:
W20260709 12:14:49.509922 12422 rpcz_store.cc:269] 0709 12:13:28.048199 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:34.261330 (+6213131us) service_pool.cc:224] Handling call
0709 12:13:35.067211 (+805881us) spinlock_profiling.cc:243] Waited 7.28 ms on lock 0x7b5400280190. stack: 00007f1485901343 00007f1485901240 00007f14852901d4 00007f148efce304 00007f148efce299 00007f14857bbdb1 00007f14858fbb71 00007f14858fc0b2 00007f148e875759 00007f148e80da76 00007f148e7ded58 00007f148e8dd04e 00007f148cd9e2a4 00007f148cd9e232 00007f148cd9e161 00007f148cd9e0dc
0709 12:14:49.509144 (+74441933us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":125306,"reported_tablets":52,"spinlock_wait_cycles":16746240}
W20260709 12:14:49.512125 12572 consensus_peers.cc:597] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:49.525063 13305 consensus_peers.cc:597] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:14:49.553064 12421 catalog_manager.cc:5697] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 27 to 29, leader changed from a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 29 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } }
W20260709 12:14:49.622047 12646 consensus_peers.cc:597] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:49.626170 12429 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58980 (request call id 187) took 90695 ms (client timeout 15000 ms). Trace:
W20260709 12:14:49.630362 12429 rpcz_store.cc:269] 0709 12:13:18.930658 (+     0us) service_pool.cc:167] Inserting onto call queue
0709 12:13:27.374440 (+8443782us) service_pool.cc:224] Handling call
0709 12:14:49.625926 (+82251486us) inbound_call.cc:177] Queueing success response
Metrics: {"mutex_wait_us":20922,"reported_tablets":52}
W20260709 12:14:49.631505 12415 connection.cc:466] server connection from 127.0.0.1:58980 torn down before Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:58980 (request call id 187) could send its response
W20260709 12:14:49.662660 12572 consensus_peers.cc:597] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:49.668465 12646 consensus_peers.cc:597] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:49.690651 12572 consensus_peers.cc:597] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:49.695174 12572 consensus_peers.cc:597] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:49.741497 12572 consensus_peers.cc:597] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:49.777774 12572 consensus_peers.cc:597] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:14:49.784992 12605 raft_consensus.cc:1275] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 27 index: 15. Preceding OpId from leader: term: 29 index: 16. (index mismatch)
I20260709 12:14:49.786950 23702 consensus_queue.cc:1048] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 16, Last known committed idx: 15, Time since last communication: 0.000s
W20260709 12:14:49.793957 12572 consensus_peers.cc:597] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:49.854017 12646 consensus_peers.cc:597] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:49.989784 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 46645.6us
W20260709 12:14:49.991603 12701 debug-util.cc:398] Leaking SignalData structure 0x7b080030a300 after lost signal to thread 11321
W20260709 12:14:49.992811 12701 debug-util.cc:398] Leaking SignalData structure 0x7b0800041d00 after lost signal to thread 12463
W20260709 12:14:49.994237 12701 debug-util.cc:398] Leaking SignalData structure 0x7b0800042960 after lost signal to thread 12630
W20260709 12:14:49.995373 12701 debug-util.cc:398] Leaking SignalData structure 0x7b08000889c0 after lost signal to thread 12704
W20260709 12:14:49.996598 12701 debug-util.cc:398] Leaking SignalData structure 0x7b08001bdae0 after lost signal to thread 13363
W20260709 12:14:49.997314 12701 debug-util.cc:398] Leaking SignalData structure 0x7b08003d4260 after lost signal to thread 23786
W20260709 12:14:50.088047 12646 consensus_peers.cc:597] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:50.197698 12572 consensus_peers.cc:597] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:50.255762 12646 consensus_peers.cc:597] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:50.280994 12572 consensus_peers.cc:597] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:50.284384 12572 consensus_peers.cc:597] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:50.361766 12646 consensus_peers.cc:597] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:50.370137 12646 consensus_peers.cc:597] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:50.438519 12572 consensus_peers.cc:597] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:50.462083 11318 thread.cc:527] Waited for 3000ms trying to join with diag-logger (tid 12552)
W20260709 12:14:50.470268 12572 consensus_peers.cc:597] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:50.471251 12572 consensus_peers.cc:597] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:50.490432 12572 consensus_peers.cc:597] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:50.545547 12646 consensus_peers.cc:597] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:50.552659 12572 consensus_peers.cc:597] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:50.561069 12646 consensus_peers.cc:597] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:50.592453 12572 consensus_peers.cc:597] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:50.606669 12646 consensus_peers.cc:597] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:50.712113 12572 consensus_peers.cc:597] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:50.730760 13305 consensus_peers.cc:597] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:14:50.776664 11318 tablet_server.cc:196] TabletServer@127.11.13.129:0 shutdown complete.
W20260709 12:14:50.815465 13305 consensus_peers.cc:597] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:14:50.825066 13305 consensus_peers.cc:597] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:14:51.594983 13305 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 129320us
W20260709 12:14:51.625856 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 69798.1us
W20260709 12:14:51.662635 13305 consensus_peers.cc:597] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:14:51.700984 13305 consensus_peers.cc:597] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:51.721789 13304 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:38613, user_credentials={real_user=slave}} blocked reactor thread for 50805.3us
W20260709 12:14:51.728122 12572 consensus_peers.cc:597] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:51.763293 12646 consensus_peers.cc:597] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:51.765269 12646 consensus_peers.cc:597] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:14:51.797271 11318 tablet_server.cc:179] TabletServer@127.11.13.130:0 shutting down...
W20260709 12:14:51.804124 12572 consensus_peers.cc:597] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:51.806515 12572 consensus_peers.cc:597] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:51.822250 12572 consensus_peers.cc:597] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:51.897866 12572 consensus_peers.cc:597] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:51.899875 12572 consensus_peers.cc:597] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:51.900394 12572 consensus_peers.cc:597] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:14:51.917303 12572 consensus_peers.cc:597] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:51.924516 13302 consensus_peers.cc:597] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:51.926270 12643 consensus_peers.cc:597] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:51.937177 13302 consensus_peers.cc:597] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:51.942656 12646 consensus_peers.cc:597] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:51.948635 13305 consensus_peers.cc:597] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:14:51.962544 13302 consensus_peers.cc:597] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:51.987558 12646 consensus_peers.cc:597] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:52.027776 12572 consensus_peers.cc:597] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:52.049371 13302 consensus_peers.cc:597] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:52.055837 12646 consensus_peers.cc:597] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:52.075930 12643 consensus_peers.cc:597] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:52.078320 12572 consensus_peers.cc:597] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:14:52.079138 12646 consensus_peers.cc:597] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:52.109364 13302 consensus_peers.cc:597] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:52.112241 12643 consensus_peers.cc:597] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:52.124895 12572 consensus_peers.cc:597] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:14:52.175100 12643 consensus_peers.cc:597] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:52.176586 12643 consensus_peers.cc:597] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:52.177670 12643 consensus_peers.cc:597] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:52.178901 12643 consensus_peers.cc:597] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:52.179616 12460 debug-util.cc:398] Leaking SignalData structure 0x7b080000b6e0 after lost signal to thread 11321
W20260709 12:14:52.180198 12643 consensus_peers.cc:597] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:52.181172 12460 debug-util.cc:398] Leaking SignalData structure 0x7b0800364880 after lost signal to thread 12463
W20260709 12:14:52.182729 12460 debug-util.cc:398] Leaking SignalData structure 0x7b080022a5e0 after lost signal to thread 12630
W20260709 12:14:52.183576 13302 consensus_peers.cc:597] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:52.184026 12460 debug-util.cc:398] Leaking SignalData structure 0x7b0800298000 after lost signal to thread 12704
W20260709 12:14:52.184666 12643 consensus_peers.cc:597] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:52.185545 12460 debug-util.cc:398] Leaking SignalData structure 0x7b08003bb360 after lost signal to thread 13363
W20260709 12:14:52.212920 12643 consensus_peers.cc:597] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:52.223222 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
W20260709 12:14:52.225915 12643 consensus_peers.cc:597] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:52.234537 11318 tablet_replica.cc:333] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:52.235837 11318 raft_consensus.cc:2243] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:52.242853 11318 raft_consensus.cc:2272] T bcef55b2ef044bcb9d9a9ffd855c20ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:52.234062 12643 consensus_peers.cc:597] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:52.258478 11318 tablet_replica.cc:333] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:52.264384 11318 raft_consensus.cc:2243] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:52.284443 11318 raft_consensus.cc:2272] T 3a68955866d543408acca9e959ad1b26 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:52.303272 11318 tablet_replica.cc:333] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
W20260709 12:14:52.344720 13305 consensus_peers.cc:597] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
W20260709 12:14:52.351898 12572 consensus_peers.cc:597] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:14:52.358938 11318 raft_consensus.cc:2243] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:52.363531 11318 raft_consensus.cc:2272] T 111842005e864eab9966b3a00ab9ae8a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:52.376011 11318 tablet_replica.cc:333] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
W20260709 12:14:52.395018 12643 consensus_peers.cc:597] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:52.397198 12643 consensus_peers.cc:597] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:52.401767 12646 consensus_peers.cc:597] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:52.399013 11318 raft_consensus.cc:2243] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:14:52.413103 11318 raft_consensus.cc:2272] T 33b7d46307bc4c33acd6c9a58055c7af P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:52.420986 11318 tablet_replica.cc:333] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
W20260709 12:14:52.434602 12646 consensus_peers.cc:597] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:14:52.436120 12643 consensus_peers.cc:597] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:52.439510 12643 consensus_peers.cc:597] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:52.442276 11318 raft_consensus.cc:2243] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:52.443130 11318 raft_consensus.cc:2272] T 6fca0ed05cd34c8e92dc950351b864a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:52.465566 12646 consensus_peers.cc:597] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:52.478097 11318 tablet_replica.cc:333] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:52.480422 11318 raft_consensus.cc:2243] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:52.490895 11318 raft_consensus.cc:2272] T 19eaf42a0f664c919842f3c2f5cb868f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:52.500209 12572 consensus_peers.cc:597] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:52.505581 12572 consensus_peers.cc:597] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:14:52.507302 11318 tablet_replica.cc:333] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:52.521340 11318 raft_consensus.cc:2243] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:14:52.536593 11318 raft_consensus.cc:2272] T 43ba56f9f94346338994fdfb09969ba8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:52.540678 11318 tablet_replica.cc:333] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
W20260709 12:14:52.563155 12572 consensus_peers.cc:597] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:14:52.567322 11318 raft_consensus.cc:2243] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:52.575445 11318 raft_consensus.cc:2272] T 502d264440884ecda3d4641918b20d0c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:52.585371 11318 tablet_replica.cc:333] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:52.602145 11318 raft_consensus.cc:2243] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:14:52.607432 11318 raft_consensus.cc:2272] T 0d1b28b57b3d4a9184fbb79b00227f3a P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:52.621987 11318 tablet_replica.cc:333] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:52.635718 11318 raft_consensus.cc:2243] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:52.642613 11318 raft_consensus.cc:2272] T e1ff01ff4ede4fc990b9a07b23102b0d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:52.663621 11318 tablet_replica.cc:333] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:52.668377 11318 raft_consensus.cc:2243] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:14:52.683094 11318 raft_consensus.cc:2272] T ef80db1de7994402abd14cc926713fb0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:52.689175 11318 tablet_replica.cc:333] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:52.698874 11318 raft_consensus.cc:2243] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:52.711052 11318 raft_consensus.cc:2272] T 3c00651839ae4132bb331bc6c2602641 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:52.718611 12572 consensus_peers.cc:597] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:14:52.724963 11318 tablet_replica.cc:333] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:52.734505 11318 raft_consensus.cc:2243] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:14:52.739899 11318 raft_consensus.cc:2272] T 5771c48bd1d24152ab96cfc24da505ed P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:52.767926 11318 tablet_replica.cc:333] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:52.778466 11318 raft_consensus.cc:2243] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 LEADER]: Raft consensus shutting down.
I20260709 12:14:52.787278 11318 raft_consensus.cc:2272] T 6cb14d97e16d424a82a91a8ce1b6cc8d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:52.795167 11318 tablet_replica.cc:333] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:52.804771 11318 raft_consensus.cc:2243] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:14:52.811362 11318 raft_consensus.cc:2272] T 317be0a176ee473b9b3631bdc1030bcc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:52.817499 11318 tablet_replica.cc:333] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:52.831596 11318 raft_consensus.cc:2243] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:14:52.836091 11318 raft_consensus.cc:2272] T 816b6f34d124434882148e507cf5b612 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:52.840584 11318 tablet_replica.cc:333] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:52.852008 11318 raft_consensus.cc:2243] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:52.854411 11318 raft_consensus.cc:2272] T 852ac65e35c7462ba906c02b9422dc5b P f5fc807f7ecc4d1ca0fd6be524f734ce [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:52.867218 11318 tablet_replica.cc:333] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:52.873458 11318 raft_consensus.cc:2243] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:14:52.876093 11318 raft_consensus.cc:2272] T d7bd864b75c34027b05582a55b152877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:52.886667 12646 consensus_peers.cc:597] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:52.903862 11318 tablet_replica.cc:333] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:52.910187 11318 raft_consensus.cc:2243] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:52.911526 11318 raft_consensus.cc:2272] T 52c63945e7484332bf4c454ec7423a21 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:52.915175 11318 tablet_replica.cc:333] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
W20260709 12:14:52.939869 12646 consensus_peers.cc:597] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:52.943985 12646 consensus_peers.cc:597] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:52.945525 11318 raft_consensus.cc:2243] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:14:52.950753 11318 raft_consensus.cc:2272] T 3af899b7fab042d0ac433430f162d106 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:52.964047 11318 tablet_replica.cc:333] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:52.972000 11318 raft_consensus.cc:2243] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:52.975095 11318 raft_consensus.cc:2272] T ee47f7c4414b4ae98233a31696e99cc0 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 36 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:52.977854 12572 consensus_peers.cc:597] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:14:52.989378 12572 consensus_peers.cc:597] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:52.999876 11318 tablet_replica.cc:333] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.009330 11318 raft_consensus.cc:2243] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:53.011646 11318 raft_consensus.cc:2272] T 15456f360e78465994eb4d0d39e90b87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.018329 11318 tablet_replica.cc:333] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.036154 11318 raft_consensus.cc:2243] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:14:53.049387 11318 raft_consensus.cc:2272] T 7b825dff40dc4290a25d71534b598330 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.060241 11318 tablet_replica.cc:333] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
W20260709 12:14:53.062908 12646 consensus_peers.cc:597] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:53.075390 11318 raft_consensus.cc:2243] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:53.076051 11318 raft_consensus.cc:2272] T 6530c20e384d4ff0915a09d0641c82f8 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:53.073460 12572 consensus_peers.cc:597] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:14:53.089744 23593 raft_consensus.cc:493] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:53.090544 23593 raft_consensus.cc:515] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:53.095170 23593 leader_election.cc:290] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:53.095494 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:53.095281 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e15d21db797e4a96b5732711a5379980" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 26 candidate_status { last_received { term: 25 index: 17 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:53.098659 12571 leader_election.cc:304] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:53.099563 23593 raft_consensus.cc:2749] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:53.116326 11318 tablet_replica.cc:333] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.117473 11318 raft_consensus.cc:2243] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:14:53.119442 11318 raft_consensus.cc:2272] T cdb0bb14b2b745b398e14d8c1a7139a2 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.120954 23593 raft_consensus.cc:493] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 36 FOLLOWER]: Starting pre-election (detected failure of leader e56d487365394ac5ac5f1002926a5645)
I20260709 12:14:53.121587 23593 raft_consensus.cc:515] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 36 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:53.125248 23593 leader_election.cc:290] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 37 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:53.125465 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:53.125877 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4270d5e0462b49d2982145525764c7fa" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 37 candidate_status { last_received { term: 36 index: 24 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:53.127552 12571 leader_election.cc:304] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 37 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
I20260709 12:14:53.128382 23593 raft_consensus.cc:2749] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 36 FOLLOWER]: Leader pre-election lost for term 37. Reason: could not achieve majority
W20260709 12:14:53.129033 12646 consensus_peers.cc:597] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:53.135387 12646 consensus_peers.cc:597] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:53.142768 11318 tablet_replica.cc:333] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.144272 11318 raft_consensus.cc:2243] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:53.151892 11318 raft_consensus.cc:2272] T 9b55aace8c0343b9808746fb1275f5cc P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.165373 11318 tablet_replica.cc:333] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.166517 11318 raft_consensus.cc:2243] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:14:53.167976 11318 raft_consensus.cc:2272] T bdfb480850c24efa8a38a916d53bdb1f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.173501 11318 tablet_replica.cc:333] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.186846 11318 raft_consensus.cc:2243] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:53.188148 11318 raft_consensus.cc:2272] T 57b456cf0a3b4f0f9f6cae5a1862caca P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.191671 11318 tablet_replica.cc:333] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.198858 11318 raft_consensus.cc:2243] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Raft consensus shutting down.
W20260709 12:14:53.202409 12646 consensus_peers.cc:597] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:14:53.204120 11318 raft_consensus.cc:2272] T ade78efb80994e1b835a9aeec05fd8cd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.207192 11318 tablet_replica.cc:333] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.208055 11318 raft_consensus.cc:2243] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:14:53.209669 11318 raft_consensus.cc:2272] T 6fa9b4ceed3443f2a6992343ab6ad877 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.228802 11318 tablet_replica.cc:333] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.246963 11318 raft_consensus.cc:2243] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 LEADER]: Raft consensus shutting down.
I20260709 12:14:53.249059 11318 raft_consensus.cc:2272] T e40144c1c60847ee81b8252b107df67d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.253168 11318 tablet_replica.cc:333] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.254856 11318 raft_consensus.cc:2243] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:14:53.263679 11318 raft_consensus.cc:2272] T aeb1937b72c34b78909bb9d4840f1f06 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.272464 23593 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:53.273285 11318 tablet_replica.cc:333] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.273105 23593 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:53.274382 11318 raft_consensus.cc:2243] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:53.275142 11318 raft_consensus.cc:2272] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.278321 23837 raft_consensus.cc:493] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:53.278874 23837 raft_consensus.cc:515] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:53.281075 23837 leader_election.cc:290] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:53.281586 23837 raft_consensus.cc:493] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:53.282279 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:53.282756 23593 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:53.285969 12673 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dd783245c5154ded95c4fe8434d0b380" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 29 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:53.286330 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "569d6ab756014cd88db2198d5d25dccd" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 29 candidate_status { last_received { term: 28 index: 14 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:53.291895 12571 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645
W20260709 12:14:53.293411 12571 raft_consensus.cc:2686] T dd783245c5154ded95c4fe8434d0b380 P f5fc807f7ecc4d1ca0fd6be524f734ce: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260709 12:14:53.294740 11318 tablet_replica.cc:333] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.282068 23837 raft_consensus.cc:515] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:53.300143 23837 leader_election.cc:290] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:53.299196 12572 leader_election.cc:336] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:53.300838 12572 leader_election.cc:304] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:53.301635 11318 raft_consensus.cc:2243] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:14:53.301853 23593 raft_consensus.cc:2749] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:53.303208 11318 raft_consensus.cc:2272] T d29f84bdbc4c49278fb44e72ddb1a33c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.309302 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3585ecc88ce149c5b13cb3e294485b62" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 30 candidate_status { last_received { term: 29 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:53.313400 11318 tablet_replica.cc:333] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.322965 11318 raft_consensus.cc:2243] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:53.323913 11318 raft_consensus.cc:2272] T 4270d5e0462b49d2982145525764c7fa P f5fc807f7ecc4d1ca0fd6be524f734ce [term 36 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:53.313237 12572 leader_election.cc:336] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:53.331730 11318 tablet_replica.cc:333] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.332751 11318 raft_consensus.cc:2243] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 LEADER]: Raft consensus shutting down.
W20260709 12:14:53.338994 12572 consensus_peers.cc:597] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:14:53.342682 12571 leader_election.cc:304] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:53.345052 11318 raft_consensus.cc:2272] T f0d97a077cac41709974a6cde7865fcb P f5fc807f7ecc4d1ca0fd6be524f734ce [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.346355 23593 raft_consensus.cc:2749] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:53.348357 11318 tablet_replica.cc:333] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.345043 23841 raft_consensus.cc:493] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:53.350016 23841 raft_consensus.cc:515] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:53.351720 11318 raft_consensus.cc:2243] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:53.352994 23841 leader_election.cc:290] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:53.355592 11318 raft_consensus.cc:2272] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.359314 11318 tablet_replica.cc:333] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.360249 11318 raft_consensus.cc:2243] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:14:53.361750 11318 raft_consensus.cc:2272] T b65110412db1492aa47f4a845ec89242 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.365121 11318 tablet_replica.cc:333] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.366017 11318 raft_consensus.cc:2243] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:14:53.368161 11318 raft_consensus.cc:2272] T 70b8c9b89e3d43fbb70422110d76702c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.371994 11318 tablet_replica.cc:333] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.373548 11318 raft_consensus.cc:2243] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:14:53.377413 11318 raft_consensus.cc:2272] T 47b417a13f2d44ef8922eb717c8ab20d P f5fc807f7ecc4d1ca0fd6be524f734ce [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.380949 11318 tablet_replica.cc:333] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.381855 11318 raft_consensus.cc:2243] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:14:53.383129 11318 raft_consensus.cc:2272] T 33a0b4d6cfba4ab2b3d54f591439b179 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.384855 23827 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:53.385502 23827 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:53.386444 11318 tablet_replica.cc:333] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.387552 11318 raft_consensus.cc:2243] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:14:53.388162 23827 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:53.389042 11318 raft_consensus.cc:2272] T cf6806e0707d42a0b80b9b9a65444604 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:53.394088 12646 leader_election.cc:336] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:53.395836 11318 tablet_replica.cc:333] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.396832 11318 raft_consensus.cc:2243] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:14:53.398862 11318 raft_consensus.cc:2272] T 05ff02a879ef4c94966348127fec27f7 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.402014 11318 tablet_replica.cc:333] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.404945 11318 raft_consensus.cc:2243] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:14:53.410274 11318 raft_consensus.cc:2272] T 3c9b5315396948b2af3fa5480c9bee87 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.411755 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "53977095735a4e34bc297c91e8c6f17f" candidate_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" candidate_term: 28 candidate_status { last_received { term: 27 index: 18 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:53.414187 11318 tablet_replica.cc:333] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.415508 11318 raft_consensus.cc:2243] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:53.416353 11318 raft_consensus.cc:2272] T 3585ecc88ce149c5b13cb3e294485b62 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:53.432447 13305 consensus_peers.cc:597] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:14:53.437515 11318 tablet_replica.cc:333] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.438498 11318 raft_consensus.cc:2243] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:14:53.439872 11318 raft_consensus.cc:2272] T 5dd1b2c01b5e495f9f31aee68f93d394 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.444697 11318 tablet_replica.cc:333] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.451385 11318 raft_consensus.cc:2243] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:53.453869 11318 raft_consensus.cc:2272] T 569d6ab756014cd88db2198d5d25dccd P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.457604 11318 tablet_replica.cc:333] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.458540 11318 raft_consensus.cc:2243] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:14:53.459894 11318 raft_consensus.cc:2272] T 29bbd70aa24b4e81b3d2764c3584ad60 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.463277 11318 tablet_replica.cc:333] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.464066 11318 raft_consensus.cc:2243] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:14:53.465400 11318 raft_consensus.cc:2272] T e479222164574b9ea8944c530fbeb919 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:53.466446 12572 leader_election.cc:336] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:53.468899 12572 leader_election.cc:304] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f5fc807f7ecc4d1ca0fd6be524f734ce; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:14:53.469609 12572 raft_consensus.cc:2686] T 53977095735a4e34bc297c91e8c6f17f P f5fc807f7ecc4d1ca0fd6be524f734ce: unable to run election callback: Service unavailable: Thread pool token was shut down
W20260709 12:14:53.472468 12643 leader_election.cc:336] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:53.473038 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:53.468468 11318 tablet_replica.cc:333] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.474321 11318 raft_consensus.cc:2243] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:14:53.475004 23827 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:53.475998 11318 raft_consensus.cc:2272] T 10395d6e75b84c3982521323f92e3309 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:53.499018 11318 tablet_replica.cc:333] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.499954 11318 raft_consensus.cc:2243] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:14:53.501711 11318 raft_consensus.cc:2272] T 8f2a3bfdbbd6440ba9cf687448fb678c P f5fc807f7ecc4d1ca0fd6be524f734ce [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:53.534171 13305 consensus_peers.cc:597] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:14:53.552959 11318 tablet_replica.cc:333] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce: stopping tablet replica
I20260709 12:14:53.554014 11318 raft_consensus.cc:2243] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:53.555011 11318 raft_consensus.cc:2272] T e15d21db797e4a96b5732711a5379980 P f5fc807f7ecc4d1ca0fd6be524f734ce [term 25 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:53.609944 12646 consensus_peers.cc:597] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:53.630337 13305 consensus_peers.cc:597] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
W20260709 12:14:53.664031 13305 consensus_peers.cc:597] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:14:53.748916 23838 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:53.749531 23838 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:53.752512 23838 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:53.759975 12646 leader_election.cc:336] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:53.760746 12643 leader_election.cc:336] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:53.763036 12643 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:53.764325 23838 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:53.842334 23827 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:53.843163 23827 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:53.845661 23827 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:53.855124 12646 leader_election.cc:336] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:53.857031 12643 leader_election.cc:336] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:53.857633 12643 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:53.858951 23827 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:53.870191 23811 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:53.871109 23811 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:53.881882 13302 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 548 similar messages]
W20260709 12:14:53.884011 13305 leader_election.cc:336] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:53.883026 23811 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:53.887861 13302 leader_election.cc:336] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:14:53.893460 13302 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:53.904630 23842 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:14:53.927687 12646 consensus_peers.cc:597] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:14:53.974594 23838 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:53.975355 23838 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:53.978183 23838 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:53.986719 12646 leader_election.cc:336] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:54.039539 12643 leader_election.cc:336] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
W20260709 12:14:54.043759 12646 consensus_peers.cc:597] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:14:54.040176 12643 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.050205 23838 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:54.061168 23838 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.061847 23838 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:54.064415 23838 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:54.072109 12646 leader_election.cc:336] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:54.073752 12643 leader_election.cc:336] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:54.074419 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.076413 23838 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.080698 23838 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:54.083312 23838 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:54.091152 23827 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:14:54.106386 12643 leader_election.cc:336] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:54.117743 23845 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.118741 23845 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:54.121523 23845 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:54.129633 12646 leader_election.cc:336] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:54.130329 12646 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.131227 23845 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:14:54.134334 12646 leader_election.cc:336] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:54.166646 12643 leader_election.cc:336] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:54.167357 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.168411 23838 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:14:54.170919 23838 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.171633 23838 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:54.177977 23838 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:54.186738 12646 leader_election.cc:336] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:54.200145 13305 consensus_peers.cc:597] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:14:54.216151 12643 leader_election.cc:336] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:54.216779 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.217648 23838 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:54.220641 23842 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.221248 23842 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:54.228029 23842 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:54.232573 13305 leader_election.cc:336] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:54.237666 13302 leader_election.cc:336] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:54.238309 13302 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.239763 23842 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:54.241999 23838 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.242709 23838 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:54.245164 23838 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:54.252120 12646 leader_election.cc:336] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:54.255434 12643 leader_election.cc:336] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:54.256049 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.262187 23811 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.262840 23811 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:54.265398 23811 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:54.266363 23838 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:54.268906 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:54.272643 23838 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.273259 23838 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:54.275681 23838 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:54.280956 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:54.281787 13338 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 27.
W20260709 12:14:54.285431 12643 leader_election.cc:336] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
W20260709 12:14:54.287338 13302 leader_election.cc:336] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:54.287786 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.288019 13302 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:54.289324 13302 consensus_peers.cc:597] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:54.289875 23827 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election won for term 28
I20260709 12:14:54.291070 23827 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.291613 23827 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Advancing to term 28
I20260709 12:14:54.289863 23811 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:14:54.305958 13302 consensus_peers.cc:597] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:54.308036 12643 consensus_peers.cc:597] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:54.313743 12643 consensus_peers.cc:597] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:54.314224 12643 consensus_peers.cc:597] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:54.312502 23827 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:54.325578 23827 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:54.329388 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "816b6f34d124434882148e507cf5b612" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 28 candidate_status { last_received { term: 27 index: 19 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645"
I20260709 12:14:54.330158 13337 raft_consensus.cc:3060] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Advancing to term 28
W20260709 12:14:54.333045 12643 leader_election.cc:336] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:54.360309 13337 raft_consensus.cc:2468] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 28.
I20260709 12:14:54.371337 12643 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.372951 23827 raft_consensus.cc:2804] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader election won for term 28
I20260709 12:14:54.373708 23827 raft_consensus.cc:697] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 LEADER]: Becoming Leader. State: Replica: 193a40da6e364ead8ab561e14c3cc8d1, State: Running, Role: LEADER
I20260709 12:14:54.375056 23827 consensus_queue.cc:237] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 19, Committed index: 19, Last appended: 27.19, Last appended by leader: 19, Current term: 28, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:54.375921 23845 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.376624 23845 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:54.380664 23845 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:54.392972 12646 leader_election.cc:336] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:54.407250 13302 consensus_peers.cc:597] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:54.407119 23811 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.408420 23811 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:54.410310 23845 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.410945 23845 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:54.414088 23845 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:54.416443 13305 leader_election.cc:336] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:54.433940 23811 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:54.440397 13302 leader_election.cc:336] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:54.440861 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.442183 23811 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:14:54.430114 12646 leader_election.cc:336] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:54.443908 23845 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.444900 23845 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:54.452617 23845 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:54.443589 12643 leader_election.cc:336] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:54.454298 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:54.455284 13305 consensus_peers.cc:597] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:14:54.457201 23845 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:14:54.476248 12643 leader_election.cc:336] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:54.476933 12643 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.477825 23845 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:14:54.479203 12643 consensus_peers.cc:597] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:54.488626 13302 consensus_peers.cc:597] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:54.502933 12646 leader_election.cc:336] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:54.503947 12643 leader_election.cc:336] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:54.505084 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.506250 23827 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:54.507326 23827 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.507881 23827 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:54.515970 13302 consensus_peers.cc:597] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:54.519009 23827 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:54.525075 23845 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.525792 23845 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:14:54.528236 23845 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
I20260709 12:14:54.534941 12421 catalog_manager.cc:5697] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 reported cstate change: term changed from 27 to 28, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131). New cstate: current_term: 28 leader_uuid: "193a40da6e364ead8ab561e14c3cc8d1" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260709 12:14:54.536856 23827 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.537827 23827 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:54.540792 23827 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:54.535770 12646 leader_election.cc:336] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:54.554423 12646 leader_election.cc:336] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:54.567253 23811 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.568037 23811 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:54.577412 13305 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:54.584551 23811 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:54.586642 12643 leader_election.cc:336] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:54.587287 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:54.588402 12643 leader_election.cc:336] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:54.588944 23827 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:14:54.589025 13302 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:54.589605 23845 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.589659 12643 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.590399 23845 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:54.591174 23827 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:54.593540 23845 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:54.596101 13302 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.597306 23811 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:14:54.599346 12643 leader_election.cc:336] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:54.599763 12677 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
W20260709 12:14:54.600288 12643 consensus_peers.cc:597] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:54.602553 12646 leader_election.cc:336] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:54.603125 12646 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.603971 23845 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:14:54.604890 12643 leader_election.cc:336] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
W20260709 12:14:54.612908 12643 consensus_peers.cc:597] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:54.616698 12643 consensus_peers.cc:597] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:54.620797 12643 consensus_peers.cc:597] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:54.621382 12643 consensus_peers.cc:597] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:54.621446 12646 leader_election.cc:336] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:54.622501 12646 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.623553 23827 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:14:54.652385 12646 consensus_peers.cc:597] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:14:54.702314 23827 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:14:54.702605 12643 consensus_peers.cc:597] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:54.703061 23827 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:54.705790 23827 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:54.708666 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
W20260709 12:14:54.712572 12643 leader_election.cc:336] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:54.713765 12643 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: e56d487365394ac5ac5f1002926a5645, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.714550 23827 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.714735 23845 raft_consensus.cc:2749] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:54.715271 23827 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:54.718262 23827 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:54.731925 12643 leader_election.cc:336] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:54.742720 23811 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.743428 23811 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:54.748126 12646 leader_election.cc:336] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:54.748687 12646 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.753454 23827 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:14:54.757017 13302 leader_election.cc:336] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
W20260709 12:14:54.759847 13305 leader_election.cc:336] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:54.760375 13305 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:54.760520 12646 consensus_peers.cc:597] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:14:54.761317 23811 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:54.761955 23842 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:54.776700 23827 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:14:54.778180 12643 consensus_peers.cc:597] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:54.777841 23827 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:54.782243 23827 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:54.784842 12643 consensus_peers.cc:597] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:54.785411 12643 consensus_peers.cc:597] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:54.787457 12643 leader_election.cc:336] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:54.788801 23827 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.789443 23827 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:54.791455 23827 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:54.792171 12646 consensus_peers.cc:597] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:54.793316 12646 leader_election.cc:336] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:54.795624 12646 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.799620 23827 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:14:54.802416 12643 leader_election.cc:336] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
W20260709 12:14:54.803257 13302 consensus_peers.cc:597] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:54.808192 23827 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.808902 23827 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:54.812009 23827 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:54.823443 13337 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:54.824321 13337 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:14:54.829252 13337 raft_consensus.cc:1275] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Refusing update from remote peer 193a40da6e364ead8ab561e14c3cc8d1: Log matching property violated. Preceding OpId in replica: term: 27 index: 19. Preceding OpId from leader: term: 28 index: 20. (index mismatch)
W20260709 12:14:54.834686 12646 leader_election.cc:336] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:54.837677 23851 consensus_queue.cc:1048] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 20, Last known committed idx: 19, Time since last communication: 0.001s
I20260709 12:14:54.838073 12646 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.839715 23851 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:14:54.844183 12643 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
W20260709 12:14:54.862725 12646 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:54.864151 12646 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.871665 23851 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:14:54.871971 12646 consensus_peers.cc:597] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:14:54.879012 12643 consensus_peers.cc:597] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:54.893059 13305 consensus_peers.cc:597] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:14:54.920795 23811 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.921559 23811 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:54.924751 12643 consensus_peers.cc:597] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:54.931932 23811 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:54.932577 13302 leader_election.cc:336] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:54.934201 23827 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.934862 23827 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:54.937227 23827 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:54.943143 23851 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.943744 23851 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:54.946014 23851 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:54.951656 12643 consensus_peers.cc:597] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:54.954174 12646 leader_election.cc:336] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:54.957512 23851 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.958266 23851 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:54.960618 23851 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:54.962252 23851 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.962927 23851 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:54.968590 12675 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1" is_pre_election: true
I20260709 12:14:54.969354 12675 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 32.
W20260709 12:14:54.971853 12646 leader_election.cc:336] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:54.980306 12646 leader_election.cc:336] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:54.984807 11318 thread.cc:527] Waited for 1000ms trying to join with diag-logger (tid 12627)
I20260709 12:14:54.985014 13304 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.987784 23842 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Leader pre-election won for term 33
I20260709 12:14:54.980361 23851 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:54.988229 12643 leader_election.cc:336] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
W20260709 12:14:54.988458 12646 leader_election.cc:336] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:54.989166 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.988222 23842 raft_consensus.cc:493] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting leader election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:54.990013 23842 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:54.990104 23851 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:14:54.990417 12643 leader_election.cc:336] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:54.995376 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:54.996306 23851 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:14:55.002993 12643 leader_election.cc:336] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:55.003664 12643 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:55.004513 23851 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:14:55.004715 12643 leader_election.cc:336] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:55.005513 12643 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:55.006547 23851 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:14:55.011565 12643 consensus_peers.cc:597] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:55.018426 23842 raft_consensus.cc:515] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:55.021773 23842 leader_election.cc:290] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 election: Requested vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:55.026391 13302 leader_election.cc:336] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:55.028085 12674 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5dd1b2c01b5e495f9f31aee68f93d394" candidate_uuid: "e56d487365394ac5ac5f1002926a5645" candidate_term: 33 candidate_status { last_received { term: 32 index: 21 } } ignore_live_leader: false dest_uuid: "193a40da6e364ead8ab561e14c3cc8d1"
I20260709 12:14:55.028836 12674 raft_consensus.cc:3060] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Advancing to term 33
I20260709 12:14:55.043404 12674 raft_consensus.cc:2468] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e56d487365394ac5ac5f1002926a5645 in term 33.
I20260709 12:14:55.044996 13304 leader_election.cc:304] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:55.046190 23842 raft_consensus.cc:2804] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Leader election won for term 33
I20260709 12:14:55.046733 23842 raft_consensus.cc:697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 33 LEADER]: Becoming Leader. State: Replica: e56d487365394ac5ac5f1002926a5645, State: Running, Role: LEADER
I20260709 12:14:55.047791 23842 consensus_queue.cc:237] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 21, Committed index: 21, Last appended: 32.21, Last appended by leader: 21, Current term: 33, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:55.052791 12646 consensus_peers.cc:597] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:14:55.071352 12421 catalog_manager.cc:5697] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 reported cstate change: term changed from 32 to 33, leader changed from f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130) to e56d487365394ac5ac5f1002926a5645 (127.11.13.132). New cstate: current_term: 33 leader_uuid: "e56d487365394ac5ac5f1002926a5645" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260709 12:14:55.198736 12646 consensus_peers.cc:597] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:55.227432 12646 consensus_peers.cc:597] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:55.274921 12646 consensus_peers.cc:597] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:55.292579 12646 consensus_peers.cc:597] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:14:55.394449 23827 raft_consensus.cc:493] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:55.395153 23827 raft_consensus.cc:515] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:55.397967 23827 leader_election.cc:290] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:55.408209 12646 leader_election.cc:336] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:55.418792 12643 leader_election.cc:336] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:55.419530 12643 leader_election.cc:304] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:55.422771 23851 raft_consensus.cc:2749] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
I20260709 12:14:55.467585 12674 raft_consensus.cc:1275] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Refusing update from remote peer e56d487365394ac5ac5f1002926a5645: Log matching property violated. Preceding OpId in replica: term: 32 index: 21. Preceding OpId from leader: term: 33 index: 22. (index mismatch)
I20260709 12:14:55.471802 23855 consensus_queue.cc:1048] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [LEADER]: Connected to new peer: Peer: permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 22, Last known committed idx: 21, Time since last communication: 0.001s
W20260709 12:14:55.538839 13302 consensus_peers.cc:597] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:55.551187 23851 raft_consensus.cc:493] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:55.551891 23851 raft_consensus.cc:515] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:55.561877 23851 leader_election.cc:290] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:55.574007 12643 leader_election.cc:336] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
W20260709 12:14:55.581620 12646 leader_election.cc:336] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:55.582250 12646 leader_election.cc:304] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:55.583050 23827 raft_consensus.cc:2749] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:14:55.601432 12646 consensus_peers.cc:597] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:14:55.638260 12646 consensus_peers.cc:597] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:14:55.654304 23827 raft_consensus.cc:493] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:55.655081 23827 raft_consensus.cc:515] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:55.660742 23851 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:55.661352 23851 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:55.666855 23851 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:55.669961 12646 leader_election.cc:336] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:55.673592 12643 leader_election.cc:336] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:55.674286 23845 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:55.674891 23845 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:55.675668 12643 leader_election.cc:304] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:55.671806 23827 leader_election.cc:290] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:55.676793 23827 raft_consensus.cc:2749] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:55.681185 23845 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:55.682897 12646 leader_election.cc:336] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:55.683547 12646 leader_election.cc:336] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:55.695879 12643 leader_election.cc:336] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:55.696507 12643 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:55.699980 23845 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:14:55.719491 12646 consensus_peers.cc:597] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:55.734876 12643 leader_election.cc:336] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:55.741513 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:55.745777 23862 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:14:55.759526 23862 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:55.760233 23862 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:55.763468 23862 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:55.775075 23855 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:55.779098 23855 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:55.781374 23855 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:55.778345 12646 consensus_peers.cc:597] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:14:55.790640 13302 leader_election.cc:336] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
W20260709 12:14:55.793349 13305 leader_election.cc:336] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:55.794167 13305 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:55.796878 23861 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
W20260709 12:14:55.806773 12646 leader_election.cc:336] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:55.835186 12643 leader_election.cc:336] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:55.836586 12643 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:55.837734 23862 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:14:55.849522 12646 consensus_peers.cc:597] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:14:55.867641 23862 raft_consensus.cc:493] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:55.868223 23862 raft_consensus.cc:515] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:55.871501 23862 leader_election.cc:290] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:55.878046 12646 leader_election.cc:336] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:55.888978 13305 consensus_peers.cc:597] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:14:55.898932 12643 leader_election.cc:336] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:55.899714 12643 leader_election.cc:304] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:55.900739 23862 raft_consensus.cc:2749] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:14:55.963044 13305 consensus_peers.cc:597] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:14:55.967587 23855 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:55.968276 23855 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:55.981397 23855 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:55.997437 13305 leader_election.cc:336] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:56.007728 13302 leader_election.cc:336] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:56.008601 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:56.010061 23855 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:14:56.028352 23862 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:56.029455 23862 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:56.031984 23862 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:56.032732 12646 consensus_peers.cc:597] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:14:56.036322 12643 leader_election.cc:336] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
W20260709 12:14:56.057948 12646 leader_election.cc:336] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:56.058990 12646 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:56.060299 23862 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:56.061676 23862 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:56.062279 23862 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:56.064939 23862 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:56.070432 12643 leader_election.cc:336] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
W20260709 12:14:56.077346 12646 leader_election.cc:336] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:56.083763 12646 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:56.084807 23862 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:56.184770 23851 raft_consensus.cc:493] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:56.185369 23851 raft_consensus.cc:515] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:56.187824 23851 leader_election.cc:290] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:56.194056 12646 leader_election.cc:336] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:56.203037 12643 leader_election.cc:336] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:56.204218 12643 leader_election.cc:304] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:56.204872 13305 consensus_peers.cc:597] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:14:56.206063 23862 raft_consensus.cc:2749] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:14:56.229408 13305 consensus_peers.cc:597] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:14:56.278385 23851 raft_consensus.cc:493] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:56.279052 23851 raft_consensus.cc:515] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:56.279304 23862 raft_consensus.cc:493] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:56.279978 23862 raft_consensus.cc:515] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:56.281343 23851 leader_election.cc:290] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:56.286018 23862 leader_election.cc:290] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:56.324676 12646 leader_election.cc:336] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:56.330493 12646 leader_election.cc:336] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:56.326567 23855 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:14:56.337620 12646 consensus_peers.cc:597] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:14:56.337663 23855 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:56.340240 12643 leader_election.cc:336] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:56.340807 12643 leader_election.cc:304] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:56.344219 23862 raft_consensus.cc:2749] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:14:56.345498 12643 leader_election.cc:336] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:56.349437 12643 leader_election.cc:304] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:56.351981 13302 leader_election.cc:336] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:56.340323 23855 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:56.354107 13305 leader_election.cc:336] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:56.354581 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:56.355584 23862 raft_consensus.cc:2749] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:56.357574 23861 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:56.363626 23862 raft_consensus.cc:493] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:56.364436 23862 raft_consensus.cc:515] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:56.367054 23862 leader_election.cc:290] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:56.381932 12646 leader_election.cc:336] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:56.408861 12646 consensus_peers.cc:597] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:14:56.418707 12643 leader_election.cc:336] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:56.423079 12643 leader_election.cc:304] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:56.425194 23862 raft_consensus.cc:2749] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:14:56.434185 23862 raft_consensus.cc:493] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:56.435060 23862 raft_consensus.cc:515] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } }
I20260709 12:14:56.437482 23862 leader_election.cc:290] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811)
W20260709 12:14:56.453375 12646 leader_election.cc:336] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:56.473733 12643 leader_election.cc:336] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:56.482367 12643 leader_election.cc:304] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:56.490626 23862 raft_consensus.cc:493] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:56.491467 23862 raft_consensus.cc:515] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:56.493942 23862 leader_election.cc:290] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:56.496552 23862 raft_consensus.cc:2749] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:14:56.516366 12646 leader_election.cc:336] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:56.521572 12643 leader_election.cc:336] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:56.525184 12643 leader_election.cc:304] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:56.538851 23862 raft_consensus.cc:2749] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
I20260709 12:14:56.540144 23862 raft_consensus.cc:493] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:56.540834 23862 raft_consensus.cc:515] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:56.546680 23862 leader_election.cc:290] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:56.553347 23851 raft_consensus.cc:493] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:56.566277 23851 raft_consensus.cc:515] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:56.567387 23862 raft_consensus.cc:493] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:56.568492 23851 leader_election.cc:290] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:56.578572 23862 raft_consensus.cc:515] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:56.582796 23862 leader_election.cc:290] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:56.592151 23864 raft_consensus.cc:493] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:56.602730 23864 raft_consensus.cc:515] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:56.605124 23864 leader_election.cc:290] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:56.593138 12646 leader_election.cc:336] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:56.622416 23855 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:56.631580 23855 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:56.635643 23855 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:56.642935 13305 leader_election.cc:336] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:56.645649 13302 leader_election.cc:336] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:56.646113 13302 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:56.647753 23855 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:14:56.648684 12646 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:38811, user_credentials={real_user=slave}} blocked reactor thread for 55573.8us
W20260709 12:14:56.651329 12643 leader_election.cc:336] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:56.655445 12643 leader_election.cc:304] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 28 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:56.660274 23862 raft_consensus.cc:493] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:56.664490 23862 raft_consensus.cc:515] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:56.666708 23862 leader_election.cc:290] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:56.673277 12643 leader_election.cc:336] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:56.673969 23862 raft_consensus.cc:2749] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Leader pre-election lost for term 28. Reason: could not achieve majority
W20260709 12:14:56.678481 12646 leader_election.cc:336] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:56.680997 12646 leader_election.cc:304] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 31 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:56.692874 23862 raft_consensus.cc:2749] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Leader pre-election lost for term 31. Reason: could not achieve majority
W20260709 12:14:56.694284 12646 leader_election.cc:336] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:56.707552 12643 leader_election.cc:336] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:56.713868 12643 leader_election.cc:304] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:56.716526 11318 tablet_server.cc:196] TabletServer@127.11.13.130:0 shutdown complete.
I20260709 12:14:56.718204 23864 raft_consensus.cc:493] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:56.719432 23864 raft_consensus.cc:515] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:56.722060 23864 leader_election.cc:290] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:56.717386 23862 raft_consensus.cc:2749] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:14:56.715056 12646 leader_election.cc:336] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:56.735817 12643 leader_election.cc:336] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
W20260709 12:14:56.736243 12646 leader_election.cc:336] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:56.737448 12643 leader_election.cc:304] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:56.738536 23864 raft_consensus.cc:2749] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:14:56.740352 12646 leader_election.cc:336] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:56.777202 13302 consensus_peers.cc:597] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:56.800081 12643 leader_election.cc:336] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:56.800871 12643 leader_election.cc:304] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:56.802011 23864 raft_consensus.cc:2749] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:14:56.809033 13305 consensus_peers.cc:597] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:14:56.813362 23864 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:56.815435 23864 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:56.821857 23864 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), e56d487365394ac5ac5f1002926a5645 (127.11.13.132:45801)
I20260709 12:14:56.832350 13338 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "70b8c9b89e3d43fbb70422110d76702c" candidate_uuid: "193a40da6e364ead8ab561e14c3cc8d1" candidate_term: 27 candidate_status { last_received { term: 26 index: 16 } } ignore_live_leader: false dest_uuid: "e56d487365394ac5ac5f1002926a5645" is_pre_election: true
I20260709 12:14:56.833314 13338 raft_consensus.cc:2468] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 193a40da6e364ead8ab561e14c3cc8d1 in term 26.
I20260709 12:14:56.835723 23864 raft_consensus.cc:493] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:56.836489 23864 raft_consensus.cc:515] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:56.839157 23864 leader_election.cc:290] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:56.842046 12643 leader_election.cc:336] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:56.842676 12643 leader_election.cc:304] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:56.848542 13302 consensus_peers.cc:597] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:56.848809 23863 raft_consensus.cc:2749] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:14:56.853426 12646 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:56.854533 12643 consensus_peers.cc:597] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:56.858776 12643 consensus_peers.cc:597] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:56.860067 12643 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:56.860513 12643 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1, e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:56.862085 23863 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:14:56.862361 12646 leader_election.cc:336] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:56.868219 12643 leader_election.cc:336] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:56.868702 12643 leader_election.cc:304] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:56.869668 23863 raft_consensus.cc:2749] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:14:56.909001 13302 consensus_peers.cc:597] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:56.917526 13305 consensus_peers.cc:597] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:14:56.943397 12643 consensus_peers.cc:597] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:57.011211 12643 consensus_peers.cc:597] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:57.046229 12643 consensus_peers.cc:597] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:57.079633 23863 raft_consensus.cc:493] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
W20260709 12:14:57.082067 13302 consensus_peers.cc:597] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:57.080384 23863 raft_consensus.cc:515] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:57.085649 13302 consensus_peers.cc:597] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:57.091882 12643 leader_election.cc:336] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:57.093022 23863 leader_election.cc:290] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:57.098448 12646 leader_election.cc:336] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:57.099005 12646 leader_election.cc:304] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:57.100461 23863 raft_consensus.cc:2749] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:14:57.123592 13302 consensus_peers.cc:597] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:57.161335 12643 consensus_peers.cc:597] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:57.189841 12643 consensus_peers.cc:597] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:57.190467 12643 consensus_peers.cc:597] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:57.235574 11318 tablet_server.cc:179] TabletServer@127.11.13.131:0 shutting down...
W20260709 12:14:57.251677 13304 consensus_peers.cc:597] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 -> Peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Couldn't send request to peer 193a40da6e364ead8ab561e14c3cc8d1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:57.253906 12643 consensus_peers.cc:597] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:57.265645 12643 consensus_peers.cc:597] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:57.276328 12643 consensus_peers.cc:597] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:57.277127 12643 consensus_peers.cc:597] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:14:57.293334 12643 consensus_peers.cc:597] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:57.303045 23863 raft_consensus.cc:493] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:57.303665 23863 raft_consensus.cc:515] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:57.305958 23863 leader_election.cc:290] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:57.311239 12646 leader_election.cc:336] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:57.331439 13304 consensus_peers.cc:597] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 -> Peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Couldn't send request to peer 193a40da6e364ead8ab561e14c3cc8d1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:57.336930 12643 leader_election.cc:336] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:57.337568 12643 leader_election.cc:304] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:57.338487 23863 raft_consensus.cc:2749] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
W20260709 12:14:57.339915 13304 consensus_peers.cc:597] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 -> Peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Couldn't send request to peer 193a40da6e364ead8ab561e14c3cc8d1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:57.346285 13304 consensus_peers.cc:597] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 -> Peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Couldn't send request to peer 193a40da6e364ead8ab561e14c3cc8d1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:57.407457 12643 consensus_peers.cc:597] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:57.435241 23862 raft_consensus.cc:493] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:57.435832 23862 raft_consensus.cc:515] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:57.437973 23862 leader_election.cc:290] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:57.445921 12646 consensus_peers.cc:597] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
W20260709 12:14:57.450707 12643 leader_election.cc:336] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
W20260709 12:14:57.452351 12646 leader_election.cc:336] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:57.452898 12646 leader_election.cc:304] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:57.455976 23862 raft_consensus.cc:2749] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
W20260709 12:14:57.467432 13304 consensus_peers.cc:597] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 -> Peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Couldn't send request to peer 193a40da6e364ead8ab561e14c3cc8d1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:57.475034 13304 consensus_peers.cc:597] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 -> Peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Couldn't send request to peer 193a40da6e364ead8ab561e14c3cc8d1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:57.481192 12646 consensus_peers.cc:597] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 26: this message will repeat every 5th retry.
I20260709 12:14:57.487344 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:14:57.491096 11318 tablet_replica.cc:333] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.492431 11318 raft_consensus.cc:2243] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 LEADER]: Raft consensus shutting down.
W20260709 12:14:57.494210 12643 consensus_peers.cc:597] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:57.497198 11318 raft_consensus.cc:2272] T bcef55b2ef044bcb9d9a9ffd855c20ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.504069 11318 tablet_replica.cc:333] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
W20260709 12:14:57.509223 12646 consensus_peers.cc:597] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:14:57.512388 11318 raft_consensus.cc:2243] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.516299 11318 raft_consensus.cc:2272] T 3a68955866d543408acca9e959ad1b26 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:57.516391 12643 consensus_peers.cc:597] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:57.520136 11318 tablet_replica.cc:333] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
W20260709 12:14:57.524091 12643 consensus_peers.cc:597] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:14:57.528214 11318 raft_consensus.cc:2243] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.529019 11318 raft_consensus.cc:2272] T 33b7d46307bc4c33acd6c9a58055c7af P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:57.531340 13304 consensus_peers.cc:597] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 -> Peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Couldn't send request to peer 193a40da6e364ead8ab561e14c3cc8d1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:57.532671 11318 tablet_replica.cc:333] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.537196 11318 raft_consensus.cc:2243] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.539234 11318 raft_consensus.cc:2272] T 111842005e864eab9966b3a00ab9ae8a P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.542304 11318 tablet_replica.cc:333] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.543363 11318 raft_consensus.cc:2243] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.544404 11318 raft_consensus.cc:2272] T 6fca0ed05cd34c8e92dc950351b864a2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.547713 11318 tablet_replica.cc:333] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.548674 11318 raft_consensus.cc:2243] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.550756 11318 raft_consensus.cc:2272] T 19eaf42a0f664c919842f3c2f5cb868f P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:57.559962 13305 consensus_peers.cc:597] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
I20260709 12:14:57.561653 23864 raft_consensus.cc:493] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:57.562273 23864 raft_consensus.cc:515] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
I20260709 12:14:57.571039 23864 leader_election.cc:290] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:57.577860 11318 tablet_replica.cc:333] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
W20260709 12:14:57.579722 12646 leader_election.cc:336] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:57.580375 12643 leader_election.cc:336] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:57.580914 11318 raft_consensus.cc:2243] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.581154 12643 leader_election.cc:304] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:57.582212 11318 raft_consensus.cc:2272] T 6cb14d97e16d424a82a91a8ce1b6cc8d P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.582697 23864 raft_consensus.cc:2749] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:57.585752 11318 tablet_replica.cc:333] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.586826 11318 raft_consensus.cc:2243] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.587944 11318 raft_consensus.cc:2272] T 43ba56f9f94346338994fdfb09969ba8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.591385 11318 tablet_replica.cc:333] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.592581 11318 raft_consensus.cc:2243] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.593494 11318 raft_consensus.cc:2272] T ef80db1de7994402abd14cc926713fb0 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.598851 11318 tablet_replica.cc:333] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.600167 11318 raft_consensus.cc:2243] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.601162 11318 raft_consensus.cc:2272] T 5771c48bd1d24152ab96cfc24da505ed P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.604244 11318 tablet_replica.cc:333] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
W20260709 12:14:57.604983 12646 consensus_peers.cc:597] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:14:57.605170 11318 raft_consensus.cc:2243] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.605940 11318 raft_consensus.cc:2272] T 0d1b28b57b3d4a9184fbb79b00227f3a P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.608902 11318 tablet_replica.cc:333] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.609797 11318 raft_consensus.cc:2243] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.610888 11318 raft_consensus.cc:2272] T e1ff01ff4ede4fc990b9a07b23102b0d P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:57.612144 12646 consensus_peers.cc:597] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:14:57.613158 23855 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:57.613818 23855 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:57.622936 13305 leader_election.cc:336] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:57.629840 13302 leader_election.cc:336] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:57.630337 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:57.632756 23861 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:14:57.633994 23855 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:57.641435 11318 tablet_replica.cc:333] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.642596 11318 raft_consensus.cc:2243] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.643985 11318 raft_consensus.cc:2272] T 852ac65e35c7462ba906c02b9422dc5b P 193a40da6e364ead8ab561e14c3cc8d1 [term 34 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.655179 11318 tablet_replica.cc:333] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.656925 11318 raft_consensus.cc:2243] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Raft consensus shutting down.
W20260709 12:14:57.668885 12646 consensus_peers.cc:597] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 21: this message will repeat every 5th retry.
I20260709 12:14:57.672153 11318 raft_consensus.cc:2272] T 502d264440884ecda3d4641918b20d0c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.682317 11318 tablet_replica.cc:333] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.683316 11318 raft_consensus.cc:2243] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.684083 11318 raft_consensus.cc:2272] T 952cdab9198b44e4bdf1b472109715c3 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.691737 11318 tablet_replica.cc:333] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.693084 11318 raft_consensus.cc:2243] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.694113 11318 raft_consensus.cc:2272] T 317be0a176ee473b9b3631bdc1030bcc P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.697314 11318 tablet_replica.cc:333] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.698319 11318 raft_consensus.cc:2243] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.700639 11318 raft_consensus.cc:2272] T 816b6f34d124434882148e507cf5b612 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.703882 11318 tablet_replica.cc:333] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.705449 11318 raft_consensus.cc:2243] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.715067 11318 raft_consensus.cc:2272] T 8c7e0eddcc2b4449b7ed8821854646e2 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.718678 11318 tablet_replica.cc:333] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.719686 11318 raft_consensus.cc:2243] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 LEADER]: Raft consensus shutting down.
W20260709 12:14:57.721261 13304 consensus_peers.cc:597] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 -> Peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Couldn't send request to peer 193a40da6e364ead8ab561e14c3cc8d1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:14:57.736140 12646 consensus_peers.cc:597] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
I20260709 12:14:57.738559 11318 raft_consensus.cc:2272] T 52c63945e7484332bf4c454ec7423a21 P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.742154 11318 tablet_replica.cc:333] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.743311 11318 raft_consensus.cc:2243] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.744038 11318 raft_consensus.cc:2272] T 3af899b7fab042d0ac433430f162d106 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.747097 11318 tablet_replica.cc:333] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.747931 11318 raft_consensus.cc:2243] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.758247 11318 raft_consensus.cc:2272] T 6fa9b4ceed3443f2a6992343ab6ad877 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:57.774966 13304 consensus_peers.cc:597] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 -> Peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Couldn't send request to peer 193a40da6e364ead8ab561e14c3cc8d1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:57.779338 23864 raft_consensus.cc:493] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:57.779876 23864 raft_consensus.cc:515] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:57.793643 12646 leader_election.cc:336] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:57.795495 23864 leader_election.cc:290] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:57.801571 13304 consensus_peers.cc:597] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 -> Peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Couldn't send request to peer 193a40da6e364ead8ab561e14c3cc8d1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:14:57.805655 11318 tablet_replica.cc:333] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.807760 11318 raft_consensus.cc:2243] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.809085 11318 raft_consensus.cc:2272] T ade78efb80994e1b835a9aeec05fd8cd P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:57.806720 12643 leader_election.cc:336] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:57.809805 12643 leader_election.cc:304] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:57.810664 23864 raft_consensus.cc:2749] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:57.814836 11318 tablet_replica.cc:333] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.815903 11318 raft_consensus.cc:2243] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.818611 11318 raft_consensus.cc:2272] T 6530c20e384d4ff0915a09d0641c82f8 P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.823460 11318 tablet_replica.cc:333] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.824434 11318 raft_consensus.cc:2243] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.825194 11318 raft_consensus.cc:2272] T ee30d7cd60b04485b7e19c8d119d67bd P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.829042 11318 tablet_replica.cc:333] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.829792 23864 raft_consensus.cc:493] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:57.830750 23864 raft_consensus.cc:515] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } }
W20260709 12:14:57.851795 12646 leader_election.cc:336] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:57.855540 23864 leader_election.cc:290] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:57.862102 12643 leader_election.cc:336] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:57.862785 12643 leader_election.cc:304] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [CANDIDATE]: Term 26 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 193a40da6e364ead8ab561e14c3cc8d1; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:57.864269 23864 raft_consensus.cc:2749] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Leader pre-election lost for term 26. Reason: could not achieve majority
I20260709 12:14:57.865953 11318 raft_consensus.cc:2243] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.867508 11318 raft_consensus.cc:2272] T 9b55aace8c0343b9808746fb1275f5cc P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.893263 11318 tablet_replica.cc:333] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.894692 11318 raft_consensus.cc:2243] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.896109 11318 raft_consensus.cc:2272] T bdfb480850c24efa8a38a916d53bdb1f P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.906673 11318 tablet_replica.cc:333] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.907673 11318 raft_consensus.cc:2243] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.909072 11318 raft_consensus.cc:2272] T 57b456cf0a3b4f0f9f6cae5a1862caca P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.912842 11318 tablet_replica.cc:333] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.914044 11318 raft_consensus.cc:2243] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.915169 11318 raft_consensus.cc:2272] T aeb1937b72c34b78909bb9d4840f1f06 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.918869 11318 tablet_replica.cc:333] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.920107 11318 raft_consensus.cc:2243] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.921707 11318 raft_consensus.cc:2272] T cf6806e0707d42a0b80b9b9a65444604 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.924794 11318 tablet_replica.cc:333] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.925661 11318 raft_consensus.cc:2243] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.928203 11318 raft_consensus.cc:2272] T 8f2a3bfdbbd6440ba9cf687448fb678c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.931152 11318 tablet_replica.cc:333] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.932053 11318 raft_consensus.cc:2243] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.932818 11318 raft_consensus.cc:2272] T e15d21db797e4a96b5732711a5379980 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.937220 11318 tablet_replica.cc:333] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.938251 11318 raft_consensus.cc:2243] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.939571 11318 raft_consensus.cc:2272] T dd783245c5154ded95c4fe8434d0b380 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:57.946311 13302 consensus_peers.cc:597] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:57.946982 11318 tablet_replica.cc:333] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.947880 11318 raft_consensus.cc:2243] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.948729 11318 raft_consensus.cc:2272] T d29f84bdbc4c49278fb44e72ddb1a33c P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.952509 11318 tablet_replica.cc:333] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.953675 11318 raft_consensus.cc:2243] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.954396 11318 raft_consensus.cc:2272] T 4270d5e0462b49d2982145525764c7fa P 193a40da6e364ead8ab561e14c3cc8d1 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.958511 11318 tablet_replica.cc:333] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.959564 11318 raft_consensus.cc:2243] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:14:57.961247 11318 raft_consensus.cc:2272] T 618bbf28983a42beaaf337df12784ee1 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.963840 23855 raft_consensus.cc:493] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:57.964519 23855 raft_consensus.cc:515] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:57.967003 23855 leader_election.cc:290] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
I20260709 12:14:57.969780 11318 tablet_replica.cc:333] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.970778 11318 raft_consensus.cc:2243] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.971460 11318 raft_consensus.cc:2272] T f0d97a077cac41709974a6cde7865fcb P 193a40da6e364ead8ab561e14c3cc8d1 [term 31 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:57.974309 13305 leader_election.cc:336] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:57.974535 11318 tablet_replica.cc:333] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.975569 11318 raft_consensus.cc:2243] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 LEADER]: Raft consensus shutting down.
W20260709 12:14:57.976325 13302 leader_election.cc:336] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:14:57.976778 13302 leader_election.cc:304] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 32 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:57.977494 23861 raft_consensus.cc:2749] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Leader pre-election lost for term 32. Reason: could not achieve majority
I20260709 12:14:57.982224 11318 raft_consensus.cc:2272] T 53977095735a4e34bc297c91e8c6f17f P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:57.985157 11318 tablet_replica.cc:333] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:57.986083 11318 raft_consensus.cc:2243] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:57.986864 11318 raft_consensus.cc:2272] T 70b8c9b89e3d43fbb70422110d76702c P 193a40da6e364ead8ab561e14c3cc8d1 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:58.000516 11318 tablet_replica.cc:333] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:58.001381 11318 raft_consensus.cc:2243] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:58.002198 11318 raft_consensus.cc:2272] T b65110412db1492aa47f4a845ec89242 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:58.014164 11318 tablet_replica.cc:333] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:58.015376 11318 raft_consensus.cc:2243] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:58.016613 11318 raft_consensus.cc:2272] T 05ff02a879ef4c94966348127fec27f7 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:58.021292 11318 tablet_replica.cc:333] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:58.030798 11318 raft_consensus.cc:2243] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:58.032091 11318 raft_consensus.cc:2272] T 3c9b5315396948b2af3fa5480c9bee87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:58.035704 11318 tablet_replica.cc:333] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:58.036936 11318 raft_consensus.cc:2243] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:14:58.038915 11318 raft_consensus.cc:2272] T 3585ecc88ce149c5b13cb3e294485b62 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:58.042495 11318 tablet_replica.cc:333] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:58.043695 11318 raft_consensus.cc:2243] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:58.044344 11318 raft_consensus.cc:2272] T 5dd1b2c01b5e495f9f31aee68f93d394 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:58.047044 11318 tablet_replica.cc:333] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:58.047782 11318 raft_consensus.cc:2243] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:14:58.049243 11318 raft_consensus.cc:2272] T 569d6ab756014cd88db2198d5d25dccd P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:58.052342 11318 tablet_replica.cc:333] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:58.053151 11318 raft_consensus.cc:2243] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:58.053951 11318 raft_consensus.cc:2272] T 47b417a13f2d44ef8922eb717c8ab20d P 193a40da6e364ead8ab561e14c3cc8d1 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:58.056880 11318 tablet_replica.cc:333] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:58.057659 11318 raft_consensus.cc:2243] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:58.058517 11318 raft_consensus.cc:2272] T b7cccd56fa854cbc84a922238ad4add9 P 193a40da6e364ead8ab561e14c3cc8d1 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:58.061295 11318 tablet_replica.cc:333] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:58.062029 11318 raft_consensus.cc:2243] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 LEADER]: Raft consensus shutting down.
I20260709 12:14:58.063244 11318 raft_consensus.cc:2272] T 43d4539f52e24ea8adc368a4c9529e66 P 193a40da6e364ead8ab561e14c3cc8d1 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:58.066000 11318 tablet_replica.cc:333] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:58.066717 11318 raft_consensus.cc:2243] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:58.067704 11318 raft_consensus.cc:2272] T e479222164574b9ea8944c530fbeb919 P 193a40da6e364ead8ab561e14c3cc8d1 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:58.070506 11318 tablet_replica.cc:333] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:58.071198 11318 raft_consensus.cc:2243] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:58.071790 11318 raft_consensus.cc:2272] T 10395d6e75b84c3982521323f92e3309 P 193a40da6e364ead8ab561e14c3cc8d1 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:58.074651 11318 tablet_replica.cc:333] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:58.075455 11318 raft_consensus.cc:2243] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 36 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:58.076638 11318 raft_consensus.cc:2272] T a92d1d91cce04cec98366aa6021356df P 193a40da6e364ead8ab561e14c3cc8d1 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:58.080313 11318 tablet_replica.cc:333] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:58.081146 11318 raft_consensus.cc:2243] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:58.082041 11318 raft_consensus.cc:2272] T 29bbd70aa24b4e81b3d2764c3584ad60 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:58.085289 11318 tablet_replica.cc:333] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:58.086001 11318 raft_consensus.cc:2243] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Raft consensus shutting down.
I20260709 12:14:58.086969 11318 raft_consensus.cc:2272] T 7b825dff40dc4290a25d71534b598330 P 193a40da6e364ead8ab561e14c3cc8d1 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:14:58.089706 11318 tablet_replica.cc:333] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1: stopping tablet replica
I20260709 12:14:58.090341 11318 raft_consensus.cc:2243] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 LEADER]: Raft consensus shutting down.
I20260709 12:14:58.100322 11318 raft_consensus.cc:2272] T 15456f360e78465994eb4d0d39e90b87 P 193a40da6e364ead8ab561e14c3cc8d1 [term 32 FOLLOWER]: Raft consensus is shut down!
W20260709 12:14:58.498777 13305 consensus_peers.cc:597] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 36: this message will repeat every 5th retry.
W20260709 12:14:58.506415 13305 consensus_peers.cc:597] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:14:58.549736 13305 consensus_peers.cc:597] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
I20260709 12:14:58.565992 23861 raft_consensus.cc:493] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:58.566663 23861 raft_consensus.cc:515] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:58.569203 23861 leader_election.cc:290] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:58.575729 13302 leader_election.cc:336] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
W20260709 12:14:58.589432 13305 leader_election.cc:336] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:58.590085 13305 leader_election.cc:304] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:58.591025 23861 raft_consensus.cc:2749] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:58.662456 23861 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:58.663090 23861 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
W20260709 12:14:58.677157 13302 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
W20260709 12:14:58.681555 13305 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:58.682153 13305 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
W20260709 12:14:58.688879 13304 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Network error: Client connection negotiation failed: client connection to 127.11.13.131:38613: connect: Connection refused (error 111)
I20260709 12:14:58.690358 23868 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
I20260709 12:14:58.689898 23861 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
I20260709 12:14:58.705675 23868 raft_consensus.cc:493] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:14:58.706342 23868 raft_consensus.cc:515] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:58.709275 23868 leader_election.cc:290] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:14:58.722321 13302 leader_election.cc:336] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
W20260709 12:14:58.729318 13305 leader_election.cc:336] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:58.730010 13305 leader_election.cc:304] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 34 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:58.731236 23861 raft_consensus.cc:2749] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Leader pre-election lost for term 34. Reason: could not achieve majority
W20260709 12:14:58.887858 13302 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111) [suppressed 556 similar messages]
W20260709 12:14:58.923429 13305 consensus_peers.cc:597] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 31: this message will repeat every 5th retry.
I20260709 12:14:59.072860 23861 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:59.073470 23861 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:59.076845 23861 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:59.083037 13302 leader_election.cc:336] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
W20260709 12:14:59.088335 13304 leader_election.cc:336] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Network error: Client connection negotiation failed: client connection to 127.11.13.131:38613: connect: Connection refused (error 111)
I20260709 12:14:59.088788 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:59.089509 23861 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:14:59.146185 23861 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:59.146806 23861 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:59.150738 23861 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:59.161060 13305 leader_election.cc:336] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:14:59.168956 13304 leader_election.cc:336] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Network error: Client connection negotiation failed: client connection to 127.11.13.131:38613: connect: Connection refused (error 111)
I20260709 12:14:59.169476 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:59.170312 23861 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:14:59.180275 23861 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:59.180940 23861 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:59.184796 23861 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:59.189364 13302 leader_election.cc:336] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
W20260709 12:14:59.212479 13304 leader_election.cc:336] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Network error: Client connection negotiation failed: client connection to 127.11.13.131:38613: connect: Connection refused (error 111)
I20260709 12:14:59.213006 13304 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:14:59.213771 23861 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
W20260709 12:14:59.360008 13305 consensus_peers.cc:597] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:14:59.396631 13305 consensus_peers.cc:597] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
W20260709 12:14:59.428236 13302 consensus_peers.cc:597] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:59.441283 13302 consensus_peers.cc:597] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:59.507666 13304 consensus_peers.cc:597] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 -> Peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Couldn't send request to peer 193a40da6e364ead8ab561e14c3cc8d1. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.131:38613: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:14:59.530951 23861 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:14:59.531606 23861 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:14:59.534037 23861 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:14:59.542572 13304 leader_election.cc:336] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Network error: Client connection negotiation failed: client connection to 127.11.13.131:38613: connect: Connection refused (error 111)
W20260709 12:14:59.548491 13305 leader_election.cc:336] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:14:59.548976 13305 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:14:59.551052 23861 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:14:59.560211 13302 consensus_peers.cc:597] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:59.593140 13304 consensus_peers.cc:597] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 -> Peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Couldn't send request to peer 193a40da6e364ead8ab561e14c3cc8d1. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.131:38613: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:14:59.636363 13302 consensus_peers.cc:597] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:59.644673 13302 consensus_peers.cc:597] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:59.767961 13302 consensus_peers.cc:597] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 16: this message will repeat every 5th retry.
W20260709 12:14:59.800717 13304 consensus_peers.cc:597] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 -> Peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Couldn't send request to peer 193a40da6e364ead8ab561e14c3cc8d1. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.131:38613: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:15:00.542143 23876 raft_consensus.cc:493] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:15:00.542855 23876 raft_consensus.cc:515] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:15:00.549137 23876 leader_election.cc:290] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365)
W20260709 12:15:00.563910 13304 consensus_peers.cc:597] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 -> Peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Couldn't send request to peer 193a40da6e364ead8ab561e14c3cc8d1. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.131:38613: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:15:00.564332 13305 consensus_peers.cc:597] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 41: this message will repeat every 5th retry.
W20260709 12:15:00.564543 13304 consensus_peers.cc:597] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 -> Peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Couldn't send request to peer 193a40da6e364ead8ab561e14c3cc8d1. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.131:38613: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:15:00.565038 13304 consensus_peers.cc:597] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 -> Peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Couldn't send request to peer 193a40da6e364ead8ab561e14c3cc8d1. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.131:38613: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:15:00.565531 13304 consensus_peers.cc:597] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 -> Peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Couldn't send request to peer 193a40da6e364ead8ab561e14c3cc8d1. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.131:38613: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:15:00.566044 13304 consensus_peers.cc:597] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 -> Peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Couldn't send request to peer 193a40da6e364ead8ab561e14c3cc8d1. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.131:38613: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:15:00.566521 13304 consensus_peers.cc:597] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 -> Peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Couldn't send request to peer 193a40da6e364ead8ab561e14c3cc8d1. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.131:38613: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:15:00.567432 13305 leader_election.cc:336] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:15:00.572489 13302 leader_election.cc:336] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:15:00.572912 13302 leader_election.cc:304] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 36 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: a2fdbe7eef6748a9b4611f6c9bc9b201, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:15:00.573597 23876 raft_consensus.cc:2749] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Leader pre-election lost for term 36. Reason: could not achieve majority
I20260709 12:15:00.664892 23876 raft_consensus.cc:493] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election (detected failure of leader f5fc807f7ecc4d1ca0fd6be524f734ce)
I20260709 12:15:00.665509 23876 raft_consensus.cc:515] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:15:00.669492 23876 leader_election.cc:290] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:15:00.675176 13305 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:15:00.683267 13304 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Network error: Client connection negotiation failed: client connection to 127.11.13.131:38613: connect: Connection refused (error 111)
I20260709 12:15:00.683763 13304 leader_election.cc:304] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:15:00.684882 23876 raft_consensus.cc:2749] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Leader pre-election lost for term 27. Reason: could not achieve majority
W20260709 12:15:00.690174 13302 leader_election.cc:336] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 27 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:15:01.019500 23876 raft_consensus.cc:493] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:15:01.020105 23876 raft_consensus.cc:515] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:15:01.024071 23876 leader_election.cc:290] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:15:01.036674 13302 leader_election.cc:336] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
W20260709 12:15:01.044623 13304 leader_election.cc:336] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Network error: Client connection negotiation failed: client connection to 127.11.13.131:38613: connect: Connection refused (error 111)
I20260709 12:15:01.045100 13304 leader_election.cc:304] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:15:01.046381 23880 raft_consensus.cc:2749] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
W20260709 12:15:01.047819 13302 consensus_peers.cc:597] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 -> Peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Couldn't send request to peer f5fc807f7ecc4d1ca0fd6be524f734ce. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:15:01.073855 13304 consensus_peers.cc:597] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 -> Peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Couldn't send request to peer 193a40da6e364ead8ab561e14c3cc8d1. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.131:38613: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:15:01.078470 23880 raft_consensus.cc:493] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:15:01.079222 23880 raft_consensus.cc:515] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:15:01.081566 23880 leader_election.cc:290] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:15:01.086426 13305 leader_election.cc:336] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
W20260709 12:15:01.095151 13304 leader_election.cc:336] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Network error: Client connection negotiation failed: client connection to 127.11.13.131:38613: connect: Connection refused (error 111)
I20260709 12:15:01.095769 13304 leader_election.cc:304] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 33 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
I20260709 12:15:01.096936 23880 raft_consensus.cc:2749] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Leader pre-election lost for term 33. Reason: could not achieve majority
I20260709 12:15:01.116858 23880 raft_consensus.cc:493] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:15:01.117617 23880 raft_consensus.cc:515] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:15:01.120158 23880 leader_election.cc:290] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 pre-election: Requested pre-vote from peers f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:15:01.125603 13302 leader_election.cc:336] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
W20260709 12:15:01.128821 13304 leader_election.cc:336] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Network error: Client connection negotiation failed: client connection to 127.11.13.131:38613: connect: Connection refused (error 111)
I20260709 12:15:01.129398 13304 leader_election.cc:304] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 29 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, f5fc807f7ecc4d1ca0fd6be524f734ce
I20260709 12:15:01.130277 23880 raft_consensus.cc:2749] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Leader pre-election lost for term 29. Reason: could not achieve majority
I20260709 12:15:01.178004 11318 tablet_server.cc:196] TabletServer@127.11.13.131:0 shutdown complete.
I20260709 12:15:01.373852 11318 tablet_server.cc:179] TabletServer@127.11.13.132:0 shutting down...
I20260709 12:15:01.425096 23880 raft_consensus.cc:493] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election (detected failure of leader 193a40da6e364ead8ab561e14c3cc8d1)
I20260709 12:15:01.426803 23880 raft_consensus.cc:515] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: false } }
I20260709 12:15:01.443117 23880 leader_election.cc:290] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Requested pre-vote from peers a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811), 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613)
W20260709 12:15:01.446043 13304 leader_election.cc:336] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer 193a40da6e364ead8ab561e14c3cc8d1 (127.11.13.131:38613): Network error: Client connection negotiation failed: client connection to 127.11.13.131:38613: connect: Connection refused (error 111)
W20260709 12:15:01.451571 13305 leader_election.cc:336] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: RPC error from VoteRequest() call to peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111)
I20260709 12:15:01.452193 13305 leader_election.cc:304] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [CANDIDATE]: Term 30 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e56d487365394ac5ac5f1002926a5645; no voters: 193a40da6e364ead8ab561e14c3cc8d1, a2fdbe7eef6748a9b4611f6c9bc9b201
W20260709 12:15:01.453459 13305 consensus_peers.cc:597] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 51: this message will repeat every 5th retry.
I20260709 12:15:01.454955 23880 raft_consensus.cc:2749] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Leader pre-election lost for term 30. Reason: could not achieve majority
I20260709 12:15:01.465286 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:15:01.467195 11318 tablet_replica.cc:333] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
I20260709 12:15:01.472519 11318 raft_consensus.cc:2243] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:01.474047 11318 raft_consensus.cc:2272] T 70b8c9b89e3d43fbb70422110d76702c P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:01.480994 11318 tablet_replica.cc:333] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
I20260709 12:15:01.482484 11318 raft_consensus.cc:2243] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:01.484419 11318 raft_consensus.cc:2272] T 816b6f34d124434882148e507cf5b612 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Raft consensus is shut down!
W20260709 12:15:01.485667 13360 debug-util.cc:398] Leaking SignalData structure 0x7b080020cf40 after lost signal to thread 11321
W20260709 12:15:01.488245 13360 debug-util.cc:398] Leaking SignalData structure 0x7b0800365180 after lost signal to thread 12463
I20260709 12:15:01.489485 11318 tablet_replica.cc:333] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
W20260709 12:15:01.491127 13360 debug-util.cc:398] Leaking SignalData structure 0x7b08002e0420 after lost signal to thread 13363
I20260709 12:15:01.491134 11318 raft_consensus.cc:2243] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 28 LEADER]: Raft consensus shutting down.
I20260709 12:15:01.498847 11318 raft_consensus.cc:2272] T 952cdab9198b44e4bdf1b472109715c3 P e56d487365394ac5ac5f1002926a5645 [term 28 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:01.504702 11318 tablet_replica.cc:333] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
I20260709 12:15:01.506522 11318 raft_consensus.cc:2243] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 36 LEADER]: Raft consensus shutting down.
I20260709 12:15:01.508716 11318 raft_consensus.cc:2272] T ee47f7c4414b4ae98233a31696e99cc0 P e56d487365394ac5ac5f1002926a5645 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:01.515733 11318 tablet_replica.cc:333] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
I20260709 12:15:01.518384 11318 raft_consensus.cc:2243] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 33 LEADER]: Raft consensus shutting down.
I20260709 12:15:01.523145 11318 raft_consensus.cc:2272] T 5dd1b2c01b5e495f9f31aee68f93d394 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:01.527992 11318 tablet_replica.cc:333] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
I20260709 12:15:01.530480 11318 raft_consensus.cc:2243] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:01.532150 11318 raft_consensus.cc:2272] T cdb0bb14b2b745b398e14d8c1a7139a2 P e56d487365394ac5ac5f1002926a5645 [term 33 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:01.538151 11318 tablet_replica.cc:333] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
I20260709 12:15:01.540278 11318 raft_consensus.cc:2243] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:15:01.543157 11318 raft_consensus.cc:2272] T 3c00651839ae4132bb331bc6c2602641 P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Raft consensus is shut down!
W20260709 12:15:01.543653 13305 consensus_peers.cc:597] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 -> Peer a2fdbe7eef6748a9b4611f6c9bc9b201 (127.11.13.129:38811): Couldn't send request to peer a2fdbe7eef6748a9b4611f6c9bc9b201. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:38811: connect: Connection refused (error 111). This is attempt 46: this message will repeat every 5th retry.
I20260709 12:15:01.548477 11318 tablet_replica.cc:333] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
I20260709 12:15:01.549898 11318 raft_consensus.cc:2243] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:01.551184 11318 raft_consensus.cc:2272] T 43d4539f52e24ea8adc368a4c9529e66 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:01.556506 11318 tablet_replica.cc:333] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
I20260709 12:15:01.558434 11318 raft_consensus.cc:2243] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:01.559851 11318 raft_consensus.cc:2272] T dd783245c5154ded95c4fe8434d0b380 P e56d487365394ac5ac5f1002926a5645 [term 29 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:01.564543 11318 tablet_replica.cc:333] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
I20260709 12:15:01.565594 11318 raft_consensus.cc:2243] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 27 LEADER]: Raft consensus shutting down.
I20260709 12:15:01.568539 11318 raft_consensus.cc:2272] T 6fca0ed05cd34c8e92dc950351b864a2 P e56d487365394ac5ac5f1002926a5645 [term 27 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:01.573408 11318 tablet_replica.cc:333] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
I20260709 12:15:01.574556 11318 raft_consensus.cc:2243] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 31 LEADER]: Raft consensus shutting down.
I20260709 12:15:01.576979 11318 raft_consensus.cc:2272] T ee30d7cd60b04485b7e19c8d119d67bd P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:01.581506 11318 tablet_replica.cc:333] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
I20260709 12:15:01.582854 11318 raft_consensus.cc:2243] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 36 LEADER]: Raft consensus shutting down.
I20260709 12:15:01.585162 11318 raft_consensus.cc:2272] T a92d1d91cce04cec98366aa6021356df P e56d487365394ac5ac5f1002926a5645 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:01.589407 11318 tablet_replica.cc:333] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
I20260709 12:15:01.590653 11318 raft_consensus.cc:2243] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 26 LEADER]: Raft consensus shutting down.
I20260709 12:15:01.592989 11318 raft_consensus.cc:2272] T 8c7e0eddcc2b4449b7ed8821854646e2 P e56d487365394ac5ac5f1002926a5645 [term 26 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:01.597347 11318 tablet_replica.cc:333] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
I20260709 12:15:01.598486 11318 raft_consensus.cc:2243] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 25 LEADER]: Raft consensus shutting down.
I20260709 12:15:01.600402 11318 raft_consensus.cc:2272] T e15d21db797e4a96b5732711a5379980 P e56d487365394ac5ac5f1002926a5645 [term 25 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:01.606628 11318 tablet_replica.cc:333] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
I20260709 12:15:01.607893 11318 raft_consensus.cc:2243] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645 [term 4 LEARNER]: Raft consensus shutting down.
I20260709 12:15:01.608767 11318 pending_rounds.cc:70] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645: Trying to abort 1 pending ops.
I20260709 12:15:01.609073 11318 pending_rounds.cc:77] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645: Aborting op as it isn't in flight: id { term: 4 index: 4 } timestamp: 7305621573063065600 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "3c9b5315396948b2af3fa5480c9bee87" old_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } } } new_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a2fdbe7eef6748a9b4611f6c9bc9b201" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 38811 } } peers { permanent_uuid: "f5fc807f7ecc4d1ca0fd6be524f734ce" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 39365 } } peers { permanent_uuid: "193a40da6e364ead8ab561e14c3cc8d1" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38613 } attrs { replace: true } } peers { permanent_uuid: "e56d487365394ac5ac5f1002926a5645" member_type: NON_VOTER last_known_addr { host: "127.11.13.132" port: 45801 } attrs { promote: true } } } }
I20260709 12:15:01.611335 11318 raft_consensus.cc:2917] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645 [term 4 LEARNER]: Aborting config change with OpId 4.4: Aborted: Op aborted
I20260709 12:15:01.612403 11318 raft_consensus.cc:2889] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645 [term 4 NON_PARTICIPANT]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260709 12:15:01.612838 11318 raft_consensus.cc:2272] T 3c9b5315396948b2af3fa5480c9bee87 P e56d487365394ac5ac5f1002926a5645 [term 4 NON_PARTICIPANT]: Raft consensus is shut down!
I20260709 12:15:01.617929 11318 tablet_replica.cc:333] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
I20260709 12:15:01.619376 11318 raft_consensus.cc:2243] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 30 LEADER]: Raft consensus shutting down.
I20260709 12:15:01.621498 11318 raft_consensus.cc:2272] T e1ff01ff4ede4fc990b9a07b23102b0d P e56d487365394ac5ac5f1002926a5645 [term 30 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:01.626107 11318 tablet_replica.cc:333] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
I20260709 12:15:01.627319 11318 raft_consensus.cc:2243] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 36 LEADER]: Raft consensus shutting down.
I20260709 12:15:01.629216 11318 raft_consensus.cc:2272] T 4270d5e0462b49d2982145525764c7fa P e56d487365394ac5ac5f1002926a5645 [term 36 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:01.634418 11318 tablet_replica.cc:333] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
I20260709 12:15:01.635496 11318 raft_consensus.cc:2243] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:01.637084 11318 raft_consensus.cc:2272] T 618bbf28983a42beaaf337df12784ee1 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:01.641377 11318 tablet_replica.cc:333] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
I20260709 12:15:01.642375 11318 raft_consensus.cc:2243] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:01.643555 11318 raft_consensus.cc:2272] T 33a0b4d6cfba4ab2b3d54f591439b179 P e56d487365394ac5ac5f1002926a5645 [term 32 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:01.647634 11318 tablet_replica.cc:333] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
I20260709 12:15:01.648577 11318 raft_consensus.cc:2243] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:01.649772 11318 raft_consensus.cc:2272] T d7bd864b75c34027b05582a55b152877 P e56d487365394ac5ac5f1002926a5645 [term 31 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:01.653587 11318 tablet_replica.cc:333] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
I20260709 12:15:01.654534 11318 raft_consensus.cc:2243] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 35 LEADER]: Raft consensus shutting down.
I20260709 12:15:01.655973 11318 raft_consensus.cc:2272] T b7cccd56fa854cbc84a922238ad4add9 P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:01.659777 11318 tablet_replica.cc:333] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645: stopping tablet replica
I20260709 12:15:01.660676 11318 raft_consensus.cc:2243] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Raft consensus shutting down.
I20260709 12:15:01.662367 11318 raft_consensus.cc:2272] T e40144c1c60847ee81b8252b107df67d P e56d487365394ac5ac5f1002926a5645 [term 35 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:02.404685 11318 tablet_server.cc:196] TabletServer@127.11.13.132:0 shutdown complete.
I20260709 12:15:02.473834 11318 master.cc:562] Master@127.11.13.190:46629 shutting down...
W20260709 12:15:02.524108 12477 auto_rebalancer.cc:873] Removing replace marker failed: Catalog manager is not initialized. State: Closing
W20260709 12:15:02.524540 12477 auto_rebalancer.cc:901] Could not move replica: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:15:02.524829 12477 auto_rebalancer.cc:330] scheduled replica move failed to complete: Service unavailable: Catalog manager is not initialized. State: Closing
W20260709 12:15:28.013841 12460 debug-util.cc:398] Leaking SignalData structure 0x7b0800242a40 after lost signal to thread 11321
W20260709 12:15:28.014560 12460 debug-util.cc:398] Leaking SignalData structure 0x7b08003b3f20 after lost signal to thread 12463
W20260709 12:15:44.115217 12413 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111) [suppressed 136 similar messages]
W20260709 12:15:44.119683 12413 catalog_manager.cc:4755] TS f5fc807f7ecc4d1ca0fd6be524f734ce (127.11.13.130:39365): ChangeConfig:REMOVE_PEER RPC failed for tablet 70b8c9b89e3d43fbb70422110d76702c: Network error: Client connection negotiation failed: client connection to 127.11.13.130:39365: connect: Connection refused (error 111)
I20260709 12:15:54.003278 11318 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:15:54.005102 11318 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:15:54.005697 11318 tablet_replica.cc:333] T 00000000000000000000000000000000 P d5d8248387cf49cfbd3416fcacedae86: stopping tablet replica
I20260709 12:15:54.324491 11318 master.cc:584] Master@127.11.13.190:46629 shutdown complete.
I20260709 12:15:54.391652 11318 test_util.cc:182] -----------------------------------------------
I20260709 12:15:54.391952 11318 test_util.cc:183] Had failures, leaving test files at /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1783598892056905-11318-0
[  FAILED  ] LeaderRebalancerTest.AddTserver (399553 ms)
[ RUN      ] LeaderRebalancerTest.RestartTserver
I20260709 12:15:54.396106 11318 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.11.13.190:46705
I20260709 12:15:54.397449 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:54.404431 23891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:54.405105 23894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:54.404479 23892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:54.405970 11318 server_base.cc:1061] running on GCE node
I20260709 12:15:54.411255 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:54.411489 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:54.411679 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599354411661 us; error 0 us; skew 500 ppm
I20260709 12:15:54.412312 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:54.415175 11318 webserver.cc:533] Webserver started at http://127.11.13.190:36591/ using document root <none> and password file <none>
I20260709 12:15:54.415733 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:54.415900 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:54.416239 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:54.417662 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/master-0-root/instance:
uuid: "49fa0f839a734d2b834f455413dda82b"
format_stamp: "Formatted at 2026-07-09 12:15:54 on dist-test-slave-zqnl"
I20260709 12:15:54.425408 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.005s	sys 0.001s
I20260709 12:15:54.430843 23899 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:54.431897 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.004s
I20260709 12:15:54.432385 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/master-0-root
uuid: "49fa0f839a734d2b834f455413dda82b"
format_stamp: "Formatted at 2026-07-09 12:15:54 on dist-test-slave-zqnl"
I20260709 12:15:54.432736 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:54.460704 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:54.462469 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:54.508455 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.190:46705
I20260709 12:15:54.508592 23950 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.190:46705 every 8 connection(s)
I20260709 12:15:54.513129 23951 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:54.527573 23951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b: Bootstrap starting.
I20260709 12:15:54.533263 23951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:54.538708 23951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b: No bootstrap required, opened a new log
I20260709 12:15:54.541544 23951 raft_consensus.cc:359] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49fa0f839a734d2b834f455413dda82b" member_type: VOTER }
I20260709 12:15:54.542068 23951 raft_consensus.cc:385] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:54.542445 23951 raft_consensus.cc:740] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49fa0f839a734d2b834f455413dda82b, State: Initialized, Role: FOLLOWER
I20260709 12:15:54.543135 23951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49fa0f839a734d2b834f455413dda82b" member_type: VOTER }
I20260709 12:15:54.543663 23951 raft_consensus.cc:399] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:15:54.543889 23951 raft_consensus.cc:493] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:15:54.544188 23951 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:54.551023 23951 raft_consensus.cc:515] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49fa0f839a734d2b834f455413dda82b" member_type: VOTER }
I20260709 12:15:54.551687 23951 leader_election.cc:304] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49fa0f839a734d2b834f455413dda82b; no voters: 
I20260709 12:15:54.552991 23951 leader_election.cc:290] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:15:54.553386 23954 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:54.555387 23954 raft_consensus.cc:697] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b [term 1 LEADER]: Becoming Leader. State: Replica: 49fa0f839a734d2b834f455413dda82b, State: Running, Role: LEADER
I20260709 12:15:54.556257 23954 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49fa0f839a734d2b834f455413dda82b" member_type: VOTER }
I20260709 12:15:54.556890 23951 sys_catalog.cc:565] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:15:54.568871 23955 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49fa0f839a734d2b834f455413dda82b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49fa0f839a734d2b834f455413dda82b" member_type: VOTER } }
I20260709 12:15:54.568706 23956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49fa0f839a734d2b834f455413dda82b. Latest consensus state: current_term: 1 leader_uuid: "49fa0f839a734d2b834f455413dda82b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49fa0f839a734d2b834f455413dda82b" member_type: VOTER } }
I20260709 12:15:54.569885 23956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b [sys.catalog]: This master's current role is: LEADER
I20260709 12:15:54.569649 23955 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b [sys.catalog]: This master's current role is: LEADER
I20260709 12:15:54.572373 23964 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:15:54.578765 23964 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:15:54.580029 11318 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:15:54.589352 23964 catalog_manager.cc:1383] Generated new cluster ID: 67d3f775a0b2432dbe2dde5079590c77
I20260709 12:15:54.589745 23964 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:15:54.609203 23964 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:15:54.610843 23964 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:15:54.626873 23964 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b: Generated new TSK 0
I20260709 12:15:54.627570 23964 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:15:54.647905 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:54.655712 23972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:54.656888 23973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:54.659019 23975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:54.661414 11318 server_base.cc:1061] running on GCE node
I20260709 12:15:54.664726 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:54.665184 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:54.665344 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599354665330 us; error 0 us; skew 500 ppm
I20260709 12:15:54.665953 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:54.668684 11318 webserver.cc:533] Webserver started at http://127.11.13.129:36291/ using document root <none> and password file <none>
I20260709 12:15:54.669142 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:54.669297 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:54.669668 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:54.671068 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-0-root/instance:
uuid: "c359614416104a7ba9db1ede97d13802"
format_stamp: "Formatted at 2026-07-09 12:15:54 on dist-test-slave-zqnl"
I20260709 12:15:54.676002 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:15:54.679690 23980 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:54.681717 11318 fs_manager.cc:730] Time spent opening block manager: real 0.004s	user 0.004s	sys 0.000s
I20260709 12:15:54.682070 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-0-root
uuid: "c359614416104a7ba9db1ede97d13802"
format_stamp: "Formatted at 2026-07-09 12:15:54 on dist-test-slave-zqnl"
I20260709 12:15:54.682479 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:54.698675 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:54.700070 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:54.701758 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:54.704752 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:54.705046 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:54.705358 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:54.705574 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:54.755656 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.129:42669
I20260709 12:15:54.755795 24042 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.129:42669 every 8 connection(s)
I20260709 12:15:54.760897 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:54.771075 24047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:54.772616 24048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:54.777070 11318 server_base.cc:1061] running on GCE node
W20260709 12:15:54.777735 24050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:54.779379 24043 heartbeater.cc:344] Connected to a master server at 127.11.13.190:46705
I20260709 12:15:54.779810 24043 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:54.780826 24043 heartbeater.cc:507] Master 127.11.13.190:46705 requested a full tablet report, sending...
I20260709 12:15:54.781213 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:54.781497 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:54.781697 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599354781661 us; error 0 us; skew 500 ppm
I20260709 12:15:54.782473 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:54.783516 23916 ts_manager.cc:194] Registered new tserver with Master: c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:15:54.785431 11318 webserver.cc:533] Webserver started at http://127.11.13.130:42009/ using document root <none> and password file <none>
I20260709 12:15:54.786020 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:54.785928 23916 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34188
I20260709 12:15:54.786430 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:54.786826 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:54.788270 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-1-root/instance:
uuid: "6c74a5203b594708921cd793aaca0b8b"
format_stamp: "Formatted at 2026-07-09 12:15:54 on dist-test-slave-zqnl"
I20260709 12:15:54.793195 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.002s	sys 0.004s
I20260709 12:15:54.798214 24055 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:54.799383 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:15:54.799775 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-1-root
uuid: "6c74a5203b594708921cd793aaca0b8b"
format_stamp: "Formatted at 2026-07-09 12:15:54 on dist-test-slave-zqnl"
I20260709 12:15:54.800132 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:54.819558 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:54.820976 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:54.822744 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:54.825662 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:54.825973 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:54.826282 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:54.826474 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:54.875798 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.130:42247
I20260709 12:15:54.875952 24117 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.130:42247 every 8 connection(s)
I20260709 12:15:54.881443 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:54.891465 24122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:54.892973 24121 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:54.893278 24124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:54.895597 11318 server_base.cc:1061] running on GCE node
I20260709 12:15:54.896925 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:15:54.897291 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:54.897526 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599354897482 us; error 0 us; skew 500 ppm
I20260709 12:15:54.898211 24118 heartbeater.cc:344] Connected to a master server at 127.11.13.190:46705
I20260709 12:15:54.898327 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:54.898753 24118 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:54.899672 24118 heartbeater.cc:507] Master 127.11.13.190:46705 requested a full tablet report, sending...
I20260709 12:15:54.901793 11318 webserver.cc:533] Webserver started at http://127.11.13.131:33843/ using document root <none> and password file <none>
I20260709 12:15:54.902283 23916 ts_manager.cc:194] Registered new tserver with Master: 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:54.902562 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:54.902900 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:54.903283 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:54.904352 23916 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34200
I20260709 12:15:54.905110 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-2-root/instance:
uuid: "008a79e516614c809c2d3afdb5cbc327"
format_stamp: "Formatted at 2026-07-09 12:15:54 on dist-test-slave-zqnl"
I20260709 12:15:54.910696 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.003s	sys 0.004s
I20260709 12:15:54.914536 24129 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:54.915482 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:15:54.915822 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-2-root
uuid: "008a79e516614c809c2d3afdb5cbc327"
format_stamp: "Formatted at 2026-07-09 12:15:54 on dist-test-slave-zqnl"
I20260709 12:15:54.916150 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:54.928136 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:54.930231 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:54.932200 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:54.935045 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:54.935360 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:54.935619 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:54.935750 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:54.986992 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.131:36699
I20260709 12:15:54.987123 24191 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.131:36699 every 8 connection(s)
I20260709 12:15:54.991873 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:15:54.999683 24195 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:15:55.003297 24196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:55.007542 11318 server_base.cc:1061] running on GCE node
W20260709 12:15:55.007654 24198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:15:55.008252 24192 heartbeater.cc:344] Connected to a master server at 127.11.13.190:46705
I20260709 12:15:55.008924 24192 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:55.009512 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:15:55.009862 24192 heartbeater.cc:507] Master 127.11.13.190:46705 requested a full tablet report, sending...
W20260709 12:15:55.010061 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:15:55.010469 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599355010439 us; error 0 us; skew 500 ppm
I20260709 12:15:55.011294 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:15:55.012678 23916 ts_manager.cc:194] Registered new tserver with Master: 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:15:55.014497 23916 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34210
I20260709 12:15:55.015173 11318 webserver.cc:533] Webserver started at http://127.11.13.132:35905/ using document root <none> and password file <none>
I20260709 12:15:55.016028 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:15:55.016295 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:15:55.016670 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:15:55.018894 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-3-root/instance:
uuid: "a0df0eff47b64e3eb45ab074139ed187"
format_stamp: "Formatted at 2026-07-09 12:15:55 on dist-test-slave-zqnl"
I20260709 12:15:55.024236 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:15:55.028525 24203 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:55.029801 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:15:55.030397 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-3-root
uuid: "a0df0eff47b64e3eb45ab074139ed187"
format_stamp: "Formatted at 2026-07-09 12:15:55 on dist-test-slave-zqnl"
I20260709 12:15:55.030794 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:15:55.077878 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:15:55.079738 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:15:55.081370 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:15:55.083879 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:15:55.084151 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:55.084457 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:15:55.084656 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:15:55.133371 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.132:41011
I20260709 12:15:55.133468 24265 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.132:41011 every 8 connection(s)
I20260709 12:15:55.152256 24266 heartbeater.cc:344] Connected to a master server at 127.11.13.190:46705
I20260709 12:15:55.152776 24266 heartbeater.cc:461] Registering TS with master...
I20260709 12:15:55.153635 24266 heartbeater.cc:507] Master 127.11.13.190:46705 requested a full tablet report, sending...
I20260709 12:15:55.156008 23916 ts_manager.cc:194] Registered new tserver with Master: a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:15:55.156184 11318 internal_mini_cluster.cc:371] 4 TS(s) registered with all masters after 0.019137412s
I20260709 12:15:55.158236 23916 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:34214
I20260709 12:15:55.192059 23916 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34230:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
I20260709 12:15:55.465680 24230 tablet_service.cc:1511] Processing CreateTablet for tablet bc53eac4ce984dbaa561d9b719143d18 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:15:55.467483 24230 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc53eac4ce984dbaa561d9b719143d18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.469971 24229 tablet_service.cc:1511] Processing CreateTablet for tablet ceb6c32822cf44158ca1ea0d62e0b001 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:15:55.471940 24229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ceb6c32822cf44158ca1ea0d62e0b001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.474756 24231 tablet_service.cc:1511] Processing CreateTablet for tablet 659e684fa2a0433faf027bf99c82dd6b (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:15:55.478075 24228 tablet_service.cc:1511] Processing CreateTablet for tablet 1df16f5650964ece90109902b6b40882 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:15:55.476558 24231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 659e684fa2a0433faf027bf99c82dd6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.484285 24228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1df16f5650964ece90109902b6b40882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.488266 24227 tablet_service.cc:1511] Processing CreateTablet for tablet e679d0d8ffd84a23afc6d7f8faf84ca6 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:15:55.489566 24004 tablet_service.cc:1511] Processing CreateTablet for tablet 1df16f5650964ece90109902b6b40882 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:15:55.489694 24003 tablet_service.cc:1511] Processing CreateTablet for tablet e679d0d8ffd84a23afc6d7f8faf84ca6 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:15:55.489969 24227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e679d0d8ffd84a23afc6d7f8faf84ca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.492036 24007 tablet_service.cc:1511] Processing CreateTablet for tablet bc53eac4ce984dbaa561d9b719143d18 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:15:55.494544 24006 tablet_service.cc:1511] Processing CreateTablet for tablet ceb6c32822cf44158ca1ea0d62e0b001 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:15:55.495186 24226 tablet_service.cc:1511] Processing CreateTablet for tablet fbb90520a319469fa1664276964749ea (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:15:55.496335 24002 tablet_service.cc:1511] Processing CreateTablet for tablet 5587d307c2f64ab9a9b39c1b190e85be (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:15:55.497079 24226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb90520a319469fa1664276964749ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.496919 24008 tablet_service.cc:1511] Processing CreateTablet for tablet 11a4ed5e3c5d47f189b17261d317eb7b (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:15:55.498459 24004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1df16f5650964ece90109902b6b40882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.499935 24083 tablet_service.cc:1511] Processing CreateTablet for tablet 659e684fa2a0433faf027bf99c82dd6b (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:15:55.501509 24083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 659e684fa2a0433faf027bf99c82dd6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.502089 24079 tablet_service.cc:1511] Processing CreateTablet for tablet 1df16f5650964ece90109902b6b40882 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260709 12:15:55.488268 24005 tablet_service.cc:1511] Processing CreateTablet for tablet cf791a63ce7546f38a312c7f77142a61 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:15:55.503223 24003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e679d0d8ffd84a23afc6d7f8faf84ca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.503677 24079 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1df16f5650964ece90109902b6b40882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.504163 24005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf791a63ce7546f38a312c7f77142a61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.505199 24002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5587d307c2f64ab9a9b39c1b190e85be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.504982 24081 tablet_service.cc:1511] Processing CreateTablet for tablet bc53eac4ce984dbaa561d9b719143d18 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260709 12:15:55.507489 24081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc53eac4ce984dbaa561d9b719143d18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.508337 24078 tablet_service.cc:1511] Processing CreateTablet for tablet fbb90520a319469fa1664276964749ea (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:15:55.503669 24082 tablet_service.cc:1511] Processing CreateTablet for tablet 11a4ed5e3c5d47f189b17261d317eb7b (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:15:55.510617 24080 tablet_service.cc:1511] Processing CreateTablet for tablet cf791a63ce7546f38a312c7f77142a61 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:15:55.511503 24008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11a4ed5e3c5d47f189b17261d317eb7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.512326 24007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bc53eac4ce984dbaa561d9b719143d18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.513242 24077 tablet_service.cc:1511] Processing CreateTablet for tablet 5587d307c2f64ab9a9b39c1b190e85be (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:15:55.514600 24006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ceb6c32822cf44158ca1ea0d62e0b001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.518411 24080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf791a63ce7546f38a312c7f77142a61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.519259 24078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb90520a319469fa1664276964749ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.520417 24077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5587d307c2f64ab9a9b39c1b190e85be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.521888 24082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11a4ed5e3c5d47f189b17261d317eb7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.545996 24225 tablet_service.cc:1511] Processing CreateTablet for tablet 55819e09f3e245179f0dd80b9e60eb6a (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:15:55.547797 24225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55819e09f3e245179f0dd80b9e60eb6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.558921 24001 tablet_service.cc:1511] Processing CreateTablet for tablet 55819e09f3e245179f0dd80b9e60eb6a (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:15:55.560717 24001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55819e09f3e245179f0dd80b9e60eb6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.565109 24076 tablet_service.cc:1511] Processing CreateTablet for tablet 55819e09f3e245179f0dd80b9e60eb6a (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260709 12:15:55.566776 24076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 55819e09f3e245179f0dd80b9e60eb6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.568698 24224 tablet_service.cc:1511] Processing CreateTablet for tablet 7e815126e7554106a097f95846d7f663 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:15:55.570474 24224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e815126e7554106a097f95846d7f663. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:15:55.571445 23963 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:15:55.583168 24000 tablet_service.cc:1511] Processing CreateTablet for tablet 7e815126e7554106a097f95846d7f663 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:15:55.584888 24000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e815126e7554106a097f95846d7f663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.591842 24075 tablet_service.cc:1511] Processing CreateTablet for tablet cd2d7fad08bc4376bb38cf21a6a9b611 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:15:55.593535 24075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd2d7fad08bc4376bb38cf21a6a9b611. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.607414 24157 tablet_service.cc:1511] Processing CreateTablet for tablet 659e684fa2a0433faf027bf99c82dd6b (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260709 12:15:55.607740 24155 tablet_service.cc:1511] Processing CreateTablet for tablet ceb6c32822cf44158ca1ea0d62e0b001 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260709 12:15:55.608952 24157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 659e684fa2a0433faf027bf99c82dd6b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.611353 24153 tablet_service.cc:1511] Processing CreateTablet for tablet e679d0d8ffd84a23afc6d7f8faf84ca6 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260709 12:15:55.612936 24153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e679d0d8ffd84a23afc6d7f8faf84ca6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.615561 24152 tablet_service.cc:1511] Processing CreateTablet for tablet fbb90520a319469fa1664276964749ea (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260709 12:15:55.617158 24152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fbb90520a319469fa1664276964749ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.619504 24151 tablet_service.cc:1511] Processing CreateTablet for tablet 5587d307c2f64ab9a9b39c1b190e85be (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260709 12:15:55.621162 24151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 5587d307c2f64ab9a9b39c1b190e85be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.623615 24150 tablet_service.cc:1511] Processing CreateTablet for tablet 7e815126e7554106a097f95846d7f663 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260709 12:15:55.625249 24150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e815126e7554106a097f95846d7f663. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.627656 24155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ceb6c32822cf44158ca1ea0d62e0b001. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.629215 24149 tablet_service.cc:1511] Processing CreateTablet for tablet cd2d7fad08bc4376bb38cf21a6a9b611 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:15:55.630246 24154 tablet_service.cc:1511] Processing CreateTablet for tablet cf791a63ce7546f38a312c7f77142a61 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260709 12:15:55.631183 24149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd2d7fad08bc4376bb38cf21a6a9b611. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.635152 24154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf791a63ce7546f38a312c7f77142a61. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.604707 24156 tablet_service.cc:1511] Processing CreateTablet for tablet 11a4ed5e3c5d47f189b17261d317eb7b (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260709 12:15:55.647815 24156 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11a4ed5e3c5d47f189b17261d317eb7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.685508 24223 tablet_service.cc:1511] Processing CreateTablet for tablet cd2d7fad08bc4376bb38cf21a6a9b611 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260709 12:15:55.686141 24007 tablet_service.cc:1511] Processing CreateTablet for tablet 0015560e665c48f0bd17fe4be5c8761d (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:15:55.688045 24223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cd2d7fad08bc4376bb38cf21a6a9b611. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.688433 24007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0015560e665c48f0bd17fe4be5c8761d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.691128 24285 tablet_bootstrap.cc:492] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:15:55.726609 24148 tablet_service.cc:1511] Processing CreateTablet for tablet 0015560e665c48f0bd17fe4be5c8761d (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:15:55.728216 24148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0015560e665c48f0bd17fe4be5c8761d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.732318 24222 tablet_service.cc:1511] Processing CreateTablet for tablet 3052da843efd4ef4861a13e2240bb466 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:15:55.735127 24078 tablet_service.cc:1511] Processing CreateTablet for tablet 0015560e665c48f0bd17fe4be5c8761d (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260709 12:15:55.736711 24078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0015560e665c48f0bd17fe4be5c8761d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.741636 24003 tablet_service.cc:1511] Processing CreateTablet for tablet 3052da843efd4ef4861a13e2240bb466 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:15:55.748040 24006 tablet_service.cc:1511] Processing CreateTablet for tablet ab48aa491dc84193a39fdbf1a1c52927 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:15:55.748652 24081 tablet_service.cc:1511] Processing CreateTablet for tablet 3052da843efd4ef4861a13e2240bb466 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260709 12:15:55.754019 24006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab48aa491dc84193a39fdbf1a1c52927. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.754477 24081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3052da843efd4ef4861a13e2240bb466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.755363 24083 tablet_service.cc:1511] Processing CreateTablet for tablet 802231ce402445309306d2cbffa2f92c (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:15:55.756928 24083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 802231ce402445309306d2cbffa2f92c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.759268 24003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3052da843efd4ef4861a13e2240bb466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.765575 24286 tablet_bootstrap.cc:492] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:15:55.767395 24222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3052da843efd4ef4861a13e2240bb466. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.770637 24285 tablet_bootstrap.cc:654] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:55.773288 24224 tablet_service.cc:1511] Processing CreateTablet for tablet ab48aa491dc84193a39fdbf1a1c52927 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:15:55.774973 24224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab48aa491dc84193a39fdbf1a1c52927. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.789808 24043 heartbeater.cc:499] Master 127.11.13.190:46705 was elected leader, sending a full tablet report...
I20260709 12:15:55.813388 24286 tablet_bootstrap.cc:654] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:55.805966 24151 tablet_service.cc:1511] Processing CreateTablet for tablet ab48aa491dc84193a39fdbf1a1c52927 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260709 12:15:55.817075 24151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ab48aa491dc84193a39fdbf1a1c52927. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.817708 24157 tablet_service.cc:1511] Processing CreateTablet for tablet 802231ce402445309306d2cbffa2f92c (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:15:55.819109 24157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 802231ce402445309306d2cbffa2f92c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.839659 24285 tablet_bootstrap.cc:492] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:15:55.840363 24285 ts_tablet_manager.cc:1403] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.150s	user 0.012s	sys 0.000s
I20260709 12:15:55.843948 24285 raft_consensus.cc:359] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:55.844758 24285 raft_consensus.cc:385] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:55.846997 24230 tablet_service.cc:1511] Processing CreateTablet for tablet 802231ce402445309306d2cbffa2f92c (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260709 12:15:55.850088 24230 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 802231ce402445309306d2cbffa2f92c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.856635 24006 tablet_service.cc:1511] Processing CreateTablet for tablet e35efdb2f1c24db0a45fbc04abea260e (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:15:55.856559 24076 tablet_service.cc:1511] Processing CreateTablet for tablet e35efdb2f1c24db0a45fbc04abea260e (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:15:55.851539 24285 raft_consensus.cc:740] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:15:55.859304 24076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e35efdb2f1c24db0a45fbc04abea260e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.861584 24287 tablet_bootstrap.cc:492] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:15:55.873778 24006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e35efdb2f1c24db0a45fbc04abea260e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.877132 24229 tablet_service.cc:1511] Processing CreateTablet for tablet 4c91f626cde148838a6a979d4fd1a5ed (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:15:55.878679 24153 tablet_service.cc:1511] Processing CreateTablet for tablet e35efdb2f1c24db0a45fbc04abea260e (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260709 12:15:55.879238 24229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c91f626cde148838a6a979d4fd1a5ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.880300 24153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e35efdb2f1c24db0a45fbc04abea260e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.859231 24285 consensus_queue.cc:260] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:55.889739 24078 tablet_service.cc:1511] Processing CreateTablet for tablet 4c91f626cde148838a6a979d4fd1a5ed (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:15:55.891264 24078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c91f626cde148838a6a979d4fd1a5ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.895282 24007 tablet_service.cc:1511] Processing CreateTablet for tablet 4c91f626cde148838a6a979d4fd1a5ed (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260709 12:15:55.896760 24007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 4c91f626cde148838a6a979d4fd1a5ed. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.907409 24118 heartbeater.cc:499] Master 127.11.13.190:46705 was elected leader, sending a full tablet report...
I20260709 12:15:55.909915 24288 tablet_bootstrap.cc:492] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:15:55.943710 24288 tablet_bootstrap.cc:654] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:55.951877 24287 tablet_bootstrap.cc:654] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:55.951974 24285 ts_tablet_manager.cc:1434] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.111s	user 0.011s	sys 0.007s
I20260709 12:15:55.953565 24224 tablet_service.cc:1511] Processing CreateTablet for tablet 0756db96a05a45bc9ebc1f91ee907496 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:15:55.958173 24224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0756db96a05a45bc9ebc1f91ee907496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.961095 24153 tablet_service.cc:1511] Processing CreateTablet for tablet 0756db96a05a45bc9ebc1f91ee907496 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:15:55.967971 24223 tablet_service.cc:1511] Processing CreateTablet for tablet 531e8a2f98124f13910b3a8ce4d279c1 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:15:55.969501 24223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 531e8a2f98124f13910b3a8ce4d279c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.967993 24153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0756db96a05a45bc9ebc1f91ee907496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.971127 24076 tablet_service.cc:1511] Processing CreateTablet for tablet d7366aa6480d4d1e909db5717506432b (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:15:55.974942 24076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7366aa6480d4d1e909db5717506432b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.971243 24007 tablet_service.cc:1511] Processing CreateTablet for tablet d7366aa6480d4d1e909db5717506432b (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:15:55.977841 24007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7366aa6480d4d1e909db5717506432b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.977671 24152 tablet_service.cc:1511] Processing CreateTablet for tablet d7366aa6480d4d1e909db5717506432b (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260709 12:15:55.979277 24152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d7366aa6480d4d1e909db5717506432b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.980862 24006 tablet_service.cc:1511] Processing CreateTablet for tablet 0756db96a05a45bc9ebc1f91ee907496 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260709 12:15:55.972896 24285 tablet_bootstrap.cc:492] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:15:55.972973 24157 tablet_service.cc:1511] Processing CreateTablet for tablet 531e8a2f98124f13910b3a8ce4d279c1 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:15:55.987360 24157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 531e8a2f98124f13910b3a8ce4d279c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.988229 24078 tablet_service.cc:1511] Processing CreateTablet for tablet 531e8a2f98124f13910b3a8ce4d279c1 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260709 12:15:55.965489 24286 tablet_bootstrap.cc:492] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:15:55.997839 24286 ts_tablet_manager.cc:1403] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.233s	user 0.016s	sys 0.013s
I20260709 12:15:55.986186 24006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0756db96a05a45bc9ebc1f91ee907496. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:55.998871 24078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 531e8a2f98124f13910b3a8ce4d279c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.005579 24285 tablet_bootstrap.cc:654] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:56.007290 24229 tablet_service.cc:1511] Processing CreateTablet for tablet 41c505fd92654536b72dbd8a3f1bc728 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:15:56.009356 24229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41c505fd92654536b72dbd8a3f1bc728. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:15:56.014890 24044 tablet.cc:2406] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:15:56.001462 24286 raft_consensus.cc:359] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:56.015877 24286 raft_consensus.cc:385] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:56.018765 24192 heartbeater.cc:499] Master 127.11.13.190:46705 was elected leader, sending a full tablet report...
I20260709 12:15:56.025411 24288 tablet_bootstrap.cc:492] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:15:56.026026 24288 ts_tablet_manager.cc:1403] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.116s	user 0.012s	sys 0.013s
I20260709 12:15:56.016224 24286 raft_consensus.cc:740] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:15:56.027123 24286 consensus_queue.cc:260] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:56.031066 24287 tablet_bootstrap.cc:492] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:15:56.031661 24287 ts_tablet_manager.cc:1403] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.170s	user 0.012s	sys 0.007s
I20260709 12:15:56.033392 24003 tablet_service.cc:1511] Processing CreateTablet for tablet 41c505fd92654536b72dbd8a3f1bc728 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:15:56.035012 24003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41c505fd92654536b72dbd8a3f1bc728. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.029692 24288 raft_consensus.cc:359] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:56.039397 24288 raft_consensus.cc:385] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:56.039959 24288 raft_consensus.cc:740] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:15:56.040812 24288 consensus_queue.cc:260] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:56.041828 24287 raft_consensus.cc:359] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:56.042701 24287 raft_consensus.cc:385] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:56.043032 24287 raft_consensus.cc:740] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:15:56.043879 24287 consensus_queue.cc:260] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:56.046684 24286 ts_tablet_manager.cc:1434] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.048s	user 0.008s	sys 0.000s
I20260709 12:15:56.048300 24286 tablet_bootstrap.cc:492] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:15:56.049054 24076 tablet_service.cc:1511] Processing CreateTablet for tablet 41c505fd92654536b72dbd8a3f1bc728 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260709 12:15:56.054097 24151 tablet_service.cc:1511] Processing CreateTablet for tablet 21f0988e53da475c8274bcb13b34b3a8 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:15:56.055708 24151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21f0988e53da475c8274bcb13b34b3a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.055663 24224 tablet_service.cc:1511] Processing CreateTablet for tablet 21f0988e53da475c8274bcb13b34b3a8 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:15:56.057466 24224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21f0988e53da475c8274bcb13b34b3a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.067096 24076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 41c505fd92654536b72dbd8a3f1bc728. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.069387 24288 ts_tablet_manager.cc:1434] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.043s	user 0.006s	sys 0.005s
I20260709 12:15:56.070245 24266 heartbeater.cc:499] Master 127.11.13.190:46705 was elected leader, sending a full tablet report...
I20260709 12:15:56.070689 24288 tablet_bootstrap.cc:492] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:15:56.071369 24287 ts_tablet_manager.cc:1434] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.039s	user 0.008s	sys 0.000s
I20260709 12:15:56.072494 24287 tablet_bootstrap.cc:492] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:15:56.081692 24286 tablet_bootstrap.cc:654] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:56.088218 24285 tablet_bootstrap.cc:492] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:15:56.088842 24285 ts_tablet_manager.cc:1403] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.116s	user 0.018s	sys 0.006s
I20260709 12:15:56.092145 24285 raft_consensus.cc:359] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:56.092903 24285 raft_consensus.cc:385] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:56.093214 24285 raft_consensus.cc:740] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:15:56.097700 24007 tablet_service.cc:1511] Processing CreateTablet for tablet 21f0988e53da475c8274bcb13b34b3a8 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260709 12:15:56.099293 24007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21f0988e53da475c8274bcb13b34b3a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.104089 24295 raft_consensus.cc:493] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.104763 24295 raft_consensus.cc:515] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:56.106197 24288 tablet_bootstrap.cc:654] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:56.093924 24285 consensus_queue.cc:260] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:56.109581 24295 leader_election.cc:290] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:15:56.122593 24285 ts_tablet_manager.cc:1434] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.033s	user 0.006s	sys 0.004s
I20260709 12:15:56.130270 24285 tablet_bootstrap.cc:492] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:15:56.130429 24078 tablet_service.cc:1511] Processing CreateTablet for tablet 8e3a9c266ee343eb903d5138d1de95d2 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:15:56.132534 24078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e3a9c266ee343eb903d5138d1de95d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.118321 24152 tablet_service.cc:1511] Processing CreateTablet for tablet 8e3a9c266ee343eb903d5138d1de95d2 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:15:56.120505 24287 tablet_bootstrap.cc:654] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:56.132228 24223 tablet_service.cc:1511] Processing CreateTablet for tablet 8e3a9c266ee343eb903d5138d1de95d2 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260709 12:15:56.140177 24152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e3a9c266ee343eb903d5138d1de95d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.140497 24223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8e3a9c266ee343eb903d5138d1de95d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.138829 24285 tablet_bootstrap.cc:654] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:56.172554 24286 tablet_bootstrap.cc:492] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:15:56.173230 24286 ts_tablet_manager.cc:1403] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.125s	user 0.017s	sys 0.009s
I20260709 12:15:56.174846 24005 tablet_service.cc:1511] Processing CreateTablet for tablet dcccd54cb1fa4e9d9cab5afd3e8d4748 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:15:56.175027 24224 tablet_service.cc:1511] Processing CreateTablet for tablet 7dd0b13fa2ed4f4489eef33489d4ebf1 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:15:56.176388 24005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcccd54cb1fa4e9d9cab5afd3e8d4748. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.174715 24083 tablet_service.cc:1511] Processing CreateTablet for tablet dcccd54cb1fa4e9d9cab5afd3e8d4748 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:15:56.185920 24224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dd0b13fa2ed4f4489eef33489d4ebf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.191128 24083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcccd54cb1fa4e9d9cab5afd3e8d4748. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.177673 24286 raft_consensus.cc:359] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:56.195717 24286 raft_consensus.cc:385] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:56.196122 24286 raft_consensus.cc:740] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:15:56.208293 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb90520a319469fa1664276964749ea" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:15:56.209826 24286 consensus_queue.cc:260] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
W20260709 12:15:56.210493 24058 leader_election.cc:343] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:56.212360 24286 ts_tablet_manager.cc:1434] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.039s	user 0.010s	sys 0.000s
I20260709 12:15:56.213683 24151 tablet_service.cc:1511] Processing CreateTablet for tablet dcccd54cb1fa4e9d9cab5afd3e8d4748 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260709 12:15:56.215260 24151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet dcccd54cb1fa4e9d9cab5afd3e8d4748. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.221058 24290 raft_consensus.cc:493] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.221829 24290 raft_consensus.cc:515] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:56.229668 24290 leader_election.cc:290] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:56.243861 24286 tablet_bootstrap.cc:492] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:15:56.245721 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb90520a319469fa1664276964749ea" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
W20260709 12:15:56.248090 24056 leader_election.cc:343] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:56.248714 24056 leader_election.cc:304] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:15:56.250144 24295 raft_consensus.cc:2749] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:56.252892 24288 tablet_bootstrap.cc:492] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:15:56.253647 24288 ts_tablet_manager.cc:1403] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.183s	user 0.018s	sys 0.023s
I20260709 12:15:56.257577 24288 raft_consensus.cc:359] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:56.258987 24288 raft_consensus.cc:385] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:56.259584 24288 raft_consensus.cc:740] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:15:56.261567 24285 tablet_bootstrap.cc:492] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:15:56.262288 24285 ts_tablet_manager.cc:1403] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.132s	user 0.025s	sys 0.000s
I20260709 12:15:56.262359 24288 consensus_queue.cc:260] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:56.263716 24287 tablet_bootstrap.cc:492] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:15:56.266774 24285 raft_consensus.cc:359] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:56.267576 24287 ts_tablet_manager.cc:1403] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.195s	user 0.014s	sys 0.041s
I20260709 12:15:56.267961 24285 raft_consensus.cc:385] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:56.268500 24285 raft_consensus.cc:740] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:15:56.264664 24286 tablet_bootstrap.cc:654] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:56.271752 24287 raft_consensus.cc:359] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:56.272809 24287 raft_consensus.cc:385] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:56.273231 24287 raft_consensus.cc:740] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:15:56.274307 24285 consensus_queue.cc:260] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:56.274215 24287 consensus_queue.cc:260] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:56.277351 24287 ts_tablet_manager.cc:1434] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:15:56.278527 24287 tablet_bootstrap.cc:492] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:15:56.279243 24285 ts_tablet_manager.cc:1434] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.016s	user 0.009s	sys 0.000s
I20260709 12:15:56.280465 24285 tablet_bootstrap.cc:492] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:15:56.282768 24078 tablet_service.cc:1511] Processing CreateTablet for tablet 7dd0b13fa2ed4f4489eef33489d4ebf1 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:15:56.283720 24006 tablet_service.cc:1511] Processing CreateTablet for tablet 7dd0b13fa2ed4f4489eef33489d4ebf1 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260709 12:15:56.284333 24078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dd0b13fa2ed4f4489eef33489d4ebf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.285141 24006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7dd0b13fa2ed4f4489eef33489d4ebf1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.287642 24288 ts_tablet_manager.cc:1434] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 12:15:56.288746 24288 tablet_bootstrap.cc:492] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:15:56.297580 24287 tablet_bootstrap.cc:654] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:56.298319 24285 tablet_bootstrap.cc:654] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:56.305047 24288 tablet_bootstrap.cc:654] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:56.324254 24229 tablet_service.cc:1511] Processing CreateTablet for tablet f4c368e2cc6c4869a85a53ae022a0785 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:15:56.325821 24229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4c368e2cc6c4869a85a53ae022a0785. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.331429 24223 tablet_service.cc:1511] Processing CreateTablet for tablet 1012da668d874f27a9563ae3f53ce4eb (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:15:56.332913 24223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1012da668d874f27a9563ae3f53ce4eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.333160 24151 tablet_service.cc:1511] Processing CreateTablet for tablet f4c368e2cc6c4869a85a53ae022a0785 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:15:56.335446 24151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4c368e2cc6c4869a85a53ae022a0785. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.335589 24076 tablet_service.cc:1511] Processing CreateTablet for tablet 1012da668d874f27a9563ae3f53ce4eb (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:15:56.336980 24076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1012da668d874f27a9563ae3f53ce4eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.320403 24286 tablet_bootstrap.cc:492] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:15:56.347702 24286 ts_tablet_manager.cc:1403] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.117s	user 0.020s	sys 0.008s
I20260709 12:15:56.352269 24080 tablet_service.cc:1511] Processing CreateTablet for tablet b3effea5c11740418bd8b7bedd33e0f5 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:15:56.353794 24153 tablet_service.cc:1511] Processing CreateTablet for tablet b3effea5c11740418bd8b7bedd33e0f5 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:15:56.355008 24230 tablet_service.cc:1511] Processing CreateTablet for tablet be4e681eef304bb8bc812e313a67414d (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:15:56.355960 24153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3effea5c11740418bd8b7bedd33e0f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.356405 24230 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be4e681eef304bb8bc812e313a67414d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.352126 24286 raft_consensus.cc:359] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:56.357267 24286 raft_consensus.cc:385] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:56.342631 24007 tablet_service.cc:1511] Processing CreateTablet for tablet f4c368e2cc6c4869a85a53ae022a0785 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260709 12:15:56.357635 24286 raft_consensus.cc:740] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:15:56.358538 24007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f4c368e2cc6c4869a85a53ae022a0785. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.358582 24286 consensus_queue.cc:260] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:56.344300 24152 tablet_service.cc:1511] Processing CreateTablet for tablet 1012da668d874f27a9563ae3f53ce4eb (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260709 12:15:56.361830 24152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1012da668d874f27a9563ae3f53ce4eb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.353854 24080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3effea5c11740418bd8b7bedd33e0f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.387629 24006 tablet_service.cc:1511] Processing CreateTablet for tablet be4e681eef304bb8bc812e313a67414d (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260709 12:15:56.389174 24006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be4e681eef304bb8bc812e313a67414d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.389838 24295 raft_consensus.cc:493] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.390399 24295 raft_consensus.cc:515] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:56.392884 24295 leader_election.cc:290] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:15:56.395009 24286 ts_tablet_manager.cc:1434] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.047s	user 0.003s	sys 0.008s
I20260709 12:15:56.397236 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11a4ed5e3c5d47f189b17261d317eb7b" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:15:56.399513 24286 tablet_bootstrap.cc:492] T 659e684fa2a0433faf027bf99c82dd6b P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:15:56.402827 24296 raft_consensus.cc:493] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.410566 24296 raft_consensus.cc:515] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:56.402827 24081 tablet_service.cc:1511] Processing CreateTablet for tablet be4e681eef304bb8bc812e313a67414d (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
W20260709 12:15:56.405128 24056 leader_election.cc:343] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:56.405001 24157 tablet_service.cc:1511] Processing CreateTablet for tablet b4638bcb9174442184bc8d8a27d45072 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:15:56.414741 24081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be4e681eef304bb8bc812e313a67414d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.415313 24157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4638bcb9174442184bc8d8a27d45072. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.416710 24287 tablet_bootstrap.cc:492] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:15:56.417820 24287 ts_tablet_manager.cc:1403] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.140s	user 0.012s	sys 0.007s
I20260709 12:15:56.412356 24224 tablet_service.cc:1511] Processing CreateTablet for tablet b4638bcb9174442184bc8d8a27d45072 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:15:56.420246 24224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4638bcb9174442184bc8d8a27d45072. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.436807 24297 raft_consensus.cc:493] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.437723 24297 raft_consensus.cc:515] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:56.420862 24005 tablet_service.cc:1511] Processing CreateTablet for tablet b3effea5c11740418bd8b7bedd33e0f5 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260709 12:15:56.446159 24005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3effea5c11740418bd8b7bedd33e0f5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.469498 24290 raft_consensus.cc:493] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.470966 24285 tablet_bootstrap.cc:492] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:15:56.507858 24285 ts_tablet_manager.cc:1403] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.228s	user 0.012s	sys 0.017s
I20260709 12:15:56.508484 24152 tablet_service.cc:1511] Processing CreateTablet for tablet 3bd7c9c11f834f15974a65477ef9258a (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:15:56.477968 24288 tablet_bootstrap.cc:492] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:15:56.479070 24003 tablet_service.cc:1511] Processing CreateTablet for tablet b4638bcb9174442184bc8d8a27d45072 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260709 12:15:56.510591 24288 ts_tablet_manager.cc:1403] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.222s	user 0.014s	sys 0.006s
I20260709 12:15:56.511703 24003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b4638bcb9174442184bc8d8a27d45072. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.511962 24285 raft_consensus.cc:359] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:56.512861 24285 raft_consensus.cc:385] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:56.513203 24285 raft_consensus.cc:740] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:15:56.470326 24290 raft_consensus.cc:515] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:56.514554 24288 raft_consensus.cc:359] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:56.516021 24288 raft_consensus.cc:385] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:56.516467 24288 raft_consensus.cc:740] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:15:56.517549 24288 consensus_queue.cc:260] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:56.518744 24290 leader_election.cc:290] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:15:56.510037 24152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bd7c9c11f834f15974a65477ef9258a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.497534 24287 raft_consensus.cc:359] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:56.524478 24287 raft_consensus.cc:385] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:56.524899 24287 raft_consensus.cc:740] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:15:56.525986 24287 consensus_queue.cc:260] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:56.495884 24296 leader_election.cc:290] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:56.532536 24296 raft_consensus.cc:493] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.533403 24296 raft_consensus.cc:515] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:56.498147 24297 leader_election.cc:290] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:15:56.506403 24076 tablet_service.cc:1511] Processing CreateTablet for tablet 3bd7c9c11f834f15974a65477ef9258a (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:15:56.536121 24076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bd7c9c11f834f15974a65477ef9258a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.537552 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc53eac4ce984dbaa561d9b719143d18" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
W20260709 12:15:56.542403 23983 leader_election.cc:343] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:56.545770 24092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5587d307c2f64ab9a9b39c1b190e85be" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:56.547644 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc53eac4ce984dbaa561d9b719143d18" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
W20260709 12:15:56.548380 23981 leader_election.cc:343] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:15:56.561254 23981 leader_election.cc:343] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:56.561981 23981 leader_election.cc:304] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:15:56.562009 24006 tablet_service.cc:1511] Processing CreateTablet for tablet 7e76d3d2239249bb806deb5ad8232c1c (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:15:56.563439 24290 raft_consensus.cc:2749] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:56.563962 24006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e76d3d2239249bb806deb5ad8232c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.566390 24290 raft_consensus.cc:493] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.572567 24290 raft_consensus.cc:515] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
W20260709 12:15:56.574653 23963 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:15:56.580518 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11a4ed5e3c5d47f189b17261d317eb7b" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
W20260709 12:15:56.582616 24056 leader_election.cc:343] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:56.583101 24290 leader_election.cc:290] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:15:56.583177 24056 leader_election.cc:304] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:15:56.585582 24295 raft_consensus.cc:2749] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:56.586673 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb6c32822cf44158ca1ea0d62e0b001" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
W20260709 12:15:56.588398 23983 leader_election.cc:343] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:56.591291 24224 tablet_service.cc:1511] Processing CreateTablet for tablet 3bd7c9c11f834f15974a65477ef9258a (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260709 12:15:56.594695 24224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3bd7c9c11f834f15974a65477ef9258a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.605319 24296 leader_election.cc:290] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:15:56.607244 24287 ts_tablet_manager.cc:1434] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.182s	user 0.010s	sys 0.000s
I20260709 12:15:56.608352 24288 ts_tablet_manager.cc:1434] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.097s	user 0.014s	sys 0.009s
I20260709 12:15:56.609622 24288 tablet_bootstrap.cc:492] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:15:56.552526 24285 consensus_queue.cc:260] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:56.620496 24287 tablet_bootstrap.cc:492] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:15:56.604400 24286 tablet_bootstrap.cc:654] T 659e684fa2a0433faf027bf99c82dd6b P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:56.633271 24285 ts_tablet_manager.cc:1434] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.125s	user 0.009s	sys 0.000s
I20260709 12:15:56.634594 24285 tablet_bootstrap.cc:492] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:15:56.634454 24080 tablet_service.cc:1511] Processing CreateTablet for tablet 7e76d3d2239249bb806deb5ad8232c1c (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:15:56.636451 24080 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e76d3d2239249bb806deb5ad8232c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.638708 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5587d307c2f64ab9a9b39c1b190e85be" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:15:56.639504 24167 raft_consensus.cc:2468] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 0.
I20260709 12:15:56.646373 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb6c32822cf44158ca1ea0d62e0b001" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:15:56.664012 24078 tablet_service.cc:1511] Processing CreateTablet for tablet 56ed2939d1454de181106eec39d0e4bc (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:15:56.667589 24078 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56ed2939d1454de181106eec39d0e4bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.667385 24007 tablet_service.cc:1511] Processing CreateTablet for tablet 56ed2939d1454de181106eec39d0e4bc (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:15:56.667266 24008 tablet_service.cc:1511] Processing CreateTablet for tablet be5ea3171ad74d74bede5875414d92ac (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:15:56.669234 24007 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56ed2939d1454de181106eec39d0e4bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.670042 24008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be5ea3171ad74d74bede5875414d92ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.667438 24083 tablet_service.cc:1511] Processing CreateTablet for tablet 2e4765d7c78c480fa4ba08d2812aeac3 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:15:56.671655 24083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e4765d7c78c480fa4ba08d2812aeac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.673404 24004 tablet_service.cc:1511] Processing CreateTablet for tablet 95824f48a95746289333a9da8e265178 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:15:56.674925 24004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95824f48a95746289333a9da8e265178. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.676291 24153 tablet_service.cc:1511] Processing CreateTablet for tablet 2e4765d7c78c480fa4ba08d2812aeac3 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:15:56.679134 23981 leader_election.cc:304] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 6c74a5203b594708921cd793aaca0b8b
W20260709 12:15:56.680485 23981 leader_election.cc:343] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:56.680956 23981 leader_election.cc:304] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:15:56.681730 24290 raft_consensus.cc:2804] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:56.682107 24290 raft_consensus.cc:493] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:56.682502 24290 raft_consensus.cc:3060] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:56.687022 24324 raft_consensus.cc:2749] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:56.688882 24151 tablet_service.cc:1511] Processing CreateTablet for tablet 7e76d3d2239249bb806deb5ad8232c1c (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260709 12:15:56.696820 24152 tablet_service.cc:1511] Processing CreateTablet for tablet be5ea3171ad74d74bede5875414d92ac (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:15:56.677927 24153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e4765d7c78c480fa4ba08d2812aeac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.708621 24229 tablet_service.cc:1511] Processing CreateTablet for tablet 56ed2939d1454de181106eec39d0e4bc (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260709 12:15:56.713810 24079 tablet_service.cc:1511] Processing CreateTablet for tablet 1e98aa59558f4e58925a3c87026b6fa4 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:15:56.715595 24079 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e98aa59558f4e58925a3c87026b6fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.716215 24075 tablet_service.cc:1511] Processing CreateTablet for tablet 56fb07e9eb574b1fbc5da95063610dd5 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:15:56.717708 24075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56fb07e9eb574b1fbc5da95063610dd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.719087 24287 tablet_bootstrap.cc:654] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:56.719177 24156 tablet_service.cc:1511] Processing CreateTablet for tablet 1e98aa59558f4e58925a3c87026b6fa4 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:15:56.720181 24148 tablet_service.cc:1511] Processing CreateTablet for tablet 31e3897eed824c5fa96e04ce0eca32a8 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:15:56.721148 24082 tablet_service.cc:1511] Processing CreateTablet for tablet 31e3897eed824c5fa96e04ce0eca32a8 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:15:56.714090 24077 tablet_service.cc:1511] Processing CreateTablet for tablet 95824f48a95746289333a9da8e265178 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:15:56.724303 24296 raft_consensus.cc:493] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.719177 24149 tablet_service.cc:1511] Processing CreateTablet for tablet ba257bd0f16a4a7da0cb4a7953ca1869 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:15:56.725045 24296 raft_consensus.cc:515] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:56.728117 24001 tablet_service.cc:1511] Processing CreateTablet for tablet 31e3897eed824c5fa96e04ce0eca32a8 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260709 12:15:56.729799 24001 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31e3897eed824c5fa96e04ce0eca32a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.730094 24150 tablet_service.cc:1511] Processing CreateTablet for tablet c385253c4ad94570b7dbb6875a755b36 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:15:56.732311 24151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7e76d3d2239249bb806deb5ad8232c1c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.733069 24156 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e98aa59558f4e58925a3c87026b6fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.728384 24154 tablet_service.cc:1511] Processing CreateTablet for tablet e2eabb81feec483ea9b303c5e516b7cf (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:15:56.734488 24002 tablet_service.cc:1511] Processing CreateTablet for tablet e2eabb81feec483ea9b303c5e516b7cf (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:15:56.735476 24148 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31e3897eed824c5fa96e04ce0eca32a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.731882 24000 tablet_service.cc:1511] Processing CreateTablet for tablet 131c7392ac614d87ab3d9b6713a46f7b (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:15:56.724115 24082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 31e3897eed824c5fa96e04ce0eca32a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.737406 24296 leader_election.cc:290] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:56.738469 24149 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba257bd0f16a4a7da0cb4a7953ca1869. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.738548 24077 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95824f48a95746289333a9da8e265178. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.740312 24157 tablet_service.cc:1511] Processing CreateTablet for tablet 95824f48a95746289333a9da8e265178 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260709 12:15:56.741750 24157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 95824f48a95746289333a9da8e265178. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.723579 24290 raft_consensus.cc:515] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:56.742442 24154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2eabb81feec483ea9b303c5e516b7cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.744527 24150 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c385253c4ad94570b7dbb6875a755b36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.744844 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5587d307c2f64ab9a9b39c1b190e85be" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:15:56.746414 24000 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 131c7392ac614d87ab3d9b6713a46f7b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:15:56.746685 23981 leader_election.cc:343] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:56.750594 24002 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2eabb81feec483ea9b303c5e516b7cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.752054 24074 tablet_service.cc:1511] Processing CreateTablet for tablet 131c7392ac614d87ab3d9b6713a46f7b (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:15:56.753577 24074 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 131c7392ac614d87ab3d9b6713a46f7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.754374 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5587d307c2f64ab9a9b39c1b190e85be" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:15:56.758211 24167 raft_consensus.cc:3060] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:56.759270 24223 tablet_service.cc:1511] Processing CreateTablet for tablet be5ea3171ad74d74bede5875414d92ac (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260709 12:15:56.760787 24320 raft_consensus.cc:493] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.761515 24320 raft_consensus.cc:515] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:56.765333 24320 leader_election.cc:290] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:56.755770 24290 leader_election.cc:290] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:15:56.754321 24229 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56ed2939d1454de181106eec39d0e4bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.779403 24152 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be5ea3171ad74d74bede5875414d92ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.790097 24223 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet be5ea3171ad74d74bede5875414d92ac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.792604 24081 tablet_service.cc:1511] Processing CreateTablet for tablet c385253c4ad94570b7dbb6875a755b36 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:15:56.794286 24081 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c385253c4ad94570b7dbb6875a755b36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.794806 24008 tablet_service.cc:1511] Processing CreateTablet for tablet 56fb07e9eb574b1fbc5da95063610dd5 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:15:56.796418 24008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56fb07e9eb574b1fbc5da95063610dd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.812721 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd2d7fad08bc4376bb38cf21a6a9b611" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:15:56.814883 24240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659e684fa2a0433faf027bf99c82dd6b" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:15:56.815994 24240 raft_consensus.cc:2468] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
W20260709 12:15:56.820978 24132 leader_election.cc:343] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:56.822023 24132 leader_election.cc:304] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:15:56.824892 24320 raft_consensus.cc:2804] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:56.825352 24320 raft_consensus.cc:493] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:56.825780 24320 raft_consensus.cc:3060] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:56.836306 24222 tablet_service.cc:1511] Processing CreateTablet for tablet 56fb07e9eb574b1fbc5da95063610dd5 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260709 12:15:56.838109 24222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 56fb07e9eb574b1fbc5da95063610dd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.839383 24230 tablet_service.cc:1511] Processing CreateTablet for tablet 2e4765d7c78c480fa4ba08d2812aeac3 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260709 12:15:56.842176 24295 raft_consensus.cc:493] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.843495 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd2d7fad08bc4376bb38cf21a6a9b611" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:56.843609 24295 raft_consensus.cc:515] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:56.841513 24230 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2e4765d7c78c480fa4ba08d2812aeac3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.825150 24288 tablet_bootstrap.cc:654] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:56.766613 24003 tablet_service.cc:1511] Processing CreateTablet for tablet ba257bd0f16a4a7da0cb4a7953ca1869 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:15:56.879097 24320 raft_consensus.cc:515] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:56.881588 24228 tablet_service.cc:1511] Processing CreateTablet for tablet ba257bd0f16a4a7da0cb4a7953ca1869 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260709 12:15:56.882139 24226 tablet_service.cc:1511] Processing CreateTablet for tablet 1e98aa59558f4e58925a3c87026b6fa4 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260709 12:15:56.882506 24225 tablet_service.cc:1511] Processing CreateTablet for tablet 131c7392ac614d87ab3d9b6713a46f7b (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260709 12:15:56.883435 24240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659e684fa2a0433faf027bf99c82dd6b" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:15:56.884037 24228 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba257bd0f16a4a7da0cb4a7953ca1869. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.884135 24240 raft_consensus.cc:3060] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:56.885279 24227 tablet_service.cc:1511] Processing CreateTablet for tablet e2eabb81feec483ea9b303c5e516b7cf (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260709 12:15:56.891839 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659e684fa2a0433faf027bf99c82dd6b" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:56.894424 24225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 131c7392ac614d87ab3d9b6713a46f7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.895355 24227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2eabb81feec483ea9b303c5e516b7cf. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:15:56.895701 24130 leader_election.cc:343] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:56.896545 24130 leader_election.cc:304] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327; no voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:15:56.897636 24297 raft_consensus.cc:2749] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:56.898289 24290 raft_consensus.cc:493] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:56.898783 24226 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1e98aa59558f4e58925a3c87026b6fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.898864 24290 raft_consensus.cc:515] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:56.892997 24295 leader_election.cc:290] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:15:56.903160 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e679d0d8ffd84a23afc6d7f8faf84ca6" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:15:56.903999 24231 tablet_service.cc:1511] Processing CreateTablet for tablet c385253c4ad94570b7dbb6875a755b36 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260709 12:15:56.904646 24290 leader_election.cc:290] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:15:56.905498 24231 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c385253c4ad94570b7dbb6875a755b36. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.906903 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e679d0d8ffd84a23afc6d7f8faf84ca6" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
W20260709 12:15:56.908684 23983 leader_election.cc:343] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:15:56.910305 23981 leader_election.cc:343] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:56.910801 23981 leader_election.cc:304] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:15:56.911319 24005 tablet_service.cc:1511] Processing CreateTablet for tablet 576c4469eb9c49dbbba40a54832fa33f (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:15:56.912169 24290 raft_consensus.cc:2749] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:56.916064 24092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659e684fa2a0433faf027bf99c82dd6b" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:56.917119 24287 tablet_bootstrap.cc:492] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:15:56.917830 24287 ts_tablet_manager.cc:1403] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.298s	user 0.030s	sys 0.027s
I20260709 12:15:56.917183 24286 tablet_bootstrap.cc:492] T 659e684fa2a0433faf027bf99c82dd6b P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:15:56.919422 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16f5650964ece90109902b6b40882" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:56.920006 24286 ts_tablet_manager.cc:1403] T 659e684fa2a0433faf027bf99c82dd6b P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.522s	user 0.014s	sys 0.009s
I20260709 12:15:56.920114 24093 raft_consensus.cc:2468] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 0.
I20260709 12:15:56.921168 24287 raft_consensus.cc:359] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:56.922474 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16f5650964ece90109902b6b40882" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:15:56.923151 24241 raft_consensus.cc:2468] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:15:56.923591 24167 raft_consensus.cc:2468] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 1.
I20260709 12:15:56.923722 24286 raft_consensus.cc:359] T 659e684fa2a0433faf027bf99c82dd6b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:56.924571 24286 raft_consensus.cc:385] T 659e684fa2a0433faf027bf99c82dd6b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:56.924904 24286 raft_consensus.cc:740] T 659e684fa2a0433faf027bf99c82dd6b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:15:56.925210 24058 leader_election.cc:304] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:15:56.926396 23981 leader_election.cc:304] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 6c74a5203b594708921cd793aaca0b8b
W20260709 12:15:56.927810 24130 leader_election.cc:343] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:56.928936 24295 raft_consensus.cc:2804] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:56.929414 24295 raft_consensus.cc:493] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:56.929674 24285 tablet_bootstrap.cc:654] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:56.933558 24290 raft_consensus.cc:2804] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:56.934693 24290 raft_consensus.cc:697] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:15:56.935698 24290 consensus_queue.cc:237] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:56.929838 24295 raft_consensus.cc:3060] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:56.938355 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e815126e7554106a097f95846d7f663" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:56.940032 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16f5650964ece90109902b6b40882" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:56.941005 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16f5650964ece90109902b6b40882" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:56.942229 24003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ba257bd0f16a4a7da0cb4a7953ca1869. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:15:56.942727 24056 leader_election.cc:343] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:15:56.941154 24204 leader_election.cc:343] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:56.946730 24204 leader_election.cc:304] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:15:56.947631 24296 raft_consensus.cc:2804] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:56.948017 24296 raft_consensus.cc:493] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:56.948370 24296 raft_consensus.cc:3060] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:15:56.954675 24204 leader_election.cc:343] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:15:56.955581 24204 leader_election.cc:343] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:56.958053 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659e684fa2a0433faf027bf99c82dd6b" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:15:56.959105 24166 raft_consensus.cc:2393] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0df0eff47b64e3eb45ab074139ed187 in current term 1: Already voted for candidate 008a79e516614c809c2d3afdb5cbc327 in this term.
I20260709 12:15:56.959884 24296 raft_consensus.cc:515] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:56.956579 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659e684fa2a0433faf027bf99c82dd6b" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:15:56.963773 24155 tablet_service.cc:1511] Processing CreateTablet for tablet 576c4469eb9c49dbbba40a54832fa33f (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:15:56.965574 24155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 576c4469eb9c49dbbba40a54832fa33f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.966287 24005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 576c4469eb9c49dbbba40a54832fa33f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.967336 24006 tablet_service.cc:1511] Processing CreateTablet for tablet 77079c31ca114c49a916e49cfd8faf2c (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:15:56.968910 24006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77079c31ca114c49a916e49cfd8faf2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.973522 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e815126e7554106a097f95846d7f663" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:15:56.974249 24296 leader_election.cc:290] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:56.978329 24286 consensus_queue.cc:260] T 659e684fa2a0433faf027bf99c82dd6b P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:56.979460 24093 raft_consensus.cc:3060] T 659e684fa2a0433faf027bf99c82dd6b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:56.980685 24076 tablet_service.cc:1511] Processing CreateTablet for tablet 576c4469eb9c49dbbba40a54832fa33f (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260709 12:15:56.985095 24240 raft_consensus.cc:2468] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:15:56.954716 24320 leader_election.cc:290] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:56.986655 24132 leader_election.cc:304] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:15:56.987619 24320 raft_consensus.cc:2804] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:56.988685 24295 raft_consensus.cc:515] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:56.989041 24320 raft_consensus.cc:697] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:15:56.990161 24320 consensus_queue.cc:237] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:56.993793 24083 tablet_service.cc:1511] Processing CreateTablet for tablet 77079c31ca114c49a916e49cfd8faf2c (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:15:56.991649 24093 raft_consensus.cc:2468] T 659e684fa2a0433faf027bf99c82dd6b P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:15:56.999734 24004 tablet_service.cc:1511] Processing CreateTablet for tablet 2bd0baecffd2454894539252387f4f2d (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:15:57.002307 24004 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bd0baecffd2454894539252387f4f2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:56.921922 24287 raft_consensus.cc:385] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:57.011320 24076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 576c4469eb9c49dbbba40a54832fa33f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.035633 24151 tablet_service.cc:1511] Processing CreateTablet for tablet 2bd0baecffd2454894539252387f4f2d (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:15:57.040490 24285 tablet_bootstrap.cc:492] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:15:57.041213 24285 ts_tablet_manager.cc:1403] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.407s	user 0.012s	sys 0.006s
I20260709 12:15:57.045027 24285 raft_consensus.cc:359] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:57.051066 24204 leader_election.cc:304] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b
I20260709 12:15:57.068420 24151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bd0baecffd2454894539252387f4f2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.075821 24075 tablet_service.cc:1511] Processing CreateTablet for tablet b0805608ea264bef894c413af00a0687 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260709 12:15:57.079784 24240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16f5650964ece90109902b6b40882" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:15:57.122982 24240 raft_consensus.cc:2393] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c74a5203b594708921cd793aaca0b8b in current term 1: Already voted for candidate a0df0eff47b64e3eb45ab074139ed187 in this term.
I20260709 12:15:57.080000 24295 leader_election.cc:290] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:15:56.965546 24224 tablet_service.cc:1511] Processing CreateTablet for tablet 77079c31ca114c49a916e49cfd8faf2c (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260709 12:15:57.017395 24287 raft_consensus.cc:740] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:15:57.081256 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16f5650964ece90109902b6b40882" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:15:57.087006 24083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77079c31ca114c49a916e49cfd8faf2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.102519 24157 tablet_service.cc:1511] Processing CreateTablet for tablet b0805608ea264bef894c413af00a0687 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
W20260709 12:15:57.116604 24204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:36699, user_credentials={real_user=slave}} blocked reactor thread for 65651us
I20260709 12:15:57.117635 24296 raft_consensus.cc:2749] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:57.124204 24285 raft_consensus.cc:385] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:15:57.140743 24056 leader_election.cc:343] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:57.141340 24056 leader_election.cc:304] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802
I20260709 12:15:57.146952 24157 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0805608ea264bef894c413af00a0687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.147956 23916 catalog_manager.cc:5697] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } }
W20260709 12:15:57.143353 24204 leader_election.cc:343] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:57.151628 24006 tablet_service.cc:1511] Processing CreateTablet for tablet 809a23266b1646e0be710ddab5fff291 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:15:57.065821 24227 tablet_service.cc:1511] Processing CreateTablet for tablet 2bd0baecffd2454894539252387f4f2d (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260709 12:15:57.159125 24286 ts_tablet_manager.cc:1434] T 659e684fa2a0433faf027bf99c82dd6b P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.239s	user 0.011s	sys 0.049s
I20260709 12:15:57.160840 24286 tablet_bootstrap.cc:492] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:15:57.163172 24288 tablet_bootstrap.cc:492] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:15:57.156065 24204 leader_election.cc:304] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:15:57.164791 24006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 809a23266b1646e0be710ddab5fff291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.166713 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16f5650964ece90109902b6b40882" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:15:57.146844 24332 raft_consensus.cc:2749] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:15:57.145187 24075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0805608ea264bef894c413af00a0687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.164979 24296 raft_consensus.cc:2749] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:57.169373 23914 catalog_manager.cc:5697] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 0 to 1, leader changed from <none> to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:57.174220 24222 tablet_service.cc:1511] Processing CreateTablet for tablet b0805608ea264bef894c413af00a0687 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
W20260709 12:15:57.176328 24204 leader_election.cc:343] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:57.165192 24288 ts_tablet_manager.cc:1403] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.556s	user 0.026s	sys 0.027s
I20260709 12:15:57.192876 24288 raft_consensus.cc:359] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:57.192653 24287 consensus_queue.cc:260] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:57.194402 24296 raft_consensus.cc:493] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:57.194928 24296 raft_consensus.cc:515] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:57.196593 24287 ts_tablet_manager.cc:1434] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.278s	user 0.009s	sys 0.000s
I20260709 12:15:57.197922 24287 tablet_bootstrap.cc:492] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:15:57.198372 24224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 77079c31ca114c49a916e49cfd8faf2c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.200014 24222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b0805608ea264bef894c413af00a0687. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.137748 24285 raft_consensus.cc:740] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:15:57.205451 24296 leader_election.cc:290] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:15:57.205564 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16f5650964ece90109902b6b40882" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:15:57.206856 24093 raft_consensus.cc:2393] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a0df0eff47b64e3eb45ab074139ed187 in current term 1: Already voted for candidate 6c74a5203b594708921cd793aaca0b8b in this term.
I20260709 12:15:57.208307 24285 consensus_queue.cc:260] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:57.215389 24227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bd0baecffd2454894539252387f4f2d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.193967 24288 raft_consensus.cc:385] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:57.216727 24288 raft_consensus.cc:740] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:15:57.218760 24285 ts_tablet_manager.cc:1434] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.177s	user 0.006s	sys 0.004s
I20260709 12:15:57.218422 24288 consensus_queue.cc:260] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:57.219970 24285 tablet_bootstrap.cc:492] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:15:57.224937 24288 ts_tablet_manager.cc:1434] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 12:15:57.226351 24288 tablet_bootstrap.cc:492] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:15:57.228194 24287 tablet_bootstrap.cc:654] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:57.214211 24154 tablet_service.cc:1511] Processing CreateTablet for tablet 809a23266b1646e0be710ddab5fff291 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:15:57.233687 24154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 809a23266b1646e0be710ddab5fff291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.247401 24008 tablet_service.cc:1511] Processing CreateTablet for tablet b498d1ba220843a5939d152e0382bff7 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:15:57.247895 24076 tablet_service.cc:1511] Processing CreateTablet for tablet 809a23266b1646e0be710ddab5fff291 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260709 12:15:57.249351 24008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b498d1ba220843a5939d152e0382bff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.249476 24076 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 809a23266b1646e0be710ddab5fff291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.250473 24288 tablet_bootstrap.cc:654] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:57.252815 24079 tablet_service.cc:1511] Processing CreateTablet for tablet b498d1ba220843a5939d152e0382bff7 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:15:57.254526 24079 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b498d1ba220843a5939d152e0382bff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.259086 24285 tablet_bootstrap.cc:654] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:57.259224 24155 tablet_service.cc:1511] Processing CreateTablet for tablet ddbc768ef2c54c3391f0558cc562a01d (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:15:57.260715 24155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddbc768ef2c54c3391f0558cc562a01d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.264328 24005 tablet_service.cc:1511] Processing CreateTablet for tablet ddbc768ef2c54c3391f0558cc562a01d (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:15:57.266237 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e679d0d8ffd84a23afc6d7f8faf84ca6" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:15:57.266508 24320 raft_consensus.cc:493] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:57.267468 24005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddbc768ef2c54c3391f0558cc562a01d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.269598 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e679d0d8ffd84a23afc6d7f8faf84ca6" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:57.271313 24018 raft_consensus.cc:2468] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 0.
I20260709 12:15:57.277493 24082 tablet_service.cc:1511] Processing CreateTablet for tablet 6db45947fce14c98b26a8ed1ed9bbce4 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:15:57.280543 24082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6db45947fce14c98b26a8ed1ed9bbce4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.267014 24320 raft_consensus.cc:515] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:57.283288 24153 tablet_service.cc:1511] Processing CreateTablet for tablet 6db45947fce14c98b26a8ed1ed9bbce4 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:15:57.284752 24153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6db45947fce14c98b26a8ed1ed9bbce4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.286823 24320 leader_election.cc:290] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:15:57.308269 24286 tablet_bootstrap.cc:654] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:57.304064 24204 leader_election.cc:304] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:15:57.311021 24296 raft_consensus.cc:2804] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:57.311422 24296 raft_consensus.cc:493] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:57.311756 24296 raft_consensus.cc:3060] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:57.311806 24240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb6c32822cf44158ca1ea0d62e0b001" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
W20260709 12:15:57.313678 24132 leader_election.cc:343] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:15:57.317904 24204 leader_election.cc:343] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:57.317968 24296 raft_consensus.cc:515] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:57.348640 24287 tablet_bootstrap.cc:492] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:15:57.349432 24287 ts_tablet_manager.cc:1403] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.152s	user 0.018s	sys 0.005s
I20260709 12:15:57.351013 24296 leader_election.cc:290] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Requested vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
W20260709 12:15:57.356762 23981 consensus_peers.cc:597] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:15:57.360386 24224 tablet_service.cc:1511] Processing CreateTablet for tablet b498d1ba220843a5939d152e0382bff7 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260709 12:15:57.362771 24224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b498d1ba220843a5939d152e0382bff7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.367873 24204 leader_election.cc:304] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:15:57.368369 24230 tablet_service.cc:1511] Processing CreateTablet for tablet ddbc768ef2c54c3391f0558cc562a01d (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260709 12:15:57.369445 24335 raft_consensus.cc:2749] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:15:57.391937 24230 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ddbc768ef2c54c3391f0558cc562a01d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.391629 24006 tablet_service.cc:1511] Processing CreateTablet for tablet 96039384602f4057b875e1d873138d47 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:15:57.392108 24075 tablet_service.cc:1511] Processing CreateTablet for tablet 96039384602f4057b875e1d873138d47 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:15:57.393565 24075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96039384602f4057b875e1d873138d47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.393591 24006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96039384602f4057b875e1d873138d47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.395567 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e679d0d8ffd84a23afc6d7f8faf84ca6" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:15:57.396886 24225 tablet_service.cc:1511] Processing CreateTablet for tablet 6db45947fce14c98b26a8ed1ed9bbce4 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260709 12:15:57.398394 24225 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6db45947fce14c98b26a8ed1ed9bbce4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.399621 24320 consensus_queue.cc:1048] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:57.401952 24285 tablet_bootstrap.cc:492] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:15:57.402637 24285 ts_tablet_manager.cc:1403] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.183s	user 0.015s	sys 0.017s
I20260709 12:15:57.410277 24290 raft_consensus.cc:493] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:57.410826 24290 raft_consensus.cc:515] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:57.413794 24290 leader_election.cc:290] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:15:57.414620 24240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e815126e7554106a097f95846d7f663" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:15:57.415436 24240 raft_consensus.cc:2468] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 0.
I20260709 12:15:57.415807 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e815126e7554106a097f95846d7f663" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:15:57.417189 23983 leader_election.cc:304] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802; no voters: 
W20260709 12:15:57.419180 23981 leader_election.cc:343] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:57.423646 24290 raft_consensus.cc:2804] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:57.424296 24290 raft_consensus.cc:493] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:57.424736 24290 raft_consensus.cc:3060] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:57.435913 24320 consensus_queue.cc:1048] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:57.444057 24287 raft_consensus.cc:359] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:57.445091 24287 raft_consensus.cc:385] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:57.445472 24287 raft_consensus.cc:740] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:15:57.446292 24287 consensus_queue.cc:260] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:57.448900 24287 ts_tablet_manager.cc:1434] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.099s	user 0.006s	sys 0.003s
I20260709 12:15:57.454324 24285 raft_consensus.cc:359] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:57.455161 24003 tablet_service.cc:1511] Processing CreateTablet for tablet 94568d4dbfd440478c83b50b50976be7 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:15:57.459406 24083 tablet_service.cc:1511] Processing CreateTablet for tablet 94568d4dbfd440478c83b50b50976be7 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:15:57.466646 24285 raft_consensus.cc:385] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:57.483326 24287 tablet_bootstrap.cc:492] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:15:57.485605 24286 tablet_bootstrap.cc:492] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:15:57.475135 24285 raft_consensus.cc:740] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:15:57.460402 24154 tablet_service.cc:1511] Processing CreateTablet for tablet 96039384602f4057b875e1d873138d47 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260709 12:15:57.487190 24154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 96039384602f4057b875e1d873138d47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.486842 24285 consensus_queue.cc:260] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:57.488250 24286 ts_tablet_manager.cc:1403] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.328s	user 0.016s	sys 0.016s
I20260709 12:15:57.491144 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e679d0d8ffd84a23afc6d7f8faf84ca6" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:15:57.491302 24287 tablet_bootstrap.cc:654] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:57.496492 24083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94568d4dbfd440478c83b50b50976be7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.501331 24151 tablet_service.cc:1511] Processing CreateTablet for tablet 6f15aae25cad42abb701c295e56963e3 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
W20260709 12:15:57.506887 24204 leader_election.cc:343] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:57.507231 24151 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f15aae25cad42abb701c295e56963e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.507354 24288 tablet_bootstrap.cc:492] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:15:57.510913 24288 ts_tablet_manager.cc:1403] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.285s	user 0.044s	sys 0.061s
I20260709 12:15:57.496908 24003 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94568d4dbfd440478c83b50b50976be7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.514149 24018 raft_consensus.cc:3060] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:57.514734 24288 raft_consensus.cc:359] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:57.515611 24288 raft_consensus.cc:385] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:57.515933 24288 raft_consensus.cc:740] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:15:57.516863 24288 consensus_queue.cc:260] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:57.516727 24227 tablet_service.cc:1511] Processing CreateTablet for tablet 6f15aae25cad42abb701c295e56963e3 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:15:57.520081 24227 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f15aae25cad42abb701c295e56963e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.522987 24288 ts_tablet_manager.cc:1434] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:15:57.525770 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb6c32822cf44158ca1ea0d62e0b001" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:57.526559 24017 raft_consensus.cc:2468] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:15:57.528213 24288 tablet_bootstrap.cc:492] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:15:57.528100 24290 raft_consensus.cc:515] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:57.542851 24286 raft_consensus.cc:359] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:57.544966 24286 raft_consensus.cc:385] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:57.545640 24286 raft_consensus.cc:740] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:15:57.547204 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e815126e7554106a097f95846d7f663" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:15:57.551396 24224 tablet_service.cc:1511] Processing CreateTablet for tablet 94568d4dbfd440478c83b50b50976be7 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260709 12:15:57.553098 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e815126e7554106a097f95846d7f663" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:15:57.554270 24290 leader_election.cc:290] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Requested vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:15:57.560187 24008 tablet_service.cc:1511] Processing CreateTablet for tablet 6f15aae25cad42abb701c295e56963e3 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260709 12:15:57.566488 24224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 94568d4dbfd440478c83b50b50976be7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.574226 24241 raft_consensus.cc:3060] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:15:57.580132 23981 leader_election.cc:343] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:57.580456 24153 tablet_service.cc:1511] Processing CreateTablet for tablet 93f3b3f298544987a7b9fc3dd0af496c (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:15:57.582360 24153 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93f3b3f298544987a7b9fc3dd0af496c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.581966 24075 tablet_service.cc:1511] Processing CreateTablet for tablet 93f3b3f298544987a7b9fc3dd0af496c (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:15:57.583488 24075 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93f3b3f298544987a7b9fc3dd0af496c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.572717 24018 raft_consensus.cc:2468] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 1.
I20260709 12:15:57.590675 24008 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6f15aae25cad42abb701c295e56963e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.591681 24204 leader_election.cc:304] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327
I20260709 12:15:57.592613 24335 raft_consensus.cc:2804] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:57.593096 24335 raft_consensus.cc:697] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Becoming Leader. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Running, Role: LEADER
I20260709 12:15:57.593659 24345 consensus_queue.cc:1048] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:57.593950 24335 consensus_queue.cc:237] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:57.617370 24230 tablet_service.cc:1511] Processing CreateTablet for tablet 93f3b3f298544987a7b9fc3dd0af496c (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260709 12:15:57.607268 24286 consensus_queue.cc:260] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:57.651405 24286 ts_tablet_manager.cc:1434] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.163s	user 0.011s	sys 0.000s
I20260709 12:15:57.652590 24286 tablet_bootstrap.cc:492] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:15:57.654719 24006 tablet_service.cc:1511] Processing CreateTablet for tablet 0192ff9d150a47feb68f490efce1e3f2 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:15:57.655930 24230 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 93f3b3f298544987a7b9fc3dd0af496c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.656177 24006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0192ff9d150a47feb68f490efce1e3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.662647 24288 tablet_bootstrap.cc:654] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:57.670415 24285 ts_tablet_manager.cc:1434] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.267s	user 0.027s	sys 0.049s
I20260709 12:15:57.680707 24348 raft_consensus.cc:493] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:57.682996 24005 tablet_service.cc:1511] Processing CreateTablet for tablet 2147e5dc12974e6c852f8961496066ae (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:15:57.707294 24348 raft_consensus.cc:515] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:57.680919 24222 tablet_service.cc:1511] Processing CreateTablet for tablet 2147e5dc12974e6c852f8961496066ae (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:15:57.667626 24082 tablet_service.cc:1511] Processing CreateTablet for tablet 0192ff9d150a47feb68f490efce1e3f2 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260709 12:15:57.709859 24082 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0192ff9d150a47feb68f490efce1e3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.684155 24155 tablet_service.cc:1511] Processing CreateTablet for tablet 0192ff9d150a47feb68f490efce1e3f2 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
W20260709 12:15:57.676810 24130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:42247, user_credentials={real_user=slave}} blocked reactor thread for 58035.3us
I20260709 12:15:57.711639 24155 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0192ff9d150a47feb68f490efce1e3f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.723749 24005 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2147e5dc12974e6c852f8961496066ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.727659 24222 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2147e5dc12974e6c852f8961496066ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.728950 24285 tablet_bootstrap.cc:492] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:15:57.725390 24083 tablet_service.cc:1511] Processing CreateTablet for tablet 2147e5dc12974e6c852f8961496066ae (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260709 12:15:57.732107 24083 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2147e5dc12974e6c852f8961496066ae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.670625 24241 raft_consensus.cc:2468] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 1.
I20260709 12:15:57.735632 23983 leader_election.cc:304] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327
I20260709 12:15:57.737576 24345 raft_consensus.cc:2804] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:57.744614 24287 tablet_bootstrap.cc:492] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:15:57.745381 24287 ts_tablet_manager.cc:1403] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.262s	user 0.001s	sys 0.015s
I20260709 12:15:57.746311 24154 tablet_service.cc:1511] Processing CreateTablet for tablet 57c7696ee73146b59bd8b17640333077 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:15:57.747853 24154 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57c7696ee73146b59bd8b17640333077. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.762626 24130 leader_election.cc:304] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: a0df0eff47b64e3eb45ab074139ed187
I20260709 12:15:57.763223 24092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11a4ed5e3c5d47f189b17261d317eb7b" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:57.764035 24092 raft_consensus.cc:2468] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:15:57.764384 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11a4ed5e3c5d47f189b17261d317eb7b" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:57.766068 24356 raft_consensus.cc:2804] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:57.767895 24356 raft_consensus.cc:493] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:57.768289 24356 raft_consensus.cc:3060] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:57.710021 24348 leader_election.cc:290] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:57.776862 24224 tablet_service.cc:1511] Processing CreateTablet for tablet 57c7696ee73146b59bd8b17640333077 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:15:57.778316 24224 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57c7696ee73146b59bd8b17640333077. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.773866 24287 raft_consensus.cc:359] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:57.786263 24006 tablet_service.cc:1511] Processing CreateTablet for tablet 57c7696ee73146b59bd8b17640333077 (DEFAULT_TABLE table=test-workload [id=f04c18275833486ba2933269c7fb79c9]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260709 12:15:57.786841 24287 raft_consensus.cc:385] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:57.787297 24287 raft_consensus.cc:740] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:15:57.787652 24006 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 57c7696ee73146b59bd8b17640333077. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:15:57.788199 24287 consensus_queue.cc:260] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:57.793550 24130 leader_election.cc:304] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: 
W20260709 12:15:57.794652 24130 leader_election.cc:343] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:57.798842 24287 ts_tablet_manager.cc:1434] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.053s	user 0.006s	sys 0.004s
I20260709 12:15:57.777758 24356 raft_consensus.cc:515] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:57.801708 24348 raft_consensus.cc:2804] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:57.802235 24348 raft_consensus.cc:493] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:57.802589 24348 raft_consensus.cc:3060] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:57.804466 24335 raft_consensus.cc:493] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:57.805088 24335 raft_consensus.cc:515] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:57.814767 24345 raft_consensus.cc:697] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:15:57.828613 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb6c32822cf44158ca1ea0d62e0b001" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:15:57.830034 24335 leader_election.cc:290] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:15:57.830663 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ceb6c32822cf44158ca1ea0d62e0b001" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:15:57.830847 24092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb90520a319469fa1664276964749ea" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:57.831349 24018 raft_consensus.cc:3060] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:57.831622 24092 raft_consensus.cc:2468] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 0.
I20260709 12:15:57.833838 24204 leader_election.cc:304] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
W20260709 12:15:57.835652 24132 leader_election.cc:343] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:57.835809 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb90520a319469fa1664276964749ea" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:15:57.837018 24356 leader_election.cc:290] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
W20260709 12:15:57.837790 24204 leader_election.cc:343] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:57.840462 24285 tablet_bootstrap.cc:654] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:57.838486 24335 raft_consensus.cc:2804] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:57.841099 24335 raft_consensus.cc:493] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:57.846562 24335 raft_consensus.cc:3060] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:57.842295 24287 tablet_bootstrap.cc:492] T bc53eac4ce984dbaa561d9b719143d18 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:15:57.845429 24286 tablet_bootstrap.cc:654] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
W20260709 12:15:57.854868 23963 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:15:57.863000 24359 raft_consensus.cc:493] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:57.863664 24359 raft_consensus.cc:515] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:57.864140 24018 raft_consensus.cc:2468] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:15:57.866325 24359 leader_election.cc:290] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:57.867620 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55819e09f3e245179f0dd80b9e60eb6a" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:15:57.868060 24130 leader_election.cc:304] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: a0df0eff47b64e3eb45ab074139ed187
I20260709 12:15:57.868412 24241 raft_consensus.cc:2468] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 0.
I20260709 12:15:57.868427 24335 raft_consensus.cc:515] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:57.869879 23983 leader_election.cc:304] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:15:57.871201 24335 leader_election.cc:290] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:15:57.870970 24359 raft_consensus.cc:2804] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:57.871929 24359 raft_consensus.cc:493] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:57.872342 24359 raft_consensus.cc:3060] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:57.874023 24092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb90520a319469fa1664276964749ea" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:15:57.874835 24092 raft_consensus.cc:3060] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:57.876600 24287 tablet_bootstrap.cc:654] T bc53eac4ce984dbaa561d9b719143d18 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:57.886242 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fbb90520a319469fa1664276964749ea" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
W20260709 12:15:57.887562 24204 leader_election.cc:343] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:57.887329 23912 catalog_manager.cc:5697] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132). New cstate: current_term: 1 leader_uuid: "a0df0eff47b64e3eb45ab074139ed187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:57.889765 24356 raft_consensus.cc:2804] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:57.890435 24356 raft_consensus.cc:697] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:15:57.891317 24356 consensus_queue.cc:237] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:57.898859 24092 raft_consensus.cc:2468] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 1.
I20260709 12:15:57.901392 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55819e09f3e245179f0dd80b9e60eb6a" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
W20260709 12:15:57.903447 23981 leader_election.cc:343] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:57.901495 24204 leader_election.cc:304] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327
I20260709 12:15:57.905649 24335 raft_consensus.cc:2804] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:57.906246 24335 raft_consensus.cc:697] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Becoming Leader. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Running, Role: LEADER
I20260709 12:15:57.907202 24335 consensus_queue.cc:237] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:57.870559 24345 consensus_queue.cc:237] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:57.884711 24348 raft_consensus.cc:515] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:57.918972 24286 tablet_bootstrap.cc:492] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:15:57.919735 24286 ts_tablet_manager.cc:1403] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.267s	user 0.026s	sys 0.021s
I20260709 12:15:57.925010 24332 raft_consensus.cc:493] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:57.924234 23914 catalog_manager.cc:5697] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:57.925565 24332 raft_consensus.cc:515] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:57.926832 24286 raft_consensus.cc:359] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:57.927726 24286 raft_consensus.cc:385] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:57.927831 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11a4ed5e3c5d47f189b17261d317eb7b" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:15:57.928025 24332 leader_election.cc:290] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:15:57.928215 24286 raft_consensus.cc:740] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:15:57.929400 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc53eac4ce984dbaa561d9b719143d18" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:15:57.929418 24286 consensus_queue.cc:260] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:57.930245 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc53eac4ce984dbaa561d9b719143d18" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:57.930958 24018 raft_consensus.cc:2468] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
W20260709 12:15:57.931348 24058 leader_election.cc:343] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:57.932367 24056 leader_election.cc:304] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802; no voters: a0df0eff47b64e3eb45ab074139ed187
W20260709 12:15:57.929653 24130 leader_election.cc:343] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:57.936326 24332 raft_consensus.cc:2804] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:57.939574 24332 raft_consensus.cc:493] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:57.948788 24288 tablet_bootstrap.cc:492] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:15:57.953935 24348 leader_election.cc:290] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:57.954432 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11a4ed5e3c5d47f189b17261d317eb7b" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:15:57.945760 24332 raft_consensus.cc:3060] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:57.957981 24091 raft_consensus.cc:3060] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:57.960806 23916 catalog_manager.cc:5697] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132). New cstate: current_term: 1 leader_uuid: "a0df0eff47b64e3eb45ab074139ed187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:57.967406 24091 raft_consensus.cc:2468] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:15:57.967864 24286 ts_tablet_manager.cc:1434] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.048s	user 0.004s	sys 0.003s
I20260709 12:15:57.956890 24288 ts_tablet_manager.cc:1403] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.429s	user 0.009s	sys 0.014s
I20260709 12:15:57.971412 24287 tablet_bootstrap.cc:492] T bc53eac4ce984dbaa561d9b719143d18 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:15:57.973223 24287 ts_tablet_manager.cc:1403] T bc53eac4ce984dbaa561d9b719143d18 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.159s	user 0.012s	sys 0.013s
I20260709 12:15:57.973444 24359 raft_consensus.cc:515] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:57.976711 24359 leader_election.cc:290] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:57.977454 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55819e09f3e245179f0dd80b9e60eb6a" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:15:57.979553 24241 raft_consensus.cc:3060] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:57.977819 24130 leader_election.cc:304] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:15:57.978297 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55819e09f3e245179f0dd80b9e60eb6a" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:15:57.981771 24332 raft_consensus.cc:515] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:57.983632 23916 catalog_manager.cc:5697] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 0 to 1, leader changed from <none> to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:57.986063 24356 raft_consensus.cc:2804] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:57.986691 24356 raft_consensus.cc:697] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:15:57.987587 24356 consensus_queue.cc:237] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
W20260709 12:15:57.990267 24204 consensus_peers.cc:597] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 -> Peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Couldn't send request to peer 008a79e516614c809c2d3afdb5cbc327. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:15:57.990935 23981 leader_election.cc:343] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:57.993577 24240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc53eac4ce984dbaa561d9b719143d18" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:15:57.994092 24241 raft_consensus.cc:2468] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 1.
W20260709 12:15:57.995652 24058 leader_election.cc:343] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:57.987541 24288 raft_consensus.cc:359] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:57.996389 24288 raft_consensus.cc:385] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:57.996703 24288 raft_consensus.cc:740] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:15:57.996641 23983 leader_election.cc:304] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802; no voters: 6c74a5203b594708921cd793aaca0b8b
I20260709 12:15:57.997812 24359 raft_consensus.cc:2804] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:57.998620 24359 raft_consensus.cc:697] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:15:57.998927 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc53eac4ce984dbaa561d9b719143d18" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:15:57.999478 24286 tablet_bootstrap.cc:492] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:15:57.999611 24018 raft_consensus.cc:3060] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:57.999591 24359 consensus_queue.cc:237] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:58.002264 24332 leader_election.cc:290] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:15:57.997781 24288 consensus_queue.cc:260] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:58.010064 24285 tablet_bootstrap.cc:492] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:15:58.014809 24018 raft_consensus.cc:2468] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:15:58.003551 24287 raft_consensus.cc:359] T bc53eac4ce984dbaa561d9b719143d18 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:58.015609 24285 ts_tablet_manager.cc:1403] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.287s	user 0.020s	sys 0.008s
I20260709 12:15:58.018637 24288 ts_tablet_manager.cc:1434] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.046s	user 0.009s	sys 0.000s
I20260709 12:15:58.020479 24288 tablet_bootstrap.cc:492] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:15:58.016614 24287 raft_consensus.cc:385] T bc53eac4ce984dbaa561d9b719143d18 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.021337 24287 raft_consensus.cc:740] T bc53eac4ce984dbaa561d9b719143d18 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.022320 24056 leader_election.cc:304] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802; no voters: a0df0eff47b64e3eb45ab074139ed187
I20260709 12:15:58.022395 24287 consensus_queue.cc:260] T bc53eac4ce984dbaa561d9b719143d18 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:58.024063 24332 raft_consensus.cc:2804] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:58.024606 24332 raft_consensus.cc:697] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Becoming Leader. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Running, Role: LEADER
I20260709 12:15:58.025874 24287 ts_tablet_manager.cc:1434] T bc53eac4ce984dbaa561d9b719143d18 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.052s	user 0.006s	sys 0.002s
I20260709 12:15:58.025563 24332 consensus_queue.cc:237] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:58.033318 24286 tablet_bootstrap.cc:654] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.032546 23916 catalog_manager.cc:5697] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:58.034063 24356 raft_consensus.cc:493] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.020440 24285 raft_consensus.cc:359] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:58.032691 23912 catalog_manager.cc:5697] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 0 to 1, leader changed from <none> to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:58.035523 24287 tablet_bootstrap.cc:492] T ceb6c32822cf44158ca1ea0d62e0b001 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:15:58.046010 24288 tablet_bootstrap.cc:654] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.035581 24285 raft_consensus.cc:385] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.035795 24356 raft_consensus.cc:515] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:58.058341 23914 catalog_manager.cc:5697] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c74a5203b594708921cd793aaca0b8b (127.11.13.130). New cstate: current_term: 1 leader_uuid: "6c74a5203b594708921cd793aaca0b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:58.059592 24285 raft_consensus.cc:740] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.062635 24285 consensus_queue.cc:260] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:58.064693 24356 leader_election.cc:290] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:15:58.065135 24239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e815126e7554106a097f95846d7f663" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:15:58.066073 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e815126e7554106a097f95846d7f663" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:58.066211 24239 raft_consensus.cc:2393] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 008a79e516614c809c2d3afdb5cbc327 in current term 1: Already voted for candidate c359614416104a7ba9db1ede97d13802 in this term.
I20260709 12:15:58.068832 24132 leader_election.cc:304] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327; no voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802
I20260709 12:15:58.070297 24356 raft_consensus.cc:3060] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.077872 24335 raft_consensus.cc:493] T bc53eac4ce984dbaa561d9b719143d18 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.078567 24335 raft_consensus.cc:515] T bc53eac4ce984dbaa561d9b719143d18 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:58.080654 24356 raft_consensus.cc:2749] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:58.080821 24335 leader_election.cc:290] T bc53eac4ce984dbaa561d9b719143d18 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:58.083181 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc53eac4ce984dbaa561d9b719143d18" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:58.084141 24018 raft_consensus.cc:2393] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0df0eff47b64e3eb45ab074139ed187 in current term 1: Already voted for candidate 6c74a5203b594708921cd793aaca0b8b in this term.
I20260709 12:15:58.086688 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bc53eac4ce984dbaa561d9b719143d18" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:58.087425 24288 tablet_bootstrap.cc:492] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:15:58.088328 24288 ts_tablet_manager.cc:1403] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.068s	user 0.016s	sys 0.005s
I20260709 12:15:58.088887 24287 tablet_bootstrap.cc:654] T ceb6c32822cf44158ca1ea0d62e0b001 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.089095 24204 leader_election.cc:304] T bc53eac4ce984dbaa561d9b719143d18 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:15:58.090456 24335 raft_consensus.cc:3060] T bc53eac4ce984dbaa561d9b719143d18 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.091832 24288 raft_consensus.cc:359] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.092568 24288 raft_consensus.cc:385] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.092921 24288 raft_consensus.cc:740] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.094012 24288 consensus_queue.cc:260] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.095412 24287 tablet_bootstrap.cc:492] T ceb6c32822cf44158ca1ea0d62e0b001 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:15:58.096019 24287 ts_tablet_manager.cc:1403] T ceb6c32822cf44158ca1ea0d62e0b001 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.061s	user 0.012s	sys 0.012s
I20260709 12:15:58.096792 24286 tablet_bootstrap.cc:492] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:15:58.097376 24286 ts_tablet_manager.cc:1403] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.098s	user 0.017s	sys 0.008s
I20260709 12:15:58.098817 24335 raft_consensus.cc:2749] T bc53eac4ce984dbaa561d9b719143d18 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:58.099341 24287 raft_consensus.cc:359] T ceb6c32822cf44158ca1ea0d62e0b001 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:58.101050 24286 raft_consensus.cc:359] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.102015 24287 raft_consensus.cc:385] T ceb6c32822cf44158ca1ea0d62e0b001 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.102077 24286 raft_consensus.cc:385] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.102574 24287 raft_consensus.cc:740] T ceb6c32822cf44158ca1ea0d62e0b001 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.102722 24286 raft_consensus.cc:740] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.103591 24296 consensus_queue.cc:1048] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:58.103793 24287 consensus_queue.cc:260] T ceb6c32822cf44158ca1ea0d62e0b001 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:58.104316 24286 consensus_queue.cc:260] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.108072 24286 ts_tablet_manager.cc:1434] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.010s	user 0.001s	sys 0.010s
I20260709 12:15:58.109354 24286 tablet_bootstrap.cc:492] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:15:58.110613 24288 ts_tablet_manager.cc:1434] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.022s	user 0.004s	sys 0.004s
I20260709 12:15:58.111912 24288 tablet_bootstrap.cc:492] T fbb90520a319469fa1664276964749ea P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:15:58.117525 24285 ts_tablet_manager.cc:1434] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.101s	user 0.004s	sys 0.004s
I20260709 12:15:58.119131 24285 tablet_bootstrap.cc:492] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:15:58.122829 24286 tablet_bootstrap.cc:654] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.125639 24287 ts_tablet_manager.cc:1434] T ceb6c32822cf44158ca1ea0d62e0b001 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.029s	user 0.013s	sys 0.004s
I20260709 12:15:58.126895 24287 tablet_bootstrap.cc:492] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:15:58.132314 24356 raft_consensus.cc:493] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.132957 24356 raft_consensus.cc:515] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.134243 24288 tablet_bootstrap.cc:654] T fbb90520a319469fa1664276964749ea P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.147091 24287 tablet_bootstrap.cc:654] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.148088 24356 leader_election.cc:290] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:58.150295 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf791a63ce7546f38a312c7f77142a61" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
W20260709 12:15:58.153236 24130 leader_election.cc:343] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:58.147418 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf791a63ce7546f38a312c7f77142a61" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
W20260709 12:15:58.156157 24130 leader_election.cc:343] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:58.156736 24130 leader_election.cc:304] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:15:58.161274 24356 raft_consensus.cc:2749] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:58.163332 24285 tablet_bootstrap.cc:654] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.165704 24287 tablet_bootstrap.cc:492] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:15:58.166486 24287 ts_tablet_manager.cc:1403] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.040s	user 0.016s	sys 0.000s
I20260709 12:15:58.169999 24287 raft_consensus.cc:359] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:58.170763 24287 raft_consensus.cc:385] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.171049 24287 raft_consensus.cc:740] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.171260 24286 tablet_bootstrap.cc:492] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:15:58.171792 24286 ts_tablet_manager.cc:1403] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.063s	user 0.010s	sys 0.029s
I20260709 12:15:58.171754 24287 consensus_queue.cc:260] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:58.174276 24287 ts_tablet_manager.cc:1434] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:15:58.175412 24287 tablet_bootstrap.cc:492] T cd2d7fad08bc4376bb38cf21a6a9b611 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:15:58.175181 24286 raft_consensus.cc:359] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:58.175976 24286 raft_consensus.cc:385] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.176265 24286 raft_consensus.cc:740] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.177037 24286 consensus_queue.cc:260] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:58.179652 24286 ts_tablet_manager.cc:1434] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.002s
I20260709 12:15:58.180984 24286 tablet_bootstrap.cc:492] T 3052da843efd4ef4861a13e2240bb466 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:15:58.189400 24285 tablet_bootstrap.cc:492] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:15:58.190110 24285 ts_tablet_manager.cc:1403] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.071s	user 0.011s	sys 0.006s
I20260709 12:15:58.192811 24287 tablet_bootstrap.cc:654] T cd2d7fad08bc4376bb38cf21a6a9b611 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.193563 24285 raft_consensus.cc:359] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.194725 24285 raft_consensus.cc:385] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.195142 24285 raft_consensus.cc:740] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.196163 24285 consensus_queue.cc:260] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.203904 24286 tablet_bootstrap.cc:654] T 3052da843efd4ef4861a13e2240bb466 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.205627 24288 tablet_bootstrap.cc:492] T fbb90520a319469fa1664276964749ea P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:15:58.206427 24288 ts_tablet_manager.cc:1403] T fbb90520a319469fa1664276964749ea P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.095s	user 0.017s	sys 0.022s
I20260709 12:15:58.207029 24332 raft_consensus.cc:493] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.207846 24332 raft_consensus.cc:515] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:58.208660 24285 ts_tablet_manager.cc:1434] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:15:58.209425 24288 raft_consensus.cc:359] T fbb90520a319469fa1664276964749ea P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:58.211552 24332 leader_election.cc:290] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:15:58.211645 24239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55819e09f3e245179f0dd80b9e60eb6a" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:15:58.210184 24288 raft_consensus.cc:385] T fbb90520a319469fa1664276964749ea P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.212145 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55819e09f3e245179f0dd80b9e60eb6a" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:58.218479 24239 raft_consensus.cc:2393] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c74a5203b594708921cd793aaca0b8b in current term 1: Already voted for candidate c359614416104a7ba9db1ede97d13802 in this term.
I20260709 12:15:58.219785 24058 leader_election.cc:304] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802
I20260709 12:15:58.220580 24285 tablet_bootstrap.cc:492] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:15:58.221129 24287 tablet_bootstrap.cc:492] T cd2d7fad08bc4376bb38cf21a6a9b611 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:15:58.221735 24332 raft_consensus.cc:3060] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.221899 24287 ts_tablet_manager.cc:1403] T cd2d7fad08bc4376bb38cf21a6a9b611 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.012s
I20260709 12:15:58.228251 24287 raft_consensus.cc:359] T cd2d7fad08bc4376bb38cf21a6a9b611 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:58.217722 24288 raft_consensus.cc:740] T fbb90520a319469fa1664276964749ea P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.229347 24287 raft_consensus.cc:385] T cd2d7fad08bc4376bb38cf21a6a9b611 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.229840 24287 raft_consensus.cc:740] T cd2d7fad08bc4376bb38cf21a6a9b611 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.230615 24285 tablet_bootstrap.cc:654] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.231289 24287 consensus_queue.cc:260] T cd2d7fad08bc4376bb38cf21a6a9b611 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:58.233657 24288 consensus_queue.cc:260] T fbb90520a319469fa1664276964749ea P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:58.235062 24287 ts_tablet_manager.cc:1434] T cd2d7fad08bc4376bb38cf21a6a9b611 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 12:15:58.236699 24287 tablet_bootstrap.cc:492] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:15:58.236817 24288 ts_tablet_manager.cc:1434] T fbb90520a319469fa1664276964749ea P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 12:15:58.239159 24288 tablet_bootstrap.cc:492] T e679d0d8ffd84a23afc6d7f8faf84ca6 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:15:58.237654 24332 raft_consensus.cc:2749] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:58.247520 24288 tablet_bootstrap.cc:654] T e679d0d8ffd84a23afc6d7f8faf84ca6 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.255077 24286 tablet_bootstrap.cc:492] T 3052da843efd4ef4861a13e2240bb466 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:15:58.255827 24286 ts_tablet_manager.cc:1403] T 3052da843efd4ef4861a13e2240bb466 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.075s	user 0.025s	sys 0.018s
I20260709 12:15:58.261684 24287 tablet_bootstrap.cc:654] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.263796 24286 raft_consensus.cc:359] T 3052da843efd4ef4861a13e2240bb466 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:58.264673 24286 raft_consensus.cc:385] T 3052da843efd4ef4861a13e2240bb466 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.265025 24286 raft_consensus.cc:740] T 3052da843efd4ef4861a13e2240bb466 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.265985 24286 consensus_queue.cc:260] T 3052da843efd4ef4861a13e2240bb466 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:58.268558 24285 tablet_bootstrap.cc:492] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:15:58.268682 24288 tablet_bootstrap.cc:492] T e679d0d8ffd84a23afc6d7f8faf84ca6 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:15:58.268678 24287 tablet_bootstrap.cc:492] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:15:58.269224 24285 ts_tablet_manager.cc:1403] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.049s	user 0.013s	sys 0.005s
I20260709 12:15:58.269356 24288 ts_tablet_manager.cc:1403] T e679d0d8ffd84a23afc6d7f8faf84ca6 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.031s	user 0.019s	sys 0.008s
I20260709 12:15:58.269590 24287 ts_tablet_manager.cc:1403] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.033s	user 0.007s	sys 0.006s
I20260709 12:15:58.271550 24286 ts_tablet_manager.cc:1434] T 3052da843efd4ef4861a13e2240bb466 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.002s
I20260709 12:15:58.272933 24286 tablet_bootstrap.cc:492] T 0015560e665c48f0bd17fe4be5c8761d P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:15:58.272627 24288 raft_consensus.cc:359] T e679d0d8ffd84a23afc6d7f8faf84ca6 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:58.273432 24288 raft_consensus.cc:385] T e679d0d8ffd84a23afc6d7f8faf84ca6 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.273839 24288 raft_consensus.cc:740] T e679d0d8ffd84a23afc6d7f8faf84ca6 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.274696 24288 consensus_queue.cc:260] T e679d0d8ffd84a23afc6d7f8faf84ca6 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:58.277098 24288 ts_tablet_manager.cc:1434] T e679d0d8ffd84a23afc6d7f8faf84ca6 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:15:58.273347 24287 raft_consensus.cc:359] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:58.277621 24287 raft_consensus.cc:385] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.278007 24287 raft_consensus.cc:740] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.278898 24288 tablet_bootstrap.cc:492] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:15:58.278851 24287 consensus_queue.cc:260] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:58.281214 24287 ts_tablet_manager.cc:1434] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.000s
I20260709 12:15:58.282315 24287 tablet_bootstrap.cc:492] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:15:58.287070 24288 tablet_bootstrap.cc:654] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.289495 24287 tablet_bootstrap.cc:654] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.292599 24332 raft_consensus.cc:493] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.293411 24332 raft_consensus.cc:515] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:58.295063 24286 tablet_bootstrap.cc:654] T 0015560e665c48f0bd17fe4be5c8761d P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.297338 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd2d7fad08bc4376bb38cf21a6a9b611" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:15:58.298134 24167 raft_consensus.cc:2468] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:15:58.299767 24056 leader_election.cc:304] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: 
I20260709 12:15:58.300776 24367 raft_consensus.cc:2804] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:58.301170 24367 raft_consensus.cc:493] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:58.301518 24367 raft_consensus.cc:3060] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.304524 24285 raft_consensus.cc:359] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.305292 24285 raft_consensus.cc:385] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.305585 24285 raft_consensus.cc:740] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.306433 24285 consensus_queue.cc:260] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.308668 24285 ts_tablet_manager.cc:1434] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.039s	user 0.003s	sys 0.003s
I20260709 12:15:58.310595 24285 tablet_bootstrap.cc:492] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:15:58.310720 24239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd2d7fad08bc4376bb38cf21a6a9b611" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:15:58.311904 24239 raft_consensus.cc:2468] T cd2d7fad08bc4376bb38cf21a6a9b611 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:15:58.312142 24367 raft_consensus.cc:515] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:58.314491 24359 raft_consensus.cc:493] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.315568 24359 raft_consensus.cc:515] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:58.318081 24359 leader_election.cc:290] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:58.318423 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd2d7fad08bc4376bb38cf21a6a9b611" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:15:58.319108 24167 raft_consensus.cc:3060] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.321233 24239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16f5650964ece90109902b6b40882" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:15:58.322602 24285 tablet_bootstrap.cc:654] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.323771 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16f5650964ece90109902b6b40882" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:58.324719 24093 raft_consensus.cc:2393] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c359614416104a7ba9db1ede97d13802 in current term 1: Already voted for candidate 6c74a5203b594708921cd793aaca0b8b in this term.
I20260709 12:15:58.325387 24332 leader_election.cc:290] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:15:58.322872 24239 raft_consensus.cc:2393] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c359614416104a7ba9db1ede97d13802 in current term 1: Already voted for candidate a0df0eff47b64e3eb45ab074139ed187 in this term.
I20260709 12:15:58.330660 23983 leader_election.cc:304] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:15:58.331956 24359 raft_consensus.cc:3060] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.344043 24167 raft_consensus.cc:2468] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:15:58.349026 24287 tablet_bootstrap.cc:492] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:15:58.346948 24239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cd2d7fad08bc4376bb38cf21a6a9b611" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:15:58.346820 24356 consensus_queue.cc:1048] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:58.348522 24359 raft_consensus.cc:2749] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:58.349375 24056 leader_election.cc:304] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: 
I20260709 12:15:58.345026 24367 leader_election.cc:290] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Requested vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:15:58.351042 24367 raft_consensus.cc:2804] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:58.351223 24239 raft_consensus.cc:3060] T cd2d7fad08bc4376bb38cf21a6a9b611 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.351914 24367 raft_consensus.cc:697] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Becoming Leader. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Running, Role: LEADER
I20260709 12:15:58.353363 24367 consensus_queue.cc:237] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:58.359315 24286 tablet_bootstrap.cc:492] T 0015560e665c48f0bd17fe4be5c8761d P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:15:58.348644 24288 tablet_bootstrap.cc:492] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:15:58.360720 24288 ts_tablet_manager.cc:1403] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.082s	user 0.012s	sys 0.003s
I20260709 12:15:58.361392 24286 ts_tablet_manager.cc:1403] T 0015560e665c48f0bd17fe4be5c8761d P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.089s	user 0.010s	sys 0.006s
I20260709 12:15:58.361820 24287 ts_tablet_manager.cc:1403] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.080s	user 0.025s	sys 0.040s
I20260709 12:15:58.362201 24239 raft_consensus.cc:2468] T cd2d7fad08bc4376bb38cf21a6a9b611 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:15:58.366696 24287 raft_consensus.cc:359] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:58.369622 24287 raft_consensus.cc:385] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.370187 24287 raft_consensus.cc:740] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.371474 24287 consensus_queue.cc:260] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:58.372578 24286 raft_consensus.cc:359] T 0015560e665c48f0bd17fe4be5c8761d P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.373355 24286 raft_consensus.cc:385] T 0015560e665c48f0bd17fe4be5c8761d P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.373687 24286 raft_consensus.cc:740] T 0015560e665c48f0bd17fe4be5c8761d P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.374866 24287 ts_tablet_manager.cc:1434] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:15:58.374717 24286 consensus_queue.cc:260] T 0015560e665c48f0bd17fe4be5c8761d P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.376310 24287 tablet_bootstrap.cc:492] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:15:58.384539 24287 tablet_bootstrap.cc:654] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.384881 24286 ts_tablet_manager.cc:1434] T 0015560e665c48f0bd17fe4be5c8761d P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 12:15:58.386296 24286 tablet_bootstrap.cc:492] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:15:58.386613 23916 catalog_manager.cc:5697] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c74a5203b594708921cd793aaca0b8b (127.11.13.130). New cstate: current_term: 1 leader_uuid: "6c74a5203b594708921cd793aaca0b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:58.369462 24288 raft_consensus.cc:359] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.391621 24288 raft_consensus.cc:385] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.392078 24288 raft_consensus.cc:740] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.392970 24288 consensus_queue.cc:260] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.397496 24286 tablet_bootstrap.cc:654] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.398914 24285 tablet_bootstrap.cc:492] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:15:58.399684 24285 ts_tablet_manager.cc:1403] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.089s	user 0.022s	sys 0.018s
I20260709 12:15:58.410205 24288 ts_tablet_manager.cc:1434] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.049s	user 0.007s	sys 0.002s
I20260709 12:15:58.412003 24288 tablet_bootstrap.cc:492] T ab48aa491dc84193a39fdbf1a1c52927 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:15:58.416044 24285 raft_consensus.cc:359] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:58.416870 24285 raft_consensus.cc:385] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.417223 24285 raft_consensus.cc:740] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.424602 24285 consensus_queue.cc:260] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:58.427541 24239 raft_consensus.cc:3060] T ceb6c32822cf44158ca1ea0d62e0b001 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.428313 24287 tablet_bootstrap.cc:492] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:15:58.428856 24287 ts_tablet_manager.cc:1403] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.053s	user 0.011s	sys 0.018s
I20260709 12:15:58.429029 24367 raft_consensus.cc:493] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.429636 24367 raft_consensus.cc:515] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.430110 24288 tablet_bootstrap.cc:654] T ab48aa491dc84193a39fdbf1a1c52927 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.432263 24367 leader_election.cc:290] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:15:58.435578 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5587d307c2f64ab9a9b39c1b190e85be" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:15:58.439131 24285 ts_tablet_manager.cc:1434] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.039s	user 0.009s	sys 0.000s
I20260709 12:15:58.441784 24018 raft_consensus.cc:3060] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.445694 24287 raft_consensus.cc:359] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:58.446238 24167 raft_consensus.cc:3060] T fbb90520a319469fa1664276964749ea P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.446611 24287 raft_consensus.cc:385] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.446318 24356 consensus_queue.cc:1048] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:58.447022 24287 raft_consensus.cc:740] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.447839 24287 consensus_queue.cc:260] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:58.449524 24375 raft_consensus.cc:493] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.450040 24375 raft_consensus.cc:515] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:58.451097 24287 ts_tablet_manager.cc:1434] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.022s	user 0.007s	sys 0.002s
I20260709 12:15:58.451592 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5587d307c2f64ab9a9b39c1b190e85be" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:58.452283 24287 tablet_bootstrap.cc:492] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:15:58.460394 24287 tablet_bootstrap.cc:654] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.460136 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "802231ce402445309306d2cbffa2f92c" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:58.463518 24056 leader_election.cc:304] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:15:58.462373 24285 tablet_bootstrap.cc:492] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:15:58.460769 24335 consensus_queue.cc:1048] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:15:58.465831 24375 leader_election.cc:290] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:15:58.466336 24356 consensus_queue.cc:1048] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:15:58.468047 24367 raft_consensus.cc:2749] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:58.474823 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "802231ce402445309306d2cbffa2f92c" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
W20260709 12:15:58.475652 24204 leader_election.cc:343] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:58.477249 24288 tablet_bootstrap.cc:492] T ab48aa491dc84193a39fdbf1a1c52927 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:15:58.478704 24367 consensus_queue.cc:1048] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260709 12:15:58.482425 24093 raft_consensus.cc:3060] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.494982 24345 consensus_queue.cc:1048] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:58.495055 24335 raft_consensus.cc:493] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.497291 24335 raft_consensus.cc:515] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:58.499624 24335 leader_election.cc:290] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:58.500217 24335 consensus_queue.cc:1048] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:58.477867 24288 ts_tablet_manager.cc:1403] T ab48aa491dc84193a39fdbf1a1c52927 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.066s	user 0.010s	sys 0.004s
I20260709 12:15:58.502029 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3052da843efd4ef4861a13e2240bb466" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:58.508262 24091 raft_consensus.cc:2468] T 3052da843efd4ef4861a13e2240bb466 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 0.
I20260709 12:15:58.510735 24204 leader_election.cc:304] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:15:58.512068 24375 raft_consensus.cc:2804] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:58.512566 24375 raft_consensus.cc:493] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:58.512993 24375 raft_consensus.cc:3060] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.513816 24288 raft_consensus.cc:359] T ab48aa491dc84193a39fdbf1a1c52927 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:58.514925 24288 raft_consensus.cc:385] T ab48aa491dc84193a39fdbf1a1c52927 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.515100 24287 tablet_bootstrap.cc:492] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:15:58.515347 24288 raft_consensus.cc:740] T ab48aa491dc84193a39fdbf1a1c52927 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.515949 24287 ts_tablet_manager.cc:1403] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.064s	user 0.018s	sys 0.002s
I20260709 12:15:58.516338 24288 consensus_queue.cc:260] T ab48aa491dc84193a39fdbf1a1c52927 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:58.527344 24288 ts_tablet_manager.cc:1434] T ab48aa491dc84193a39fdbf1a1c52927 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 12:15:58.528815 24288 tablet_bootstrap.cc:492] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:15:58.528640 24345 consensus_queue.cc:1048] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:58.536689 24288 tablet_bootstrap.cc:654] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.542543 24287 raft_consensus.cc:359] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:58.543457 24287 raft_consensus.cc:385] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.543799 24287 raft_consensus.cc:740] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.544659 24287 consensus_queue.cc:260] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:58.549338 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3052da843efd4ef4861a13e2240bb466" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:58.553043 24351 consensus_queue.cc:1048] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:58.554561 24286 tablet_bootstrap.cc:492] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:15:58.555140 24286 ts_tablet_manager.cc:1403] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.169s	user 0.015s	sys 0.009s
W20260709 12:15:58.555083 24204 leader_election.cc:343] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:58.561985 24286 raft_consensus.cc:359] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.562769 24286 raft_consensus.cc:385] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.563055 24286 raft_consensus.cc:740] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.563748 24286 consensus_queue.cc:260] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.568959 24288 tablet_bootstrap.cc:492] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:15:58.569571 24288 ts_tablet_manager.cc:1403] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.041s	user 0.018s	sys 0.002s
W20260709 12:15:58.571619 24204 leader_election.cc:343] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:58.572431 24288 raft_consensus.cc:359] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.572489 24345 consensus_queue.cc:1048] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:58.573182 24288 raft_consensus.cc:385] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.573541 24288 raft_consensus.cc:740] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.574232 24288 consensus_queue.cc:260] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.572190 24204 leader_election.cc:304] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b
I20260709 12:15:58.575729 24286 ts_tablet_manager.cc:1434] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.020s	user 0.007s	sys 0.001s
I20260709 12:15:58.576088 24296 raft_consensus.cc:2749] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:58.576584 24288 ts_tablet_manager.cc:1434] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.004s
I20260709 12:15:58.577857 24288 tablet_bootstrap.cc:492] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:15:58.581594 24285 tablet_bootstrap.cc:654] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.585129 24375 raft_consensus.cc:515] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:58.587811 24288 tablet_bootstrap.cc:654] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.592020 24287 ts_tablet_manager.cc:1434] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.076s	user 0.009s	sys 0.000s
I20260709 12:15:58.593509 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3052da843efd4ef4861a13e2240bb466" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:15:58.594033 24367 consensus_queue.cc:1048] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:58.594456 24093 raft_consensus.cc:3060] T 3052da843efd4ef4861a13e2240bb466 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.596673 24166 raft_consensus.cc:3060] T e679d0d8ffd84a23afc6d7f8faf84ca6 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.593652 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3052da843efd4ef4861a13e2240bb466" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:15:58.603325 24093 raft_consensus.cc:2468] T 3052da843efd4ef4861a13e2240bb466 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 1.
I20260709 12:15:58.593508 24287 tablet_bootstrap.cc:492] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:15:58.610090 24204 leader_election.cc:304] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:15:58.611171 24296 raft_consensus.cc:2804] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:58.611732 24296 raft_consensus.cc:697] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Becoming Leader. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Running, Role: LEADER
I20260709 12:15:58.612874 24351 raft_consensus.cc:493] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.612700 24296 consensus_queue.cc:237] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:58.613598 24351 raft_consensus.cc:515] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.576840 24286 tablet_bootstrap.cc:492] T 802231ce402445309306d2cbffa2f92c P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:15:58.619417 24351 leader_election.cc:290] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
W20260709 12:15:58.620232 24204 leader_election.cc:343] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:58.620428 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35efdb2f1c24db0a45fbc04abea260e" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
W20260709 12:15:58.622617 24130 leader_election.cc:343] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:58.587738 24375 leader_election.cc:290] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:58.631043 24287 tablet_bootstrap.cc:654] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.630924 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35efdb2f1c24db0a45fbc04abea260e" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:58.630982 24286 tablet_bootstrap.cc:654] T 802231ce402445309306d2cbffa2f92c P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
W20260709 12:15:58.634950 24130 leader_election.cc:343] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:58.638439 24130 leader_election.cc:304] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:15:58.640043 24351 raft_consensus.cc:2749] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:58.637668 24285 tablet_bootstrap.cc:492] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:15:58.644106 24285 ts_tablet_manager.cc:1403] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.182s	user 0.014s	sys 0.001s
I20260709 12:15:58.648362 24285 raft_consensus.cc:359] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.657500 24285 raft_consensus.cc:385] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.659456 24285 raft_consensus.cc:740] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.661628 24288 tablet_bootstrap.cc:492] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:15:58.662406 24288 ts_tablet_manager.cc:1403] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.085s	user 0.029s	sys 0.007s
I20260709 12:15:58.666280 24288 raft_consensus.cc:359] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:58.667244 24288 raft_consensus.cc:385] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.667598 24288 raft_consensus.cc:740] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.668404 24296 raft_consensus.cc:493] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.669109 24296 raft_consensus.cc:515] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:58.672473 24296 leader_election.cc:290] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:15:58.675786 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab48aa491dc84193a39fdbf1a1c52927" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:58.676842 24017 raft_consensus.cc:2468] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 0.
I20260709 12:15:58.678584 24204 leader_election.cc:304] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:15:58.679661 24422 raft_consensus.cc:493] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.679710 24296 raft_consensus.cc:2804] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:58.680392 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab48aa491dc84193a39fdbf1a1c52927" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:15:58.681473 24288 consensus_queue.cc:260] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:58.683440 24296 raft_consensus.cc:493] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:58.684065 24296 raft_consensus.cc:3060] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.685989 24288 ts_tablet_manager.cc:1434] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:15:58.686664 24166 raft_consensus.cc:2468] T ab48aa491dc84193a39fdbf1a1c52927 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 0.
I20260709 12:15:58.689855 24288 tablet_bootstrap.cc:492] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:15:58.661270 24285 consensus_queue.cc:260] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.705309 24288 tablet_bootstrap.cc:654] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.683386 24422 raft_consensus.cc:515] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:58.709774 24422 leader_election.cc:290] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:15:58.710601 24351 raft_consensus.cc:493] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.715495 24351 raft_consensus.cc:515] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.711457 24240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab48aa491dc84193a39fdbf1a1c52927" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:15:58.713352 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab48aa491dc84193a39fdbf1a1c52927" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:15:58.718779 24166 raft_consensus.cc:2468] T ab48aa491dc84193a39fdbf1a1c52927 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 0.
I20260709 12:15:58.721598 24296 raft_consensus.cc:515] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:58.726718 23915 catalog_manager.cc:5697] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132). New cstate: current_term: 1 leader_uuid: "a0df0eff47b64e3eb45ab074139ed187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:58.722054 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0015560e665c48f0bd17fe4be5c8761d" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:58.729487 24018 raft_consensus.cc:2468] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:15:58.722561 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0015560e665c48f0bd17fe4be5c8761d" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:58.730501 24093 raft_consensus.cc:2468] T 0015560e665c48f0bd17fe4be5c8761d P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:15:58.718052 24351 leader_election.cc:290] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:58.731662 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab48aa491dc84193a39fdbf1a1c52927" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:15:58.724598 23981 leader_election.cc:304] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:15:58.732635 24166 raft_consensus.cc:3060] T ab48aa491dc84193a39fdbf1a1c52927 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.722775 24412 raft_consensus.cc:493] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.733640 24412 raft_consensus.cc:515] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.731732 24130 leader_election.cc:304] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:15:58.736160 24296 leader_election.cc:290] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Requested vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:15:58.738350 24351 raft_consensus.cc:2804] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:58.738938 24351 raft_consensus.cc:493] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:58.739346 24351 raft_consensus.cc:3060] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.739722 24418 raft_consensus.cc:2804] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:58.740195 24418 raft_consensus.cc:493] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:58.740547 24418 raft_consensus.cc:3060] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.728726 24240 raft_consensus.cc:2393] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c359614416104a7ba9db1ede97d13802 in current term 1: Already voted for candidate a0df0eff47b64e3eb45ab074139ed187 in this term.
I20260709 12:15:58.743773 24345 consensus_queue.cc:1048] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:58.744870 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab48aa491dc84193a39fdbf1a1c52927" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:15:58.745352 24345 raft_consensus.cc:493] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.767863 24296 raft_consensus.cc:493] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.745884 24345 raft_consensus.cc:515] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.768535 24296 raft_consensus.cc:515] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:58.769285 24288 tablet_bootstrap.cc:492] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:15:58.769825 24288 ts_tablet_manager.cc:1403] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.080s	user 0.019s	sys 0.010s
I20260709 12:15:58.771196 24296 leader_election.cc:290] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:15:58.771548 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf791a63ce7546f38a312c7f77142a61" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:58.772305 24091 raft_consensus.cc:2468] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 0.
I20260709 12:15:58.772997 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0756db96a05a45bc9ebc1f91ee907496" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:15:58.774325 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0756db96a05a45bc9ebc1f91ee907496" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:58.774420 23981 leader_election.cc:304] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802; no voters: 
W20260709 12:15:58.775300 24204 leader_election.cc:343] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:15:58.777666 24204 leader_election.cc:343] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:58.778294 24204 leader_election.cc:304] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:15:58.780807 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf791a63ce7546f38a312c7f77142a61" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:58.782608 24017 raft_consensus.cc:2468] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:15:58.782346 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf791a63ce7546f38a312c7f77142a61" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:15:58.783116 24412 leader_election.cc:290] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:15:58.783419 24167 raft_consensus.cc:2468] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:15:58.783748 24375 raft_consensus.cc:2749] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:58.785336 24056 leader_election.cc:304] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:15:58.786154 24286 tablet_bootstrap.cc:492] T 802231ce402445309306d2cbffa2f92c P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:15:58.785840 24288 raft_consensus.cc:359] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:58.786607 24288 raft_consensus.cc:385] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.786691 24286 ts_tablet_manager.cc:1403] T 802231ce402445309306d2cbffa2f92c P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.210s	user 0.007s	sys 0.051s
I20260709 12:15:58.786921 24288 raft_consensus.cc:740] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.787060 24423 raft_consensus.cc:2804] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:58.787535 24423 raft_consensus.cc:493] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:58.787930 24423 raft_consensus.cc:3060] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.787855 24288 consensus_queue.cc:260] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:58.791932 24288 ts_tablet_manager.cc:1434] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.022s	user 0.010s	sys 0.000s
I20260709 12:15:58.789775 24286 raft_consensus.cc:359] T 802231ce402445309306d2cbffa2f92c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:58.793781 24286 raft_consensus.cc:385] T 802231ce402445309306d2cbffa2f92c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.794104 24286 raft_consensus.cc:740] T 802231ce402445309306d2cbffa2f92c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.794586 24412 raft_consensus.cc:2804] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:58.795200 24285 ts_tablet_manager.cc:1434] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.150s	user 0.010s	sys 0.003s
I20260709 12:15:58.796907 24285 tablet_bootstrap.cc:492] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:15:58.800109 24288 tablet_bootstrap.cc:492] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:15:58.794943 24286 consensus_queue.cc:260] T 802231ce402445309306d2cbffa2f92c P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:58.795135 24412 raft_consensus.cc:493] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:58.807402 24412 raft_consensus.cc:3060] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.814930 24351 raft_consensus.cc:515] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.818490 24351 leader_election.cc:290] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:58.820430 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0015560e665c48f0bd17fe4be5c8761d" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:15:58.821409 24017 raft_consensus.cc:3060] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.823944 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0015560e665c48f0bd17fe4be5c8761d" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:15:58.824855 24091 raft_consensus.cc:3060] T 0015560e665c48f0bd17fe4be5c8761d P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.773198 24166 raft_consensus.cc:2468] T ab48aa491dc84193a39fdbf1a1c52927 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 1.
I20260709 12:15:58.843590 24286 ts_tablet_manager.cc:1434] T 802231ce402445309306d2cbffa2f92c P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.057s	user 0.008s	sys 0.000s
I20260709 12:15:58.846002 24286 tablet_bootstrap.cc:492] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:15:58.848680 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf791a63ce7546f38a312c7f77142a61" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:15:58.849324 24204 leader_election.cc:304] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:15:58.849758 24166 raft_consensus.cc:2468] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 0.
I20260709 12:15:58.850834 24296 raft_consensus.cc:2804] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:58.851496 24296 raft_consensus.cc:697] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Becoming Leader. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Running, Role: LEADER
I20260709 12:15:58.852411 24296 consensus_queue.cc:237] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:58.860299 24287 tablet_bootstrap.cc:492] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:15:58.861301 24287 ts_tablet_manager.cc:1403] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.268s	user 0.033s	sys 0.067s
I20260709 12:15:58.866195 24375 raft_consensus.cc:493] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.866729 24375 raft_consensus.cc:515] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:58.868515 24287 raft_consensus.cc:359] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:58.868969 24375 leader_election.cc:290] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:58.869534 24287 raft_consensus.cc:385] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.869946 24287 raft_consensus.cc:740] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.870735 24287 consensus_queue.cc:260] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:58.872160 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91f626cde148838a6a979d4fd1a5ed" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:58.873371 24287 ts_tablet_manager.cc:1434] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.003s
I20260709 12:15:58.874557 24287 tablet_bootstrap.cc:492] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:15:58.860163 24017 raft_consensus.cc:2468] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:15:58.858582 24367 consensus_queue.cc:1048] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
W20260709 12:15:58.875303 24204 leader_election.cc:343] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:58.861105 24423 raft_consensus.cc:515] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.865278 24288 tablet_bootstrap.cc:654] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.878159 24130 leader_election.cc:304] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:15:58.879266 24356 raft_consensus.cc:2804] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:58.879814 24356 raft_consensus.cc:697] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:15:58.881146 24286 tablet_bootstrap.cc:654] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.881093 24356 consensus_queue.cc:237] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
W20260709 12:15:58.888203 23963 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:15:58.883869 24423 leader_election.cc:290] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:15:58.884590 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf791a63ce7546f38a312c7f77142a61" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:15:58.870936 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91f626cde148838a6a979d4fd1a5ed" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:58.882787 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf791a63ce7546f38a312c7f77142a61" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:15:58.845249 24345 leader_election.cc:290] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:15:58.905093 24412 raft_consensus.cc:515] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.907655 24412 leader_election.cc:290] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:15:58.910082 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf791a63ce7546f38a312c7f77142a61" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:15:58.914187 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf791a63ce7546f38a312c7f77142a61" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:15:58.915508 24017 raft_consensus.cc:2393] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c74a5203b594708921cd793aaca0b8b in current term 1: Already voted for candidate c359614416104a7ba9db1ede97d13802 in this term.
I20260709 12:15:58.915625 23913 catalog_manager.cc:5697] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132). New cstate: current_term: 1 leader_uuid: "a0df0eff47b64e3eb45ab074139ed187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } }
W20260709 12:15:58.916971 24204 leader_election.cc:343] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:58.917158 24056 leader_election.cc:304] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:15:58.917754 24204 leader_election.cc:304] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:15:58.913901 24090 raft_consensus.cc:2393] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c359614416104a7ba9db1ede97d13802 in current term 1: Already voted for candidate 6c74a5203b594708921cd793aaca0b8b in this term.
I20260709 12:15:58.919382 24367 raft_consensus.cc:2749] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:15:58.919453 24296 raft_consensus.cc:2749] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:58.900336 24167 raft_consensus.cc:3060] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.924496 24091 raft_consensus.cc:2468] T 0015560e665c48f0bd17fe4be5c8761d P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:15:58.930114 24367 consensus_queue.cc:1048] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:15:58.936489 24418 raft_consensus.cc:515] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:58.937676 24018 raft_consensus.cc:2393] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a0df0eff47b64e3eb45ab074139ed187 in current term 1: Already voted for candidate c359614416104a7ba9db1ede97d13802 in this term.
I20260709 12:15:58.941273 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab48aa491dc84193a39fdbf1a1c52927" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:15:58.942296 24165 raft_consensus.cc:2393] T ab48aa491dc84193a39fdbf1a1c52927 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c359614416104a7ba9db1ede97d13802 in current term 1: Already voted for candidate a0df0eff47b64e3eb45ab074139ed187 in this term.
I20260709 12:15:58.943889 24240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab48aa491dc84193a39fdbf1a1c52927" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:15:58.944347 24287 tablet_bootstrap.cc:654] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.946285 24348 raft_consensus.cc:493] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:58.953292 23981 leader_election.cc:304] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:15:58.943945 24418 leader_election.cc:290] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Requested vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:15:58.957253 24348 raft_consensus.cc:515] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:58.935506 24167 raft_consensus.cc:2468] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 1.
I20260709 12:15:58.960826 24285 tablet_bootstrap.cc:654] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:58.957890 24418 raft_consensus.cc:2749] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:15:58.964329 24240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "802231ce402445309306d2cbffa2f92c" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:15:58.965250 24240 raft_consensus.cc:2468] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:15:58.967790 24286 tablet_bootstrap.cc:492] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:15:58.968276 24286 ts_tablet_manager.cc:1403] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.122s	user 0.029s	sys 0.005s
I20260709 12:15:58.962148 24348 leader_election.cc:290] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:15:58.971045 24132 leader_election.cc:304] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:15:58.969998 23981 leader_election.cc:304] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 6c74a5203b594708921cd793aaca0b8b
I20260709 12:15:58.974548 24348 raft_consensus.cc:2804] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:58.974967 24348 raft_consensus.cc:493] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:58.978530 24288 tablet_bootstrap.cc:492] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:15:58.979177 24288 ts_tablet_manager.cc:1403] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.186s	user 0.008s	sys 0.024s
I20260709 12:15:58.979252 24418 raft_consensus.cc:2804] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:58.980470 24418 raft_consensus.cc:697] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:15:58.981559 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "802231ce402445309306d2cbffa2f92c" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:58.981841 24418 consensus_queue.cc:237] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.982453 24288 raft_consensus.cc:359] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:58.983292 24288 raft_consensus.cc:385] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.983628 24288 raft_consensus.cc:740] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.982322 24091 raft_consensus.cc:2468] T 802231ce402445309306d2cbffa2f92c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:15:58.984428 24288 consensus_queue.cc:260] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:58.975333 24348 raft_consensus.cc:3060] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:58.993055 24288 ts_tablet_manager.cc:1434] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:15:58.992775 23911 catalog_manager.cc:5697] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:58.994387 24286 raft_consensus.cc:359] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:58.995302 24286 raft_consensus.cc:385] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:58.995635 24286 raft_consensus.cc:740] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:15:58.995735 24288 tablet_bootstrap.cc:492] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:15:58.996685 24286 consensus_queue.cc:260] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:59.001390 24286 ts_tablet_manager.cc:1434] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.033s	user 0.010s	sys 0.000s
I20260709 12:15:59.003118 24286 tablet_bootstrap.cc:492] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:15:59.007448 24285 tablet_bootstrap.cc:492] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:15:59.008045 24285 ts_tablet_manager.cc:1403] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.211s	user 0.014s	sys 0.001s
I20260709 12:15:59.021166 24285 raft_consensus.cc:359] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.021910 24285 raft_consensus.cc:385] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:59.022264 24285 raft_consensus.cc:740] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:15:59.023092 24285 consensus_queue.cc:260] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.025156 24288 tablet_bootstrap.cc:654] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:59.027276 24348 raft_consensus.cc:515] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:59.028024 24287 tablet_bootstrap.cc:492] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:15:59.029587 24287 ts_tablet_manager.cc:1403] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.155s	user 0.017s	sys 0.003s
I20260709 12:15:59.031687 24285 ts_tablet_manager.cc:1434] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.002s
I20260709 12:15:59.032538 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "802231ce402445309306d2cbffa2f92c" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:15:59.032936 24285 tablet_bootstrap.cc:492] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:15:59.033277 24090 raft_consensus.cc:3060] T 802231ce402445309306d2cbffa2f92c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:59.032814 24287 raft_consensus.cc:359] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:59.033847 24287 raft_consensus.cc:385] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:59.034250 24287 raft_consensus.cc:740] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:15:59.035259 24287 consensus_queue.cc:260] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:59.037436 24417 raft_consensus.cc:493] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.037844 24287 ts_tablet_manager.cc:1434] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:15:59.038025 24417 raft_consensus.cc:515] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.039002 24287 tablet_bootstrap.cc:492] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:15:59.041175 24296 raft_consensus.cc:493] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.041744 24296 raft_consensus.cc:515] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.043942 24296 leader_election.cc:290] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:59.044214 24090 raft_consensus.cc:2468] T 802231ce402445309306d2cbffa2f92c P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:15:59.046496 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16f5650964ece90109902b6b40882" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:59.053817 24287 tablet_bootstrap.cc:654] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:59.054256 24018 raft_consensus.cc:2468] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 1.
I20260709 12:15:59.054713 24130 leader_election.cc:304] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: 
I20260709 12:15:59.055593 24240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "802231ce402445309306d2cbffa2f92c" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:15:59.055908 24432 raft_consensus.cc:2804] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:59.056195 24240 raft_consensus.cc:3060] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:59.056547 24432 raft_consensus.cc:697] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:15:59.057466 24432 consensus_queue.cc:237] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:59.049981 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3052da843efd4ef4861a13e2240bb466" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:59.046923 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16f5650964ece90109902b6b40882" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:59.049134 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3052da843efd4ef4861a13e2240bb466" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:15:59.059747 24093 raft_consensus.cc:2393] T 3052da843efd4ef4861a13e2240bb466 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c359614416104a7ba9db1ede97d13802 in current term 1: Already voted for candidate a0df0eff47b64e3eb45ab074139ed187 in this term.
I20260709 12:15:59.059664 24091 raft_consensus.cc:2468] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 1.
I20260709 12:15:59.061527 23981 leader_election.cc:304] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:15:59.051905 23913 catalog_manager.cc:5697] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 0 to 1, leader changed from <none> to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:59.058672 24204 leader_election.cc:304] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:15:59.053095 24348 leader_election.cc:290] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:15:59.064036 24423 raft_consensus.cc:3060] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:59.065641 24285 tablet_bootstrap.cc:654] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:59.069154 24296 raft_consensus.cc:2804] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:15:59.050077 24417 leader_election.cc:290] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:59.076797 24286 tablet_bootstrap.cc:654] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:59.080159 24285 tablet_bootstrap.cc:492] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:15:59.077827 24296 raft_consensus.cc:493] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:59.080694 24285 ts_tablet_manager.cc:1403] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.048s	user 0.014s	sys 0.002s
I20260709 12:15:59.080791 24296 raft_consensus.cc:3060] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:59.083817 24285 raft_consensus.cc:359] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.084591 24335 raft_consensus.cc:493] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.085232 24335 raft_consensus.cc:515] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:59.084748 24285 raft_consensus.cc:385] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:59.085985 24285 raft_consensus.cc:740] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:15:59.086915 24285 consensus_queue.cc:260] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.087311 24287 tablet_bootstrap.cc:492] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:15:59.087849 24287 ts_tablet_manager.cc:1403] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.049s	user 0.012s	sys 0.000s
I20260709 12:15:59.088019 24335 leader_election.cc:290] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:15:59.089913 24285 ts_tablet_manager.cc:1434] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:15:59.090062 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21f0988e53da475c8274bcb13b34b3a8" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:59.091238 24285 tablet_bootstrap.cc:492] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:15:59.090893 24287 raft_consensus.cc:359] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.091682 24287 raft_consensus.cc:385] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:59.091969 24287 raft_consensus.cc:740] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:15:59.092393 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21f0988e53da475c8274bcb13b34b3a8" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:15:59.092749 24287 consensus_queue.cc:260] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.095567 24287 ts_tablet_manager.cc:1434] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:15:59.096906 24287 tablet_bootstrap.cc:492] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:15:59.096210 23911 catalog_manager.cc:5697] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } }
W20260709 12:15:59.098194 24204 leader_election.cc:343] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:15:59.103220 24204 leader_election.cc:343] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:59.103749 24204 leader_election.cc:304] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:15:59.104624 24288 tablet_bootstrap.cc:492] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:15:59.105072 24335 raft_consensus.cc:2749] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:59.105654 24285 tablet_bootstrap.cc:654] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:59.108893 24348 raft_consensus.cc:493] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.109979 24348 raft_consensus.cc:515] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:59.105195 24288 ts_tablet_manager.cc:1403] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.110s	user 0.020s	sys 0.010s
I20260709 12:15:59.112457 24348 leader_election.cc:290] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:15:59.113770 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "531e8a2f98124f13910b3a8ce4d279c1" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:15:59.114593 24238 raft_consensus.cc:2468] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:15:59.115763 24288 raft_consensus.cc:359] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:59.116562 24132 leader_election.cc:304] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:15:59.118839 24432 raft_consensus.cc:2804] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:59.119352 24432 raft_consensus.cc:493] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:59.116701 24288 raft_consensus.cc:385] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:59.119752 24432 raft_consensus.cc:3060] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:59.120325 24288 raft_consensus.cc:740] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:15:59.116993 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "531e8a2f98124f13910b3a8ce4d279c1" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:59.100822 24286 tablet_bootstrap.cc:492] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:15:59.121817 24286 ts_tablet_manager.cc:1403] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.119s	user 0.016s	sys 0.000s
W20260709 12:15:59.122291 24130 leader_election.cc:343] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:59.123903 24288 consensus_queue.cc:260] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:59.126636 24288 ts_tablet_manager.cc:1434] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.002s
I20260709 12:15:59.119688 24287 tablet_bootstrap.cc:654] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:59.127738 24286 raft_consensus.cc:359] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.128569 24286 raft_consensus.cc:385] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:59.128881 24286 raft_consensus.cc:740] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:15:59.129807 24286 consensus_queue.cc:260] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.131450 24288 tablet_bootstrap.cc:492] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:15:59.143091 24286 ts_tablet_manager.cc:1434] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.000s
I20260709 12:15:59.146476 24423 raft_consensus.cc:2749] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:59.155962 24288 tablet_bootstrap.cc:654] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:59.156854 24432 raft_consensus.cc:515] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:59.161312 24286 tablet_bootstrap.cc:492] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:15:59.166774 24375 raft_consensus.cc:493] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.159659 24240 raft_consensus.cc:2468] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:15:59.167464 24375 raft_consensus.cc:515] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.168589 24296 raft_consensus.cc:515] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.172808 24375 leader_election.cc:290] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:59.175290 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c505fd92654536b72dbd8a3f1bc728" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:59.171388 24335 consensus_queue.cc:1048] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:15:59.177893 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c505fd92654536b72dbd8a3f1bc728" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:59.178467 24286 tablet_bootstrap.cc:654] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:59.179131 24296 leader_election.cc:290] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
W20260709 12:15:59.180553 24204 leader_election.cc:343] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:59.185878 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16f5650964ece90109902b6b40882" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:15:59.186743 24432 leader_election.cc:290] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:15:59.187335 24018 raft_consensus.cc:3060] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:59.187916 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1df16f5650964ece90109902b6b40882" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:15:59.188652 24090 raft_consensus.cc:3060] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:15:59.190824 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "531e8a2f98124f13910b3a8ce4d279c1" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
W20260709 12:15:59.192492 24130 leader_election.cc:343] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:15:59.193670 24204 leader_election.cc:343] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:59.194298 24204 leader_election.cc:304] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:15:59.191041 24240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "531e8a2f98124f13910b3a8ce4d279c1" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:15:59.202925 24240 raft_consensus.cc:3060] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:59.201098 24018 raft_consensus.cc:2468] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 2.
I20260709 12:15:59.206202 24296 consensus_queue.cc:1048] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:59.212694 24285 tablet_bootstrap.cc:492] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:15:59.213301 24285 ts_tablet_manager.cc:1403] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.122s	user 0.020s	sys 0.021s
I20260709 12:15:59.216697 24285 raft_consensus.cc:359] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:59.217410 24285 raft_consensus.cc:385] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:59.217787 24285 raft_consensus.cc:740] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:15:59.218581 24285 consensus_queue.cc:260] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:59.221261 24285 ts_tablet_manager.cc:1434] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:15:59.209272 24335 raft_consensus.cc:2749] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:59.208827 24090 raft_consensus.cc:2468] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 2.
I20260709 12:15:59.223618 24204 leader_election.cc:304] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:15:59.225104 24285 tablet_bootstrap.cc:492] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:15:59.228415 24335 raft_consensus.cc:2804] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:15:59.236052 24287 tablet_bootstrap.cc:492] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:15:59.236713 24287 ts_tablet_manager.cc:1403] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.140s	user 0.028s	sys 0.017s
I20260709 12:15:59.239605 24335 raft_consensus.cc:697] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 2 LEADER]: Becoming Leader. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Running, Role: LEADER
I20260709 12:15:59.234300 24367 raft_consensus.cc:493] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.240883 24335 consensus_queue.cc:237] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.241590 24367 raft_consensus.cc:515] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:59.242249 24332 raft_consensus.cc:493] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.242820 24332 raft_consensus.cc:515] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.245043 24332 leader_election.cc:290] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:15:59.246850 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91f626cde148838a6a979d4fd1a5ed" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:15:59.247670 24238 raft_consensus.cc:2468] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:15:59.249109 24058 leader_election.cc:304] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:15:59.246578 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35efdb2f1c24db0a45fbc04abea260e" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:59.250839 24018 raft_consensus.cc:2468] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:15:59.251046 24412 raft_consensus.cc:2804] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:59.251718 24412 raft_consensus.cc:493] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:59.252180 24412 raft_consensus.cc:3060] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:59.245050 24367 leader_election.cc:290] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:15:59.252173 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35efdb2f1c24db0a45fbc04abea260e" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:15:59.259698 24166 raft_consensus.cc:2468] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:15:59.261581 24285 tablet_bootstrap.cc:654] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:59.262580 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91f626cde148838a6a979d4fd1a5ed" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:59.266690 24056 leader_election.cc:304] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:15:59.267836 24367 raft_consensus.cc:2804] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:59.268795 24367 raft_consensus.cc:493] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:59.269197 24018 raft_consensus.cc:2468] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:15:59.269703 24367 raft_consensus.cc:3060] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:59.271715 24287 raft_consensus.cc:359] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:59.264214 24240 raft_consensus.cc:2468] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:15:59.272536 24287 raft_consensus.cc:385] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:59.272907 24287 raft_consensus.cc:740] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:15:59.273696 24287 consensus_queue.cc:260] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:59.276628 24132 leader_election.cc:304] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b
I20260709 12:15:59.277856 24432 raft_consensus.cc:2804] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:59.281283 24286 tablet_bootstrap.cc:492] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:15:59.281807 24286 ts_tablet_manager.cc:1403] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.121s	user 0.004s	sys 0.025s
I20260709 12:15:59.278409 24432 raft_consensus.cc:697] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:15:59.285514 24432 consensus_queue.cc:237] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:59.291872 24412 raft_consensus.cc:515] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.298705 24287 ts_tablet_manager.cc:1434] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.062s	user 0.006s	sys 0.002s
I20260709 12:15:59.300727 24287 tablet_bootstrap.cc:492] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:15:59.299839 24375 consensus_queue.cc:1048] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:15:59.304723 24286 raft_consensus.cc:359] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:59.305685 24286 raft_consensus.cc:385] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:59.306020 24286 raft_consensus.cc:740] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:15:59.306809 24288 tablet_bootstrap.cc:492] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:15:59.307612 24288 ts_tablet_manager.cc:1403] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.176s	user 0.020s	sys 0.034s
I20260709 12:15:59.312994 24286 consensus_queue.cc:260] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:59.324982 24240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91f626cde148838a6a979d4fd1a5ed" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:15:59.326056 24240 raft_consensus.cc:3060] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:59.341596 23913 catalog_manager.cc:5697] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 reported cstate change: term changed from 0 to 2, leader changed from <none> to a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132). New cstate: current_term: 2 leader_uuid: "a0df0eff47b64e3eb45ab074139ed187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:59.344465 24348 consensus_queue.cc:1048] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:59.353039 24335 consensus_queue.cc:1048] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:59.313766 24432 raft_consensus.cc:493] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.366255 24286 ts_tablet_manager.cc:1434] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.084s	user 0.010s	sys 0.000s
I20260709 12:15:59.367734 24286 tablet_bootstrap.cc:492] T 531e8a2f98124f13910b3a8ce4d279c1 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:15:59.361023 24412 leader_election.cc:290] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:15:59.359794 24432 raft_consensus.cc:515] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:59.388785 24288 raft_consensus.cc:359] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:59.389457 24288 raft_consensus.cc:385] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:59.389715 24288 raft_consensus.cc:740] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:15:59.397168 24285 tablet_bootstrap.cc:492] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:15:59.397905 24285 ts_tablet_manager.cc:1403] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.173s	user 0.030s	sys 0.047s
I20260709 12:15:59.402866 24288 consensus_queue.cc:260] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:59.405527 24288 ts_tablet_manager.cc:1434] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.098s	user 0.003s	sys 0.006s
I20260709 12:15:59.406647 24288 tablet_bootstrap.cc:492] T 8e3a9c266ee343eb903d5138d1de95d2 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:15:59.408774 24432 leader_election.cc:290] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:15:59.411629 24287 tablet_bootstrap.cc:654] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:59.412348 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0756db96a05a45bc9ebc1f91ee907496" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:15:59.413108 24238 raft_consensus.cc:2468] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:15:59.412914 24367 raft_consensus.cc:515] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
W20260709 12:15:59.392076 24056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:42669, user_credentials={real_user=slave}} blocked reactor thread for 52131.3us
I20260709 12:15:59.432379 24132 leader_election.cc:304] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:15:59.426591 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0756db96a05a45bc9ebc1f91ee907496" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
W20260709 12:15:59.437390 24130 leader_election.cc:343] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:59.438022 24286 tablet_bootstrap.cc:654] T 531e8a2f98124f13910b3a8ce4d279c1 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:59.448503 24417 consensus_queue.cc:1048] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:59.443634 24432 raft_consensus.cc:2804] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:59.452186 24432 raft_consensus.cc:493] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:59.452549 24432 raft_consensus.cc:3060] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:59.455206 24240 raft_consensus.cc:2468] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:15:59.450968 24285 raft_consensus.cc:359] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:59.457259 24285 raft_consensus.cc:385] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:59.458001 24058 leader_election.cc:304] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:15:59.459234 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91f626cde148838a6a979d4fd1a5ed" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:15:59.460587 24285 raft_consensus.cc:740] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:15:59.462620 24453 raft_consensus.cc:2804] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:59.463326 24453 raft_consensus.cc:697] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Becoming Leader. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Running, Role: LEADER
I20260709 12:15:59.463508 24015 raft_consensus.cc:3060] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:59.464900 24288 tablet_bootstrap.cc:654] T 8e3a9c266ee343eb903d5138d1de95d2 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:59.464370 24453 consensus_queue.cc:237] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.469905 24285 consensus_queue.cc:260] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:59.481069 24348 raft_consensus.cc:493] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.481941 24348 raft_consensus.cc:515] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:59.485917 24367 leader_election.cc:290] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:15:59.487443 24287 tablet_bootstrap.cc:492] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:15:59.488101 24287 ts_tablet_manager.cc:1403] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.188s	user 0.016s	sys 0.006s
I20260709 12:15:59.491093 24285 ts_tablet_manager.cc:1434] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.093s	user 0.008s	sys 0.000s
I20260709 12:15:59.493438 24348 leader_election.cc:290] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:15:59.495635 24015 raft_consensus.cc:2468] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:15:59.488974 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21f0988e53da475c8274bcb13b34b3a8" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:15:59.498919 24285 tablet_bootstrap.cc:492] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:15:59.497999 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21f0988e53da475c8274bcb13b34b3a8" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
W20260709 12:15:59.503647 24130 leader_election.cc:343] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:59.508210 24348 consensus_queue.cc:1048] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:59.508836 24287 raft_consensus.cc:359] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.509496 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35efdb2f1c24db0a45fbc04abea260e" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:15:59.517469 24287 raft_consensus.cc:385] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:59.517894 24287 raft_consensus.cc:740] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:15:59.501473 24238 raft_consensus.cc:2468] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:15:59.519670 24167 raft_consensus.cc:3060] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:59.522585 24132 leader_election.cc:304] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:15:59.489409 23914 catalog_manager.cc:5697] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:59.532222 24356 raft_consensus.cc:2804] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:59.536032 24356 raft_consensus.cc:493] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:59.536759 24356 raft_consensus.cc:3060] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:59.543725 24288 tablet_bootstrap.cc:492] T 8e3a9c266ee343eb903d5138d1de95d2 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:15:59.545676 24288 ts_tablet_manager.cc:1403] T 8e3a9c266ee343eb903d5138d1de95d2 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.139s	user 0.008s	sys 0.008s
I20260709 12:15:59.540827 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e35efdb2f1c24db0a45fbc04abea260e" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:15:59.548707 24015 raft_consensus.cc:3060] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:59.549789 24286 tablet_bootstrap.cc:492] T 531e8a2f98124f13910b3a8ce4d279c1 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:15:59.550558 24286 ts_tablet_manager.cc:1403] T 531e8a2f98124f13910b3a8ce4d279c1 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.183s	user 0.018s	sys 0.016s
I20260709 12:15:59.554221 24286 raft_consensus.cc:359] T 531e8a2f98124f13910b3a8ce4d279c1 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:59.557678 24286 raft_consensus.cc:385] T 531e8a2f98124f13910b3a8ce4d279c1 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:59.558485 24286 raft_consensus.cc:740] T 531e8a2f98124f13910b3a8ce4d279c1 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:15:59.541702 24287 consensus_queue.cc:260] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.561314 24335 raft_consensus.cc:493] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.564320 24286 consensus_queue.cc:260] T 531e8a2f98124f13910b3a8ce4d279c1 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:59.567327 24287 ts_tablet_manager.cc:1434] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.079s	user 0.009s	sys 0.000s
I20260709 12:15:59.558656 24417 consensus_queue.cc:1048] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:15:59.569114 24287 tablet_bootstrap.cc:492] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:15:59.565814 24335 raft_consensus.cc:515] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.576495 24287 tablet_bootstrap.cc:654] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:59.580909 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd0b13fa2ed4f4489eef33489d4ebf1" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
W20260709 12:15:59.582949 24204 leader_election.cc:343] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:59.584139 24335 leader_election.cc:290] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:15:59.585352 24367 raft_consensus.cc:493] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.585069 23911 catalog_manager.cc:5697] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c74a5203b594708921cd793aaca0b8b (127.11.13.130). New cstate: current_term: 1 leader_uuid: "6c74a5203b594708921cd793aaca0b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: HEALTHY } } }
I20260709 12:15:59.585878 24288 raft_consensus.cc:359] T 8e3a9c266ee343eb903d5138d1de95d2 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:59.585937 24367 raft_consensus.cc:515] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:59.586709 24288 raft_consensus.cc:385] T 8e3a9c266ee343eb903d5138d1de95d2 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:59.587039 24288 raft_consensus.cc:740] T 8e3a9c266ee343eb903d5138d1de95d2 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:15:59.587785 24288 consensus_queue.cc:260] T 8e3a9c266ee343eb903d5138d1de95d2 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:59.588284 24367 leader_election.cc:290] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:15:59.589627 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd0b13fa2ed4f4489eef33489d4ebf1" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:59.576207 24348 raft_consensus.cc:493] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.590806 24164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7366aa6480d4d1e909db5717506432b" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:15:59.595669 24288 ts_tablet_manager.cc:1434] T 8e3a9c266ee343eb903d5138d1de95d2 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.050s	user 0.006s	sys 0.002s
I20260709 12:15:59.603726 24288 tablet_bootstrap.cc:492] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:15:59.599959 24286 ts_tablet_manager.cc:1434] T 531e8a2f98124f13910b3a8ce4d279c1 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.049s	user 0.008s	sys 0.002s
I20260709 12:15:59.607419 24286 tablet_bootstrap.cc:492] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:15:59.601182 24348 raft_consensus.cc:515] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:59.612999 24348 leader_election.cc:290] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
W20260709 12:15:59.613241 24204 leader_election.cc:343] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:59.613874 24204 leader_election.cc:304] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:15:59.614531 24164 raft_consensus.cc:2468] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:15:59.615588 24335 raft_consensus.cc:2749] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:59.616842 24285 tablet_bootstrap.cc:654] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:59.618716 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7366aa6480d4d1e909db5717506432b" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:59.651556 24375 raft_consensus.cc:493] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.652383 24375 raft_consensus.cc:515] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:59.665889 24288 tablet_bootstrap.cc:654] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:59.674453 24015 raft_consensus.cc:2468] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:15:59.677704 24356 raft_consensus.cc:515] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:59.645877 24018 raft_consensus.cc:2468] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:15:59.644142 24056 leader_election.cc:304] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: 
I20260709 12:15:59.693230 24453 raft_consensus.cc:2804] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:59.694111 24453 raft_consensus.cc:493] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:59.694204 24375 leader_election.cc:290] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:15:59.698982 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3a9c266ee343eb903d5138d1de95d2" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:15:59.694800 24453 raft_consensus.cc:3060] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:59.690888 24167 raft_consensus.cc:2468] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
W20260709 12:15:59.695094 24056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:36699, user_credentials={real_user=slave}} blocked reactor thread for 54641.6us
I20260709 12:15:59.713148 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3a9c266ee343eb903d5138d1de95d2" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:59.716149 24287 tablet_bootstrap.cc:492] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:15:59.702445 24163 raft_consensus.cc:2468] T 8e3a9c266ee343eb903d5138d1de95d2 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 0.
I20260709 12:15:59.717518 24287 ts_tablet_manager.cc:1403] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.149s	user 0.016s	sys 0.009s
I20260709 12:15:59.723543 24356 leader_election.cc:290] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:15:59.724785 24286 tablet_bootstrap.cc:654] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:59.734148 24285 tablet_bootstrap.cc:492] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:15:59.734304 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7366aa6480d4d1e909db5717506432b" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:15:59.734828 24285 ts_tablet_manager.cc:1403] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.236s	user 0.026s	sys 0.006s
I20260709 12:15:59.735169 24015 raft_consensus.cc:2468] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:15:59.735975 24204 leader_election.cc:304] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:15:59.736937 24240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21f0988e53da475c8274bcb13b34b3a8" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:15:59.737690 24240 raft_consensus.cc:3060] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:15:59.737094 24204 leader_election.cc:343] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:59.738181 24375 raft_consensus.cc:2804] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:15:59.739455 24375 raft_consensus.cc:493] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:15:59.739854 24375 raft_consensus.cc:3060] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:59.740128 24090 raft_consensus.cc:1275] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Refusing update from remote peer a0df0eff47b64e3eb45ab074139ed187: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:15:59.727357 24432 raft_consensus.cc:515] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:59.743762 24432 leader_election.cc:290] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:15:59.744611 24335 consensus_queue.cc:1048] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:59.745915 24056 leader_election.cc:304] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:15:59.752434 24367 raft_consensus.cc:2804] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:59.752954 24367 raft_consensus.cc:697] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Becoming Leader. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Running, Role: LEADER
I20260709 12:15:59.747792 24285 raft_consensus.cc:359] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.754179 24285 raft_consensus.cc:385] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:59.753837 24367 consensus_queue.cc:237] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:59.754514 24285 raft_consensus.cc:740] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:15:59.748243 24287 raft_consensus.cc:359] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:59.748790 24453 raft_consensus.cc:515] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:59.755591 24287 raft_consensus.cc:385] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:59.756098 24287 raft_consensus.cc:740] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:15:59.754655 24412 raft_consensus.cc:493] T 531e8a2f98124f13910b3a8ce4d279c1 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:15:59.758327 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7366aa6480d4d1e909db5717506432b" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:15:59.759251 24093 raft_consensus.cc:2393] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 008a79e516614c809c2d3afdb5cbc327 in current term 1: Already voted for candidate 6c74a5203b594708921cd793aaca0b8b in this term.
I20260709 12:15:59.758363 24412 raft_consensus.cc:515] T 531e8a2f98124f13910b3a8ce4d279c1 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:59.762894 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0756db96a05a45bc9ebc1f91ee907496" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:15:59.768774 24432 consensus_queue.cc:1048] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:15:59.769809 24464 consensus_queue.cc:1048] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:59.771798 24375 raft_consensus.cc:515] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:59.769124 24238 raft_consensus.cc:3060] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:59.763682 24285 consensus_queue.cc:260] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.761528 24240 raft_consensus.cc:2468] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:15:59.791468 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7366aa6480d4d1e909db5717506432b" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:15:59.793179 23915 catalog_manager.cc:5697] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c74a5203b594708921cd793aaca0b8b (127.11.13.130). New cstate: current_term: 1 leader_uuid: "6c74a5203b594708921cd793aaca0b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:59.781289 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3a9c266ee343eb903d5138d1de95d2" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:15:59.820683 24238 raft_consensus.cc:2468] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:15:59.785600 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "531e8a2f98124f13910b3a8ce4d279c1" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:15:59.794224 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7366aa6480d4d1e909db5717506432b" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:15:59.822866 24166 raft_consensus.cc:3060] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:59.823153 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8e3a9c266ee343eb903d5138d1de95d2" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:15:59.824050 24163 raft_consensus.cc:3060] T 8e3a9c266ee343eb903d5138d1de95d2 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:59.826679 24241 raft_consensus.cc:2393] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c74a5203b594708921cd793aaca0b8b in current term 1: Already voted for candidate 008a79e516614c809c2d3afdb5cbc327 in this term.
I20260709 12:15:59.810957 24132 leader_election.cc:304] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:15:59.827950 24132 leader_election.cc:304] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:15:59.811524 24285 ts_tablet_manager.cc:1434] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.076s	user 0.007s	sys 0.000s
I20260709 12:15:59.812089 24288 tablet_bootstrap.cc:492] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:15:59.829207 24288 ts_tablet_manager.cc:1403] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.226s	user 0.015s	sys 0.011s
I20260709 12:15:59.829794 24464 raft_consensus.cc:2804] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:59.830385 24464 raft_consensus.cc:697] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:15:59.831264 24464 consensus_queue.cc:237] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:59.832522 24288 raft_consensus.cc:359] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:59.833633 24288 raft_consensus.cc:385] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:59.833921 24288 raft_consensus.cc:740] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:15:59.834538 24468 raft_consensus.cc:2804] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:59.835147 24468 raft_consensus.cc:697] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:15:59.835559 24288 consensus_queue.cc:260] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:59.836069 24468 consensus_queue.cc:237] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:15:59.839146 24288 ts_tablet_manager.cc:1434] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:15:59.841147 24163 raft_consensus.cc:2468] T 8e3a9c266ee343eb903d5138d1de95d2 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 1.
I20260709 12:15:59.841459 24166 raft_consensus.cc:2468] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:15:59.843425 24056 leader_election.cc:304] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: 
I20260709 12:15:59.843945 24285 tablet_bootstrap.cc:492] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:15:59.844763 24367 raft_consensus.cc:2804] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:15:59.845455 24367 raft_consensus.cc:697] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Becoming Leader. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Running, Role: LEADER
I20260709 12:15:59.847204 24288 tablet_bootstrap.cc:492] T 1012da668d874f27a9563ae3f53ce4eb P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:15:59.850383 24367 consensus_queue.cc:237] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:15:59.854276 24288 tablet_bootstrap.cc:654] T 1012da668d874f27a9563ae3f53ce4eb P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:59.782861 24412 leader_election.cc:290] T 531e8a2f98124f13910b3a8ce4d279c1 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:15:59.794516 24453 leader_election.cc:290] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:15:59.769539 24130 leader_election.cc:304] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 6c74a5203b594708921cd793aaca0b8b
I20260709 12:15:59.856501 24356 raft_consensus.cc:2764] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
W20260709 12:15:59.856712 24130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:42669, user_credentials={real_user=slave}} blocked reactor thread for 87314.1us
I20260709 12:15:59.818465 24375 leader_election.cc:290] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:15:59.858644 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21f0988e53da475c8274bcb13b34b3a8" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:15:59.860656 24453 consensus_queue.cc:1048] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:59.861004 24356 consensus_queue.cc:1048] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:59.862702 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0756db96a05a45bc9ebc1f91ee907496" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:15:59.819317 24286 tablet_bootstrap.cc:492] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
W20260709 12:15:59.820801 24204 leader_election.cc:343] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:15:59.881611 24204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:42247, user_credentials={real_user=slave}} blocked reactor thread for 60859.4us
I20260709 12:15:59.819298 24015 raft_consensus.cc:3060] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:59.802950 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "531e8a2f98124f13910b3a8ce4d279c1" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:15:59.899353 24056 leader_election.cc:304] T 531e8a2f98124f13910b3a8ce4d279c1 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:15:59.900346 24412 raft_consensus.cc:3060] T 531e8a2f98124f13910b3a8ce4d279c1 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:15:59.780346 24287 consensus_queue.cc:260] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:15:59.903383 24287 ts_tablet_manager.cc:1434] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.183s	user 0.006s	sys 0.001s
I20260709 12:15:59.904781 24287 tablet_bootstrap.cc:492] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:15:59.911733 24412 raft_consensus.cc:2749] T 531e8a2f98124f13910b3a8ce4d279c1 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:15:59.913250 24015 raft_consensus.cc:2468] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:15:59.914525 24412 consensus_queue.cc:1048] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:15:59.918556 24018 raft_consensus.cc:1275] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Refusing update from remote peer a0df0eff47b64e3eb45ab074139ed187: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:15:59.927196 24375 consensus_queue.cc:1048] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:15:59.931182 24130 leader_election.cc:343] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:15:59.933167 24130 leader_election.cc:343] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:15:59.921269 23913 catalog_manager.cc:5697] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c74a5203b594708921cd793aaca0b8b (127.11.13.130). New cstate: current_term: 1 leader_uuid: "6c74a5203b594708921cd793aaca0b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:59.923404 24286 ts_tablet_manager.cc:1403] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.316s	user 0.019s	sys 0.019s
I20260709 12:15:59.951045 24286 raft_consensus.cc:359] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.951467 24468 consensus_queue.cc:1048] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:15:59.964574 23963 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:15:59.968024 23911 catalog_manager.cc:5697] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:59.971350 23911 catalog_manager.cc:5697] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } }
I20260709 12:15:59.968487 24286 raft_consensus.cc:385] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:15:59.976534 24286 raft_consensus.cc:740] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:15:59.977341 24286 consensus_queue.cc:260] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:15:59.980294 24287 tablet_bootstrap.cc:654] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:59.980963 24285 tablet_bootstrap.cc:700] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802: Time spent opening tablet: real 0.131s	user 0.004s	sys 0.000s
I20260709 12:15:59.998790 24285 tablet_bootstrap.cc:654] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:15:59.993006 24288 tablet_bootstrap.cc:492] T 1012da668d874f27a9563ae3f53ce4eb P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:15:59.999819 24288 ts_tablet_manager.cc:1403] T 1012da668d874f27a9563ae3f53ce4eb P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.160s	user 0.015s	sys 0.026s
I20260709 12:15:59.996881 24286 ts_tablet_manager.cc:1434] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.049s	user 0.006s	sys 0.000s
W20260709 12:15:59.997717 24484 log.cc:927] Time spent T 802231ce402445309306d2cbffa2f92c P 6c74a5203b594708921cd793aaca0b8b: Append to log took a long time: real 0.054s	user 0.000s	sys 0.002s
I20260709 12:15:59.984032 24204 leader_election.cc:304] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b
I20260709 12:16:00.003054 24296 raft_consensus.cc:2804] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:00.003731 24296 raft_consensus.cc:697] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Becoming Leader. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Running, Role: LEADER
I20260709 12:16:00.005244 24286 tablet_bootstrap.cc:492] T 8e3a9c266ee343eb903d5138d1de95d2 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:00.004912 24296 consensus_queue.cc:237] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:00.015265 24288 raft_consensus.cc:359] T 1012da668d874f27a9563ae3f53ce4eb P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:00.016332 24288 raft_consensus.cc:385] T 1012da668d874f27a9563ae3f53ce4eb P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.016673 24288 raft_consensus.cc:740] T 1012da668d874f27a9563ae3f53ce4eb P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.017089 24464 raft_consensus.cc:493] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.017702 24464 raft_consensus.cc:515] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:00.017477 24288 consensus_queue.cc:260] T 1012da668d874f27a9563ae3f53ce4eb P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:00.020205 24464 leader_election.cc:290] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:00.020777 24288 ts_tablet_manager.cc:1434] T 1012da668d874f27a9563ae3f53ce4eb P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.004s
I20260709 12:16:00.021900 24288 tablet_bootstrap.cc:492] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:16:00.023492 24287 tablet_bootstrap.cc:492] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:00.024081 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcccd54cb1fa4e9d9cab5afd3e8d4748" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:00.024477 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcccd54cb1fa4e9d9cab5afd3e8d4748" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
W20260709 12:16:00.025833 24130 leader_election.cc:343] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:00.026584 24130 leader_election.cc:343] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:00.026996 24130 leader_election.cc:304] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:16:00.027774 24464 raft_consensus.cc:2749] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:00.024081 24287 ts_tablet_manager.cc:1403] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.120s	user 0.007s	sys 0.005s
I20260709 12:16:00.031946 24287 raft_consensus.cc:359] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.032910 24287 raft_consensus.cc:385] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.033354 24287 raft_consensus.cc:740] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.035382 24287 consensus_queue.cc:260] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.028592 24288 tablet_bootstrap.cc:654] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.037329 24286 tablet_bootstrap.cc:654] T 8e3a9c266ee343eb903d5138d1de95d2 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.042008 24287 ts_tablet_manager.cc:1434] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:16:00.042598 24285 tablet_bootstrap.cc:492] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:00.043252 24287 tablet_bootstrap.cc:492] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:00.043402 24285 ts_tablet_manager.cc:1403] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.200s	user 0.015s	sys 0.001s
I20260709 12:16:00.043605 23915 catalog_manager.cc:5697] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132). New cstate: current_term: 1 leader_uuid: "a0df0eff47b64e3eb45ab074139ed187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:00.049124 24285 raft_consensus.cc:359] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.050204 24285 raft_consensus.cc:385] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.050484 24287 tablet_bootstrap.cc:654] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.050566 24285 raft_consensus.cc:740] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.072453 24287 tablet_bootstrap.cc:492] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:00.073107 24287 ts_tablet_manager.cc:1403] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.030s	user 0.022s	sys 0.004s
I20260709 12:16:00.076234 24287 raft_consensus.cc:359] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.077096 24287 raft_consensus.cc:385] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.076982 24285 consensus_queue.cc:260] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.077572 24287 raft_consensus.cc:740] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.080435 24285 ts_tablet_manager.cc:1434] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.001s
I20260709 12:16:00.078657 24287 consensus_queue.cc:260] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.084103 24288 tablet_bootstrap.cc:492] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:00.084735 24288 ts_tablet_manager.cc:1403] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.063s	user 0.015s	sys 0.023s
I20260709 12:16:00.087913 24287 ts_tablet_manager.cc:1434] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:16:00.089015 24287 tablet_bootstrap.cc:492] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:00.087962 24288 raft_consensus.cc:359] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:00.091917 24288 raft_consensus.cc:385] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.092602 24288 raft_consensus.cc:740] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.093838 24288 consensus_queue.cc:260] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:00.094978 24285 tablet_bootstrap.cc:492] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:00.098536 24286 tablet_bootstrap.cc:492] T 8e3a9c266ee343eb903d5138d1de95d2 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:00.099179 24286 ts_tablet_manager.cc:1403] T 8e3a9c266ee343eb903d5138d1de95d2 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.094s	user 0.019s	sys 0.000s
I20260709 12:16:00.103478 24288 ts_tablet_manager.cc:1434] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:16:00.103520 24287 tablet_bootstrap.cc:654] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.104812 24288 tablet_bootstrap.cc:492] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:16:00.105129 24286 raft_consensus.cc:359] T 8e3a9c266ee343eb903d5138d1de95d2 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:00.105994 24286 raft_consensus.cc:385] T 8e3a9c266ee343eb903d5138d1de95d2 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.106380 24286 raft_consensus.cc:740] T 8e3a9c266ee343eb903d5138d1de95d2 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.105314 24285 tablet_bootstrap.cc:654] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.107268 24286 consensus_queue.cc:260] T 8e3a9c266ee343eb903d5138d1de95d2 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:00.110399 24286 ts_tablet_manager.cc:1434] T 8e3a9c266ee343eb903d5138d1de95d2 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.000s
I20260709 12:16:00.118423 24417 raft_consensus.cc:493] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.119068 24417 raft_consensus.cc:515] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:00.121754 24417 leader_election.cc:290] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:00.122692 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c505fd92654536b72dbd8a3f1bc728" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:00.123484 24238 raft_consensus.cc:2468] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 0.
I20260709 12:16:00.125017 23983 leader_election.cc:304] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:00.126405 24417 raft_consensus.cc:2804] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:00.126816 24417 raft_consensus.cc:493] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:00.127195 24417 raft_consensus.cc:3060] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:00.128552 24286 tablet_bootstrap.cc:492] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:00.125937 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c505fd92654536b72dbd8a3f1bc728" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:00.129835 24091 raft_consensus.cc:2468] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 0.
I20260709 12:16:00.135317 24288 tablet_bootstrap.cc:654] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.136302 24286 tablet_bootstrap.cc:654] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.135921 24417 raft_consensus.cc:515] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:00.140479 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c505fd92654536b72dbd8a3f1bc728" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:00.141208 24238 raft_consensus.cc:3060] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:00.143724 24285 tablet_bootstrap.cc:492] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:00.144678 24285 ts_tablet_manager.cc:1403] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.050s	user 0.012s	sys 0.002s
I20260709 12:16:00.147964 24285 raft_consensus.cc:359] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.148659 24285 raft_consensus.cc:385] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.148934 24285 raft_consensus.cc:740] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.151026 24285 consensus_queue.cc:260] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.152961 24288 tablet_bootstrap.cc:492] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:00.153589 24288 ts_tablet_manager.cc:1403] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.049s	user 0.012s	sys 0.000s
I20260709 12:16:00.154672 24285 ts_tablet_manager.cc:1434] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.001s
I20260709 12:16:00.156629 24288 raft_consensus.cc:359] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.157608 24296 raft_consensus.cc:493] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.157557 24288 raft_consensus.cc:385] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.158833 24285 tablet_bootstrap.cc:492] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:00.158445 24296 raft_consensus.cc:515] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:00.161839 24296 leader_election.cc:290] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:00.158861 24288 raft_consensus.cc:740] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.163000 24288 consensus_queue.cc:260] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.163645 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1012da668d874f27a9563ae3f53ce4eb" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:00.164347 24163 raft_consensus.cc:2468] T 1012da668d874f27a9563ae3f53ce4eb P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 0.
I20260709 12:16:00.165802 24288 ts_tablet_manager.cc:1434] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.012s	user 0.004s	sys 0.004s
I20260709 12:16:00.166371 24204 leader_election.cc:304] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:00.167068 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1012da668d874f27a9563ae3f53ce4eb" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:00.167598 24296 raft_consensus.cc:2804] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:00.168062 24288 tablet_bootstrap.cc:492] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:16:00.168026 24296 raft_consensus.cc:493] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:00.168512 24296 raft_consensus.cc:3060] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:00.169919 24204 leader_election.cc:343] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:00.167124 24238 raft_consensus.cc:2468] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 1.
I20260709 12:16:00.172255 23983 leader_election.cc:304] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:00.173218 24418 raft_consensus.cc:2804] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:00.174538 24287 tablet_bootstrap.cc:492] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:00.175274 24287 ts_tablet_manager.cc:1403] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.086s	user 0.022s	sys 0.005s
I20260709 12:16:00.180238 24296 raft_consensus.cc:515] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:00.181648 24288 tablet_bootstrap.cc:654] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.186007 24417 leader_election.cc:290] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:00.187355 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c505fd92654536b72dbd8a3f1bc728" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:00.188050 24091 raft_consensus.cc:3060] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:00.191728 24367 consensus_queue.cc:1048] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:00.196347 24285 tablet_bootstrap.cc:654] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.200286 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1012da668d874f27a9563ae3f53ce4eb" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:00.200287 24286 tablet_bootstrap.cc:492] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:00.201592 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1012da668d874f27a9563ae3f53ce4eb" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:00.202497 24286 ts_tablet_manager.cc:1403] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.074s	user 0.034s	sys 0.012s
I20260709 12:16:00.202654 24163 raft_consensus.cc:3060] T 1012da668d874f27a9563ae3f53ce4eb P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:00.206727 24204 leader_election.cc:343] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:00.210814 24418 raft_consensus.cc:697] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:16:00.212212 24418 consensus_queue.cc:237] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:00.214828 24296 leader_election.cc:290] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:00.217598 24367 consensus_queue.cc:1048] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:00.205720 24286 raft_consensus.cc:359] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:00.218999 24091 raft_consensus.cc:2468] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 1.
I20260709 12:16:00.191730 24287 raft_consensus.cc:359] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:00.221573 24417 raft_consensus.cc:493] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.223297 24287 raft_consensus.cc:385] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.224005 24287 raft_consensus.cc:740] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.223766 24417 raft_consensus.cc:515] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.225153 24287 consensus_queue.cc:260] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:00.226620 24417 leader_election.cc:290] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:00.220147 24286 raft_consensus.cc:385] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.227231 24286 raft_consensus.cc:740] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.228258 24286 consensus_queue.cc:260] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:00.231104 24286 ts_tablet_manager.cc:1434] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.028s	user 0.006s	sys 0.001s
I20260709 12:16:00.232748 24163 raft_consensus.cc:2468] T 1012da668d874f27a9563ae3f53ce4eb P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 1.
I20260709 12:16:00.233330 24287 ts_tablet_manager.cc:1434] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.046s	user 0.009s	sys 0.001s
I20260709 12:16:00.234890 24287 tablet_bootstrap.cc:492] T 3bd7c9c11f834f15974a65477ef9258a P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:00.236387 24204 leader_election.cc:304] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b
I20260709 12:16:00.238229 24286 tablet_bootstrap.cc:492] T 1012da668d874f27a9563ae3f53ce4eb P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:00.245508 24287 tablet_bootstrap.cc:654] T 3bd7c9c11f834f15974a65477ef9258a P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.256333 24332 consensus_queue.cc:1048] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:00.272533 24285 tablet_bootstrap.cc:492] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:00.273233 24288 tablet_bootstrap.cc:492] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:00.273264 24285 ts_tablet_manager.cc:1403] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.115s	user 0.018s	sys 0.011s
I20260709 12:16:00.273968 24288 ts_tablet_manager.cc:1403] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.106s	user 0.017s	sys 0.026s
W20260709 12:16:00.276686 23981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:36699, user_credentials={real_user=slave}} blocked reactor thread for 68916.4us
I20260709 12:16:00.285526 24285 raft_consensus.cc:359] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:00.287714 24285 raft_consensus.cc:385] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.288101 24285 raft_consensus.cc:740] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.289042 24285 consensus_queue.cc:260] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:00.291429 24285 ts_tablet_manager.cc:1434] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:16:00.292513 24285 tablet_bootstrap.cc:492] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:00.283901 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21f0988e53da475c8274bcb13b34b3a8" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:00.295776 24286 tablet_bootstrap.cc:654] T 1012da668d874f27a9563ae3f53ce4eb P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.280633 24367 consensus_queue.cc:1048] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 12:16:00.278348 24296 raft_consensus.cc:2804] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:00.302841 24296 raft_consensus.cc:697] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Becoming Leader. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Running, Role: LEADER
I20260709 12:16:00.303828 24296 consensus_queue.cc:237] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:00.305570 24288 raft_consensus.cc:359] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:00.306428 24288 raft_consensus.cc:385] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.306734 24288 raft_consensus.cc:740] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.307531 24288 consensus_queue.cc:260] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:00.310282 24288 ts_tablet_manager.cc:1434] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.036s	user 0.008s	sys 0.000s
I20260709 12:16:00.311434 24288 tablet_bootstrap.cc:492] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:16:00.311127 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21f0988e53da475c8274bcb13b34b3a8" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:00.325057 24418 raft_consensus.cc:493] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.325994 24418 raft_consensus.cc:515] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.328605 24418 leader_election.cc:290] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:00.329876 24240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0756db96a05a45bc9ebc1f91ee907496" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:00.330601 24287 tablet_bootstrap.cc:492] T 3bd7c9c11f834f15974a65477ef9258a P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:00.330801 24240 raft_consensus.cc:2393] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c359614416104a7ba9db1ede97d13802 in current term 1: Already voted for candidate 008a79e516614c809c2d3afdb5cbc327 in this term.
I20260709 12:16:00.331149 24287 ts_tablet_manager.cc:1403] T 3bd7c9c11f834f15974a65477ef9258a P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.096s	user 0.013s	sys 0.014s
I20260709 12:16:00.327772 23912 catalog_manager.cc:5697] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 0 to 1, leader changed from <none> to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:00.340925 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0756db96a05a45bc9ebc1f91ee907496" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:00.341662 24287 raft_consensus.cc:359] T 3bd7c9c11f834f15974a65477ef9258a P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:00.340149 24238 raft_consensus.cc:2393] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c359614416104a7ba9db1ede97d13802 in current term 1: Already voted for candidate 008a79e516614c809c2d3afdb5cbc327 in this term.
I20260709 12:16:00.357062 23981 leader_election.cc:304] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:00.358570 24418 raft_consensus.cc:3060] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:00.358817 24348 consensus_queue.cc:1048] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:00.362572 23983 leader_election.cc:304] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:00.363595 24503 raft_consensus.cc:3060] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:00.367619 24418 raft_consensus.cc:2749] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:00.348690 24285 tablet_bootstrap.cc:654] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.352161 24288 tablet_bootstrap.cc:654] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.353070 24287 raft_consensus.cc:385] T 3bd7c9c11f834f15974a65477ef9258a P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.353681 24356 raft_consensus.cc:493] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.379477 24356 raft_consensus.cc:515] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.385059 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4c368e2cc6c4869a85a53ae022a0785" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:00.386548 24356 leader_election.cc:290] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:00.387310 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4c368e2cc6c4869a85a53ae022a0785" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:00.388072 24238 raft_consensus.cc:2468] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:16:00.389971 24287 raft_consensus.cc:740] T 3bd7c9c11f834f15974a65477ef9258a P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
W20260709 12:16:00.390941 24130 leader_election.cc:343] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:00.392194 24132 leader_election.cc:304] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:00.393186 24356 raft_consensus.cc:2804] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:00.393579 24356 raft_consensus.cc:493] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:00.393931 24356 raft_consensus.cc:3060] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:00.395182 24286 tablet_bootstrap.cc:492] T 1012da668d874f27a9563ae3f53ce4eb P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:00.395409 24469 raft_consensus.cc:493] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.396745 24286 ts_tablet_manager.cc:1403] T 1012da668d874f27a9563ae3f53ce4eb P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.159s	user 0.009s	sys 0.011s
I20260709 12:16:00.396731 24469 raft_consensus.cc:515] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:00.398360 24432 consensus_queue.cc:1048] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:00.399564 24469 leader_election.cc:290] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:00.400338 24286 raft_consensus.cc:359] T 1012da668d874f27a9563ae3f53ce4eb P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:00.390991 24287 consensus_queue.cc:260] T 3bd7c9c11f834f15974a65477ef9258a P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:00.401218 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bd7c9c11f834f15974a65477ef9258a" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:00.401190 24286 raft_consensus.cc:385] T 1012da668d874f27a9563ae3f53ce4eb P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.401741 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bd7c9c11f834f15974a65477ef9258a" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:00.402032 24286 raft_consensus.cc:740] T 1012da668d874f27a9563ae3f53ce4eb P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.402665 24241 raft_consensus.cc:2468] T 3bd7c9c11f834f15974a65477ef9258a P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:16:00.407212 24375 raft_consensus.cc:493] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.407804 24503 raft_consensus.cc:2749] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:00.403126 24286 consensus_queue.cc:260] T 1012da668d874f27a9563ae3f53ce4eb P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:00.407877 24375 raft_consensus.cc:515] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.413749 24375 leader_election.cc:290] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:00.416508 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4638bcb9174442184bc8d8a27d45072" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:00.417325 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4638bcb9174442184bc8d8a27d45072" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
W20260709 12:16:00.420171 24204 leader_election.cc:343] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:00.420993 24204 leader_election.cc:343] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:00.421481 24204 leader_election.cc:304] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:00.411279 24091 raft_consensus.cc:3060] T 8e3a9c266ee343eb903d5138d1de95d2 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:00.426450 24335 raft_consensus.cc:2749] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:00.428718 24285 tablet_bootstrap.cc:492] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:00.429380 24285 ts_tablet_manager.cc:1403] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.137s	user 0.009s	sys 0.003s
W20260709 12:16:00.405040 24130 leader_election.cc:343] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:00.430970 24469 consensus_queue.cc:1048] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:00.431420 24418 raft_consensus.cc:493] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.432029 24418 raft_consensus.cc:515] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:00.437018 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd0b13fa2ed4f4489eef33489d4ebf1" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:00.437839 24286 ts_tablet_manager.cc:1434] T 1012da668d874f27a9563ae3f53ce4eb P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.000s
W20260709 12:16:00.439083 23981 leader_election.cc:343] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:00.440145 24132 leader_election.cc:304] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b
I20260709 12:16:00.445791 24469 raft_consensus.cc:2804] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:00.446295 24469 raft_consensus.cc:493] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:00.446735 24469 raft_consensus.cc:3060] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:00.447813 24432 raft_consensus.cc:493] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.448380 24432 raft_consensus.cc:515] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:00.449316 24356 raft_consensus.cc:515] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.450618 24432 leader_election.cc:290] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:00.451570 24356 leader_election.cc:290] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:00.453425 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3effea5c11740418bd8b7bedd33e0f5" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:00.454286 24285 raft_consensus.cc:359] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:00.455168 24285 raft_consensus.cc:385] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.455533 24285 raft_consensus.cc:740] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.455279 24286 tablet_bootstrap.cc:492] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
W20260709 12:16:00.456538 24130 leader_election.cc:343] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:00.441637 24418 leader_election.cc:290] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:00.457960 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3effea5c11740418bd8b7bedd33e0f5" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:00.458767 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4c368e2cc6c4869a85a53ae022a0785" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:00.468083 24241 raft_consensus.cc:3060] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:00.461930 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4c368e2cc6c4869a85a53ae022a0785" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:16:00.461589 24285 consensus_queue.cc:260] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
W20260709 12:16:00.466696 24130 leader_election.cc:343] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:00.467087 24335 consensus_queue.cc:1048] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:00.474377 24130 leader_election.cc:304] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:16:00.475881 24356 raft_consensus.cc:2749] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:00.476943 24464 consensus_queue.cc:1048] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:00.479843 24469 raft_consensus.cc:515] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:00.404350 24287 ts_tablet_manager.cc:1434] T 3bd7c9c11f834f15974a65477ef9258a P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.073s	user 0.007s	sys 0.002s
I20260709 12:16:00.471031 23912 catalog_manager.cc:5697] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132). New cstate: current_term: 1 leader_uuid: "a0df0eff47b64e3eb45ab074139ed187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:00.479843 24130 leader_election.cc:343] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:00.481629 24285 ts_tablet_manager.cc:1434] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.052s	user 0.009s	sys 0.000s
I20260709 12:16:00.486953 24287 tablet_bootstrap.cc:492] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:00.490058 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bd7c9c11f834f15974a65477ef9258a" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:00.491009 24288 tablet_bootstrap.cc:492] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:00.491566 24288 ts_tablet_manager.cc:1403] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.180s	user 0.011s	sys 0.006s
W20260709 12:16:00.491597 24130 leader_election.cc:343] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:00.490101 24285 tablet_bootstrap.cc:492] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:00.490881 24469 leader_election.cc:290] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:00.499068 24285 tablet_bootstrap.cc:654] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.500124 24239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bd7c9c11f834f15974a65477ef9258a" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:00.500887 24239 raft_consensus.cc:3060] T 3bd7c9c11f834f15974a65477ef9258a P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:00.504716 24287 tablet_bootstrap.cc:654] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.460023 24240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd0b13fa2ed4f4489eef33489d4ebf1" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:00.522758 24240 raft_consensus.cc:2468] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 0.
I20260709 12:16:00.509792 24288 raft_consensus.cc:359] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.524019 24288 raft_consensus.cc:385] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.524372 24288 raft_consensus.cc:740] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.524520 23983 leader_election.cc:304] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802; no voters: 6c74a5203b594708921cd793aaca0b8b
I20260709 12:16:00.525959 24503 raft_consensus.cc:2804] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:00.482708 24286 tablet_bootstrap.cc:654] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.526371 24503 raft_consensus.cc:493] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:00.526831 24503 raft_consensus.cc:3060] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:00.526703 24517 consensus_queue.cc:1048] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:00.531068 24239 raft_consensus.cc:2468] T 3bd7c9c11f834f15974a65477ef9258a P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:16:00.532757 24132 leader_election.cc:304] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b
I20260709 12:16:00.534204 24464 raft_consensus.cc:2804] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:00.534793 24464 raft_consensus.cc:697] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:00.539495 24288 consensus_queue.cc:260] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.545969 24288 ts_tablet_manager.cc:1434] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.054s	user 0.009s	sys 0.000s
I20260709 12:16:00.551062 24285 tablet_bootstrap.cc:492] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:00.552357 24285 ts_tablet_manager.cc:1403] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.065s	user 0.024s	sys 0.004s
I20260709 12:16:00.528585 24241 raft_consensus.cc:2468] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:16:00.537746 24464 consensus_queue.cc:237] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:00.555824 24288 tablet_bootstrap.cc:492] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:16:00.572171 24285 raft_consensus.cc:359] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:00.590721 24285 raft_consensus.cc:385] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.569402 24503 raft_consensus.cc:515] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:00.591215 24285 raft_consensus.cc:740] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.598156 24285 consensus_queue.cc:260] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:00.600940 24285 ts_tablet_manager.cc:1434] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.048s	user 0.012s	sys 0.010s
I20260709 12:16:00.590988 24132 leader_election.cc:304] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:00.602146 24285 tablet_bootstrap.cc:492] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:00.603668 24286 tablet_bootstrap.cc:492] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:00.604189 24286 ts_tablet_manager.cc:1403] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.152s	user 0.011s	sys 0.008s
I20260709 12:16:00.606288 24356 raft_consensus.cc:2804] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:00.606863 24503 leader_election.cc:290] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:00.608693 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd0b13fa2ed4f4489eef33489d4ebf1" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:00.609197 24285 tablet_bootstrap.cc:654] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:00.611158 23981 leader_election.cc:343] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:00.611944 24418 raft_consensus.cc:493] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.612457 24418 raft_consensus.cc:515] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:00.614056 24286 raft_consensus.cc:359] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:00.614892 24286 raft_consensus.cc:385] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.615182 24286 raft_consensus.cc:740] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.595656 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd0b13fa2ed4f4489eef33489d4ebf1" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:00.616255 24241 raft_consensus.cc:3060] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:00.616858 24286 consensus_queue.cc:260] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:00.619357 24517 raft_consensus.cc:493] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.619972 24517 raft_consensus.cc:515] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:00.620638 24288 tablet_bootstrap.cc:654] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.627498 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be4e681eef304bb8bc812e313a67414d" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:00.628597 24018 raft_consensus.cc:2468] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 0.
I20260709 12:16:00.638695 24286 ts_tablet_manager.cc:1434] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.034s	user 0.002s	sys 0.006s
I20260709 12:16:00.638553 24418 leader_election.cc:290] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:00.641428 24286 tablet_bootstrap.cc:492] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:00.659981 24287 tablet_bootstrap.cc:492] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:00.660807 24287 ts_tablet_manager.cc:1403] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.174s	user 0.021s	sys 0.045s
I20260709 12:16:00.664929 24287 raft_consensus.cc:359] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:00.665990 24287 raft_consensus.cc:385] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.666416 24287 raft_consensus.cc:740] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.667399 24287 consensus_queue.cc:260] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:00.647194 24356 raft_consensus.cc:697] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:00.667070 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcccd54cb1fa4e9d9cab5afd3e8d4748" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:00.670476 24517 leader_election.cc:290] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:00.670881 24090 raft_consensus.cc:2468] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 0.
I20260709 12:16:00.672381 23981 leader_election.cc:304] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:00.674283 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcccd54cb1fa4e9d9cab5afd3e8d4748" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:00.675077 24163 raft_consensus.cc:2468] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 0.
I20260709 12:16:00.677656 24204 leader_election.cc:304] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802; no voters: 
W20260709 12:16:00.678485 24204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:42669, user_credentials={real_user=slave}} blocked reactor thread for 45427.4us
I20260709 12:16:00.678647 24356 consensus_queue.cc:237] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.680178 24296 raft_consensus.cc:2804] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:00.680565 24296 raft_consensus.cc:493] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:00.680929 24296 raft_consensus.cc:3060] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:00.678544 24241 raft_consensus.cc:2468] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 1.
I20260709 12:16:00.683511 24287 ts_tablet_manager.cc:1434] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.001s
I20260709 12:16:00.684576 24287 tablet_bootstrap.cc:492] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:00.686630 23983 leader_election.cc:304] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802; no voters: 6c74a5203b594708921cd793aaca0b8b
I20260709 12:16:00.687619 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be4e681eef304bb8bc812e313a67414d" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:00.688195 24503 raft_consensus.cc:2804] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:00.688745 24503 raft_consensus.cc:697] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:16:00.689893 24503 consensus_queue.cc:237] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:00.693265 24286 tablet_bootstrap.cc:654] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.692830 24418 raft_consensus.cc:2804] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:00.703787 24418 raft_consensus.cc:493] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:00.707051 24418 raft_consensus.cc:3060] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:00.712092 24204 leader_election.cc:343] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:00.718992 24335 consensus_queue.cc:1048] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:00.721263 24296 raft_consensus.cc:515] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:00.730532 24285 tablet_bootstrap.cc:492] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:00.725298 23912 catalog_manager.cc:5697] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:00.731123 24285 ts_tablet_manager.cc:1403] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.129s	user 0.029s	sys 0.028s
I20260709 12:16:00.732007 24367 raft_consensus.cc:493] T 1012da668d874f27a9563ae3f53ce4eb P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.732556 24367 raft_consensus.cc:515] T 1012da668d874f27a9563ae3f53ce4eb P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:00.734158 24285 raft_consensus.cc:359] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.735033 24285 raft_consensus.cc:385] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.735312 24285 raft_consensus.cc:740] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.736508 24367 leader_election.cc:290] T 1012da668d874f27a9563ae3f53ce4eb P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:00.740422 24499 consensus_queue.cc:1048] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:00.740653 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1012da668d874f27a9563ae3f53ce4eb" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:00.742446 24163 raft_consensus.cc:2393] T 1012da668d874f27a9563ae3f53ce4eb P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c74a5203b594708921cd793aaca0b8b in current term 1: Already voted for candidate a0df0eff47b64e3eb45ab074139ed187 in this term.
I20260709 12:16:00.743598 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1012da668d874f27a9563ae3f53ce4eb" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:00.743943 24418 raft_consensus.cc:515] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:00.740495 24285 consensus_queue.cc:260] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.750658 24285 ts_tablet_manager.cc:1434] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:16:00.752118 24285 tablet_bootstrap.cc:492] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:00.754433 24296 leader_election.cc:290] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:00.757284 24502 consensus_queue.cc:1048] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:00.752156 24288 tablet_bootstrap.cc:492] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:00.753283 24418 leader_election.cc:290] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:00.752883 24058 leader_election.cc:304] T 1012da668d874f27a9563ae3f53ce4eb P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:00.771735 24288 ts_tablet_manager.cc:1403] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.217s	user 0.033s	sys 0.017s
I20260709 12:16:00.781325 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcccd54cb1fa4e9d9cab5afd3e8d4748" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:00.782239 24091 raft_consensus.cc:3060] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:00.782383 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcccd54cb1fa4e9d9cab5afd3e8d4748" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:00.783159 24163 raft_consensus.cc:3060] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:00.778225 24367 raft_consensus.cc:3060] T 1012da668d874f27a9563ae3f53ce4eb P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:00.785983 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be4e681eef304bb8bc812e313a67414d" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:00.787521 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be4e681eef304bb8bc812e313a67414d" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:16:00.788142 24018 raft_consensus.cc:3060] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:00.790786 23911 catalog_manager.cc:5697] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 0 to 1, leader changed from <none> to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:00.783311 24286 tablet_bootstrap.cc:492] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:00.792814 24286 ts_tablet_manager.cc:1403] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.152s	user 0.011s	sys 0.009s
I20260709 12:16:00.793951 24287 tablet_bootstrap.cc:654] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.796386 23912 catalog_manager.cc:5697] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:00.804636 24375 raft_consensus.cc:493] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.805205 24375 raft_consensus.cc:515] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:00.809923 24163 raft_consensus.cc:2468] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 1.
I20260709 12:16:00.811998 24286 raft_consensus.cc:359] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:00.812788 24286 raft_consensus.cc:385] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.813102 24286 raft_consensus.cc:740] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
W20260709 12:16:00.800475 24204 leader_election.cc:343] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:00.814309 23981 leader_election.cc:304] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:00.818434 24502 raft_consensus.cc:2804] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:00.819000 24502 raft_consensus.cc:697] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:16:00.819587 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56fb07e9eb574b1fbc5da95063610dd5" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:00.819869 24502 consensus_queue.cc:237] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:00.831084 24468 raft_consensus.cc:493] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.831794 24375 leader_election.cc:290] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
W20260709 12:16:00.832204 24204 leader_election.cc:343] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:00.831673 24468 raft_consensus.cc:515] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.833910 24286 consensus_queue.cc:260] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:00.849231 24285 tablet_bootstrap.cc:654] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.849314 24286 ts_tablet_manager.cc:1434] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.000s
I20260709 12:16:00.839450 24018 raft_consensus.cc:2468] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 1.
I20260709 12:16:00.850901 24286 tablet_bootstrap.cc:492] T be4e681eef304bb8bc812e313a67414d P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:00.851385 24288 raft_consensus.cc:359] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:00.852505 24288 raft_consensus.cc:385] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.852906 24288 raft_consensus.cc:740] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.855185 24288 consensus_queue.cc:260] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:00.856694 24468 leader_election.cc:290] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:00.858642 24239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4638bcb9174442184bc8d8a27d45072" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:00.859364 24239 raft_consensus.cc:2468] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:16:00.859503 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4638bcb9174442184bc8d8a27d45072" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
W20260709 12:16:00.861109 24130 leader_election.cc:343] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:00.869817 24288 ts_tablet_manager.cc:1434] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.096s	user 0.000s	sys 0.008s
I20260709 12:16:00.871133 24288 tablet_bootstrap.cc:492] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:16:00.873039 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56fb07e9eb574b1fbc5da95063610dd5" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:00.876017 24132 leader_election.cc:304] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:00.877460 24453 raft_consensus.cc:493] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.878031 24453 raft_consensus.cc:515] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:00.878849 23914 catalog_manager.cc:5697] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 0 to 1, leader changed from <none> to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:00.880252 24453 leader_election.cc:290] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:00.877523 24356 raft_consensus.cc:2804] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:00.881625 24356 raft_consensus.cc:493] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:00.887379 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd0b13fa2ed4f4489eef33489d4ebf1" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:00.888345 24241 raft_consensus.cc:2393] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c74a5203b594708921cd793aaca0b8b in current term 1: Already voted for candidate c359614416104a7ba9db1ede97d13802 in this term.
I20260709 12:16:00.883879 24356 raft_consensus.cc:3060] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:00.901185 24367 raft_consensus.cc:2749] T 1012da668d874f27a9563ae3f53ce4eb P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:00.869012 24204 leader_election.cc:304] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802; no voters: 6c74a5203b594708921cd793aaca0b8b
I20260709 12:16:00.911500 24375 raft_consensus.cc:2804] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:00.914084 24375 raft_consensus.cc:697] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Becoming Leader. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Running, Role: LEADER
W20260709 12:16:00.915153 24204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:42669, user_credentials={real_user=slave}} blocked reactor thread for 49386.2us
W20260709 12:16:00.916558 24204 leader_election.cc:343] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:00.917065 24204 leader_election.cc:304] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:16:00.917104 24091 raft_consensus.cc:2468] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 1.
I20260709 12:16:00.921036 24335 raft_consensus.cc:2749] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:00.912806 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd0b13fa2ed4f4489eef33489d4ebf1" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:00.926663 24286 tablet_bootstrap.cc:654] T be4e681eef304bb8bc812e313a67414d P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:00.915212 24375 consensus_queue.cc:237] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:00.930918 24056 leader_election.cc:304] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802
I20260709 12:16:00.932775 24367 raft_consensus.cc:3060] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:00.950927 24285 tablet_bootstrap.cc:492] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:00.951817 24285 ts_tablet_manager.cc:1403] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.200s	user 0.022s	sys 0.001s
I20260709 12:16:00.959270 24517 consensus_queue.cc:1048] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:00.962805 24287 tablet_bootstrap.cc:492] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:00.963406 24287 ts_tablet_manager.cc:1403] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.279s	user 0.016s	sys 0.016s
I20260709 12:16:00.964622 24285 raft_consensus.cc:359] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.965468 24285 raft_consensus.cc:385] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:00.965812 24285 raft_consensus.cc:740] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:00.966866 24285 consensus_queue.cc:260] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
W20260709 12:16:00.978442 23983 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:41011, user_credentials={real_user=slave}} blocked reactor thread for 56317.3us
I20260709 12:16:00.985383 24285 ts_tablet_manager.cc:1434] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.002s
I20260709 12:16:00.987041 24285 tablet_bootstrap.cc:492] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:00.991811 24418 raft_consensus.cc:493] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:00.992478 24418 raft_consensus.cc:515] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:00.995024 24418 leader_election.cc:290] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:00.996873 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4c368e2cc6c4869a85a53ae022a0785" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:00.969825 24356 raft_consensus.cc:515] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:01.005384 24287 raft_consensus.cc:359] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:01.006392 24287 raft_consensus.cc:385] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.006760 24287 raft_consensus.cc:740] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.007439 24287 consensus_queue.cc:260] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:01.015833 24287 ts_tablet_manager.cc:1434] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.052s	user 0.008s	sys 0.000s
I20260709 12:16:01.025168 24288 tablet_bootstrap.cc:654] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.028170 24285 tablet_bootstrap.cc:654] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.029552 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4c368e2cc6c4869a85a53ae022a0785" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
W20260709 12:16:01.033416 24130 consensus_peers.cc:597] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:01.033942 24241 raft_consensus.cc:2393] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c359614416104a7ba9db1ede97d13802 in current term 1: Already voted for candidate 008a79e516614c809c2d3afdb5cbc327 in this term.
I20260709 12:16:01.036439 24418 raft_consensus.cc:493] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.037068 24418 raft_consensus.cc:515] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:01.037314 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4638bcb9174442184bc8d8a27d45072" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:01.038070 24241 raft_consensus.cc:3060] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:01.040318 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4638bcb9174442184bc8d8a27d45072" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:01.040326 24418 leader_election.cc:290] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:01.041222 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4638bcb9174442184bc8d8a27d45072" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:16:01.041905 24018 raft_consensus.cc:3060] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:01.042373 23983 leader_election.cc:304] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:01.042954 24286 tablet_bootstrap.cc:492] T be4e681eef304bb8bc812e313a67414d P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:01.043300 24499 raft_consensus.cc:3060] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:01.043475 24286 ts_tablet_manager.cc:1403] T be4e681eef304bb8bc812e313a67414d P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.193s	user 0.015s	sys 0.010s
I20260709 12:16:01.041237 24163 raft_consensus.cc:2393] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c359614416104a7ba9db1ede97d13802 in current term 1: Already voted for candidate 008a79e516614c809c2d3afdb5cbc327 in this term.
I20260709 12:16:01.048465 24367 raft_consensus.cc:2749] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:01.049434 24356 leader_election.cc:290] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:01.054106 24432 consensus_queue.cc:1048] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:01.056360 24287 tablet_bootstrap.cc:492] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:01.061347 24286 raft_consensus.cc:359] T be4e681eef304bb8bc812e313a67414d P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:01.062187 24286 raft_consensus.cc:385] T be4e681eef304bb8bc812e313a67414d P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.062538 24286 raft_consensus.cc:740] T be4e681eef304bb8bc812e313a67414d P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.065228 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4638bcb9174442184bc8d8a27d45072" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:01.066548 24286 consensus_queue.cc:260] T be4e681eef304bb8bc812e313a67414d P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:01.076257 23983 leader_election.cc:304] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:01.080375 24499 raft_consensus.cc:2749] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:01.083232 24285 tablet_bootstrap.cc:492] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:01.083850 24285 ts_tablet_manager.cc:1403] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.097s	user 0.008s	sys 0.008s
I20260709 12:16:01.085009 24286 ts_tablet_manager.cc:1434] T be4e681eef304bb8bc812e313a67414d P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.041s	user 0.008s	sys 0.000s
I20260709 12:16:01.086153 24286 tablet_bootstrap.cc:492] T 7e76d3d2239249bb806deb5ad8232c1c P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
W20260709 12:16:01.087538 23963 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:01.091913 24285 raft_consensus.cc:359] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:01.103915 24287 tablet_bootstrap.cc:654] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.104177 24285 raft_consensus.cc:385] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.092100 24018 raft_consensus.cc:2468] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:16:01.104789 24285 raft_consensus.cc:740] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.105338 24418 raft_consensus.cc:2749] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:01.105635 24285 consensus_queue.cc:260] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:01.108556 24285 ts_tablet_manager.cc:1434] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.024s	user 0.007s	sys 0.000s
I20260709 12:16:01.109937 24285 tablet_bootstrap.cc:492] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:01.103915 24286 tablet_bootstrap.cc:654] T 7e76d3d2239249bb806deb5ad8232c1c P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.117498 24288 tablet_bootstrap.cc:492] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:01.118229 24288 ts_tablet_manager.cc:1403] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.247s	user 0.008s	sys 0.012s
I20260709 12:16:01.121544 24285 tablet_bootstrap.cc:654] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.124681 24432 consensus_queue.cc:1048] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:01.126143 24418 consensus_queue.cc:1048] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:01.125483 24288 raft_consensus.cc:359] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:01.128700 24130 leader_election.cc:304] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:01.129725 24288 raft_consensus.cc:385] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.091810 24241 raft_consensus.cc:2468] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:16:01.132776 24288 raft_consensus.cc:740] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.133769 24288 consensus_queue.cc:260] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:01.140398 24432 raft_consensus.cc:2804] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:01.158244 24432 raft_consensus.cc:697] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:01.158968 24288 ts_tablet_manager.cc:1434] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.040s	user 0.008s	sys 0.002s
I20260709 12:16:01.159689 24432 consensus_queue.cc:237] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:01.166145 24418 consensus_queue.cc:1048] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:01.172438 24288 tablet_bootstrap.cc:492] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:16:01.172174 23914 catalog_manager.cc:5697] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132). New cstate: current_term: 1 leader_uuid: "a0df0eff47b64e3eb45ab074139ed187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:01.201368 24356 raft_consensus.cc:493] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.202033 24356 raft_consensus.cc:515] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:01.203575 24288 tablet_bootstrap.cc:654] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.213722 24356 leader_election.cc:290] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:01.218173 24286 tablet_bootstrap.cc:492] T 7e76d3d2239249bb806deb5ad8232c1c P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:01.218752 24286 ts_tablet_manager.cc:1403] T 7e76d3d2239249bb806deb5ad8232c1c P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.133s	user 0.017s	sys 0.011s
I20260709 12:16:01.221545 24286 raft_consensus.cc:359] T 7e76d3d2239249bb806deb5ad8232c1c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:01.222074 24286 raft_consensus.cc:385] T 7e76d3d2239249bb806deb5ad8232c1c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.223593 24286 raft_consensus.cc:740] T 7e76d3d2239249bb806deb5ad8232c1c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.224321 24286 consensus_queue.cc:260] T 7e76d3d2239249bb806deb5ad8232c1c P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:01.230400 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e76d3d2239249bb806deb5ad8232c1c" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:01.243453 24287 tablet_bootstrap.cc:492] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:01.234217 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e76d3d2239249bb806deb5ad8232c1c" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:01.244166 24287 ts_tablet_manager.cc:1403] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.188s	user 0.026s	sys 0.032s
I20260709 12:16:01.243783 24093 raft_consensus.cc:2468] T 7e76d3d2239249bb806deb5ad8232c1c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:16:01.247423 24285 tablet_bootstrap.cc:492] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:01.252609 24285 ts_tablet_manager.cc:1403] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.143s	user 0.013s	sys 0.014s
I20260709 12:16:01.250798 24453 raft_consensus.cc:493] T be4e681eef304bb8bc812e313a67414d P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.248312 24130 leader_election.cc:304] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: 
I20260709 12:16:01.248478 24367 raft_consensus.cc:493] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.253871 24453 raft_consensus.cc:515] T be4e681eef304bb8bc812e313a67414d P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:01.254465 24367 raft_consensus.cc:515] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:01.255003 24287 raft_consensus.cc:359] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:01.255795 24287 raft_consensus.cc:385] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.256083 24287 raft_consensus.cc:740] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.257305 24453 leader_election.cc:290] T be4e681eef304bb8bc812e313a67414d P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:01.257563 24464 raft_consensus.cc:2804] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:01.233134 24286 ts_tablet_manager.cc:1434] T 7e76d3d2239249bb806deb5ad8232c1c P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.014s	user 0.007s	sys 0.000s
I20260709 12:16:01.259109 24286 tablet_bootstrap.cc:492] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:01.259135 24287 consensus_queue.cc:260] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:01.262945 24418 consensus_queue.cc:1048] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:16:01.270819 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be4e681eef304bb8bc812e313a67414d" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:01.272126 24018 raft_consensus.cc:2393] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c74a5203b594708921cd793aaca0b8b in current term 1: Already voted for candidate a0df0eff47b64e3eb45ab074139ed187 in this term.
I20260709 12:16:01.258090 24464 raft_consensus.cc:493] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:01.273954 24464 raft_consensus.cc:3060] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:01.278715 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3effea5c11740418bd8b7bedd33e0f5" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:01.263371 24468 consensus_queue.cc:1048] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 12:16:01.262059 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be4e681eef304bb8bc812e313a67414d" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
W20260709 12:16:01.283672 24056 leader_election.cc:343] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:01.285606 24058 leader_election.cc:304] T be4e681eef304bb8bc812e313a67414d P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802
I20260709 12:16:01.287102 24367 leader_election.cc:290] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:01.287678 24453 raft_consensus.cc:3060] T be4e681eef304bb8bc812e313a67414d P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:01.263545 24130 leader_election.cc:343] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:01.287577 24285 raft_consensus.cc:359] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:01.293295 24285 raft_consensus.cc:385] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.293895 24285 raft_consensus.cc:740] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.294441 24287 ts_tablet_manager.cc:1434] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.050s	user 0.009s	sys 0.003s
I20260709 12:16:01.295536 24287 tablet_bootstrap.cc:492] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:01.294890 24285 consensus_queue.cc:260] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:01.300302 24464 raft_consensus.cc:515] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:01.306066 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3effea5c11740418bd8b7bedd33e0f5" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:01.307197 24163 raft_consensus.cc:2468] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:16:01.315248 24296 raft_consensus.cc:493] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.315434 24056 leader_election.cc:304] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:01.315977 24296 raft_consensus.cc:515] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:01.321647 24367 raft_consensus.cc:2804] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:01.322252 24367 raft_consensus.cc:493] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:01.322697 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2eabb81feec483ea9b303c5e516b7cf" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:01.326730 23912 catalog_manager.cc:5697] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:01.322619 24367 raft_consensus.cc:3060] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:01.329170 24204 leader_election.cc:343] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:01.332871 24296 leader_election.cc:290] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:01.343852 24453 raft_consensus.cc:2749] T be4e681eef304bb8bc812e313a67414d P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:01.351755 24285 ts_tablet_manager.cc:1434] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.099s	user 0.009s	sys 0.000s
I20260709 12:16:01.353368 24285 tablet_bootstrap.cc:492] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:01.368438 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2eabb81feec483ea9b303c5e516b7cf" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
W20260709 12:16:01.373580 24204 leader_election.cc:343] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:01.374092 24204 leader_election.cc:304] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:01.375811 24296 raft_consensus.cc:2749] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:01.393543 24286 tablet_bootstrap.cc:654] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.389954 24499 consensus_queue.cc:1048] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
W20260709 12:16:01.391880 24056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:36699, user_credentials={real_user=slave}} blocked reactor thread for 76612.8us
I20260709 12:16:01.406919 24464 leader_election.cc:290] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:01.411808 24287 tablet_bootstrap.cc:654] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.412930 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e76d3d2239249bb806deb5ad8232c1c" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:16:01.411759 24285 tablet_bootstrap.cc:654] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.416232 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e76d3d2239249bb806deb5ad8232c1c" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:01.423995 24296 raft_consensus.cc:493] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.430045 24536 raft_consensus.cc:493] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.430725 24536 raft_consensus.cc:515] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:01.432231 24288 tablet_bootstrap.cc:492] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:01.432941 24288 ts_tablet_manager.cc:1403] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.261s	user 0.021s	sys 0.035s
W20260709 12:16:01.427290 24130 leader_election.cc:343] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:01.425304 24091 raft_consensus.cc:3060] T 7e76d3d2239249bb806deb5ad8232c1c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:01.424638 24296 raft_consensus.cc:515] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:01.433102 24536 leader_election.cc:290] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:01.434615 24517 consensus_queue.cc:1048] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.011s
I20260709 12:16:01.445725 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ed2939d1454de181106eec39d0e4bc" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:01.449371 24288 raft_consensus.cc:359] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:01.440382 24296 leader_election.cc:290] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:01.435473 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be5ea3171ad74d74bede5875414d92ac" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:01.452430 24288 raft_consensus.cc:385] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.435617 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be5ea3171ad74d74bede5875414d92ac" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:01.454088 24288 raft_consensus.cc:740] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.456547 24288 consensus_queue.cc:260] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
W20260709 12:16:01.460656 24204 leader_election.cc:343] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:01.437690 24367 raft_consensus.cc:515] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
W20260709 12:16:01.461963 23983 leader_election.cc:343] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:01.463647 24367 leader_election.cc:290] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:01.463930 24286 tablet_bootstrap.cc:492] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:01.464933 24286 ts_tablet_manager.cc:1403] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.206s	user 0.023s	sys 0.000s
I20260709 12:16:01.466475 24287 tablet_bootstrap.cc:492] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:01.467042 24287 ts_tablet_manager.cc:1403] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.172s	user 0.018s	sys 0.002s
I20260709 12:16:01.467975 24286 raft_consensus.cc:359] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:01.468798 24286 raft_consensus.cc:385] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.469107 24286 raft_consensus.cc:740] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.470110 24286 consensus_queue.cc:260] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:01.470391 24287 raft_consensus.cc:359] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:01.471240 24287 raft_consensus.cc:385] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.471531 24287 raft_consensus.cc:740] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.472221 24287 consensus_queue.cc:260] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:01.479136 24286 ts_tablet_manager.cc:1434] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.014s	user 0.009s	sys 0.000s
I20260709 12:16:01.480561 24288 ts_tablet_manager.cc:1434] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.047s	user 0.004s	sys 0.004s
I20260709 12:16:01.480634 24286 tablet_bootstrap.cc:492] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:01.481870 24288 tablet_bootstrap.cc:492] T c385253c4ad94570b7dbb6875a755b36 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:16:01.484323 24287 ts_tablet_manager.cc:1434] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.017s	user 0.007s	sys 0.000s
I20260709 12:16:01.455018 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ed2939d1454de181106eec39d0e4bc" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:01.496454 24464 raft_consensus.cc:493] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.497149 24464 raft_consensus.cc:515] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:01.498944 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3effea5c11740418bd8b7bedd33e0f5" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:16:01.499549 24464 leader_election.cc:290] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:01.502269 24286 tablet_bootstrap.cc:654] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.509423 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e3897eed824c5fa96e04ce0eca32a8" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:01.511093 24375 consensus_queue.cc:1048] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:01.514441 24091 raft_consensus.cc:2468] T 7e76d3d2239249bb806deb5ad8232c1c P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:16:01.514557 24092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e3897eed824c5fa96e04ce0eca32a8" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
W20260709 12:16:01.518123 23981 leader_election.cc:343] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:01.518999 23981 leader_election.cc:304] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:01.520620 24499 raft_consensus.cc:2749] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:01.525341 24287 tablet_bootstrap.cc:492] T c385253c4ad94570b7dbb6875a755b36 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
W20260709 12:16:01.527544 24130 leader_election.cc:343] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:01.528676 24130 leader_election.cc:304] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: c359614416104a7ba9db1ede97d13802
W20260709 12:16:01.529927 24130 leader_election.cc:343] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:01.532284 24296 raft_consensus.cc:493] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.535207 24296 raft_consensus.cc:515] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
W20260709 12:16:01.543708 24204 leader_election.cc:343] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:01.544255 24204 leader_election.cc:304] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:16:01.544596 24287 tablet_bootstrap.cc:654] T c385253c4ad94570b7dbb6875a755b36 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.533842 24464 raft_consensus.cc:2804] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:01.547192 24288 tablet_bootstrap.cc:654] T c385253c4ad94570b7dbb6875a755b36 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:01.548269 24056 leader_election.cc:343] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:01.549225 24432 raft_consensus.cc:493] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.549805 24432 raft_consensus.cc:515] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:01.550861 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3effea5c11740418bd8b7bedd33e0f5" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:01.551535 24163 raft_consensus.cc:3060] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:01.552119 24432 leader_election.cc:290] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:01.549129 24464 raft_consensus.cc:697] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:01.546562 24335 raft_consensus.cc:2749] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:01.554323 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba257bd0f16a4a7da0cb4a7953ca1869" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:01.555145 24241 raft_consensus.cc:2468] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:16:01.555121 24464 consensus_queue.cc:237] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:01.557273 24296 leader_election.cc:290] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:01.561000 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba257bd0f16a4a7da0cb4a7953ca1869" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:01.566219 24165 raft_consensus.cc:2468] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 0.
I20260709 12:16:01.567266 24503 raft_consensus.cc:493] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.568841 24204 leader_election.cc:304] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:01.570013 24296 raft_consensus.cc:2804] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:01.570604 24296 raft_consensus.cc:493] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:01.571110 24296 raft_consensus.cc:3060] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:01.568264 24503 raft_consensus.cc:515] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:01.582033 24286 tablet_bootstrap.cc:492] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:01.582763 24286 ts_tablet_manager.cc:1403] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.103s	user 0.014s	sys 0.004s
I20260709 12:16:01.585032 24503 leader_election.cc:290] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:01.539600 24130 leader_election.cc:304] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:16:01.589365 24432 raft_consensus.cc:2749] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:01.586179 24286 raft_consensus.cc:359] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:01.592780 24286 raft_consensus.cc:385] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.593156 24286 raft_consensus.cc:740] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.597810 24286 consensus_queue.cc:260] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:01.566095 24132 leader_election.cc:304] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:01.585780 24163 raft_consensus.cc:2468] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
W20260709 12:16:01.611292 24132 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:41011, user_credentials={real_user=slave}} blocked reactor thread for 46768.2us
I20260709 12:16:01.612341 24432 raft_consensus.cc:2804] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:01.612828 24432 raft_consensus.cc:493] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:01.613202 24432 raft_consensus.cc:3060] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:01.618701 24286 ts_tablet_manager.cc:1434] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.036s	user 0.009s	sys 0.000s
I20260709 12:16:01.619066 24296 raft_consensus.cc:515] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:01.619931 24286 tablet_bootstrap.cc:492] T 3bd7c9c11f834f15974a65477ef9258a P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:01.621400 24296 leader_election.cc:290] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Requested vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:01.622219 24285 tablet_bootstrap.cc:492] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:01.622987 24285 ts_tablet_manager.cc:1403] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.270s	user 0.027s	sys 0.017s
I20260709 12:16:01.629251 24285 raft_consensus.cc:359] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:01.630386 24285 raft_consensus.cc:385] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.630730 24285 raft_consensus.cc:740] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.633464 24286 tablet_bootstrap.cc:654] T 3bd7c9c11f834f15974a65477ef9258a P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.644235 24287 tablet_bootstrap.cc:492] T c385253c4ad94570b7dbb6875a755b36 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:01.630561 24432 raft_consensus.cc:515] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:01.645087 24287 ts_tablet_manager.cc:1403] T c385253c4ad94570b7dbb6875a755b36 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.120s	user 0.009s	sys 0.027s
I20260709 12:16:01.647424 24288 tablet_bootstrap.cc:492] T c385253c4ad94570b7dbb6875a755b36 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:01.647709 24432 leader_election.cc:290] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:01.648204 24288 ts_tablet_manager.cc:1403] T c385253c4ad94570b7dbb6875a755b36 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.166s	user 0.027s	sys 0.006s
I20260709 12:16:01.649019 24287 raft_consensus.cc:359] T c385253c4ad94570b7dbb6875a755b36 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:01.649964 24287 raft_consensus.cc:385] T c385253c4ad94570b7dbb6875a755b36 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.650298 24287 raft_consensus.cc:740] T c385253c4ad94570b7dbb6875a755b36 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.651594 24286 tablet_bootstrap.cc:492] T 3bd7c9c11f834f15974a65477ef9258a P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:01.651336 24287 consensus_queue.cc:260] T c385253c4ad94570b7dbb6875a755b36 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:01.652249 24286 ts_tablet_manager.cc:1403] T 3bd7c9c11f834f15974a65477ef9258a P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.033s	user 0.014s	sys 0.001s
I20260709 12:16:01.651875 24288 raft_consensus.cc:359] T c385253c4ad94570b7dbb6875a755b36 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:01.653681 24288 raft_consensus.cc:385] T c385253c4ad94570b7dbb6875a755b36 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.653968 24288 raft_consensus.cc:740] T c385253c4ad94570b7dbb6875a755b36 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.654198 24287 ts_tablet_manager.cc:1434] T c385253c4ad94570b7dbb6875a755b36 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.001s
I20260709 12:16:01.654722 24288 consensus_queue.cc:260] T c385253c4ad94570b7dbb6875a755b36 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:01.655846 24287 tablet_bootstrap.cc:492] T 1e98aa59558f4e58925a3c87026b6fa4 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:01.657338 24288 ts_tablet_manager.cc:1434] T c385253c4ad94570b7dbb6875a755b36 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:16:01.567979 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba257bd0f16a4a7da0cb4a7953ca1869" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:01.731925 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95824f48a95746289333a9da8e265178" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
W20260709 12:16:01.597716 24130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:42247, user_credentials={real_user=slave}} blocked reactor thread for 63828.3us
W20260709 12:16:01.739318 24204 leader_election.cc:343] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:01.739588 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba257bd0f16a4a7da0cb4a7953ca1869" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:01.741565 24165 raft_consensus.cc:2393] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a0df0eff47b64e3eb45ab074139ed187 in current term 1: Already voted for candidate 008a79e516614c809c2d3afdb5cbc327 in this term.
I20260709 12:16:01.747505 24288 tablet_bootstrap.cc:492] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:16:01.748047 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba257bd0f16a4a7da0cb4a7953ca1869" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:01.749509 24241 raft_consensus.cc:2393] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 008a79e516614c809c2d3afdb5cbc327 in current term 1: Already voted for candidate a0df0eff47b64e3eb45ab074139ed187 in this term.
I20260709 12:16:01.657919 24286 raft_consensus.cc:359] T 3bd7c9c11f834f15974a65477ef9258a P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:01.758635 24285 consensus_queue.cc:260] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:01.759611 24367 raft_consensus.cc:493] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.761078 24367 raft_consensus.cc:515] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:01.766389 24286 raft_consensus.cc:385] T 3bd7c9c11f834f15974a65477ef9258a P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:01.766731 24056 leader_election.cc:304] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:01.768271 24239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c385253c4ad94570b7dbb6875a755b36" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:01.778421 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba257bd0f16a4a7da0cb4a7953ca1869" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:16:01.778785 24239 raft_consensus.cc:2468] T c385253c4ad94570b7dbb6875a755b36 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:16:01.781306 24286 raft_consensus.cc:740] T 3bd7c9c11f834f15974a65477ef9258a P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:01.782516 24286 consensus_queue.cc:260] T 3bd7c9c11f834f15974a65477ef9258a P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:01.783358 24058 leader_election.cc:304] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
W20260709 12:16:01.777339 24056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:36699, user_credentials={real_user=slave}} blocked reactor thread for 47239.8us
I20260709 12:16:01.773576 24287 tablet_bootstrap.cc:654] T 1e98aa59558f4e58925a3c87026b6fa4 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:01.806842 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba257bd0f16a4a7da0cb4a7953ca1869" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
W20260709 12:16:01.810024 23981 leader_election.cc:343] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:01.814296 24553 raft_consensus.cc:2804] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:01.817047 24553 raft_consensus.cc:697] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Becoming Leader. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Running, Role: LEADER
I20260709 12:16:01.819237 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba257bd0f16a4a7da0cb4a7953ca1869" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:01.819042 24553 consensus_queue.cc:237] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:01.815156 24468 consensus_queue.cc:1048] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:01.826447 24552 raft_consensus.cc:493] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.827706 24503 raft_consensus.cc:493] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.827392 24552 raft_consensus.cc:515] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:01.828301 24503 raft_consensus.cc:515] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:01.831091 24503 leader_election.cc:290] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
W20260709 12:16:01.835038 24058 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:41011, user_credentials={real_user=slave}} blocked reactor thread for 52102.3us
I20260709 12:16:01.836422 24453 raft_consensus.cc:2804] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:01.837071 24453 raft_consensus.cc:493] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:01.837589 24453 raft_consensus.cc:3060] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:01.847301 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4765d7c78c480fa4ba08d2812aeac3" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:01.850690 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c385253c4ad94570b7dbb6875a755b36" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:01.851995 24165 raft_consensus.cc:2468] T c385253c4ad94570b7dbb6875a755b36 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:16:01.859259 24285 ts_tablet_manager.cc:1434] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.236s	user 0.017s	sys 0.000s
I20260709 12:16:01.805460 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95824f48a95746289333a9da8e265178" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:01.866356 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3effea5c11740418bd8b7bedd33e0f5" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:01.870442 24552 leader_election.cc:290] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:01.765972 24367 leader_election.cc:290] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:01.908097 24286 ts_tablet_manager.cc:1434] T 3bd7c9c11f834f15974a65477ef9258a P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.255s	user 0.028s	sys 0.024s
I20260709 12:16:01.912993 24240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4765d7c78c480fa4ba08d2812aeac3" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:01.920513 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3effea5c11740418bd8b7bedd33e0f5" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:01.923049 24286 tablet_bootstrap.cc:492] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:01.927202 24165 raft_consensus.cc:2393] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c359614416104a7ba9db1ede97d13802 in current term 1: Already voted for candidate 6c74a5203b594708921cd793aaca0b8b in this term.
I20260709 12:16:01.923303 24288 tablet_bootstrap.cc:654] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:01.942473 24130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:42247, user_credentials={real_user=slave}} blocked reactor thread for 112488us
W20260709 12:16:01.943192 24058 leader_election.cc:343] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:01.972671 24367 raft_consensus.cc:493] T 3bd7c9c11f834f15974a65477ef9258a P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:01.981577 24367 raft_consensus.cc:515] T 3bd7c9c11f834f15974a65477ef9258a P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:01.984721 24367 leader_election.cc:290] T 3bd7c9c11f834f15974a65477ef9258a P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
W20260709 12:16:01.987933 24130 leader_election.cc:343] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:01.988583 24130 leader_election.cc:304] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327; no voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802
I20260709 12:16:01.991554 24286 tablet_bootstrap.cc:654] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:01.992079 24204 leader_election.cc:343] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:01.993136 24204 leader_election.cc:304] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:01.993203 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bd7c9c11f834f15974a65477ef9258a" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:02.003099 24568 raft_consensus.cc:2749] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:02.005359 24285 tablet_bootstrap.cc:492] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
W20260709 12:16:02.025094 24132 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:41011, user_credentials={real_user=slave}} blocked reactor thread for 65105.5us
I20260709 12:16:02.029570 23915 catalog_manager.cc:5697] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:02.035676 24130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:42669, user_credentials={real_user=slave}} blocked reactor thread for 47830.2us
W20260709 12:16:02.036958 24130 leader_election.cc:343] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:02.042310 24056 leader_election.cc:343] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.043040 24056 leader_election.cc:304] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:02.045441 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3bd7c9c11f834f15974a65477ef9258a" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:02.045719 24572 raft_consensus.cc:2749] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:16:02.052547 23981 leader_election.cc:343] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.060390 23981 leader_election.cc:304] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b
I20260709 12:16:02.075139 24288 tablet_bootstrap.cc:492] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
W20260709 12:16:02.065869 23981 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:36699, user_credentials={real_user=slave}} blocked reactor thread for 54720.4us
I20260709 12:16:02.098699 23981 leader_election.cc:304] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b
I20260709 12:16:02.100334 24536 raft_consensus.cc:3060] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:02.110443 24130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:42669, user_credentials={real_user=slave}} blocked reactor thread for 73573.4us
I20260709 12:16:02.065907 24503 raft_consensus.cc:2749] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:02.068025 24569 raft_consensus.cc:2749] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:02.065505 24287 tablet_bootstrap.cc:492] T 1e98aa59558f4e58925a3c87026b6fa4 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:02.116394 24288 ts_tablet_manager.cc:1403] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.458s	user 0.028s	sys 0.017s
I20260709 12:16:02.117496 24287 ts_tablet_manager.cc:1403] T 1e98aa59558f4e58925a3c87026b6fa4 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.462s	user 0.014s	sys 0.029s
I20260709 12:16:02.118559 24536 raft_consensus.cc:2749] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:02.068933 24204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:42669, user_credentials={real_user=slave}} blocked reactor thread for 76946.9us
I20260709 12:16:02.116245 24056 leader_election.cc:304] T 3bd7c9c11f834f15974a65477ef9258a P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:02.130533 24553 raft_consensus.cc:2749] T 3bd7c9c11f834f15974a65477ef9258a P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:02.125085 24468 consensus_queue.cc:1048] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.017s
I20260709 12:16:02.124012 24288 raft_consensus.cc:359] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:02.147277 24288 raft_consensus.cc:385] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.147800 24288 raft_consensus.cc:740] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.154282 24285 tablet_bootstrap.cc:654] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.168735 24286 tablet_bootstrap.cc:492] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:02.169397 24286 ts_tablet_manager.cc:1403] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.247s	user 0.020s	sys 0.038s
I20260709 12:16:02.060963 24453 raft_consensus.cc:515] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:02.188515 24286 raft_consensus.cc:359] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:02.189658 24286 raft_consensus.cc:385] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.190037 24286 raft_consensus.cc:740] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.191143 24286 consensus_queue.cc:260] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:02.194931 24092 raft_consensus.cc:3060] T 3bd7c9c11f834f15974a65477ef9258a P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.196938 24286 ts_tablet_manager.cc:1434] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.027s	user 0.012s	sys 0.000s
I20260709 12:16:02.165719 23911 catalog_manager.cc:5697] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c74a5203b594708921cd793aaca0b8b (127.11.13.130). New cstate: current_term: 1 leader_uuid: "6c74a5203b594708921cd793aaca0b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:02.199080 24286 tablet_bootstrap.cc:492] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:02.208884 24288 consensus_queue.cc:260] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:02.245301 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c385253c4ad94570b7dbb6875a755b36" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:02.251662 24166 raft_consensus.cc:3060] T c385253c4ad94570b7dbb6875a755b36 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.270464 24569 consensus_queue.cc:1048] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:02.276942 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c385253c4ad94570b7dbb6875a755b36" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:02.278080 24241 raft_consensus.cc:3060] T c385253c4ad94570b7dbb6875a755b36 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.286904 24286 tablet_bootstrap.cc:654] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.288455 24166 raft_consensus.cc:2468] T c385253c4ad94570b7dbb6875a755b36 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:16:02.288084 24453 leader_election.cc:290] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Requested vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:02.295894 24288 ts_tablet_manager.cc:1434] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.178s	user 0.010s	sys 0.000s
I20260709 12:16:02.296314 24287 raft_consensus.cc:359] T 1e98aa59558f4e58925a3c87026b6fa4 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:02.299474 24285 tablet_bootstrap.cc:492] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:02.299903 24287 raft_consensus.cc:385] T 1e98aa59558f4e58925a3c87026b6fa4 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.300433 24287 raft_consensus.cc:740] T 1e98aa59558f4e58925a3c87026b6fa4 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.300539 24285 ts_tablet_manager.cc:1403] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.333s	user 0.011s	sys 0.015s
I20260709 12:16:02.297482 24288 tablet_bootstrap.cc:492] T 95824f48a95746289333a9da8e265178 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:16:02.306706 24285 raft_consensus.cc:359] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:02.307569 24285 raft_consensus.cc:385] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.307857 24285 raft_consensus.cc:740] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.308666 24285 consensus_queue.cc:260] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:02.326628 24453 raft_consensus.cc:493] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.328902 24287 consensus_queue.cc:260] T 1e98aa59558f4e58925a3c87026b6fa4 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:02.334529 24056 leader_election.cc:304] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: 
I20260709 12:16:02.337598 24570 raft_consensus.cc:2804] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:02.338217 24570 raft_consensus.cc:697] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Becoming Leader. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Running, Role: LEADER
I20260709 12:16:02.339308 24570 consensus_queue.cc:237] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:02.328563 24453 raft_consensus.cc:515] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:02.344794 24453 leader_election.cc:290] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:02.345552 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ed2939d1454de181106eec39d0e4bc" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:02.346526 24238 raft_consensus.cc:2468] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:16:02.349792 24058 leader_election.cc:304] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:02.350824 24552 raft_consensus.cc:2804] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:02.351053 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ed2939d1454de181106eec39d0e4bc" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:02.351291 24552 raft_consensus.cc:493] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:02.353511 24552 raft_consensus.cc:3060] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.364046 24017 raft_consensus.cc:3060] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.367625 24356 consensus_queue.cc:1048] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:02.372177 24552 raft_consensus.cc:515] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:02.376471 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ed2939d1454de181106eec39d0e4bc" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:02.377277 24238 raft_consensus.cc:3060] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.377637 24286 tablet_bootstrap.cc:492] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
W20260709 12:16:02.377875 24056 leader_election.cc:343] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.356122 24241 raft_consensus.cc:2468] T c385253c4ad94570b7dbb6875a755b36 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:16:02.378327 24286 ts_tablet_manager.cc:1403] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.180s	user 0.022s	sys 0.017s
I20260709 12:16:02.378993 24552 leader_election.cc:290] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:02.383643 24287 ts_tablet_manager.cc:1434] T 1e98aa59558f4e58925a3c87026b6fa4 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.266s	user 0.000s	sys 0.010s
I20260709 12:16:02.385418 24287 tablet_bootstrap.cc:492] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:02.386300 24367 consensus_queue.cc:1048] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:02.388605 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ed2939d1454de181106eec39d0e4bc" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:16:02.391012 24286 raft_consensus.cc:359] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:02.398488 24286 raft_consensus.cc:385] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.398979 24286 raft_consensus.cc:740] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.399847 24286 consensus_queue.cc:260] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
W20260709 12:16:02.401227 24056 leader_election.cc:343] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.398062 24287 tablet_bootstrap.cc:654] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.403600 24285 ts_tablet_manager.cc:1434] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.103s	user 0.007s	sys 0.001s
I20260709 12:16:02.406992 24468 consensus_queue.cc:1048] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:02.412878 24286 ts_tablet_manager.cc:1434] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 12:16:02.414358 24286 tablet_bootstrap.cc:492] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:02.423954 24356 raft_consensus.cc:493] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.424757 24356 raft_consensus.cc:515] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:02.430352 24288 tablet_bootstrap.cc:654] T 95824f48a95746289333a9da8e265178 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.443081 24356 leader_election.cc:290] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:02.444350 24285 tablet_bootstrap.cc:492] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:02.445067 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e98aa59558f4e58925a3c87026b6fa4" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:02.446381 24241 raft_consensus.cc:2468] T 1e98aa59558f4e58925a3c87026b6fa4 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:16:02.447923 24089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e98aa59558f4e58925a3c87026b6fa4" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:02.448616 24132 leader_election.cc:304] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:02.450822 24468 raft_consensus.cc:2804] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:02.451391 24468 raft_consensus.cc:493] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:02.451840 24468 raft_consensus.cc:3060] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:02.454643 24130 leader_election.cc:343] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:02.464485 24585 log.cc:927] Time spent T 3bd7c9c11f834f15974a65477ef9258a P 6c74a5203b594708921cd793aaca0b8b: Append to log took a long time: real 0.085s	user 0.001s	sys 0.000s
I20260709 12:16:02.482862 24285 tablet_bootstrap.cc:654] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.493096 24288 tablet_bootstrap.cc:492] T 95824f48a95746289333a9da8e265178 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:02.493824 24288 ts_tablet_manager.cc:1403] T 95824f48a95746289333a9da8e265178 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.197s	user 0.011s	sys 0.007s
I20260709 12:16:02.496636 24286 tablet_bootstrap.cc:654] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.503118 24468 raft_consensus.cc:515] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:02.505151 24238 raft_consensus.cc:2468] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:16:02.505504 24468 leader_election.cc:290] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:02.506531 24058 leader_election.cc:304] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:02.508111 24552 raft_consensus.cc:2804] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:02.508961 24552 raft_consensus.cc:697] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Becoming Leader. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Running, Role: LEADER
I20260709 12:16:02.519922 24287 tablet_bootstrap.cc:492] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:02.520576 24287 ts_tablet_manager.cc:1403] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.135s	user 0.012s	sys 0.008s
I20260709 12:16:02.520529 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e98aa59558f4e58925a3c87026b6fa4" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:02.520903 24092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e98aa59558f4e58925a3c87026b6fa4" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:02.521380 24241 raft_consensus.cc:3060] T 1e98aa59558f4e58925a3c87026b6fa4 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:02.522593 24130 leader_election.cc:343] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.524133 24552 consensus_queue.cc:237] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:02.526587 24287 raft_consensus.cc:359] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:02.527412 24287 raft_consensus.cc:385] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.528756 24287 raft_consensus.cc:740] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.531033 24287 consensus_queue.cc:260] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:02.546689 24288 raft_consensus.cc:359] T 95824f48a95746289333a9da8e265178 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:02.547814 24288 raft_consensus.cc:385] T 95824f48a95746289333a9da8e265178 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.548189 24288 raft_consensus.cc:740] T 95824f48a95746289333a9da8e265178 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.549126 24288 consensus_queue.cc:260] T 95824f48a95746289333a9da8e265178 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:02.552145 24287 ts_tablet_manager.cc:1434] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.031s	user 0.006s	sys 0.001s
I20260709 12:16:02.553865 24287 tablet_bootstrap.cc:492] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:02.566814 23914 catalog_manager.cc:5697] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c74a5203b594708921cd793aaca0b8b (127.11.13.130). New cstate: current_term: 1 leader_uuid: "6c74a5203b594708921cd793aaca0b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:02.582197 24288 ts_tablet_manager.cc:1434] T 95824f48a95746289333a9da8e265178 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.088s	user 0.009s	sys 0.000s
I20260709 12:16:02.583851 24288 tablet_bootstrap.cc:492] T be5ea3171ad74d74bede5875414d92ac P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
W20260709 12:16:02.586638 23963 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:02.589521 24286 tablet_bootstrap.cc:492] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:02.602267 24286 ts_tablet_manager.cc:1403] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.188s	user 0.030s	sys 0.000s
I20260709 12:16:02.595016 24367 raft_consensus.cc:493] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.600486 24288 tablet_bootstrap.cc:654] T be5ea3171ad74d74bede5875414d92ac P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.592620 24287 tablet_bootstrap.cc:654] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.600807 23911 catalog_manager.cc:5697] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c74a5203b594708921cd793aaca0b8b (127.11.13.130). New cstate: current_term: 1 leader_uuid: "6c74a5203b594708921cd793aaca0b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:02.613839 24367 raft_consensus.cc:515] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:02.616276 24367 leader_election.cc:290] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:02.604264 24241 raft_consensus.cc:2468] T 1e98aa59558f4e58925a3c87026b6fa4 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:16:02.617600 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "131c7392ac614d87ab3d9b6713a46f7b" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:02.619491 24132 leader_election.cc:304] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b
I20260709 12:16:02.620463 24568 raft_consensus.cc:2804] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:16:02.621668 24058 leader_election.cc:343] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.622221 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "131c7392ac614d87ab3d9b6713a46f7b" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:02.624547 24568 raft_consensus.cc:697] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:02.625813 24568 consensus_queue.cc:237] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
W20260709 12:16:02.630703 24056 leader_election.cc:343] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.631532 24056 leader_election.cc:304] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802
I20260709 12:16:02.633579 24453 raft_consensus.cc:2749] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:02.616897 24296 raft_consensus.cc:493] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.642578 24296 raft_consensus.cc:515] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:02.646350 24567 raft_consensus.cc:493] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.646938 24567 raft_consensus.cc:515] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:02.646628 24296 leader_election.cc:290] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:02.641521 24286 raft_consensus.cc:359] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:02.649945 24286 raft_consensus.cc:385] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.650437 24286 raft_consensus.cc:740] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.654529 24286 consensus_queue.cc:260] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:02.653967 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56fb07e9eb574b1fbc5da95063610dd5" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:02.656289 24567 leader_election.cc:290] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:02.659574 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be5ea3171ad74d74bede5875414d92ac" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:02.660940 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be5ea3171ad74d74bede5875414d92ac" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
W20260709 12:16:02.692366 24204 leader_election.cc:343] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.692817 24015 raft_consensus.cc:2468] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 0.
I20260709 12:16:02.693351 24286 ts_tablet_manager.cc:1434] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.090s	user 0.008s	sys 0.000s
I20260709 12:16:02.693655 24552 raft_consensus.cc:493] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.694303 24552 raft_consensus.cc:515] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:02.694896 24286 tablet_bootstrap.cc:492] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:02.663431 24285 tablet_bootstrap.cc:492] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:02.699461 24285 ts_tablet_manager.cc:1403] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.256s	user 0.035s	sys 0.032s
I20260709 12:16:02.699935 24287 tablet_bootstrap.cc:492] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:02.700628 24287 ts_tablet_manager.cc:1403] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.147s	user 0.049s	sys 0.018s
I20260709 12:16:02.703286 24286 tablet_bootstrap.cc:654] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.703761 24287 raft_consensus.cc:359] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:02.704540 24287 raft_consensus.cc:385] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.704825 24287 raft_consensus.cc:740] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.706634 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95824f48a95746289333a9da8e265178" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:02.707532 24015 raft_consensus.cc:2468] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:16:02.705729 24287 consensus_queue.cc:260] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:02.710167 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95824f48a95746289333a9da8e265178" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:02.710846 24166 raft_consensus.cc:2468] T 95824f48a95746289333a9da8e265178 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:16:02.712424 24056 leader_election.cc:304] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: 
I20260709 12:16:02.713393 24453 raft_consensus.cc:2804] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:02.713773 24453 raft_consensus.cc:493] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:02.714140 24453 raft_consensus.cc:3060] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.717283 24287 ts_tablet_manager.cc:1434] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.016s	user 0.008s	sys 0.000s
I20260709 12:16:02.718928 24287 tablet_bootstrap.cc:492] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:02.719556 24204 leader_election.cc:304] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802; no voters: 
W20260709 12:16:02.720557 24204 leader_election.cc:343] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.720784 24567 raft_consensus.cc:2804] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:02.669862 24288 tablet_bootstrap.cc:492] T be5ea3171ad74d74bede5875414d92ac P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:02.722877 24567 raft_consensus.cc:493] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:02.723187 24288 ts_tablet_manager.cc:1403] T be5ea3171ad74d74bede5875414d92ac P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.140s	user 0.027s	sys 0.001s
I20260709 12:16:02.723278 24567 raft_consensus.cc:3060] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.678246 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56fb07e9eb574b1fbc5da95063610dd5" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:02.726738 24285 raft_consensus.cc:359] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:02.727628 24285 raft_consensus.cc:385] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.728001 24285 raft_consensus.cc:740] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.728827 24285 consensus_queue.cc:260] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:02.730701 24367 consensus_queue.cc:1048] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:02.730608 24285 ts_tablet_manager.cc:1434] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.031s	user 0.007s	sys 0.000s
I20260709 12:16:02.735407 24552 leader_election.cc:290] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:02.736667 24285 tablet_bootstrap.cc:492] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
W20260709 12:16:02.746454 24204 leader_election.cc:343] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.737711 24288 raft_consensus.cc:359] T be5ea3171ad74d74bede5875414d92ac P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:02.747036 24204 leader_election.cc:304] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:16:02.747897 24296 raft_consensus.cc:2749] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:02.752242 23914 catalog_manager.cc:5697] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:02.747202 24288 raft_consensus.cc:385] T be5ea3171ad74d74bede5875414d92ac P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.759212 24288 raft_consensus.cc:740] T be5ea3171ad74d74bede5875414d92ac P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.760751 24288 consensus_queue.cc:260] T be5ea3171ad74d74bede5875414d92ac P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:02.746203 24567 raft_consensus.cc:515] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:02.753890 24285 tablet_bootstrap.cc:654] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.755266 24453 raft_consensus.cc:515] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:02.762557 24288 ts_tablet_manager.cc:1434] T be5ea3171ad74d74bede5875414d92ac P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.001s
I20260709 12:16:02.767156 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95824f48a95746289333a9da8e265178" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:16:02.767629 24567 leader_election.cc:290] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Requested vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:02.767843 24015 raft_consensus.cc:3060] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.768373 24288 tablet_bootstrap.cc:492] T 2e4765d7c78c480fa4ba08d2812aeac3 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:16:02.769097 24287 tablet_bootstrap.cc:654] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.772238 24453 leader_election.cc:290] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:02.774279 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95824f48a95746289333a9da8e265178" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:02.775259 24165 raft_consensus.cc:3060] T 95824f48a95746289333a9da8e265178 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.776050 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be5ea3171ad74d74bede5875414d92ac" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:02.777004 24167 raft_consensus.cc:3060] T be5ea3171ad74d74bede5875414d92ac P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.791945 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be5ea3171ad74d74bede5875414d92ac" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:16:02.793093 24017 raft_consensus.cc:3060] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.797267 24567 raft_consensus.cc:493] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.798231 24567 raft_consensus.cc:515] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:02.801636 24567 leader_election.cc:290] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:02.804630 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "131c7392ac614d87ab3d9b6713a46f7b" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:02.808528 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "131c7392ac614d87ab3d9b6713a46f7b" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:02.809552 24093 raft_consensus.cc:2468] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 0.
I20260709 12:16:02.811779 24204 leader_election.cc:304] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:02.823089 24567 raft_consensus.cc:2804] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:02.823594 24567 raft_consensus.cc:493] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:02.824044 24567 raft_consensus.cc:3060] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.814968 24017 raft_consensus.cc:2468] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 1.
I20260709 12:16:02.821347 24285 tablet_bootstrap.cc:492] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:02.831771 24285 ts_tablet_manager.cc:1403] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.097s	user 0.011s	sys 0.003s
W20260709 12:16:02.824395 24204 leader_election.cc:343] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.833549 24204 leader_election.cc:304] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:02.793076 24288 tablet_bootstrap.cc:654] T 2e4765d7c78c480fa4ba08d2812aeac3 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:02.840056 24167 raft_consensus.cc:2468] T be5ea3171ad74d74bede5875414d92ac P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 1.
I20260709 12:16:02.831192 24570 consensus_queue.cc:1048] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:02.843313 24296 raft_consensus.cc:2804] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:02.848203 24296 raft_consensus.cc:697] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Becoming Leader. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Running, Role: LEADER
I20260709 12:16:02.848917 24285 raft_consensus.cc:359] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:02.849103 24296 consensus_queue.cc:237] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:02.849781 24285 raft_consensus.cc:385] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.850116 24285 raft_consensus.cc:740] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.850865 24285 consensus_queue.cc:260] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:02.843891 24499 raft_consensus.cc:493] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.852645 24499 raft_consensus.cc:515] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:02.853639 24567 raft_consensus.cc:515] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:02.856158 24567 leader_election.cc:290] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:02.873224 24287 tablet_bootstrap.cc:492] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:02.873967 24287 ts_tablet_manager.cc:1403] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.155s	user 0.039s	sys 0.036s
I20260709 12:16:02.824021 24165 raft_consensus.cc:2468] T 95824f48a95746289333a9da8e265178 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:16:02.884770 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e3897eed824c5fa96e04ce0eca32a8" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
W20260709 12:16:02.886960 23981 leader_election.cc:343] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:02.884858 24287 raft_consensus.cc:359] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:02.888188 24287 raft_consensus.cc:385] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.888505 24287 raft_consensus.cc:740] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.889156 24499 leader_election.cc:290] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:02.889321 24287 consensus_queue.cc:260] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:02.822084 24015 raft_consensus.cc:2468] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:16:02.881134 24285 ts_tablet_manager.cc:1434] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.049s	user 0.010s	sys 0.000s
I20260709 12:16:02.922634 24287 ts_tablet_manager.cc:1434] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.048s	user 0.003s	sys 0.006s
I20260709 12:16:02.927186 24288 tablet_bootstrap.cc:492] T 2e4765d7c78c480fa4ba08d2812aeac3 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:02.927788 24288 ts_tablet_manager.cc:1403] T 2e4765d7c78c480fa4ba08d2812aeac3 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.160s	user 0.012s	sys 0.005s
I20260709 12:16:02.933595 24056 leader_election.cc:304] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: 
I20260709 12:16:02.934973 24569 raft_consensus.cc:2804] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:02.935516 24569 raft_consensus.cc:697] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Becoming Leader. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Running, Role: LEADER
I20260709 12:16:02.936450 24569 consensus_queue.cc:237] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:02.955614 24285 tablet_bootstrap.cc:492] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:02.956928 24286 tablet_bootstrap.cc:492] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:02.957653 24286 ts_tablet_manager.cc:1403] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.263s	user 0.051s	sys 0.059s
I20260709 12:16:02.951552 24570 consensus_queue.cc:1048] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260709 12:16:02.960650 24286 raft_consensus.cc:359] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:02.961745 24286 raft_consensus.cc:385] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.962044 24286 raft_consensus.cc:740] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.962972 24286 consensus_queue.cc:260] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:02.965852 24286 ts_tablet_manager.cc:1434] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:16:02.968616 24499 raft_consensus.cc:493] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:02.969365 24499 raft_consensus.cc:515] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:02.972783 24499 leader_election.cc:290] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:02.974576 24092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "131c7392ac614d87ab3d9b6713a46f7b" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:02.975375 24092 raft_consensus.cc:2468] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 0.
I20260709 12:16:02.975601 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "131c7392ac614d87ab3d9b6713a46f7b" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:02.976452 24238 raft_consensus.cc:2393] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c359614416104a7ba9db1ede97d13802 in current term 1: Already voted for candidate a0df0eff47b64e3eb45ab074139ed187 in this term.
I20260709 12:16:02.976816 23981 leader_election.cc:304] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:02.978266 24499 raft_consensus.cc:2804] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:02.979763 24499 raft_consensus.cc:493] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:02.980525 24499 raft_consensus.cc:3060] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:02.979301 24287 tablet_bootstrap.cc:492] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:02.964596 24288 raft_consensus.cc:359] T 2e4765d7c78c480fa4ba08d2812aeac3 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:02.983035 24288 raft_consensus.cc:385] T 2e4765d7c78c480fa4ba08d2812aeac3 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:02.994237 24288 raft_consensus.cc:740] T 2e4765d7c78c480fa4ba08d2812aeac3 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:02.962654 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e3897eed824c5fa96e04ce0eca32a8" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:03.004171 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "131c7392ac614d87ab3d9b6713a46f7b" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:16:03.006160 24092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "131c7392ac614d87ab3d9b6713a46f7b" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:03.002692 24286 tablet_bootstrap.cc:492] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:03.032671 24285 tablet_bootstrap.cc:654] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:03.047851 24288 consensus_queue.cc:260] T 2e4765d7c78c480fa4ba08d2812aeac3 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:03.046625 23911 catalog_manager.cc:5697] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132). New cstate: current_term: 1 leader_uuid: "a0df0eff47b64e3eb45ab074139ed187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:03.048379 24092 raft_consensus.cc:3060] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:03.049418 24167 raft_consensus.cc:2468] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 0.
I20260709 12:16:03.065754 24288 ts_tablet_manager.cc:1434] T 2e4765d7c78c480fa4ba08d2812aeac3 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.138s	user 0.008s	sys 0.000s
I20260709 12:16:03.067111 24288 tablet_bootstrap.cc:492] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:16:03.071431 24287 tablet_bootstrap.cc:654] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:03.053362 24056 consensus_peers.cc:597] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:03.091043 24130 consensus_peers.cc:597] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:03.114980 24567 raft_consensus.cc:493] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.115585 24567 raft_consensus.cc:515] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:03.115931 23915 catalog_manager.cc:5697] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c74a5203b594708921cd793aaca0b8b (127.11.13.130). New cstate: current_term: 1 leader_uuid: "6c74a5203b594708921cd793aaca0b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:03.119819 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4765d7c78c480fa4ba08d2812aeac3" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:03.120620 24093 raft_consensus.cc:2468] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 0.
I20260709 12:16:03.122161 24204 leader_election.cc:304] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:03.123039 24296 raft_consensus.cc:2804] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:03.123413 24296 raft_consensus.cc:493] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:03.123749 24296 raft_consensus.cc:3060] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:03.114185 24056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:42669, user_credentials={real_user=slave}} blocked reactor thread for 71895.4us
I20260709 12:16:03.117992 24567 leader_election.cc:290] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:03.136020 24499 raft_consensus.cc:515] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:03.137485 24015 raft_consensus.cc:2393] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a0df0eff47b64e3eb45ab074139ed187 in current term 1: Already voted for candidate c359614416104a7ba9db1ede97d13802 in this term.
I20260709 12:16:03.145218 23981 leader_election.cc:304] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 6c74a5203b594708921cd793aaca0b8b
I20260709 12:16:03.146569 24604 raft_consensus.cc:2804] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:03.146965 24604 raft_consensus.cc:493] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:03.147316 24604 raft_consensus.cc:3060] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:03.150036 24296 raft_consensus.cc:515] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:03.152230 24296 leader_election.cc:290] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:03.155841 24604 raft_consensus.cc:515] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:03.157029 24286 tablet_bootstrap.cc:654] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:03.159811 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e3897eed824c5fa96e04ce0eca32a8" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:03.084189 24092 raft_consensus.cc:2468] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 1.
W20260709 12:16:03.175282 24204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:36699, user_credentials={real_user=slave}} blocked reactor thread for 44623.8us
I20260709 12:16:03.175627 24288 tablet_bootstrap.cc:654] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:03.179390 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4765d7c78c480fa4ba08d2812aeac3" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:03.180439 24093 raft_consensus.cc:3060] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:03.175561 24604 leader_election.cc:290] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
W20260709 12:16:03.170763 23981 leader_election.cc:343] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:03.191736 24204 leader_election.cc:304] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:03.192813 24572 raft_consensus.cc:2804] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:03.193298 24572 raft_consensus.cc:697] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Becoming Leader. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Running, Role: LEADER
I20260709 12:16:03.194245 24572 consensus_queue.cc:237] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:03.197546 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4765d7c78c480fa4ba08d2812aeac3" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:03.198817 24163 raft_consensus.cc:2468] T 2e4765d7c78c480fa4ba08d2812aeac3 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 0.
I20260709 12:16:03.205456 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e3897eed824c5fa96e04ce0eca32a8" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:03.206149 24167 raft_consensus.cc:3060] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:03.211126 24287 tablet_bootstrap.cc:492] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:03.211664 24287 ts_tablet_manager.cc:1403] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.233s	user 0.038s	sys 0.008s
I20260709 12:16:03.215847 24285 tablet_bootstrap.cc:492] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:03.216142 24239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "131c7392ac614d87ab3d9b6713a46f7b" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:03.216430 24285 ts_tablet_manager.cc:1403] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.264s	user 0.035s	sys 0.029s
I20260709 12:16:03.218918 24499 leader_election.cc:290] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:03.220067 24285 raft_consensus.cc:359] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:03.220845 24285 raft_consensus.cc:385] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:03.220775 24167 raft_consensus.cc:2468] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 1.
I20260709 12:16:03.221517 24285 raft_consensus.cc:740] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:03.222026 24092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "131c7392ac614d87ab3d9b6713a46f7b" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:03.223029 24092 raft_consensus.cc:2393] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c359614416104a7ba9db1ede97d13802 in current term 1: Already voted for candidate a0df0eff47b64e3eb45ab074139ed187 in this term.
I20260709 12:16:03.222715 24285 consensus_queue.cc:260] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:03.225651 24285 ts_tablet_manager.cc:1434] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.009s	user 0.009s	sys 0.001s
I20260709 12:16:03.224067 24287 raft_consensus.cc:359] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:03.229898 24287 raft_consensus.cc:385] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:03.231165 24287 raft_consensus.cc:740] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:03.234618 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2e4765d7c78c480fa4ba08d2812aeac3" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:03.236193 23981 leader_election.cc:304] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:03.239792 24287 consensus_queue.cc:260] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:03.241343 24093 raft_consensus.cc:2468] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 1.
I20260709 12:16:03.242676 24204 leader_election.cc:304] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:03.243517 24296 raft_consensus.cc:2804] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:03.243988 24296 raft_consensus.cc:697] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Becoming Leader. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Running, Role: LEADER
I20260709 12:16:03.244722 24296 consensus_queue.cc:237] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:03.246944 24285 tablet_bootstrap.cc:492] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:03.247864 23911 catalog_manager.cc:5697] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132). New cstate: current_term: 1 leader_uuid: "a0df0eff47b64e3eb45ab074139ed187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:03.239524 24163 raft_consensus.cc:3060] T 2e4765d7c78c480fa4ba08d2812aeac3 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:03.251286 23981 leader_election.cc:304] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 6c74a5203b594708921cd793aaca0b8b
I20260709 12:16:03.252537 24604 raft_consensus.cc:2804] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:03.253127 24604 raft_consensus.cc:697] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:16:03.254060 24604 consensus_queue.cc:237] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:03.248683 24499 raft_consensus.cc:2749] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:03.263353 24287 ts_tablet_manager.cc:1434] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.051s	user 0.008s	sys 0.001s
I20260709 12:16:03.265029 24163 raft_consensus.cc:2468] T 2e4765d7c78c480fa4ba08d2812aeac3 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 1.
I20260709 12:16:03.265053 24287 tablet_bootstrap.cc:492] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:03.284046 24286 tablet_bootstrap.cc:492] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:03.284673 24286 ts_tablet_manager.cc:1403] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.283s	user 0.024s	sys 0.008s
I20260709 12:16:03.286953 24432 consensus_queue.cc:1048] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:03.291412 24287 tablet_bootstrap.cc:654] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:03.294770 24367 raft_consensus.cc:493] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.295401 24367 raft_consensus.cc:515] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:03.298094 24288 tablet_bootstrap.cc:492] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:03.298544 24286 raft_consensus.cc:359] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:03.298920 24288 ts_tablet_manager.cc:1403] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.232s	user 0.014s	sys 0.006s
I20260709 12:16:03.299396 24286 raft_consensus.cc:385] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:03.299794 24286 raft_consensus.cc:740] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:03.300179 24367 leader_election.cc:290] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:03.302659 24285 tablet_bootstrap.cc:654] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:03.302034 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56fb07e9eb574b1fbc5da95063610dd5" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:03.303231 23911 catalog_manager.cc:5697] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132). New cstate: current_term: 1 leader_uuid: "a0df0eff47b64e3eb45ab074139ed187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:03.304088 24241 raft_consensus.cc:2468] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:16:03.305336 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56fb07e9eb574b1fbc5da95063610dd5" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:03.305509 24058 leader_election.cc:304] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:03.304939 24286 consensus_queue.cc:260] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:03.306708 24367 raft_consensus.cc:2804] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:03.307088 24367 raft_consensus.cc:493] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:03.307536 24367 raft_consensus.cc:3060] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:03.313426 24286 ts_tablet_manager.cc:1434] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.028s	user 0.008s	sys 0.000s
I20260709 12:16:03.313531 24288 raft_consensus.cc:359] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:03.314462 24288 raft_consensus.cc:385] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:03.314749 24288 raft_consensus.cc:740] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:03.315546 24288 consensus_queue.cc:260] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:03.318162 24286 tablet_bootstrap.cc:492] T 1e98aa59558f4e58925a3c87026b6fa4 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
W20260709 12:16:03.307368 24056 leader_election.cc:343] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:03.328235 24288 ts_tablet_manager.cc:1434] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.029s	user 0.008s	sys 0.000s
I20260709 12:16:03.330264 24469 raft_consensus.cc:493] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.331014 24288 tablet_bootstrap.cc:492] T 576c4469eb9c49dbbba40a54832fa33f P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:16:03.330830 24469 raft_consensus.cc:515] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:03.333241 24469 leader_election.cc:290] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:03.339069 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2eabb81feec483ea9b303c5e516b7cf" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:03.345337 24367 raft_consensus.cc:515] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
W20260709 12:16:03.350369 24130 leader_election.cc:343] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:03.352216 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2eabb81feec483ea9b303c5e516b7cf" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:03.352423 24240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56fb07e9eb574b1fbc5da95063610dd5" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:03.353056 24241 raft_consensus.cc:2468] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:16:03.353200 24240 raft_consensus.cc:3060] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:03.354660 24132 leader_election.cc:304] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:03.355851 24288 tablet_bootstrap.cc:654] T 576c4469eb9c49dbbba40a54832fa33f P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:03.356002 24469 raft_consensus.cc:2804] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:03.356514 24469 raft_consensus.cc:493] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:03.357084 24469 raft_consensus.cc:3060] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:03.358788 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56fb07e9eb574b1fbc5da95063610dd5" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:16:03.360270 24286 tablet_bootstrap.cc:654] T 1e98aa59558f4e58925a3c87026b6fa4 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:03.360993 24056 leader_election.cc:343] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:03.371830 24367 leader_election.cc:290] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:03.370779 24517 consensus_queue.cc:1048] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:03.373200 24287 tablet_bootstrap.cc:492] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:03.373795 24287 ts_tablet_manager.cc:1403] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.109s	user 0.014s	sys 0.011s
I20260709 12:16:03.377013 24287 raft_consensus.cc:359] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:03.377782 24287 raft_consensus.cc:385] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:03.378212 24287 raft_consensus.cc:740] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:03.377748 24469 raft_consensus.cc:515] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:03.385452 23912 catalog_manager.cc:5697] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 0 to 1, leader changed from <none> to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:03.402719 24240 raft_consensus.cc:2468] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:16:03.382443 24287 consensus_queue.cc:260] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:03.405556 24570 consensus_queue.cc:1048] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:03.408504 24287 ts_tablet_manager.cc:1434] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.034s	user 0.008s	sys 0.000s
I20260709 12:16:03.409793 24287 tablet_bootstrap.cc:492] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:03.413252 24517 consensus_queue.cc:1048] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:03.415025 24469 leader_election.cc:290] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:03.416532 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2eabb81feec483ea9b303c5e516b7cf" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:03.417174 24241 raft_consensus.cc:3060] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:03.418507 24287 tablet_bootstrap.cc:654] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:03.422370 24058 leader_election.cc:304] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:03.424242 24552 raft_consensus.cc:2804] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:03.425165 24552 raft_consensus.cc:697] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Becoming Leader. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Running, Role: LEADER
I20260709 12:16:03.426407 24552 consensus_queue.cc:237] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:03.449496 24570 raft_consensus.cc:493] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.450182 24570 raft_consensus.cc:515] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:03.452195 24286 tablet_bootstrap.cc:492] T 1e98aa59558f4e58925a3c87026b6fa4 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:03.452780 24570 leader_election.cc:290] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:03.453012 24286 ts_tablet_manager.cc:1403] T 1e98aa59558f4e58925a3c87026b6fa4 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.135s	user 0.025s	sys 0.004s
I20260709 12:16:03.454919 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e3897eed824c5fa96e04ce0eca32a8" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:03.456921 24286 raft_consensus.cc:359] T 1e98aa59558f4e58925a3c87026b6fa4 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:03.457971 24286 raft_consensus.cc:385] T 1e98aa59558f4e58925a3c87026b6fa4 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:03.469262 24286 raft_consensus.cc:740] T 1e98aa59558f4e58925a3c87026b6fa4 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:03.470209 24286 consensus_queue.cc:260] T 1e98aa59558f4e58925a3c87026b6fa4 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
W20260709 12:16:03.475600 24130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:42669, user_credentials={real_user=slave}} blocked reactor thread for 75050.2us
I20260709 12:16:03.477722 24287 tablet_bootstrap.cc:492] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:03.478389 24287 ts_tablet_manager.cc:1403] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.069s	user 0.017s	sys 0.024s
I20260709 12:16:03.462838 24499 raft_consensus.cc:493] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.479297 24499 raft_consensus.cc:515] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:03.482160 24287 raft_consensus.cc:359] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:03.483121 24287 raft_consensus.cc:385] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:03.483475 24287 raft_consensus.cc:740] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:03.465975 24241 raft_consensus.cc:2468] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:16:03.485041 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2eabb81feec483ea9b303c5e516b7cf" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:16:03.483501 24499 leader_election.cc:290] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:03.484400 24287 consensus_queue.cc:260] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:03.486589 24092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ed2939d1454de181106eec39d0e4bc" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:03.487461 24132 leader_election.cc:304] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:03.488428 24432 raft_consensus.cc:2804] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:03.488709 24287 ts_tablet_manager.cc:1434] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 12:16:03.488919 24432 raft_consensus.cc:697] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:03.490345 24287 tablet_bootstrap.cc:492] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:03.490108 24432 consensus_queue.cc:237] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:03.491593 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ed2939d1454de181106eec39d0e4bc" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:03.494669 23983 leader_election.cc:304] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:03.496766 24499 raft_consensus.cc:2749] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:03.498780 24287 tablet_bootstrap.cc:654] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:03.464470 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e3897eed824c5fa96e04ce0eca32a8" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:03.520748 24167 raft_consensus.cc:2393] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c74a5203b594708921cd793aaca0b8b in current term 1: Already voted for candidate c359614416104a7ba9db1ede97d13802 in this term.
I20260709 12:16:03.511204 24287 tablet_bootstrap.cc:492] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:03.511869 24286 ts_tablet_manager.cc:1434] T 1e98aa59558f4e58925a3c87026b6fa4 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.058s	user 0.010s	sys 0.000s
I20260709 12:16:03.522182 24287 ts_tablet_manager.cc:1403] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.032s	user 0.010s	sys 0.011s
I20260709 12:16:03.522945 24286 tablet_bootstrap.cc:492] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:03.526903 24056 leader_election.cc:304] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:03.527738 24572 raft_consensus.cc:493] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.528360 24572 raft_consensus.cc:515] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:03.525846 24287 raft_consensus.cc:359] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:03.530280 24287 raft_consensus.cc:385] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:03.530683 24286 tablet_bootstrap.cc:654] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:03.538208 24287 raft_consensus.cc:740] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:03.539155 24287 consensus_queue.cc:260] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:03.540860 24092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b498d1ba220843a5939d152e0382bff7" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:03.531070 24572 leader_election.cc:290] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:03.533208 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b498d1ba220843a5939d152e0382bff7" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:03.541584 24569 raft_consensus.cc:493] T 1e98aa59558f4e58925a3c87026b6fa4 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.542143 24569 raft_consensus.cc:515] T 1e98aa59558f4e58925a3c87026b6fa4 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
W20260709 12:16:03.543103 24204 leader_election.cc:343] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:03.543929 24204 leader_election.cc:343] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:03.544544 24569 leader_election.cc:290] T 1e98aa59558f4e58925a3c87026b6fa4 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:03.545140 24569 raft_consensus.cc:3060] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:03.544338 24204 leader_election.cc:304] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:16:03.546317 24570 consensus_queue.cc:1048] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:03.547640 24287 ts_tablet_manager.cc:1434] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.000s
I20260709 12:16:03.547788 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e98aa59558f4e58925a3c87026b6fa4" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:03.548345 24572 raft_consensus.cc:2749] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:03.543108 23914 catalog_manager.cc:5697] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c74a5203b594708921cd793aaca0b8b (127.11.13.130). New cstate: current_term: 1 leader_uuid: "6c74a5203b594708921cd793aaca0b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:03.551199 24287 tablet_bootstrap.cc:492] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:03.563062 24517 raft_consensus.cc:493] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.563840 24517 raft_consensus.cc:515] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
W20260709 12:16:03.566103 24130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:42669, user_credentials={real_user=slave}} blocked reactor thread for 46386.1us
I20260709 12:16:03.561281 23915 catalog_manager.cc:5697] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:03.566944 24130 leader_election.cc:343] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:03.567842 24517 leader_election.cc:290] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:03.569265 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd0baecffd2454894539252387f4f2d" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
W20260709 12:16:03.572293 24204 leader_election.cc:343] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:03.573570 24287 tablet_bootstrap.cc:654] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:03.575834 24016 raft_consensus.cc:3060] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:03.577209 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1e98aa59558f4e58925a3c87026b6fa4" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:03.585863 24092 raft_consensus.cc:3060] T 1e98aa59558f4e58925a3c87026b6fa4 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:03.602092 24285 tablet_bootstrap.cc:492] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:03.586678 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd0baecffd2454894539252387f4f2d" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:03.604283 24285 ts_tablet_manager.cc:1403] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.366s	user 0.019s	sys 0.001s
W20260709 12:16:03.605981 24204 leader_election.cc:343] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:03.606699 24204 leader_election.cc:304] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:03.588891 24056 leader_election.cc:304] T 1e98aa59558f4e58925a3c87026b6fa4 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:03.617008 24286 tablet_bootstrap.cc:492] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:03.617590 24286 ts_tablet_manager.cc:1403] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.095s	user 0.011s	sys 0.001s
I20260709 12:16:03.631250 24572 raft_consensus.cc:2749] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:03.683314 24288 tablet_bootstrap.cc:492] T 576c4469eb9c49dbbba40a54832fa33f P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:03.683178 24286 raft_consensus.cc:359] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:03.684109 24288 ts_tablet_manager.cc:1403] T 576c4469eb9c49dbbba40a54832fa33f P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.353s	user 0.051s	sys 0.062s
I20260709 12:16:03.684800 24613 raft_consensus.cc:493] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.685420 24613 raft_consensus.cc:515] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:03.683221 24572 raft_consensus.cc:493] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.687904 24613 leader_election.cc:290] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:03.688032 24572 raft_consensus.cc:515] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:03.689319 24288 raft_consensus.cc:359] T 576c4469eb9c49dbbba40a54832fa33f P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:03.690088 24288 raft_consensus.cc:385] T 576c4469eb9c49dbbba40a54832fa33f P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:03.690691 24286 raft_consensus.cc:385] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:03.694519 24286 raft_consensus.cc:740] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:03.691789 24285 raft_consensus.cc:359] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:03.695679 24285 raft_consensus.cc:385] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:03.695900 24285 raft_consensus.cc:740] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:03.693573 24288 raft_consensus.cc:740] T 576c4469eb9c49dbbba40a54832fa33f P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:03.697260 24285 consensus_queue.cc:260] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:03.697922 24286 consensus_queue.cc:260] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:03.701905 24285 ts_tablet_manager.cc:1434] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.097s	user 0.008s	sys 0.000s
I20260709 12:16:03.703187 24285 tablet_bootstrap.cc:492] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:03.708705 24569 raft_consensus.cc:2749] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:03.709789 24285 tablet_bootstrap.cc:654] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:03.711374 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db45947fce14c98b26a8ed1ed9bbce4" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
W20260709 12:16:03.715024 23963 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:03.715775 24288 consensus_queue.cc:260] T 576c4469eb9c49dbbba40a54832fa33f P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:03.719266 24572 leader_election.cc:290] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:03.722306 24286 ts_tablet_manager.cc:1434] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.104s	user 0.011s	sys 0.000s
I20260709 12:16:03.731209 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db45947fce14c98b26a8ed1ed9bbce4" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:03.731720 24499 consensus_queue.cc:1048] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:03.737288 24286 tablet_bootstrap.cc:492] T b0805608ea264bef894c413af00a0687 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:03.746603 24604 consensus_queue.cc:1048] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:03.747963 24367 raft_consensus.cc:2749] T 1e98aa59558f4e58925a3c87026b6fa4 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:03.753243 24296 consensus_queue.cc:1048] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:03.755822 24204 leader_election.cc:343] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:03.759691 24204 leader_election.cc:343] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:03.760156 24204 leader_election.cc:304] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b
I20260709 12:16:03.764783 24567 raft_consensus.cc:2749] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:03.768851 24287 tablet_bootstrap.cc:492] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:03.769621 24287 ts_tablet_manager.cc:1403] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.219s	user 0.021s	sys 0.012s
I20260709 12:16:03.779027 24288 ts_tablet_manager.cc:1434] T 576c4469eb9c49dbbba40a54832fa33f P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.095s	user 0.003s	sys 0.007s
I20260709 12:16:03.799609 24288 tablet_bootstrap.cc:492] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:16:03.798070 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77079c31ca114c49a916e49cfd8faf2c" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:03.809839 24517 consensus_queue.cc:1048] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:03.810951 24089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77079c31ca114c49a916e49cfd8faf2c" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
W20260709 12:16:03.815243 24204 leader_election.cc:343] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:03.816831 24204 leader_election.cc:343] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:03.817473 24613 consensus_queue.cc:1048] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:03.818395 24569 raft_consensus.cc:493] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.819010 24569 raft_consensus.cc:515] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:03.832252 24604 raft_consensus.cc:493] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.833083 24604 raft_consensus.cc:515] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:03.837433 24604 leader_election.cc:290] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:03.839141 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2eabb81feec483ea9b303c5e516b7cf" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:03.817463 24204 leader_election.cc:304] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:16:03.840487 24286 tablet_bootstrap.cc:654] T b0805608ea264bef894c413af00a0687 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:03.842427 24572 raft_consensus.cc:2749] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:03.845580 24287 raft_consensus.cc:359] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:03.846506 24287 raft_consensus.cc:385] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:03.846774 24287 raft_consensus.cc:740] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:03.847530 24287 consensus_queue.cc:260] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:03.821939 24569 leader_election.cc:290] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:03.852936 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "576c4469eb9c49dbbba40a54832fa33f" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:03.853878 24165 raft_consensus.cc:2468] T 576c4469eb9c49dbbba40a54832fa33f P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:16:03.856949 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "576c4469eb9c49dbbba40a54832fa33f" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:03.863443 24287 ts_tablet_manager.cc:1434] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.093s	user 0.008s	sys 0.000s
I20260709 12:16:03.868299 24287 tablet_bootstrap.cc:492] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:03.874265 24240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2eabb81feec483ea9b303c5e516b7cf" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:03.875303 24240 raft_consensus.cc:2393] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c359614416104a7ba9db1ede97d13802 in current term 1: Already voted for candidate 008a79e516614c809c2d3afdb5cbc327 in this term.
I20260709 12:16:03.881567 23983 leader_election.cc:304] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:03.883111 24536 raft_consensus.cc:3060] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:03.889278 24056 leader_election.cc:304] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: 
I20260709 12:16:03.890296 24453 raft_consensus.cc:2804] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:03.891702 24453 raft_consensus.cc:493] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:03.889653 24517 consensus_queue.cc:1048] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:16:03.892082 24453 raft_consensus.cc:3060] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:03.891156 24056 leader_election.cc:343] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:03.895335 24285 tablet_bootstrap.cc:492] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:03.896219 24285 ts_tablet_manager.cc:1403] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.193s	user 0.025s	sys 0.051s
I20260709 12:16:03.912216 24517 raft_consensus.cc:493] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.912807 24517 raft_consensus.cc:515] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:03.913564 24288 tablet_bootstrap.cc:654] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:03.914762 24517 leader_election.cc:290] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:03.920137 24285 raft_consensus.cc:359] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:03.921195 24285 raft_consensus.cc:385] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:03.921528 24285 raft_consensus.cc:740] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:03.932103 24536 raft_consensus.cc:2749] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:03.933574 24287 tablet_bootstrap.cc:654] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:03.936563 24572 raft_consensus.cc:493] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:03.937181 24572 raft_consensus.cc:515] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:03.938819 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba257bd0f16a4a7da0cb4a7953ca1869" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:03.939847 24165 raft_consensus.cc:2468] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 1.
I20260709 12:16:03.941097 24572 leader_election.cc:290] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:03.934510 24285 consensus_queue.cc:260] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:03.944411 24204 leader_election.cc:304] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:03.947160 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0805608ea264bef894c413af00a0687" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
W20260709 12:16:03.949995 24204 leader_election.cc:343] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:03.958447 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0805608ea264bef894c413af00a0687" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:03.951709 24572 raft_consensus.cc:2804] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:16:03.973127 24204 leader_election.cc:343] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:03.975215 24204 leader_election.cc:304] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b
I20260709 12:16:03.981873 24285 ts_tablet_manager.cc:1434] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.085s	user 0.009s	sys 0.000s
I20260709 12:16:03.983462 24285 tablet_bootstrap.cc:492] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:03.951563 24626 consensus_queue.cc:1048] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.025s
I20260709 12:16:03.975127 24572 raft_consensus.cc:493] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:03.949437 24453 raft_consensus.cc:515] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:03.978789 24613 raft_consensus.cc:2749] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:03.979341 24056 consensus_peers.cc:597] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:04.011312 24286 tablet_bootstrap.cc:492] T b0805608ea264bef894c413af00a0687 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:04.011826 24288 tablet_bootstrap.cc:492] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:04.011968 24286 ts_tablet_manager.cc:1403] T b0805608ea264bef894c413af00a0687 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.275s	user 0.031s	sys 0.008s
I20260709 12:16:03.979269 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba257bd0f16a4a7da0cb4a7953ca1869" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:04.012347 24288 ts_tablet_manager.cc:1403] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.213s	user 0.022s	sys 0.009s
I20260709 12:16:03.995692 24572 raft_consensus.cc:3060] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:04.043586 24286 raft_consensus.cc:359] T b0805608ea264bef894c413af00a0687 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:04.044792 24286 raft_consensus.cc:385] T b0805608ea264bef894c413af00a0687 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:04.045224 24286 raft_consensus.cc:740] T b0805608ea264bef894c413af00a0687 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:04.046015 24288 raft_consensus.cc:359] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:04.046834 24288 raft_consensus.cc:385] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:04.047143 24288 raft_consensus.cc:740] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:04.059275 24286 consensus_queue.cc:260] T b0805608ea264bef894c413af00a0687 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:04.064172 24288 consensus_queue.cc:260] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:04.066826 24286 ts_tablet_manager.cc:1434] T b0805608ea264bef894c413af00a0687 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.053s	user 0.010s	sys 0.000s
I20260709 12:16:04.070808 24288 ts_tablet_manager.cc:1434] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.055s	user 0.006s	sys 0.001s
I20260709 12:16:04.071094 24287 tablet_bootstrap.cc:492] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:04.071707 24287 ts_tablet_manager.cc:1403] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.204s	user 0.021s	sys 0.003s
I20260709 12:16:04.072132 24288 tablet_bootstrap.cc:492] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:16:04.075973 24287 raft_consensus.cc:359] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.076891 24287 raft_consensus.cc:385] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:04.077193 24287 raft_consensus.cc:740] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:04.077433 24469 consensus_queue.cc:1048] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:04.077841 24287 consensus_queue.cc:260] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.081684 24287 ts_tablet_manager.cc:1434] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
W20260709 12:16:04.085023 24204 leader_election.cc:343] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:04.090065 24285 tablet_bootstrap.cc:654] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:04.099918 24287 tablet_bootstrap.cc:492] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:04.109607 24286 tablet_bootstrap.cc:492] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:04.138164 24453 leader_election.cc:290] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:04.137899 24287 tablet_bootstrap.cc:654] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:04.143776 24288 tablet_bootstrap.cc:654] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:04.145298 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "576c4469eb9c49dbbba40a54832fa33f" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:16:04.146816 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "576c4469eb9c49dbbba40a54832fa33f" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
W20260709 12:16:04.147380 24056 leader_election.cc:343] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:04.147776 24166 raft_consensus.cc:3060] T 576c4469eb9c49dbbba40a54832fa33f P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:04.158551 24569 consensus_queue.cc:1048] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:04.160648 24166 raft_consensus.cc:2468] T 576c4469eb9c49dbbba40a54832fa33f P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:16:04.162245 24056 leader_election.cc:304] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:04.164717 24569 raft_consensus.cc:2804] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:04.165915 24569 raft_consensus.cc:697] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Becoming Leader. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Running, Role: LEADER
I20260709 12:16:04.167174 24569 consensus_queue.cc:237] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:04.206290 24286 tablet_bootstrap.cc:654] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:04.207342 24572 raft_consensus.cc:515] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.240862 24285 tablet_bootstrap.cc:492] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:04.241820 24285 ts_tablet_manager.cc:1403] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.259s	user 0.026s	sys 0.032s
I20260709 12:16:04.243961 24572 leader_election.cc:290] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 election: Requested vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:04.245550 24285 raft_consensus.cc:359] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:04.246544 24285 raft_consensus.cc:385] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:04.246897 24285 raft_consensus.cc:740] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:04.251765 24285 consensus_queue.cc:260] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:04.254194 24285 ts_tablet_manager.cc:1434] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.000s
I20260709 12:16:04.257424 24285 tablet_bootstrap.cc:492] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:04.260350 24654 raft_consensus.cc:493] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.261046 24654 raft_consensus.cc:515] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:04.287199 24285 tablet_bootstrap.cc:654] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:04.306684 24572 raft_consensus.cc:493] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.310998 24572 raft_consensus.cc:515] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.320497 24286 tablet_bootstrap.cc:492] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:04.321156 24286 ts_tablet_manager.cc:1403] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.212s	user 0.040s	sys 0.027s
I20260709 12:16:04.322105 24572 leader_election.cc:290] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:04.324450 24654 leader_election.cc:290] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:04.326256 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0805608ea264bef894c413af00a0687" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:04.327330 24241 raft_consensus.cc:2468] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:16:04.329918 24288 tablet_bootstrap.cc:492] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:04.330551 24288 ts_tablet_manager.cc:1403] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.259s	user 0.015s	sys 0.011s
I20260709 12:16:04.332612 24132 leader_election.cc:304] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:04.332530 24286 raft_consensus.cc:359] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:04.333601 24286 raft_consensus.cc:385] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:04.334095 24286 raft_consensus.cc:740] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:04.334445 24633 raft_consensus.cc:2804] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:04.334934 24633 raft_consensus.cc:493] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:04.335395 24633 raft_consensus.cc:3060] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:04.335040 24286 consensus_queue.cc:260] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:04.337359 24286 ts_tablet_manager.cc:1434] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.016s	user 0.002s	sys 0.005s
I20260709 12:16:04.333534 24288 raft_consensus.cc:359] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.338505 24288 raft_consensus.cc:385] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:04.339598 24288 raft_consensus.cc:740] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:04.340441 24288 consensus_queue.cc:260] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.342592 24288 ts_tablet_manager.cc:1434] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 12:16:04.343812 24288 tablet_bootstrap.cc:492] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:16:04.347865 24633 raft_consensus.cc:515] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:04.350854 24286 tablet_bootstrap.cc:492] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:04.351619 24633 leader_election.cc:290] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:04.351820 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0805608ea264bef894c413af00a0687" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:04.352469 24241 raft_consensus.cc:3060] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:04.359822 24286 tablet_bootstrap.cc:654] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:04.359884 24288 tablet_bootstrap.cc:654] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:04.456072 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba257bd0f16a4a7da0cb4a7953ca1869" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:16:04.456403 24015 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbc768ef2c54c3391f0558cc562a01d" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:04.455322 24536 raft_consensus.cc:493] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.458112 24536 raft_consensus.cc:515] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.460156 24536 leader_election.cc:290] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:04.521884 24287 tablet_bootstrap.cc:492] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:04.523571 24287 ts_tablet_manager.cc:1403] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.424s	user 0.015s	sys 0.017s
I20260709 12:16:04.526759 24287 raft_consensus.cc:359] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:04.527554 24287 raft_consensus.cc:385] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:04.527845 24287 raft_consensus.cc:740] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:04.528596 24287 consensus_queue.cc:260] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:04.531945 24287 ts_tablet_manager.cc:1434] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:16:04.533258 24287 tablet_bootstrap.cc:492] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:04.541251 24287 tablet_bootstrap.cc:654] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:04.547201 24287 tablet_bootstrap.cc:492] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:04.547777 24287 ts_tablet_manager.cc:1403] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.015s	user 0.003s	sys 0.009s
I20260709 12:16:04.551039 24536 raft_consensus.cc:493] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.550978 24287 raft_consensus.cc:359] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.551800 24287 raft_consensus.cc:385] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:04.551651 24536 raft_consensus.cc:515] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:04.552114 24287 raft_consensus.cc:740] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:04.554083 24536 leader_election.cc:290] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:04.555343 24092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77079c31ca114c49a916e49cfd8faf2c" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:04.556151 24092 raft_consensus.cc:2468] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 0.
I20260709 12:16:04.553975 24287 consensus_queue.cc:260] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.558072 23981 leader_election.cc:304] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:04.559087 24536 raft_consensus.cc:2804] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:04.559264 24287 ts_tablet_manager.cc:1434] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:16:04.559559 24536 raft_consensus.cc:493] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:04.559958 24536 raft_consensus.cc:3060] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:04.560668 24287 tablet_bootstrap.cc:492] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:04.564911 24552 raft_consensus.cc:493] T b0805608ea264bef894c413af00a0687 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.565620 24552 raft_consensus.cc:515] T b0805608ea264bef894c413af00a0687 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:04.568011 24287 tablet_bootstrap.cc:654] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:04.568584 24536 raft_consensus.cc:515] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:04.571523 24552 leader_election.cc:290] T b0805608ea264bef894c413af00a0687 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:04.581199 24092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0805608ea264bef894c413af00a0687" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:04.581825 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77079c31ca114c49a916e49cfd8faf2c" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:04.582146 24092 raft_consensus.cc:2468] T b0805608ea264bef894c413af00a0687 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:16:04.582654 24091 raft_consensus.cc:3060] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:04.586874 24092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0805608ea264bef894c413af00a0687" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:04.587718 24092 raft_consensus.cc:3060] T b0805608ea264bef894c413af00a0687 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:04.593216 24241 raft_consensus.cc:2468] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:16:04.595499 24091 raft_consensus.cc:2468] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 1.
I20260709 12:16:04.597404 23981 leader_election.cc:304] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:04.598814 24604 raft_consensus.cc:2804] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:04.599397 24092 raft_consensus.cc:2468] T b0805608ea264bef894c413af00a0687 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:16:04.599558 24604 raft_consensus.cc:697] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:16:04.601012 24604 consensus_queue.cc:237] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:04.606828 24536 leader_election.cc:290] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:04.613915 24285 tablet_bootstrap.cc:492] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:04.616186 24285 ts_tablet_manager.cc:1403] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.359s	user 0.024s	sys 0.006s
I20260709 12:16:04.617441 24287 tablet_bootstrap.cc:492] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:04.618197 24287 ts_tablet_manager.cc:1403] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.058s	user 0.016s	sys 0.030s
I20260709 12:16:04.621672 24287 raft_consensus.cc:359] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.622820 24287 raft_consensus.cc:385] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:04.623111 24287 raft_consensus.cc:740] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:04.623263 24285 raft_consensus.cc:359] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.624032 24285 raft_consensus.cc:385] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:04.623910 24287 consensus_queue.cc:260] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.624436 24285 raft_consensus.cc:740] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:04.625415 24285 consensus_queue.cc:260] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.627028 24287 ts_tablet_manager.cc:1434] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.009s	user 0.004s	sys 0.003s
I20260709 12:16:04.627950 24285 ts_tablet_manager.cc:1434] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.001s
I20260709 12:16:04.628275 24287 tablet_bootstrap.cc:492] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:04.629192 24285 tablet_bootstrap.cc:492] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:04.636899 24287 tablet_bootstrap.cc:654] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:04.639343 24285 tablet_bootstrap.cc:654] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:04.651757 24240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0805608ea264bef894c413af00a0687" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:04.653213 24240 raft_consensus.cc:2393] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c74a5203b594708921cd793aaca0b8b in current term 1: Already voted for candidate 008a79e516614c809c2d3afdb5cbc327 in this term.
I20260709 12:16:04.658792 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba257bd0f16a4a7da0cb4a7953ca1869" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:04.659633 24166 raft_consensus.cc:3060] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:04.662489 24287 tablet_bootstrap.cc:492] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:04.663105 24287 ts_tablet_manager.cc:1403] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.035s	user 0.010s	sys 0.010s
I20260709 12:16:04.666302 24287 raft_consensus.cc:359] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:04.667057 24287 raft_consensus.cc:385] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:04.667089 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77079c31ca114c49a916e49cfd8faf2c" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:04.667296 24240 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77079c31ca114c49a916e49cfd8faf2c" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:04.667440 24287 raft_consensus.cc:740] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:04.668736 24287 consensus_queue.cc:260] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:04.669320 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd0baecffd2454894539252387f4f2d" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:04.670077 24238 raft_consensus.cc:2468] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 0.
I20260709 12:16:04.671044 24287 ts_tablet_manager.cc:1434] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:16:04.672228 24287 tablet_bootstrap.cc:492] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187: Bootstrap starting.
I20260709 12:16:04.672863 24132 leader_election.cc:304] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:04.677299 24633 raft_consensus.cc:2804] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:04.667825 24241 raft_consensus.cc:3060] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:04.678969 24287 tablet_bootstrap.cc:654] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:04.680680 23983 leader_election.cc:304] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:04.682291 24604 raft_consensus.cc:2804] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:04.682688 24604 raft_consensus.cc:493] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:04.683132 24604 raft_consensus.cc:3060] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:04.684532 24633 raft_consensus.cc:697] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:04.685515 24633 consensus_queue.cc:237] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:04.692241 24287 tablet_bootstrap.cc:492] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187: No bootstrap required, opened a new log
I20260709 12:16:04.692869 24287 ts_tablet_manager.cc:1403] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.000s
I20260709 12:16:04.703670 24166 raft_consensus.cc:2468] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 2.
I20260709 12:16:04.710726 24287 raft_consensus.cc:359] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:04.711652 24287 raft_consensus.cc:385] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:04.711941 24287 raft_consensus.cc:740] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Initialized, Role: FOLLOWER
I20260709 12:16:04.712832 24287 consensus_queue.cc:260] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:04.715740 24287 ts_tablet_manager.cc:1434] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187: Time spent starting tablet: real 0.023s	user 0.006s	sys 0.000s
I20260709 12:16:04.722818 24626 raft_consensus.cc:493] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.723569 24626 raft_consensus.cc:515] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.726230 24286 tablet_bootstrap.cc:492] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:04.726379 24626 leader_election.cc:290] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:04.726995 24286 ts_tablet_manager.cc:1403] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.376s	user 0.012s	sys 0.008s
I20260709 12:16:04.728906 24288 tablet_bootstrap.cc:492] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:04.729467 24288 ts_tablet_manager.cc:1403] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.386s	user 0.024s	sys 0.012s
I20260709 12:16:04.729908 24671 raft_consensus.cc:493] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.730707 24671 raft_consensus.cc:515] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.733349 24671 leader_election.cc:290] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:04.736683 24164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd0baecffd2454894539252387f4f2d" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
W20260709 12:16:04.741660 23981 leader_election.cc:343] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:04.736831 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbc768ef2c54c3391f0558cc562a01d" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:04.743670 24238 raft_consensus.cc:2468] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:16:04.736855 24286 raft_consensus.cc:359] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:04.744458 24286 raft_consensus.cc:385] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:04.744794 24286 raft_consensus.cc:740] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:04.734930 24604 raft_consensus.cc:515] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.750190 24241 raft_consensus.cc:2468] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 1.
I20260709 12:16:04.745735 24286 consensus_queue.cc:260] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:04.748225 24132 leader_election.cc:304] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:04.766057 24654 raft_consensus.cc:2804] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:04.753540 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbc768ef2c54c3391f0558cc562a01d" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:04.766506 24654 raft_consensus.cc:493] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:04.766940 24654 raft_consensus.cc:3060] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:04.756270 24164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd0baecffd2454894539252387f4f2d" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
W20260709 12:16:04.770452 23981 leader_election.cc:343] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:04.757584 24604 leader_election.cc:290] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Requested vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:04.765120 24671 raft_consensus.cc:493] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.751209 24288 raft_consensus.cc:359] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:04.780952 24288 raft_consensus.cc:385] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:04.782037 24288 raft_consensus.cc:740] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:04.784849 24288 consensus_queue.cc:260] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:04.788094 24671 raft_consensus.cc:515] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:04.790287 24671 leader_election.cc:290] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:04.791674 24288 ts_tablet_manager.cc:1434] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.057s	user 0.009s	sys 0.000s
I20260709 12:16:04.798632 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbc768ef2c54c3391f0558cc562a01d" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:04.811177 24286 ts_tablet_manager.cc:1434] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.084s	user 0.008s	sys 0.000s
I20260709 12:16:04.812713 24286 tablet_bootstrap.cc:492] T b498d1ba220843a5939d152e0382bff7 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:04.813676 24569 raft_consensus.cc:493] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.814217 24569 raft_consensus.cc:515] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:04.816030 24569 leader_election.cc:290] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:04.821821 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd0baecffd2454894539252387f4f2d" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:04.822562 24241 raft_consensus.cc:3060] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:04.823679 24130 leader_election.cc:343] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:04.827837 24285 tablet_bootstrap.cc:492] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:04.828361 24285 ts_tablet_manager.cc:1403] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.199s	user 0.021s	sys 0.000s
I20260709 12:16:04.831494 24285 raft_consensus.cc:359] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:04.832180 24285 raft_consensus.cc:385] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:04.832476 24285 raft_consensus.cc:740] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:04.833461 24285 consensus_queue.cc:260] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:04.836848 24241 raft_consensus.cc:2468] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 1.
I20260709 12:16:04.821877 24288 tablet_bootstrap.cc:492] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
W20260709 12:16:04.848333 23963 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:04.847373 24204 leader_election.cc:304] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:04.850894 24285 ts_tablet_manager.cc:1434] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.022s	user 0.008s	sys 0.000s
I20260709 12:16:04.852799 24286 tablet_bootstrap.cc:654] T b498d1ba220843a5939d152e0382bff7 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:04.852288 24204 leader_election.cc:343] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:04.855085 24204 leader_election.cc:343] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:04.857393 24657 raft_consensus.cc:2804] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:04.857946 24657 raft_consensus.cc:697] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 2 LEADER]: Becoming Leader. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Running, Role: LEADER
I20260709 12:16:04.858908 24657 consensus_queue.cc:237] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.870515 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f15aae25cad42abb701c295e56963e3" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:04.870900 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f15aae25cad42abb701c295e56963e3" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:04.889657 24285 tablet_bootstrap.cc:492] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:04.892879 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "809a23266b1646e0be710ddab5fff291" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:04.893323 24653 raft_consensus.cc:493] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.893868 24653 raft_consensus.cc:515] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.894407 24517 raft_consensus.cc:493] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.894955 24517 raft_consensus.cc:515] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.895345 24654 raft_consensus.cc:515] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.891765 23915 catalog_manager.cc:5697] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c74a5203b594708921cd793aaca0b8b (127.11.13.130). New cstate: current_term: 1 leader_uuid: "6c74a5203b594708921cd793aaca0b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:04.896224 24653 leader_election.cc:290] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:04.904695 24092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94568d4dbfd440478c83b50b50976be7" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:04.886691 24613 raft_consensus.cc:493] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.907486 24613 raft_consensus.cc:515] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:04.912966 24164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba257bd0f16a4a7da0cb4a7953ca1869" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
W20260709 12:16:04.901295 24204 leader_election.cc:343] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:04.902362 24165 raft_consensus.cc:2393] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a0df0eff47b64e3eb45ab074139ed187 in current term 1: Already voted for candidate 008a79e516614c809c2d3afdb5cbc327 in this term.
I20260709 12:16:04.902881 24517 leader_election.cc:290] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
W20260709 12:16:04.915167 24056 leader_election.cc:343] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:04.914069 23983 leader_election.cc:304] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327
I20260709 12:16:04.925912 24164 raft_consensus.cc:2368] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c359614416104a7ba9db1ede97d13802 for earlier term 1. Current term is 2.
I20260709 12:16:04.929284 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147e5dc12974e6c852f8961496066ae" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:04.909961 24613 leader_election.cc:290] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:04.930326 24653 raft_consensus.cc:2804] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:04.930908 24653 raft_consensus.cc:697] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
W20260709 12:16:04.936570 24204 leader_election.cc:343] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:04.937206 24204 leader_election.cc:304] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:04.938030 24572 raft_consensus.cc:2749] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:04.938905 24204 leader_election.cc:343] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:04.939155 24654 leader_election.cc:290] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
W20260709 12:16:04.940145 24204 leader_election.cc:343] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:04.931823 24653 consensus_queue.cc:237] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.944954 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbc768ef2c54c3391f0558cc562a01d" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:04.945755 24241 raft_consensus.cc:3060] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:04.951262 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94568d4dbfd440478c83b50b50976be7" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:04.952224 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c7696ee73146b59bd8b17640333077" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:04.959568 24241 raft_consensus.cc:2468] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:16:04.956063 24572 raft_consensus.cc:493] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.976245 23981 leader_election.cc:400] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Vote denied by peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699) with higher term. Message: Invalid argument: T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c359614416104a7ba9db1ede97d13802 for earlier term 1. Current term is 2.
I20260709 12:16:04.976652 23981 leader_election.cc:403] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Cancelling election due to peer responding with higher term
I20260709 12:16:04.978235 24204 leader_election.cc:304] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:04.978519 24649 raft_consensus.cc:3060] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 2
I20260709 12:16:04.979190 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147e5dc12974e6c852f8961496066ae" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:04.979699 24678 raft_consensus.cc:2749] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:04.974036 24132 leader_election.cc:304] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:04.983114 24626 raft_consensus.cc:2804] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:04.983363 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbc768ef2c54c3391f0558cc562a01d" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:16:04.983646 24626 raft_consensus.cc:697] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:04.984542 24626 consensus_queue.cc:237] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:04.988000 24651 raft_consensus.cc:493] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:04.988500 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c7696ee73146b59bd8b17640333077" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:04.988576 24651 raft_consensus.cc:515] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:04.990402 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ba257bd0f16a4a7da0cb4a7953ca1869" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:04.994313 24092 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "809a23266b1646e0be710ddab5fff291" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:04.995410 24092 raft_consensus.cc:2468] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:16:04.997225 24130 leader_election.cc:304] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: 
W20260709 12:16:05.014251 24204 leader_election.cc:343] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:05.017020 24204 leader_election.cc:304] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
W20260709 12:16:05.018705 24204 leader_election.cc:343] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:05.020922 24204 leader_election.cc:343] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:05.021683 24204 leader_election.cc:304] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:05.023365 24296 raft_consensus.cc:2749] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:05.025352 24130 leader_election.cc:343] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:04.997171 24651 leader_election.cc:290] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
W20260709 12:16:05.031167 24204 leader_election.cc:343] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:04.975674 24572 raft_consensus.cc:515] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:05.032925 24204 leader_election.cc:304] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:16:05.020763 24633 raft_consensus.cc:2804] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:05.039521 24296 raft_consensus.cc:2749] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:05.026676 24678 raft_consensus.cc:2749] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:05.030058 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b0805608ea264bef894c413af00a0687" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:05.054658 24164 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "809a23266b1646e0be710ddab5fff291" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:05.056130 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "809a23266b1646e0be710ddab5fff291" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:05.060236 24572 leader_election.cc:290] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:05.064182 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f3b3f298544987a7b9fc3dd0af496c" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:05.044816 24633 raft_consensus.cc:493] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:05.065316 24633 raft_consensus.cc:3060] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:05.065985 24204 leader_election.cc:343] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:05.079942 24056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:42669, user_credentials={real_user=slave}} blocked reactor thread for 46207.3us
I20260709 12:16:05.082929 24288 tablet_bootstrap.cc:654] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:05.138782 24130 leader_election.cc:343] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:05.084152 24016 raft_consensus.cc:3060] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:05.140916 24056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:36699, user_credentials={real_user=slave}} blocked reactor thread for 59087us
I20260709 12:16:05.142153 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f3b3f298544987a7b9fc3dd0af496c" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:05.145231 24286 tablet_bootstrap.cc:492] T b498d1ba220843a5939d152e0382bff7 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:05.151652 24286 ts_tablet_manager.cc:1403] T b498d1ba220843a5939d152e0382bff7 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.339s	user 0.017s	sys 0.010s
I20260709 12:16:05.167510 24056 leader_election.cc:304] T b0805608ea264bef894c413af00a0687 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187
W20260709 12:16:05.170467 24204 leader_election.cc:343] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:05.171211 24204 leader_election.cc:304] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b
I20260709 12:16:05.172585 24649 raft_consensus.cc:2749] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Leader pre-election lost for term 1. Reason: Vote denied by peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699) with higher term. Message: Invalid argument: T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c359614416104a7ba9db1ede97d13802 for earlier term 1. Current term is 2.
I20260709 12:16:05.174623 24572 raft_consensus.cc:2749] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:05.169590 24453 raft_consensus.cc:2749] T b0805608ea264bef894c413af00a0687 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:05.180783 24285 tablet_bootstrap.cc:654] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.183017 24286 raft_consensus.cc:359] T b498d1ba220843a5939d152e0382bff7 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:05.183888 24286 raft_consensus.cc:385] T b498d1ba220843a5939d152e0382bff7 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:05.184257 24286 raft_consensus.cc:740] T b498d1ba220843a5939d152e0382bff7 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:05.185536 24570 consensus_queue.cc:1048] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260709 12:16:05.195122 24604 raft_consensus.cc:493] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.195761 24604 raft_consensus.cc:515] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:05.200470 24604 leader_election.cc:290] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:05.219820 24286 consensus_queue.cc:260] T b498d1ba220843a5939d152e0382bff7 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:05.229979 24286 ts_tablet_manager.cc:1434] T b498d1ba220843a5939d152e0382bff7 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.078s	user 0.008s	sys 0.000s
I20260709 12:16:05.232441 24286 tablet_bootstrap.cc:492] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:05.278065 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "576c4469eb9c49dbbba40a54832fa33f" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:05.261765 24569 consensus_queue.cc:1048] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.021s
I20260709 12:16:05.315006 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "576c4469eb9c49dbbba40a54832fa33f" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:05.331545 24288 tablet_bootstrap.cc:492] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:05.332533 24288 ts_tablet_manager.cc:1403] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.533s	user 0.023s	sys 0.020s
I20260709 12:16:05.358309 24536 consensus_queue.cc:1048] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:05.374503 24286 tablet_bootstrap.cc:654] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:05.393548 24686 log.cc:927] Time spent T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802: Append to log took a long time: real 0.074s	user 0.001s	sys 0.000s
I20260709 12:16:05.391004 23981 leader_election.cc:304] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b
W20260709 12:16:05.412487 24204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:42669, user_credentials={real_user=slave}} blocked reactor thread for 48909.5us
I20260709 12:16:05.421422 24654 consensus_queue.cc:1048] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:05.422497 24285 tablet_bootstrap.cc:492] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:05.423062 24285 ts_tablet_manager.cc:1403] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.534s	user 0.031s	sys 0.002s
I20260709 12:16:05.414803 23912 catalog_manager.cc:5697] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 0 to 1, leader changed from <none> to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:05.397419 24649 consensus_queue.cc:1048] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.036s
W20260709 12:16:05.428469 24114 debug-util.cc:398] Leaking SignalData structure 0x7b0800284980 after lost signal to thread 11321
W20260709 12:16:05.445663 24114 debug-util.cc:398] Leaking SignalData structure 0x7b080003b460 after lost signal to thread 23950
W20260709 12:16:05.447264 24114 debug-util.cc:398] Leaking SignalData structure 0x7b080020fce0 after lost signal to thread 24042
I20260709 12:16:05.443400 24604 raft_consensus.cc:2749] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:05.452100 24016 raft_consensus.cc:1275] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Refusing update from remote peer a0df0eff47b64e3eb45ab074139ed187: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:16:05.444211 24285 raft_consensus.cc:359] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:05.464432 24167 raft_consensus.cc:1275] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Refusing update from remote peer a0df0eff47b64e3eb45ab074139ed187: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:16:05.466068 24517 consensus_queue.cc:1048] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:05.470486 24651 consensus_queue.cc:1048] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:05.458596 24285 raft_consensus.cc:385] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260709 12:16:05.480672 24114 debug-util.cc:398] Leaking SignalData structure 0x7b080014f1a0 after lost signal to thread 24117
I20260709 12:16:05.482002 24567 consensus_queue.cc:1048] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:05.484230 24114 debug-util.cc:398] Leaking SignalData structure 0x7b080039e7e0 after lost signal to thread 24191
I20260709 12:16:05.426316 24633 raft_consensus.cc:515] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:05.473977 24285 raft_consensus.cc:740] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:05.487179 24285 consensus_queue.cc:260] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:05.487675 24164 raft_consensus.cc:2393] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6c74a5203b594708921cd793aaca0b8b in current term 1: Already voted for candidate 008a79e516614c809c2d3afdb5cbc327 in this term.
I20260709 12:16:05.487901 24633 leader_election.cc:290] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:05.489423 24285 ts_tablet_manager.cc:1434] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.066s	user 0.005s	sys 0.000s
I20260709 12:16:05.480463 24288 raft_consensus.cc:359] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:05.490777 24288 raft_consensus.cc:385] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:05.491151 24288 raft_consensus.cc:740] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:05.497354 24288 consensus_queue.cc:260] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
W20260709 12:16:05.507978 24132 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:41011, user_credentials={real_user=slave}} blocked reactor thread for 87800.9us
I20260709 12:16:05.518651 24285 tablet_bootstrap.cc:492] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:05.508502 23911 catalog_manager.cc:5697] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:05.503990 24114 debug-util.cc:398] Leaking SignalData structure 0x7b08003a8b40 after lost signal to thread 24265
W20260709 12:16:05.522710 23981 consensus_peers.cc:597] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 -> Peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Couldn't send request to peer 008a79e516614c809c2d3afdb5cbc327. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:05.526183 24288 ts_tablet_manager.cc:1434] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.193s	user 0.007s	sys 0.004s
I20260709 12:16:05.528276 24288 tablet_bootstrap.cc:492] T 6db45947fce14c98b26a8ed1ed9bbce4 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
W20260709 12:16:05.554813 24114 debug-util.cc:398] Leaking SignalData structure 0x7b0800032f80 after lost signal to thread 24643
I20260709 12:16:05.561678 24286 tablet_bootstrap.cc:492] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:05.562451 24286 ts_tablet_manager.cc:1403] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.330s	user 0.026s	sys 0.018s
I20260709 12:16:05.585027 24288 tablet_bootstrap.cc:654] T 6db45947fce14c98b26a8ed1ed9bbce4 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.593253 24285 tablet_bootstrap.cc:654] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.605404 24286 raft_consensus.cc:359] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:05.613102 24286 raft_consensus.cc:385] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:05.613448 24286 raft_consensus.cc:740] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:05.618244 24286 consensus_queue.cc:260] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:05.624315 24286 ts_tablet_manager.cc:1434] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.062s	user 0.007s	sys 0.000s
I20260709 12:16:05.628603 24286 tablet_bootstrap.cc:492] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:05.589660 24056 leader_election.cc:304] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:05.637639 24570 raft_consensus.cc:3060] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:05.647152 24286 tablet_bootstrap.cc:654] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:05.661195 24671 raft_consensus.cc:493] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.661863 24671 raft_consensus.cc:515] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:05.664742 24671 leader_election.cc:290] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:05.669991 23914 catalog_manager.cc:5697] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 reported cstate change: term changed from 0 to 2, leader changed from <none> to a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132). New cstate: current_term: 2 leader_uuid: "a0df0eff47b64e3eb45ab074139ed187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:05.711450 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b498d1ba220843a5939d152e0382bff7" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:05.728179 24453 raft_consensus.cc:493] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:05.734042 24130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:42247, user_credentials={real_user=slave}} blocked reactor thread for 276923us
W20260709 12:16:05.719704 24058 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:41011, user_credentials={real_user=slave}} blocked reactor thread for 186866us
I20260709 12:16:05.729231 24697 raft_consensus.cc:493] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.731076 24090 raft_consensus.cc:2468] T b498d1ba220843a5939d152e0382bff7 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 0.
I20260709 12:16:05.749303 24204 leader_election.cc:304] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:05.754511 24697 raft_consensus.cc:515] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:05.737571 24453 raft_consensus.cc:515] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:05.787274 24453 leader_election.cc:290] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:05.797675 24570 raft_consensus.cc:2749] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:05.762809 24567 raft_consensus.cc:2804] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:05.806042 24657 raft_consensus.cc:493] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.812302 24567 raft_consensus.cc:493] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:05.838784 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "809a23266b1646e0be710ddab5fff291" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:16:05.848714 24567 raft_consensus.cc:3060] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:05.852483 24016 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b498d1ba220843a5939d152e0382bff7" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:05.859951 24604 raft_consensus.cc:493] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:05.889204 24285 tablet_bootstrap.cc:492] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
W20260709 12:16:06.047473 24696 log.cc:927] Time spent T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802: Append to log took a long time: real 0.141s	user 0.000s	sys 0.000s
I20260709 12:16:06.106319 24286 tablet_bootstrap.cc:492] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:05.849689 24657 raft_consensus.cc:515] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:05.867910 24697 leader_election.cc:290] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
W20260709 12:16:06.021404 24712 log.cc:927] Time spent T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b: Append to log took a long time: real 0.111s	user 0.000s	sys 0.009s
W20260709 12:16:05.875523 24706 log.cc:927] Time spent T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327: Append to log took a long time: real 0.120s	user 0.001s	sys 0.000s
I20260709 12:16:06.116277 24286 ts_tablet_manager.cc:1403] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.488s	user 0.031s	sys 0.012s
I20260709 12:16:06.117028 24285 ts_tablet_manager.cc:1403] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.609s	user 0.023s	sys 0.037s
I20260709 12:16:06.117648 24288 tablet_bootstrap.cc:492] T 6db45947fce14c98b26a8ed1ed9bbce4 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
W20260709 12:16:06.107952 24130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:42669, user_credentials={real_user=slave}} blocked reactor thread for 204341us
I20260709 12:16:06.125949 24288 ts_tablet_manager.cc:1403] T 6db45947fce14c98b26a8ed1ed9bbce4 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.598s	user 0.023s	sys 0.015s
I20260709 12:16:06.140003 24657 leader_election.cc:290] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
W20260709 12:16:06.144312 23963 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:06.155249 24235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db45947fce14c98b26a8ed1ed9bbce4" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:06.156148 24235 raft_consensus.cc:2468] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:16:06.160238 24166 raft_consensus.cc:3060] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:06.188387 24286 raft_consensus.cc:359] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:06.189252 24286 raft_consensus.cc:385] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:06.189540 24286 raft_consensus.cc:740] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:06.191665 24235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd0baecffd2454894539252387f4f2d" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:06.192757 24235 raft_consensus.cc:2393] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 008a79e516614c809c2d3afdb5cbc327 in current term 1: Already voted for candidate c359614416104a7ba9db1ede97d13802 in this term.
I20260709 12:16:06.206089 24160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db45947fce14c98b26a8ed1ed9bbce4" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:06.107662 24604 raft_consensus.cc:515] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:06.219501 24567 raft_consensus.cc:515] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:06.221043 24604 leader_election.cc:290] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:06.222349 24017 raft_consensus.cc:3060] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:06.231730 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "809a23266b1646e0be710ddab5fff291" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:06.214586 24286 consensus_queue.cc:260] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:06.216166 24090 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db45947fce14c98b26a8ed1ed9bbce4" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:06.217375 24167 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db45947fce14c98b26a8ed1ed9bbce4" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:06.261089 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b498d1ba220843a5939d152e0382bff7" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:06.262277 24084 raft_consensus.cc:3060] T b498d1ba220843a5939d152e0382bff7 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:06.222033 24604 consensus_queue.cc:1048] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.044s
I20260709 12:16:06.273885 24236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56fb07e9eb574b1fbc5da95063610dd5" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
W20260709 12:16:06.275034 24130 leader_election.cc:343] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:06.276517 24013 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd0baecffd2454894539252387f4f2d" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:06.264098 24285 raft_consensus.cc:359] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:06.287878 24285 raft_consensus.cc:385] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:06.258404 24058 leader_election.cc:304] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:06.288285 24285 raft_consensus.cc:740] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:06.221943 24567 leader_election.cc:290] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:06.293768 24285 consensus_queue.cc:260] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
W20260709 12:16:06.311345 24204 leader_election.cc:343] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:06.315029 24286 ts_tablet_manager.cc:1434] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.192s	user 0.005s	sys 0.003s
W20260709 12:16:06.304780 24058 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:41011, user_credentials={real_user=slave}} blocked reactor thread for 46756us
I20260709 12:16:06.259395 24090 raft_consensus.cc:2468] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 0.
I20260709 12:16:06.313032 24089 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56fb07e9eb574b1fbc5da95063610dd5" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:06.301090 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b498d1ba220843a5939d152e0382bff7" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:16:06.335350 24285 ts_tablet_manager.cc:1434] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.210s	user 0.005s	sys 0.004s
I20260709 12:16:06.373440 24285 tablet_bootstrap.cc:492] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:06.358891 24569 raft_consensus.cc:2804] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:06.377115 24569 raft_consensus.cc:493] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:06.377609 24569 raft_consensus.cc:3060] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:06.391048 24569 raft_consensus.cc:515] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:06.394941 24084 raft_consensus.cc:2468] T b498d1ba220843a5939d152e0382bff7 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 1.
I20260709 12:16:06.400624 23981 leader_election.cc:304] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:06.411682 24286 tablet_bootstrap.cc:492] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:06.411005 24651 consensus_queue.cc:1048] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:16:06.366410 24204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:42669, user_credentials={real_user=slave}} blocked reactor thread for 45208.3us
I20260709 12:16:06.434478 24130 leader_election.cc:304] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327; no voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802
W20260709 12:16:06.432452 24204 leader_election.cc:343] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:06.475203 24130 consensus_peers.cc:597] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:06.481876 24286 tablet_bootstrap.cc:654] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:06.488147 24288 raft_consensus.cc:359] T 6db45947fce14c98b26a8ed1ed9bbce4 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:06.488934 24569 leader_election.cc:290] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Requested vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:06.489200 24288 raft_consensus.cc:385] T 6db45947fce14c98b26a8ed1ed9bbce4 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:06.489655 24288 raft_consensus.cc:740] T 6db45947fce14c98b26a8ed1ed9bbce4 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:06.490489 24288 consensus_queue.cc:260] T 6db45947fce14c98b26a8ed1ed9bbce4 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:06.492151 24167 raft_consensus.cc:2468] T 6db45947fce14c98b26a8ed1ed9bbce4 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:16:06.492491 24288 ts_tablet_manager.cc:1434] T 6db45947fce14c98b26a8ed1ed9bbce4 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.366s	user 0.007s	sys 0.000s
I20260709 12:16:06.492905 24091 raft_consensus.cc:2468] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:16:06.493697 24288 tablet_bootstrap.cc:492] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:16:06.495077 24285 tablet_bootstrap.cc:654] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:06.499023 24130 leader_election.cc:304] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:06.499356 24204 leader_election.cc:304] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:06.503736 24204 leader_election.cc:304] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:06.506621 24715 raft_consensus.cc:2804] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:16:06.508644 24056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:36699, user_credentials={real_user=slave}} blocked reactor thread for 141288us
I20260709 12:16:06.507424 24613 raft_consensus.cc:2804] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:06.507294 24633 raft_consensus.cc:2749] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:06.509356 24678 raft_consensus.cc:2804] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:06.510762 24715 raft_consensus.cc:697] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:06.547729 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db45947fce14c98b26a8ed1ed9bbce4" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:06.524906 24613 raft_consensus.cc:493] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:06.555861 24613 raft_consensus.cc:3060] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:06.540812 24678 raft_consensus.cc:697] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Becoming Leader. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Running, Role: LEADER
I20260709 12:16:06.563104 24678 consensus_queue.cc:237] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:06.566186 24715 consensus_queue.cc:237] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:06.540310 24704 raft_consensus.cc:493] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.539969 24160 raft_consensus.cc:2468] T 6db45947fce14c98b26a8ed1ed9bbce4 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 0.
I20260709 12:16:06.569192 24704 raft_consensus.cc:515] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:06.572309 24704 leader_election.cc:290] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:06.582917 24725 raft_consensus.cc:2749] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:06.600278 24160 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db45947fce14c98b26a8ed1ed9bbce4" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:06.601039 24160 raft_consensus.cc:3060] T 6db45947fce14c98b26a8ed1ed9bbce4 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:06.609294 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96039384602f4057b875e1d873138d47" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:06.615914 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96039384602f4057b875e1d873138d47" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:06.619647 24160 raft_consensus.cc:2468] T 6db45947fce14c98b26a8ed1ed9bbce4 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:16:06.622413 24657 raft_consensus.cc:493] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.640280 24657 raft_consensus.cc:515] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:06.646384 24657 leader_election.cc:290] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:06.652769 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147e5dc12974e6c852f8961496066ae" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:06.657881 24286 tablet_bootstrap.cc:492] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:06.660719 24286 ts_tablet_manager.cc:1403] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.249s	user 0.029s	sys 0.005s
I20260709 12:16:06.662281 24088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147e5dc12974e6c852f8961496066ae" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
W20260709 12:16:06.661522 24204 leader_election.cc:343] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:06.665807 24204 leader_election.cc:343] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:06.666381 24204 leader_election.cc:304] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
W20260709 12:16:06.668821 24056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:42669, user_credentials={real_user=slave}} blocked reactor thread for 92785.5us
I20260709 12:16:06.688687 24288 tablet_bootstrap.cc:654] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:06.683889 24056 leader_election.cc:343] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:06.690805 24056 leader_election.cc:304] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: 
I20260709 12:16:06.685797 24728 raft_consensus.cc:493] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.695649 24613 raft_consensus.cc:515] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:06.695945 24728 raft_consensus.cc:515] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:06.697196 24241 raft_consensus.cc:2393] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c74a5203b594708921cd793aaca0b8b in current term 1: Already voted for candidate a0df0eff47b64e3eb45ab074139ed187 in this term.
I20260709 12:16:06.700608 24088 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "809a23266b1646e0be710ddab5fff291" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:06.701604 24088 raft_consensus.cc:2393] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c359614416104a7ba9db1ede97d13802 in current term 1: Already voted for candidate 008a79e516614c809c2d3afdb5cbc327 in this term.
I20260709 12:16:06.701786 24728 leader_election.cc:290] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:06.681713 24671 raft_consensus.cc:2749] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:06.712177 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "809a23266b1646e0be710ddab5fff291" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:06.717208 23914 catalog_manager.cc:5697] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:06.720770 23914 catalog_manager.cc:5697] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 0 to 1, leader changed from <none> to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } }
I20260709 12:16:06.731290 23981 leader_election.cc:304] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b
W20260709 12:16:06.750358 24056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:36699, user_credentials={real_user=slave}} blocked reactor thread for 59648.5us
W20260709 12:16:06.759428 24056 leader_election.cc:343] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:06.759949 24056 leader_election.cc:304] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:06.756588 24453 raft_consensus.cc:2804] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:06.764446 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db45947fce14c98b26a8ed1ed9bbce4" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:06.774391 24286 raft_consensus.cc:359] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:06.775269 24286 raft_consensus.cc:385] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:06.775671 24286 raft_consensus.cc:740] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:06.779109 24286 consensus_queue.cc:260] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:06.765978 24613 leader_election.cc:290] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:06.772234 24570 raft_consensus.cc:2749] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:06.771875 24453 raft_consensus.cc:697] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Becoming Leader. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Running, Role: LEADER
I20260709 12:16:06.804857 24286 ts_tablet_manager.cc:1434] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.143s	user 0.008s	sys 0.000s
I20260709 12:16:06.812842 24453 consensus_queue.cc:237] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:06.822608 24657 raft_consensus.cc:493] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.823283 24657 raft_consensus.cc:515] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
W20260709 12:16:06.823743 24740 log.cc:927] Time spent T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802: Append to log took a long time: real 0.077s	user 0.001s	sys 0.000s
I20260709 12:16:06.824769 24718 raft_consensus.cc:3060] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:06.825599 24657 leader_election.cc:290] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:06.831856 24286 tablet_bootstrap.cc:492] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:06.854902 24285 tablet_bootstrap.cc:492] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:06.855518 24285 ts_tablet_manager.cc:1403] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.482s	user 0.036s	sys 0.009s
W20260709 12:16:06.859016 24204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:36699, user_credentials={real_user=slave}} blocked reactor thread for 109404us
I20260709 12:16:06.867913 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6db45947fce14c98b26a8ed1ed9bbce4" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:06.868798 24166 raft_consensus.cc:2393] T 6db45947fce14c98b26a8ed1ed9bbce4 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a0df0eff47b64e3eb45ab074139ed187 in current term 1: Already voted for candidate 6c74a5203b594708921cd793aaca0b8b in this term.
I20260709 12:16:06.873138 24718 raft_consensus.cc:2749] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:06.884173 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c7696ee73146b59bd8b17640333077" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:06.886339 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c7696ee73146b59bd8b17640333077" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:06.875862 24691 raft_consensus.cc:493] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.888619 24691 raft_consensus.cc:515] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:06.890763 24691 leader_election.cc:290] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:06.901908 24716 raft_consensus.cc:493] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.902632 24716 raft_consensus.cc:515] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
W20260709 12:16:06.908632 24204 leader_election.cc:343] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:06.912774 24204 leader_election.cc:343] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:06.913883 24204 leader_election.cc:304] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:06.935453 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f15aae25cad42abb701c295e56963e3" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:06.938670 24703 raft_consensus.cc:493] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:06.939390 24703 raft_consensus.cc:515] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:06.941820 24703 leader_election.cc:290] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:06.931658 24162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f3b3f298544987a7b9fc3dd0af496c" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:06.944869 24716 leader_election.cc:290] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:06.930445 24678 raft_consensus.cc:2749] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:06.951691 24285 raft_consensus.cc:359] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:06.957002 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f15aae25cad42abb701c295e56963e3" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:06.957510 24285 raft_consensus.cc:385] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:06.960536 24285 raft_consensus.cc:740] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:06.948758 24237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94568d4dbfd440478c83b50b50976be7" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:06.962445 24237 raft_consensus.cc:2468] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:16:06.961725 24285 consensus_queue.cc:260] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:06.966593 24288 tablet_bootstrap.cc:492] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:06.978145 24288 ts_tablet_manager.cc:1403] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.485s	user 0.028s	sys 0.025s
I20260709 12:16:06.983875 24285 ts_tablet_manager.cc:1434] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.128s	user 0.009s	sys 0.001s
I20260709 12:16:06.984089 24286 tablet_bootstrap.cc:654] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:06.984531 24204 leader_election.cc:343] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:06.984840 24058 leader_election.cc:304] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:07.004142 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f3b3f298544987a7b9fc3dd0af496c" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
W20260709 12:16:07.016566 24204 leader_election.cc:343] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:07.018867 24204 leader_election.cc:343] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:07.019352 24204 leader_election.cc:304] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:07.019487 24285 tablet_bootstrap.cc:492] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:07.022683 24657 raft_consensus.cc:2749] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:07.023351 24727 raft_consensus.cc:2804] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:07.023808 24727 raft_consensus.cc:493] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:07.024130 24727 raft_consensus.cc:3060] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:07.027841 24288 raft_consensus.cc:359] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:07.028666 24288 raft_consensus.cc:385] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:07.028985 24288 raft_consensus.cc:740] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:07.030150 24288 consensus_queue.cc:260] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:07.042722 24204 leader_election.cc:304] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b
W20260709 12:16:07.042935 24132 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:41011, user_credentials={real_user=slave}} blocked reactor thread for 87711.9us
I20260709 12:16:07.044914 24751 raft_consensus.cc:2749] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
W20260709 12:16:07.046329 24204 leader_election.cc:343] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:07.044700 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94568d4dbfd440478c83b50b50976be7" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:07.046806 24204 leader_election.cc:304] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b
I20260709 12:16:07.061980 24286 tablet_bootstrap.cc:492] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:07.048215 24751 raft_consensus.cc:2749] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:07.065025 24056 leader_election.cc:343] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:07.066975 24286 ts_tablet_manager.cc:1403] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.235s	user 0.023s	sys 0.013s
W20260709 12:16:07.068389 24204 consensus_peers.cc:597] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:07.069913 24286 raft_consensus.cc:359] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:07.070868 24286 raft_consensus.cc:385] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:07.071194 24286 raft_consensus.cc:740] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:07.072022 24286 consensus_queue.cc:260] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:07.074446 24286 ts_tablet_manager.cc:1434] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:16:07.075500 24286 tablet_bootstrap.cc:492] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:07.084864 24288 ts_tablet_manager.cc:1434] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.106s	user 0.008s	sys 0.000s
I20260709 12:16:07.088886 24288 tablet_bootstrap.cc:492] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:16:07.095793 24288 tablet_bootstrap.cc:654] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:07.095176 23911 catalog_manager.cc:5697] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c74a5203b594708921cd793aaca0b8b (127.11.13.130). New cstate: current_term: 1 leader_uuid: "6c74a5203b594708921cd793aaca0b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:07.101639 24727 raft_consensus.cc:515] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:07.103446 24285 tablet_bootstrap.cc:654] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:07.111739 24747 consensus_queue.cc:1048] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:07.113902 24288 tablet_bootstrap.cc:492] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:07.123154 24288 ts_tablet_manager.cc:1403] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.034s	user 0.016s	sys 0.004s
I20260709 12:16:07.114378 24012 raft_consensus.cc:3060] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:07.126739 24288 raft_consensus.cc:359] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:07.127776 24288 raft_consensus.cc:385] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:07.128273 24288 raft_consensus.cc:740] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:07.129340 24288 consensus_queue.cc:260] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:07.116107 24751 consensus_queue.cc:1048] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:07.117933 24286 tablet_bootstrap.cc:654] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:07.136026 24018 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94568d4dbfd440478c83b50b50976be7" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:16:07.137392 24239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "94568d4dbfd440478c83b50b50976be7" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:07.138175 24239 raft_consensus.cc:3060] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:07.142705 24727 leader_election.cc:290] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
W20260709 12:16:07.148607 24056 leader_election.cc:343] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:07.162868 24286 tablet_bootstrap.cc:492] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:07.163806 24286 ts_tablet_manager.cc:1403] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.089s	user 0.017s	sys 0.007s
I20260709 12:16:07.167047 24285 tablet_bootstrap.cc:492] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:07.167704 24285 ts_tablet_manager.cc:1403] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.149s	user 0.014s	sys 0.007s
I20260709 12:16:07.168414 24286 raft_consensus.cc:359] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:07.170646 24286 raft_consensus.cc:385] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:07.171481 24286 raft_consensus.cc:740] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:07.175740 24722 consensus_queue.cc:1048] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:07.178637 24288 ts_tablet_manager.cc:1434] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.055s	user 0.009s	sys 0.000s
I20260709 12:16:07.173069 24286 consensus_queue.cc:260] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:07.186733 24285 raft_consensus.cc:359] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:07.187575 24285 raft_consensus.cc:385] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:07.187872 24285 raft_consensus.cc:740] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:07.191401 24285 consensus_queue.cc:260] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:07.195899 24286 ts_tablet_manager.cc:1434] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.032s	user 0.007s	sys 0.002s
I20260709 12:16:07.200062 24286 tablet_bootstrap.cc:492] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b: Bootstrap starting.
I20260709 12:16:07.204494 24285 ts_tablet_manager.cc:1434] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.036s	user 0.007s	sys 0.002s
I20260709 12:16:07.212276 24288 tablet_bootstrap.cc:492] T 93f3b3f298544987a7b9fc3dd0af496c P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:16:07.226668 24286 tablet_bootstrap.cc:654] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:07.227269 24285 tablet_bootstrap.cc:492] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:07.250453 24288 tablet_bootstrap.cc:654] T 93f3b3f298544987a7b9fc3dd0af496c P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:07.261024 24704 consensus_queue.cc:1048] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:07.214869 24239 raft_consensus.cc:2468] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:16:07.271998 24718 raft_consensus.cc:493] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 008a79e516614c809c2d3afdb5cbc327)
I20260709 12:16:07.274765 24718 raft_consensus.cc:515] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:07.279608 24058 leader_election.cc:304] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:07.283726 24703 raft_consensus.cc:2804] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:07.287104 24703 raft_consensus.cc:697] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Becoming Leader. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Running, Role: LEADER
I20260709 12:16:07.287801 24161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbc768ef2c54c3391f0558cc562a01d" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:07.288009 24703 consensus_queue.cc:237] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:07.292222 24732 raft_consensus.cc:493] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.292812 24732 raft_consensus.cc:515] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:07.295459 24732 leader_election.cc:290] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:07.298055 24718 leader_election.cc:290] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:07.308388 24237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbc768ef2c54c3391f0558cc562a01d" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:07.311636 24285 tablet_bootstrap.cc:654] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:07.313617 23983 leader_election.cc:304] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:07.329960 24728 raft_consensus.cc:2749] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:07.349231 24237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f3b3f298544987a7b9fc3dd0af496c" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:07.351891 24161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f3b3f298544987a7b9fc3dd0af496c" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:07.384346 24237 raft_consensus.cc:2468] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:16:07.388584 23914 catalog_manager.cc:5697] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:07.393201 24750 raft_consensus.cc:493] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260709 12:16:07.402405 24056 leader_election.cc:343] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:07.415599 24750 raft_consensus.cc:515] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:07.466194 24058 leader_election.cc:304] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327
I20260709 12:16:07.450541 24569 consensus_queue.cc:1048] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
I20260709 12:16:07.472543 24453 raft_consensus.cc:2804] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:07.487460 24286 tablet_bootstrap.cc:492] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b: No bootstrap required, opened a new log
I20260709 12:16:07.488013 24286 ts_tablet_manager.cc:1403] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b: Time spent bootstrapping tablet: real 0.288s	user 0.031s	sys 0.003s
I20260709 12:16:07.490199 24288 tablet_bootstrap.cc:492] T 93f3b3f298544987a7b9fc3dd0af496c P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:07.488520 24453 raft_consensus.cc:493] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:07.490793 24288 ts_tablet_manager.cc:1403] T 93f3b3f298544987a7b9fc3dd0af496c P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.279s	user 0.038s	sys 0.058s
I20260709 12:16:07.476630 24705 raft_consensus.cc:493] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.507473 24718 raft_consensus.cc:493] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.502058 24705 raft_consensus.cc:515] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:07.508249 24718 raft_consensus.cc:515] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:07.510059 24705 leader_election.cc:290] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:07.510540 24718 leader_election.cc:290] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:07.495571 24453 raft_consensus.cc:3060] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:07.514866 24286 raft_consensus.cc:359] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:07.522205 24286 raft_consensus.cc:385] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:07.522575 24286 raft_consensus.cc:740] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Initialized, Role: FOLLOWER
I20260709 12:16:07.523046 24161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96039384602f4057b875e1d873138d47" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:07.523540 24286 consensus_queue.cc:260] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:07.526489 24286 ts_tablet_manager.cc:1434] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b: Time spent starting tablet: real 0.038s	user 0.010s	sys 0.000s
I20260709 12:16:07.519232 24288 raft_consensus.cc:359] T 93f3b3f298544987a7b9fc3dd0af496c P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:07.526974 24288 raft_consensus.cc:385] T 93f3b3f298544987a7b9fc3dd0af496c P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:07.527309 24288 raft_consensus.cc:740] T 93f3b3f298544987a7b9fc3dd0af496c P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:07.527982 24288 consensus_queue.cc:260] T 93f3b3f298544987a7b9fc3dd0af496c P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:07.523949 24161 raft_consensus.cc:2468] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 0.
I20260709 12:16:07.536881 24285 tablet_bootstrap.cc:492] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:07.541216 24285 ts_tablet_manager.cc:1403] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.314s	user 0.019s	sys 0.010s
I20260709 12:16:07.545015 24285 raft_consensus.cc:359] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:07.518570 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f15aae25cad42abb701c295e56963e3" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:07.547473 24285 raft_consensus.cc:385] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:07.547786 24241 raft_consensus.cc:2468] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:16:07.534866 24288 ts_tablet_manager.cc:1434] T 93f3b3f298544987a7b9fc3dd0af496c P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.044s	user 0.007s	sys 0.000s
I20260709 12:16:07.549194 24132 leader_election.cc:304] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:07.549422 24288 tablet_bootstrap.cc:492] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:16:07.547914 24285 raft_consensus.cc:740] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:07.551236 24285 consensus_queue.cc:260] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:07.557687 24017 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96039384602f4057b875e1d873138d47" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:07.558506 24017 raft_consensus.cc:2468] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:16:07.559376 23911 catalog_manager.cc:5697] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132). New cstate: current_term: 1 leader_uuid: "a0df0eff47b64e3eb45ab074139ed187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:07.562727 24285 ts_tablet_manager.cc:1434] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.021s	user 0.004s	sys 0.005s
I20260709 12:16:07.564105 24285 tablet_bootstrap.cc:492] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:07.564742 23981 leader_election.cc:304] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:07.565685 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96039384602f4057b875e1d873138d47" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:07.566219 24723 raft_consensus.cc:2804] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:07.566428 24091 raft_consensus.cc:2468] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 0.
I20260709 12:16:07.566749 24723 raft_consensus.cc:493] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:07.567267 24723 raft_consensus.cc:3060] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:07.572019 24776 raft_consensus.cc:493] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.573355 24776 raft_consensus.cc:515] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:07.577373 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ff9d150a47feb68f490efce1e3f2" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:07.567412 24690 raft_consensus.cc:2804] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:07.579185 24750 leader_election.cc:290] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
W20260709 12:16:07.579602 24056 leader_election.cc:343] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:07.580494 24690 raft_consensus.cc:493] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:07.582161 24690 raft_consensus.cc:3060] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:07.583961 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ff9d150a47feb68f490efce1e3f2" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
W20260709 12:16:07.586036 24056 leader_election.cc:343] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:07.586565 24056 leader_election.cc:304] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:07.587392 24775 raft_consensus.cc:2749] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:07.588225 24776 leader_election.cc:290] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:07.590260 24453 raft_consensus.cc:515] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:07.593633 24130 leader_election.cc:304] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:07.594027 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f3b3f298544987a7b9fc3dd0af496c" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:07.594775 24166 raft_consensus.cc:3060] T 93f3b3f298544987a7b9fc3dd0af496c P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:07.595130 24722 raft_consensus.cc:2804] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:07.595661 24722 raft_consensus.cc:493] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:07.596124 24722 raft_consensus.cc:3060] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:07.596364 24690 raft_consensus.cc:515] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:07.596139 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f15aae25cad42abb701c295e56963e3" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
W20260709 12:16:07.599390 24130 leader_election.cc:343] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:07.600087 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f15aae25cad42abb701c295e56963e3" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:07.600744 24241 raft_consensus.cc:3060] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:07.605099 23963 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:07.610659 24166 raft_consensus.cc:2468] T 93f3b3f298544987a7b9fc3dd0af496c P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:16:07.639995 24722 raft_consensus.cc:515] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:07.641139 24241 raft_consensus.cc:2468] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:16:07.642436 24132 leader_election.cc:304] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:07.643328 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96039384602f4057b875e1d873138d47" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:16:07.624256 24453 leader_election.cc:290] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Requested vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:07.645313 24722 leader_election.cc:290] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:07.646147 24722 raft_consensus.cc:2804] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:07.647009 24722 raft_consensus.cc:697] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:07.648269 24722 consensus_queue.cc:237] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:07.625408 24690 leader_election.cc:290] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:07.627017 24288 tablet_bootstrap.cc:654] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:07.627704 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96039384602f4057b875e1d873138d47" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:07.656018 24091 raft_consensus.cc:2468] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:16:07.627799 24723 raft_consensus.cc:515] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:07.628149 24014 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f15aae25cad42abb701c295e56963e3" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:16:07.658392 24012 raft_consensus.cc:2393] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 008a79e516614c809c2d3afdb5cbc327 in current term 1: Already voted for candidate c359614416104a7ba9db1ede97d13802 in this term.
I20260709 12:16:07.659888 24723 leader_election.cc:290] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:07.659860 24056 leader_election.cc:304] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: 
I20260709 12:16:07.661433 24453 raft_consensus.cc:2804] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election won for term 1
W20260709 12:16:07.661985 24130 leader_election.cc:343] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:07.662055 24453 raft_consensus.cc:697] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Becoming Leader. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Running, Role: LEADER
I20260709 12:16:07.630384 24285 tablet_bootstrap.cc:654] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:07.637411 24776 raft_consensus.cc:493] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.663586 24453 consensus_queue.cc:237] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:07.663949 24776 raft_consensus.cc:515] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:07.666761 24776 leader_election.cc:290] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:07.669610 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96039384602f4057b875e1d873138d47" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:07.669945 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96039384602f4057b875e1d873138d47" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:07.670396 24091 raft_consensus.cc:3060] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:07.670917 24166 raft_consensus.cc:2393] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate c359614416104a7ba9db1ede97d13802 in current term 1: Already voted for candidate 008a79e516614c809c2d3afdb5cbc327 in this term.
I20260709 12:16:07.687311 24012 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147e5dc12974e6c852f8961496066ae" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:07.692113 24285 tablet_bootstrap.cc:492] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:07.693102 24285 ts_tablet_manager.cc:1403] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.129s	user 0.006s	sys 0.011s
I20260709 12:16:07.694726 24091 raft_consensus.cc:2468] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 1.
I20260709 12:16:07.696146 23981 leader_election.cc:304] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327
I20260709 12:16:07.697564 24723 raft_consensus.cc:2804] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:07.698280 24723 raft_consensus.cc:697] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:16:07.700052 24723 consensus_queue.cc:237] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:07.702543 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96039384602f4057b875e1d873138d47" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:07.703459 24093 raft_consensus.cc:2393] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 008a79e516614c809c2d3afdb5cbc327 in current term 1: Already voted for candidate c359614416104a7ba9db1ede97d13802 in this term.
I20260709 12:16:07.706566 24285 raft_consensus.cc:359] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:07.719491 24285 raft_consensus.cc:385] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:07.720392 24285 raft_consensus.cc:740] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:07.722235 24285 consensus_queue.cc:260] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:07.732208 24285 ts_tablet_manager.cc:1434] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.039s	user 0.007s	sys 0.000s
I20260709 12:16:07.736104 24285 tablet_bootstrap.cc:492] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:07.738307 23915 catalog_manager.cc:5697] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c74a5203b594708921cd793aaca0b8b (127.11.13.130). New cstate: current_term: 1 leader_uuid: "6c74a5203b594708921cd793aaca0b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:07.754901 24285 tablet_bootstrap.cc:654] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:07.736913 24056 leader_election.cc:343] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:07.793644 24288 tablet_bootstrap.cc:492] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:07.794303 24288 ts_tablet_manager.cc:1403] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.245s	user 0.017s	sys 0.016s
I20260709 12:16:07.799669 24288 raft_consensus.cc:359] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:07.800441 24288 raft_consensus.cc:385] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:07.800690 24288 raft_consensus.cc:740] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:07.801326 24288 consensus_queue.cc:260] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:07.804133 24288 ts_tablet_manager.cc:1434] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.000s
I20260709 12:16:07.805171 24288 tablet_bootstrap.cc:492] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327: Bootstrap starting.
I20260709 12:16:07.816113 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "93f3b3f298544987a7b9fc3dd0af496c" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:07.818344 24241 raft_consensus.cc:3060] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:07.819591 24236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147e5dc12974e6c852f8961496066ae" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:07.820392 24236 raft_consensus.cc:2468] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 0.
I20260709 12:16:07.822075 24014 raft_consensus.cc:3060] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:07.813585 24285 tablet_bootstrap.cc:492] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:07.826839 24285 ts_tablet_manager.cc:1403] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.091s	user 0.018s	sys 0.020s
I20260709 12:16:07.827812 24241 raft_consensus.cc:2468] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:16:07.830914 24058 leader_election.cc:304] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:07.832000 24453 raft_consensus.cc:2804] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:07.830324 23915 catalog_manager.cc:5697] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 0 to 1, leader changed from <none> to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:07.832532 24453 raft_consensus.cc:493] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:07.834210 24288 tablet_bootstrap.cc:654] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:07.834218 24453 raft_consensus.cc:3060] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:07.829054 23913 catalog_manager.cc:5697] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c74a5203b594708921cd793aaca0b8b (127.11.13.130). New cstate: current_term: 1 leader_uuid: "6c74a5203b594708921cd793aaca0b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:07.833534 24130 leader_election.cc:304] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:16:07.844411 24747 raft_consensus.cc:2749] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:07.847131 24285 raft_consensus.cc:359] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:07.858108 24285 raft_consensus.cc:385] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:07.858491 24285 raft_consensus.cc:740] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:07.859207 24285 consensus_queue.cc:260] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:07.861860 24285 ts_tablet_manager.cc:1434] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.034s	user 0.007s	sys 0.004s
I20260709 12:16:07.863538 24285 tablet_bootstrap.cc:492] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:07.870424 24285 tablet_bootstrap.cc:654] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:07.876540 24569 consensus_queue.cc:1048] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260709 12:16:07.931850 24723 raft_consensus.cc:493] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:07.932477 24723 raft_consensus.cc:515] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:07.907631 23914 catalog_manager.cc:5697] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:07.932680 24453 raft_consensus.cc:515] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:07.943238 24727 consensus_queue.cc:1048] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:07.953012 24723 leader_election.cc:290] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:07.955070 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b498d1ba220843a5939d152e0382bff7" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:07.958237 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147e5dc12974e6c852f8961496066ae" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:07.958931 24238 raft_consensus.cc:3060] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:07.966998 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b498d1ba220843a5939d152e0382bff7" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:07.968070 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147e5dc12974e6c852f8961496066ae" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
I20260709 12:16:07.968573 24453 leader_election.cc:290] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
W20260709 12:16:07.982371 24056 leader_election.cc:343] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:07.984365 23981 leader_election.cc:304] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:08.013679 24288 tablet_bootstrap.cc:492] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327: No bootstrap required, opened a new log
I20260709 12:16:08.015123 24288 ts_tablet_manager.cc:1403] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327: Time spent bootstrapping tablet: real 0.210s	user 0.021s	sys 0.020s
I20260709 12:16:07.993294 24728 raft_consensus.cc:2749] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:08.064105 24288 raft_consensus.cc:359] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:08.064889 24288 raft_consensus.cc:385] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:08.065201 24288 raft_consensus.cc:740] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Initialized, Role: FOLLOWER
I20260709 12:16:08.067548 24288 consensus_queue.cc:260] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:08.085440 24288 ts_tablet_manager.cc:1434] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327: Time spent starting tablet: real 0.070s	user 0.007s	sys 0.001s
I20260709 12:16:08.092777 24238 raft_consensus.cc:2468] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:16:08.108515 24010 raft_consensus.cc:3060] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:08.111215 24058 leader_election.cc:304] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:08.115370 24704 raft_consensus.cc:2804] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:08.115949 24704 raft_consensus.cc:697] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Becoming Leader. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Running, Role: LEADER
I20260709 12:16:08.116799 24704 consensus_queue.cc:237] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:08.138911 24469 consensus_queue.cc:1048] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:08.151643 24285 tablet_bootstrap.cc:492] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:08.155644 24285 ts_tablet_manager.cc:1403] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.293s	user 0.026s	sys 0.019s
I20260709 12:16:08.171988 24285 raft_consensus.cc:359] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:08.174386 24285 raft_consensus.cc:385] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:08.175264 24285 raft_consensus.cc:740] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
W20260709 12:16:08.182271 24130 consensus_peers.cc:597] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:08.182329 24718 consensus_queue.cc:1048] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260709 12:16:08.184453 24285 consensus_queue.cc:260] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:08.200805 24285 ts_tablet_manager.cc:1434] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.045s	user 0.008s	sys 0.001s
I20260709 12:16:08.206326 23911 catalog_manager.cc:5697] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b reported cstate change: term changed from 0 to 1, leader changed from <none> to 6c74a5203b594708921cd793aaca0b8b (127.11.13.130). New cstate: current_term: 1 leader_uuid: "6c74a5203b594708921cd793aaca0b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:08.222024 24285 tablet_bootstrap.cc:492] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:08.227821 24718 consensus_queue.cc:1048] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:08.233568 24285 tablet_bootstrap.cc:654] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:08.241849 24747 raft_consensus.cc:493] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:08.242502 24747 raft_consensus.cc:515] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:08.245041 24747 leader_election.cc:290] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:08.247998 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c7696ee73146b59bd8b17640333077" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802" is_pre_election: true
I20260709 12:16:08.248343 24453 consensus_queue.cc:1048] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:08.250561 24234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c7696ee73146b59bd8b17640333077" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:08.251255 24234 raft_consensus.cc:2468] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 0.
I20260709 12:16:08.252821 24132 leader_election.cc:304] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:08.253626 24747 raft_consensus.cc:2804] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:08.255551 24747 raft_consensus.cc:493] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:08.255935 24747 raft_consensus.cc:3060] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:08.262156 24130 leader_election.cc:343] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:08.267694 24704 consensus_queue.cc:1048] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:08.278491 24285 tablet_bootstrap.cc:492] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:08.279320 24285 ts_tablet_manager.cc:1403] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.058s	user 0.018s	sys 0.001s
I20260709 12:16:08.303735 24285 raft_consensus.cc:359] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:08.315564 24285 raft_consensus.cc:385] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:08.315970 24285 raft_consensus.cc:740] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:08.324030 24728 raft_consensus.cc:493] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:08.324753 24728 raft_consensus.cc:515] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:08.327829 24728 leader_election.cc:290] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:08.317674 24285 consensus_queue.cc:260] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:08.330722 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ff9d150a47feb68f490efce1e3f2" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:08.333245 24091 raft_consensus.cc:2468] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 0.
I20260709 12:16:08.334815 23981 leader_election.cc:304] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:08.335698 24285 ts_tablet_manager.cc:1434] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.056s	user 0.007s	sys 0.001s
I20260709 12:16:08.351723 24285 tablet_bootstrap.cc:492] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:08.330658 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ff9d150a47feb68f490efce1e3f2" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:08.352420 24166 raft_consensus.cc:2468] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 0.
I20260709 12:16:08.346712 24728 raft_consensus.cc:2804] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:08.353729 24728 raft_consensus.cc:493] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:08.354240 24728 raft_consensus.cc:3060] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:08.343844 24747 raft_consensus.cc:515] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:08.360215 24747 leader_election.cc:290] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:08.365888 24010 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c7696ee73146b59bd8b17640333077" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c359614416104a7ba9db1ede97d13802"
W20260709 12:16:08.367977 24130 leader_election.cc:343] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:08.369644 24237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c7696ee73146b59bd8b17640333077" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:08.373816 24237 raft_consensus.cc:3060] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:08.395810 24728 raft_consensus.cc:515] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:08.397969 24728 leader_election.cc:290] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:08.398965 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ff9d150a47feb68f490efce1e3f2" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:08.399636 24091 raft_consensus.cc:3060] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:08.400544 24285 tablet_bootstrap.cc:654] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:08.407510 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ff9d150a47feb68f490efce1e3f2" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:08.408267 24166 raft_consensus.cc:3060] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:08.412092 24237 raft_consensus.cc:2468] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:16:08.418442 24166 raft_consensus.cc:2468] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 1.
I20260709 12:16:08.421663 24091 raft_consensus.cc:2468] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 1.
I20260709 12:16:08.427289 23981 leader_election.cc:304] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:08.444000 24285 tablet_bootstrap.cc:492] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802: No bootstrap required, opened a new log
I20260709 12:16:08.444586 24285 ts_tablet_manager.cc:1403] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.093s	user 0.011s	sys 0.004s
I20260709 12:16:08.437592 24728 raft_consensus.cc:2804] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:08.447295 24728 raft_consensus.cc:697] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:16:08.448298 24728 consensus_queue.cc:237] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:08.455098 24132 leader_election.cc:304] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:08.456095 24802 raft_consensus.cc:2804] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:08.456629 24802 raft_consensus.cc:697] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:08.457512 24802 consensus_queue.cc:237] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:08.461021 24285 raft_consensus.cc:359] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:08.461802 24285 raft_consensus.cc:385] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:08.462143 24285 raft_consensus.cc:740] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:08.462860 24285 consensus_queue.cc:260] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:08.468219 24285 ts_tablet_manager.cc:1434] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.023s	user 0.008s	sys 0.000s
I20260709 12:16:08.479766 23911 catalog_manager.cc:5697] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 0 to 1, leader changed from <none> to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:08.485323 23914 catalog_manager.cc:5697] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 0 to 1, leader changed from <none> to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:08.546398 24728 raft_consensus.cc:493] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:08.547200 24728 raft_consensus.cc:515] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:08.549242 24728 leader_election.cc:290] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:08.550158 24237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147e5dc12974e6c852f8961496066ae" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:08.550474 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2147e5dc12974e6c852f8961496066ae" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:08.551177 24237 raft_consensus.cc:2393] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c359614416104a7ba9db1ede97d13802 in current term 1: Already voted for candidate 6c74a5203b594708921cd793aaca0b8b in this term.
I20260709 12:16:08.552850 23983 leader_election.cc:304] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:08.553820 24728 raft_consensus.cc:3060] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:08.564177 24728 raft_consensus.cc:2749] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:08.607208 24728 raft_consensus.cc:493] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:08.607720 24728 raft_consensus.cc:515] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:08.609608 24728 leader_election.cc:290] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:08.610505 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f15aae25cad42abb701c295e56963e3" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:08.610683 24237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6f15aae25cad42abb701c295e56963e3" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:08.612416 23983 leader_election.cc:304] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:08.613319 24728 raft_consensus.cc:2749] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:08.697985 24012 raft_consensus.cc:3060] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:08.703064 24728 raft_consensus.cc:493] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:08.703780 24728 raft_consensus.cc:515] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:08.708519 24704 consensus_queue.cc:1048] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:08.708868 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c7696ee73146b59bd8b17640333077" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:08.712241 24728 leader_election.cc:290] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:08.719587 24234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "57c7696ee73146b59bd8b17640333077" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:08.721093 24234 raft_consensus.cc:2393] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c359614416104a7ba9db1ede97d13802 in current term 1: Already voted for candidate 008a79e516614c809c2d3afdb5cbc327 in this term.
I20260709 12:16:08.722506 23983 leader_election.cc:304] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:08.723263 24728 raft_consensus.cc:3060] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:08.731370 24728 raft_consensus.cc:2749] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:08.746239 24704 consensus_queue.cc:1048] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
I20260709 12:16:08.864854 24725 consensus_queue.cc:1048] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:08.886955 24728 consensus_queue.cc:1048] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:08.893790 24722 consensus_queue.cc:1048] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:08.926879 24722 consensus_queue.cc:1048] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:09.918052 11318 tablet_server.cc:179] TabletServer@127.11.13.129:0 shutting down...
W20260709 12:16:09.948176 24130 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:09.959882 24130 consensus_peers.cc:597] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:09.968138 24130 consensus_peers.cc:597] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.006268 24130 consensus_peers.cc:597] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.008939 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:10.015264 11318 tablet_replica.cc:333] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.022689 11318 raft_consensus.cc:2243] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.023313 11318 raft_consensus.cc:2272] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.026081 11318 tablet_replica.cc:333] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
W20260709 12:16:10.026767 24204 consensus_peers.cc:597] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.027870 11318 raft_consensus.cc:2243] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.028554 11318 raft_consensus.cc:2272] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.033043 11318 tablet_replica.cc:333] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.034423 11318 raft_consensus.cc:2243] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.035320 11318 raft_consensus.cc:2272] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:10.036317 24204 consensus_peers.cc:597] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.038945 11318 tablet_replica.cc:333] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.040253 11318 raft_consensus.cc:2243] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:16:10.042621 24204 consensus_peers.cc:597] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.050583 11318 raft_consensus.cc:2272] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:10.051926 24204 consensus_peers.cc:597] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.052788 24204 consensus_peers.cc:597] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.054457 24130 consensus_peers.cc:597] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.055231 11318 tablet_replica.cc:333] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.059031 11318 raft_consensus.cc:2243] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.059659 11318 raft_consensus.cc:2272] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:10.061416 24130 consensus_peers.cc:597] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.063853 11318 tablet_replica.cc:333] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.064560 11318 raft_consensus.cc:2243] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.065024 11318 raft_consensus.cc:2272] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:10.068979 24056 consensus_peers.cc:597] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.070040 11318 tablet_replica.cc:333] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802: stopping tablet replica
W20260709 12:16:10.070761 24056 consensus_peers.cc:597] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.071033 11318 raft_consensus.cc:2243] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.071620 11318 raft_consensus.cc:2272] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.073664 11318 tablet_replica.cc:333] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.074445 11318 raft_consensus.cc:2243] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:10.075316 11318 raft_consensus.cc:2272] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.077333 11318 tablet_replica.cc:333] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.078202 11318 raft_consensus.cc:2243] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.078658 11318 raft_consensus.cc:2272] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.080715 11318 tablet_replica.cc:333] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.081444 11318 raft_consensus.cc:2243] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:10.082415 11318 raft_consensus.cc:2272] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.085004 11318 tablet_replica.cc:333] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.086024 11318 raft_consensus.cc:2243] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.086663 11318 raft_consensus.cc:2272] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.090039 11318 tablet_replica.cc:333] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802: stopping tablet replica
W20260709 12:16:10.091385 24130 consensus_peers.cc:597] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.091702 11318 raft_consensus.cc:2243] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.092921 11318 raft_consensus.cc:2272] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.095788 11318 tablet_replica.cc:333] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.096820 11318 raft_consensus.cc:2243] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.097671 11318 raft_consensus.cc:2272] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.099989 11318 tablet_replica.cc:333] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.100689 11318 raft_consensus.cc:2243] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.101197 11318 raft_consensus.cc:2272] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.103340 11318 tablet_replica.cc:333] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.103986 11318 raft_consensus.cc:2243] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.104441 11318 raft_consensus.cc:2272] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.106756 11318 tablet_replica.cc:333] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.107455 11318 raft_consensus.cc:2243] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.107933 11318 raft_consensus.cc:2272] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.110239 11318 tablet_replica.cc:333] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.110862 11318 raft_consensus.cc:2243] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.111279 11318 raft_consensus.cc:2272] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.114039 11318 tablet_replica.cc:333] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.115154 11318 raft_consensus.cc:2243] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.115836 11318 raft_consensus.cc:2272] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.118860 11318 tablet_replica.cc:333] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.119820 11318 raft_consensus.cc:2243] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.120638 11318 raft_consensus.cc:2272] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:10.127005 24130 consensus_peers.cc:597] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.128114 24204 consensus_peers.cc:597] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.130833 11318 tablet_replica.cc:333] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.132167 11318 raft_consensus.cc:2243] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:10.133605 11318 raft_consensus.cc:2272] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.137756 11318 tablet_replica.cc:333] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.138908 11318 raft_consensus.cc:2243] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:10.140393 11318 raft_consensus.cc:2272] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.144122 11318 tablet_replica.cc:333] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.145150 11318 raft_consensus.cc:2243] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.146026 11318 raft_consensus.cc:2272] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:10.148109 24130 consensus_peers.cc:597] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.150362 11318 tablet_replica.cc:333] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.151417 11318 raft_consensus.cc:2243] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.152573 11318 raft_consensus.cc:2272] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.156811 11318 tablet_replica.cc:333] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.157725 11318 raft_consensus.cc:2243] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:10.159628 11318 raft_consensus.cc:2272] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:10.161362 24056 consensus_peers.cc:597] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.162374 24056 consensus_peers.cc:597] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.162787 11318 tablet_replica.cc:333] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802: stopping tablet replica
W20260709 12:16:10.163614 24130 consensus_peers.cc:597] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.163743 11318 raft_consensus.cc:2243] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:16:10.164386 24130 consensus_peers.cc:597] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.165197 11318 raft_consensus.cc:2272] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.167780 11318 tablet_replica.cc:333] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.168386 11318 raft_consensus.cc:2243] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.169090 11318 raft_consensus.cc:2272] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.171859 11318 tablet_replica.cc:333] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.172653 11318 raft_consensus.cc:2243] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.173182 11318 raft_consensus.cc:2272] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:10.174891 24204 consensus_peers.cc:597] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.175791 11318 tablet_replica.cc:333] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.176455 11318 raft_consensus.cc:2243] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.176949 11318 raft_consensus.cc:2272] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.179219 11318 tablet_replica.cc:333] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.180378 11318 raft_consensus.cc:2243] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.180928 11318 raft_consensus.cc:2272] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.183933 11318 tablet_replica.cc:333] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.184643 11318 raft_consensus.cc:2243] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:16:10.184566 24130 consensus_peers.cc:597] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.185480 11318 raft_consensus.cc:2272] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.188882 11318 tablet_replica.cc:333] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.189946 11318 raft_consensus.cc:2243] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.190615 11318 raft_consensus.cc:2272] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:10.191946 24204 consensus_peers.cc:597] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.193128 11318 tablet_replica.cc:333] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.194399 11318 raft_consensus.cc:2243] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:10.195369 11318 raft_consensus.cc:2272] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.197706 11318 tablet_replica.cc:333] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.198395 11318 raft_consensus.cc:2243] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.198980 11318 raft_consensus.cc:2272] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.202178 11318 tablet_replica.cc:333] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.203075 11318 raft_consensus.cc:2243] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:10.204200 11318 raft_consensus.cc:2272] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.207232 11318 tablet_replica.cc:333] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
W20260709 12:16:10.207814 24056 consensus_peers.cc:597] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.208128 11318 raft_consensus.cc:2243] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.208786 11318 raft_consensus.cc:2272] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.211511 11318 tablet_replica.cc:333] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.212139 11318 raft_consensus.cc:2243] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:10.213104 11318 raft_consensus.cc:2272] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.215395 11318 tablet_replica.cc:333] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.216010 11318 raft_consensus.cc:2243] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.216486 11318 raft_consensus.cc:2272] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.218871 11318 tablet_replica.cc:333] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.219764 11318 raft_consensus.cc:2243] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.220336 11318 raft_consensus.cc:2272] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.223275 11318 tablet_replica.cc:333] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.224085 11318 raft_consensus.cc:2243] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.224723 11318 raft_consensus.cc:2272] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.228619 11318 tablet_replica.cc:333] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
W20260709 12:16:10.228713 24130 consensus_peers.cc:597] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.229799 11318 raft_consensus.cc:2243] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:10.231104 11318 raft_consensus.cc:2272] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.233858 11318 tablet_replica.cc:333] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.234516 11318 raft_consensus.cc:2243] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:10.235466 11318 raft_consensus.cc:2272] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.237602 11318 tablet_replica.cc:333] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.238291 11318 raft_consensus.cc:2243] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.239079 11318 raft_consensus.cc:2272] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.244154 11318 tablet_replica.cc:333] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.244858 11318 raft_consensus.cc:2243] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.245455 11318 raft_consensus.cc:2272] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.248360 11318 tablet_replica.cc:333] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.249006 11318 raft_consensus.cc:2243] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.249668 11318 raft_consensus.cc:2272] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:10.252045 11318 tablet_replica.cc:333] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802: stopping tablet replica
I20260709 12:16:10.252622 11318 raft_consensus.cc:2243] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:10.253153 11318 raft_consensus.cc:2272] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:10.271740 24056 consensus_peers.cc:597] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.310781 24056 consensus_peers.cc:597] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.318078 24056 consensus_peers.cc:597] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.338955 24130 connection.cc:573] client connection to 127.11.13.129:42669 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260709 12:16:10.355369 24204 consensus_peers.cc:597] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.365213 11318 tablet_server.cc:196] TabletServer@127.11.13.129:0 shutdown complete.
W20260709 12:16:10.508890 24056 consensus_peers.cc:597] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.509660 24056 consensus_peers.cc:597] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.513068 24130 consensus_peers.cc:597] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:10.542326 24056 consensus_peers.cc:597] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:10.609337 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:10.630025 24831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:10.728477 24832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:10.748577 24834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:10.754078 11318 server_base.cc:1061] running on GCE node
I20260709 12:16:10.766031 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:10.766453 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:10.766763 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599370766740 us; error 0 us; skew 500 ppm
I20260709 12:16:10.767570 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:10.773038 11318 webserver.cc:533] Webserver started at http://127.11.13.129:36291/ using document root <none> and password file <none>
I20260709 12:16:10.773927 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:10.774225 11318 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:16:10.780193 11318 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:16:10.794198 24839 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:10.796831 11318 fs_manager.cc:730] Time spent opening block manager: real 0.005s	user 0.003s	sys 0.001s
I20260709 12:16:10.797374 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-0-root
uuid: "c359614416104a7ba9db1ede97d13802"
format_stamp: "Formatted at 2026-07-09 12:15:54 on dist-test-slave-zqnl"
I20260709 12:16:10.798086 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.RestartTserver.1783598892056905-11318-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:10.850049 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:10.851922 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:10.854202 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:10.889555 24846 ts_tablet_manager.cc:548] Loading tablet metadata (0/45 complete)
I20260709 12:16:11.062588 24827 raft_consensus.cc:493] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.063300 24827 raft_consensus.cc:515] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:11.065732 24827 leader_election.cc:290] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:11.066679 24657 raft_consensus.cc:493] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.066944 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77079c31ca114c49a916e49cfd8faf2c" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:11.067296 24657 raft_consensus.cc:515] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:11.070071 24657 leader_election.cc:290] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:11.070770 24238 raft_consensus.cc:2468] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:16:11.073021 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77079c31ca114c49a916e49cfd8faf2c" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:11.073765 24093 raft_consensus.cc:2468] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 1.
I20260709 12:16:11.073992 24058 leader_election.cc:304] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:11.075134 24827 raft_consensus.cc:2804] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:11.075690 24827 raft_consensus.cc:493] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting leader election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.076512 24827 raft_consensus.cc:3060] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.075938 24204 leader_election.cc:304] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:11.077672 24657 raft_consensus.cc:2804] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:11.078415 24657 raft_consensus.cc:493] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.078820 24657 raft_consensus.cc:3060] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:16:11.081010 24204 leader_election.cc:336] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.088388 24657 raft_consensus.cc:515] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
W20260709 12:16:11.091015 24056 leader_election.cc:336] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
W20260709 12:16:11.107146 24204 leader_election.cc:336] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.111452 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77079c31ca114c49a916e49cfd8faf2c" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:11.108702 24657 leader_election.cc:290] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:11.119009 24827 raft_consensus.cc:515] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:11.121174 24093 raft_consensus.cc:2393] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a0df0eff47b64e3eb45ab074139ed187 in current term 2: Already voted for candidate 6c74a5203b594708921cd793aaca0b8b in this term.
I20260709 12:16:11.129558 24657 raft_consensus.cc:493] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.130400 24657 raft_consensus.cc:515] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:11.129714 24204 leader_election.cc:304] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.132004 24830 raft_consensus.cc:2749] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:16:11.132844 24657 leader_election.cc:290] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:11.134716 24162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd0baecffd2454894539252387f4f2d" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
W20260709 12:16:11.144400 24204 leader_election.cc:336] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.148136 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77079c31ca114c49a916e49cfd8faf2c" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:11.149108 24238 raft_consensus.cc:2393] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6c74a5203b594708921cd793aaca0b8b in current term 2: Already voted for candidate a0df0eff47b64e3eb45ab074139ed187 in this term.
I20260709 12:16:11.158533 24819 raft_consensus.cc:493] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.159159 24819 raft_consensus.cc:515] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:11.163794 24828 raft_consensus.cc:493] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.164459 24828 raft_consensus.cc:515] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:11.167295 24828 leader_election.cc:290] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:11.170312 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd0b13fa2ed4f4489eef33489d4ebf1" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:11.171932 24819 leader_election.cc:290] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
W20260709 12:16:11.173038 24056 leader_election.cc:336] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.173678 24056 leader_election.cc:304] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.175312 24819 raft_consensus.cc:2749] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260709 12:16:11.178808 24162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcccd54cb1fa4e9d9cab5afd3e8d4748" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:11.181785 24204 leader_election.cc:304] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.183435 24827 leader_election.cc:290] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:11.184929 24747 raft_consensus.cc:493] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.185984 24747 raft_consensus.cc:515] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:11.188189 24747 leader_election.cc:290] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:11.191901 24824 raft_consensus.cc:2749] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:11.194767 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcccd54cb1fa4e9d9cab5afd3e8d4748" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:11.195616 24091 raft_consensus.cc:2468] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:16:11.199877 24130 leader_election.cc:304] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: 
I20260709 12:16:11.200956 24802 raft_consensus.cc:2804] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:11.201463 24802 raft_consensus.cc:493] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.201882 24802 raft_consensus.cc:3060] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:16:11.208700 24056 leader_election.cc:336] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.209244 24056 leader_election.cc:304] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
W20260709 12:16:11.210189 24056 leader_election.cc:336] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.213811 24056 leader_election.cc:304] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.212988 24827 raft_consensus.cc:2749] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:11.215940 24828 raft_consensus.cc:2749] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:16:11.220192 24130 leader_election.cc:336] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.232146 24802 raft_consensus.cc:515] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:11.234545 24802 leader_election.cc:290] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:11.241564 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "dcccd54cb1fa4e9d9cab5afd3e8d4748" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:11.242293 24091 raft_consensus.cc:3060] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:16:11.249195 24130 leader_election.cc:336] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.250941 24091 raft_consensus.cc:2468] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 2.
I20260709 12:16:11.265954 24130 leader_election.cc:304] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.269975 24802 raft_consensus.cc:2804] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:11.270586 24802 raft_consensus.cc:697] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:11.271998 24819 raft_consensus.cc:493] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.272607 24819 raft_consensus.cc:515] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:11.275383 24819 leader_election.cc:290] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:11.272010 24802 consensus_queue.cc:237] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:11.290597 24830 raft_consensus.cc:493] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.291204 24830 raft_consensus.cc:515] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:11.293478 24830 leader_election.cc:290] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
W20260709 12:16:11.301777 24204 leader_election.cc:336] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.303755 24824 raft_consensus.cc:493] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.304330 24824 raft_consensus.cc:515] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:11.304787 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c505fd92654536b72dbd8a3f1bc728" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:11.306277 24204 leader_election.cc:304] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.306744 24824 leader_election.cc:290] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:11.307277 24824 raft_consensus.cc:2749] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:11.308037 24747 raft_consensus.cc:493] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.308585 24747 raft_consensus.cc:515] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:11.310724 24747 leader_election.cc:290] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:11.311507 24162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e815126e7554106a097f95846d7f663" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:11.315663 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd0baecffd2454894539252387f4f2d" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
W20260709 12:16:11.313426 24204 leader_election.cc:336] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.317401 24204 leader_election.cc:304] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.318517 24824 raft_consensus.cc:2749] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:11.316390 24238 raft_consensus.cc:2468] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:16:11.323954 24132 leader_election.cc:304] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:11.326020 24750 raft_consensus.cc:2804] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:11.326459 24750 raft_consensus.cc:493] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.326820 24750 raft_consensus.cc:3060] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:16:11.339239 24130 leader_election.cc:336] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.355437 24848 raft_consensus.cc:493] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.356215 24848 raft_consensus.cc:515] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:11.358914 24848 leader_election.cc:290] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:11.362146 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ff9d150a47feb68f490efce1e3f2" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:11.384023 24750 raft_consensus.cc:515] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:11.388749 24162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5587d307c2f64ab9a9b39c1b190e85be" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:11.391109 24819 raft_consensus.cc:493] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.412190 24819 raft_consensus.cc:515] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:11.414292 24819 leader_election.cc:290] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:11.399348 24827 raft_consensus.cc:493] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.415047 24827 raft_consensus.cc:515] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:11.417163 24827 leader_election.cc:290] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
W20260709 12:16:11.403544 24132 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:41011, user_credentials={real_user=slave}} blocked reactor thread for 44636.3us
W20260709 12:16:11.405215 24130 leader_election.cc:336] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.419471 24130 leader_election.cc:304] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.420425 24849 raft_consensus.cc:2749] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:11.427497 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (45 total tablets, 45 live tablets)
I20260709 12:16:11.427956 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.564s	user 0.023s	sys 0.012s
I20260709 12:16:11.428333 11318 ts_tablet_manager.cc:600] Registering tablets (0/45 complete)
I20260709 12:16:11.432116 24750 leader_election.cc:290] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:11.432720 24830 raft_consensus.cc:493] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.433363 24830 raft_consensus.cc:515] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:11.436009 24830 leader_election.cc:290] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:11.439303 24827 raft_consensus.cc:493] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.440176 24846 tablet_bootstrap.cc:492] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:11.439908 24827 raft_consensus.cc:515] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:11.442464 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd0b13fa2ed4f4489eef33489d4ebf1" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
W20260709 12:16:11.443208 24204 leader_election.cc:336] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.443622 24091 raft_consensus.cc:2468] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 1.
I20260709 12:16:11.436018 24237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd0baecffd2454894539252387f4f2d" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:11.444705 24237 raft_consensus.cc:3060] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.445140 24204 leader_election.cc:304] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.446358 24830 raft_consensus.cc:2804] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:11.446820 24830 raft_consensus.cc:493] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.447809 24827 leader_election.cc:290] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:11.447239 24830 raft_consensus.cc:3060] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.453186 24237 raft_consensus.cc:2468] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 2.
I20260709 12:16:11.471529 24162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf791a63ce7546f38a312c7f77142a61" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:11.475528 24162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ff9d150a47feb68f490efce1e3f2" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:11.476277 24162 raft_consensus.cc:2468] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:16:11.479166 24827 raft_consensus.cc:493] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.480341 24827 raft_consensus.cc:515] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:11.480872 24162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96039384602f4057b875e1d873138d47" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:11.479758 24846 tablet_bootstrap.cc:492] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:11.483518 24846 tablet_bootstrap.cc:492] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:11.483999 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55819e09f3e245179f0dd80b9e60eb6a" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:11.484545 24846 ts_tablet_manager.cc:1403] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.045s	user 0.016s	sys 0.004s
I20260709 12:16:11.484998 23911 catalog_manager.cc:5697] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 1 to 2, leader changed from c359614416104a7ba9db1ede97d13802 (127.11.13.129) to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 2 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:11.486625 24827 leader_election.cc:290] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
W20260709 12:16:11.483314 24130 leader_election.cc:336] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.487679 24846 raft_consensus.cc:359] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:11.499119 24846 raft_consensus.cc:740] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:11.500146 24846 consensus_queue.cc:260] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:11.502601 24056 leader_election.cc:304] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: 
I20260709 12:16:11.457592 24830 raft_consensus.cc:515] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:11.507019 24846 ts_tablet_manager.cc:1434] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 12:16:11.508210 24846 tablet_bootstrap.cc:492] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:11.508412 24132 leader_election.cc:304] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.509349 24827 raft_consensus.cc:2804] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:11.509806 24827 raft_consensus.cc:493] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting leader election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.510202 24827 raft_consensus.cc:3060] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.511571 24747 raft_consensus.cc:2804] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:11.512102 24747 raft_consensus.cc:697] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:11.512976 24747 consensus_queue.cc:237] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:11.515049 24852 raft_consensus.cc:493] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.515627 24852 raft_consensus.cc:515] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:11.519160 24802 raft_consensus.cc:493] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.519789 24830 leader_election.cc:290] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:11.520023 24802 raft_consensus.cc:515] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:11.520642 24819 raft_consensus.cc:493] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.521255 24819 raft_consensus.cc:515] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:11.521958 24848 raft_consensus.cc:493] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.522754 24848 raft_consensus.cc:515] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:11.523825 24819 leader_election.cc:290] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:11.524483 24802 leader_election.cc:290] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:11.524869 24848 leader_election.cc:290] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:11.525494 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c505fd92654536b72dbd8a3f1bc728" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:11.525687 24852 leader_election.cc:290] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:11.526224 24238 raft_consensus.cc:2468] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
W20260709 12:16:11.527842 24204 leader_election.cc:336] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.528211 24058 leader_election.cc:304] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:11.530521 24819 raft_consensus.cc:2804] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:11.530977 24819 raft_consensus.cc:493] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting leader election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.531625 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd0b13fa2ed4f4489eef33489d4ebf1" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:11.532836 24091 raft_consensus.cc:3060] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.533761 24824 raft_consensus.cc:493] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.534662 24824 raft_consensus.cc:515] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:11.536935 24824 leader_election.cc:290] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:11.538403 24819 raft_consensus.cc:3060] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.544440 24091 raft_consensus.cc:2468] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 2.
I20260709 12:16:11.548172 24747 raft_consensus.cc:493] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.549047 24747 raft_consensus.cc:515] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:11.551891 24747 leader_election.cc:290] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:11.553740 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e815126e7554106a097f95846d7f663" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
W20260709 12:16:11.553980 24130 leader_election.cc:336] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.554500 24238 raft_consensus.cc:2468] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:16:11.555740 24132 leader_election.cc:304] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:11.556802 24747 raft_consensus.cc:2804] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:11.557498 24747 raft_consensus.cc:493] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c359614416104a7ba9db1ede97d13802)
W20260709 12:16:11.557765 24056 leader_election.cc:336] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.557997 24747 raft_consensus.cc:3060] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.563030 24056 leader_election.cc:304] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.555941 24827 raft_consensus.cc:515] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
W20260709 12:16:11.565809 24204 leader_election.cc:336] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.566774 24828 raft_consensus.cc:2749] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:16:11.554563 24130 leader_election.cc:336] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
W20260709 12:16:11.571686 24130 leader_election.cc:336] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.584955 24747 raft_consensus.cc:515] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
W20260709 12:16:11.597687 24130 leader_election.cc:336] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.590725 24827 leader_election.cc:290] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
W20260709 12:16:11.601323 24058 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.132:41011, user_credentials={real_user=slave}} blocked reactor thread for 73186us
I20260709 12:16:11.602408 24204 leader_election.cc:304] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.602892 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e3897eed824c5fa96e04ce0eca32a8" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:11.604733 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e815126e7554106a097f95846d7f663" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:11.604951 24747 leader_election.cc:290] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:11.605221 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96039384602f4057b875e1d873138d47" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:11.605424 24238 raft_consensus.cc:3060] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.605916 24093 raft_consensus.cc:2468] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:16:11.607337 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf791a63ce7546f38a312c7f77142a61" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:11.608117 24093 raft_consensus.cc:2468] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:16:11.608358 24830 raft_consensus.cc:2804] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:11.609115 24830 raft_consensus.cc:697] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [term 2 LEADER]: Becoming Leader. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Running, Role: LEADER
I20260709 12:16:11.610033 24830 consensus_queue.cc:237] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
W20260709 12:16:11.614583 24130 leader_election.cc:336] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.620532 24130 leader_election.cc:304] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.623195 24846 tablet_bootstrap.cc:492] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:11.617233 24819 raft_consensus.cc:515] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:11.633055 24819 leader_election.cc:290] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:11.634722 24237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "41c505fd92654536b72dbd8a3f1bc728" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:11.635378 24237 raft_consensus.cc:3060] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.628726 24238 raft_consensus.cc:2468] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 2.
I20260709 12:16:11.624221 24162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0192ff9d150a47feb68f490efce1e3f2" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:11.638800 24162 raft_consensus.cc:3060] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.639532 24132 leader_election.cc:304] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.640946 24747 raft_consensus.cc:2804] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:11.629966 24852 raft_consensus.cc:2749] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:11.629974 24846 tablet_bootstrap.cc:492] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:11.630986 24130 leader_election.cc:304] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.643316 24846 ts_tablet_manager.cc:1403] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.135s	user 0.030s	sys 0.017s
I20260709 12:16:11.644459 24852 raft_consensus.cc:2804] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:11.645047 24852 raft_consensus.cc:493] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.645555 24852 raft_consensus.cc:3060] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.647315 24846 raft_consensus.cc:359] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:11.648168 24846 raft_consensus.cc:740] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:11.649132 24846 consensus_queue.cc:260] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:11.626468 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55819e09f3e245179f0dd80b9e60eb6a" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:11.650681 24091 raft_consensus.cc:2468] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 1.
I20260709 12:16:11.651549 24846 ts_tablet_manager.cc:1434] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:16:11.652565 24846 tablet_bootstrap.cc:492] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:11.643748 24130 leader_election.cc:304] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.663782 24802 raft_consensus.cc:2804] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:11.664451 24802 raft_consensus.cc:493] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.664932 24802 raft_consensus.cc:3060] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.654170 24237 raft_consensus.cc:2468] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 2.
I20260709 12:16:11.654461 24204 leader_election.cc:304] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: c359614416104a7ba9db1ede97d13802
W20260709 12:16:11.654759 24056 leader_election.cc:336] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.641964 24747 raft_consensus.cc:697] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:11.676759 24830 raft_consensus.cc:2804] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:11.678359 24830 raft_consensus.cc:493] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.678925 24830 raft_consensus.cc:3060] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.678710 24747 consensus_queue.cc:237] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:11.682085 24848 raft_consensus.cc:493] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.682811 24848 raft_consensus.cc:515] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:11.685441 24848 leader_election.cc:290] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:11.676115 24056 leader_election.cc:304] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.702824 24802 raft_consensus.cc:515] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:11.704555 24852 raft_consensus.cc:515] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:11.705262 24802 leader_election.cc:290] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:11.707443 24162 raft_consensus.cc:2468] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 2.
W20260709 12:16:11.714022 24056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:42669, user_credentials={real_user=slave}} blocked reactor thread for 59290us
W20260709 12:16:11.714687 24056 leader_election.cc:336] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.715265 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5587d307c2f64ab9a9b39c1b190e85be" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:11.715930 24091 raft_consensus.cc:2468] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 1.
I20260709 12:16:11.717408 24827 raft_consensus.cc:2749] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:16:11.718906 24130 leader_election.cc:336] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.693755 24830 raft_consensus.cc:515] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:11.724264 24830 leader_election.cc:290] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
W20260709 12:16:11.725710 24130 leader_election.cc:336] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.727046 24058 leader_election.cc:304] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:11.728497 24827 raft_consensus.cc:2804] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:11.729138 24827 raft_consensus.cc:697] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [term 2 LEADER]: Becoming Leader. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Running, Role: LEADER
I20260709 12:16:11.727208 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf791a63ce7546f38a312c7f77142a61" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:11.733165 24091 raft_consensus.cc:3060] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.735543 24827 consensus_queue.cc:237] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:11.743856 24091 raft_consensus.cc:2468] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 2.
I20260709 12:16:11.744943 23911 catalog_manager.cc:5697] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 reported cstate change: term changed from 1 to 2, leader changed from c359614416104a7ba9db1ede97d13802 (127.11.13.129) to a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132). New cstate: current_term: 2 leader_uuid: "a0df0eff47b64e3eb45ab074139ed187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:11.729141 24130 leader_election.cc:304] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: c359614416104a7ba9db1ede97d13802
W20260709 12:16:11.715763 24056 leader_election.cc:336] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.758786 24056 leader_election.cc:304] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
W20260709 12:16:11.761570 24056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:42669, user_credentials={real_user=slave}} blocked reactor thread for 45826.7us
I20260709 12:16:11.764508 24802 raft_consensus.cc:2804] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:11.765285 24802 raft_consensus.cc:493] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Starting leader election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.765817 24802 raft_consensus.cc:3060] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.764313 24819 raft_consensus.cc:2749] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:11.780063 24852 leader_election.cc:290] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:11.792811 24130 leader_election.cc:304] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.797050 24802 raft_consensus.cc:515] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:11.813268 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "96039384602f4057b875e1d873138d47" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:11.813553 24849 raft_consensus.cc:2804] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:11.813966 24084 raft_consensus.cc:3060] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.814395 24849 raft_consensus.cc:697] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:11.815428 24849 consensus_queue.cc:237] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
W20260709 12:16:11.822618 24056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:42669, user_credentials={real_user=slave}} blocked reactor thread for 58644.7us
I20260709 12:16:11.825340 24802 leader_election.cc:290] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:11.829121 24085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5587d307c2f64ab9a9b39c1b190e85be" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:11.829842 24085 raft_consensus.cc:3060] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.827006 24087 raft_consensus.cc:1275] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Refusing update from remote peer 008a79e516614c809c2d3afdb5cbc327: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:11.832532 24860 raft_consensus.cc:493] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:11.833101 24860 raft_consensus.cc:515] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
W20260709 12:16:11.836047 24130 leader_election.cc:336] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.842344 24802 consensus_queue.cc:1048] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:11.843659 24860 leader_election.cc:290] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:11.819942 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "55819e09f3e245179f0dd80b9e60eb6a" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:11.862900 24091 raft_consensus.cc:3060] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:11.848940 24056 leader_election.cc:304] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: 
W20260709 12:16:11.853363 24130 consensus_peers.cc:597] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:11.854635 24204 leader_election.cc:336] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.859197 24846 tablet_bootstrap.cc:492] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:11.847541 24084 raft_consensus.cc:2468] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 2.
W20260709 12:16:11.868832 24130 leader_election.cc:336] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.880398 24091 raft_consensus.cc:2468] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 2.
I20260709 12:16:11.883836 23911 catalog_manager.cc:5697] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b reported cstate change: term changed from 1 to 2, leader changed from c359614416104a7ba9db1ede97d13802 (127.11.13.129) to 6c74a5203b594708921cd793aaca0b8b (127.11.13.130). New cstate: current_term: 2 leader_uuid: "6c74a5203b594708921cd793aaca0b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:11.888283 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e3897eed824c5fa96e04ce0eca32a8" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:11.889104 24163 raft_consensus.cc:2468] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 1.
I20260709 12:16:11.889597 24857 raft_consensus.cc:2804] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:11.888636 24846 tablet_bootstrap.cc:492] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:11.891235 24846 ts_tablet_manager.cc:1403] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.239s	user 0.032s	sys 0.019s
I20260709 12:16:11.891155 24130 leader_election.cc:304] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.872748 24085 raft_consensus.cc:2468] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 2.
I20260709 12:16:11.894141 24204 leader_election.cc:304] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.895110 24130 leader_election.cc:304] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.898206 24865 raft_consensus.cc:2804] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:11.898831 24865 raft_consensus.cc:697] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [term 2 LEADER]: Becoming Leader. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Running, Role: LEADER
I20260709 12:16:11.899720 24865 consensus_queue.cc:237] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:11.907079 24857 raft_consensus.cc:697] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [term 2 LEADER]: Becoming Leader. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Running, Role: LEADER
I20260709 12:16:11.907892 24846 raft_consensus.cc:359] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:11.908726 24846 raft_consensus.cc:740] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:11.908262 24857 consensus_queue.cc:237] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:11.910475 24750 raft_consensus.cc:2804] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:11.911052 24750 raft_consensus.cc:697] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:11.912415 24750 consensus_queue.cc:237] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
W20260709 12:16:11.915099 24056 leader_election.cc:336] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:11.915639 24056 leader_election.cc:304] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b; no voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802
I20260709 12:16:11.909560 24846 consensus_queue.cc:260] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
W20260709 12:16:11.917591 24056 leader_election.cc:336] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
W20260709 12:16:11.920042 24056 leader_election.cc:336] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
W20260709 12:16:11.987393 24056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:42669, user_credentials={real_user=slave}} blocked reactor thread for 67355.9us
I20260709 12:16:11.928483 24846 ts_tablet_manager.cc:1434] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.037s	user 0.003s	sys 0.003s
I20260709 12:16:11.920447 24819 raft_consensus.cc:2749] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:12.024292 24056 leader_election.cc:304] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: 
I20260709 12:16:11.920022 24849 raft_consensus.cc:2804] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:16:12.091626 24056 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:36699, user_credentials={real_user=slave}} blocked reactor thread for 67663.4us
I20260709 12:16:12.068461 23911 catalog_manager.cc:5697] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 1 to 2, leader changed from c359614416104a7ba9db1ede97d13802 (127.11.13.129) to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 2 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:12.096248 23911 catalog_manager.cc:5697] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 1 to 2, leader changed from c359614416104a7ba9db1ede97d13802 (127.11.13.129) to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 2 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:12.080206 24849 raft_consensus.cc:697] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:12.105405 24846 tablet_bootstrap.cc:492] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:12.093951 24828 raft_consensus.cc:2804] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:12.108978 24828 raft_consensus.cc:493] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Starting leader election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:12.109570 24828 raft_consensus.cc:3060] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:12.111176 24849 consensus_queue.cc:237] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:12.123737 24084 raft_consensus.cc:1275] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Refusing update from remote peer a0df0eff47b64e3eb45ab074139ed187: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:12.124046 24237 raft_consensus.cc:1275] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Refusing update from remote peer 008a79e516614c809c2d3afdb5cbc327: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:12.181347 24877 consensus_queue.cc:1048] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:12.201967 24657 consensus_queue.cc:1048] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:16:12.204032 24130 consensus_peers.cc:597] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:12.233233 23911 catalog_manager.cc:5697] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 reported cstate change: term changed from 1 to 2, leader changed from c359614416104a7ba9db1ede97d13802 (127.11.13.129) to a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132). New cstate: current_term: 2 leader_uuid: "a0df0eff47b64e3eb45ab074139ed187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:12.233636 23914 catalog_manager.cc:5697] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b reported cstate change: term changed from 1 to 2, leader changed from c359614416104a7ba9db1ede97d13802 (127.11.13.129) to 6c74a5203b594708921cd793aaca0b8b (127.11.13.130). New cstate: current_term: 2 leader_uuid: "6c74a5203b594708921cd793aaca0b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:12.231842 24056 leader_election.cc:336] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
W20260709 12:16:12.244755 24056 leader_election.cc:336] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:12.257884 24828 raft_consensus.cc:515] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:12.302239 23914 catalog_manager.cc:5697] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 1 to 2, leader changed from c359614416104a7ba9db1ede97d13802 (127.11.13.129) to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 2 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:12.306528 23914 catalog_manager.cc:5697] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 1 to 2, leader changed from c359614416104a7ba9db1ede97d13802 (127.11.13.129) to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 2 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:12.309536 24234 raft_consensus.cc:1275] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Refusing update from remote peer 6c74a5203b594708921cd793aaca0b8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:12.313879 24239 raft_consensus.cc:1275] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Refusing update from remote peer 008a79e516614c809c2d3afdb5cbc327: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:12.314561 23914 catalog_manager.cc:5697] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 1 to 2, leader changed from c359614416104a7ba9db1ede97d13802 (127.11.13.129) to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 2 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:12.321573 24828 leader_election.cc:290] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:12.328941 24819 consensus_queue.cc:1048] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:16:12.334244 24204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:36699, user_credentials={real_user=slave}} blocked reactor thread for 71748.6us
W20260709 12:16:12.339214 24204 consensus_peers.cc:597] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:12.339367 24877 consensus_queue.cc:1048] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:12.363322 24161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "31e3897eed824c5fa96e04ce0eca32a8" candidate_uuid: "6c74a5203b594708921cd793aaca0b8b" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:12.364387 24161 raft_consensus.cc:3060] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:12.375598 24161 raft_consensus.cc:2468] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c74a5203b594708921cd793aaca0b8b in term 2.
W20260709 12:16:12.383490 24056 consensus_peers.cc:597] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:12.395543 24056 leader_election.cc:304] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b; no voters: 
W20260709 12:16:12.426316 24056 leader_election.cc:336] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
W20260709 12:16:12.432281 24056 consensus_peers.cc:597] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:12.407785 24827 raft_consensus.cc:2804] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:16:12.436573 24130 consensus_peers.cc:597] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:12.437502 24130 consensus_peers.cc:597] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:12.433749 24827 raft_consensus.cc:697] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [term 2 LEADER]: Becoming Leader. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Running, Role: LEADER
I20260709 12:16:12.447927 24827 consensus_queue.cc:237] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:12.482020 11318 ts_tablet_manager.cc:600] Registering tablets (35/45 complete)
I20260709 12:16:12.486896 24846 tablet_bootstrap.cc:492] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:12.488225 24846 tablet_bootstrap.cc:492] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
W20260709 12:16:12.488643 24130 consensus_peers.cc:597] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:12.489094 24846 ts_tablet_manager.cc:1403] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.461s	user 0.041s	sys 0.008s
I20260709 12:16:12.498978 24085 raft_consensus.cc:1275] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Refusing update from remote peer 008a79e516614c809c2d3afdb5cbc327: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:12.501945 24088 raft_consensus.cc:1275] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Refusing update from remote peer a0df0eff47b64e3eb45ab074139ed187: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:12.513972 24162 raft_consensus.cc:1275] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Refusing update from remote peer 6c74a5203b594708921cd793aaca0b8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:16:12.531664 24204 consensus_peers.cc:597] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:12.541709 24846 raft_consensus.cc:359] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:12.542654 24846 raft_consensus.cc:740] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:12.547727 24876 consensus_queue.cc:1048] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:16:12.552883 24056 consensus_peers.cc:597] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:12.545284 24846 consensus_queue.cc:260] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:12.585729 24849 consensus_queue.cc:1048] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:16:12.591097 24130 consensus_peers.cc:597] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:12.612542 24204 consensus_peers.cc:597] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:12.613265 24204 consensus_peers.cc:597] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:12.617821 24883 consensus_queue.cc:1048] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:16:12.624025 24130 consensus_peers.cc:597] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:12.624709 24130 consensus_peers.cc:597] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:12.631580 24091 raft_consensus.cc:1275] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Refusing update from remote peer 008a79e516614c809c2d3afdb5cbc327: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:16:12.643141 24056 consensus_peers.cc:597] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:12.647219 24846 ts_tablet_manager.cc:1434] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.158s	user 0.016s	sys 0.004s
I20260709 12:16:12.648551 24846 tablet_bootstrap.cc:492] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:12.673375 24849 consensus_queue.cc:1048] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:16:12.676214 24204 consensus_peers.cc:597] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:12.677928 23911 catalog_manager.cc:5697] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b reported cstate change: term changed from 1 to 2, leader changed from c359614416104a7ba9db1ede97d13802 (127.11.13.129) to 6c74a5203b594708921cd793aaca0b8b (127.11.13.130). New cstate: current_term: 2 leader_uuid: "6c74a5203b594708921cd793aaca0b8b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:12.707392 24130 consensus_peers.cc:597] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:12.711681 24130 consensus_peers.cc:597] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:12.742861 24204 consensus_peers.cc:597] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:12.745632 24130 consensus_peers.cc:597] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:12.751077 24056 consensus_peers.cc:597] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:12.764782 24894 raft_consensus.cc:493] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:12.765573 24894 raft_consensus.cc:515] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:12.772367 24894 leader_election.cc:290] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
W20260709 12:16:12.782194 24204 consensus_peers.cc:597] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:12.784672 24865 raft_consensus.cc:493] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:12.786262 24865 raft_consensus.cc:515] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:12.789055 24865 leader_election.cc:290] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:12.797539 24087 raft_consensus.cc:1275] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Refusing update from remote peer 008a79e516614c809c2d3afdb5cbc327: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:12.802932 24091 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77079c31ca114c49a916e49cfd8faf2c" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:12.803766 24091 raft_consensus.cc:2468] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 2.
I20260709 12:16:12.806600 24204 leader_election.cc:304] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:12.807587 24824 raft_consensus.cc:2804] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:16:12.807417 24848 consensus_queue.cc:1048] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:12.808125 24824 raft_consensus.cc:493] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:12.808523 24824 raft_consensus.cc:3060] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:16:12.811295 24056 consensus_peers.cc:597] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:12.820787 24824 raft_consensus.cc:515] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
W20260709 12:16:12.823397 24130 consensus_peers.cc:597] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:12.827670 24130 consensus_peers.cc:597] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:12.833317 24056 consensus_peers.cc:597] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:12.839476 24204 consensus_peers.cc:597] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:12.841044 24204 leader_election.cc:336] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:12.860625 24824 leader_election.cc:290] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 election: Requested vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:12.879338 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95824f48a95746289333a9da8e265178" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:12.904804 24087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "77079c31ca114c49a916e49cfd8faf2c" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:12.905076 24162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "95824f48a95746289333a9da8e265178" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:12.905628 24087 raft_consensus.cc:3060] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:12.907632 24840 leader_election.cc:304] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327, 6c74a5203b594708921cd793aaca0b8b
I20260709 12:16:12.908627 24894 raft_consensus.cc:2749] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260709 12:16:12.911271 24204 consensus_peers.cc:597] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:12.918555 24130 consensus_peers.cc:597] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:12.887938 24846 tablet_bootstrap.cc:492] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:12.923928 24087 raft_consensus.cc:2468] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 3.
I20260709 12:16:12.924234 24846 tablet_bootstrap.cc:492] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:12.927445 24846 ts_tablet_manager.cc:1403] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.279s	user 0.022s	sys 0.006s
I20260709 12:16:12.930167 11318 ts_tablet_manager.cc:616] Registered 45 tablets
I20260709 12:16:12.930615 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 1.502s	user 0.359s	sys 0.015s
I20260709 12:16:12.933638 24846 raft_consensus.cc:359] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:12.934381 24846 raft_consensus.cc:740] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
W20260709 12:16:12.934966 24204 consensus_peers.cc:597] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:12.935526 24204 leader_election.cc:336] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:12.935163 24846 consensus_queue.cc:260] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:12.936538 24204 leader_election.cc:304] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: c359614416104a7ba9db1ede97d13802
I20260709 12:16:12.937439 24824 raft_consensus.cc:2804] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:12.938848 24824 raft_consensus.cc:697] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 3 LEADER]: Becoming Leader. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Running, Role: LEADER
I20260709 12:16:12.939967 24824 consensus_queue.cc:237] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:12.941515 24846 ts_tablet_manager.cc:1434] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.014s	user 0.006s	sys 0.000s
I20260709 12:16:12.944693 24846 tablet_bootstrap.cc:492] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
W20260709 12:16:12.946178 24056 consensus_peers.cc:597] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:12.952240 24162 raft_consensus.cc:1275] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Refusing update from remote peer 6c74a5203b594708921cd793aaca0b8b: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:12.953904 24876 consensus_queue.cc:1048] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:12.955830 23913 catalog_manager.cc:5697] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 reported cstate change: term changed from 1 to 3, leader changed from c359614416104a7ba9db1ede97d13802 (127.11.13.129) to a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132). New cstate: current_term: 3 leader_uuid: "a0df0eff47b64e3eb45ab074139ed187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:12.974418 24204 consensus_peers.cc:597] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:12.994997 24130 consensus_peers.cc:597] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:13.005046 24130 consensus_peers.cc:597] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:13.037925 24906 log.cc:927] Time spent T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187: Append to log took a long time: real 0.092s	user 0.000s	sys 0.000s
W20260709 12:16:13.061110 24130 consensus_peers.cc:597] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:13.064034 24894 raft_consensus.cc:493] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.066442 24894 raft_consensus.cc:515] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:13.086614 24894 leader_election.cc:290] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
W20260709 12:16:13.120394 24130 consensus_peers.cc:597] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:13.152175 24087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd0b13fa2ed4f4489eef33489d4ebf1" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:13.198567 24846 tablet_bootstrap.cc:492] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:13.199736 24846 tablet_bootstrap.cc:492] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:13.208442 24846 ts_tablet_manager.cc:1403] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.266s	user 0.086s	sys 0.118s
W20260709 12:16:13.214541 24056 consensus_peers.cc:597] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:13.219938 24056 consensus_peers.cc:597] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:13.211813 24846 raft_consensus.cc:359] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:13.223421 24846 raft_consensus.cc:740] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:13.219069 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7dd0b13fa2ed4f4489eef33489d4ebf1" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:13.225629 24842 leader_election.cc:304] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:13.227068 24894 raft_consensus.cc:2749] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:13.230657 24846 consensus_queue.cc:260] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
W20260709 12:16:13.233783 24056 consensus_peers.cc:597] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:13.233960 24846 ts_tablet_manager.cc:1434] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 12:16:13.237344 24846 tablet_bootstrap.cc:492] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
W20260709 12:16:13.241096 24056 consensus_peers.cc:597] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:13.297984 24846 tablet_bootstrap.cc:492] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:13.298858 24894 raft_consensus.cc:493] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.299722 24846 tablet_bootstrap.cc:492] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:13.299541 24894 raft_consensus.cc:515] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:13.301362 24846 ts_tablet_manager.cc:1403] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.064s	user 0.016s	sys 0.035s
I20260709 12:16:13.304250 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91f626cde148838a6a979d4fd1a5ed" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:13.304936 24846 raft_consensus.cc:359] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:13.305976 24846 raft_consensus.cc:740] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:13.307050 24846 consensus_queue.cc:260] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:13.302000 24894 leader_election.cc:290] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:13.312213 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "4c91f626cde148838a6a979d4fd1a5ed" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "6c74a5203b594708921cd793aaca0b8b" is_pre_election: true
I20260709 12:16:13.319984 24840 leader_election.cc:304] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:13.321040 24894 raft_consensus.cc:2749] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:13.337749 24846 ts_tablet_manager.cc:1434] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.036s	user 0.010s	sys 0.007s
I20260709 12:16:13.338953 24846 tablet_bootstrap.cc:492] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
W20260709 12:16:13.352073 24204 consensus_peers.cc:597] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:13.357942 24056 consensus_peers.cc:597] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:13.407181 24846 tablet_bootstrap.cc:492] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:13.408442 24846 tablet_bootstrap.cc:492] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:13.412241 24846 ts_tablet_manager.cc:1403] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.074s	user 0.033s	sys 0.005s
I20260709 12:16:13.416622 24846 raft_consensus.cc:359] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:13.417435 24846 raft_consensus.cc:740] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:13.427223 24846 consensus_queue.cc:260] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:13.451509 24846 ts_tablet_manager.cc:1434] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.022s	user 0.009s	sys 0.000s
I20260709 12:16:13.455991 24846 tablet_bootstrap.cc:492] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
W20260709 12:16:13.488366 24130 consensus_peers.cc:597] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:13.506327 24056 consensus_peers.cc:597] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:13.530810 24084 raft_consensus.cc:1275] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [term 3 FOLLOWER]: Refusing update from remote peer a0df0eff47b64e3eb45ab074139ed187: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:16:13.533167 24882 consensus_queue.cc:1048] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:13.543087 24846 tablet_bootstrap.cc:492] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:13.544381 24846 tablet_bootstrap.cc:492] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:13.562906 24846 ts_tablet_manager.cc:1403] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.107s	user 0.024s	sys 0.004s
I20260709 12:16:13.602656 24846 raft_consensus.cc:359] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:13.607815 24846 raft_consensus.cc:740] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:13.609105 24846 consensus_queue.cc:260] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:13.611274 24846 ts_tablet_manager.cc:1434] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.048s	user 0.007s	sys 0.000s
I20260709 12:16:13.612268 24846 tablet_bootstrap.cc:492] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:13.655050 24846 tablet_bootstrap.cc:492] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:13.656298 24846 tablet_bootstrap.cc:492] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:13.661429 24846 ts_tablet_manager.cc:1403] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.049s	user 0.017s	sys 0.003s
I20260709 12:16:13.665488 24846 raft_consensus.cc:359] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:13.666499 24846 raft_consensus.cc:740] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:13.667447 24846 consensus_queue.cc:260] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:13.671634 24846 ts_tablet_manager.cc:1434] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.010s	user 0.004s	sys 0.003s
I20260709 12:16:13.674394 24846 tablet_bootstrap.cc:492] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:13.686053 24920 raft_consensus.cc:493] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:13.686728 24920 raft_consensus.cc:515] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:13.691027 24162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e679d0d8ffd84a23afc6d7f8faf84ca6" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "008a79e516614c809c2d3afdb5cbc327" is_pre_election: true
I20260709 12:16:13.706077 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e679d0d8ffd84a23afc6d7f8faf84ca6" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a0df0eff47b64e3eb45ab074139ed187" is_pre_election: true
I20260709 12:16:13.708995 24842 leader_election.cc:304] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c359614416104a7ba9db1ede97d13802; no voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:13.710837 24894 raft_consensus.cc:2749] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:13.716699 24920 leader_election.cc:290] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:13.728777 24846 tablet_bootstrap.cc:492] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:13.734104 24846 tablet_bootstrap.cc:492] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:13.734979 24846 ts_tablet_manager.cc:1403] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.061s	user 0.039s	sys 0.006s
I20260709 12:16:13.740288 24846 raft_consensus.cc:359] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:13.741053 24846 raft_consensus.cc:740] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:13.741878 24846 consensus_queue.cc:260] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:13.744681 24846 ts_tablet_manager.cc:1434] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:16:13.745723 24846 tablet_bootstrap.cc:492] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:13.784467 24846 tablet_bootstrap.cc:492] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:13.787209 24846 tablet_bootstrap.cc:492] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:13.787956 24846 ts_tablet_manager.cc:1403] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.043s	user 0.008s	sys 0.011s
W20260709 12:16:13.841928 24204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:42247, user_credentials={real_user=slave}} blocked reactor thread for 44469.4us
I20260709 12:16:13.854552 24846 raft_consensus.cc:359] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:13.855700 24846 raft_consensus.cc:740] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:13.859483 24846 consensus_queue.cc:260] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:13.874501 24846 ts_tablet_manager.cc:1434] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.086s	user 0.010s	sys 0.001s
I20260709 12:16:13.875804 24846 tablet_bootstrap.cc:492] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:13.940570 24846 tablet_bootstrap.cc:492] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:13.941682 24846 tablet_bootstrap.cc:492] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:13.942699 24846 ts_tablet_manager.cc:1403] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.067s	user 0.024s	sys 0.007s
I20260709 12:16:13.946491 24846 raft_consensus.cc:359] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:13.947640 24846 raft_consensus.cc:740] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:13.948671 24846 consensus_queue.cc:260] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:13.951221 24846 ts_tablet_manager.cc:1434] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:16:13.952131 24846 tablet_bootstrap.cc:492] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:13.975103 24846 tablet_bootstrap.cc:492] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:13.976231 24846 tablet_bootstrap.cc:492] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:13.980913 24846 ts_tablet_manager.cc:1403] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.011s
I20260709 12:16:13.986271 24846 raft_consensus.cc:359] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:13.987195 24846 raft_consensus.cc:740] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:13.988077 24846 consensus_queue.cc:260] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:13.990396 24846 ts_tablet_manager.cc:1434] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:16:13.991447 24846 tablet_bootstrap.cc:492] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:14.013276 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.129:42669
I20260709 12:16:14.015005 24979 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.129:42669 every 8 connection(s)
I20260709 12:16:14.018731 24846 tablet_bootstrap.cc:492] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:14.020164 24846 tablet_bootstrap.cc:492] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:14.020931 24846 ts_tablet_manager.cc:1403] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.030s	user 0.019s	sys 0.001s
I20260709 12:16:14.025053 24846 raft_consensus.cc:359] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:14.025833 24846 raft_consensus.cc:740] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.028184 24846 consensus_queue.cc:260] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:14.031040 24846 ts_tablet_manager.cc:1434] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:16:14.032085 24846 tablet_bootstrap.cc:492] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:14.070559 24947 raft_consensus.cc:3060] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:14.074973 24946 raft_consensus.cc:3060] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:14.082252 24944 raft_consensus.cc:3060] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:14.130752 24846 tablet_bootstrap.cc:492] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:14.131923 24846 tablet_bootstrap.cc:492] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:14.132690 24846 ts_tablet_manager.cc:1403] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.101s	user 0.022s	sys 0.030s
I20260709 12:16:14.136178 24945 raft_consensus.cc:3060] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:14.135950 24846 raft_consensus.cc:359] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:14.137133 24846 raft_consensus.cc:740] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.138320 24846 consensus_queue.cc:260] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:14.141000 24846 ts_tablet_manager.cc:1434] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.003s
I20260709 12:16:14.142187 24846 tablet_bootstrap.cc:492] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:14.146554 24982 heartbeater.cc:344] Connected to a master server at 127.11.13.190:46705
I20260709 12:16:14.147245 24982 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:14.158056 24982 heartbeater.cc:507] Master 127.11.13.190:46705 requested a full tablet report, sending...
I20260709 12:16:14.195966 24942 raft_consensus.cc:3060] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:16:14.234690 24130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:42669, user_credentials={real_user=slave}} blocked reactor thread for 62579.7us
I20260709 12:16:14.246240 24846 tablet_bootstrap.cc:492] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:14.249217 24846 tablet_bootstrap.cc:492] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:14.250105 24846 ts_tablet_manager.cc:1403] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.108s	user 0.035s	sys 0.022s
I20260709 12:16:14.253533 24846 raft_consensus.cc:359] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:14.254297 24846 raft_consensus.cc:740] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.255220 24846 consensus_queue.cc:260] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:14.257659 24846 ts_tablet_manager.cc:1434] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:14.258702 24846 tablet_bootstrap.cc:492] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
W20260709 12:16:14.311451 24130 consensus_peers.cc:597] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:14.325081 24846 tablet_bootstrap.cc:492] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:14.326067 24846 tablet_bootstrap.cc:492] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:14.327145 24846 ts_tablet_manager.cc:1403] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.069s	user 0.020s	sys 0.015s
I20260709 12:16:14.330971 24846 raft_consensus.cc:359] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:14.331719 24846 raft_consensus.cc:740] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.332448 24846 consensus_queue.cc:260] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:14.332823 23911 ts_manager.cc:194] Re-registered known tserver with Master: c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:14.334619 24947 raft_consensus.cc:3060] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:14.345070 24846 ts_tablet_manager.cc:1434] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.018s	user 0.006s	sys 0.000s
I20260709 12:16:14.353156 24846 tablet_bootstrap.cc:492] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
W20260709 12:16:14.367122 24130 consensus_peers.cc:597] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 6: this message will repeat every 5th retry.
I20260709 12:16:14.397327 24846 tablet_bootstrap.cc:492] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:14.398608 24846 tablet_bootstrap.cc:492] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:14.399367 24846 ts_tablet_manager.cc:1403] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.046s	user 0.011s	sys 0.008s
I20260709 12:16:14.403172 24846 raft_consensus.cc:359] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:14.404006 24846 raft_consensus.cc:740] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.404958 24846 consensus_queue.cc:260] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:14.407379 24846 ts_tablet_manager.cc:1434] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.002s
I20260709 12:16:14.408370 24846 tablet_bootstrap.cc:492] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:14.418004 23911 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:46188
I20260709 12:16:14.424789 24982 heartbeater.cc:499] Master 127.11.13.190:46705 was elected leader, sending a full tablet report...
I20260709 12:16:14.434270 24947 raft_consensus.cc:3060] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:14.472688 24846 tablet_bootstrap.cc:492] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:14.473865 24846 tablet_bootstrap.cc:492] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:14.474699 24846 ts_tablet_manager.cc:1403] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.067s	user 0.025s	sys 0.016s
I20260709 12:16:14.477960 24846 raft_consensus.cc:359] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:14.478758 24846 raft_consensus.cc:740] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.479660 24846 consensus_queue.cc:260] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:14.481914 24846 ts_tablet_manager.cc:1434] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:16:14.482880 24846 tablet_bootstrap.cc:492] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:14.486397 24942 raft_consensus.cc:3060] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:14.518510 24846 tablet_bootstrap.cc:492] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:14.519711 24846 tablet_bootstrap.cc:492] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:14.520440 24846 ts_tablet_manager.cc:1403] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.038s	user 0.024s	sys 0.005s
I20260709 12:16:14.526316 24846 raft_consensus.cc:359] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:14.529304 24846 raft_consensus.cc:740] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
W20260709 12:16:14.537955 24204 consensus_peers.cc:597] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:14.540747 24846 consensus_queue.cc:260] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:14.545249 24846 ts_tablet_manager.cc:1434] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.024s	user 0.012s	sys 0.000s
I20260709 12:16:14.547848 24846 tablet_bootstrap.cc:492] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:14.606782 24846 tablet_bootstrap.cc:492] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:14.607777 24846 tablet_bootstrap.cc:492] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:14.608451 24846 ts_tablet_manager.cc:1403] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.061s	user 0.023s	sys 0.016s
I20260709 12:16:14.612301 24846 raft_consensus.cc:359] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:14.612988 24846 raft_consensus.cc:740] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.613647 24846 consensus_queue.cc:260] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:14.621613 24846 ts_tablet_manager.cc:1434] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.013s	user 0.007s	sys 0.001s
I20260709 12:16:14.622637 24846 tablet_bootstrap.cc:492] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
W20260709 12:16:14.631156 24130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:42247, user_credentials={real_user=slave}} blocked reactor thread for 46296.3us
I20260709 12:16:14.639950 24846 tablet_bootstrap.cc:492] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:14.641593 24846 tablet_bootstrap.cc:492] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:14.642719 24846 ts_tablet_manager.cc:1403] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.020s	user 0.019s	sys 0.000s
I20260709 12:16:14.646552 24846 raft_consensus.cc:359] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:14.647748 24846 raft_consensus.cc:740] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.648638 24846 consensus_queue.cc:260] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:14.653235 24846 ts_tablet_manager.cc:1434] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.000s
I20260709 12:16:14.654371 24846 tablet_bootstrap.cc:492] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:14.699919 24942 raft_consensus.cc:3060] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:16:14.701645 24130 consensus_peers.cc:597] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:14.706215 24846 tablet_bootstrap.cc:492] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:14.707273 24846 tablet_bootstrap.cc:492] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:14.707959 24846 ts_tablet_manager.cc:1403] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.054s	user 0.017s	sys 0.009s
I20260709 12:16:14.713558 24846 raft_consensus.cc:359] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:14.714385 24846 raft_consensus.cc:740] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.715138 24846 consensus_queue.cc:260] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:14.716773 24846 ts_tablet_manager.cc:1434] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.009s	user 0.005s	sys 0.000s
I20260709 12:16:14.717567 24846 tablet_bootstrap.cc:492] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:14.728690 24944 raft_consensus.cc:3060] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:16:14.761685 24846 tablet_bootstrap.cc:492] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:14.763038 24846 tablet_bootstrap.cc:492] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:14.763903 24846 ts_tablet_manager.cc:1403] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.047s	user 0.019s	sys 0.003s
I20260709 12:16:14.767156 24846 raft_consensus.cc:359] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:14.768268 24846 raft_consensus.cc:740] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.769269 24846 consensus_queue.cc:260] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:14.772390 24846 ts_tablet_manager.cc:1434] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:16:14.777837 24846 tablet_bootstrap.cc:492] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:14.805825 24846 tablet_bootstrap.cc:492] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:14.807067 24846 tablet_bootstrap.cc:492] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:14.807900 24846 ts_tablet_manager.cc:1403] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.030s	user 0.012s	sys 0.008s
I20260709 12:16:14.810881 24846 raft_consensus.cc:359] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:14.812789 24846 raft_consensus.cc:740] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.813771 24846 consensus_queue.cc:260] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:14.816838 24846 ts_tablet_manager.cc:1434] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:16:14.818982 24846 tablet_bootstrap.cc:492] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:14.837301 24846 tablet_bootstrap.cc:492] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:14.838881 24846 tablet_bootstrap.cc:492] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:14.839814 24846 ts_tablet_manager.cc:1403] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.021s	user 0.015s	sys 0.002s
I20260709 12:16:14.843246 24846 raft_consensus.cc:359] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:14.844616 24846 raft_consensus.cc:740] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.845503 24846 consensus_queue.cc:260] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:14.848335 24846 ts_tablet_manager.cc:1434] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.000s
I20260709 12:16:14.850142 24846 tablet_bootstrap.cc:492] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:14.872100 24846 tablet_bootstrap.cc:492] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:14.873167 24846 tablet_bootstrap.cc:492] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:14.873919 24846 ts_tablet_manager.cc:1403] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.007s
I20260709 12:16:14.881469 24846 raft_consensus.cc:359] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:14.882310 24846 raft_consensus.cc:740] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.883121 24846 consensus_queue.cc:260] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:14.886113 24846 ts_tablet_manager.cc:1434] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.000s
I20260709 12:16:14.887117 24846 tablet_bootstrap.cc:492] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:14.922168 24846 tablet_bootstrap.cc:492] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:14.923158 24846 tablet_bootstrap.cc:492] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:14.923913 24846 ts_tablet_manager.cc:1403] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.037s	user 0.019s	sys 0.001s
I20260709 12:16:14.926676 24846 raft_consensus.cc:359] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:14.927379 24846 raft_consensus.cc:740] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.928092 24846 consensus_queue.cc:260] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:14.930030 24846 ts_tablet_manager.cc:1434] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:16:14.931022 24846 tablet_bootstrap.cc:492] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:14.933372 24942 raft_consensus.cc:3060] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:14.960785 24846 tablet_bootstrap.cc:492] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:14.962045 24846 tablet_bootstrap.cc:492] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:14.962858 24846 ts_tablet_manager.cc:1403] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.032s	user 0.021s	sys 0.009s
I20260709 12:16:14.967116 24846 raft_consensus.cc:359] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:14.968180 24846 raft_consensus.cc:740] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:14.972162 24846 consensus_queue.cc:260] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:14.974839 24846 ts_tablet_manager.cc:1434] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.001s
I20260709 12:16:15.005975 24846 tablet_bootstrap.cc:492] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
W20260709 12:16:15.015393 24204 consensus_peers.cc:597] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:15.090230 24846 tablet_bootstrap.cc:492] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:15.091385 24846 tablet_bootstrap.cc:492] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:15.092860 24846 ts_tablet_manager.cc:1403] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.087s	user 0.019s	sys 0.001s
W20260709 12:16:15.100704 24056 consensus_peers.cc:597] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:15.095711 24846 raft_consensus.cc:359] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:15.103785 24846 raft_consensus.cc:740] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.104768 24846 consensus_queue.cc:260] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:15.113270 24846 ts_tablet_manager.cc:1434] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.020s	user 0.005s	sys 0.000s
I20260709 12:16:15.114691 24846 tablet_bootstrap.cc:492] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:15.141656 24846 tablet_bootstrap.cc:492] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:15.142864 24846 tablet_bootstrap.cc:492] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:15.143680 24846 ts_tablet_manager.cc:1403] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.029s	user 0.017s	sys 0.004s
I20260709 12:16:15.147394 24846 raft_consensus.cc:359] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:15.148335 24846 raft_consensus.cc:740] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.149142 24846 consensus_queue.cc:260] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:15.151613 24846 ts_tablet_manager.cc:1434] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:15.152568 24846 tablet_bootstrap.cc:492] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:15.172698 24846 tablet_bootstrap.cc:492] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:15.173811 24846 tablet_bootstrap.cc:492] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:15.174674 24846 ts_tablet_manager.cc:1403] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.022s	user 0.020s	sys 0.000s
I20260709 12:16:15.178041 24846 raft_consensus.cc:359] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:15.178829 24846 raft_consensus.cc:740] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.179620 24846 consensus_queue.cc:260] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:15.182041 24846 ts_tablet_manager.cc:1434] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
I20260709 12:16:15.183224 24846 tablet_bootstrap.cc:492] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:15.205156 24846 tablet_bootstrap.cc:492] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:15.206255 24846 tablet_bootstrap.cc:492] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:15.207120 24846 ts_tablet_manager.cc:1403] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.024s	user 0.013s	sys 0.007s
W20260709 12:16:15.212334 24130 consensus_peers.cc:597] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:15.214673 24846 raft_consensus.cc:359] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:15.215605 24846 raft_consensus.cc:740] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.216423 24846 consensus_queue.cc:260] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:15.218992 24846 ts_tablet_manager.cc:1434] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:16:15.219934 24846 tablet_bootstrap.cc:492] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:15.243323 24846 tablet_bootstrap.cc:492] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:15.244510 24846 tablet_bootstrap.cc:492] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:15.245410 24846 ts_tablet_manager.cc:1403] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.026s	user 0.015s	sys 0.005s
I20260709 12:16:15.248450 24846 raft_consensus.cc:359] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:15.249262 24846 raft_consensus.cc:740] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.249995 24846 consensus_queue.cc:260] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:15.252630 24846 ts_tablet_manager.cc:1434] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.003s
I20260709 12:16:15.253584 24846 tablet_bootstrap.cc:492] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:15.272814 24846 tablet_bootstrap.cc:492] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:15.273705 24846 tablet_bootstrap.cc:492] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:15.274509 24846 ts_tablet_manager.cc:1403] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.021s	user 0.017s	sys 0.003s
I20260709 12:16:15.277208 24846 raft_consensus.cc:359] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:15.277956 24846 raft_consensus.cc:740] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.278738 24846 consensus_queue.cc:260] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:15.281850 24846 ts_tablet_manager.cc:1434] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.001s
I20260709 12:16:15.282877 24846 tablet_bootstrap.cc:492] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:15.320493 24846 tablet_bootstrap.cc:492] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:15.321977 24846 tablet_bootstrap.cc:492] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:15.323383 24846 ts_tablet_manager.cc:1403] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.041s	user 0.023s	sys 0.005s
I20260709 12:16:15.326397 24846 raft_consensus.cc:359] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:15.327178 24846 raft_consensus.cc:740] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.327922 24846 consensus_queue.cc:260] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:15.330801 24846 ts_tablet_manager.cc:1434] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:15.348392 24846 tablet_bootstrap.cc:492] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:15.377100 24846 tablet_bootstrap.cc:492] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:15.378201 24846 tablet_bootstrap.cc:492] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:15.379336 24846 ts_tablet_manager.cc:1403] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.031s	user 0.012s	sys 0.008s
I20260709 12:16:15.382432 24846 raft_consensus.cc:359] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:15.383248 24846 raft_consensus.cc:740] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.384091 24846 consensus_queue.cc:260] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:15.386730 24846 ts_tablet_manager.cc:1434] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:16:15.387699 24846 tablet_bootstrap.cc:492] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:15.780463 24846 tablet_bootstrap.cc:492] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:15.820444 24846 tablet_bootstrap.cc:492] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:15.821076 24846 ts_tablet_manager.cc:1403] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.434s	user 0.055s	sys 0.068s
I20260709 12:16:15.829147 24938 raft_consensus.cc:3060] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:16:15.843497 24132 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:41011, user_credentials={real_user=slave}} blocked reactor thread for 53331.8us
I20260709 12:16:15.860500 24846 raft_consensus.cc:359] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
W20260709 12:16:15.861099 24056 consensus_peers.cc:597] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:15.877789 24056 consensus_peers.cc:597] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:15.879530 24846 raft_consensus.cc:740] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:15.881810 24846 consensus_queue.cc:260] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:15.889309 24846 ts_tablet_manager.cc:1434] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.068s	user 0.008s	sys 0.000s
I20260709 12:16:15.896414 24846 tablet_bootstrap.cc:492] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
W20260709 12:16:15.887143 24130 consensus_peers.cc:597] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:15.994695 24846 tablet_bootstrap.cc:492] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:15.996239 24846 tablet_bootstrap.cc:492] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:15.996975 24846 ts_tablet_manager.cc:1403] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.101s	user 0.017s	sys 0.000s
I20260709 12:16:16.000101 24846 raft_consensus.cc:359] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:16.001303 24846 raft_consensus.cc:740] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.002048 24846 consensus_queue.cc:260] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:16.003925 24846 ts_tablet_manager.cc:1434] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:16:16.004829 24846 tablet_bootstrap.cc:492] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:16.022198 24846 tablet_bootstrap.cc:492] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:16.026666 24846 tablet_bootstrap.cc:492] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:16.027475 24846 ts_tablet_manager.cc:1403] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.023s	user 0.017s	sys 0.001s
I20260709 12:16:16.030483 24846 raft_consensus.cc:359] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:16.031213 24846 raft_consensus.cc:740] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.031977 24846 consensus_queue.cc:260] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:16.034564 24846 ts_tablet_manager.cc:1434] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:16:16.035689 24846 tablet_bootstrap.cc:492] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:16.068197 24846 tablet_bootstrap.cc:492] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:16.069790 24846 tablet_bootstrap.cc:492] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:16.070816 24846 ts_tablet_manager.cc:1403] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.035s	user 0.024s	sys 0.007s
I20260709 12:16:16.073923 24846 raft_consensus.cc:359] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:16.074923 24846 raft_consensus.cc:740] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.075758 24846 consensus_queue.cc:260] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:16.078331 24846 ts_tablet_manager.cc:1434] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.007s	user 0.008s	sys 0.000s
I20260709 12:16:16.079668 24846 tablet_bootstrap.cc:492] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:16.099498 24846 tablet_bootstrap.cc:492] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:16.100943 24846 tablet_bootstrap.cc:492] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:16.101884 24846 ts_tablet_manager.cc:1403] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.005s
I20260709 12:16:16.105584 24846 raft_consensus.cc:359] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:16.106796 24846 raft_consensus.cc:740] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.107623 24846 consensus_queue.cc:260] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:16.110405 24846 ts_tablet_manager.cc:1434] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.001s
I20260709 12:16:16.111462 24846 tablet_bootstrap.cc:492] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:16.145601 24846 tablet_bootstrap.cc:492] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:16.151397 24846 tablet_bootstrap.cc:492] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:16.152709 24846 ts_tablet_manager.cc:1403] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.041s	user 0.024s	sys 0.001s
I20260709 12:16:16.160822 24846 raft_consensus.cc:359] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:16.161940 24846 raft_consensus.cc:740] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.162918 24846 consensus_queue.cc:260] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:16.176157 24846 ts_tablet_manager.cc:1434] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.018s	user 0.005s	sys 0.000s
I20260709 12:16:16.177291 24846 tablet_bootstrap.cc:492] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802: Bootstrap starting.
I20260709 12:16:16.214208 24846 tablet_bootstrap.cc:492] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:16.215516 24846 tablet_bootstrap.cc:492] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802: Bootstrap complete.
I20260709 12:16:16.216336 24846 ts_tablet_manager.cc:1403] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802: Time spent bootstrapping tablet: real 0.039s	user 0.018s	sys 0.004s
I20260709 12:16:16.222064 24846 raft_consensus.cc:359] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:16.234565 24846 raft_consensus.cc:740] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c359614416104a7ba9db1ede97d13802, State: Initialized, Role: FOLLOWER
I20260709 12:16:16.235646 24846 consensus_queue.cc:260] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:16.253433 24846 ts_tablet_manager.cc:1434] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802: Time spent starting tablet: real 0.034s	user 0.004s	sys 0.003s
W20260709 12:16:21.258016 24130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:42247, user_credentials={real_user=slave}} blocked reactor thread for 175344us
W20260709 12:16:21.457852 24132 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.132:41011, user_credentials={real_user=slave}} blocked reactor thread for 57232.2us
W20260709 12:16:22.399091 24262 debug-util.cc:398] Leaking SignalData structure 0x7b0800170080 after lost signal to thread 11321
W20260709 12:16:22.401515 24262 debug-util.cc:398] Leaking SignalData structure 0x7b08000b6e40 after lost signal to thread 23950
W20260709 12:16:22.404639 24262 debug-util.cc:398] Leaking SignalData structure 0x7b08001a6a60 after lost signal to thread 24117
W20260709 12:16:22.407382 24262 debug-util.cc:398] Leaking SignalData structure 0x7b0800223a80 after lost signal to thread 24191
W20260709 12:16:22.410420 24262 debug-util.cc:398] Leaking SignalData structure 0x7b08001bea00 after lost signal to thread 24265
W20260709 12:16:22.413641 24262 debug-util.cc:398] Leaking SignalData structure 0x7b08002120c0 after lost signal to thread 24979
I20260709 12:16:24.220897 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:24.293643 24162 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "21f0988e53da475c8274bcb13b34b3a8"
dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
mode: GRACEFUL
new_leader_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:57632
I20260709 12:16:24.294680 24162 raft_consensus.cc:606] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Received request to transfer leadership to TS c359614416104a7ba9db1ede97d13802
I20260709 12:16:24.297529 24162 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2bd0baecffd2454894539252387f4f2d"
dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
mode: GRACEFUL
new_leader_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:57632
I20260709 12:16:24.298207 24162 raft_consensus.cc:606] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Received request to transfer leadership to TS c359614416104a7ba9db1ede97d13802
I20260709 12:16:24.302701 24162 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "531e8a2f98124f13910b3a8ce4d279c1"
dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
mode: GRACEFUL
new_leader_uuid: "a0df0eff47b64e3eb45ab074139ed187"
 from {username='slave'} at 127.0.0.1:57632
I20260709 12:16:24.303547 24162 raft_consensus.cc:606] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Received request to transfer leadership to TS a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:24.310513 24166 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7e815126e7554106a097f95846d7f663"
dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
mode: GRACEFUL
new_leader_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:57632
I20260709 12:16:24.311195 24166 raft_consensus.cc:606] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Received request to transfer leadership to TS c359614416104a7ba9db1ede97d13802
I20260709 12:16:24.336103 24093 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3effea5c11740418bd8b7bedd33e0f5"
dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
mode: GRACEFUL
new_leader_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:48346
I20260709 12:16:24.336686 24093 raft_consensus.cc:606] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Received request to transfer leadership to TS c359614416104a7ba9db1ede97d13802
I20260709 12:16:24.358421 24238 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "be5ea3171ad74d74bede5875414d92ac"
dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
mode: GRACEFUL
new_leader_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:52536
I20260709 12:16:24.359102 24238 raft_consensus.cc:606] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Received request to transfer leadership to TS c359614416104a7ba9db1ede97d13802
I20260709 12:16:24.362378 24166 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4c368e2cc6c4869a85a53ae022a0785"
dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
mode: GRACEFUL
new_leader_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:57632
I20260709 12:16:24.362931 24166 raft_consensus.cc:606] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Received request to transfer leadership to TS c359614416104a7ba9db1ede97d13802
I20260709 12:16:24.364362 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 12:16:24.367067 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:24.368209 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:24.431476 25101 raft_consensus.cc:993] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b: : Instructing follower c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:24.444387 25090 raft_consensus.cc:1081] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Signalling peer c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:24.484367 24947 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3effea5c11740418bd8b7bedd33e0f5"
dest_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:34790
I20260709 12:16:24.485634 24947 raft_consensus.cc:493] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:24.486222 24947 raft_consensus.cc:3060] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:24.494570 25073 raft_consensus.cc:993] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327: : Instructing follower a0df0eff47b64e3eb45ab074139ed187 to start an election
I20260709 12:16:24.495674 25073 raft_consensus.cc:1081] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Signalling peer a0df0eff47b64e3eb45ab074139ed187 to start an election
I20260709 12:16:24.498548 24238 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "531e8a2f98124f13910b3a8ce4d279c1"
dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
 from {username='slave'} at 127.0.0.1:48496
I20260709 12:16:24.499358 24238 raft_consensus.cc:493] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:24.499825 24238 raft_consensus.cc:3060] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:24.520269 25076 raft_consensus.cc:993] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187: : Instructing follower c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:24.524326 25094 raft_consensus.cc:1081] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Signalling peer c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:24.527895 24938 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "be5ea3171ad74d74bede5875414d92ac"
dest_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:34776
I20260709 12:16:24.528587 24938 raft_consensus.cc:493] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:24.528930 24938 raft_consensus.cc:3060] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:24.541232 24947 raft_consensus.cc:515] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:24.551797 24238 raft_consensus.cc:515] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:24.556819 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3effea5c11740418bd8b7bedd33e0f5" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:24.557541 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "531e8a2f98124f13910b3a8ce4d279c1" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:24.557832 24093 raft_consensus.cc:3055] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:24.555954 24947 leader_election.cc:290] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:24.558871 24162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3effea5c11740418bd8b7bedd33e0f5" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:24.559510 24162 raft_consensus.cc:3060] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:24.558228 24084 raft_consensus.cc:3060] T 531e8a2f98124f13910b3a8ce4d279c1 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:24.558243 24093 raft_consensus.cc:740] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Running, Role: LEADER
I20260709 12:16:24.570166 24093 consensus_queue.cc:260] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:24.572057 24093 raft_consensus.cc:3060] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:24.576869 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "531e8a2f98124f13910b3a8ce4d279c1" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:24.577785 24166 raft_consensus.cc:3055] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:24.578111 24166 raft_consensus.cc:740] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:24.579258 24238 leader_election.cc:290] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:24.579531 24166 consensus_queue.cc:260] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:24.560688 25107 raft_consensus.cc:993] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327: : Instructing follower c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:24.583473 25108 raft_consensus.cc:1081] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Signalling peer c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:24.587133 25096 raft_consensus.cc:993] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327: : Instructing follower c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:24.588685 25096 raft_consensus.cc:1081] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Signalling peer c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:24.591682 24936 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7e815126e7554106a097f95846d7f663"
dest_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:34764
I20260709 12:16:24.592285 24936 raft_consensus.cc:493] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:24.592675 24936 raft_consensus.cc:3060] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:24.593600 24945 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2bd0baecffd2454894539252387f4f2d"
dest_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:34764
I20260709 12:16:24.594364 24945 raft_consensus.cc:493] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:24.594760 24945 raft_consensus.cc:3060] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:24.581876 24166 raft_consensus.cc:3060] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:24.601624 24936 raft_consensus.cc:515] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:24.608783 24162 raft_consensus.cc:2468] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 2.
I20260709 12:16:24.614254 24936 leader_election.cc:290] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 3 election: Requested vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:24.615195 24161 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e815126e7554106a097f95846d7f663" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:24.653157 24161 raft_consensus.cc:3055] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:24.653558 24161 raft_consensus.cc:740] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:24.654361 24840 leader_election.cc:304] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:24.654477 24161 consensus_queue.cc:260] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:24.655828 24161 raft_consensus.cc:3060] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:24.624553 24084 raft_consensus.cc:2468] T 531e8a2f98124f13910b3a8ce4d279c1 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 2.
I20260709 12:16:24.612895 24945 raft_consensus.cc:515] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:24.656934 25109 raft_consensus.cc:2804] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:24.659118 24945 leader_election.cc:290] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 3 election: Requested vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:24.663242 24204 leader_election.cc:304] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:24.664654 25094 raft_consensus.cc:2804] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:24.670056 24241 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd0baecffd2454894539252387f4f2d" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:24.670456 24234 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7e815126e7554106a097f95846d7f663" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:24.671052 24241 raft_consensus.cc:3060] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:24.671264 24234 raft_consensus.cc:3060] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:24.675024 24161 raft_consensus.cc:2468] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 3.
I20260709 12:16:24.676170 25108 raft_consensus.cc:993] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327: : Instructing follower c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:24.676959 25108 raft_consensus.cc:1081] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Signalling peer c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:24.680629 24936 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "21f0988e53da475c8274bcb13b34b3a8"
dest_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:34764
I20260709 12:16:24.681298 24936 raft_consensus.cc:493] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:24.681734 24936 raft_consensus.cc:3060] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:24.684629 25094 raft_consensus.cc:697] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 [term 2 LEADER]: Becoming Leader. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Running, Role: LEADER
I20260709 12:16:24.685766 25094 consensus_queue.cc:237] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } }
I20260709 12:16:24.695179 25109 raft_consensus.cc:697] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [term 2 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:16:24.696617 25109 consensus_queue.cc:237] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:24.645169 24938 raft_consensus.cc:515] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:24.698658 24936 raft_consensus.cc:515] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:24.700176 24938 leader_election.cc:290] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 election: Requested vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:24.635236 24093 raft_consensus.cc:2468] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 2.
I20260709 12:16:24.700897 24936 leader_election.cc:290] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 election: Requested vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
W20260709 12:16:24.700924 24840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:36699, user_credentials={real_user=slave}} blocked reactor thread for 46820.4us
I20260709 12:16:24.703776 24840 leader_election.cc:304] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:24.705392 24162 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bd0baecffd2454894539252387f4f2d" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:24.705574 24238 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be5ea3171ad74d74bede5875414d92ac" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:24.705737 24163 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "be5ea3171ad74d74bede5875414d92ac" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:24.706175 24238 raft_consensus.cc:3055] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:24.706476 24163 raft_consensus.cc:3060] T be5ea3171ad74d74bede5875414d92ac P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:24.706576 24238 raft_consensus.cc:740] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Running, Role: LEADER
I20260709 12:16:24.706677 24237 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21f0988e53da475c8274bcb13b34b3a8" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:24.707474 24237 raft_consensus.cc:3060] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:24.707774 24238 consensus_queue.cc:260] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:24.644313 24166 raft_consensus.cc:2468] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 2.
I20260709 12:16:24.709236 24238 raft_consensus.cc:3060] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:24.710705 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21f0988e53da475c8274bcb13b34b3a8" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:24.711165 24162 raft_consensus.cc:3055] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:24.711390 24165 raft_consensus.cc:3055] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:24.711609 24162 raft_consensus.cc:740] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:24.712464 25110 raft_consensus.cc:2804] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:24.711866 24165 raft_consensus.cc:740] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:24.714689 24165 consensus_queue.cc:260] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:24.716328 24165 raft_consensus.cc:3060] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:24.724431 24241 raft_consensus.cc:2468] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 3.
I20260709 12:16:24.722072 24162 consensus_queue.cc:260] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:24.735919 24163 raft_consensus.cc:2468] T be5ea3171ad74d74bede5875414d92ac P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 2.
I20260709 12:16:24.737366 24840 leader_election.cc:304] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:24.737888 24162 raft_consensus.cc:3060] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:24.743285 24165 raft_consensus.cc:2468] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 2.
I20260709 12:16:24.745466 25113 raft_consensus.cc:2804] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:24.745589 25110 raft_consensus.cc:697] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 3 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:16:24.756193 25110 consensus_queue.cc:237] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:24.747128 24840 leader_election.cc:304] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:24.761740 25114 raft_consensus.cc:2804] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:24.766037 24162 raft_consensus.cc:2468] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 3.
I20260709 12:16:24.767422 24840 leader_election.cc:304] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:24.697943 24234 raft_consensus.cc:2468] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 3.
I20260709 12:16:24.785225 23913 catalog_manager.cc:5697] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 reported cstate change: term changed from 1 to 2, leader changed from 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131) to a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132). New cstate: current_term: 2 leader_uuid: "a0df0eff47b64e3eb45ab074139ed187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:24.767375 24238 raft_consensus.cc:2468] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 2.
I20260709 12:16:24.811293 24237 raft_consensus.cc:2468] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 2.
I20260709 12:16:24.785878 25117 raft_consensus.cc:2804] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 3 FOLLOWER]: Leader election won for term 3
W20260709 12:16:24.813189 24840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.131:36699, user_credentials={real_user=slave}} blocked reactor thread for 45890.7us
I20260709 12:16:24.813437 25117 raft_consensus.cc:697] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 3 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:16:24.799736 25113 raft_consensus.cc:697] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [term 2 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:16:24.816751 25113 consensus_queue.cc:237] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:24.820137 25114 raft_consensus.cc:697] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [term 2 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:16:24.814606 25117 consensus_queue.cc:237] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
W20260709 12:16:24.879488 24204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:42247, user_credentials={real_user=slave}} blocked reactor thread for 67613.9us
I20260709 12:16:24.879223 25114 consensus_queue.cc:237] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
W20260709 12:16:25.157452 24204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:42247, user_credentials={real_user=slave}} blocked reactor thread for 228874us
I20260709 12:16:25.168164 24162 raft_consensus.cc:1275] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 3 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:25.172006 25109 consensus_queue.cc:1048] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:25.435102 24234 raft_consensus.cc:1275] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:25.442430 24241 raft_consensus.cc:1275] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 3 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:25.466737 24166 raft_consensus.cc:1275] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Refusing update from remote peer a0df0eff47b64e3eb45ab074139ed187: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:25.476373 25127 consensus_queue.cc:1048] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:25.486501 24235 raft_consensus.cc:1275] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 3 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:25.502086 24162 raft_consensus.cc:1275] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 3 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:25.509631 25117 consensus_queue.cc:1048] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:25.548676 24236 raft_consensus.cc:1275] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:25.571377 24167 raft_consensus.cc:1275] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:25.581115 24165 raft_consensus.cc:1275] T be5ea3171ad74d74bede5875414d92ac P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:25.586337 24163 raft_consensus.cc:1275] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:25.591998 25113 consensus_queue.cc:1048] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:25.592887 24085 raft_consensus.cc:1275] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:25.599924 25114 consensus_queue.cc:1048] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:25.600664 25109 consensus_queue.cc:1048] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:25.602231 25109 consensus_queue.cc:1048] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:25.604842 23915 catalog_manager.cc:5697] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 1 to 2, leader changed from 6c74a5203b594708921cd793aaca0b8b (127.11.13.130) to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:25.608223 25113 consensus_queue.cc:1048] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:25.610666 25117 consensus_queue.cc:1048] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:25.619436 25113 consensus_queue.cc:1048] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:25.627641 24085 raft_consensus.cc:1275] T 531e8a2f98124f13910b3a8ce4d279c1 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Refusing update from remote peer a0df0eff47b64e3eb45ab074139ed187: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:25.629603 25105 consensus_queue.cc:1048] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:25.601923 25114 consensus_queue.cc:1048] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
I20260709 12:16:25.637681 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:25.625276 23913 catalog_manager.cc:5697] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 2 to 3, leader changed from 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131) to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 3 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } }
I20260709 12:16:25.651978 23913 catalog_manager.cc:5697] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 2 to 3, leader changed from 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131) to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 3 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } }
I20260709 12:16:25.673563 23911 catalog_manager.cc:5697] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 reported cstate change: term changed from 1 to 2, leader changed from a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132) to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } }
I20260709 12:16:25.826313 23911 catalog_manager.cc:5697] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 1 to 2, leader changed from 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131) to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:25.850841 24165 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "11a4ed5e3c5d47f189b17261d317eb7b"
dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
mode: GRACEFUL
new_leader_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:57632
I20260709 12:16:25.851617 24165 raft_consensus.cc:606] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Received request to transfer leadership to TS c359614416104a7ba9db1ede97d13802
W20260709 12:16:25.853853 24130 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:42669, user_credentials={real_user=slave}} blocked reactor thread for 75709.6us
I20260709 12:16:25.860900 24165 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2bd0baecffd2454894539252387f4f2d"
dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
mode: GRACEFUL
new_leader_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:57632
I20260709 12:16:25.861646 24165 raft_consensus.cc:606] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 3 FOLLOWER]: Received request to transfer leadership to TS c359614416104a7ba9db1ede97d13802
I20260709 12:16:25.863279 24165 raft_consensus.cc:614] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260709 12:16:25.866298 24093 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "56ed2939d1454de181106eec39d0e4bc"
dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
mode: GRACEFUL
new_leader_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:48346
I20260709 12:16:25.866962 24093 raft_consensus.cc:606] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Received request to transfer leadership to TS c359614416104a7ba9db1ede97d13802
I20260709 12:16:25.868919 25128 raft_consensus.cc:993] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327: : Instructing follower c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:25.869510 25128 raft_consensus.cc:1081] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Signalling peer c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:25.872078 24165 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "659e684fa2a0433faf027bf99c82dd6b"
dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
mode: GRACEFUL
new_leader_uuid: "a0df0eff47b64e3eb45ab074139ed187"
 from {username='slave'} at 127.0.0.1:57632
I20260709 12:16:25.872848 24165 raft_consensus.cc:606] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Received request to transfer leadership to TS a0df0eff47b64e3eb45ab074139ed187
I20260709 12:16:25.882762 24943 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "11a4ed5e3c5d47f189b17261d317eb7b"
dest_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:34764
I20260709 12:16:25.883646 24943 raft_consensus.cc:493] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:25.884127 24943 raft_consensus.cc:3060] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:25.891053 24236 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ab48aa491dc84193a39fdbf1a1c52927"
dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
mode: GRACEFUL
new_leader_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:52536
I20260709 12:16:25.891856 24236 raft_consensus.cc:606] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Received request to transfer leadership to TS c359614416104a7ba9db1ede97d13802
I20260709 12:16:25.901343 24943 raft_consensus.cc:515] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:25.902454 24165 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf791a63ce7546f38a312c7f77142a61"
dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
mode: GRACEFUL
new_leader_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:57632
I20260709 12:16:25.903220 24165 raft_consensus.cc:606] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Received request to transfer leadership to TS c359614416104a7ba9db1ede97d13802
I20260709 12:16:25.904742 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11a4ed5e3c5d47f189b17261d317eb7b" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:25.905368 24084 raft_consensus.cc:3060] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:25.908305 24166 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f4c368e2cc6c4869a85a53ae022a0785"
dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
mode: GRACEFUL
new_leader_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:57632
I20260709 12:16:25.908929 24166 raft_consensus.cc:606] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Received request to transfer leadership to TS c359614416104a7ba9db1ede97d13802
I20260709 12:16:25.910691 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260709 12:16:25.911976 24943 leader_election.cc:290] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:25.913470 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11a4ed5e3c5d47f189b17261d317eb7b" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:25.914254 24165 raft_consensus.cc:3055] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:25.914686 24165 raft_consensus.cc:740] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:25.915781 24165 consensus_queue.cc:260] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:25.916420 24084 raft_consensus.cc:2468] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 2.
I20260709 12:16:25.917117 24165 raft_consensus.cc:3060] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:25.918634 24840 leader_election.cc:304] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:25.920121 25114 raft_consensus.cc:2804] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:25.920943 25114 raft_consensus.cc:697] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [term 2 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:16:25.922179 25114 consensus_queue.cc:237] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:25.926920 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:25.927856 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:25.933851 24165 raft_consensus.cc:2468] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 2.
I20260709 12:16:25.967134 25126 raft_consensus.cc:993] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327: : Instructing follower a0df0eff47b64e3eb45ab074139ed187 to start an election
I20260709 12:16:25.967674 25126 raft_consensus.cc:1081] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Signalling peer a0df0eff47b64e3eb45ab074139ed187 to start an election
I20260709 12:16:25.970682 24235 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "659e684fa2a0433faf027bf99c82dd6b"
dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
 from {username='slave'} at 127.0.0.1:48496
I20260709 12:16:25.971303 24235 raft_consensus.cc:493] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:25.971652 24235 raft_consensus.cc:3060] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:25.999236 25096 raft_consensus.cc:993] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327: : Instructing follower c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:25.999826 25126 raft_consensus.cc:1081] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Signalling peer c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:26.002190 24942 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cf791a63ce7546f38a312c7f77142a61"
dest_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:34764
I20260709 12:16:26.002818 24942 raft_consensus.cc:493] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:26.003280 24942 raft_consensus.cc:3060] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:26.011494 25094 raft_consensus.cc:993] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187: : Instructing follower c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:26.013193 25094 raft_consensus.cc:1081] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Signalling peer c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:26.021881 24938 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ab48aa491dc84193a39fdbf1a1c52927"
dest_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:34776
I20260709 12:16:26.022665 24938 raft_consensus.cc:493] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:26.023061 24938 raft_consensus.cc:3060] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:26.017012 23913 catalog_manager.cc:5697] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 1 to 2, leader changed from 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131) to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:26.082161 24938 raft_consensus.cc:515] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:26.085098 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab48aa491dc84193a39fdbf1a1c52927" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:26.085768 24165 raft_consensus.cc:3060] T ab48aa491dc84193a39fdbf1a1c52927 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:26.088843 24235 raft_consensus.cc:515] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:26.106585 25090 raft_consensus.cc:993] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b: : Instructing follower c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:26.138219 25090 raft_consensus.cc:1081] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Signalling peer c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:26.138473 24942 raft_consensus.cc:515] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:26.140908 24943 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "56ed2939d1454de181106eec39d0e4bc"
dest_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:34790
I20260709 12:16:26.141695 24943 raft_consensus.cc:493] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:26.142485 24943 raft_consensus.cc:3060] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:26.142616 24084 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf791a63ce7546f38a312c7f77142a61" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:26.143303 24084 raft_consensus.cc:3060] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:26.127595 24165 raft_consensus.cc:2468] T ab48aa491dc84193a39fdbf1a1c52927 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 2.
I20260709 12:16:26.152488 24840 leader_election.cc:304] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:26.153514 25117 raft_consensus.cc:2804] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:26.154011 25117 raft_consensus.cc:697] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 2 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:16:26.154958 25117 consensus_queue.cc:237] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:26.131492 24236 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ab48aa491dc84193a39fdbf1a1c52927" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:26.164291 24236 raft_consensus.cc:3055] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:26.164683 24236 raft_consensus.cc:740] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Running, Role: LEADER
I20260709 12:16:26.165060 24235 leader_election.cc:290] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 election: Requested vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:26.165673 24236 consensus_queue.cc:260] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:26.167394 24085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659e684fa2a0433faf027bf99c82dd6b" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:26.168301 24085 raft_consensus.cc:3060] T 659e684fa2a0433faf027bf99c82dd6b P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:26.163188 24943 raft_consensus.cc:515] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:26.175863 24943 leader_election.cc:290] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:26.177409 24239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ed2939d1454de181106eec39d0e4bc" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:26.178139 24239 raft_consensus.cc:3060] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:26.178748 24087 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "56ed2939d1454de181106eec39d0e4bc" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:26.179373 24087 raft_consensus.cc:3055] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:26.179697 24087 raft_consensus.cc:740] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Running, Role: LEADER
I20260709 12:16:26.180547 24087 consensus_queue.cc:260] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:26.181914 24087 raft_consensus.cc:3060] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:26.120123 24938 leader_election.cc:290] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 election: Requested vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:26.183666 24236 raft_consensus.cc:3060] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:26.177304 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659e684fa2a0433faf027bf99c82dd6b" candidate_uuid: "a0df0eff47b64e3eb45ab074139ed187" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:26.184873 24165 raft_consensus.cc:3055] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:26.188102 24165 raft_consensus.cc:740] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:26.194491 24084 raft_consensus.cc:2468] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 3.
I20260709 12:16:26.200724 24165 consensus_queue.cc:260] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:26.204180 24087 raft_consensus.cc:2468] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 2.
I20260709 12:16:26.203016 24085 raft_consensus.cc:2468] T 659e684fa2a0433faf027bf99c82dd6b P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 2.
I20260709 12:16:26.205693 24204 leader_election.cc:304] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:26.206622 25076 raft_consensus.cc:2804] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:26.207172 25076 raft_consensus.cc:697] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 2 LEADER]: Becoming Leader. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Running, Role: LEADER
I20260709 12:16:26.208205 25076 consensus_queue.cc:237] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:26.208359 24840 leader_election.cc:304] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:26.211259 25113 raft_consensus.cc:2804] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:26.206554 24165 raft_consensus.cc:3060] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:26.215907 25126 raft_consensus.cc:993] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327: : Instructing follower c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:26.216046 24239 raft_consensus.cc:2468] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 2.
I20260709 12:16:26.216553 25126 raft_consensus.cc:1081] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Signalling peer c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:26.212589 25113 raft_consensus.cc:697] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [term 2 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:16:26.217949 25113 consensus_queue.cc:237] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:26.218673 24943 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f4c368e2cc6c4869a85a53ae022a0785"
dest_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:34764
I20260709 12:16:26.219596 24943 raft_consensus.cc:493] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:26.219641 24840 leader_election.cc:304] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:26.220337 24943 raft_consensus.cc:3060] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:26.220783 25114 raft_consensus.cc:2804] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:26.221338 25114 raft_consensus.cc:697] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 3 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:16:26.222368 25114 consensus_queue.cc:237] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:26.228163 24942 leader_election.cc:290] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 3 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:26.231139 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf791a63ce7546f38a312c7f77142a61" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:26.231791 24166 raft_consensus.cc:3055] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:26.232200 24166 raft_consensus.cc:740] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:26.233165 24166 consensus_queue.cc:260] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:26.233781 24165 raft_consensus.cc:2468] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0df0eff47b64e3eb45ab074139ed187 in term 2.
I20260709 12:16:26.236752 24166 raft_consensus.cc:3060] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Advancing to term 3
W20260709 12:16:26.241189 24114 debug-util.cc:398] Leaking SignalData structure 0x7b080024a7e0 after lost signal to thread 11321
W20260709 12:16:26.242995 24114 debug-util.cc:398] Leaking SignalData structure 0x7b08000a7ac0 after lost signal to thread 23950
W20260709 12:16:26.244686 24114 debug-util.cc:398] Leaking SignalData structure 0x7b08000fe4a0 after lost signal to thread 24117
W20260709 12:16:26.246328 24114 debug-util.cc:398] Leaking SignalData structure 0x7b08003f5b80 after lost signal to thread 24191
W20260709 12:16:26.247757 24114 debug-util.cc:398] Leaking SignalData structure 0x7b08000cdfe0 after lost signal to thread 24265
I20260709 12:16:26.257309 23913 catalog_manager.cc:5697] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 reported cstate change: term changed from 1 to 2, leader changed from 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131) to a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132). New cstate: current_term: 2 leader_uuid: "a0df0eff47b64e3eb45ab074139ed187" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:26.264521 24236 raft_consensus.cc:2468] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 2.
W20260709 12:16:26.268170 24114 debug-util.cc:398] Leaking SignalData structure 0x7b080030f220 after lost signal to thread 24979
I20260709 12:16:26.233026 24943 raft_consensus.cc:515] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:26.276057 24943 leader_election.cc:290] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 election: Requested vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:26.277428 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4c368e2cc6c4869a85a53ae022a0785" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:26.279842 24239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f4c368e2cc6c4869a85a53ae022a0785" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:26.281040 24239 raft_consensus.cc:3060] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:26.281828 24166 raft_consensus.cc:2468] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 3.
I20260709 12:16:26.279124 24165 raft_consensus.cc:3055] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:26.286985 24165 raft_consensus.cc:740] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:26.293399 24165 consensus_queue.cc:260] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:26.295353 24239 raft_consensus.cc:2468] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 2.
I20260709 12:16:26.299110 24165 raft_consensus.cc:3060] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:26.297506 23913 catalog_manager.cc:5697] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 1 to 2, leader changed from a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132) to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:26.306890 24842 leader_election.cc:304] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:26.307869 25117 raft_consensus.cc:2804] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:26.309520 24165 raft_consensus.cc:2468] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 2.
I20260709 12:16:26.313428 25117 raft_consensus.cc:697] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [term 2 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:16:26.315657 25117 consensus_queue.cc:237] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:26.355569 23913 catalog_manager.cc:5697] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 1 to 2, leader changed from 6c74a5203b594708921cd793aaca0b8b (127.11.13.130) to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:26.357723 23913 catalog_manager.cc:5697] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 1 to 2, leader changed from 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131) to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:26.359418 23913 catalog_manager.cc:5697] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 2 to 3, leader changed from 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131) to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 3 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:26.475085 24084 raft_consensus.cc:1275] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:26.478757 25117 consensus_queue.cc:1048] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:26.516645 24165 raft_consensus.cc:1275] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:26.522720 25113 consensus_queue.cc:1048] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:26.543864 24167 raft_consensus.cc:1275] T ab48aa491dc84193a39fdbf1a1c52927 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:26.552160 25113 consensus_queue.cc:1048] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:26.574074 24236 raft_consensus.cc:1275] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:26.577003 25114 consensus_queue.cc:1048] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:26.616130 24084 raft_consensus.cc:1275] T 659e684fa2a0433faf027bf99c82dd6b P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Refusing update from remote peer a0df0eff47b64e3eb45ab074139ed187: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:26.622864 25076 consensus_queue.cc:1048] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:26.722865 24084 raft_consensus.cc:1275] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:26.725690 25114 consensus_queue.cc:1048] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:26.746449 24166 raft_consensus.cc:1275] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:26.748086 25117 consensus_queue.cc:1048] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:26.749753 24239 raft_consensus.cc:1275] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:26.753439 25117 consensus_queue.cc:1048] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
W20260709 12:16:26.783396 24204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:42247, user_credentials={real_user=slave}} blocked reactor thread for 55814.3us
I20260709 12:16:26.819623 24165 raft_consensus.cc:1275] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Refusing update from remote peer a0df0eff47b64e3eb45ab074139ed187: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:26.841044 25076 consensus_queue.cc:1048] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:26.899259 24167 raft_consensus.cc:1275] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 3 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
W20260709 12:16:26.937716 25174 log.cc:927] Time spent T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327: Append to log took a long time: real 0.068s	user 0.000s	sys 0.001s
I20260709 12:16:26.956337 25168 consensus_queue.cc:1048] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:27.003054 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:16:27.011993 24204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:42247, user_credentials={real_user=slave}} blocked reactor thread for 132785us
I20260709 12:16:27.071023 24235 raft_consensus.cc:1275] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:27.073148 25118 consensus_queue.cc:1048] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260709 12:16:27.081609 24840 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:36699, user_credentials={real_user=slave}} blocked reactor thread for 127877us
I20260709 12:16:27.133940 24084 raft_consensus.cc:1275] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 3 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:27.193957 25118 consensus_queue.cc:1048] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:27.331872 24235 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "659e684fa2a0433faf027bf99c82dd6b"
dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
mode: GRACEFUL
new_leader_uuid: "008a79e516614c809c2d3afdb5cbc327"
 from {username='slave'} at 127.0.0.1:52536
I20260709 12:16:27.332584 24235 raft_consensus.cc:606] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 2 LEADER]: Received request to transfer leadership to TS 008a79e516614c809c2d3afdb5cbc327
I20260709 12:16:27.342965 24165 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b4638bcb9174442184bc8d8a27d45072"
dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
mode: GRACEFUL
new_leader_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:57632
I20260709 12:16:27.344311 24165 raft_consensus.cc:606] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Received request to transfer leadership to TS c359614416104a7ba9db1ede97d13802
I20260709 12:16:27.352834 24085 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d7366aa6480d4d1e909db5717506432b"
dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
mode: GRACEFUL
new_leader_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:48346
I20260709 12:16:27.353678 24085 raft_consensus.cc:606] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Received request to transfer leadership to TS c359614416104a7ba9db1ede97d13802
I20260709 12:16:27.363916 24165 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ddbc768ef2c54c3391f0558cc562a01d"
dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
mode: GRACEFUL
new_leader_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:57632
I20260709 12:16:27.364961 24165 raft_consensus.cc:606] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Received request to transfer leadership to TS c359614416104a7ba9db1ede97d13802
I20260709 12:16:27.366959 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260709 12:16:27.369153 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:27.370205 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:27.380209 25172 raft_consensus.cc:993] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187: : Instructing follower 008a79e516614c809c2d3afdb5cbc327 to start an election
I20260709 12:16:27.380970 25182 raft_consensus.cc:1081] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 2 LEADER]: Signalling peer 008a79e516614c809c2d3afdb5cbc327 to start an election
I20260709 12:16:27.385946 24165 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "659e684fa2a0433faf027bf99c82dd6b"
dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
 from {username='slave'} at 127.0.0.1:35436
I20260709 12:16:27.387045 24165 raft_consensus.cc:493] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:27.387544 24165 raft_consensus.cc:3060] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:27.435438 24165 raft_consensus.cc:515] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:27.439651 24235 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659e684fa2a0433faf027bf99c82dd6b" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:27.440419 24235 raft_consensus.cc:3055] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:27.440795 24235 raft_consensus.cc:740] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a0df0eff47b64e3eb45ab074139ed187, State: Running, Role: LEADER
I20260709 12:16:27.441809 24235 consensus_queue.cc:260] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:27.443301 24235 raft_consensus.cc:3060] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:27.445197 24085 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "659e684fa2a0433faf027bf99c82dd6b" candidate_uuid: "008a79e516614c809c2d3afdb5cbc327" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:27.445969 24085 raft_consensus.cc:3060] T 659e684fa2a0433faf027bf99c82dd6b P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:27.441723 24165 leader_election.cc:290] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 3 election: Requested vote from peers a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
I20260709 12:16:27.454440 24235 raft_consensus.cc:2468] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 3.
I20260709 12:16:27.462532 24132 leader_election.cc:304] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, a0df0eff47b64e3eb45ab074139ed187; no voters: 
I20260709 12:16:27.464561 25126 raft_consensus.cc:2804] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:27.465140 25126 raft_consensus.cc:697] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 3 LEADER]: Becoming Leader. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:27.466159 25126 consensus_queue.cc:237] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:27.487627 23913 catalog_manager.cc:5697] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 reported cstate change: term changed from 2 to 3, leader changed from a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132) to 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131). New cstate: current_term: 3 leader_uuid: "008a79e516614c809c2d3afdb5cbc327" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:27.552650 25180 raft_consensus.cc:993] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b: : Instructing follower c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:27.553236 25180 raft_consensus.cc:1081] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Signalling peer c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:27.556659 24943 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d7366aa6480d4d1e909db5717506432b"
dest_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:34790
I20260709 12:16:27.557538 24943 raft_consensus.cc:493] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:27.558053 24943 raft_consensus.cc:3060] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:27.580403 24085 raft_consensus.cc:2468] T 659e684fa2a0433faf027bf99c82dd6b P 6c74a5203b594708921cd793aaca0b8b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 008a79e516614c809c2d3afdb5cbc327 in term 3.
I20260709 12:16:27.617508 24943 raft_consensus.cc:515] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:27.624624 24943 leader_election.cc:290] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:27.625156 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7366aa6480d4d1e909db5717506432b" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:27.625792 24165 raft_consensus.cc:3060] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:27.626564 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d7366aa6480d4d1e909db5717506432b" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:27.627278 24093 raft_consensus.cc:3055] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:27.627599 24093 raft_consensus.cc:740] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 6c74a5203b594708921cd793aaca0b8b, State: Running, Role: LEADER
I20260709 12:16:27.628484 24093 consensus_queue.cc:260] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:27.629913 24093 raft_consensus.cc:3060] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:27.633986 24165 raft_consensus.cc:2468] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 2.
I20260709 12:16:27.649601 24840 leader_election.cc:304] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:27.650715 25185 raft_consensus.cc:2804] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:27.651274 25185 raft_consensus.cc:697] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [term 2 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:16:27.652237 25185 consensus_queue.cc:237] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:27.677949 24093 raft_consensus.cc:2468] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 2.
I20260709 12:16:27.686755 25096 raft_consensus.cc:993] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327: : Instructing follower c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:27.687435 25084 raft_consensus.cc:1081] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Signalling peer c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:27.689486 24943 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b4638bcb9174442184bc8d8a27d45072"
dest_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:34764
I20260709 12:16:27.690105 24943 raft_consensus.cc:493] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:27.690659 24943 raft_consensus.cc:3060] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:27.707780 23914 catalog_manager.cc:5697] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 1 to 2, leader changed from 6c74a5203b594708921cd793aaca0b8b (127.11.13.130) to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:27.720890 24943 raft_consensus.cc:515] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:27.728693 25126 raft_consensus.cc:993] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327: : Instructing follower c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:27.731757 25126 raft_consensus.cc:1081] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Signalling peer c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:27.734370 24942 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ddbc768ef2c54c3391f0558cc562a01d"
dest_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:34764
I20260709 12:16:27.734956 24942 raft_consensus.cc:493] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:27.735304 24942 raft_consensus.cc:3060] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:27.736400 24943 leader_election.cc:290] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 election: Requested vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:27.737545 24239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4638bcb9174442184bc8d8a27d45072" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:27.738188 24239 raft_consensus.cc:3060] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:27.738430 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b4638bcb9174442184bc8d8a27d45072" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:27.739019 24165 raft_consensus.cc:3055] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:27.739300 24165 raft_consensus.cc:740] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:27.740188 24165 consensus_queue.cc:260] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:27.741520 24165 raft_consensus.cc:3060] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:27.752354 24165 raft_consensus.cc:2468] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 2.
I20260709 12:16:27.753542 24239 raft_consensus.cc:2468] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 2.
I20260709 12:16:27.754982 24840 leader_election.cc:304] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:27.769655 25185 raft_consensus.cc:2804] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:27.770352 25185 raft_consensus.cc:697] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [term 2 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:16:27.771766 25185 consensus_queue.cc:237] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:27.764350 24942 raft_consensus.cc:515] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:27.807595 24165 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbc768ef2c54c3391f0558cc562a01d" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:27.808535 24165 raft_consensus.cc:3055] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:27.808882 24165 raft_consensus.cc:740] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:27.809996 24165 consensus_queue.cc:260] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:27.811390 24165 raft_consensus.cc:3060] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:27.822026 24165 raft_consensus.cc:2468] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 2.
I20260709 12:16:27.822145 23914 catalog_manager.cc:5697] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 1 to 2, leader changed from 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131) to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:27.823629 24840 leader_election.cc:304] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:27.825124 25185 raft_consensus.cc:2804] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:27.825678 25185 raft_consensus.cc:697] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [term 2 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:16:27.826864 25185 consensus_queue.cc:237] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:27.827724 24239 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ddbc768ef2c54c3391f0558cc562a01d" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a0df0eff47b64e3eb45ab074139ed187"
I20260709 12:16:27.828526 24239 raft_consensus.cc:3060] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:27.835656 24942 leader_election.cc:290] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 2 election: Requested vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), a0df0eff47b64e3eb45ab074139ed187 (127.11.13.132:41011)
I20260709 12:16:27.845570 24239 raft_consensus.cc:2468] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 2.
I20260709 12:16:27.862387 23914 catalog_manager.cc:5697] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 1 to 2, leader changed from 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131) to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:27.904947 24239 raft_consensus.cc:1275] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 3 FOLLOWER]: Refusing update from remote peer 008a79e516614c809c2d3afdb5cbc327: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:27.907285 25096 consensus_queue.cc:1048] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:27.948341 24093 raft_consensus.cc:1275] T 659e684fa2a0433faf027bf99c82dd6b P 6c74a5203b594708921cd793aaca0b8b [term 3 FOLLOWER]: Refusing update from remote peer 008a79e516614c809c2d3afdb5cbc327: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:27.950840 25126 consensus_queue.cc:1048] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:28.082486 24165 raft_consensus.cc:1275] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:28.097851 25117 consensus_queue.cc:1048] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:28.118971 24084 raft_consensus.cc:1275] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:28.125916 25203 consensus_queue.cc:1048] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:28.255950 24166 raft_consensus.cc:1275] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:28.319926 25185 consensus_queue.cc:1048] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:28.381137 24163 raft_consensus.cc:1275] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:28.387383 25117 consensus_queue.cc:1048] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:28.410557 24241 raft_consensus.cc:1275] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:16:28.414256 24204 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:36699, user_credentials={real_user=slave}} blocked reactor thread for 62899us
I20260709 12:16:28.420595 24235 raft_consensus.cc:1275] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:28.422650 25203 consensus_queue.cc:1048] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260709 12:16:28.456061 25185 consensus_queue.cc:1048] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:28.633023 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:28.688323 24163 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "5587d307c2f64ab9a9b39c1b190e85be"
dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
mode: GRACEFUL
new_leader_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:57632
I20260709 12:16:28.688942 24163 raft_consensus.cc:606] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Received request to transfer leadership to TS c359614416104a7ba9db1ede97d13802
I20260709 12:16:28.690388 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260709 12:16:28.692370 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:28.693204 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:29.237020 25096 raft_consensus.cc:993] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327: : Instructing follower c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:29.237900 25096 raft_consensus.cc:1081] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Signalling peer c359614416104a7ba9db1ede97d13802 to start an election
I20260709 12:16:29.240690 24943 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "5587d307c2f64ab9a9b39c1b190e85be"
dest_uuid: "c359614416104a7ba9db1ede97d13802"
 from {username='slave'} at 127.0.0.1:34764
I20260709 12:16:29.241343 24943 raft_consensus.cc:493] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:29.241743 24943 raft_consensus.cc:3060] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:29.252592 24943 raft_consensus.cc:515] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:29.256810 24093 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5587d307c2f64ab9a9b39c1b190e85be" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "6c74a5203b594708921cd793aaca0b8b"
I20260709 12:16:29.257540 24093 raft_consensus.cc:3060] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:29.257722 24166 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "5587d307c2f64ab9a9b39c1b190e85be" candidate_uuid: "c359614416104a7ba9db1ede97d13802" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "008a79e516614c809c2d3afdb5cbc327"
I20260709 12:16:29.258426 24166 raft_consensus.cc:3055] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:29.258730 24166 raft_consensus.cc:740] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 008a79e516614c809c2d3afdb5cbc327, State: Running, Role: LEADER
I20260709 12:16:29.259560 24166 consensus_queue.cc:260] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:29.260738 24166 raft_consensus.cc:3060] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:29.276695 24166 raft_consensus.cc:2468] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 3.
I20260709 12:16:29.277237 24943 leader_election.cc:290] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 3 election: Requested vote from peers 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247), 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699)
I20260709 12:16:29.283496 24840 leader_election.cc:304] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802; no voters: 
I20260709 12:16:29.284512 25210 raft_consensus.cc:2804] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:29.285962 25210 raft_consensus.cc:697] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 3 LEADER]: Becoming Leader. State: Replica: c359614416104a7ba9db1ede97d13802, State: Running, Role: LEADER
I20260709 12:16:29.287231 24093 raft_consensus.cc:2468] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c359614416104a7ba9db1ede97d13802 in term 3.
I20260709 12:16:29.287140 25210 consensus_queue.cc:237] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } }
I20260709 12:16:29.309294 23914 catalog_manager.cc:5697] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 reported cstate change: term changed from 2 to 3, leader changed from 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131) to c359614416104a7ba9db1ede97d13802 (127.11.13.129). New cstate: current_term: 3 leader_uuid: "c359614416104a7ba9db1ede97d13802" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:29.707311 24084 raft_consensus.cc:1275] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [term 3 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:29.709388 25210 consensus_queue.cc:1048] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:29.714099 24165 raft_consensus.cc:1275] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 3 FOLLOWER]: Refusing update from remote peer c359614416104a7ba9db1ede97d13802: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:29.730276 25185 consensus_queue.cc:1048] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [LEADER]: Connected to new peer: Peer: permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:29.753701 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:29.870729 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:29.873204 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:29.874327 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:29.947374 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:30.008047 11318 tablet_server.cc:179] TabletServer@127.11.13.129:42669 shutting down...
W20260709 12:16:30.040206 24130 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 342 similar messages]
W20260709 12:16:30.050693 24130 consensus_peers.cc:597] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.055069 24204 consensus_peers.cc:597] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.076026 24130 consensus_peers.cc:597] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.103029 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:30.105470 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.106698 11318 raft_consensus.cc:2243] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.107404 11318 raft_consensus.cc:2272] T 7e76d3d2239249bb806deb5ad8232c1c P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.111464 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.112960 11318 raft_consensus.cc:2243] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.113564 11318 raft_consensus.cc:2272] T 1df16f5650964ece90109902b6b40882 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.116958 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.118033 11318 raft_consensus.cc:2243] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.118705 11318 raft_consensus.cc:2272] T 56fb07e9eb574b1fbc5da95063610dd5 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:30.122028 24056 consensus_peers.cc:597] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.122984 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.124171 11318 raft_consensus.cc:2243] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.125432 11318 raft_consensus.cc:2272] T 21f0988e53da475c8274bcb13b34b3a8 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.129228 11318 tablet_replica.cc:333] stopping tablet replica
W20260709 12:16:30.129498 24056 consensus_peers.cc:597] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.130486 11318 raft_consensus.cc:2243] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.131382 11318 raft_consensus.cc:2272] T 2147e5dc12974e6c852f8961496066ae P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:30.134615 24204 consensus_peers.cc:597] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.135196 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.136325 11318 raft_consensus.cc:2243] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.136871 11318 raft_consensus.cc:2272] T bc53eac4ce984dbaa561d9b719143d18 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.140555 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.141626 11318 raft_consensus.cc:2243] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.142284 11318 raft_consensus.cc:2272] T e2eabb81feec483ea9b303c5e516b7cf P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.145838 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.146943 11318 raft_consensus.cc:2243] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.148128 11318 raft_consensus.cc:2272] T ddbc768ef2c54c3391f0558cc562a01d P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:30.149320 24130 consensus_peers.cc:597] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.153208 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.154392 11318 raft_consensus.cc:2243] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.156405 11318 raft_consensus.cc:2272] T be5ea3171ad74d74bede5875414d92ac P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.161460 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.162616 11318 raft_consensus.cc:2243] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.163956 11318 raft_consensus.cc:2272] T b3effea5c11740418bd8b7bedd33e0f5 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.168664 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.170154 11318 raft_consensus.cc:2243] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.171849 11318 raft_consensus.cc:2272] T ab48aa491dc84193a39fdbf1a1c52927 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:30.173122 24056 consensus_peers.cc:597] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.175486 24056 consensus_peers.cc:597] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.176431 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.177860 11318 raft_consensus.cc:2243] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.179117 11318 raft_consensus.cc:2272] T 0756db96a05a45bc9ebc1f91ee907496 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:30.180297 24056 consensus_peers.cc:597] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.182996 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.184109 11318 raft_consensus.cc:2243] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.185559 11318 raft_consensus.cc:2272] T 11a4ed5e3c5d47f189b17261d317eb7b P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.189013 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.189967 11318 raft_consensus.cc:2243] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.190591 11318 raft_consensus.cc:2272] T 131c7392ac614d87ab3d9b6713a46f7b P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.194274 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.195801 11318 raft_consensus.cc:2243] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.197350 11318 raft_consensus.cc:2272] T f4c368e2cc6c4869a85a53ae022a0785 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.201519 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.202936 11318 raft_consensus.cc:2243] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.204299 11318 raft_consensus.cc:2272] T b4638bcb9174442184bc8d8a27d45072 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.208097 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.209358 11318 raft_consensus.cc:2243] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.210289 11318 raft_consensus.cc:2272] T 57c7696ee73146b59bd8b17640333077 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:30.213336 24204 consensus_peers.cc:597] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.214442 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.217206 11318 raft_consensus.cc:2243] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.217964 11318 raft_consensus.cc:2272] T 77079c31ca114c49a916e49cfd8faf2c P c359614416104a7ba9db1ede97d13802 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:30.218473 24204 consensus_peers.cc:597] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.221289 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.222385 11318 raft_consensus.cc:2243] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.222972 11318 raft_consensus.cc:2272] T b498d1ba220843a5939d152e0382bff7 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.226821 11318 tablet_replica.cc:333] stopping tablet replica
W20260709 12:16:30.226853 24130 consensus_peers.cc:597] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.227954 11318 raft_consensus.cc:2243] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.228615 11318 raft_consensus.cc:2272] T 41c505fd92654536b72dbd8a3f1bc728 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.230991 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.231683 11318 raft_consensus.cc:2243] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.232148 11318 raft_consensus.cc:2272] T 4c91f626cde148838a6a979d4fd1a5ed P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.235170 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.236303 11318 raft_consensus.cc:2243] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.237128 11318 raft_consensus.cc:2272] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:30.239580 24056 consensus_peers.cc:597] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.241066 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.241889 11318 raft_consensus.cc:2243] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.242398 11318 raft_consensus.cc:2272] T 576c4469eb9c49dbbba40a54832fa33f P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.245916 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.246867 11318 raft_consensus.cc:2243] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.247526 11318 raft_consensus.cc:2272] T 96039384602f4057b875e1d873138d47 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.251715 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.252485 11318 raft_consensus.cc:2243] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.253456 11318 raft_consensus.cc:2272] T d7366aa6480d4d1e909db5717506432b P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.256675 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.257404 11318 raft_consensus.cc:2243] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.258270 11318 raft_consensus.cc:2272] T ceb6c32822cf44158ca1ea0d62e0b001 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.262248 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.263195 11318 raft_consensus.cc:2243] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.264340 11318 raft_consensus.cc:2272] T cf791a63ce7546f38a312c7f77142a61 P c359614416104a7ba9db1ede97d13802 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.267422 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.268244 11318 raft_consensus.cc:2243] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.268790 11318 raft_consensus.cc:2272] T ba257bd0f16a4a7da0cb4a7953ca1869 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.272514 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.273339 11318 raft_consensus.cc:2243] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.273953 11318 raft_consensus.cc:2272] T 55819e09f3e245179f0dd80b9e60eb6a P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.277361 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.278105 11318 raft_consensus.cc:2243] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.278697 11318 raft_consensus.cc:2272] T 809a23266b1646e0be710ddab5fff291 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:30.280961 24130 consensus_peers.cc:597] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.281761 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.282637 11318 raft_consensus.cc:2243] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.283201 11318 raft_consensus.cc:2272] T 0015560e665c48f0bd17fe4be5c8761d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.285753 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.286517 11318 raft_consensus.cc:2243] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.287786 11318 raft_consensus.cc:2272] T 2bd0baecffd2454894539252387f4f2d P c359614416104a7ba9db1ede97d13802 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.290673 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.291359 11318 raft_consensus.cc:2243] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.292577 11318 raft_consensus.cc:2272] T 56ed2939d1454de181106eec39d0e4bc P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.295571 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.296243 11318 raft_consensus.cc:2243] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.296767 11318 raft_consensus.cc:2272] T e35efdb2f1c24db0a45fbc04abea260e P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.299304 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.299854 11318 raft_consensus.cc:2243] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.301038 11318 raft_consensus.cc:2272] T 7e815126e7554106a097f95846d7f663 P c359614416104a7ba9db1ede97d13802 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.304509 11318 tablet_replica.cc:333] stopping tablet replica
W20260709 12:16:30.304486 24130 consensus_peers.cc:597] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.305650 11318 raft_consensus.cc:2243] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:16:30.306394 24056 consensus_peers.cc:597] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.306409 11318 raft_consensus.cc:2272] T 31e3897eed824c5fa96e04ce0eca32a8 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.309259 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.309968 11318 raft_consensus.cc:2243] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.310552 11318 raft_consensus.cc:2272] T 0192ff9d150a47feb68f490efce1e3f2 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.313210 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.313876 11318 raft_consensus.cc:2243] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.314500 11318 raft_consensus.cc:2272] T be4e681eef304bb8bc812e313a67414d P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.317709 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.319015 11318 raft_consensus.cc:2243] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:16:30.319063 24056 consensus_peers.cc:597] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.319972 11318 raft_consensus.cc:2272] T 95824f48a95746289333a9da8e265178 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:30.321672 24130 consensus_peers.cc:597] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.323102 24204 consensus_peers.cc:597] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.323730 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.324544 11318 raft_consensus.cc:2243] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.325071 11318 raft_consensus.cc:2272] T e679d0d8ffd84a23afc6d7f8faf84ca6 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.328805 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.329670 11318 raft_consensus.cc:2243] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.330292 11318 raft_consensus.cc:2272] T 6f15aae25cad42abb701c295e56963e3 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.334962 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.335877 11318 raft_consensus.cc:2243] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.336540 11318 raft_consensus.cc:2272] T 94568d4dbfd440478c83b50b50976be7 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:30.339069 24056 consensus_peers.cc:597] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.341678 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.342751 11318 raft_consensus.cc:2243] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.343482 11318 raft_consensus.cc:2272] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P c359614416104a7ba9db1ede97d13802 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.346199 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.346931 11318 raft_consensus.cc:2243] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.347505 11318 raft_consensus.cc:2272] T 3052da843efd4ef4861a13e2240bb466 P c359614416104a7ba9db1ede97d13802 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.349984 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:30.350582 11318 raft_consensus.cc:2243] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.351475 11318 raft_consensus.cc:2272] T 5587d307c2f64ab9a9b39c1b190e85be P c359614416104a7ba9db1ede97d13802 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:30.361328 24204 consensus_peers.cc:597] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.374074 24056 consensus_peers.cc:597] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.392831 24204 consensus_peers.cc:597] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.417011 24130 consensus_peers.cc:597] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.430649 24204 consensus_peers.cc:597] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.431809 24130 consensus_peers.cc:597] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.436178 24204 consensus_peers.cc:597] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.447942 24843 connection.cc:466] server connection from 127.0.0.1:34776 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:34776 (request call id 373) could send its response
I20260709 12:16:30.469316 11318 tablet_server.cc:196] TabletServer@127.11.13.129:42669 shutdown complete.
W20260709 12:16:30.585367 24056 consensus_peers.cc:597] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.585367 24204 consensus_peers.cc:597] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.587962 11318 tablet_server.cc:179] TabletServer@127.11.13.130:0 shutting down...
W20260709 12:16:30.617635 24204 consensus_peers.cc:597] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.623428 24130 consensus_peers.cc:597] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 -> Peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Couldn't send request to peer c359614416104a7ba9db1ede97d13802. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.654619 24130 consensus_peers.cc:597] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.661620 24204 consensus_peers.cc:597] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.664326 24130 consensus_peers.cc:597] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.665218 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:30.666296 11318 tablet_replica.cc:333] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.667361 11318 raft_consensus.cc:2243] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.668777 11318 raft_consensus.cc:2272] T 2147e5dc12974e6c852f8961496066ae P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.672516 11318 tablet_replica.cc:333] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.673784 11318 raft_consensus.cc:2243] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.674891 11318 raft_consensus.cc:2272] T 93f3b3f298544987a7b9fc3dd0af496c P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.678474 11318 tablet_replica.cc:333] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.679352 11318 raft_consensus.cc:2243] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.679796 11318 raft_consensus.cc:2272] T 96039384602f4057b875e1d873138d47 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.682286 11318 tablet_replica.cc:333] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.683108 11318 raft_consensus.cc:2243] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.684216 11318 raft_consensus.cc:2272] T 6db45947fce14c98b26a8ed1ed9bbce4 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:30.687376 24204 consensus_peers.cc:597] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.687754 11318 tablet_replica.cc:333] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.691171 11318 raft_consensus.cc:2243] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.691932 11318 raft_consensus.cc:2272] T 77079c31ca114c49a916e49cfd8faf2c P 6c74a5203b594708921cd793aaca0b8b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.694649 11318 tablet_replica.cc:333] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.695366 11318 raft_consensus.cc:2243] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.696247 11318 raft_consensus.cc:2272] T 576c4469eb9c49dbbba40a54832fa33f P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.698539 11318 tablet_replica.cc:333] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.699277 11318 raft_consensus.cc:2243] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.700448 11318 raft_consensus.cc:2272] T 56fb07e9eb574b1fbc5da95063610dd5 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.703806 11318 tablet_replica.cc:333] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.704857 11318 raft_consensus.cc:2243] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.705896 11318 raft_consensus.cc:2272] T 131c7392ac614d87ab3d9b6713a46f7b P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:30.706619 24204 consensus_peers.cc:597] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.708838 11318 tablet_replica.cc:333] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.709731 11318 raft_consensus.cc:2243] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.710297 11318 raft_consensus.cc:2272] T 56ed2939d1454de181106eec39d0e4bc P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.713403 11318 tablet_replica.cc:333] T 3bd7c9c11f834f15974a65477ef9258a P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.714357 11318 raft_consensus.cc:2243] T 3bd7c9c11f834f15974a65477ef9258a P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:16:30.718581 24130 consensus_peers.cc:597] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.719376 11318 raft_consensus.cc:2272] T 3bd7c9c11f834f15974a65477ef9258a P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.722675 11318 tablet_replica.cc:333] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.723780 11318 raft_consensus.cc:2243] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.725767 11318 raft_consensus.cc:2272] T 94568d4dbfd440478c83b50b50976be7 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.728811 11318 tablet_replica.cc:333] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.729494 11318 raft_consensus.cc:2243] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.729984 11318 raft_consensus.cc:2272] T 2e4765d7c78c480fa4ba08d2812aeac3 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.732234 11318 tablet_replica.cc:333] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.733031 11318 raft_consensus.cc:2243] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.733855 11318 raft_consensus.cc:2272] T c385253c4ad94570b7dbb6875a755b36 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.736142 11318 tablet_replica.cc:333] T 7e76d3d2239249bb806deb5ad8232c1c P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.736776 11318 raft_consensus.cc:2243] T 7e76d3d2239249bb806deb5ad8232c1c P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.737212 11318 raft_consensus.cc:2272] T 7e76d3d2239249bb806deb5ad8232c1c P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.740996 11318 tablet_replica.cc:333] T b498d1ba220843a5939d152e0382bff7 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.742260 11318 raft_consensus.cc:2243] T b498d1ba220843a5939d152e0382bff7 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.743038 11318 raft_consensus.cc:2272] T b498d1ba220843a5939d152e0382bff7 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.747530 11318 tablet_replica.cc:333] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.748596 11318 raft_consensus.cc:2243] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.749769 11318 raft_consensus.cc:2272] T 4c91f626cde148838a6a979d4fd1a5ed P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.752882 11318 tablet_replica.cc:333] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.753546 11318 raft_consensus.cc:2243] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.754532 11318 raft_consensus.cc:2272] T 41c505fd92654536b72dbd8a3f1bc728 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:30.754565 24130 consensus_peers.cc:597] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.757561 11318 tablet_replica.cc:333] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.758637 11318 raft_consensus.cc:2243] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.759256 11318 raft_consensus.cc:2272] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.762027 11318 tablet_replica.cc:333] T b0805608ea264bef894c413af00a0687 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.762970 11318 raft_consensus.cc:2243] T b0805608ea264bef894c413af00a0687 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.763947 11318 raft_consensus.cc:2272] T b0805608ea264bef894c413af00a0687 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.768173 11318 tablet_replica.cc:333] T 8e3a9c266ee343eb903d5138d1de95d2 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.769114 11318 raft_consensus.cc:2243] T 8e3a9c266ee343eb903d5138d1de95d2 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.769913 11318 raft_consensus.cc:2272] T 8e3a9c266ee343eb903d5138d1de95d2 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.772814 11318 tablet_replica.cc:333] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.773654 11318 raft_consensus.cc:2243] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.774309 11318 raft_consensus.cc:2272] T b3effea5c11740418bd8b7bedd33e0f5 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.777321 11318 tablet_replica.cc:333] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.778371 11318 raft_consensus.cc:2243] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.778967 11318 raft_consensus.cc:2272] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.781623 11318 tablet_replica.cc:333] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.782769 11318 raft_consensus.cc:2243] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.784471 11318 raft_consensus.cc:2272] T 0192ff9d150a47feb68f490efce1e3f2 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.787577 11318 tablet_replica.cc:333] T 1e98aa59558f4e58925a3c87026b6fa4 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.788260 11318 raft_consensus.cc:2243] T 1e98aa59558f4e58925a3c87026b6fa4 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.788723 11318 raft_consensus.cc:2272] T 1e98aa59558f4e58925a3c87026b6fa4 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.791106 11318 tablet_replica.cc:333] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.791690 11318 raft_consensus.cc:2243] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.792534 11318 raft_consensus.cc:2272] T 31e3897eed824c5fa96e04ce0eca32a8 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.794656 11318 tablet_replica.cc:333] T be4e681eef304bb8bc812e313a67414d P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.795229 11318 raft_consensus.cc:2243] T be4e681eef304bb8bc812e313a67414d P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.795663 11318 raft_consensus.cc:2272] T be4e681eef304bb8bc812e313a67414d P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.797540 11318 tablet_replica.cc:333] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.798090 11318 raft_consensus.cc:2243] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.798954 11318 raft_consensus.cc:2272] T e35efdb2f1c24db0a45fbc04abea260e P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.801479 11318 tablet_replica.cc:333] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.802078 11318 raft_consensus.cc:2243] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.802911 11318 raft_consensus.cc:2272] T 95824f48a95746289333a9da8e265178 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.805330 11318 tablet_replica.cc:333] T 802231ce402445309306d2cbffa2f92c P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.805966 11318 raft_consensus.cc:2243] T 802231ce402445309306d2cbffa2f92c P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.806576 11318 raft_consensus.cc:2272] T 802231ce402445309306d2cbffa2f92c P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:30.808861 24130 consensus_peers.cc:597] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.809041 11318 tablet_replica.cc:333] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.809963 11318 raft_consensus.cc:2243] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.810509 11318 raft_consensus.cc:2272] T 5587d307c2f64ab9a9b39c1b190e85be P 6c74a5203b594708921cd793aaca0b8b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.812819 11318 tablet_replica.cc:333] T 3052da843efd4ef4861a13e2240bb466 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.813434 11318 raft_consensus.cc:2243] T 3052da843efd4ef4861a13e2240bb466 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.814070 11318 raft_consensus.cc:2272] T 3052da843efd4ef4861a13e2240bb466 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.817492 11318 tablet_replica.cc:333] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.818303 11318 raft_consensus.cc:2243] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.818918 11318 raft_consensus.cc:2272] T 1df16f5650964ece90109902b6b40882 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:30.820039 24130 consensus_peers.cc:597] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.822822 11318 tablet_replica.cc:333] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.823712 11318 raft_consensus.cc:2243] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.824302 11318 raft_consensus.cc:2272] T 55819e09f3e245179f0dd80b9e60eb6a P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.827521 11318 tablet_replica.cc:333] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.828346 11318 raft_consensus.cc:2243] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.828845 11318 raft_consensus.cc:2272] T d7366aa6480d4d1e909db5717506432b P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.831713 11318 tablet_replica.cc:333] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.832703 11318 raft_consensus.cc:2243] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.833243 11318 raft_consensus.cc:2272] T fbb90520a319469fa1664276964749ea P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.835850 11318 tablet_replica.cc:333] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.836447 11318 raft_consensus.cc:2243] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.836951 11318 raft_consensus.cc:2272] T cf791a63ce7546f38a312c7f77142a61 P 6c74a5203b594708921cd793aaca0b8b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.839107 11318 tablet_replica.cc:333] T 1012da668d874f27a9563ae3f53ce4eb P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.839809 11318 raft_consensus.cc:2243] T 1012da668d874f27a9563ae3f53ce4eb P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.840339 11318 raft_consensus.cc:2272] T 1012da668d874f27a9563ae3f53ce4eb P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.844167 11318 tablet_replica.cc:333] T 531e8a2f98124f13910b3a8ce4d279c1 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.845028 11318 raft_consensus.cc:2243] T 531e8a2f98124f13910b3a8ce4d279c1 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.845707 11318 raft_consensus.cc:2272] T 531e8a2f98124f13910b3a8ce4d279c1 P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.848604 11318 tablet_replica.cc:333] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.849378 11318 raft_consensus.cc:2243] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.850555 11318 raft_consensus.cc:2272] T cd2d7fad08bc4376bb38cf21a6a9b611 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.853557 11318 tablet_replica.cc:333] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
W20260709 12:16:30.853583 24204 consensus_peers.cc:597] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:30.854561 11318 raft_consensus.cc:2243] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.854991 11318 raft_consensus.cc:2272] T 11a4ed5e3c5d47f189b17261d317eb7b P 6c74a5203b594708921cd793aaca0b8b [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.857077 11318 tablet_replica.cc:333] T 659e684fa2a0433faf027bf99c82dd6b P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.857578 11318 raft_consensus.cc:2243] T 659e684fa2a0433faf027bf99c82dd6b P 6c74a5203b594708921cd793aaca0b8b [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.858016 11318 raft_consensus.cc:2272] T 659e684fa2a0433faf027bf99c82dd6b P 6c74a5203b594708921cd793aaca0b8b [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.860818 11318 tablet_replica.cc:333] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.861603 11318 raft_consensus.cc:2243] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:30.862560 11318 raft_consensus.cc:2272] T bc53eac4ce984dbaa561d9b719143d18 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.865564 11318 tablet_replica.cc:333] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.866336 11318 raft_consensus.cc:2243] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.866926 11318 raft_consensus.cc:2272] T 809a23266b1646e0be710ddab5fff291 P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:30.870042 11318 tablet_replica.cc:333] T 0015560e665c48f0bd17fe4be5c8761d P 6c74a5203b594708921cd793aaca0b8b: stopping tablet replica
I20260709 12:16:30.870786 11318 raft_consensus.cc:2243] T 0015560e665c48f0bd17fe4be5c8761d P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:30.871395 11318 raft_consensus.cc:2272] T 0015560e665c48f0bd17fe4be5c8761d P 6c74a5203b594708921cd793aaca0b8b [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:30.913247 24130 consensus_peers.cc:597] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.919955 24204 consensus_peers.cc:597] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.940325 24058 connection.cc:466] server connection from 127.0.0.1:58466 torn down before Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:58466 (request call id 733) could send its response
W20260709 12:16:30.945576 24130 consensus_peers.cc:597] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:42247: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.953001 24204 consensus_peers.cc:597] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:42247: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.955323 24204 consensus_peers.cc:597] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:42247: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.964447 24204 consensus_peers.cc:597] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:42247: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:30.987207 24130 consensus_peers.cc:597] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:42247: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:31.013731 24204 consensus_peers.cc:597] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:42247: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:31.027271 11318 tablet_server.cc:196] TabletServer@127.11.13.130:0 shutdown complete.
W20260709 12:16:31.070606 24204 consensus_peers.cc:597] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:42247: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:31.113759 11318 tablet_server.cc:179] TabletServer@127.11.13.131:0 shutting down...
W20260709 12:16:31.127777 24130 consensus_peers.cc:597] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:42247: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:31.162384 24204 consensus_peers.cc:597] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:42247: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:31.174252 24204 consensus_peers.cc:597] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 -> Peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Couldn't send request to peer 6c74a5203b594708921cd793aaca0b8b. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:42247: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:31.175084 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:31.176100 11318 tablet_replica.cc:333] T 93f3b3f298544987a7b9fc3dd0af496c P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.177162 11318 raft_consensus.cc:2243] T 93f3b3f298544987a7b9fc3dd0af496c P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.177911 11318 raft_consensus.cc:2272] T 93f3b3f298544987a7b9fc3dd0af496c P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.180253 11318 tablet_replica.cc:333] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.180830 11318 raft_consensus.cc:2243] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.181566 11318 raft_consensus.cc:2272] T 6f15aae25cad42abb701c295e56963e3 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.183791 11318 tablet_replica.cc:333] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.184412 11318 raft_consensus.cc:2243] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.185452 11318 raft_consensus.cc:2272] T 96039384602f4057b875e1d873138d47 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.187618 11318 tablet_replica.cc:333] T 6db45947fce14c98b26a8ed1ed9bbce4 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.188215 11318 raft_consensus.cc:2243] T 6db45947fce14c98b26a8ed1ed9bbce4 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.188623 11318 raft_consensus.cc:2272] T 6db45947fce14c98b26a8ed1ed9bbce4 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.190833 11318 tablet_replica.cc:333] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.191401 11318 raft_consensus.cc:2243] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.191774 11318 raft_consensus.cc:2272] T 2bd0baecffd2454894539252387f4f2d P 008a79e516614c809c2d3afdb5cbc327 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.193706 11318 tablet_replica.cc:333] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.194403 11318 raft_consensus.cc:2243] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.194840 11318 raft_consensus.cc:2272] T ddbc768ef2c54c3391f0558cc562a01d P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.196756 11318 tablet_replica.cc:333] T 576c4469eb9c49dbbba40a54832fa33f P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.197348 11318 raft_consensus.cc:2243] T 576c4469eb9c49dbbba40a54832fa33f P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.197942 11318 raft_consensus.cc:2272] T 576c4469eb9c49dbbba40a54832fa33f P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.200964 11318 tablet_replica.cc:333] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.201813 11318 raft_consensus.cc:2243] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.202996 11318 raft_consensus.cc:2272] T e2eabb81feec483ea9b303c5e516b7cf P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.206158 11318 tablet_replica.cc:333] T 2e4765d7c78c480fa4ba08d2812aeac3 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.207070 11318 raft_consensus.cc:2243] T 2e4765d7c78c480fa4ba08d2812aeac3 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.207720 11318 raft_consensus.cc:2272] T 2e4765d7c78c480fa4ba08d2812aeac3 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.210996 11318 tablet_replica.cc:333] T be5ea3171ad74d74bede5875414d92ac P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.211790 11318 raft_consensus.cc:2243] T be5ea3171ad74d74bede5875414d92ac P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.212298 11318 raft_consensus.cc:2272] T be5ea3171ad74d74bede5875414d92ac P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.215061 11318 tablet_replica.cc:333] T c385253c4ad94570b7dbb6875a755b36 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.215698 11318 raft_consensus.cc:2243] T c385253c4ad94570b7dbb6875a755b36 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.216180 11318 raft_consensus.cc:2272] T c385253c4ad94570b7dbb6875a755b36 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.218885 11318 tablet_replica.cc:333] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.219550 11318 raft_consensus.cc:2243] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.220222 11318 raft_consensus.cc:2272] T 0192ff9d150a47feb68f490efce1e3f2 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.223443 11318 tablet_replica.cc:333] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.224189 11318 raft_consensus.cc:2243] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.225101 11318 raft_consensus.cc:2272] T 1e98aa59558f4e58925a3c87026b6fa4 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.227427 11318 tablet_replica.cc:333] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.228034 11318 raft_consensus.cc:2243] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.228526 11318 raft_consensus.cc:2272] T 31e3897eed824c5fa96e04ce0eca32a8 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.230969 11318 tablet_replica.cc:333] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.231573 11318 raft_consensus.cc:2243] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.232051 11318 raft_consensus.cc:2272] T ba257bd0f16a4a7da0cb4a7953ca1869 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.234357 11318 tablet_replica.cc:333] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.235002 11318 raft_consensus.cc:2243] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.235901 11318 raft_consensus.cc:2272] T 7e76d3d2239249bb806deb5ad8232c1c P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.238761 11318 tablet_replica.cc:333] T ab48aa491dc84193a39fdbf1a1c52927 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.239535 11318 raft_consensus.cc:2243] T ab48aa491dc84193a39fdbf1a1c52927 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.240084 11318 raft_consensus.cc:2272] T ab48aa491dc84193a39fdbf1a1c52927 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.243075 11318 tablet_replica.cc:333] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.244107 11318 raft_consensus.cc:2243] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.244831 11318 raft_consensus.cc:2272] T b3effea5c11740418bd8b7bedd33e0f5 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.247673 11318 tablet_replica.cc:333] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.248306 11318 raft_consensus.cc:2243] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.249378 11318 raft_consensus.cc:2272] T dcccd54cb1fa4e9d9cab5afd3e8d4748 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.251878 11318 tablet_replica.cc:333] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.252524 11318 raft_consensus.cc:2243] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.253460 11318 raft_consensus.cc:2272] T b0805608ea264bef894c413af00a0687 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.255847 11318 tablet_replica.cc:333] T 8e3a9c266ee343eb903d5138d1de95d2 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.256510 11318 raft_consensus.cc:2243] T 8e3a9c266ee343eb903d5138d1de95d2 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.257067 11318 raft_consensus.cc:2272] T 8e3a9c266ee343eb903d5138d1de95d2 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.259419 11318 tablet_replica.cc:333] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.260351 11318 raft_consensus.cc:2243] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.260954 11318 raft_consensus.cc:2272] T 21f0988e53da475c8274bcb13b34b3a8 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.263911 11318 tablet_replica.cc:333] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.264590 11318 raft_consensus.cc:2243] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.265489 11318 raft_consensus.cc:2272] T 659e684fa2a0433faf027bf99c82dd6b P 008a79e516614c809c2d3afdb5cbc327 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.268362 11318 tablet_replica.cc:333] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.269277 11318 raft_consensus.cc:2243] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.269950 11318 raft_consensus.cc:2272] T 11a4ed5e3c5d47f189b17261d317eb7b P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.273077 11318 tablet_replica.cc:333] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.273994 11318 raft_consensus.cc:2243] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.275043 11318 raft_consensus.cc:2272] T 0756db96a05a45bc9ebc1f91ee907496 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.278393 11318 tablet_replica.cc:333] T 95824f48a95746289333a9da8e265178 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.279130 11318 raft_consensus.cc:2243] T 95824f48a95746289333a9da8e265178 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.279680 11318 raft_consensus.cc:2272] T 95824f48a95746289333a9da8e265178 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.282780 11318 tablet_replica.cc:333] T e679d0d8ffd84a23afc6d7f8faf84ca6 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.283746 11318 raft_consensus.cc:2243] T e679d0d8ffd84a23afc6d7f8faf84ca6 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.284365 11318 raft_consensus.cc:2272] T e679d0d8ffd84a23afc6d7f8faf84ca6 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.287611 11318 tablet_replica.cc:333] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.288450 11318 raft_consensus.cc:2243] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.289628 11318 raft_consensus.cc:2272] T 802231ce402445309306d2cbffa2f92c P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.292560 11318 tablet_replica.cc:333] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.293316 11318 raft_consensus.cc:2243] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Raft consensus shutting down.
W20260709 12:16:31.293975 24204 consensus_peers.cc:597] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 -> Peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Couldn't send request to peer 008a79e516614c809c2d3afdb5cbc327. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:31.294414 11318 raft_consensus.cc:2272] T ceb6c32822cf44158ca1ea0d62e0b001 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.296788 11318 tablet_replica.cc:333] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.297420 11318 raft_consensus.cc:2243] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.297890 11318 raft_consensus.cc:2272] T cf791a63ce7546f38a312c7f77142a61 P 008a79e516614c809c2d3afdb5cbc327 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.300046 11318 tablet_replica.cc:333] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.300679 11318 raft_consensus.cc:2243] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.301136 11318 raft_consensus.cc:2272] T d7366aa6480d4d1e909db5717506432b P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.303196 11318 tablet_replica.cc:333] T fbb90520a319469fa1664276964749ea P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.303814 11318 raft_consensus.cc:2243] T fbb90520a319469fa1664276964749ea P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.304265 11318 raft_consensus.cc:2272] T fbb90520a319469fa1664276964749ea P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.306491 11318 tablet_replica.cc:333] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.307049 11318 raft_consensus.cc:2243] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.307565 11318 raft_consensus.cc:2272] T e35efdb2f1c24db0a45fbc04abea260e P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.309981 11318 tablet_replica.cc:333] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.310541 11318 raft_consensus.cc:2243] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 3 FOLLOWER]: Raft consensus shutting down.
W20260709 12:16:31.310753 24204 consensus_peers.cc:597] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 -> Peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Couldn't send request to peer 008a79e516614c809c2d3afdb5cbc327. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:31.310974 11318 raft_consensus.cc:2272] T 7e815126e7554106a097f95846d7f663 P 008a79e516614c809c2d3afdb5cbc327 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.313325 11318 tablet_replica.cc:333] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.313907 11318 raft_consensus.cc:2243] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.314817 11318 raft_consensus.cc:2272] T 57c7696ee73146b59bd8b17640333077 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.316960 11318 tablet_replica.cc:333] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.317546 11318 raft_consensus.cc:2243] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.317997 11318 raft_consensus.cc:2272] T b4638bcb9174442184bc8d8a27d45072 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.320185 11318 tablet_replica.cc:333] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.320787 11318 raft_consensus.cc:2243] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.321664 11318 raft_consensus.cc:2272] T 3bd7c9c11f834f15974a65477ef9258a P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.324388 11318 tablet_replica.cc:333] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.325206 11318 raft_consensus.cc:2243] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.325814 11318 raft_consensus.cc:2272] T f4c368e2cc6c4869a85a53ae022a0785 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.328771 11318 tablet_replica.cc:333] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.329406 11318 raft_consensus.cc:2243] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.329892 11318 raft_consensus.cc:2272] T cd2d7fad08bc4376bb38cf21a6a9b611 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.332444 11318 tablet_replica.cc:333] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.333194 11318 raft_consensus.cc:2243] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.333801 11318 raft_consensus.cc:2272] T 531e8a2f98124f13910b3a8ce4d279c1 P 008a79e516614c809c2d3afdb5cbc327 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.336575 11318 tablet_replica.cc:333] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.337165 11318 raft_consensus.cc:2243] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.338083 11318 raft_consensus.cc:2272] T 809a23266b1646e0be710ddab5fff291 P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.340852 11318 tablet_replica.cc:333] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.341507 11318 raft_consensus.cc:2243] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.342412 11318 raft_consensus.cc:2272] T 0015560e665c48f0bd17fe4be5c8761d P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.342944 25207 raft_consensus.cc:493] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:31.343596 25207 raft_consensus.cc:515] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:31.345214 11318 tablet_replica.cc:333] T 1012da668d874f27a9563ae3f53ce4eb P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.345983 25207 leader_election.cc:290] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:31.346098 11318 raft_consensus.cc:2243] T 1012da668d874f27a9563ae3f53ce4eb P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.346860 11318 raft_consensus.cc:2272] T 1012da668d874f27a9563ae3f53ce4eb P 008a79e516614c809c2d3afdb5cbc327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.347579 25207 raft_consensus.cc:493] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:31.348227 25207 raft_consensus.cc:515] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:31.349975 11318 tablet_replica.cc:333] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327: stopping tablet replica
I20260709 12:16:31.350627 25207 leader_election.cc:290] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:31.350863 11318 raft_consensus.cc:2243] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.351511 11318 raft_consensus.cc:2272] T 5587d307c2f64ab9a9b39c1b190e85be P 008a79e516614c809c2d3afdb5cbc327 [term 3 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:31.359582 24204 leader_election.cc:336] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:31.360460 24204 leader_election.cc:336] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:31.360939 24204 leader_election.cc:304] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
W20260709 12:16:31.361884 24204 leader_election.cc:336] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:31.361876 25207 raft_consensus.cc:2749] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:16:31.364398 25207 raft_consensus.cc:493] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:31.365044 25207 raft_consensus.cc:515] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
W20260709 12:16:31.365975 24204 leader_election.cc:336] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
I20260709 12:16:31.366781 24204 leader_election.cc:304] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:31.367297 25207 leader_election.cc:290] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:31.367434 25182 raft_consensus.cc:2749] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:16:31.372140 24204 leader_election.cc:336] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:31.372902 24204 leader_election.cc:336] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:31.373265 24204 leader_election.cc:304] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:31.373899 25182 raft_consensus.cc:2749] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:16:31.388036 24204 consensus_peers.cc:597] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 -> Peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Couldn't send request to peer 008a79e516614c809c2d3afdb5cbc327. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:31.404000 25182 raft_consensus.cc:493] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:31.404642 25182 raft_consensus.cc:515] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:31.406791 25182 leader_election.cc:290] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
W20260709 12:16:31.410811 24204 leader_election.cc:336] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:16:31.411878 24204 leader_election.cc:336] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:31.412279 24204 leader_election.cc:304] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:31.412992 25182 raft_consensus.cc:2749] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260709 12:16:31.442839 11318 tablet_server.cc:196] TabletServer@127.11.13.131:0 shutdown complete.
I20260709 12:16:31.448176 25182 raft_consensus.cc:493] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:31.448894 25182 raft_consensus.cc:515] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:31.453503 25182 leader_election.cc:290] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
W20260709 12:16:31.456995 24204 leader_election.cc:336] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Network error: Client connection negotiation failed: client connection to 127.11.13.131:36699: connect: Connection refused (error 111)
W20260709 12:16:31.460186 24204 leader_election.cc:336] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:31.460692 24204 leader_election.cc:304] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:31.461568 25182 raft_consensus.cc:2749] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:16:31.508522 24204 consensus_peers.cc:597] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 -> Peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Couldn't send request to peer 008a79e516614c809c2d3afdb5cbc327. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.131:36699: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:31.527259 24204 consensus_peers.cc:597] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 -> Peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Couldn't send request to peer 008a79e516614c809c2d3afdb5cbc327. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.131:36699: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:31.547967 25182 raft_consensus.cc:493] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:31.548640 11318 tablet_server.cc:179] TabletServer@127.11.13.132:0 shutting down...
I20260709 12:16:31.548537 25182 raft_consensus.cc:515] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } } peers { permanent_uuid: "6c74a5203b594708921cd793aaca0b8b" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42247 } }
I20260709 12:16:31.553609 25182 leader_election.cc:290] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669), 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247)
W20260709 12:16:31.559597 24204 leader_election.cc:336] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
W20260709 12:16:31.560369 24204 leader_election.cc:336] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 6c74a5203b594708921cd793aaca0b8b (127.11.13.130:42247): Network error: Client connection negotiation failed: client connection to 127.11.13.130:42247: connect: Connection refused (error 111)
I20260709 12:16:31.560705 24204 leader_election.cc:304] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 6c74a5203b594708921cd793aaca0b8b, c359614416104a7ba9db1ede97d13802
I20260709 12:16:31.561354 25182 raft_consensus.cc:2749] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:16:31.586979 25182 raft_consensus.cc:493] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:31.590883 25182 raft_consensus.cc:515] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:31.594099 25182 leader_election.cc:290] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
I20260709 12:16:31.594909 25182 raft_consensus.cc:493] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c359614416104a7ba9db1ede97d13802)
I20260709 12:16:31.595631 25182 raft_consensus.cc:515] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "008a79e516614c809c2d3afdb5cbc327" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 36699 } } peers { permanent_uuid: "a0df0eff47b64e3eb45ab074139ed187" member_type: VOTER last_known_addr { host: "127.11.13.132" port: 41011 } } peers { permanent_uuid: "c359614416104a7ba9db1ede97d13802" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 42669 } }
I20260709 12:16:31.602100 25182 leader_election.cc:290] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699), c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669)
W20260709 12:16:31.605284 24204 leader_election.cc:336] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
W20260709 12:16:31.610625 24204 leader_election.cc:336] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Network error: Client connection negotiation failed: client connection to 127.11.13.131:36699: connect: Connection refused (error 111)
I20260709 12:16:31.611091 24204 leader_election.cc:304] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:31.611967 25246 raft_consensus.cc:2749] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260709 12:16:31.612042 24204 leader_election.cc:336] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Network error: Client connection negotiation failed: client connection to 127.11.13.131:36699: connect: Connection refused (error 111)
W20260709 12:16:31.615197 24204 consensus_peers.cc:597] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 -> Peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Couldn't send request to peer 008a79e516614c809c2d3afdb5cbc327. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.131:36699: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:31.616956 24204 leader_election.cc:336] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c359614416104a7ba9db1ede97d13802 (127.11.13.129:42669): Network error: Client connection negotiation failed: client connection to 127.11.13.129:42669: connect: Connection refused (error 111)
I20260709 12:16:31.617451 24204 leader_election.cc:304] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a0df0eff47b64e3eb45ab074139ed187; no voters: 008a79e516614c809c2d3afdb5cbc327, c359614416104a7ba9db1ede97d13802
I20260709 12:16:31.617668 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:31.618177 25246 raft_consensus.cc:2749] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:16:31.618808 11318 tablet_replica.cc:333] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.619684 11318 raft_consensus.cc:2243] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.620230 11318 raft_consensus.cc:2272] T 93f3b3f298544987a7b9fc3dd0af496c P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.622745 11318 tablet_replica.cc:333] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.623371 11318 raft_consensus.cc:2243] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.623819 11318 raft_consensus.cc:2272] T 2147e5dc12974e6c852f8961496066ae P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.626338 11318 tablet_replica.cc:333] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.626952 11318 raft_consensus.cc:2243] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.627399 11318 raft_consensus.cc:2272] T 6f15aae25cad42abb701c295e56963e3 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.629516 11318 tablet_replica.cc:333] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.630259 11318 raft_consensus.cc:2243] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.630803 11318 raft_consensus.cc:2272] T ddbc768ef2c54c3391f0558cc562a01d P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.633106 11318 tablet_replica.cc:333] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.633776 11318 raft_consensus.cc:2243] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.634302 11318 raft_consensus.cc:2272] T 6db45947fce14c98b26a8ed1ed9bbce4 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.636371 11318 tablet_replica.cc:333] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.636972 11318 raft_consensus.cc:2243] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.637828 11318 raft_consensus.cc:2272] T 77079c31ca114c49a916e49cfd8faf2c P a0df0eff47b64e3eb45ab074139ed187 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.640179 11318 tablet_replica.cc:333] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.640872 11318 raft_consensus.cc:2243] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.641691 11318 raft_consensus.cc:2272] T 94568d4dbfd440478c83b50b50976be7 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.644351 11318 tablet_replica.cc:333] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.645087 11318 raft_consensus.cc:2243] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.646093 11318 raft_consensus.cc:2272] T 2e4765d7c78c480fa4ba08d2812aeac3 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.648222 11318 tablet_replica.cc:333] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.648823 11318 raft_consensus.cc:2243] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.650238 11318 raft_consensus.cc:2272] T 131c7392ac614d87ab3d9b6713a46f7b P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.652335 11318 tablet_replica.cc:333] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.652951 11318 raft_consensus.cc:2243] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.653395 11318 raft_consensus.cc:2272] T be5ea3171ad74d74bede5875414d92ac P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.655555 11318 tablet_replica.cc:333] T c385253c4ad94570b7dbb6875a755b36 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.656208 11318 raft_consensus.cc:2243] T c385253c4ad94570b7dbb6875a755b36 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.656685 11318 raft_consensus.cc:2272] T c385253c4ad94570b7dbb6875a755b36 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.659353 11318 tablet_replica.cc:333] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.660007 11318 raft_consensus.cc:2243] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.660621 11318 raft_consensus.cc:2272] T 2bd0baecffd2454894539252387f4f2d P a0df0eff47b64e3eb45ab074139ed187 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.663983 11318 tablet_replica.cc:333] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
W20260709 12:16:31.664745 24204 consensus_peers.cc:597] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 -> Peer 008a79e516614c809c2d3afdb5cbc327 (127.11.13.131:36699): Couldn't send request to peer 008a79e516614c809c2d3afdb5cbc327. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.131:36699: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:31.664969 11318 raft_consensus.cc:2243] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.665931 11318 raft_consensus.cc:2272] T 56ed2939d1454de181106eec39d0e4bc P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.668426 11318 tablet_replica.cc:333] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.669082 11318 raft_consensus.cc:2243] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.669523 11318 raft_consensus.cc:2272] T e2eabb81feec483ea9b303c5e516b7cf P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.671805 11318 tablet_replica.cc:333] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.672397 11318 raft_consensus.cc:2243] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.672833 11318 raft_consensus.cc:2272] T 56fb07e9eb574b1fbc5da95063610dd5 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.675024 11318 tablet_replica.cc:333] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.675800 11318 raft_consensus.cc:2243] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.676373 11318 raft_consensus.cc:2272] T 57c7696ee73146b59bd8b17640333077 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.678421 11318 tablet_replica.cc:333] T 3bd7c9c11f834f15974a65477ef9258a P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.678977 11318 raft_consensus.cc:2243] T 3bd7c9c11f834f15974a65477ef9258a P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.679404 11318 raft_consensus.cc:2272] T 3bd7c9c11f834f15974a65477ef9258a P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.681375 11318 tablet_replica.cc:333] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.681927 11318 raft_consensus.cc:2243] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.682367 11318 raft_consensus.cc:2272] T f4c368e2cc6c4869a85a53ae022a0785 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.684345 11318 tablet_replica.cc:333] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.684890 11318 raft_consensus.cc:2243] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.685289 11318 raft_consensus.cc:2272] T b4638bcb9174442184bc8d8a27d45072 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.687201 11318 tablet_replica.cc:333] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.687723 11318 raft_consensus.cc:2243] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.688127 11318 raft_consensus.cc:2272] T b0805608ea264bef894c413af00a0687 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.689926 11318 tablet_replica.cc:333] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.690531 11318 raft_consensus.cc:2243] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.691334 11318 raft_consensus.cc:2272] T 8e3a9c266ee343eb903d5138d1de95d2 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.693234 11318 tablet_replica.cc:333] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.693939 11318 raft_consensus.cc:2243] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.694980 11318 raft_consensus.cc:2272] T 7dd0b13fa2ed4f4489eef33489d4ebf1 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.697178 11318 tablet_replica.cc:333] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.697867 11318 raft_consensus.cc:2243] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.698704 11318 raft_consensus.cc:2272] T b498d1ba220843a5939d152e0382bff7 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.700744 11318 tablet_replica.cc:333] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.701326 11318 raft_consensus.cc:2243] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.701753 11318 raft_consensus.cc:2272] T 4c91f626cde148838a6a979d4fd1a5ed P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.703858 11318 tablet_replica.cc:333] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.704427 11318 raft_consensus.cc:2243] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.704909 11318 raft_consensus.cc:2272] T 41c505fd92654536b72dbd8a3f1bc728 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.707170 11318 tablet_replica.cc:333] T cd2d7fad08bc4376bb38cf21a6a9b611 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.708196 11318 raft_consensus.cc:2243] T cd2d7fad08bc4376bb38cf21a6a9b611 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.708817 11318 raft_consensus.cc:2272] T cd2d7fad08bc4376bb38cf21a6a9b611 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.711731 11318 tablet_replica.cc:333] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.712651 11318 raft_consensus.cc:2243] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.713869 11318 raft_consensus.cc:2272] T 531e8a2f98124f13910b3a8ce4d279c1 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.716712 11318 tablet_replica.cc:333] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.717471 11318 raft_consensus.cc:2243] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.718207 11318 raft_consensus.cc:2272] T 659e684fa2a0433faf027bf99c82dd6b P a0df0eff47b64e3eb45ab074139ed187 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.720657 11318 tablet_replica.cc:333] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.721207 11318 raft_consensus.cc:2243] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.721635 11318 raft_consensus.cc:2272] T 0756db96a05a45bc9ebc1f91ee907496 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.723781 11318 tablet_replica.cc:333] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.724294 11318 raft_consensus.cc:2243] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.725167 11318 raft_consensus.cc:2272] T e679d0d8ffd84a23afc6d7f8faf84ca6 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.727344 11318 tablet_replica.cc:333] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.727856 11318 raft_consensus.cc:2243] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.728272 11318 raft_consensus.cc:2272] T 802231ce402445309306d2cbffa2f92c P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.730269 11318 tablet_replica.cc:333] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.730808 11318 raft_consensus.cc:2243] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.731254 11318 raft_consensus.cc:2272] T ab48aa491dc84193a39fdbf1a1c52927 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.733348 11318 tablet_replica.cc:333] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.733850 11318 raft_consensus.cc:2243] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.735005 11318 raft_consensus.cc:2272] T 3052da843efd4ef4861a13e2240bb466 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.737121 11318 tablet_replica.cc:333] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.737645 11318 raft_consensus.cc:2243] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.738519 11318 raft_consensus.cc:2272] T 1012da668d874f27a9563ae3f53ce4eb P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.740480 11318 tablet_replica.cc:333] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.741134 11318 raft_consensus.cc:2243] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.741952 11318 raft_consensus.cc:2272] T fbb90520a319469fa1664276964749ea P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.744010 11318 tablet_replica.cc:333] T ceb6c32822cf44158ca1ea0d62e0b001 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.744606 11318 raft_consensus.cc:2243] T ceb6c32822cf44158ca1ea0d62e0b001 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.745015 11318 raft_consensus.cc:2272] T ceb6c32822cf44158ca1ea0d62e0b001 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.747022 11318 tablet_replica.cc:333] T bc53eac4ce984dbaa561d9b719143d18 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.747583 11318 raft_consensus.cc:2243] T bc53eac4ce984dbaa561d9b719143d18 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.747993 11318 raft_consensus.cc:2272] T bc53eac4ce984dbaa561d9b719143d18 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.749955 11318 tablet_replica.cc:333] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.750550 11318 raft_consensus.cc:2243] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.750980 11318 raft_consensus.cc:2272] T 21f0988e53da475c8274bcb13b34b3a8 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.752916 11318 tablet_replica.cc:333] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.753510 11318 raft_consensus.cc:2243] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.754379 11318 raft_consensus.cc:2272] T 1df16f5650964ece90109902b6b40882 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.756407 11318 tablet_replica.cc:333] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.756953 11318 raft_consensus.cc:2243] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.757781 11318 raft_consensus.cc:2272] T ba257bd0f16a4a7da0cb4a7953ca1869 P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.760327 11318 tablet_replica.cc:333] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.760918 11318 raft_consensus.cc:2243] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.761795 11318 raft_consensus.cc:2272] T 55819e09f3e245179f0dd80b9e60eb6a P a0df0eff47b64e3eb45ab074139ed187 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.763891 11318 tablet_replica.cc:333] T 1e98aa59558f4e58925a3c87026b6fa4 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.764437 11318 raft_consensus.cc:2243] T 1e98aa59558f4e58925a3c87026b6fa4 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.764878 11318 raft_consensus.cc:2272] T 1e98aa59558f4e58925a3c87026b6fa4 P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.766980 11318 tablet_replica.cc:333] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.767536 11318 raft_consensus.cc:2243] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.768391 11318 raft_consensus.cc:2272] T be4e681eef304bb8bc812e313a67414d P a0df0eff47b64e3eb45ab074139ed187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.770401 11318 tablet_replica.cc:333] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187: stopping tablet replica
I20260709 12:16:31.770895 11318 raft_consensus.cc:2243] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:31.771265 11318 raft_consensus.cc:2272] T 7e815126e7554106a097f95846d7f663 P a0df0eff47b64e3eb45ab074139ed187 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.867497 11318 tablet_server.cc:196] TabletServer@127.11.13.132:0 shutdown complete.
I20260709 12:16:31.955271 11318 master.cc:562] Master@127.11.13.190:46705 shutting down...
I20260709 12:16:31.987922 11318 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:31.988839 11318 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:31.989214 11318 tablet_replica.cc:333] T 00000000000000000000000000000000 P 49fa0f839a734d2b834f455413dda82b: stopping tablet replica
I20260709 12:16:32.013149 11318 master.cc:584] Master@127.11.13.190:46705 shutdown complete.
[       OK ] LeaderRebalancerTest.RestartTserver (37726 ms)
[ RUN      ] LeaderRebalancerTest.TestMaintenanceMode
I20260709 12:16:32.123922 11318 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.11.13.190:40877
I20260709 12:16:32.125126 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:32.132077 25248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:32.132077 25247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:32.132509 25250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:32.133473 11318 server_base.cc:1061] running on GCE node
I20260709 12:16:32.135187 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:32.135445 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:32.135645 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599392135625 us; error 0 us; skew 500 ppm
I20260709 12:16:32.136363 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:32.140130 11318 webserver.cc:533] Webserver started at http://127.11.13.190:46747/ using document root <none> and password file <none>
I20260709 12:16:32.140811 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:32.141005 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:32.141392 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:32.142753 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/master-0-root/instance:
uuid: "5983d1727349477db4c65c4c602bd694"
format_stamp: "Formatted at 2026-07-09 12:16:32 on dist-test-slave-zqnl"
I20260709 12:16:32.148708 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:16:32.152732 25255 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:32.153780 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:16:32.154188 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/master-0-root
uuid: "5983d1727349477db4c65c4c602bd694"
format_stamp: "Formatted at 2026-07-09 12:16:32 on dist-test-slave-zqnl"
I20260709 12:16:32.154496 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:32.210966 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:32.212829 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:32.262707 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.190:40877
I20260709 12:16:32.263131 25307 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.190:40877 every 8 connection(s)
I20260709 12:16:32.268301 25308 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.282519 25308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694: Bootstrap starting.
I20260709 12:16:32.289110 25308 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:32.294951 25308 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694: No bootstrap required, opened a new log
I20260709 12:16:32.297528 25308 raft_consensus.cc:359] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5983d1727349477db4c65c4c602bd694" member_type: VOTER }
I20260709 12:16:32.298269 25308 raft_consensus.cc:385] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:32.298516 25308 raft_consensus.cc:740] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5983d1727349477db4c65c4c602bd694, State: Initialized, Role: FOLLOWER
I20260709 12:16:32.299127 25308 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5983d1727349477db4c65c4c602bd694" member_type: VOTER }
I20260709 12:16:32.299598 25308 raft_consensus.cc:399] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:16:32.299818 25308 raft_consensus.cc:493] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:16:32.300076 25308 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:32.305876 25308 raft_consensus.cc:515] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5983d1727349477db4c65c4c602bd694" member_type: VOTER }
I20260709 12:16:32.306713 25308 leader_election.cc:304] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5983d1727349477db4c65c4c602bd694; no voters: 
I20260709 12:16:32.307924 25308 leader_election.cc:290] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:16:32.308408 25311 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:32.309918 25311 raft_consensus.cc:697] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694 [term 1 LEADER]: Becoming Leader. State: Replica: 5983d1727349477db4c65c4c602bd694, State: Running, Role: LEADER
I20260709 12:16:32.310757 25311 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5983d1727349477db4c65c4c602bd694" member_type: VOTER }
I20260709 12:16:32.311282 25308 sys_catalog.cc:565] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:16:32.320415 25312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5983d1727349477db4c65c4c602bd694" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5983d1727349477db4c65c4c602bd694" member_type: VOTER } }
I20260709 12:16:32.321143 25312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694 [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:32.322836 25313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5983d1727349477db4c65c4c602bd694. Latest consensus state: current_term: 1 leader_uuid: "5983d1727349477db4c65c4c602bd694" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5983d1727349477db4c65c4c602bd694" member_type: VOTER } }
I20260709 12:16:32.323494 25313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694 [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:32.324410 25318 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:16:32.334322 25318 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:16:32.336992 11318 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:16:32.345671 25318 catalog_manager.cc:1383] Generated new cluster ID: c659d20461f3494d91a42136c6d454e0
I20260709 12:16:32.346154 25318 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:16:32.369120 25318 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:16:32.371033 25318 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:16:32.386747 25318 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694: Generated new TSK 0
I20260709 12:16:32.387583 25318 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:16:32.405449 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:32.414984 25329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:32.415727 25330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:32.418174 25332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:32.418812 11318 server_base.cc:1061] running on GCE node
I20260709 12:16:32.421250 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:32.421769 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:32.422020 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599392421998 us; error 0 us; skew 500 ppm
I20260709 12:16:32.422969 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:32.426445 11318 webserver.cc:533] Webserver started at http://127.11.13.129:35127/ using document root <none> and password file <none>
I20260709 12:16:32.427371 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:32.427675 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:32.428138 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:32.430238 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/ts-0-root/instance:
uuid: "da7a166dda6c4e6580e13213788f18da"
format_stamp: "Formatted at 2026-07-09 12:16:32 on dist-test-slave-zqnl"
I20260709 12:16:32.436560 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.005s	sys 0.000s
I20260709 12:16:32.441766 25337 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:32.442795 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:16:32.443151 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/ts-0-root
uuid: "da7a166dda6c4e6580e13213788f18da"
format_stamp: "Formatted at 2026-07-09 12:16:32 on dist-test-slave-zqnl"
I20260709 12:16:32.443462 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:32.468990 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:32.470480 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:32.472211 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:32.474925 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:32.475170 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:32.475445 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:32.475643 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:32.525946 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.129:33415
I20260709 12:16:32.526345 25399 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.129:33415 every 8 connection(s)
I20260709 12:16:32.533129 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:32.541525 25404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:32.543563 25405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:32.547676 25407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:32.547717 11318 server_base.cc:1061] running on GCE node
I20260709 12:16:32.550616 25400 heartbeater.cc:344] Connected to a master server at 127.11.13.190:40877
I20260709 12:16:32.550661 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:16:32.551182 25400 heartbeater.cc:461] Registering TS with master...
W20260709 12:16:32.551301 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:32.551632 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599392551606 us; error 0 us; skew 500 ppm
I20260709 12:16:32.552394 25400 heartbeater.cc:507] Master 127.11.13.190:40877 requested a full tablet report, sending...
I20260709 12:16:32.552493 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:32.555984 25273 ts_manager.cc:194] Registered new tserver with Master: da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415)
I20260709 12:16:32.556346 11318 webserver.cc:533] Webserver started at http://127.11.13.130:46449/ using document root <none> and password file <none>
I20260709 12:16:32.557221 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:32.557490 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:32.557844 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:32.559023 25273 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39438
I20260709 12:16:32.559985 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/ts-1-root/instance:
uuid: "d0987de3c9724390936aa3bd641fa419"
format_stamp: "Formatted at 2026-07-09 12:16:32 on dist-test-slave-zqnl"
I20260709 12:16:32.566398 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.000s	sys 0.007s
I20260709 12:16:32.571446 25412 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:32.572515 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.005s	sys 0.000s
I20260709 12:16:32.572958 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/ts-1-root
uuid: "d0987de3c9724390936aa3bd641fa419"
format_stamp: "Formatted at 2026-07-09 12:16:32 on dist-test-slave-zqnl"
I20260709 12:16:32.573316 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:32.594563 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:32.595880 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:32.597584 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:32.600291 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:32.600544 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:32.600814 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:32.601016 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:32.653445 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.130:34305
I20260709 12:16:32.653818 25474 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.130:34305 every 8 connection(s)
I20260709 12:16:32.659499 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:32.667989 25478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:32.669098 25479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:32.670917 25481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:32.672501 11318 server_base.cc:1061] running on GCE node
I20260709 12:16:32.673853 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:32.674140 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:32.674448 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599392674426 us; error 0 us; skew 500 ppm
I20260709 12:16:32.675215 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:32.675947 25475 heartbeater.cc:344] Connected to a master server at 127.11.13.190:40877
I20260709 12:16:32.676470 25475 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:32.677619 25475 heartbeater.cc:507] Master 127.11.13.190:40877 requested a full tablet report, sending...
I20260709 12:16:32.678580 11318 webserver.cc:533] Webserver started at http://127.11.13.131:44165/ using document root <none> and password file <none>
I20260709 12:16:32.679267 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:32.679492 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:32.679811 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:32.680573 25273 ts_manager.cc:194] Registered new tserver with Master: d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305)
I20260709 12:16:32.681243 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/ts-2-root/instance:
uuid: "676bed0fbe614782b3aae7d0db05998f"
format_stamp: "Formatted at 2026-07-09 12:16:32 on dist-test-slave-zqnl"
I20260709 12:16:32.682583 25273 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39444
I20260709 12:16:32.687355 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.002s
I20260709 12:16:32.692680 25486 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:32.693703 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:16:32.694016 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/ts-2-root
uuid: "676bed0fbe614782b3aae7d0db05998f"
format_stamp: "Formatted at 2026-07-09 12:16:32 on dist-test-slave-zqnl"
I20260709 12:16:32.694401 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:32.729076 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:32.730516 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:32.732232 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:32.734880 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:32.735136 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:32.735404 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:32.735591 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:32.789934 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.131:40749
I20260709 12:16:32.790158 25548 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.131:40749 every 8 connection(s)
I20260709 12:16:32.806967 25549 heartbeater.cc:344] Connected to a master server at 127.11.13.190:40877
I20260709 12:16:32.807408 25549 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:32.808643 25549 heartbeater.cc:507] Master 127.11.13.190:40877 requested a full tablet report, sending...
I20260709 12:16:32.811555 25273 ts_manager.cc:194] Registered new tserver with Master: 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:32.811690 11318 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018212161s
I20260709 12:16:32.813199 25273 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39458
I20260709 12:16:32.846796 25273 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:39460:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:16:32.850366 25273 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:16:32.957217 25440 tablet_service.cc:1511] Processing CreateTablet for tablet 22f011f2c17b4327a825f429b685cd0c (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:16:32.958981 25440 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22f011f2c17b4327a825f429b685cd0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.959709 25511 tablet_service.cc:1511] Processing CreateTablet for tablet 25362af9774a4e60b349e61f9a2601db (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:16:32.961441 25511 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25362af9774a4e60b349e61f9a2601db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.963375 25364 tablet_service.cc:1511] Processing CreateTablet for tablet 0879bf3038f7407693a853a4eb69d0f1 (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:16:32.963981 25437 tablet_service.cc:1511] Processing CreateTablet for tablet 25362af9774a4e60b349e61f9a2601db (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:16:32.965658 25364 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0879bf3038f7407693a853a4eb69d0f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.969643 25438 tablet_service.cc:1511] Processing CreateTablet for tablet a2f349cdc65547d6a574bbe3563db1a1 (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:16:32.990159 25358 tablet_service.cc:1511] Processing CreateTablet for tablet 14e62bfdb1564317b3ec26702c9b0172 (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:16:33.007238 25507 tablet_service.cc:1511] Processing CreateTablet for tablet 14e62bfdb1564317b3ec26702c9b0172 (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:16:32.971308 25513 tablet_service.cc:1511] Processing CreateTablet for tablet 0879bf3038f7407693a853a4eb69d0f1 (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:16:32.974506 25514 tablet_service.cc:1511] Processing CreateTablet for tablet 22f011f2c17b4327a825f429b685cd0c (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:16:33.008817 25507 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14e62bfdb1564317b3ec26702c9b0172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.967796 25512 tablet_service.cc:1511] Processing CreateTablet for tablet a2f349cdc65547d6a574bbe3563db1a1 (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:16:33.009820 25357 tablet_service.cc:1511] Processing CreateTablet for tablet ed1825085af74858b5ae5b68c999100b (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:16:32.978080 25436 tablet_service.cc:1511] Processing CreateTablet for tablet eefc812354034f0c98229c96d8e63846 (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:16:33.011700 25357 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed1825085af74858b5ae5b68c999100b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.965848 25437 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25362af9774a4e60b349e61f9a2601db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.970288 25361 tablet_service.cc:1511] Processing CreateTablet for tablet eefc812354034f0c98229c96d8e63846 (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:16:33.015926 25361 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eefc812354034f0c98229c96d8e63846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.974480 25434 tablet_service.cc:1511] Processing CreateTablet for tablet 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:16:32.973014 25439 tablet_service.cc:1511] Processing CreateTablet for tablet 0879bf3038f7407693a853a4eb69d0f1 (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260709 12:16:32.976168 25509 tablet_service.cc:1511] Processing CreateTablet for tablet 11dad3c292254bf59bd998aff5a37702 (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:16:33.021179 25439 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0879bf3038f7407693a853a4eb69d0f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:33.021940 25436 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eefc812354034f0c98229c96d8e63846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.986450 25508 tablet_service.cc:1511] Processing CreateTablet for tablet 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:16:32.986637 25359 tablet_service.cc:1511] Processing CreateTablet for tablet 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260709 12:16:33.023672 25359 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22bd5dc8f9bc4e43ae6cfd2cfd9d1086. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:33.024472 25438 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2f349cdc65547d6a574bbe3563db1a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:33.025231 25432 tablet_service.cc:1511] Processing CreateTablet for tablet ed1825085af74858b5ae5b68c999100b (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:16:33.019477 25514 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22f011f2c17b4327a825f429b685cd0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.971750 25360 tablet_service.cc:1511] Processing CreateTablet for tablet 11dad3c292254bf59bd998aff5a37702 (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:16:33.027683 25360 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11dad3c292254bf59bd998aff5a37702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:33.027429 25506 tablet_service.cc:1511] Processing CreateTablet for tablet ed1825085af74858b5ae5b68c999100b (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260709 12:16:33.028391 25513 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0879bf3038f7407693a853a4eb69d0f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:33.029423 25432 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed1825085af74858b5ae5b68c999100b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:33.029657 25512 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2f349cdc65547d6a574bbe3563db1a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:33.030439 25506 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ed1825085af74858b5ae5b68c999100b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:33.031149 25509 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11dad3c292254bf59bd998aff5a37702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:33.032445 25356 tablet_service.cc:1511] Processing CreateTablet for tablet 7411d269e0944ceaa4f08766843e4819 (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:16:33.033890 25356 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7411d269e0944ceaa4f08766843e4819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.957217 25510 tablet_service.cc:1511] Processing CreateTablet for tablet eefc812354034f0c98229c96d8e63846 (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260709 12:16:33.037250 25510 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eefc812354034f0c98229c96d8e63846. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.968866 25362 tablet_service.cc:1511] Processing CreateTablet for tablet 25362af9774a4e60b349e61f9a2601db (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260709 12:16:33.041162 25362 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 25362af9774a4e60b349e61f9a2601db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.967478 25363 tablet_service.cc:1511] Processing CreateTablet for tablet a2f349cdc65547d6a574bbe3563db1a1 (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260709 12:16:32.973649 25365 tablet_service.cc:1511] Processing CreateTablet for tablet 22f011f2c17b4327a825f429b685cd0c (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260709 12:16:33.042800 25358 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14e62bfdb1564317b3ec26702c9b0172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:33.044628 25431 tablet_service.cc:1511] Processing CreateTablet for tablet 7411d269e0944ceaa4f08766843e4819 (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:16:32.994977 25433 tablet_service.cc:1511] Processing CreateTablet for tablet 14e62bfdb1564317b3ec26702c9b0172 (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260709 12:16:33.046284 25431 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7411d269e0944ceaa4f08766843e4819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:32.966324 25435 tablet_service.cc:1511] Processing CreateTablet for tablet 11dad3c292254bf59bd998aff5a37702 (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260709 12:16:33.059578 25433 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 14e62bfdb1564317b3ec26702c9b0172. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:33.051573 25505 tablet_service.cc:1511] Processing CreateTablet for tablet 7411d269e0944ceaa4f08766843e4819 (DEFAULT_TABLE table=test-workload [id=77b91ef8a6ae4478ade4d6f793803527]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260709 12:16:33.082916 25505 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7411d269e0944ceaa4f08766843e4819. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:33.091975 25508 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22bd5dc8f9bc4e43ae6cfd2cfd9d1086. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:33.062698 25365 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22f011f2c17b4327a825f429b685cd0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:33.113101 25435 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 11dad3c292254bf59bd998aff5a37702. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:33.113492 25564 tablet_bootstrap.cc:492] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da: Bootstrap starting.
I20260709 12:16:33.120136 25363 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet a2f349cdc65547d6a574bbe3563db1a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:33.126946 25434 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 22bd5dc8f9bc4e43ae6cfd2cfd9d1086. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:33.236191 25566 tablet_bootstrap.cc:492] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f: Bootstrap starting.
I20260709 12:16:33.243444 25565 tablet_bootstrap.cc:492] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419: Bootstrap starting.
I20260709 12:16:33.251904 25564 tablet_bootstrap.cc:654] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.260299 25566 tablet_bootstrap.cc:654] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.267102 25565 tablet_bootstrap.cc:654] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.274039 25566 tablet_bootstrap.cc:492] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f: No bootstrap required, opened a new log
I20260709 12:16:33.275853 25566 ts_tablet_manager.cc:1403] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f: Time spent bootstrapping tablet: real 0.040s	user 0.010s	sys 0.012s
I20260709 12:16:33.277354 25564 tablet_bootstrap.cc:492] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da: No bootstrap required, opened a new log
I20260709 12:16:33.277981 25564 ts_tablet_manager.cc:1403] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da: Time spent bootstrapping tablet: real 0.165s	user 0.009s	sys 0.010s
I20260709 12:16:33.279273 25566 raft_consensus.cc:359] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.280193 25566 raft_consensus.cc:385] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.280587 25566 raft_consensus.cc:740] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.281102 25564 raft_consensus.cc:359] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.281945 25564 raft_consensus.cc:385] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.282346 25564 raft_consensus.cc:740] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.281656 25566 consensus_queue.cc:260] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.283115 25564 consensus_queue.cc:260] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.285794 25549 heartbeater.cc:499] Master 127.11.13.190:40877 was elected leader, sending a full tablet report...
I20260709 12:16:33.286389 25566 ts_tablet_manager.cc:1434] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:16:33.287393 25400 heartbeater.cc:499] Master 127.11.13.190:40877 was elected leader, sending a full tablet report...
I20260709 12:16:33.288319 25564 ts_tablet_manager.cc:1434] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.002s
I20260709 12:16:33.288357 25566 tablet_bootstrap.cc:492] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f: Bootstrap starting.
I20260709 12:16:33.287073 25565 tablet_bootstrap.cc:492] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419: No bootstrap required, opened a new log
I20260709 12:16:33.289410 25565 ts_tablet_manager.cc:1403] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419: Time spent bootstrapping tablet: real 0.046s	user 0.009s	sys 0.005s
I20260709 12:16:33.289681 25564 tablet_bootstrap.cc:492] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da: Bootstrap starting.
I20260709 12:16:33.292871 25565 raft_consensus.cc:359] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.293730 25565 raft_consensus.cc:385] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.294046 25565 raft_consensus.cc:740] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Initialized, Role: FOLLOWER
W20260709 12:16:33.295369 25550 tablet.cc:2406] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:16:33.296851 25566 tablet_bootstrap.cc:654] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.297187 25564 tablet_bootstrap.cc:654] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.310053 25564 tablet_bootstrap.cc:492] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da: No bootstrap required, opened a new log
I20260709 12:16:33.311013 25564 ts_tablet_manager.cc:1403] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da: Time spent bootstrapping tablet: real 0.022s	user 0.013s	sys 0.007s
I20260709 12:16:33.312477 25565 consensus_queue.cc:260] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.315901 25564 raft_consensus.cc:359] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.316936 25564 raft_consensus.cc:385] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.317278 25564 raft_consensus.cc:740] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.318457 25564 consensus_queue.cc:260] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.321645 25564 ts_tablet_manager.cc:1434] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.004s
I20260709 12:16:33.322842 25564 tablet_bootstrap.cc:492] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da: Bootstrap starting.
W20260709 12:16:33.331246 25321 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:33.332788 25475 heartbeater.cc:499] Master 127.11.13.190:40877 was elected leader, sending a full tablet report...
I20260709 12:16:33.333196 25564 tablet_bootstrap.cc:654] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.333413 25566 tablet_bootstrap.cc:492] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f: No bootstrap required, opened a new log
I20260709 12:16:33.334043 25566 ts_tablet_manager.cc:1403] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f: Time spent bootstrapping tablet: real 0.046s	user 0.019s	sys 0.023s
I20260709 12:16:33.332767 25575 raft_consensus.cc:493] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.337210 25575 raft_consensus.cc:515] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.342447 25565 ts_tablet_manager.cc:1434] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419: Time spent starting tablet: real 0.053s	user 0.015s	sys 0.001s
I20260709 12:16:33.343818 25565 tablet_bootstrap.cc:492] T 25362af9774a4e60b349e61f9a2601db P d0987de3c9724390936aa3bd641fa419: Bootstrap starting.
I20260709 12:16:33.344085 25566 raft_consensus.cc:359] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.344923 25566 raft_consensus.cc:385] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.345477 25566 raft_consensus.cc:740] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.346583 25566 consensus_queue.cc:260] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.349346 25566 ts_tablet_manager.cc:1434] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.000s
I20260709 12:16:33.350951 25566 tablet_bootstrap.cc:492] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f: Bootstrap starting.
I20260709 12:16:33.360951 25566 tablet_bootstrap.cc:654] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.376276 25575 leader_election.cc:290] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:33.381642 25566 tablet_bootstrap.cc:492] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f: No bootstrap required, opened a new log
I20260709 12:16:33.385166 25566 ts_tablet_manager.cc:1403] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f: Time spent bootstrapping tablet: real 0.035s	user 0.022s	sys 0.000s
I20260709 12:16:33.388068 25565 tablet_bootstrap.cc:654] T 25362af9774a4e60b349e61f9a2601db P d0987de3c9724390936aa3bd641fa419: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.388497 25566 raft_consensus.cc:359] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.389319 25566 raft_consensus.cc:385] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.389644 25566 raft_consensus.cc:740] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.389822 25564 tablet_bootstrap.cc:492] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da: No bootstrap required, opened a new log
I20260709 12:16:33.390477 25564 ts_tablet_manager.cc:1403] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da: Time spent bootstrapping tablet: real 0.068s	user 0.028s	sys 0.021s
I20260709 12:16:33.390626 25566 consensus_queue.cc:260] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.393469 25566 ts_tablet_manager.cc:1434] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:16:33.394775 25566 tablet_bootstrap.cc:492] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f: Bootstrap starting.
I20260709 12:16:33.398097 25564 raft_consensus.cc:359] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.399334 25564 raft_consensus.cc:385] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.399766 25564 raft_consensus.cc:740] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.408123 25564 consensus_queue.cc:260] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.408671 25375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7411d269e0944ceaa4f08766843e4819" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7a166dda6c4e6580e13213788f18da" is_pre_election: true
W20260709 12:16:33.410915 25416 leader_election.cc:343] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.414078 25564 ts_tablet_manager.cc:1434] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da: Time spent starting tablet: real 0.023s	user 0.010s	sys 0.000s
I20260709 12:16:33.415751 25564 tablet_bootstrap.cc:492] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da: Bootstrap starting.
I20260709 12:16:33.415578 25524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7411d269e0944ceaa4f08766843e4819" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676bed0fbe614782b3aae7d0db05998f" is_pre_election: true
W20260709 12:16:33.417649 25414 leader_election.cc:343] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.418200 25414 leader_election.cc:304] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d0987de3c9724390936aa3bd641fa419; no voters: 676bed0fbe614782b3aae7d0db05998f, da7a166dda6c4e6580e13213788f18da
I20260709 12:16:33.419153 25575 raft_consensus.cc:2749] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:33.421487 25566 tablet_bootstrap.cc:654] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.427608 25564 tablet_bootstrap.cc:654] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.429168 25565 tablet_bootstrap.cc:492] T 25362af9774a4e60b349e61f9a2601db P d0987de3c9724390936aa3bd641fa419: No bootstrap required, opened a new log
I20260709 12:16:33.429816 25565 ts_tablet_manager.cc:1403] T 25362af9774a4e60b349e61f9a2601db P d0987de3c9724390936aa3bd641fa419: Time spent bootstrapping tablet: real 0.086s	user 0.022s	sys 0.021s
I20260709 12:16:33.433671 25564 tablet_bootstrap.cc:492] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da: No bootstrap required, opened a new log
I20260709 12:16:33.433451 25565 raft_consensus.cc:359] T 25362af9774a4e60b349e61f9a2601db P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.434309 25565 raft_consensus.cc:385] T 25362af9774a4e60b349e61f9a2601db P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.434366 25564 ts_tablet_manager.cc:1403] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da: Time spent bootstrapping tablet: real 0.019s	user 0.013s	sys 0.003s
I20260709 12:16:33.434803 25570 raft_consensus.cc:493] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.434837 25565 raft_consensus.cc:740] T 25362af9774a4e60b349e61f9a2601db P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.435643 25570 raft_consensus.cc:515] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.436096 25565 consensus_queue.cc:260] T 25362af9774a4e60b349e61f9a2601db P d0987de3c9724390936aa3bd641fa419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.438140 25564 raft_consensus.cc:359] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.438684 25565 ts_tablet_manager.cc:1434] T 25362af9774a4e60b349e61f9a2601db P d0987de3c9724390936aa3bd641fa419: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.005s
I20260709 12:16:33.439002 25564 raft_consensus.cc:385] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.439375 25564 raft_consensus.cc:740] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.440030 25565 tablet_bootstrap.cc:492] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419: Bootstrap starting.
I20260709 12:16:33.440311 25564 consensus_queue.cc:260] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.441216 25566 tablet_bootstrap.cc:492] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f: No bootstrap required, opened a new log
I20260709 12:16:33.441880 25566 ts_tablet_manager.cc:1403] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f: Time spent bootstrapping tablet: real 0.047s	user 0.013s	sys 0.004s
I20260709 12:16:33.445308 25566 raft_consensus.cc:359] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.446215 25566 raft_consensus.cc:385] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.446605 25566 raft_consensus.cc:740] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.447829 25565 tablet_bootstrap.cc:654] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.447611 25566 consensus_queue.cc:260] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.451406 25564 ts_tablet_manager.cc:1434] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.001s
I20260709 12:16:33.452683 25570 leader_election.cc:290] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305)
I20260709 12:16:33.459638 25564 tablet_bootstrap.cc:492] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da: Bootstrap starting.
I20260709 12:16:33.470181 25584 raft_consensus.cc:493] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.471007 25584 raft_consensus.cc:515] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.474822 25565 tablet_bootstrap.cc:492] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419: No bootstrap required, opened a new log
I20260709 12:16:33.475471 25565 ts_tablet_manager.cc:1403] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419: Time spent bootstrapping tablet: real 0.036s	user 0.013s	sys 0.014s
I20260709 12:16:33.471812 25564 tablet_bootstrap.cc:654] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.473078 25566 ts_tablet_manager.cc:1434] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f: Time spent starting tablet: real 0.031s	user 0.010s	sys 0.007s
I20260709 12:16:33.479130 25565 raft_consensus.cc:359] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.479984 25565 raft_consensus.cc:385] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.480307 25565 raft_consensus.cc:740] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.480448 25375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f349cdc65547d6a574bbe3563db1a1" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7a166dda6c4e6580e13213788f18da" is_pre_election: true
I20260709 12:16:33.481359 25565 consensus_queue.cc:260] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.482018 25374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0879bf3038f7407693a853a4eb69d0f1" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7a166dda6c4e6580e13213788f18da" is_pre_election: true
I20260709 12:16:33.482825 25374 raft_consensus.cc:2468] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 0.
W20260709 12:16:33.484521 25490 leader_election.cc:343] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.485980 25565 ts_tablet_manager.cc:1434] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.006s
I20260709 12:16:33.486033 25566 tablet_bootstrap.cc:492] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f: Bootstrap starting.
I20260709 12:16:33.486280 25490 leader_election.cc:304] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, da7a166dda6c4e6580e13213788f18da; no voters: 
I20260709 12:16:33.487257 25565 tablet_bootstrap.cc:492] T eefc812354034f0c98229c96d8e63846 P d0987de3c9724390936aa3bd641fa419: Bootstrap starting.
I20260709 12:16:33.486150 25584 leader_election.cc:290] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305)
I20260709 12:16:33.491842 25570 raft_consensus.cc:2804] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:33.492364 25570 raft_consensus.cc:493] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:33.492728 25570 raft_consensus.cc:3060] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.499259 25566 tablet_bootstrap.cc:654] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.499610 25565 tablet_bootstrap.cc:654] T eefc812354034f0c98229c96d8e63846 P d0987de3c9724390936aa3bd641fa419: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.508893 25450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f349cdc65547d6a574bbe3563db1a1" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0987de3c9724390936aa3bd641fa419" is_pre_election: true
I20260709 12:16:33.510092 25450 raft_consensus.cc:2468] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 0.
I20260709 12:16:33.511623 25570 raft_consensus.cc:515] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.511276 25449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0879bf3038f7407693a853a4eb69d0f1" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0987de3c9724390936aa3bd641fa419" is_pre_election: true
I20260709 12:16:33.515168 25570 leader_election.cc:290] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 election: Requested vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305)
I20260709 12:16:33.517159 25490 leader_election.cc:304] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, d0987de3c9724390936aa3bd641fa419; no voters: da7a166dda6c4e6580e13213788f18da
I20260709 12:16:33.517617 25374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0879bf3038f7407693a853a4eb69d0f1" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7a166dda6c4e6580e13213788f18da"
I20260709 12:16:33.518751 25374 raft_consensus.cc:3060] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:33.519573 25490 leader_election.cc:343] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.522449 25571 raft_consensus.cc:493] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.523255 25571 raft_consensus.cc:515] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.530930 25449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0879bf3038f7407693a853a4eb69d0f1" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0987de3c9724390936aa3bd641fa419"
I20260709 12:16:33.532172 25564 tablet_bootstrap.cc:492] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da: No bootstrap required, opened a new log
I20260709 12:16:33.532959 25564 ts_tablet_manager.cc:1403] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da: Time spent bootstrapping tablet: real 0.074s	user 0.013s	sys 0.020s
I20260709 12:16:33.537106 25571 leader_election.cc:290] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:33.541200 25565 tablet_bootstrap.cc:492] T eefc812354034f0c98229c96d8e63846 P d0987de3c9724390936aa3bd641fa419: No bootstrap required, opened a new log
W20260709 12:16:33.541203 25490 leader_election.cc:343] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.541956 25565 ts_tablet_manager.cc:1403] T eefc812354034f0c98229c96d8e63846 P d0987de3c9724390936aa3bd641fa419: Time spent bootstrapping tablet: real 0.055s	user 0.030s	sys 0.016s
I20260709 12:16:33.537751 25564 raft_consensus.cc:359] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.543206 25564 raft_consensus.cc:385] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.543572 25564 raft_consensus.cc:740] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.541250 25374 raft_consensus.cc:2468] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 1.
I20260709 12:16:33.544586 25570 raft_consensus.cc:2804] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:33.545465 25570 raft_consensus.cc:493] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:33.545876 25570 raft_consensus.cc:3060] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.545970 25490 leader_election.cc:304] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, da7a166dda6c4e6580e13213788f18da; no voters: d0987de3c9724390936aa3bd641fa419
I20260709 12:16:33.547093 25584 raft_consensus.cc:2804] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:33.544603 25564 consensus_queue.cc:260] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.544986 25566 tablet_bootstrap.cc:492] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f: No bootstrap required, opened a new log
I20260709 12:16:33.546030 25565 raft_consensus.cc:359] T eefc812354034f0c98229c96d8e63846 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.548977 25565 raft_consensus.cc:385] T eefc812354034f0c98229c96d8e63846 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.549278 25565 raft_consensus.cc:740] T eefc812354034f0c98229c96d8e63846 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.550089 25565 consensus_queue.cc:260] T eefc812354034f0c98229c96d8e63846 P d0987de3c9724390936aa3bd641fa419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.552484 25565 ts_tablet_manager.cc:1434] T eefc812354034f0c98229c96d8e63846 P d0987de3c9724390936aa3bd641fa419: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.003s
I20260709 12:16:33.553587 25565 tablet_bootstrap.cc:492] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419: Bootstrap starting.
I20260709 12:16:33.554687 25564 ts_tablet_manager.cc:1434] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da: Time spent starting tablet: real 0.021s	user 0.005s	sys 0.007s
I20260709 12:16:33.555985 25564 tablet_bootstrap.cc:492] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da: Bootstrap starting.
I20260709 12:16:33.561216 25565 tablet_bootstrap.cc:654] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.567333 25566 ts_tablet_manager.cc:1403] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f: Time spent bootstrapping tablet: real 0.082s	user 0.036s	sys 0.013s
I20260709 12:16:33.570861 25571 raft_consensus.cc:493] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.572140 25571 raft_consensus.cc:515] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.572165 25566 raft_consensus.cc:359] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.574069 25566 raft_consensus.cc:385] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.574580 25566 raft_consensus.cc:740] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.576555 25566 consensus_queue.cc:260] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.577903 25571 leader_election.cc:290] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:33.576575 25564 tablet_bootstrap.cc:654] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.588079 25584 raft_consensus.cc:697] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [term 1 LEADER]: Becoming Leader. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Running, Role: LEADER
I20260709 12:16:33.591277 25584 consensus_queue.cc:237] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.599233 25592 raft_consensus.cc:493] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.600005 25592 raft_consensus.cc:515] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.613933 25592 leader_election.cc:290] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305)
I20260709 12:16:33.614212 25449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25362af9774a4e60b349e61f9a2601db" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0987de3c9724390936aa3bd641fa419" is_pre_election: true
I20260709 12:16:33.615267 25449 raft_consensus.cc:2468] T 25362af9774a4e60b349e61f9a2601db P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 0.
I20260709 12:16:33.614068 25374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25362af9774a4e60b349e61f9a2601db" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7a166dda6c4e6580e13213788f18da" is_pre_election: true
I20260709 12:16:33.616816 25374 raft_consensus.cc:2468] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 0.
I20260709 12:16:33.617990 25490 leader_election.cc:304] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, d0987de3c9724390936aa3bd641fa419; no voters: 
I20260709 12:16:33.599972 25570 raft_consensus.cc:515] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.619748 25524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14e62bfdb1564317b3ec26702c9b0172" candidate_uuid: "da7a166dda6c4e6580e13213788f18da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676bed0fbe614782b3aae7d0db05998f" is_pre_election: true
I20260709 12:16:33.620460 25524 raft_consensus.cc:2468] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da7a166dda6c4e6580e13213788f18da in term 0.
I20260709 12:16:33.621076 25565 tablet_bootstrap.cc:492] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419: No bootstrap required, opened a new log
I20260709 12:16:33.621181 25570 leader_election.cc:290] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 election: Requested vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305)
I20260709 12:16:33.621780 25565 ts_tablet_manager.cc:1403] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419: Time spent bootstrapping tablet: real 0.068s	user 0.010s	sys 0.010s
I20260709 12:16:33.621963 25570 raft_consensus.cc:2804] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:33.622465 25570 raft_consensus.cc:493] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:33.622813 25339 leader_election.cc:304] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, da7a166dda6c4e6580e13213788f18da; no voters: 
I20260709 12:16:33.622848 25570 raft_consensus.cc:3060] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.624697 25450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25362af9774a4e60b349e61f9a2601db" candidate_uuid: "da7a166dda6c4e6580e13213788f18da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0987de3c9724390936aa3bd641fa419" is_pre_election: true
I20260709 12:16:33.625451 25450 raft_consensus.cc:2468] T 25362af9774a4e60b349e61f9a2601db P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da7a166dda6c4e6580e13213788f18da in term 0.
I20260709 12:16:33.626428 25571 raft_consensus.cc:2804] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:33.625648 25565 raft_consensus.cc:359] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.626889 25571 raft_consensus.cc:493] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:33.627247 25565 raft_consensus.cc:385] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.627300 25571 raft_consensus.cc:3060] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.628733 25341 leader_election.cc:304] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0987de3c9724390936aa3bd641fa419, da7a166dda6c4e6580e13213788f18da; no voters: 
I20260709 12:16:33.630508 25524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25362af9774a4e60b349e61f9a2601db" candidate_uuid: "da7a166dda6c4e6580e13213788f18da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676bed0fbe614782b3aae7d0db05998f" is_pre_election: true
I20260709 12:16:33.631629 25449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14e62bfdb1564317b3ec26702c9b0172" candidate_uuid: "da7a166dda6c4e6580e13213788f18da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0987de3c9724390936aa3bd641fa419" is_pre_election: true
I20260709 12:16:33.633972 25374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f349cdc65547d6a574bbe3563db1a1" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7a166dda6c4e6580e13213788f18da"
I20260709 12:16:33.633801 25570 raft_consensus.cc:515] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.634727 25374 raft_consensus.cc:3060] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.635205 25524 raft_consensus.cc:2393] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate da7a166dda6c4e6580e13213788f18da in current term 1: Already voted for candidate 676bed0fbe614782b3aae7d0db05998f in this term.
I20260709 12:16:33.638329 25565 raft_consensus.cc:740] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.639748 25450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f349cdc65547d6a574bbe3563db1a1" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0987de3c9724390936aa3bd641fa419"
I20260709 12:16:33.640655 25450 raft_consensus.cc:3060] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.646009 25375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25362af9774a4e60b349e61f9a2601db" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7a166dda6c4e6580e13213788f18da"
I20260709 12:16:33.647079 25375 raft_consensus.cc:3060] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.648911 25565 consensus_queue.cc:260] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.654996 25449 raft_consensus.cc:2468] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da7a166dda6c4e6580e13213788f18da in term 0.
I20260709 12:16:33.665001 25565 ts_tablet_manager.cc:1434] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419: Time spent starting tablet: real 0.043s	user 0.012s	sys 0.001s
I20260709 12:16:33.653622 25571 raft_consensus.cc:515] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.666109 25566 ts_tablet_manager.cc:1434] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f: Time spent starting tablet: real 0.098s	user 0.034s	sys 0.005s
I20260709 12:16:33.669800 25449 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "25362af9774a4e60b349e61f9a2601db" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0987de3c9724390936aa3bd641fa419"
I20260709 12:16:33.674502 25449 raft_consensus.cc:3060] T 25362af9774a4e60b349e61f9a2601db P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.660432 25375 raft_consensus.cc:2468] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 1.
I20260709 12:16:33.670485 25570 leader_election.cc:290] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 election: Requested vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305)
I20260709 12:16:33.671710 25271 catalog_manager.cc:5697] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f reported cstate change: term changed from 0 to 1, leader changed from <none> to 676bed0fbe614782b3aae7d0db05998f (127.11.13.131). New cstate: current_term: 1 leader_uuid: "676bed0fbe614782b3aae7d0db05998f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:33.678611 25490 leader_election.cc:304] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, da7a166dda6c4e6580e13213788f18da; no voters: 
I20260709 12:16:33.687316 25570 raft_consensus.cc:2804] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:33.678601 25374 raft_consensus.cc:2468] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 1.
I20260709 12:16:33.680263 25571 leader_election.cc:290] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [CANDIDATE]: Term 1 election: Requested vote from peers d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:33.681908 25565 tablet_bootstrap.cc:492] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419: Bootstrap starting.
I20260709 12:16:33.687973 25570 raft_consensus.cc:697] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [term 1 LEADER]: Becoming Leader. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Running, Role: LEADER
I20260709 12:16:33.688280 25524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14e62bfdb1564317b3ec26702c9b0172" candidate_uuid: "da7a166dda6c4e6580e13213788f18da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
I20260709 12:16:33.691256 25524 raft_consensus.cc:3060] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.691881 25490 leader_election.cc:304] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, da7a166dda6c4e6580e13213788f18da; no voters: 
I20260709 12:16:33.692215 25570 consensus_queue.cc:237] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.688304 25566 tablet_bootstrap.cc:492] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f: Bootstrap starting.
I20260709 12:16:33.694525 25571 raft_consensus.cc:2764] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260709 12:16:33.695616 25564 tablet_bootstrap.cc:492] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da: No bootstrap required, opened a new log
I20260709 12:16:33.696368 25564 ts_tablet_manager.cc:1403] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da: Time spent bootstrapping tablet: real 0.141s	user 0.025s	sys 0.047s
I20260709 12:16:33.698436 25584 raft_consensus.cc:2804] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:33.699150 25584 raft_consensus.cc:697] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [term 1 LEADER]: Becoming Leader. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Running, Role: LEADER
I20260709 12:16:33.699898 25564 raft_consensus.cc:359] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.703414 25564 raft_consensus.cc:385] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.700101 25584 consensus_queue.cc:237] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.703817 25564 raft_consensus.cc:740] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.704913 25564 consensus_queue.cc:260] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.705566 25565 tablet_bootstrap.cc:654] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:33.691237 25341 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:34305, user_credentials={real_user=slave}} blocked reactor thread for 62728.7us
I20260709 12:16:33.715720 25449 raft_consensus.cc:2468] T 25362af9774a4e60b349e61f9a2601db P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 1.
I20260709 12:16:33.717772 25448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14e62bfdb1564317b3ec26702c9b0172" candidate_uuid: "da7a166dda6c4e6580e13213788f18da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0987de3c9724390936aa3bd641fa419"
I20260709 12:16:33.718636 25448 raft_consensus.cc:3060] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.724056 25524 raft_consensus.cc:2468] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da7a166dda6c4e6580e13213788f18da in term 1.
I20260709 12:16:33.726059 25339 leader_election.cc:304] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, da7a166dda6c4e6580e13213788f18da; no voters: 
I20260709 12:16:33.709221 25450 raft_consensus.cc:2468] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 1.
I20260709 12:16:33.730633 25448 raft_consensus.cc:2468] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da7a166dda6c4e6580e13213788f18da in term 1.
I20260709 12:16:33.730754 25597 raft_consensus.cc:2804] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:33.733160 25597 raft_consensus.cc:697] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 1 LEADER]: Becoming Leader. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Running, Role: LEADER
I20260709 12:16:33.734499 25597 consensus_queue.cc:237] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.735814 25566 tablet_bootstrap.cc:654] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.738473 25564 ts_tablet_manager.cc:1434] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da: Time spent starting tablet: real 0.042s	user 0.008s	sys 0.001s
I20260709 12:16:33.740510 25564 tablet_bootstrap.cc:492] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da: Bootstrap starting.
I20260709 12:16:33.743979 25565 tablet_bootstrap.cc:492] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419: No bootstrap required, opened a new log
I20260709 12:16:33.744782 25565 ts_tablet_manager.cc:1403] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419: Time spent bootstrapping tablet: real 0.063s	user 0.015s	sys 0.008s
I20260709 12:16:33.749145 25565 raft_consensus.cc:359] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } }
I20260709 12:16:33.750017 25565 raft_consensus.cc:385] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.750352 25565 raft_consensus.cc:740] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.751638 25565 consensus_queue.cc:260] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } }
I20260709 12:16:33.752921 25564 tablet_bootstrap.cc:654] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.757932 25565 ts_tablet_manager.cc:1434] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.000s
I20260709 12:16:33.759176 25565 tablet_bootstrap.cc:492] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419: Bootstrap starting.
I20260709 12:16:33.763049 25566 tablet_bootstrap.cc:492] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f: No bootstrap required, opened a new log
I20260709 12:16:33.763986 25566 ts_tablet_manager.cc:1403] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f: Time spent bootstrapping tablet: real 0.077s	user 0.020s	sys 0.009s
I20260709 12:16:33.766161 25273 catalog_manager.cc:5697] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da reported cstate change: term changed from 0 to 1, leader changed from <none> to da7a166dda6c4e6580e13213788f18da (127.11.13.129). New cstate: current_term: 1 leader_uuid: "da7a166dda6c4e6580e13213788f18da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:33.779696 25565 tablet_bootstrap.cc:654] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.782642 25271 catalog_manager.cc:5697] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f reported cstate change: term changed from 0 to 1, leader changed from <none> to 676bed0fbe614782b3aae7d0db05998f (127.11.13.131). New cstate: current_term: 1 leader_uuid: "676bed0fbe614782b3aae7d0db05998f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:33.784528 25584 raft_consensus.cc:493] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.784559 25566 raft_consensus.cc:359] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.785269 25584 raft_consensus.cc:515] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.785524 25566 raft_consensus.cc:385] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.785557 25564 tablet_bootstrap.cc:492] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da: No bootstrap required, opened a new log
I20260709 12:16:33.785953 25566 raft_consensus.cc:740] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.786356 25564 ts_tablet_manager.cc:1403] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da: Time spent bootstrapping tablet: real 0.047s	user 0.024s	sys 0.018s
I20260709 12:16:33.786834 25566 consensus_queue.cc:260] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.787448 25584 leader_election.cc:290] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305)
I20260709 12:16:33.789532 25566 ts_tablet_manager.cc:1434] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f: Time spent starting tablet: real 0.025s	user 0.006s	sys 0.001s
I20260709 12:16:33.789626 25564 raft_consensus.cc:359] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.790501 25564 raft_consensus.cc:385] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.790711 25566 tablet_bootstrap.cc:492] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f: Bootstrap starting.
I20260709 12:16:33.790850 25564 raft_consensus.cc:740] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.791735 25564 consensus_queue.cc:260] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.791742 25374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eefc812354034f0c98229c96d8e63846" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7a166dda6c4e6580e13213788f18da" is_pre_election: true
I20260709 12:16:33.793423 25564 ts_tablet_manager.cc:1434] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da: Time spent starting tablet: real 0.007s	user 0.004s	sys 0.000s
W20260709 12:16:33.794188 25490 leader_election.cc:343] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.794651 25564 tablet_bootstrap.cc:492] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da: Bootstrap starting.
I20260709 12:16:33.792081 25448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eefc812354034f0c98229c96d8e63846" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0987de3c9724390936aa3bd641fa419" is_pre_election: true
I20260709 12:16:33.797648 25448 raft_consensus.cc:2468] T eefc812354034f0c98229c96d8e63846 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 0.
I20260709 12:16:33.798718 25566 tablet_bootstrap.cc:654] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.799237 25490 leader_election.cc:304] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, d0987de3c9724390936aa3bd641fa419; no voters: da7a166dda6c4e6580e13213788f18da
I20260709 12:16:33.800714 25584 raft_consensus.cc:2804] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:33.801173 25584 raft_consensus.cc:493] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:33.801594 25584 raft_consensus.cc:3060] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.802935 25564 tablet_bootstrap.cc:654] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.804447 25271 catalog_manager.cc:5697] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f reported cstate change: term changed from 0 to 1, leader changed from <none> to 676bed0fbe614782b3aae7d0db05998f (127.11.13.131). New cstate: current_term: 1 leader_uuid: "676bed0fbe614782b3aae7d0db05998f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:33.816138 25565 tablet_bootstrap.cc:492] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419: No bootstrap required, opened a new log
I20260709 12:16:33.816794 25565 ts_tablet_manager.cc:1403] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419: Time spent bootstrapping tablet: real 0.058s	user 0.008s	sys 0.003s
I20260709 12:16:33.817253 25564 tablet_bootstrap.cc:492] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da: No bootstrap required, opened a new log
I20260709 12:16:33.817905 25564 ts_tablet_manager.cc:1403] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da: Time spent bootstrapping tablet: real 0.023s	user 0.014s	sys 0.009s
I20260709 12:16:33.818684 25584 raft_consensus.cc:515] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.820142 25565 raft_consensus.cc:359] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.821010 25565 raft_consensus.cc:385] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.820768 25564 raft_consensus.cc:359] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.821406 25565 raft_consensus.cc:740] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.821606 25564 raft_consensus.cc:385] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.821978 25564 raft_consensus.cc:740] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.822082 25374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eefc812354034f0c98229c96d8e63846" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7a166dda6c4e6580e13213788f18da"
I20260709 12:16:33.822381 25565 consensus_queue.cc:260] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.822885 25564 consensus_queue.cc:260] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.825428 25374 raft_consensus.cc:3060] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.827649 25565 ts_tablet_manager.cc:1434] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419: Time spent starting tablet: real 0.011s	user 0.009s	sys 0.000s
I20260709 12:16:33.828924 25565 tablet_bootstrap.cc:492] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419: Bootstrap starting.
I20260709 12:16:33.834062 25374 raft_consensus.cc:2468] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 1.
I20260709 12:16:33.835956 25565 tablet_bootstrap.cc:654] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.843711 25448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eefc812354034f0c98229c96d8e63846" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0987de3c9724390936aa3bd641fa419"
I20260709 12:16:33.844492 25448 raft_consensus.cc:3060] T eefc812354034f0c98229c96d8e63846 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.847921 25490 leader_election.cc:304] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, da7a166dda6c4e6580e13213788f18da; no voters: 
I20260709 12:16:33.849524 25570 raft_consensus.cc:2804] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:33.850345 25570 raft_consensus.cc:697] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [term 1 LEADER]: Becoming Leader. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Running, Role: LEADER
I20260709 12:16:33.851647 25570 consensus_queue.cc:237] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.862498 25584 leader_election.cc:290] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 election: Requested vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305)
I20260709 12:16:33.843743 25564 ts_tablet_manager.cc:1434] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.001s
I20260709 12:16:33.866089 25564 tablet_bootstrap.cc:492] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da: Bootstrap starting.
I20260709 12:16:33.869200 25566 tablet_bootstrap.cc:492] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f: No bootstrap required, opened a new log
I20260709 12:16:33.869846 25566 ts_tablet_manager.cc:1403] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f: Time spent bootstrapping tablet: real 0.079s	user 0.031s	sys 0.023s
I20260709 12:16:33.872471 25448 raft_consensus.cc:2468] T eefc812354034f0c98229c96d8e63846 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 1.
I20260709 12:16:33.873261 25566 raft_consensus.cc:359] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.874091 25566 raft_consensus.cc:385] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.874435 25566 raft_consensus.cc:740] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.875484 25566 consensus_queue.cc:260] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.877935 25597 raft_consensus.cc:493] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.878535 25566 ts_tablet_manager.cc:1434] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f: Time spent starting tablet: real 0.008s	user 0.003s	sys 0.004s
I20260709 12:16:33.878619 25597 raft_consensus.cc:515] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.879856 25566 tablet_bootstrap.cc:492] T 22f011f2c17b4327a825f429b685cd0c P 676bed0fbe614782b3aae7d0db05998f: Bootstrap starting.
I20260709 12:16:33.881934 25571 raft_consensus.cc:493] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.880656 25271 catalog_manager.cc:5697] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f reported cstate change: term changed from 0 to 1, leader changed from <none> to 676bed0fbe614782b3aae7d0db05998f (127.11.13.131). New cstate: current_term: 1 leader_uuid: "676bed0fbe614782b3aae7d0db05998f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:33.882565 25571 raft_consensus.cc:515] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.882817 25524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7411d269e0944ceaa4f08766843e4819" candidate_uuid: "da7a166dda6c4e6580e13213788f18da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676bed0fbe614782b3aae7d0db05998f" is_pre_election: true
W20260709 12:16:33.884613 25339 leader_election.cc:343] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.885102 25571 leader_election.cc:290] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:33.886502 25524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22bd5dc8f9bc4e43ae6cfd2cfd9d1086" candidate_uuid: "da7a166dda6c4e6580e13213788f18da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676bed0fbe614782b3aae7d0db05998f" is_pre_election: true
I20260709 12:16:33.888525 25450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22bd5dc8f9bc4e43ae6cfd2cfd9d1086" candidate_uuid: "da7a166dda6c4e6580e13213788f18da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0987de3c9724390936aa3bd641fa419" is_pre_election: true
I20260709 12:16:33.889603 25566 tablet_bootstrap.cc:654] T 22f011f2c17b4327a825f429b685cd0c P 676bed0fbe614782b3aae7d0db05998f: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:33.890161 25341 leader_election.cc:343] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:33.891544 25339 leader_election.cc:343] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.892014 25339 leader_election.cc:304] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: da7a166dda6c4e6580e13213788f18da; no voters: 676bed0fbe614782b3aae7d0db05998f, d0987de3c9724390936aa3bd641fa419
I20260709 12:16:33.892872 25571 raft_consensus.cc:2749] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:33.886669 25597 leader_election.cc:290] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:33.892400 25564 tablet_bootstrap.cc:654] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.894618 25448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7411d269e0944ceaa4f08766843e4819" candidate_uuid: "da7a166dda6c4e6580e13213788f18da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0987de3c9724390936aa3bd641fa419" is_pre_election: true
I20260709 12:16:33.897701 25565 tablet_bootstrap.cc:492] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419: No bootstrap required, opened a new log
I20260709 12:16:33.898334 25565 ts_tablet_manager.cc:1403] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419: Time spent bootstrapping tablet: real 0.070s	user 0.028s	sys 0.022s
I20260709 12:16:33.895606 25448 raft_consensus.cc:2468] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da7a166dda6c4e6580e13213788f18da in term 0.
I20260709 12:16:33.901618 25341 leader_election.cc:304] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d0987de3c9724390936aa3bd641fa419, da7a166dda6c4e6580e13213788f18da; no voters: 676bed0fbe614782b3aae7d0db05998f
I20260709 12:16:33.901464 25565 raft_consensus.cc:359] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.902868 25571 raft_consensus.cc:2804] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:33.903179 25565 raft_consensus.cc:385] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.903945 25565 raft_consensus.cc:740] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.903965 25571 raft_consensus.cc:493] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:33.904887 25571 raft_consensus.cc:3060] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.905014 25565 consensus_queue.cc:260] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.908806 25565 ts_tablet_manager.cc:1434] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419: Time spent starting tablet: real 0.010s	user 0.007s	sys 0.000s
I20260709 12:16:33.910065 25565 tablet_bootstrap.cc:492] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419: Bootstrap starting.
I20260709 12:16:33.909915 25584 raft_consensus.cc:493] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.910929 25584 raft_consensus.cc:515] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.913892 25566 tablet_bootstrap.cc:492] T 22f011f2c17b4327a825f429b685cd0c P 676bed0fbe614782b3aae7d0db05998f: No bootstrap required, opened a new log
I20260709 12:16:33.914619 25566 ts_tablet_manager.cc:1403] T 22f011f2c17b4327a825f429b685cd0c P 676bed0fbe614782b3aae7d0db05998f: Time spent bootstrapping tablet: real 0.035s	user 0.017s	sys 0.006s
I20260709 12:16:33.917382 25565 tablet_bootstrap.cc:654] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.917649 25566 raft_consensus.cc:359] T 22f011f2c17b4327a825f429b685cd0c P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } }
I20260709 12:16:33.918646 25566 raft_consensus.cc:385] T 22f011f2c17b4327a825f429b685cd0c P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.919045 25566 raft_consensus.cc:740] T 22f011f2c17b4327a825f429b685cd0c P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.920101 25566 consensus_queue.cc:260] T 22f011f2c17b4327a825f429b685cd0c P 676bed0fbe614782b3aae7d0db05998f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } }
I20260709 12:16:33.921154 25374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11dad3c292254bf59bd998aff5a37702" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7a166dda6c4e6580e13213788f18da" is_pre_election: true
I20260709 12:16:33.926385 25374 raft_consensus.cc:2468] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 0.
I20260709 12:16:33.927970 25490 leader_election.cc:304] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, da7a166dda6c4e6580e13213788f18da; no voters: 
I20260709 12:16:33.928952 25570 raft_consensus.cc:2804] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:33.929337 25570 raft_consensus.cc:493] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:33.929750 25570 raft_consensus.cc:3060] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.933434 25566 ts_tablet_manager.cc:1434] T 22f011f2c17b4327a825f429b685cd0c P 676bed0fbe614782b3aae7d0db05998f: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:16:33.934815 25566 tablet_bootstrap.cc:492] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f: Bootstrap starting.
I20260709 12:16:33.946419 25575 raft_consensus.cc:493] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.946723 25566 tablet_bootstrap.cc:654] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:33.947117 25575 raft_consensus.cc:515] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } }
I20260709 12:16:33.949105 25570 raft_consensus.cc:515] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.961385 25374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11dad3c292254bf59bd998aff5a37702" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7a166dda6c4e6580e13213788f18da"
I20260709 12:16:33.962483 25575 leader_election.cc:290] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749), da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415)
I20260709 12:16:33.963359 25524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f011f2c17b4327a825f429b685cd0c" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676bed0fbe614782b3aae7d0db05998f" is_pre_election: true
I20260709 12:16:33.963699 25375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f011f2c17b4327a825f429b685cd0c" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7a166dda6c4e6580e13213788f18da" is_pre_election: true
I20260709 12:16:33.964464 25524 raft_consensus.cc:2468] T 22f011f2c17b4327a825f429b685cd0c P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 0.
I20260709 12:16:33.964979 25564 tablet_bootstrap.cc:492] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da: No bootstrap required, opened a new log
I20260709 12:16:33.965579 25564 ts_tablet_manager.cc:1403] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da: Time spent bootstrapping tablet: real 0.100s	user 0.016s	sys 0.021s
W20260709 12:16:33.965512 25416 leader_election.cc:343] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.962188 25374 raft_consensus.cc:3060] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.969770 25414 leader_election.cc:304] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, d0987de3c9724390936aa3bd641fa419; no voters: da7a166dda6c4e6580e13213788f18da
I20260709 12:16:33.970659 25584 leader_election.cc:290] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305)
I20260709 12:16:33.972482 25448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11dad3c292254bf59bd998aff5a37702" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0987de3c9724390936aa3bd641fa419" is_pre_election: true
I20260709 12:16:33.963192 25575 raft_consensus.cc:493] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:33.974074 25564 raft_consensus.cc:359] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } }
W20260709 12:16:33.974671 25490 leader_election.cc:343] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.975270 25564 raft_consensus.cc:385] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.975162 25571 raft_consensus.cc:515] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.974081 25575 raft_consensus.cc:515] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.976830 25565 tablet_bootstrap.cc:492] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419: No bootstrap required, opened a new log
I20260709 12:16:33.975835 25564 raft_consensus.cc:740] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.979146 25566 tablet_bootstrap.cc:492] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f: No bootstrap required, opened a new log
I20260709 12:16:33.978993 25564 consensus_queue.cc:260] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } }
I20260709 12:16:33.979221 25375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0879bf3038f7407693a853a4eb69d0f1" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7a166dda6c4e6580e13213788f18da" is_pre_election: true
I20260709 12:16:33.980253 25566 ts_tablet_manager.cc:1403] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f: Time spent bootstrapping tablet: real 0.046s	user 0.019s	sys 0.006s
I20260709 12:16:33.980876 25375 raft_consensus.cc:2393] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d0987de3c9724390936aa3bd641fa419 in current term 1: Already voted for candidate 676bed0fbe614782b3aae7d0db05998f in this term.
I20260709 12:16:33.981133 25570 leader_election.cc:290] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 election: Requested vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305)
I20260709 12:16:33.979249 25524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7411d269e0944ceaa4f08766843e4819" candidate_uuid: "da7a166dda6c4e6580e13213788f18da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
I20260709 12:16:33.984021 25566 raft_consensus.cc:359] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.984512 25448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11dad3c292254bf59bd998aff5a37702" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0987de3c9724390936aa3bd641fa419"
I20260709 12:16:33.984808 25566 raft_consensus.cc:385] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:33.985227 25566 raft_consensus.cc:740] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Initialized, Role: FOLLOWER
I20260709 12:16:33.985911 25374 raft_consensus.cc:2468] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 1.
W20260709 12:16:33.986244 25339 leader_election.cc:343] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.986178 25566 consensus_queue.cc:260] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:33.987394 25571 leader_election.cc:290] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [CANDIDATE]: Term 1 election: Requested vote from peers d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:33.980365 25615 raft_consensus.cc:2804] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:33.988397 25615 raft_consensus.cc:493] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:33.988759 25615 raft_consensus.cc:3060] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.989719 25566 ts_tablet_manager.cc:1434] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
W20260709 12:16:33.990705 25490 leader_election.cc:343] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:33.991273 25566 tablet_bootstrap.cc:492] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f: Bootstrap starting.
I20260709 12:16:33.992601 25448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7411d269e0944ceaa4f08766843e4819" candidate_uuid: "da7a166dda6c4e6580e13213788f18da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0987de3c9724390936aa3bd641fa419"
I20260709 12:16:33.993278 25448 raft_consensus.cc:3060] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:33.993281 25570 consensus_queue.cc:1048] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Connected to new peer: Peer: permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:33.995155 25490 leader_election.cc:304] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, da7a166dda6c4e6580e13213788f18da; no voters: d0987de3c9724390936aa3bd641fa419
I20260709 12:16:33.998277 25570 raft_consensus.cc:2804] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:33.999109 25570 raft_consensus.cc:697] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [term 1 LEADER]: Becoming Leader. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Running, Role: LEADER
I20260709 12:16:34.000560 25566 tablet_bootstrap.cc:654] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.003594 25448 raft_consensus.cc:2468] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da7a166dda6c4e6580e13213788f18da in term 1.
I20260709 12:16:34.000566 25570 consensus_queue.cc:237] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:34.005254 25341 leader_election.cc:304] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d0987de3c9724390936aa3bd641fa419, da7a166dda6c4e6580e13213788f18da; no voters: 676bed0fbe614782b3aae7d0db05998f
I20260709 12:16:34.007110 25571 raft_consensus.cc:2804] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:34.008404 25571 raft_consensus.cc:697] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 1 LEADER]: Becoming Leader. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Running, Role: LEADER
I20260709 12:16:34.009121 25524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0879bf3038f7407693a853a4eb69d0f1" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676bed0fbe614782b3aae7d0db05998f" is_pre_election: true
I20260709 12:16:34.009037 25564 ts_tablet_manager.cc:1434] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.001s
I20260709 12:16:34.010808 25414 leader_election.cc:304] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d0987de3c9724390936aa3bd641fa419; no voters: 676bed0fbe614782b3aae7d0db05998f, da7a166dda6c4e6580e13213788f18da
I20260709 12:16:34.011334 25565 ts_tablet_manager.cc:1403] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419: Time spent bootstrapping tablet: real 0.101s	user 0.013s	sys 0.012s
I20260709 12:16:34.011601 25564 tablet_bootstrap.cc:492] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da: Bootstrap starting.
I20260709 12:16:34.010102 25571 consensus_queue.cc:237] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:34.008260 25575 leader_election.cc:290] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:34.018726 25575 raft_consensus.cc:3060] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.020601 25565 raft_consensus.cc:359] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:34.021535 25565 raft_consensus.cc:385] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.021847 25565 raft_consensus.cc:740] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.022791 25565 consensus_queue.cc:260] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:34.025190 25565 ts_tablet_manager.cc:1434] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419: Time spent starting tablet: real 0.013s	user 0.003s	sys 0.003s
I20260709 12:16:34.010255 25615 raft_consensus.cc:515] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } }
I20260709 12:16:34.030215 25271 catalog_manager.cc:5697] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f reported cstate change: term changed from 0 to 1, leader changed from <none> to 676bed0fbe614782b3aae7d0db05998f (127.11.13.131). New cstate: current_term: 1 leader_uuid: "676bed0fbe614782b3aae7d0db05998f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:34.034772 25565 tablet_bootstrap.cc:492] T 11dad3c292254bf59bd998aff5a37702 P d0987de3c9724390936aa3bd641fa419: Bootstrap starting.
I20260709 12:16:34.069008 25564 tablet_bootstrap.cc:654] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.085093 25615 leader_election.cc:290] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 1 election: Requested vote from peers 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749), da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415)
I20260709 12:16:34.087419 25524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f011f2c17b4327a825f429b685cd0c" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
I20260709 12:16:34.088193 25524 raft_consensus.cc:3060] T 22f011f2c17b4327a825f429b685cd0c P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.088611 25575 raft_consensus.cc:2749] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:34.090976 25375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22f011f2c17b4327a825f429b685cd0c" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7a166dda6c4e6580e13213788f18da"
I20260709 12:16:34.091820 25375 raft_consensus.cc:3060] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.093634 25565 tablet_bootstrap.cc:654] T 11dad3c292254bf59bd998aff5a37702 P d0987de3c9724390936aa3bd641fa419: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:34.093091 25271 catalog_manager.cc:5697] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da reported cstate change: term changed from 0 to 1, leader changed from <none> to da7a166dda6c4e6580e13213788f18da (127.11.13.129). New cstate: current_term: 1 leader_uuid: "da7a166dda6c4e6580e13213788f18da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:34.088645 25615 raft_consensus.cc:493] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:34.096282 25615 raft_consensus.cc:515] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:34.099848 25570 consensus_queue.cc:1048] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:34.108574 25564 tablet_bootstrap.cc:492] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da: No bootstrap required, opened a new log
I20260709 12:16:34.109294 25564 ts_tablet_manager.cc:1403] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da: Time spent bootstrapping tablet: real 0.098s	user 0.011s	sys 0.013s
I20260709 12:16:34.109362 25566 tablet_bootstrap.cc:492] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f: No bootstrap required, opened a new log
I20260709 12:16:34.110085 25566 ts_tablet_manager.cc:1403] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f: Time spent bootstrapping tablet: real 0.119s	user 0.025s	sys 0.027s
I20260709 12:16:34.108356 25523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22bd5dc8f9bc4e43ae6cfd2cfd9d1086" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676bed0fbe614782b3aae7d0db05998f" is_pre_election: true
I20260709 12:16:34.112164 25375 raft_consensus.cc:2468] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 1.
I20260709 12:16:34.112809 25564 raft_consensus.cc:359] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:34.112811 25566 raft_consensus.cc:359] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:34.113909 25564 raft_consensus.cc:385] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.114691 25564 raft_consensus.cc:740] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Initialized, Role: FOLLOWER
W20260709 12:16:34.113879 25414 leader_election.cc:343] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:34.115540 25564 consensus_queue.cc:260] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:34.115427 25416 leader_election.cc:304] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0987de3c9724390936aa3bd641fa419, da7a166dda6c4e6580e13213788f18da; no voters: 
I20260709 12:16:34.117398 25575 raft_consensus.cc:2804] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:34.117556 25566 raft_consensus.cc:385] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.118218 25566 raft_consensus.cc:740] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.118264 25575 raft_consensus.cc:697] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 [term 1 LEADER]: Becoming Leader. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Running, Role: LEADER
I20260709 12:16:34.120724 25374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22bd5dc8f9bc4e43ae6cfd2cfd9d1086" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7a166dda6c4e6580e13213788f18da" is_pre_election: true
I20260709 12:16:34.121657 25374 raft_consensus.cc:2468] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 0.
I20260709 12:16:34.123165 25416 leader_election.cc:304] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: d0987de3c9724390936aa3bd641fa419, da7a166dda6c4e6580e13213788f18da; no voters: 676bed0fbe614782b3aae7d0db05998f
I20260709 12:16:34.124042 25615 leader_election.cc:290] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:34.124388 25620 raft_consensus.cc:2804] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:34.124825 25620 raft_consensus.cc:493] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:34.125171 25620 raft_consensus.cc:3060] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.131268 25584 raft_consensus.cc:493] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:34.132082 25584 raft_consensus.cc:515] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:34.119521 25575 consensus_queue.cc:237] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } }
I20260709 12:16:34.162752 25566 consensus_queue.cc:260] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:34.164294 25564 ts_tablet_manager.cc:1434] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da: Time spent starting tablet: real 0.054s	user 0.007s	sys 0.000s
I20260709 12:16:34.166852 25524 raft_consensus.cc:2468] T 22f011f2c17b4327a825f429b685cd0c P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 1.
I20260709 12:16:34.168051 25566 ts_tablet_manager.cc:1434] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f: Time spent starting tablet: real 0.058s	user 0.019s	sys 0.003s
I20260709 12:16:34.170432 25565 tablet_bootstrap.cc:492] T 11dad3c292254bf59bd998aff5a37702 P d0987de3c9724390936aa3bd641fa419: No bootstrap required, opened a new log
I20260709 12:16:34.171115 25565 ts_tablet_manager.cc:1403] T 11dad3c292254bf59bd998aff5a37702 P d0987de3c9724390936aa3bd641fa419: Time spent bootstrapping tablet: real 0.139s	user 0.007s	sys 0.018s
I20260709 12:16:34.174028 25565 raft_consensus.cc:359] T 11dad3c292254bf59bd998aff5a37702 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:34.174798 25565 raft_consensus.cc:385] T 11dad3c292254bf59bd998aff5a37702 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:34.175091 25565 raft_consensus.cc:740] T 11dad3c292254bf59bd998aff5a37702 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Initialized, Role: FOLLOWER
I20260709 12:16:34.175673 25570 consensus_queue.cc:1048] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Connected to new peer: Peer: permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:34.175976 25565 consensus_queue.cc:260] T 11dad3c292254bf59bd998aff5a37702 P d0987de3c9724390936aa3bd641fa419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:34.176735 25450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7411d269e0944ceaa4f08766843e4819" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0987de3c9724390936aa3bd641fa419" is_pre_election: true
I20260709 12:16:34.177735 25584 leader_election.cc:290] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305)
I20260709 12:16:34.177965 25450 raft_consensus.cc:2393] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 676bed0fbe614782b3aae7d0db05998f in current term 1: Already voted for candidate da7a166dda6c4e6580e13213788f18da in this term.
I20260709 12:16:34.178618 25565 ts_tablet_manager.cc:1434] T 11dad3c292254bf59bd998aff5a37702 P d0987de3c9724390936aa3bd641fa419: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:16:34.179692 25374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7411d269e0944ceaa4f08766843e4819" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7a166dda6c4e6580e13213788f18da" is_pre_election: true
I20260709 12:16:34.183940 25490 leader_election.cc:304] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f; no voters: d0987de3c9724390936aa3bd641fa419, da7a166dda6c4e6580e13213788f18da
I20260709 12:16:34.185186 25584 raft_consensus.cc:3060] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.185256 25571 consensus_queue.cc:1048] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [LEADER]: Connected to new peer: Peer: permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:34.194731 25584 raft_consensus.cc:2749] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:34.218101 25632 consensus_queue.cc:1048] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:34.221150 25570 consensus_queue.cc:1048] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:34.244390 25620 raft_consensus.cc:515] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:34.246440 25594 consensus_queue.cc:1048] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:34.250198 25375 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22bd5dc8f9bc4e43ae6cfd2cfd9d1086" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7a166dda6c4e6580e13213788f18da"
I20260709 12:16:34.251052 25375 raft_consensus.cc:3060] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.254197 25524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22bd5dc8f9bc4e43ae6cfd2cfd9d1086" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
I20260709 12:16:34.255275 25524 raft_consensus.cc:3060] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.262822 25375 raft_consensus.cc:2468] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 1.
I20260709 12:16:34.262542 25271 catalog_manager.cc:5697] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 reported cstate change: term changed from 0 to 1, leader changed from <none> to d0987de3c9724390936aa3bd641fa419 (127.11.13.130). New cstate: current_term: 1 leader_uuid: "d0987de3c9724390936aa3bd641fa419" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:34.264048 25620 leader_election.cc:290] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 1 election: Requested vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:34.264958 25524 raft_consensus.cc:2468] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 1.
I20260709 12:16:34.264909 25416 leader_election.cc:304] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0987de3c9724390936aa3bd641fa419, da7a166dda6c4e6580e13213788f18da; no voters: 
I20260709 12:16:34.266559 25620 raft_consensus.cc:2804] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:34.268306 25620 raft_consensus.cc:697] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [term 1 LEADER]: Becoming Leader. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Running, Role: LEADER
I20260709 12:16:34.268621 25592 consensus_queue.cc:1048] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:34.269276 25620 consensus_queue.cc:237] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:34.270496 25632 raft_consensus.cc:493] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:34.271050 25632 raft_consensus.cc:515] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:34.273346 25632 leader_election.cc:290] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:34.275056 25450 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1825085af74858b5ae5b68c999100b" candidate_uuid: "da7a166dda6c4e6580e13213788f18da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0987de3c9724390936aa3bd641fa419" is_pre_election: true
I20260709 12:16:34.275295 25524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1825085af74858b5ae5b68c999100b" candidate_uuid: "da7a166dda6c4e6580e13213788f18da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676bed0fbe614782b3aae7d0db05998f" is_pre_election: true
I20260709 12:16:34.275890 25450 raft_consensus.cc:2468] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da7a166dda6c4e6580e13213788f18da in term 0.
I20260709 12:16:34.276804 25592 consensus_queue.cc:1048] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Connected to new peer: Peer: permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:34.277220 25341 leader_election.cc:304] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0987de3c9724390936aa3bd641fa419, da7a166dda6c4e6580e13213788f18da; no voters: 
I20260709 12:16:34.278239 25632 raft_consensus.cc:2804] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:34.275977 25524 raft_consensus.cc:2468] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate da7a166dda6c4e6580e13213788f18da in term 0.
I20260709 12:16:34.278613 25632 raft_consensus.cc:493] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:34.281879 25632 raft_consensus.cc:3060] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.289070 25632 raft_consensus.cc:515] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:34.291201 25632 leader_election.cc:290] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [CANDIDATE]: Term 1 election: Requested vote from peers d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:34.292016 25448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1825085af74858b5ae5b68c999100b" candidate_uuid: "da7a166dda6c4e6580e13213788f18da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d0987de3c9724390936aa3bd641fa419"
I20260709 12:16:34.292695 25448 raft_consensus.cc:3060] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.292582 25524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1825085af74858b5ae5b68c999100b" candidate_uuid: "da7a166dda6c4e6580e13213788f18da" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
I20260709 12:16:34.295084 25524 raft_consensus.cc:3060] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.294437 25620 raft_consensus.cc:493] T 11dad3c292254bf59bd998aff5a37702 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:34.299973 25620 raft_consensus.cc:515] T 11dad3c292254bf59bd998aff5a37702 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:34.302541 25620 leader_election.cc:290] T 11dad3c292254bf59bd998aff5a37702 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:34.299988 25584 consensus_queue.cc:1048] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Connected to new peer: Peer: permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260709 12:16:34.304399 25523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11dad3c292254bf59bd998aff5a37702" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676bed0fbe614782b3aae7d0db05998f" is_pre_election: true
I20260709 12:16:34.305562 25374 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "11dad3c292254bf59bd998aff5a37702" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "da7a166dda6c4e6580e13213788f18da" is_pre_election: true
I20260709 12:16:34.306424 25374 raft_consensus.cc:2393] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d0987de3c9724390936aa3bd641fa419 in current term 1: Already voted for candidate 676bed0fbe614782b3aae7d0db05998f in this term.
I20260709 12:16:34.309352 25414 leader_election.cc:304] T 11dad3c292254bf59bd998aff5a37702 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d0987de3c9724390936aa3bd641fa419; no voters: 676bed0fbe614782b3aae7d0db05998f, da7a166dda6c4e6580e13213788f18da
I20260709 12:16:34.311621 25524 raft_consensus.cc:2468] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da7a166dda6c4e6580e13213788f18da in term 1.
I20260709 12:16:34.314332 25620 raft_consensus.cc:3060] T 11dad3c292254bf59bd998aff5a37702 P d0987de3c9724390936aa3bd641fa419 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:34.314869 25448 raft_consensus.cc:2468] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate da7a166dda6c4e6580e13213788f18da in term 1.
I20260709 12:16:34.318370 25341 leader_election.cc:304] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0987de3c9724390936aa3bd641fa419, da7a166dda6c4e6580e13213788f18da; no voters: 
I20260709 12:16:34.320034 25632 raft_consensus.cc:2804] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:34.320815 25632 raft_consensus.cc:697] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 1 LEADER]: Becoming Leader. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Running, Role: LEADER
I20260709 12:16:34.321216 25273 catalog_manager.cc:5697] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 reported cstate change: term changed from 0 to 1, leader changed from <none> to d0987de3c9724390936aa3bd641fa419 (127.11.13.130). New cstate: current_term: 1 leader_uuid: "d0987de3c9724390936aa3bd641fa419" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:34.322203 25632 consensus_queue.cc:237] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:34.336122 25620 raft_consensus.cc:2749] T 11dad3c292254bf59bd998aff5a37702 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:34.343510 25321 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:34.347970 25271 catalog_manager.cc:5697] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da reported cstate change: term changed from 0 to 1, leader changed from <none> to da7a166dda6c4e6580e13213788f18da (127.11.13.129). New cstate: current_term: 1 leader_uuid: "da7a166dda6c4e6580e13213788f18da" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:34.378607 11318 ts_manager.cc:295] Set tserver state for da7a166dda6c4e6580e13213788f18da to MAINTENANCE_MODE
I20260709 12:16:34.379592 11318 tablet_server.cc:179] TabletServer@127.11.13.129:0 shutting down...
I20260709 12:16:34.410960 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:34.412106 11318 tablet_replica.cc:333] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da: stopping tablet replica
I20260709 12:16:34.413162 11318 raft_consensus.cc:2243] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:34.413640 11318 raft_consensus.cc:2272] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:34.415951 11318 tablet_replica.cc:333] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da: stopping tablet replica
I20260709 12:16:34.416565 11318 raft_consensus.cc:2243] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:34.416975 11318 raft_consensus.cc:2272] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:34.418962 11318 tablet_replica.cc:333] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da: stopping tablet replica
I20260709 12:16:34.419494 11318 raft_consensus.cc:2243] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:34.420236 11318 pending_rounds.cc:70] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da: Trying to abort 1 pending ops.
I20260709 12:16:34.420523 11318 pending_rounds.cc:77] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305623117900615680 op_type: NO_OP noop_request { }
I20260709 12:16:34.420971 11318 raft_consensus.cc:2889] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:16:34.421370 11318 raft_consensus.cc:2272] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:34.424623 11318 tablet_replica.cc:333] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da: stopping tablet replica
I20260709 12:16:34.425631 11318 raft_consensus.cc:2243] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:34.426193 11318 raft_consensus.cc:2272] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:34.428727 11318 tablet_replica.cc:333] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da: stopping tablet replica
I20260709 12:16:34.429617 11318 raft_consensus.cc:2243] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:34.430265 11318 raft_consensus.cc:2272] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:34.431933 25584 consensus_queue.cc:1048] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:34.433852 11318 tablet_replica.cc:333] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da: stopping tablet replica
I20260709 12:16:34.434723 11318 raft_consensus.cc:2243] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:34.435815 11318 raft_consensus.cc:2272] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:34.438598 11318 tablet_replica.cc:333] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da: stopping tablet replica
I20260709 12:16:34.439344 11318 raft_consensus.cc:2243] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:34.439875 11318 raft_consensus.cc:2272] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:34.443660 11318 tablet_replica.cc:333] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da: stopping tablet replica
I20260709 12:16:34.446022 11318 raft_consensus.cc:2243] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:34.447077 11318 pending_rounds.cc:70] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da: Trying to abort 1 pending ops.
I20260709 12:16:34.447585 11318 pending_rounds.cc:77] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7305623119164071936 op_type: NO_OP noop_request { }
I20260709 12:16:34.448194 11318 raft_consensus.cc:2889] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 1 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:16:34.448618 11318 raft_consensus.cc:2272] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:34.452749 11318 tablet_replica.cc:333] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da: stopping tablet replica
I20260709 12:16:34.453900 11318 raft_consensus.cc:2243] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:34.454736 11318 raft_consensus.cc:2272] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:34.458817 11318 tablet_replica.cc:333] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da: stopping tablet replica
I20260709 12:16:34.459600 11318 raft_consensus.cc:2243] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:34.460096 11318 raft_consensus.cc:2272] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:34.460493 25490 consensus_peers.cc:597] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:34.494295 11318 tablet_server.cc:196] TabletServer@127.11.13.129:0 shutdown complete.
I20260709 12:16:34.537936 25620 consensus_queue.cc:1048] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260709 12:16:34.557232 25416 consensus_peers.cc:597] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:34.575312 25490 consensus_peers.cc:597] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:34.608973 25490 consensus_peers.cc:597] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:34.669384 25416 consensus_peers.cc:597] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:34.791561 25490 consensus_peers.cc:597] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:34.814733 25575 consensus_queue.cc:1048] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:34.873441 25490 consensus_peers.cc:597] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:35.043424 25416 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111) [suppressed 143 similar messages]
W20260709 12:16:35.344693 25321 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:16:35.698776 25575 raft_consensus.cc:493] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:35.699425 25575 raft_consensus.cc:515] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:35.701413 25575 leader_election.cc:290] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:35.702796 25524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7411d269e0944ceaa4f08766843e4819" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676bed0fbe614782b3aae7d0db05998f" is_pre_election: true
I20260709 12:16:35.703553 25524 raft_consensus.cc:2468] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 1.
I20260709 12:16:35.704880 25414 leader_election.cc:304] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, d0987de3c9724390936aa3bd641fa419; no voters: 
I20260709 12:16:35.705579 25575 raft_consensus.cc:2804] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:16:35.705611 25416 leader_election.cc:336] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111)
I20260709 12:16:35.705955 25575 raft_consensus.cc:493] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:35.706354 25575 raft_consensus.cc:3060] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:35.712239 25575 raft_consensus.cc:515] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:35.714390 25575 leader_election.cc:290] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 2 election: Requested vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:35.715728 25524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7411d269e0944ceaa4f08766843e4819" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
I20260709 12:16:35.716701 25570 raft_consensus.cc:493] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da7a166dda6c4e6580e13213788f18da)
I20260709 12:16:35.717653 25570 raft_consensus.cc:515] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:35.718665 25524 raft_consensus.cc:3060] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:16:35.720366 25416 leader_election.cc:336] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111)
I20260709 12:16:35.720738 25570 leader_election.cc:290] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305)
I20260709 12:16:35.722950 25575 raft_consensus.cc:493] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader da7a166dda6c4e6580e13213788f18da)
I20260709 12:16:35.723630 25448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14e62bfdb1564317b3ec26702c9b0172" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d0987de3c9724390936aa3bd641fa419" is_pre_election: true
I20260709 12:16:35.723958 25575 raft_consensus.cc:515] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:35.725445 25448 raft_consensus.cc:2468] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 1.
W20260709 12:16:35.725791 25490 leader_election.cc:336] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111)
I20260709 12:16:35.727596 25575 leader_election.cc:290] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:35.727666 25490 leader_election.cc:304] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, d0987de3c9724390936aa3bd641fa419; no voters: da7a166dda6c4e6580e13213788f18da
I20260709 12:16:35.728698 25570 raft_consensus.cc:2804] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260709 12:16:35.729127 25570 raft_consensus.cc:493] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Starting leader election (detected failure of leader da7a166dda6c4e6580e13213788f18da)
I20260709 12:16:35.729586 25570 raft_consensus.cc:3060] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:35.729691 25524 raft_consensus.cc:2468] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 2.
I20260709 12:16:35.729796 25523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14e62bfdb1564317b3ec26702c9b0172" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "676bed0fbe614782b3aae7d0db05998f" is_pre_election: true
I20260709 12:16:35.731578 25414 leader_election.cc:304] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, d0987de3c9724390936aa3bd641fa419; no voters: da7a166dda6c4e6580e13213788f18da
I20260709 12:16:35.732590 25575 raft_consensus.cc:2804] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 2 FOLLOWER]: Leader election won for term 2
W20260709 12:16:35.732815 25416 leader_election.cc:336] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111)
I20260709 12:16:35.733909 25575 raft_consensus.cc:697] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 2 LEADER]: Becoming Leader. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Running, Role: LEADER
I20260709 12:16:35.735770 25575 consensus_queue.cc:237] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:35.738579 25570 raft_consensus.cc:515] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:35.740165 25523 raft_consensus.cc:2393] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d0987de3c9724390936aa3bd641fa419 in current term 2: Already voted for candidate 676bed0fbe614782b3aae7d0db05998f in this term.
I20260709 12:16:35.742002 25570 leader_election.cc:290] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 2 election: Requested vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305)
I20260709 12:16:35.743707 25414 leader_election.cc:304] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d0987de3c9724390936aa3bd641fa419; no voters: 676bed0fbe614782b3aae7d0db05998f, da7a166dda6c4e6580e13213788f18da
I20260709 12:16:35.744678 25448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14e62bfdb1564317b3ec26702c9b0172" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d0987de3c9724390936aa3bd641fa419"
I20260709 12:16:35.745428 25448 raft_consensus.cc:3060] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:16:35.747267 25490 leader_election.cc:336] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111)
I20260709 12:16:35.747949 25271 catalog_manager.cc:5697] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 reported cstate change: term changed from 1 to 2, leader changed from da7a166dda6c4e6580e13213788f18da (127.11.13.129) to d0987de3c9724390936aa3bd641fa419 (127.11.13.130). New cstate: current_term: 2 leader_uuid: "d0987de3c9724390936aa3bd641fa419" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:35.752594 25448 raft_consensus.cc:2468] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 2.
I20260709 12:16:35.753258 25575 raft_consensus.cc:2749] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:16:35.754881 25490 leader_election.cc:304] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, d0987de3c9724390936aa3bd641fa419; no voters: da7a166dda6c4e6580e13213788f18da
I20260709 12:16:35.755988 25570 raft_consensus.cc:2804] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:35.756520 25570 raft_consensus.cc:697] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 2 LEADER]: Becoming Leader. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Running, Role: LEADER
I20260709 12:16:35.757642 25570 consensus_queue.cc:237] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:35.771965 25271 catalog_manager.cc:5697] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f reported cstate change: term changed from 1 to 2, leader changed from da7a166dda6c4e6580e13213788f18da (127.11.13.129) to 676bed0fbe614782b3aae7d0db05998f (127.11.13.131). New cstate: current_term: 2 leader_uuid: "676bed0fbe614782b3aae7d0db05998f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:36.124788 25416 consensus_peers.cc:597] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:36.142486 25490 consensus_peers.cc:597] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:36.238492 25575 raft_consensus.cc:493] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:36.239070 25575 raft_consensus.cc:515] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:36.241039 25575 leader_election.cc:290] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:36.242339 25523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1825085af74858b5ae5b68c999100b" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676bed0fbe614782b3aae7d0db05998f" is_pre_election: true
I20260709 12:16:36.243070 25523 raft_consensus.cc:2468] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 1.
I20260709 12:16:36.244205 25414 leader_election.cc:304] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, d0987de3c9724390936aa3bd641fa419; no voters: 
I20260709 12:16:36.245118 25575 raft_consensus.cc:2804] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260709 12:16:36.245164 25416 leader_election.cc:336] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111)
I20260709 12:16:36.245669 25575 raft_consensus.cc:493] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:36.246058 25575 raft_consensus.cc:3060] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:36.251869 25575 raft_consensus.cc:515] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:36.253854 25575 leader_election.cc:290] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 2 election: Requested vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:36.255475 25523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1825085af74858b5ae5b68c999100b" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
I20260709 12:16:36.256390 25523 raft_consensus.cc:3060] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:36.259719 25450 raft_consensus.cc:1275] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 2 FOLLOWER]: Refusing update from remote peer 676bed0fbe614782b3aae7d0db05998f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260709 12:16:36.259651 25416 leader_election.cc:336] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111)
I20260709 12:16:36.262410 25570 consensus_queue.cc:1048] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:36.265707 25523 raft_consensus.cc:2468] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 2.
I20260709 12:16:36.267210 25414 leader_election.cc:304] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, d0987de3c9724390936aa3bd641fa419; no voters: da7a166dda6c4e6580e13213788f18da
I20260709 12:16:36.268230 25575 raft_consensus.cc:2804] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:36.269680 25575 raft_consensus.cc:697] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 2 LEADER]: Becoming Leader. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Running, Role: LEADER
I20260709 12:16:36.270738 25575 consensus_queue.cc:237] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:36.285550 25271 catalog_manager.cc:5697] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 reported cstate change: term changed from 1 to 2, leader changed from da7a166dda6c4e6580e13213788f18da (127.11.13.129) to d0987de3c9724390936aa3bd641fa419 (127.11.13.130). New cstate: current_term: 2 leader_uuid: "d0987de3c9724390936aa3bd641fa419" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:36.302923 25523 raft_consensus.cc:1275] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Refusing update from remote peer d0987de3c9724390936aa3bd641fa419: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:16:36.304339 25575 consensus_queue.cc:1048] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:36.346349 25321 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:16:36.756440 25523 raft_consensus.cc:1275] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Refusing update from remote peer d0987de3c9724390936aa3bd641fa419: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch)
I20260709 12:16:36.758087 25575 consensus_queue.cc:1048] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260709 12:16:36.773159 25416 consensus_peers.cc:597] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:37.038084 25490 consensus_peers.cc:597] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:37.043895 25416 consensus_peers.cc:597] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:37.054320 25490 consensus_peers.cc:597] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:37.101084 25490 consensus_peers.cc:597] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:37.147145 25416 consensus_peers.cc:597] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:37.300221 25490 consensus_peers.cc:597] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:37.347522 25321 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:16:37.598299 25490 consensus_peers.cc:597] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:38.348871 25321 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:16:38.476439 25490 consensus_peers.cc:597] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:38.624423 25416 consensus_peers.cc:597] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:39.246104 25416 consensus_peers.cc:597] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 6: this message will repeat every 5th retry.
W20260709 12:16:39.345335 25490 consensus_peers.cc:597] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:39.349925 25321 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
W20260709 12:16:39.498311 25490 consensus_peers.cc:597] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:39.519379 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:39.526739 25670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:39.527216 25671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:39.528753 25673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:39.532112 11318 server_base.cc:1061] running on GCE node
I20260709 12:16:39.533602 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:39.533874 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:39.534085 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599399534042 us; error 0 us; skew 500 ppm
I20260709 12:16:39.534830 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:39.537971 11318 webserver.cc:533] Webserver started at http://127.11.13.129:35127/ using document root <none> and password file <none>
I20260709 12:16:39.538852 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:39.539086 11318 fs_manager.cc:365] Using existing metadata directory in first data directory
I20260709 12:16:39.544518 11318 fs_manager.cc:714] Time spent opening directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:16:39.549448 25678 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:39.550557 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:16:39.550935 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/ts-0-root
uuid: "da7a166dda6c4e6580e13213788f18da"
format_stamp: "Formatted at 2026-07-09 12:16:32 on dist-test-slave-zqnl"
I20260709 12:16:39.551231 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.TestMaintenanceMode.1783598892056905-11318-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:39.573998 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:39.575793 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:39.577662 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:39.582304 25685 ts_tablet_manager.cc:548] Loading tablet metadata (0/10 complete)
W20260709 12:16:39.626881 25416 consensus_peers.cc:597] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
W20260709 12:16:39.637706 25416 consensus_peers.cc:597] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:39.648214 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (10 total tablets, 10 live tablets)
I20260709 12:16:39.648752 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.068s	user 0.003s	sys 0.000s
I20260709 12:16:39.649189 11318 ts_tablet_manager.cc:600] Registering tablets (0/10 complete)
I20260709 12:16:39.655745 25685 tablet_bootstrap.cc:492] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da: Bootstrap starting.
I20260709 12:16:39.675112 25685 tablet_bootstrap.cc:492] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:39.676321 25685 tablet_bootstrap.cc:492] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da: Bootstrap complete.
I20260709 12:16:39.677096 25685 ts_tablet_manager.cc:1403] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.000s
I20260709 12:16:39.680085 25685 raft_consensus.cc:359] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.681089 25685 raft_consensus.cc:740] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.682106 25685 consensus_queue.cc:260] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.685647 25685 ts_tablet_manager.cc:1434] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da: Time spent starting tablet: real 0.008s	user 0.010s	sys 0.000s
I20260709 12:16:39.686661 25685 tablet_bootstrap.cc:492] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da: Bootstrap starting.
I20260709 12:16:39.703197 25685 tablet_bootstrap.cc:492] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:39.704123 25685 tablet_bootstrap.cc:492] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da: Bootstrap complete.
I20260709 12:16:39.704738 25685 ts_tablet_manager.cc:1403] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da: Time spent bootstrapping tablet: real 0.018s	user 0.013s	sys 0.002s
I20260709 12:16:39.707304 25685 raft_consensus.cc:359] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.708153 25685 raft_consensus.cc:740] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.708884 25685 consensus_queue.cc:260] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.712165 25685 ts_tablet_manager.cc:1434] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 12:16:39.713274 25685 tablet_bootstrap.cc:492] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da: Bootstrap starting.
W20260709 12:16:39.712810 25490 consensus_peers.cc:597] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:39.716322 11318 ts_tablet_manager.cc:616] Registered 10 tablets
I20260709 12:16:39.716742 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.068s	user 0.063s	sys 0.003s
I20260709 12:16:39.732991 25685 tablet_bootstrap.cc:492] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:39.734066 25685 tablet_bootstrap.cc:492] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da: Bootstrap complete.
I20260709 12:16:39.734854 25685 ts_tablet_manager.cc:1403] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.004s
I20260709 12:16:39.737494 25685 raft_consensus.cc:359] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.738194 25685 raft_consensus.cc:740] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.739131 25685 consensus_queue.cc:260] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.741290 25685 ts_tablet_manager.cc:1434] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da: Time spent starting tablet: real 0.006s	user 0.004s	sys 0.000s
I20260709 12:16:39.742290 25685 tablet_bootstrap.cc:492] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da: Bootstrap starting.
I20260709 12:16:39.753597 25685 tablet_bootstrap.cc:492] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:39.754549 25685 tablet_bootstrap.cc:492] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da: Bootstrap complete.
I20260709 12:16:39.755189 25685 ts_tablet_manager.cc:1403] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da: Time spent bootstrapping tablet: real 0.013s	user 0.010s	sys 0.002s
I20260709 12:16:39.758340 25685 raft_consensus.cc:359] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.759039 25685 raft_consensus.cc:740] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.759859 25685 consensus_queue.cc:260] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.762457 25685 ts_tablet_manager.cc:1434] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.002s
I20260709 12:16:39.763569 25685 tablet_bootstrap.cc:492] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da: Bootstrap starting.
W20260709 12:16:39.769124 25490 consensus_peers.cc:597] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 11: this message will repeat every 5th retry.
I20260709 12:16:39.784487 25685 tablet_bootstrap.cc:492] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:39.785516 25685 tablet_bootstrap.cc:492] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da: Bootstrap complete.
I20260709 12:16:39.786330 25685 ts_tablet_manager.cc:1403] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.012s
I20260709 12:16:39.788714 25685 raft_consensus.cc:359] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.789274 25685 raft_consensus.cc:740] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.790186 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.129:33415
I20260709 12:16:39.790298 25747 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.129:33415 every 8 connection(s)
I20260709 12:16:39.790096 25685 consensus_queue.cc:260] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.792415 25685 ts_tablet_manager.cc:1434] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da: Time spent starting tablet: real 0.006s	user 0.000s	sys 0.004s
I20260709 12:16:39.793329 25685 tablet_bootstrap.cc:492] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da: Bootstrap starting.
I20260709 12:16:39.795461 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:39.822268 25685 tablet_bootstrap.cc:492] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:39.823292 25685 tablet_bootstrap.cc:492] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da: Bootstrap complete.
I20260709 12:16:39.824098 25685 ts_tablet_manager.cc:1403] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da: Time spent bootstrapping tablet: real 0.031s	user 0.016s	sys 0.004s
I20260709 12:16:39.826287 25685 raft_consensus.cc:359] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } }
I20260709 12:16:39.826947 25685 raft_consensus.cc:740] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.827782 25685 consensus_queue.cc:260] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } }
I20260709 12:16:39.829910 25685 ts_tablet_manager.cc:1434] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da: Time spent starting tablet: real 0.006s	user 0.003s	sys 0.004s
I20260709 12:16:39.830803 25685 tablet_bootstrap.cc:492] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da: Bootstrap starting.
I20260709 12:16:39.830834 25523 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14e62bfdb1564317b3ec26702c9b0172"
dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
mode: GRACEFUL
new_leader_uuid: "d0987de3c9724390936aa3bd641fa419"
 from {username='slave'} at 127.0.0.1:33624
I20260709 12:16:39.831734 25523 raft_consensus.cc:606] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 2 LEADER]: Received request to transfer leadership to TS d0987de3c9724390936aa3bd641fa419
I20260709 12:16:39.834556 25748 heartbeater.cc:344] Connected to a master server at 127.11.13.190:40877
I20260709 12:16:39.834594 25523 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "a2f349cdc65547d6a574bbe3563db1a1"
dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
mode: GRACEFUL
new_leader_uuid: "d0987de3c9724390936aa3bd641fa419"
 from {username='slave'} at 127.0.0.1:33624
I20260709 12:16:39.835109 25748 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:39.835492 25523 raft_consensus.cc:606] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [term 1 LEADER]: Received request to transfer leadership to TS d0987de3c9724390936aa3bd641fa419
I20260709 12:16:39.836210 25748 heartbeater.cc:507] Master 127.11.13.190:40877 requested a full tablet report, sending...
I20260709 12:16:39.837062 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:16:39.837541 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:39.838232 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:39.843634 25273 ts_manager.cc:194] Re-registered known tserver with Master: da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415)
I20260709 12:16:39.846423 25685 tablet_bootstrap.cc:492] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:16:39.847633 25685 tablet_bootstrap.cc:492] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da: Bootstrap complete.
I20260709 12:16:39.848382 25685 ts_tablet_manager.cc:1403] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da: Time spent bootstrapping tablet: real 0.018s	user 0.012s	sys 0.002s
I20260709 12:16:39.851065 25685 raft_consensus.cc:359] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.851948 25685 raft_consensus.cc:740] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.852667 25685 consensus_queue.cc:260] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.854777 25685 ts_tablet_manager.cc:1434] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da: Time spent starting tablet: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:16:39.855672 25685 tablet_bootstrap.cc:492] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da: Bootstrap starting.
I20260709 12:16:39.856261 25273 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:39462
I20260709 12:16:39.860872 25748 heartbeater.cc:499] Master 127.11.13.190:40877 was elected leader, sending a full tablet report...
I20260709 12:16:39.863569 25668 raft_consensus.cc:993] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f: : Instructing follower d0987de3c9724390936aa3bd641fa419 to start an election
I20260709 12:16:39.864220 25669 raft_consensus.cc:1081] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [term 1 LEADER]: Signalling peer d0987de3c9724390936aa3bd641fa419 to start an election
I20260709 12:16:39.866185 25448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "a2f349cdc65547d6a574bbe3563db1a1"
dest_uuid: "d0987de3c9724390936aa3bd641fa419"
 from {username='slave'} at 127.0.0.1:38720
I20260709 12:16:39.867055 25448 raft_consensus.cc:493] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:39.867576 25448 raft_consensus.cc:3060] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:39.875605 25685 tablet_bootstrap.cc:492] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=1 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:39.877195 25685 tablet_bootstrap.cc:492] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da: Bootstrap complete.
I20260709 12:16:39.877557 25448 raft_consensus.cc:515] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.878317 25685 ts_tablet_manager.cc:1403] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da: Time spent bootstrapping tablet: real 0.023s	user 0.018s	sys 0.004s
I20260709 12:16:39.882071 25685 raft_consensus.cc:359] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.883101 25685 raft_consensus.cc:740] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.884231 25685 consensus_queue.cc:260] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.885011 25448 leader_election.cc:290] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 2 election: Requested vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:39.885732 25523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f349cdc65547d6a574bbe3563db1a1" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
I20260709 12:16:39.888156 25523 raft_consensus.cc:3055] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:39.888521 25523 raft_consensus.cc:740] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Running, Role: LEADER
I20260709 12:16:39.889516 25523 consensus_queue.cc:260] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.890630 25523 raft_consensus.cc:3060] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:39.886945 25685 ts_tablet_manager.cc:1434] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.004s
I20260709 12:16:39.895468 25685 tablet_bootstrap.cc:492] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da: Bootstrap starting.
I20260709 12:16:39.900933 25523 raft_consensus.cc:2468] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 2.
I20260709 12:16:39.902518 25414 leader_election.cc:304] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, d0987de3c9724390936aa3bd641fa419; no voters: 
I20260709 12:16:39.903540 25657 raft_consensus.cc:2804] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:39.912003 25657 raft_consensus.cc:697] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419 [term 2 LEADER]: Becoming Leader. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Running, Role: LEADER
I20260709 12:16:39.913254 25657 consensus_queue.cc:237] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.920348 25722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "a2f349cdc65547d6a574bbe3563db1a1" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da7a166dda6c4e6580e13213788f18da"
I20260709 12:16:39.921303 25722 raft_consensus.cc:3060] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:39.931083 25685 tablet_bootstrap.cc:492] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da: Bootstrap replayed 1/1 log segments. Stats: ops{read=0 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260709 12:16:39.932230 25722 raft_consensus.cc:2468] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 2.
I20260709 12:16:39.932531 25685 tablet_bootstrap.cc:492] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da: Bootstrap complete.
I20260709 12:16:39.936882 25685 ts_tablet_manager.cc:1403] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da: Time spent bootstrapping tablet: real 0.042s	user 0.013s	sys 0.016s
I20260709 12:16:39.940335 25685 raft_consensus.cc:359] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.941466 25685 raft_consensus.cc:740] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.942312 25685 consensus_queue.cc:260] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.942410 25271 catalog_manager.cc:5697] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419 reported cstate change: term changed from 1 to 2, leader changed from 676bed0fbe614782b3aae7d0db05998f (127.11.13.131) to d0987de3c9724390936aa3bd641fa419 (127.11.13.130). New cstate: current_term: 2 leader_uuid: "d0987de3c9724390936aa3bd641fa419" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:39.944509 25685 ts_tablet_manager.cc:1434] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da: Time spent starting tablet: real 0.007s	user 0.002s	sys 0.003s
I20260709 12:16:39.945324 25685 tablet_bootstrap.cc:492] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da: Bootstrap starting.
I20260709 12:16:39.956619 25668 raft_consensus.cc:993] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f: : Instructing follower d0987de3c9724390936aa3bd641fa419 to start an election
I20260709 12:16:39.957226 25668 raft_consensus.cc:1081] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 2 LEADER]: Signalling peer d0987de3c9724390936aa3bd641fa419 to start an election
I20260709 12:16:39.959153 25448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "14e62bfdb1564317b3ec26702c9b0172"
dest_uuid: "d0987de3c9724390936aa3bd641fa419"
 from {username='slave'} at 127.0.0.1:38720
I20260709 12:16:39.959875 25448 raft_consensus.cc:493] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:39.960414 25448 raft_consensus.cc:3060] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:39.963438 25685 tablet_bootstrap.cc:492] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da: Bootstrap replayed 1/1 log segments. Stats: ops{read=1 overwritten=0 applied=0 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 1 replicates
I20260709 12:16:39.964609 25685 tablet_bootstrap.cc:492] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da: Bootstrap complete.
I20260709 12:16:39.965211 25685 ts_tablet_manager.cc:1403] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da: Time spent bootstrapping tablet: real 0.020s	user 0.017s	sys 0.000s
I20260709 12:16:39.967486 25685 raft_consensus.cc:359] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Replica starting. Triggering 1 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.968161 25685 raft_consensus.cc:740] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: da7a166dda6c4e6580e13213788f18da, State: Initialized, Role: FOLLOWER
I20260709 12:16:39.968827 25448 raft_consensus.cc:515] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.968860 25685 consensus_queue.cc:260] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.970886 25685 ts_tablet_manager.cc:1434] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da: Time spent starting tablet: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:16:39.971411 25448 leader_election.cc:290] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 3 election: Requested vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:39.972383 25722 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14e62bfdb1564317b3ec26702c9b0172" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "da7a166dda6c4e6580e13213788f18da"
I20260709 12:16:39.973047 25722 raft_consensus.cc:3060] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Advancing to term 3
I20260709 12:16:39.973132 25523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14e62bfdb1564317b3ec26702c9b0172" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
I20260709 12:16:39.973860 25523 raft_consensus.cc:3055] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:39.974238 25523 raft_consensus.cc:740] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Running, Role: LEADER
I20260709 12:16:39.975111 25523 consensus_queue.cc:260] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.976486 25523 raft_consensus.cc:3060] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:39.981770 25722 raft_consensus.cc:2468] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 3.
I20260709 12:16:39.982664 25523 raft_consensus.cc:2468] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 3.
I20260709 12:16:39.983002 25416 leader_election.cc:304] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0987de3c9724390936aa3bd641fa419, da7a166dda6c4e6580e13213788f18da; no voters: 
I20260709 12:16:39.983968 25657 raft_consensus.cc:2804] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:39.984417 25657 raft_consensus.cc:697] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 3 LEADER]: Becoming Leader. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Running, Role: LEADER
I20260709 12:16:39.985618 25657 consensus_queue.cc:237] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:39.993671 25271 catalog_manager.cc:5697] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 reported cstate change: term changed from 2 to 3, leader changed from 676bed0fbe614782b3aae7d0db05998f (127.11.13.131) to d0987de3c9724390936aa3bd641fa419 (127.11.13.130). New cstate: current_term: 3 leader_uuid: "d0987de3c9724390936aa3bd641fa419" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:40.317664 25720 raft_consensus.cc:3060] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:40.319509 25721 raft_consensus.cc:3060] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Advancing to term 2
W20260709 12:16:40.351202 25321 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:16:40.352370 25721 pending_rounds.cc:85] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da: Aborting all ops after (but not including) 0
I20260709 12:16:40.352803 25721 pending_rounds.cc:107] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:16:40.353235 25721 raft_consensus.cc:2889] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:40.353722 25720 pending_rounds.cc:85] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da: Aborting all ops after (but not including) 0
I20260709 12:16:40.354417 25720 pending_rounds.cc:107] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260709 12:16:40.355291 25720 raft_consensus.cc:2889] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:16:40.362716 25523 raft_consensus.cc:1275] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Refusing update from remote peer d0987de3c9724390936aa3bd641fa419: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:40.369257 25657 consensus_queue.cc:1048] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:40.375573 25721 raft_consensus.cc:1275] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da [term 2 FOLLOWER]: Refusing update from remote peer d0987de3c9724390936aa3bd641fa419: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:40.377591 25657 consensus_queue.cc:1048] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:40.440793 25523 raft_consensus.cc:1275] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 3 FOLLOWER]: Refusing update from remote peer d0987de3c9724390936aa3bd641fa419: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:40.442606 25657 consensus_queue.cc:1048] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:40.453104 25721 raft_consensus.cc:1275] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 3 FOLLOWER]: Refusing update from remote peer d0987de3c9724390936aa3bd641fa419: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:40.456221 25759 consensus_queue.cc:1048] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:40.939476 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:40.960707 25448 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "22bd5dc8f9bc4e43ae6cfd2cfd9d1086"
dest_uuid: "d0987de3c9724390936aa3bd641fa419"
mode: GRACEFUL
new_leader_uuid: "676bed0fbe614782b3aae7d0db05998f"
 from {username='slave'} at 127.0.0.1:38728
I20260709 12:16:40.961382 25448 raft_consensus.cc:606] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [term 1 LEADER]: Received request to transfer leadership to TS 676bed0fbe614782b3aae7d0db05998f
I20260709 12:16:40.964443 25448 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7411d269e0944ceaa4f08766843e4819"
dest_uuid: "d0987de3c9724390936aa3bd641fa419"
mode: GRACEFUL
new_leader_uuid: "676bed0fbe614782b3aae7d0db05998f"
 from {username='slave'} at 127.0.0.1:38728
I20260709 12:16:40.965018 25448 raft_consensus.cc:606] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 2 LEADER]: Received request to transfer leadership to TS 676bed0fbe614782b3aae7d0db05998f
I20260709 12:16:40.966194 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:16:40.966741 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:40.967454 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:41.047178 25657 raft_consensus.cc:993] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419: : Instructing follower 676bed0fbe614782b3aae7d0db05998f to start an election
I20260709 12:16:41.047678 25759 raft_consensus.cc:1081] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [term 1 LEADER]: Signalling peer 676bed0fbe614782b3aae7d0db05998f to start an election
I20260709 12:16:41.049340 25523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "22bd5dc8f9bc4e43ae6cfd2cfd9d1086"
dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
 from {username='slave'} at 127.0.0.1:33616
I20260709 12:16:41.049911 25523 raft_consensus.cc:493] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:41.050290 25523 raft_consensus.cc:3060] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:41.056633 25523 raft_consensus.cc:515] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:41.058593 25523 leader_election.cc:290] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 2 election: Requested vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305)
I20260709 12:16:41.059433 25720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22bd5dc8f9bc4e43ae6cfd2cfd9d1086" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da7a166dda6c4e6580e13213788f18da"
I20260709 12:16:41.059706 25448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "22bd5dc8f9bc4e43ae6cfd2cfd9d1086" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d0987de3c9724390936aa3bd641fa419"
I20260709 12:16:41.060184 25720 raft_consensus.cc:3060] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:41.060390 25448 raft_consensus.cc:3055] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:41.060813 25448 raft_consensus.cc:740] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Running, Role: LEADER
I20260709 12:16:41.061601 25448 consensus_queue.cc:260] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:41.062646 25448 raft_consensus.cc:3060] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:41.066433 25720 raft_consensus.cc:2468] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 2.
I20260709 12:16:41.067894 25490 leader_election.cc:304] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, da7a166dda6c4e6580e13213788f18da; no voters: 
I20260709 12:16:41.068189 25448 raft_consensus.cc:2468] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 2.
I20260709 12:16:41.068766 25773 raft_consensus.cc:2804] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:41.069320 25773 raft_consensus.cc:697] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f [term 2 LEADER]: Becoming Leader. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Running, Role: LEADER
I20260709 12:16:41.070235 25773 consensus_queue.cc:237] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:41.079986 25271 catalog_manager.cc:5697] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f reported cstate change: term changed from 1 to 2, leader changed from d0987de3c9724390936aa3bd641fa419 (127.11.13.130) to 676bed0fbe614782b3aae7d0db05998f (127.11.13.131). New cstate: current_term: 2 leader_uuid: "676bed0fbe614782b3aae7d0db05998f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:41.167685 25304 debug-util.cc:398] Leaking SignalData structure 0x7b0800134520 after lost signal to thread 11321
W20260709 12:16:41.168888 25304 debug-util.cc:398] Leaking SignalData structure 0x7b080005fe40 after lost signal to thread 25307
W20260709 12:16:41.170985 25304 debug-util.cc:398] Leaking SignalData structure 0x7b08001532e0 after lost signal to thread 25548
W20260709 12:16:41.172303 25304 debug-util.cc:398] Leaking SignalData structure 0x7b0800310640 after lost signal to thread 25747
I20260709 12:16:41.223258 25657 raft_consensus.cc:993] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419: : Instructing follower 676bed0fbe614782b3aae7d0db05998f to start an election
I20260709 12:16:41.223912 25759 raft_consensus.cc:1081] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 2 LEADER]: Signalling peer 676bed0fbe614782b3aae7d0db05998f to start an election
I20260709 12:16:41.226231 25523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7411d269e0944ceaa4f08766843e4819"
dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
 from {username='slave'} at 127.0.0.1:33616
I20260709 12:16:41.227124 25523 raft_consensus.cc:493] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:41.227512 25523 raft_consensus.cc:3060] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:41.233598 25523 raft_consensus.cc:515] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:41.235342 25523 leader_election.cc:290] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 3 election: Requested vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305)
I20260709 12:16:41.236400 25720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7411d269e0944ceaa4f08766843e4819" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "da7a166dda6c4e6580e13213788f18da"
I20260709 12:16:41.236691 25448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7411d269e0944ceaa4f08766843e4819" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "d0987de3c9724390936aa3bd641fa419"
I20260709 12:16:41.237349 25720 raft_consensus.cc:3060] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:41.237519 25448 raft_consensus.cc:3055] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:41.237913 25448 raft_consensus.cc:740] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Running, Role: LEADER
I20260709 12:16:41.238781 25448 consensus_queue.cc:260] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:41.240073 25448 raft_consensus.cc:3060] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:41.245821 25720 raft_consensus.cc:2468] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 3.
I20260709 12:16:41.247563 25490 leader_election.cc:304] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, da7a166dda6c4e6580e13213788f18da; no voters: 
I20260709 12:16:41.248277 25448 raft_consensus.cc:2468] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 3.
I20260709 12:16:41.248574 25773 raft_consensus.cc:2804] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:41.249325 25773 raft_consensus.cc:697] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 3 LEADER]: Becoming Leader. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Running, Role: LEADER
I20260709 12:16:41.250473 25773 consensus_queue.cc:237] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:41.260090 25271 catalog_manager.cc:5697] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f reported cstate change: term changed from 2 to 3, leader changed from d0987de3c9724390936aa3bd641fa419 (127.11.13.130) to 676bed0fbe614782b3aae7d0db05998f (127.11.13.131). New cstate: current_term: 3 leader_uuid: "676bed0fbe614782b3aae7d0db05998f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:41.352631 25321 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:16:41.516726 25720 raft_consensus.cc:1275] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [term 2 FOLLOWER]: Refusing update from remote peer 676bed0fbe614782b3aae7d0db05998f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:41.518923 25773 consensus_queue.cc:1048] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Connected to new peer: Peer: permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:41.534861 25448 raft_consensus.cc:1275] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [term 2 FOLLOWER]: Refusing update from remote peer 676bed0fbe614782b3aae7d0db05998f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:41.536753 25773 consensus_queue.cc:1048] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:41.776747 25720 raft_consensus.cc:1275] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 3 FOLLOWER]: Refusing update from remote peer 676bed0fbe614782b3aae7d0db05998f: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:16:41.778589 25793 consensus_queue.cc:1048] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Connected to new peer: Peer: permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:41.793255 25448 raft_consensus.cc:1275] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 3 FOLLOWER]: Refusing update from remote peer 676bed0fbe614782b3aae7d0db05998f: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:16:41.794891 25669 consensus_queue.cc:1048] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:41.974339 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:41.985225 25523 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0879bf3038f7407693a853a4eb69d0f1"
dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
mode: GRACEFUL
new_leader_uuid: "d0987de3c9724390936aa3bd641fa419"
 from {username='slave'} at 127.0.0.1:33624
I20260709 12:16:41.986083 25523 raft_consensus.cc:606] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [term 1 LEADER]: Received request to transfer leadership to TS d0987de3c9724390936aa3bd641fa419
I20260709 12:16:41.989784 25523 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7411d269e0944ceaa4f08766843e4819"
dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
mode: GRACEFUL
new_leader_uuid: "d0987de3c9724390936aa3bd641fa419"
 from {username='slave'} at 127.0.0.1:33624
I20260709 12:16:41.990553 25523 raft_consensus.cc:606] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 3 LEADER]: Received request to transfer leadership to TS d0987de3c9724390936aa3bd641fa419
I20260709 12:16:41.991925 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:16:41.992941 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:41.993800 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:16:42.355383 25321 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:16:42.361905 25773 raft_consensus.cc:993] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f: : Instructing follower d0987de3c9724390936aa3bd641fa419 to start an election
I20260709 12:16:42.362756 25773 raft_consensus.cc:1081] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 3 LEADER]: Signalling peer d0987de3c9724390936aa3bd641fa419 to start an election
I20260709 12:16:42.366618 25448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7411d269e0944ceaa4f08766843e4819"
dest_uuid: "d0987de3c9724390936aa3bd641fa419"
 from {username='slave'} at 127.0.0.1:38720
I20260709 12:16:42.367753 25448 raft_consensus.cc:493] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:42.368367 25448 raft_consensus.cc:3060] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:42.383297 25448 raft_consensus.cc:515] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:42.385823 25448 leader_election.cc:290] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 4 election: Requested vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:42.387585 25523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7411d269e0944ceaa4f08766843e4819" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
I20260709 12:16:42.387966 25720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7411d269e0944ceaa4f08766843e4819" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "da7a166dda6c4e6580e13213788f18da"
I20260709 12:16:42.388566 25523 raft_consensus.cc:3055] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:42.389011 25720 raft_consensus.cc:3060] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:42.389096 25523 raft_consensus.cc:740] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Running, Role: LEADER
I20260709 12:16:42.390782 25523 consensus_queue.cc:260] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:42.392867 25523 raft_consensus.cc:3060] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:42.400241 25720 raft_consensus.cc:2468] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 4.
I20260709 12:16:42.401861 25416 leader_election.cc:304] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0987de3c9724390936aa3bd641fa419, da7a166dda6c4e6580e13213788f18da; no voters: 
I20260709 12:16:42.402853 25657 raft_consensus.cc:2804] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:42.403344 25657 raft_consensus.cc:697] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 4 LEADER]: Becoming Leader. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Running, Role: LEADER
I20260709 12:16:42.404821 25657 consensus_queue.cc:237] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:42.405908 25523 raft_consensus.cc:2468] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 4.
I20260709 12:16:42.415944 25271 catalog_manager.cc:5697] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 reported cstate change: term changed from 3 to 4, leader changed from 676bed0fbe614782b3aae7d0db05998f (127.11.13.131) to d0987de3c9724390936aa3bd641fa419 (127.11.13.130). New cstate: current_term: 4 leader_uuid: "d0987de3c9724390936aa3bd641fa419" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:42.460350 25793 raft_consensus.cc:993] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f: : Instructing follower d0987de3c9724390936aa3bd641fa419 to start an election
I20260709 12:16:42.461212 25669 raft_consensus.cc:1081] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [term 1 LEADER]: Signalling peer d0987de3c9724390936aa3bd641fa419 to start an election
I20260709 12:16:42.463558 25448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0879bf3038f7407693a853a4eb69d0f1"
dest_uuid: "d0987de3c9724390936aa3bd641fa419"
 from {username='slave'} at 127.0.0.1:38720
I20260709 12:16:42.464138 25448 raft_consensus.cc:493] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:42.464495 25448 raft_consensus.cc:3060] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:42.470587 25448 raft_consensus.cc:515] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:42.472250 25448 leader_election.cc:290] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 2 election: Requested vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:42.473363 25523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0879bf3038f7407693a853a4eb69d0f1" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
I20260709 12:16:42.474081 25523 raft_consensus.cc:3055] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:42.473380 25720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0879bf3038f7407693a853a4eb69d0f1" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "da7a166dda6c4e6580e13213788f18da"
I20260709 12:16:42.474538 25523 raft_consensus.cc:740] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Running, Role: LEADER
I20260709 12:16:42.474763 25720 raft_consensus.cc:3060] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:42.475922 25523 consensus_queue.cc:260] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:42.477620 25523 raft_consensus.cc:3060] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:42.483904 25720 raft_consensus.cc:2468] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 2.
I20260709 12:16:42.487041 25523 raft_consensus.cc:2468] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 2.
I20260709 12:16:42.487682 25416 leader_election.cc:304] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0987de3c9724390936aa3bd641fa419, da7a166dda6c4e6580e13213788f18da; no voters: 
I20260709 12:16:42.488626 25657 raft_consensus.cc:2804] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:42.489055 25657 raft_consensus.cc:697] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [term 2 LEADER]: Becoming Leader. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Running, Role: LEADER
I20260709 12:16:42.489971 25657 consensus_queue.cc:237] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:42.498643 25271 catalog_manager.cc:5697] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 reported cstate change: term changed from 1 to 2, leader changed from 676bed0fbe614782b3aae7d0db05998f (127.11.13.131) to d0987de3c9724390936aa3bd641fa419 (127.11.13.130). New cstate: current_term: 2 leader_uuid: "d0987de3c9724390936aa3bd641fa419" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:42.901247 25523 raft_consensus.cc:1275] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Refusing update from remote peer d0987de3c9724390936aa3bd641fa419: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:42.903385 25759 consensus_queue.cc:1048] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:42.912829 25720 raft_consensus.cc:1275] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 4 FOLLOWER]: Refusing update from remote peer d0987de3c9724390936aa3bd641fa419: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:16:42.915558 25759 consensus_queue.cc:1048] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:42.929657 25720 raft_consensus.cc:1275] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da [term 2 FOLLOWER]: Refusing update from remote peer d0987de3c9724390936aa3bd641fa419: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:42.935972 25805 consensus_queue.cc:1048] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:42.945204 25523 raft_consensus.cc:1275] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 4 FOLLOWER]: Refusing update from remote peer d0987de3c9724390936aa3bd641fa419: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:16:42.947052 25759 consensus_queue.cc:1048] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:43.004508 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:43.015825 25448 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14e62bfdb1564317b3ec26702c9b0172"
dest_uuid: "d0987de3c9724390936aa3bd641fa419"
mode: GRACEFUL
new_leader_uuid: "676bed0fbe614782b3aae7d0db05998f"
 from {username='slave'} at 127.0.0.1:38728
I20260709 12:16:43.016445 25448 raft_consensus.cc:606] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 3 LEADER]: Received request to transfer leadership to TS 676bed0fbe614782b3aae7d0db05998f
I20260709 12:16:43.019789 25448 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed1825085af74858b5ae5b68c999100b"
dest_uuid: "d0987de3c9724390936aa3bd641fa419"
mode: GRACEFUL
new_leader_uuid: "676bed0fbe614782b3aae7d0db05998f"
 from {username='slave'} at 127.0.0.1:38728
I20260709 12:16:43.020439 25448 raft_consensus.cc:606] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 2 LEADER]: Received request to transfer leadership to TS 676bed0fbe614782b3aae7d0db05998f
I20260709 12:16:43.022233 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:16:43.022995 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:43.024089 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:43.188867 25759 raft_consensus.cc:993] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419: : Instructing follower 676bed0fbe614782b3aae7d0db05998f to start an election
I20260709 12:16:43.189487 25805 raft_consensus.cc:1081] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 3 LEADER]: Signalling peer 676bed0fbe614782b3aae7d0db05998f to start an election
I20260709 12:16:43.192039 25523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "14e62bfdb1564317b3ec26702c9b0172"
dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
 from {username='slave'} at 127.0.0.1:33616
I20260709 12:16:43.192704 25523 raft_consensus.cc:493] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:43.193125 25523 raft_consensus.cc:3060] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:43.202057 25523 raft_consensus.cc:515] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:43.204463 25523 leader_election.cc:290] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 4 election: Requested vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305)
I20260709 12:16:43.205621 25720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14e62bfdb1564317b3ec26702c9b0172" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "da7a166dda6c4e6580e13213788f18da"
I20260709 12:16:43.205920 25448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14e62bfdb1564317b3ec26702c9b0172" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d0987de3c9724390936aa3bd641fa419"
I20260709 12:16:43.206489 25720 raft_consensus.cc:3060] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:43.206555 25448 raft_consensus.cc:3055] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:43.207065 25448 raft_consensus.cc:740] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Running, Role: LEADER
I20260709 12:16:43.208201 25448 consensus_queue.cc:260] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:43.209791 25448 raft_consensus.cc:3060] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:43.215425 25720 raft_consensus.cc:2468] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 4.
I20260709 12:16:43.216926 25490 leader_election.cc:304] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, da7a166dda6c4e6580e13213788f18da; no voters: 
I20260709 12:16:43.217850 25669 raft_consensus.cc:2804] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:43.218438 25669 raft_consensus.cc:697] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 4 LEADER]: Becoming Leader. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Running, Role: LEADER
I20260709 12:16:43.218564 25448 raft_consensus.cc:2468] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 4.
I20260709 12:16:43.219828 25669 consensus_queue.cc:237] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:43.231191 25273 catalog_manager.cc:5697] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f reported cstate change: term changed from 3 to 4, leader changed from d0987de3c9724390936aa3bd641fa419 (127.11.13.130) to 676bed0fbe614782b3aae7d0db05998f (127.11.13.131). New cstate: current_term: 4 leader_uuid: "676bed0fbe614782b3aae7d0db05998f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: HEALTHY } } }
W20260709 12:16:43.357370 25321 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:16:43.390672 25657 raft_consensus.cc:993] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419: : Instructing follower 676bed0fbe614782b3aae7d0db05998f to start an election
I20260709 12:16:43.391450 25805 raft_consensus.cc:1081] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 2 LEADER]: Signalling peer 676bed0fbe614782b3aae7d0db05998f to start an election
I20260709 12:16:43.393530 25523 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed1825085af74858b5ae5b68c999100b"
dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
 from {username='slave'} at 127.0.0.1:33616
I20260709 12:16:43.394368 25523 raft_consensus.cc:493] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:43.394786 25523 raft_consensus.cc:3060] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:43.405597 25523 raft_consensus.cc:515] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:43.408401 25523 leader_election.cc:290] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 3 election: Requested vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), d0987de3c9724390936aa3bd641fa419 (127.11.13.130:34305)
I20260709 12:16:43.409180 25720 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1825085af74858b5ae5b68c999100b" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "da7a166dda6c4e6580e13213788f18da"
I20260709 12:16:43.409421 25448 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1825085af74858b5ae5b68c999100b" candidate_uuid: "676bed0fbe614782b3aae7d0db05998f" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: true dest_uuid: "d0987de3c9724390936aa3bd641fa419"
I20260709 12:16:43.409963 25448 raft_consensus.cc:3055] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:43.410336 25448 raft_consensus.cc:740] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Running, Role: LEADER
I20260709 12:16:43.409965 25720 raft_consensus.cc:3060] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:43.411458 25448 consensus_queue.cc:260] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:43.412892 25448 raft_consensus.cc:3060] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:43.419636 25720 raft_consensus.cc:2468] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 3.
I20260709 12:16:43.421537 25490 leader_election.cc:304] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, da7a166dda6c4e6580e13213788f18da; no voters: 
I20260709 12:16:43.422959 25669 raft_consensus.cc:2804] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:43.422793 25448 raft_consensus.cc:2468] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 676bed0fbe614782b3aae7d0db05998f in term 3.
I20260709 12:16:43.423645 25669 raft_consensus.cc:697] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 3 LEADER]: Becoming Leader. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Running, Role: LEADER
I20260709 12:16:43.425037 25669 consensus_queue.cc:237] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:43.434023 25273 catalog_manager.cc:5697] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f reported cstate change: term changed from 2 to 3, leader changed from d0987de3c9724390936aa3bd641fa419 (127.11.13.130) to 676bed0fbe614782b3aae7d0db05998f (127.11.13.131). New cstate: current_term: 3 leader_uuid: "676bed0fbe614782b3aae7d0db05998f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:43.633936 25448 raft_consensus.cc:1275] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 4 FOLLOWER]: Refusing update from remote peer 676bed0fbe614782b3aae7d0db05998f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:43.635411 25669 consensus_queue.cc:1048] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:43.655947 25720 raft_consensus.cc:1275] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 4 FOLLOWER]: Refusing update from remote peer 676bed0fbe614782b3aae7d0db05998f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260709 12:16:43.659160 25793 consensus_queue.cc:1048] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Connected to new peer: Peer: permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:43.887349 25721 raft_consensus.cc:1275] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 3 FOLLOWER]: Refusing update from remote peer 676bed0fbe614782b3aae7d0db05998f: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:16:43.889027 25821 consensus_queue.cc:1048] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Connected to new peer: Peer: permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:43.906795 25448 raft_consensus.cc:1275] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 3 FOLLOWER]: Refusing update from remote peer 676bed0fbe614782b3aae7d0db05998f: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260709 12:16:43.908905 25793 consensus_queue.cc:1048] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:44.032141 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:44.042266 25523 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "14e62bfdb1564317b3ec26702c9b0172"
dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
mode: GRACEFUL
new_leader_uuid: "d0987de3c9724390936aa3bd641fa419"
 from {username='slave'} at 127.0.0.1:33624
I20260709 12:16:44.042845 25523 raft_consensus.cc:606] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 4 LEADER]: Received request to transfer leadership to TS d0987de3c9724390936aa3bd641fa419
I20260709 12:16:44.046105 25523 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ed1825085af74858b5ae5b68c999100b"
dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
mode: GRACEFUL
new_leader_uuid: "d0987de3c9724390936aa3bd641fa419"
 from {username='slave'} at 127.0.0.1:33624
I20260709 12:16:44.046868 25523 raft_consensus.cc:606] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 3 LEADER]: Received request to transfer leadership to TS d0987de3c9724390936aa3bd641fa419
I20260709 12:16:44.048300 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:16:44.048908 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:44.049657 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:44.249119 25793 raft_consensus.cc:993] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f: : Instructing follower d0987de3c9724390936aa3bd641fa419 to start an election
I20260709 12:16:44.249678 25822 raft_consensus.cc:1081] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 4 LEADER]: Signalling peer d0987de3c9724390936aa3bd641fa419 to start an election
I20260709 12:16:44.251498 25448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "14e62bfdb1564317b3ec26702c9b0172"
dest_uuid: "d0987de3c9724390936aa3bd641fa419"
 from {username='slave'} at 127.0.0.1:38720
I20260709 12:16:44.251986 25448 raft_consensus.cc:493] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:44.252274 25448 raft_consensus.cc:3060] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:44.258622 25448 raft_consensus.cc:515] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:44.260456 25448 leader_election.cc:290] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 5 election: Requested vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:44.261932 25523 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14e62bfdb1564317b3ec26702c9b0172" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
I20260709 12:16:44.262087 25721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "14e62bfdb1564317b3ec26702c9b0172" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "da7a166dda6c4e6580e13213788f18da"
I20260709 12:16:44.262641 25523 raft_consensus.cc:3055] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 4 LEADER]: Stepping down as leader of term 4
I20260709 12:16:44.262943 25721 raft_consensus.cc:3060] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:44.263082 25523 raft_consensus.cc:740] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Running, Role: LEADER
I20260709 12:16:44.264149 25523 consensus_queue.cc:260] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:44.265180 25523 raft_consensus.cc:3060] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 4 FOLLOWER]: Advancing to term 5
I20260709 12:16:44.270897 25523 raft_consensus.cc:2468] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 5.
I20260709 12:16:44.272799 25414 leader_election.cc:304] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 676bed0fbe614782b3aae7d0db05998f, d0987de3c9724390936aa3bd641fa419; no voters: 
I20260709 12:16:44.273289 25721 raft_consensus.cc:2468] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 5.
I20260709 12:16:44.273737 25805 raft_consensus.cc:2804] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 5 FOLLOWER]: Leader election won for term 5
I20260709 12:16:44.281998 25805 raft_consensus.cc:697] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 5 LEADER]: Becoming Leader. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Running, Role: LEADER
I20260709 12:16:44.283071 25805 consensus_queue.cc:237] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:44.292846 25271 catalog_manager.cc:5697] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 reported cstate change: term changed from 4 to 5, leader changed from 676bed0fbe614782b3aae7d0db05998f (127.11.13.131) to d0987de3c9724390936aa3bd641fa419 (127.11.13.130). New cstate: current_term: 5 leader_uuid: "d0987de3c9724390936aa3bd641fa419" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: UNKNOWN } } }
W20260709 12:16:44.358675 25321 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:16:44.491511 25793 raft_consensus.cc:993] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f: : Instructing follower d0987de3c9724390936aa3bd641fa419 to start an election
I20260709 12:16:44.492131 25822 raft_consensus.cc:1081] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 3 LEADER]: Signalling peer d0987de3c9724390936aa3bd641fa419 to start an election
I20260709 12:16:44.493911 25448 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ed1825085af74858b5ae5b68c999100b"
dest_uuid: "d0987de3c9724390936aa3bd641fa419"
 from {username='slave'} at 127.0.0.1:38720
I20260709 12:16:44.494761 25448 raft_consensus.cc:493] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:44.495249 25448 raft_consensus.cc:3060] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:44.506227 25448 raft_consensus.cc:515] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:44.508759 25448 leader_election.cc:290] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 4 election: Requested vote from peers da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415), 676bed0fbe614782b3aae7d0db05998f (127.11.13.131:40749)
I20260709 12:16:44.508904 25721 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1825085af74858b5ae5b68c999100b" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "da7a166dda6c4e6580e13213788f18da"
I20260709 12:16:44.509665 25721 raft_consensus.cc:3060] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:44.510936 25524 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ed1825085af74858b5ae5b68c999100b" candidate_uuid: "d0987de3c9724390936aa3bd641fa419" candidate_term: 4 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: true dest_uuid: "676bed0fbe614782b3aae7d0db05998f"
I20260709 12:16:44.511695 25524 raft_consensus.cc:3055] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:16:44.511992 25524 raft_consensus.cc:740] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 676bed0fbe614782b3aae7d0db05998f, State: Running, Role: LEADER
I20260709 12:16:44.512871 25524 consensus_queue.cc:260] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:44.514195 25524 raft_consensus.cc:3060] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:16:44.520421 25721 raft_consensus.cc:2468] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 4.
I20260709 12:16:44.522176 25416 leader_election.cc:304] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d0987de3c9724390936aa3bd641fa419, da7a166dda6c4e6580e13213788f18da; no voters: 
I20260709 12:16:44.522910 25524 raft_consensus.cc:2468] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d0987de3c9724390936aa3bd641fa419 in term 4.
I20260709 12:16:44.523056 25829 raft_consensus.cc:2804] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:16:44.523747 25829 raft_consensus.cc:697] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 4 LEADER]: Becoming Leader. State: Replica: d0987de3c9724390936aa3bd641fa419, State: Running, Role: LEADER
I20260709 12:16:44.525743 25829 consensus_queue.cc:237] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } }
I20260709 12:16:44.534581 25271 catalog_manager.cc:5697] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 reported cstate change: term changed from 3 to 4, leader changed from 676bed0fbe614782b3aae7d0db05998f (127.11.13.131) to d0987de3c9724390936aa3bd641fa419 (127.11.13.130). New cstate: current_term: 4 leader_uuid: "d0987de3c9724390936aa3bd641fa419" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d0987de3c9724390936aa3bd641fa419" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 34305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:44.783054 25721 raft_consensus.cc:1275] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 5 FOLLOWER]: Refusing update from remote peer d0987de3c9724390936aa3bd641fa419: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:16:44.785161 25829 consensus_queue.cc:1048] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:44.805814 25524 raft_consensus.cc:1275] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 5 FOLLOWER]: Refusing update from remote peer d0987de3c9724390936aa3bd641fa419: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260709 12:16:44.807344 25805 consensus_queue.cc:1048] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260709 12:16:45.057252 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:45.081367 25523 raft_consensus.cc:1275] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 4 FOLLOWER]: Refusing update from remote peer d0987de3c9724390936aa3bd641fa419: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:16:45.083344 25829 consensus_queue.cc:1048] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "676bed0fbe614782b3aae7d0db05998f" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:45.095904 25721 raft_consensus.cc:1275] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 4 FOLLOWER]: Refusing update from remote peer d0987de3c9724390936aa3bd641fa419: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:16:45.098426 25829 consensus_queue.cc:1048] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [LEADER]: Connected to new peer: Peer: permanent_uuid: "da7a166dda6c4e6580e13213788f18da" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 33415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:45.102041 11318 tablet_server.cc:179] TabletServer@127.11.13.129:33415 shutting down...
W20260709 12:16:45.107039 25416 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 97 similar messages]
W20260709 12:16:45.114581 25416 consensus_peers.cc:597] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:45.119407 25416 consensus_peers.cc:597] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419 -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:45.144834 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:45.145784 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:45.146986 11318 raft_consensus.cc:2243] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.147576 11318 raft_consensus.cc:2272] T 14e62bfdb1564317b3ec26702c9b0172 P da7a166dda6c4e6580e13213788f18da [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.151675 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:45.152395 11318 raft_consensus.cc:2243] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.152930 11318 raft_consensus.cc:2272] T 22f011f2c17b4327a825f429b685cd0c P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.155668 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:45.156822 11318 raft_consensus.cc:2243] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.157519 11318 raft_consensus.cc:2272] T ed1825085af74858b5ae5b68c999100b P da7a166dda6c4e6580e13213788f18da [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.161615 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:45.162910 11318 raft_consensus.cc:2243] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.163525 11318 raft_consensus.cc:2272] T eefc812354034f0c98229c96d8e63846 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.166878 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:45.167587 11318 raft_consensus.cc:2243] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.168066 11318 raft_consensus.cc:2272] T 0879bf3038f7407693a853a4eb69d0f1 P da7a166dda6c4e6580e13213788f18da [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.171413 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:45.172448 11318 raft_consensus.cc:2243] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.173092 11318 raft_consensus.cc:2272] T 25362af9774a4e60b349e61f9a2601db P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.177409 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:45.178503 11318 raft_consensus.cc:2243] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.179313 11318 raft_consensus.cc:2272] T 7411d269e0944ceaa4f08766843e4819 P da7a166dda6c4e6580e13213788f18da [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:45.180039 25490 consensus_peers.cc:597] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:45.182806 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:45.183775 11318 raft_consensus.cc:2243] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.184350 11318 raft_consensus.cc:2272] T 11dad3c292254bf59bd998aff5a37702 P da7a166dda6c4e6580e13213788f18da [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:45.185333 25416 consensus_peers.cc:597] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:45.187840 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:45.189208 11318 raft_consensus.cc:2243] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.190029 11318 raft_consensus.cc:2272] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P da7a166dda6c4e6580e13213788f18da [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:45.192492 25490 consensus_peers.cc:597] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:45.194330 11318 tablet_replica.cc:333] stopping tablet replica
I20260709 12:16:45.195083 11318 raft_consensus.cc:2243] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.195946 11318 raft_consensus.cc:2272] T a2f349cdc65547d6a574bbe3563db1a1 P da7a166dda6c4e6580e13213788f18da [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:45.202193 25416 consensus_peers.cc:597] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:45.233924 11318 tablet_server.cc:196] TabletServer@127.11.13.129:33415 shutdown complete.
I20260709 12:16:45.267129 11318 tablet_server.cc:179] TabletServer@127.11.13.130:0 shutting down...
I20260709 12:16:45.302636 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:45.303580 11318 tablet_replica.cc:333] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419: stopping tablet replica
I20260709 12:16:45.304502 11318 raft_consensus.cc:2243] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:45.305630 11318 raft_consensus.cc:2272] T 22f011f2c17b4327a825f429b685cd0c P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.308470 11318 tablet_replica.cc:333] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419: stopping tablet replica
I20260709 12:16:45.309228 11318 raft_consensus.cc:2243] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:45.310204 11318 raft_consensus.cc:2272] T 0879bf3038f7407693a853a4eb69d0f1 P d0987de3c9724390936aa3bd641fa419 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.312986 11318 tablet_replica.cc:333] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419: stopping tablet replica
I20260709 12:16:45.313939 11318 raft_consensus.cc:2243] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:45.315373 11318 raft_consensus.cc:2272] T ed1825085af74858b5ae5b68c999100b P d0987de3c9724390936aa3bd641fa419 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.318912 11318 tablet_replica.cc:333] T eefc812354034f0c98229c96d8e63846 P d0987de3c9724390936aa3bd641fa419: stopping tablet replica
I20260709 12:16:45.319631 11318 raft_consensus.cc:2243] T eefc812354034f0c98229c96d8e63846 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.320185 11318 raft_consensus.cc:2272] T eefc812354034f0c98229c96d8e63846 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.322568 11318 tablet_replica.cc:333] T 25362af9774a4e60b349e61f9a2601db P d0987de3c9724390936aa3bd641fa419: stopping tablet replica
I20260709 12:16:45.323172 11318 raft_consensus.cc:2243] T 25362af9774a4e60b349e61f9a2601db P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.323828 11318 raft_consensus.cc:2272] T 25362af9774a4e60b349e61f9a2601db P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.326251 11318 tablet_replica.cc:333] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419: stopping tablet replica
I20260709 12:16:45.326900 11318 raft_consensus.cc:2243] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 5 LEADER]: Raft consensus shutting down.
I20260709 12:16:45.327929 11318 raft_consensus.cc:2272] T 14e62bfdb1564317b3ec26702c9b0172 P d0987de3c9724390936aa3bd641fa419 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.331082 11318 tablet_replica.cc:333] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419: stopping tablet replica
I20260709 12:16:45.332188 11318 raft_consensus.cc:2243] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.332971 11318 raft_consensus.cc:2272] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P d0987de3c9724390936aa3bd641fa419 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.335829 11318 tablet_replica.cc:333] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419: stopping tablet replica
I20260709 12:16:45.336407 11318 raft_consensus.cc:2243] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:45.337435 11318 raft_consensus.cc:2272] T a2f349cdc65547d6a574bbe3563db1a1 P d0987de3c9724390936aa3bd641fa419 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.340142 11318 tablet_replica.cc:333] T 11dad3c292254bf59bd998aff5a37702 P d0987de3c9724390936aa3bd641fa419: stopping tablet replica
I20260709 12:16:45.340687 11318 raft_consensus.cc:2243] T 11dad3c292254bf59bd998aff5a37702 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.341171 11318 raft_consensus.cc:2272] T 11dad3c292254bf59bd998aff5a37702 P d0987de3c9724390936aa3bd641fa419 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.343832 11318 tablet_replica.cc:333] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419: stopping tablet replica
I20260709 12:16:45.344390 11318 raft_consensus.cc:2243] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:16:45.345305 11318 raft_consensus.cc:2272] T 7411d269e0944ceaa4f08766843e4819 P d0987de3c9724390936aa3bd641fa419 [term 4 FOLLOWER]: Raft consensus is shut down!
W20260709 12:16:45.360278 25321 auto_rebalancer.cc:288] Could not retrieve cluster info: Illegal state: not all tservers available for tablet placement
I20260709 12:16:45.390829 11318 tablet_server.cc:196] TabletServer@127.11.13.130:0 shutdown complete.
I20260709 12:16:45.422943 11318 tablet_server.cc:179] TabletServer@127.11.13.131:0 shutting down...
W20260709 12:16:45.440671 25490 consensus_peers.cc:597] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:16:45.444679 25490 consensus_peers.cc:597] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f -> Peer da7a166dda6c4e6580e13213788f18da (127.11.13.129:33415): Couldn't send request to peer da7a166dda6c4e6580e13213788f18da. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:33415: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:45.461423 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:45.462423 11318 tablet_replica.cc:333] T 22f011f2c17b4327a825f429b685cd0c P 676bed0fbe614782b3aae7d0db05998f: stopping tablet replica
I20260709 12:16:45.463340 11318 raft_consensus.cc:2243] T 22f011f2c17b4327a825f429b685cd0c P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.463958 11318 raft_consensus.cc:2272] T 22f011f2c17b4327a825f429b685cd0c P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.466583 11318 tablet_replica.cc:333] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f: stopping tablet replica
I20260709 12:16:45.467253 11318 raft_consensus.cc:2243] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.467739 11318 raft_consensus.cc:2272] T 7411d269e0944ceaa4f08766843e4819 P 676bed0fbe614782b3aae7d0db05998f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.470054 11318 tablet_replica.cc:333] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f: stopping tablet replica
I20260709 12:16:45.470793 11318 raft_consensus.cc:2243] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:45.471797 11318 raft_consensus.cc:2272] T 11dad3c292254bf59bd998aff5a37702 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.474870 11318 tablet_replica.cc:333] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f: stopping tablet replica
I20260709 12:16:45.475795 11318 raft_consensus.cc:2243] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.476375 11318 raft_consensus.cc:2272] T ed1825085af74858b5ae5b68c999100b P 676bed0fbe614782b3aae7d0db05998f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.480472 11318 tablet_replica.cc:333] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f: stopping tablet replica
I20260709 12:16:45.481374 11318 raft_consensus.cc:2243] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:45.482730 11318 raft_consensus.cc:2272] T eefc812354034f0c98229c96d8e63846 P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.485337 11318 tablet_replica.cc:333] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f: stopping tablet replica
I20260709 12:16:45.485956 11318 raft_consensus.cc:2243] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.486474 11318 raft_consensus.cc:2272] T 0879bf3038f7407693a853a4eb69d0f1 P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.488790 11318 tablet_replica.cc:333] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f: stopping tablet replica
I20260709 12:16:45.489351 11318 raft_consensus.cc:2243] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:45.490291 11318 raft_consensus.cc:2272] T 25362af9774a4e60b349e61f9a2601db P 676bed0fbe614782b3aae7d0db05998f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.492774 11318 tablet_replica.cc:333] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f: stopping tablet replica
I20260709 12:16:45.493337 11318 raft_consensus.cc:2243] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:45.494442 11318 raft_consensus.cc:2272] T 22bd5dc8f9bc4e43ae6cfd2cfd9d1086 P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.496871 11318 tablet_replica.cc:333] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f: stopping tablet replica
I20260709 12:16:45.497376 11318 raft_consensus.cc:2243] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.497864 11318 raft_consensus.cc:2272] T a2f349cdc65547d6a574bbe3563db1a1 P 676bed0fbe614782b3aae7d0db05998f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.500244 11318 tablet_replica.cc:333] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f: stopping tablet replica
I20260709 12:16:45.500758 11318 raft_consensus.cc:2243] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 5 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:45.501161 11318 raft_consensus.cc:2272] T 14e62bfdb1564317b3ec26702c9b0172 P 676bed0fbe614782b3aae7d0db05998f [term 5 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.539979 11318 tablet_server.cc:196] TabletServer@127.11.13.131:0 shutdown complete.
I20260709 12:16:45.568689 11318 master.cc:562] Master@127.11.13.190:40877 shutting down...
I20260709 12:16:45.593799 11318 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:45.594475 11318 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:45.594841 11318 tablet_replica.cc:333] T 00000000000000000000000000000000 P 5983d1727349477db4c65c4c602bd694: stopping tablet replica
I20260709 12:16:45.639377 11318 master.cc:584] Master@127.11.13.190:40877 shutdown complete.
[       OK ] LeaderRebalancerTest.TestMaintenanceMode (13568 ms)
[ RUN      ] LeaderRebalancerTest.MultiTableLeaderBalance
I20260709 12:16:45.691946 11318 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.11.13.190:44883
I20260709 12:16:45.693060 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:45.699275 25846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:45.699319 25847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:45.699745 25849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:45.701234 11318 server_base.cc:1061] running on GCE node
I20260709 12:16:45.702262 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:45.702497 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:45.702654 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599405702636 us; error 0 us; skew 500 ppm
I20260709 12:16:45.703186 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:45.705729 11318 webserver.cc:533] Webserver started at http://127.11.13.190:40153/ using document root <none> and password file <none>
I20260709 12:16:45.706517 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:45.706727 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:45.707012 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:45.708170 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598892056905-11318-0/minicluster-data/master-0-root/instance:
uuid: "b8f42886d4d2406d917b306b45331bb0"
format_stamp: "Formatted at 2026-07-09 12:16:45 on dist-test-slave-zqnl"
I20260709 12:16:45.712960 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:16:45.717173 25854 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:45.718235 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:16:45.718578 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598892056905-11318-0/minicluster-data/master-0-root
uuid: "b8f42886d4d2406d917b306b45331bb0"
format_stamp: "Formatted at 2026-07-09 12:16:45 on dist-test-slave-zqnl"
I20260709 12:16:45.718888 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598892056905-11318-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598892056905-11318-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598892056905-11318-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:45.730544 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:45.732096 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:45.778504 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.190:44883
I20260709 12:16:45.778667 25905 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.190:44883 every 8 connection(s)
I20260709 12:16:45.782631 25906 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:45.794835 25906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0: Bootstrap starting.
I20260709 12:16:45.800011 25906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:45.804426 25906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0: No bootstrap required, opened a new log
I20260709 12:16:45.806903 25906 raft_consensus.cc:359] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8f42886d4d2406d917b306b45331bb0" member_type: VOTER }
I20260709 12:16:45.807307 25906 raft_consensus.cc:385] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:45.807497 25906 raft_consensus.cc:740] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8f42886d4d2406d917b306b45331bb0, State: Initialized, Role: FOLLOWER
I20260709 12:16:45.808125 25906 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8f42886d4d2406d917b306b45331bb0" member_type: VOTER }
I20260709 12:16:45.808552 25906 raft_consensus.cc:399] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:16:45.808749 25906 raft_consensus.cc:493] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:16:45.808950 25906 raft_consensus.cc:3060] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:45.814016 25906 raft_consensus.cc:515] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8f42886d4d2406d917b306b45331bb0" member_type: VOTER }
I20260709 12:16:45.814867 25906 leader_election.cc:304] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8f42886d4d2406d917b306b45331bb0; no voters: 
I20260709 12:16:45.816269 25906 leader_election.cc:290] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:16:45.816629 25909 raft_consensus.cc:2804] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:45.818041 25909 raft_consensus.cc:697] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0 [term 1 LEADER]: Becoming Leader. State: Replica: b8f42886d4d2406d917b306b45331bb0, State: Running, Role: LEADER
I20260709 12:16:45.818811 25909 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8f42886d4d2406d917b306b45331bb0" member_type: VOTER }
I20260709 12:16:45.819567 25906 sys_catalog.cc:565] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:16:45.824196 25911 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8f42886d4d2406d917b306b45331bb0. Latest consensus state: current_term: 1 leader_uuid: "b8f42886d4d2406d917b306b45331bb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8f42886d4d2406d917b306b45331bb0" member_type: VOTER } }
I20260709 12:16:45.825165 25911 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0 [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:45.826059 25910 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8f42886d4d2406d917b306b45331bb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8f42886d4d2406d917b306b45331bb0" member_type: VOTER } }
I20260709 12:16:45.826817 25910 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0 [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:45.828284 25917 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:16:45.833909 25917 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:16:45.837524 11318 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:16:45.845027 25917 catalog_manager.cc:1383] Generated new cluster ID: 7a090e4729fc43dc95569430e8bf283c
I20260709 12:16:45.845324 25917 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:16:45.883486 25917 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:16:45.884841 25917 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:16:45.903560 25917 catalog_manager.cc:6070] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0: Generated new TSK 0
I20260709 12:16:45.904232 25917 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:16:45.972599 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:45.981383 25927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:45.983640 25928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:45.985356 11318 server_base.cc:1061] running on GCE node
W20260709 12:16:45.985296 25930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:45.986963 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:45.987175 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:45.987306 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599405987294 us; error 0 us; skew 500 ppm
I20260709 12:16:45.987879 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:45.990710 11318 webserver.cc:533] Webserver started at http://127.11.13.129:43621/ using document root <none> and password file <none>
I20260709 12:16:45.991209 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:45.991389 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:45.991794 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:45.992980 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598892056905-11318-0/minicluster-data/ts-0-root/instance:
uuid: "41aed3c61bc64062a35dd612d84e1512"
format_stamp: "Formatted at 2026-07-09 12:16:45 on dist-test-slave-zqnl"
I20260709 12:16:45.998346 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.000s	sys 0.007s
I20260709 12:16:46.002359 25935 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:46.003284 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:16:46.003664 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598892056905-11318-0/minicluster-data/ts-0-root
uuid: "41aed3c61bc64062a35dd612d84e1512"
format_stamp: "Formatted at 2026-07-09 12:16:45 on dist-test-slave-zqnl"
I20260709 12:16:46.004012 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598892056905-11318-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598892056905-11318-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598892056905-11318-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:46.021320 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:46.023092 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:46.025218 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:46.028532 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:46.028825 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:46.029115 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:46.029313 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:46.082042 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.129:34807
I20260709 12:16:46.082199 25997 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.129:34807 every 8 connection(s)
I20260709 12:16:46.087718 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:46.096697 26002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:46.098773 26003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:46.101509 11318 server_base.cc:1061] running on GCE node
W20260709 12:16:46.103323 26005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:46.105175 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:46.105468 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:46.105523 25998 heartbeater.cc:344] Connected to a master server at 127.11.13.190:44883
I20260709 12:16:46.105749 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599406105728 us; error 0 us; skew 500 ppm
I20260709 12:16:46.106036 25998 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:46.106631 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:46.107080 25998 heartbeater.cc:507] Master 127.11.13.190:44883 requested a full tablet report, sending...
I20260709 12:16:46.109879 11318 webserver.cc:533] Webserver started at http://127.11.13.130:38531/ using document root <none> and password file <none>
I20260709 12:16:46.110288 25871 ts_manager.cc:194] Registered new tserver with Master: 41aed3c61bc64062a35dd612d84e1512 (127.11.13.129:34807)
I20260709 12:16:46.110610 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:46.110951 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:46.111268 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:46.112461 25871 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37092
I20260709 12:16:46.112562 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598892056905-11318-0/minicluster-data/ts-1-root/instance:
uuid: "70728ad802d141dba54935cb3a513e3c"
format_stamp: "Formatted at 2026-07-09 12:16:46 on dist-test-slave-zqnl"
I20260709 12:16:46.119014 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.008s	sys 0.000s
I20260709 12:16:46.123369 26010 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:46.124234 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:16:46.124609 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598892056905-11318-0/minicluster-data/ts-1-root
uuid: "70728ad802d141dba54935cb3a513e3c"
format_stamp: "Formatted at 2026-07-09 12:16:46 on dist-test-slave-zqnl"
I20260709 12:16:46.124930 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598892056905-11318-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598892056905-11318-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598892056905-11318-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:46.147522 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:46.148815 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:46.150502 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:46.153214 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:46.153544 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:46.153846 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:46.154026 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:46.206935 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.130:42375
I20260709 12:16:46.207039 26072 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.130:42375 every 8 connection(s)
I20260709 12:16:46.212693 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:46.220820 26076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:46.222640 26077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:46.225762 26079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:46.227116 11318 server_base.cc:1061] running on GCE node
I20260709 12:16:46.228282 26073 heartbeater.cc:344] Connected to a master server at 127.11.13.190:44883
I20260709 12:16:46.228406 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:16:46.228788 26073 heartbeater.cc:461] Registering TS with master...
W20260709 12:16:46.228793 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:46.229302 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599406229281 us; error 0 us; skew 500 ppm
I20260709 12:16:46.229885 26073 heartbeater.cc:507] Master 127.11.13.190:44883 requested a full tablet report, sending...
I20260709 12:16:46.230752 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:46.232628 25871 ts_manager.cc:194] Registered new tserver with Master: 70728ad802d141dba54935cb3a513e3c (127.11.13.130:42375)
I20260709 12:16:46.234470 11318 webserver.cc:533] Webserver started at http://127.11.13.131:37733/ using document root <none> and password file <none>
I20260709 12:16:46.234519 25871 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37094
I20260709 12:16:46.235302 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:46.235594 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:46.235999 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:46.237974 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598892056905-11318-0/minicluster-data/ts-2-root/instance:
uuid: "443b0923c6ce405dbc603e9cdea18349"
format_stamp: "Formatted at 2026-07-09 12:16:46 on dist-test-slave-zqnl"
I20260709 12:16:46.243708 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:16:46.247757 26084 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:46.248973 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:16:46.249346 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598892056905-11318-0/minicluster-data/ts-2-root
uuid: "443b0923c6ce405dbc603e9cdea18349"
format_stamp: "Formatted at 2026-07-09 12:16:46 on dist-test-slave-zqnl"
I20260709 12:16:46.249682 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598892056905-11318-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598892056905-11318-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.MultiTableLeaderBalance.1783598892056905-11318-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:46.267411 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:46.268755 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:46.270690 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:46.273474 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:46.273729 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:46.273993 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:46.274215 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:16:46.322340 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.131:34239
I20260709 12:16:46.322722 26146 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.131:34239 every 8 connection(s)
I20260709 12:16:46.340586 26147 heartbeater.cc:344] Connected to a master server at 127.11.13.190:44883
I20260709 12:16:46.341383 26147 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:46.342640 26147 heartbeater.cc:507] Master 127.11.13.190:44883 requested a full tablet report, sending...
I20260709 12:16:46.345505 25871 ts_manager.cc:194] Registered new tserver with Master: 443b0923c6ce405dbc603e9cdea18349 (127.11.13.131:34239)
I20260709 12:16:46.346064 11318 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.020190265s
I20260709 12:16:46.347749 25871 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:37102
I20260709 12:16:46.378046 25871 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37104:
name: "multi_table_leader_balance_table1"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:16:46.380467 25871 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table1 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:16:46.439118 26112 tablet_service.cc:1511] Processing CreateTablet for tablet 2a23fa39c3c64ae5a40bc5367291b968 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=d05056d1bb2248589cd3132ec8459106]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:16:46.439297 26111 tablet_service.cc:1511] Processing CreateTablet for tablet eaa515c878e14aeead8aba1f03969a7c (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=d05056d1bb2248589cd3132ec8459106]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:16:46.440850 26112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a23fa39c3c64ae5a40bc5367291b968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:46.440550 25963 tablet_service.cc:1511] Processing CreateTablet for tablet 2a23fa39c3c64ae5a40bc5367291b968 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=d05056d1bb2248589cd3132ec8459106]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:16:46.441722 26111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaa515c878e14aeead8aba1f03969a7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:46.442660 25963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a23fa39c3c64ae5a40bc5367291b968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:46.443089 25962 tablet_service.cc:1511] Processing CreateTablet for tablet eaa515c878e14aeead8aba1f03969a7c (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=d05056d1bb2248589cd3132ec8459106]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:16:46.444439 25962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaa515c878e14aeead8aba1f03969a7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:46.460881 26038 tablet_service.cc:1511] Processing CreateTablet for tablet 2a23fa39c3c64ae5a40bc5367291b968 (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=d05056d1bb2248589cd3132ec8459106]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:16:46.462498 26038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2a23fa39c3c64ae5a40bc5367291b968. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:46.467905 26037 tablet_service.cc:1511] Processing CreateTablet for tablet eaa515c878e14aeead8aba1f03969a7c (DEFAULT_TABLE table=multi_table_leader_balance_table1 [id=d05056d1bb2248589cd3132ec8459106]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:16:46.469561 26037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet eaa515c878e14aeead8aba1f03969a7c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:46.483592 26162 tablet_bootstrap.cc:492] T eaa515c878e14aeead8aba1f03969a7c P 41aed3c61bc64062a35dd612d84e1512: Bootstrap starting.
I20260709 12:16:46.487486 26163 tablet_bootstrap.cc:492] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349: Bootstrap starting.
I20260709 12:16:46.495520 26162 tablet_bootstrap.cc:654] T eaa515c878e14aeead8aba1f03969a7c P 41aed3c61bc64062a35dd612d84e1512: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.497138 26163 tablet_bootstrap.cc:654] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.503937 26164 tablet_bootstrap.cc:492] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c: Bootstrap starting.
I20260709 12:16:46.507714 26162 tablet_bootstrap.cc:492] T eaa515c878e14aeead8aba1f03969a7c P 41aed3c61bc64062a35dd612d84e1512: No bootstrap required, opened a new log
I20260709 12:16:46.508314 26162 ts_tablet_manager.cc:1403] T eaa515c878e14aeead8aba1f03969a7c P 41aed3c61bc64062a35dd612d84e1512: Time spent bootstrapping tablet: real 0.025s	user 0.007s	sys 0.012s
I20260709 12:16:46.509485 26163 tablet_bootstrap.cc:492] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349: No bootstrap required, opened a new log
I20260709 12:16:46.510354 26163 ts_tablet_manager.cc:1403] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349: Time spent bootstrapping tablet: real 0.023s	user 0.010s	sys 0.012s
I20260709 12:16:46.511206 26162 raft_consensus.cc:359] T eaa515c878e14aeead8aba1f03969a7c P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.511927 26162 raft_consensus.cc:385] T eaa515c878e14aeead8aba1f03969a7c P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.512200 26162 raft_consensus.cc:740] T eaa515c878e14aeead8aba1f03969a7c P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41aed3c61bc64062a35dd612d84e1512, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.512987 26162 consensus_queue.cc:260] T eaa515c878e14aeead8aba1f03969a7c P 41aed3c61bc64062a35dd612d84e1512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.513617 26164 tablet_bootstrap.cc:654] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.513679 26163 raft_consensus.cc:359] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.514753 26163 raft_consensus.cc:385] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.515192 26163 raft_consensus.cc:740] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 443b0923c6ce405dbc603e9cdea18349, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.516214 26163 consensus_queue.cc:260] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.518492 25998 heartbeater.cc:499] Master 127.11.13.190:44883 was elected leader, sending a full tablet report...
I20260709 12:16:46.521234 26147 heartbeater.cc:499] Master 127.11.13.190:44883 was elected leader, sending a full tablet report...
I20260709 12:16:46.521767 26164 tablet_bootstrap.cc:492] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c: No bootstrap required, opened a new log
I20260709 12:16:46.521965 26163 ts_tablet_manager.cc:1434] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.005s
I20260709 12:16:46.522419 26164 ts_tablet_manager.cc:1403] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c: Time spent bootstrapping tablet: real 0.019s	user 0.014s	sys 0.004s
I20260709 12:16:46.524015 26163 tablet_bootstrap.cc:492] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349: Bootstrap starting.
I20260709 12:16:46.525964 26164 raft_consensus.cc:359] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } } peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } }
I20260709 12:16:46.527004 26164 raft_consensus.cc:385] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.527366 26164 raft_consensus.cc:740] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70728ad802d141dba54935cb3a513e3c, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.528297 26164 consensus_queue.cc:260] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } } peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } }
I20260709 12:16:46.528966 26162 ts_tablet_manager.cc:1434] T eaa515c878e14aeead8aba1f03969a7c P 41aed3c61bc64062a35dd612d84e1512: Time spent starting tablet: real 0.020s	user 0.008s	sys 0.004s
I20260709 12:16:46.532557 26163 tablet_bootstrap.cc:654] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.539815 26162 tablet_bootstrap.cc:492] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512: Bootstrap starting.
I20260709 12:16:46.545049 26163 tablet_bootstrap.cc:492] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349: No bootstrap required, opened a new log
I20260709 12:16:46.545843 26163 ts_tablet_manager.cc:1403] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349: Time spent bootstrapping tablet: real 0.022s	user 0.016s	sys 0.005s
I20260709 12:16:46.545946 26073 heartbeater.cc:499] Master 127.11.13.190:44883 was elected leader, sending a full tablet report...
I20260709 12:16:46.546619 26164 ts_tablet_manager.cc:1434] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c: Time spent starting tablet: real 0.024s	user 0.011s	sys 0.012s
I20260709 12:16:46.549058 26164 tablet_bootstrap.cc:492] T eaa515c878e14aeead8aba1f03969a7c P 70728ad802d141dba54935cb3a513e3c: Bootstrap starting.
I20260709 12:16:46.549476 26162 tablet_bootstrap.cc:654] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.549348 26163 raft_consensus.cc:359] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } } peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } }
I20260709 12:16:46.550557 26163 raft_consensus.cc:385] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.550987 26163 raft_consensus.cc:740] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 443b0923c6ce405dbc603e9cdea18349, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.552103 26163 consensus_queue.cc:260] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } } peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } }
I20260709 12:16:46.554584 26163 ts_tablet_manager.cc:1434] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349: Time spent starting tablet: real 0.008s	user 0.005s	sys 0.002s
I20260709 12:16:46.560196 26164 tablet_bootstrap.cc:654] T eaa515c878e14aeead8aba1f03969a7c P 70728ad802d141dba54935cb3a513e3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.563230 26162 tablet_bootstrap.cc:492] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512: No bootstrap required, opened a new log
I20260709 12:16:46.564028 26162 ts_tablet_manager.cc:1403] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512: Time spent bootstrapping tablet: real 0.025s	user 0.017s	sys 0.004s
I20260709 12:16:46.569185 26162 raft_consensus.cc:359] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } } peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } }
I20260709 12:16:46.569625 26164 tablet_bootstrap.cc:492] T eaa515c878e14aeead8aba1f03969a7c P 70728ad802d141dba54935cb3a513e3c: No bootstrap required, opened a new log
I20260709 12:16:46.569851 26162 raft_consensus.cc:385] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.570343 26162 raft_consensus.cc:740] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41aed3c61bc64062a35dd612d84e1512, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.570361 26164 ts_tablet_manager.cc:1403] T eaa515c878e14aeead8aba1f03969a7c P 70728ad802d141dba54935cb3a513e3c: Time spent bootstrapping tablet: real 0.022s	user 0.018s	sys 0.003s
I20260709 12:16:46.571386 26162 consensus_queue.cc:260] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } } peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } }
I20260709 12:16:46.573904 26162 ts_tablet_manager.cc:1434] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:16:46.574054 26164 raft_consensus.cc:359] T eaa515c878e14aeead8aba1f03969a7c P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.574942 26164 raft_consensus.cc:385] T eaa515c878e14aeead8aba1f03969a7c P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.575232 26164 raft_consensus.cc:740] T eaa515c878e14aeead8aba1f03969a7c P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70728ad802d141dba54935cb3a513e3c, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.576056 26164 consensus_queue.cc:260] T eaa515c878e14aeead8aba1f03969a7c P 70728ad802d141dba54935cb3a513e3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.578336 26164 ts_tablet_manager.cc:1434] T eaa515c878e14aeead8aba1f03969a7c P 70728ad802d141dba54935cb3a513e3c: Time spent starting tablet: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:16:46.589138 26169 raft_consensus.cc:493] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.589740 26169 raft_consensus.cc:515] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.592020 26169 leader_election.cc:290] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41aed3c61bc64062a35dd612d84e1512 (127.11.13.129:34807), 70728ad802d141dba54935cb3a513e3c (127.11.13.130:42375)
I20260709 12:16:46.604095 25973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa515c878e14aeead8aba1f03969a7c" candidate_uuid: "443b0923c6ce405dbc603e9cdea18349" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41aed3c61bc64062a35dd612d84e1512" is_pre_election: true
I20260709 12:16:46.605103 25973 raft_consensus.cc:2468] T eaa515c878e14aeead8aba1f03969a7c P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 443b0923c6ce405dbc603e9cdea18349 in term 0.
I20260709 12:16:46.606776 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa515c878e14aeead8aba1f03969a7c" candidate_uuid: "443b0923c6ce405dbc603e9cdea18349" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70728ad802d141dba54935cb3a513e3c" is_pre_election: true
I20260709 12:16:46.607666 26088 leader_election.cc:304] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41aed3c61bc64062a35dd612d84e1512, 443b0923c6ce405dbc603e9cdea18349; no voters: 
I20260709 12:16:46.607959 26048 raft_consensus.cc:2468] T eaa515c878e14aeead8aba1f03969a7c P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 443b0923c6ce405dbc603e9cdea18349 in term 0.
I20260709 12:16:46.608719 26169 raft_consensus.cc:2804] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:46.609216 26169 raft_consensus.cc:493] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:46.609534 26169 raft_consensus.cc:3060] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.616096 26169 raft_consensus.cc:515] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.618185 26169 leader_election.cc:290] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 [CANDIDATE]: Term 1 election: Requested vote from peers 41aed3c61bc64062a35dd612d84e1512 (127.11.13.129:34807), 70728ad802d141dba54935cb3a513e3c (127.11.13.130:42375)
I20260709 12:16:46.619094 25973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa515c878e14aeead8aba1f03969a7c" candidate_uuid: "443b0923c6ce405dbc603e9cdea18349" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41aed3c61bc64062a35dd612d84e1512"
I20260709 12:16:46.619491 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "eaa515c878e14aeead8aba1f03969a7c" candidate_uuid: "443b0923c6ce405dbc603e9cdea18349" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70728ad802d141dba54935cb3a513e3c"
I20260709 12:16:46.619741 25973 raft_consensus.cc:3060] T eaa515c878e14aeead8aba1f03969a7c P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.620200 26048 raft_consensus.cc:3060] T eaa515c878e14aeead8aba1f03969a7c P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.626612 26048 raft_consensus.cc:2468] T eaa515c878e14aeead8aba1f03969a7c P 70728ad802d141dba54935cb3a513e3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 443b0923c6ce405dbc603e9cdea18349 in term 1.
I20260709 12:16:46.626612 25973 raft_consensus.cc:2468] T eaa515c878e14aeead8aba1f03969a7c P 41aed3c61bc64062a35dd612d84e1512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 443b0923c6ce405dbc603e9cdea18349 in term 1.
I20260709 12:16:46.627923 26086 leader_election.cc:304] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 443b0923c6ce405dbc603e9cdea18349, 70728ad802d141dba54935cb3a513e3c; no voters: 
I20260709 12:16:46.628718 26169 raft_consensus.cc:2804] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:46.630259 26169 raft_consensus.cc:697] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 [term 1 LEADER]: Becoming Leader. State: Replica: 443b0923c6ce405dbc603e9cdea18349, State: Running, Role: LEADER
I20260709 12:16:46.631053 26169 consensus_queue.cc:237] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.640203 25870 catalog_manager.cc:5697] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 reported cstate change: term changed from 0 to 1, leader changed from <none> to 443b0923c6ce405dbc603e9cdea18349 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "443b0923c6ce405dbc603e9cdea18349" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:46.667730 26170 raft_consensus.cc:493] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.667809 26169 raft_consensus.cc:493] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.668473 26170 raft_consensus.cc:515] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } } peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } }
I20260709 12:16:46.668579 26169 raft_consensus.cc:515] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } } peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } }
I20260709 12:16:46.670764 26169 leader_election.cc:290] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41aed3c61bc64062a35dd612d84e1512 (127.11.13.129:34807), 70728ad802d141dba54935cb3a513e3c (127.11.13.130:42375)
I20260709 12:16:46.672793 26170 leader_election.cc:290] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 443b0923c6ce405dbc603e9cdea18349 (127.11.13.131:34239), 41aed3c61bc64062a35dd612d84e1512 (127.11.13.129:34807)
I20260709 12:16:46.673066 25973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a23fa39c3c64ae5a40bc5367291b968" candidate_uuid: "443b0923c6ce405dbc603e9cdea18349" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41aed3c61bc64062a35dd612d84e1512" is_pre_election: true
I20260709 12:16:46.673879 25973 raft_consensus.cc:2468] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 443b0923c6ce405dbc603e9cdea18349 in term 0.
I20260709 12:16:46.673132 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a23fa39c3c64ae5a40bc5367291b968" candidate_uuid: "443b0923c6ce405dbc603e9cdea18349" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70728ad802d141dba54935cb3a513e3c" is_pre_election: true
I20260709 12:16:46.675326 26048 raft_consensus.cc:2468] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 443b0923c6ce405dbc603e9cdea18349 in term 0.
I20260709 12:16:46.675441 26088 leader_election.cc:304] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41aed3c61bc64062a35dd612d84e1512, 443b0923c6ce405dbc603e9cdea18349; no voters: 
I20260709 12:16:46.676496 26169 raft_consensus.cc:2804] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:46.676889 26169 raft_consensus.cc:493] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:46.677299 26169 raft_consensus.cc:3060] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.683218 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a23fa39c3c64ae5a40bc5367291b968" candidate_uuid: "70728ad802d141dba54935cb3a513e3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "443b0923c6ce405dbc603e9cdea18349" is_pre_election: true
I20260709 12:16:46.685051 25973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a23fa39c3c64ae5a40bc5367291b968" candidate_uuid: "70728ad802d141dba54935cb3a513e3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41aed3c61bc64062a35dd612d84e1512" is_pre_election: true
I20260709 12:16:46.685210 26169 raft_consensus.cc:515] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } } peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } }
I20260709 12:16:46.685707 25973 raft_consensus.cc:2468] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70728ad802d141dba54935cb3a513e3c in term 0.
I20260709 12:16:46.686434 26122 raft_consensus.cc:2393] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 70728ad802d141dba54935cb3a513e3c in current term 1: Already voted for candidate 443b0923c6ce405dbc603e9cdea18349 in this term.
I20260709 12:16:46.687018 26014 leader_election.cc:304] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41aed3c61bc64062a35dd612d84e1512, 70728ad802d141dba54935cb3a513e3c; no voters: 
I20260709 12:16:46.688021 26169 leader_election.cc:290] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [CANDIDATE]: Term 1 election: Requested vote from peers 41aed3c61bc64062a35dd612d84e1512 (127.11.13.129:34807), 70728ad802d141dba54935cb3a513e3c (127.11.13.130:42375)
I20260709 12:16:46.688190 26170 raft_consensus.cc:2804] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:46.688688 26170 raft_consensus.cc:493] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:46.688971 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a23fa39c3c64ae5a40bc5367291b968" candidate_uuid: "443b0923c6ce405dbc603e9cdea18349" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70728ad802d141dba54935cb3a513e3c"
I20260709 12:16:46.689036 26170 raft_consensus.cc:3060] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.689015 25973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a23fa39c3c64ae5a40bc5367291b968" candidate_uuid: "443b0923c6ce405dbc603e9cdea18349" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41aed3c61bc64062a35dd612d84e1512"
I20260709 12:16:46.690162 25973 raft_consensus.cc:3060] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.699255 25973 raft_consensus.cc:2468] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 443b0923c6ce405dbc603e9cdea18349 in term 1.
I20260709 12:16:46.699280 26170 raft_consensus.cc:515] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } } peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } }
I20260709 12:16:46.700721 26048 raft_consensus.cc:2393] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 443b0923c6ce405dbc603e9cdea18349 in current term 1: Already voted for candidate 70728ad802d141dba54935cb3a513e3c in this term.
I20260709 12:16:46.701176 26088 leader_election.cc:304] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41aed3c61bc64062a35dd612d84e1512, 443b0923c6ce405dbc603e9cdea18349; no voters: 
I20260709 12:16:46.702728 26169 raft_consensus.cc:2804] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:46.703193 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a23fa39c3c64ae5a40bc5367291b968" candidate_uuid: "70728ad802d141dba54935cb3a513e3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "443b0923c6ce405dbc603e9cdea18349"
I20260709 12:16:46.703222 26169 raft_consensus.cc:697] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [term 1 LEADER]: Becoming Leader. State: Replica: 443b0923c6ce405dbc603e9cdea18349, State: Running, Role: LEADER
I20260709 12:16:46.704033 26170 leader_election.cc:290] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [CANDIDATE]: Term 1 election: Requested vote from peers 443b0923c6ce405dbc603e9cdea18349 (127.11.13.131:34239), 41aed3c61bc64062a35dd612d84e1512 (127.11.13.129:34807)
I20260709 12:16:46.704398 25973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a23fa39c3c64ae5a40bc5367291b968" candidate_uuid: "70728ad802d141dba54935cb3a513e3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41aed3c61bc64062a35dd612d84e1512"
I20260709 12:16:46.704435 26169 consensus_queue.cc:237] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } } peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } }
I20260709 12:16:46.705123 25973 raft_consensus.cc:2393] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 70728ad802d141dba54935cb3a513e3c in current term 1: Already voted for candidate 443b0923c6ce405dbc603e9cdea18349 in this term.
I20260709 12:16:46.711828 26011 leader_election.cc:304] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70728ad802d141dba54935cb3a513e3c; no voters: 41aed3c61bc64062a35dd612d84e1512, 443b0923c6ce405dbc603e9cdea18349
I20260709 12:16:46.712980 26170 raft_consensus.cc:2749] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:16:46.715758 25870 catalog_manager.cc:5697] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 reported cstate change: term changed from 0 to 1, leader changed from <none> to 443b0923c6ce405dbc603e9cdea18349 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "443b0923c6ce405dbc603e9cdea18349" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:46.792217 25870 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37112:
name: "multi_table_leader_balance_table2"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\377\377\377?""\004\001\000\377\377\377?"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:16:46.794806 25870 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table multi_table_leader_balance_table2 in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:16:46.825068 25962 tablet_service.cc:1511] Processing CreateTablet for tablet 7f25f3dd156149cab07950fc9670b346 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=f6ee03792acd4e4fb8e54ec8b9a923d5]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:16:46.826277 26038 tablet_service.cc:1511] Processing CreateTablet for tablet 7f25f3dd156149cab07950fc9670b346 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=f6ee03792acd4e4fb8e54ec8b9a923d5]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:16:46.826974 25962 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f25f3dd156149cab07950fc9670b346. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:46.827994 26038 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f25f3dd156149cab07950fc9670b346. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:46.828881 26112 tablet_service.cc:1511] Processing CreateTablet for tablet 7f25f3dd156149cab07950fc9670b346 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=f6ee03792acd4e4fb8e54ec8b9a923d5]), partition=RANGE (key) PARTITION VALUES < 1073741823
I20260709 12:16:46.830302 26112 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 7f25f3dd156149cab07950fc9670b346. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:46.830935 25919 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:46.831048 25963 tablet_service.cc:1511] Processing CreateTablet for tablet cf004ba446e748b2b44ee97aac888df0 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=f6ee03792acd4e4fb8e54ec8b9a923d5]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:16:46.832517 25963 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf004ba446e748b2b44ee97aac888df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:46.834148 26111 tablet_service.cc:1511] Processing CreateTablet for tablet cf004ba446e748b2b44ee97aac888df0 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=f6ee03792acd4e4fb8e54ec8b9a923d5]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:16:46.835575 26111 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf004ba446e748b2b44ee97aac888df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:46.836068 26037 tablet_service.cc:1511] Processing CreateTablet for tablet cf004ba446e748b2b44ee97aac888df0 (DEFAULT_TABLE table=multi_table_leader_balance_table2 [id=f6ee03792acd4e4fb8e54ec8b9a923d5]), partition=RANGE (key) PARTITION 1073741823 <= VALUES
I20260709 12:16:46.837517 26037 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet cf004ba446e748b2b44ee97aac888df0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:46.850350 26162 tablet_bootstrap.cc:492] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512: Bootstrap starting.
I20260709 12:16:46.850849 26163 tablet_bootstrap.cc:492] T 7f25f3dd156149cab07950fc9670b346 P 443b0923c6ce405dbc603e9cdea18349: Bootstrap starting.
I20260709 12:16:46.860508 26162 tablet_bootstrap.cc:654] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.862212 26163 tablet_bootstrap.cc:654] T 7f25f3dd156149cab07950fc9670b346 P 443b0923c6ce405dbc603e9cdea18349: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.872777 26163 tablet_bootstrap.cc:492] T 7f25f3dd156149cab07950fc9670b346 P 443b0923c6ce405dbc603e9cdea18349: No bootstrap required, opened a new log
I20260709 12:16:46.873006 26162 tablet_bootstrap.cc:492] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512: No bootstrap required, opened a new log
I20260709 12:16:46.873379 26163 ts_tablet_manager.cc:1403] T 7f25f3dd156149cab07950fc9670b346 P 443b0923c6ce405dbc603e9cdea18349: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.000s
I20260709 12:16:46.873553 26162 ts_tablet_manager.cc:1403] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512: Time spent bootstrapping tablet: real 0.023s	user 0.015s	sys 0.004s
I20260709 12:16:46.876806 26163 raft_consensus.cc:359] T 7f25f3dd156149cab07950fc9670b346 P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.877655 26163 raft_consensus.cc:385] T 7f25f3dd156149cab07950fc9670b346 P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.877971 26163 raft_consensus.cc:740] T 7f25f3dd156149cab07950fc9670b346 P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 443b0923c6ce405dbc603e9cdea18349, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.878285 26164 tablet_bootstrap.cc:492] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c: Bootstrap starting.
I20260709 12:16:46.878836 26163 consensus_queue.cc:260] T 7f25f3dd156149cab07950fc9670b346 P 443b0923c6ce405dbc603e9cdea18349 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.879825 26162 raft_consensus.cc:359] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.880658 26162 raft_consensus.cc:385] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.880997 26162 raft_consensus.cc:740] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41aed3c61bc64062a35dd612d84e1512, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.881742 26163 ts_tablet_manager.cc:1434] T 7f25f3dd156149cab07950fc9670b346 P 443b0923c6ce405dbc603e9cdea18349: Time spent starting tablet: real 0.008s	user 0.001s	sys 0.007s
I20260709 12:16:46.881846 26162 consensus_queue.cc:260] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.883091 26163 tablet_bootstrap.cc:492] T cf004ba446e748b2b44ee97aac888df0 P 443b0923c6ce405dbc603e9cdea18349: Bootstrap starting.
I20260709 12:16:46.884614 26162 ts_tablet_manager.cc:1434] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512: Time spent starting tablet: real 0.011s	user 0.004s	sys 0.004s
I20260709 12:16:46.890010 26164 tablet_bootstrap.cc:654] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.890762 26162 tablet_bootstrap.cc:492] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512: Bootstrap starting.
I20260709 12:16:46.891706 26163 tablet_bootstrap.cc:654] T cf004ba446e748b2b44ee97aac888df0 P 443b0923c6ce405dbc603e9cdea18349: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.900978 26162 tablet_bootstrap.cc:654] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.902628 26164 tablet_bootstrap.cc:492] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c: No bootstrap required, opened a new log
I20260709 12:16:46.903626 26164 ts_tablet_manager.cc:1403] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c: Time spent bootstrapping tablet: real 0.026s	user 0.014s	sys 0.007s
I20260709 12:16:46.907999 26164 raft_consensus.cc:359] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.910563 26164 raft_consensus.cc:385] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.910949 26164 raft_consensus.cc:740] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70728ad802d141dba54935cb3a513e3c, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.911761 26164 consensus_queue.cc:260] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.913990 26163 tablet_bootstrap.cc:492] T cf004ba446e748b2b44ee97aac888df0 P 443b0923c6ce405dbc603e9cdea18349: No bootstrap required, opened a new log
I20260709 12:16:46.914239 26164 ts_tablet_manager.cc:1434] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c: Time spent starting tablet: real 0.010s	user 0.009s	sys 0.000s
I20260709 12:16:46.914990 26163 ts_tablet_manager.cc:1403] T cf004ba446e748b2b44ee97aac888df0 P 443b0923c6ce405dbc603e9cdea18349: Time spent bootstrapping tablet: real 0.032s	user 0.013s	sys 0.016s
I20260709 12:16:46.916010 26164 tablet_bootstrap.cc:492] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c: Bootstrap starting.
I20260709 12:16:46.917973 26162 tablet_bootstrap.cc:492] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512: No bootstrap required, opened a new log
I20260709 12:16:46.919796 26162 ts_tablet_manager.cc:1403] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512: Time spent bootstrapping tablet: real 0.029s	user 0.009s	sys 0.017s
I20260709 12:16:46.918560 26163 raft_consensus.cc:359] T cf004ba446e748b2b44ee97aac888df0 P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.920979 26163 raft_consensus.cc:385] T cf004ba446e748b2b44ee97aac888df0 P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.921530 26163 raft_consensus.cc:740] T cf004ba446e748b2b44ee97aac888df0 P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 443b0923c6ce405dbc603e9cdea18349, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.922508 26163 consensus_queue.cc:260] T cf004ba446e748b2b44ee97aac888df0 P 443b0923c6ce405dbc603e9cdea18349 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.923821 26162 raft_consensus.cc:359] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.924736 26162 raft_consensus.cc:385] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.925022 26163 ts_tablet_manager.cc:1434] T cf004ba446e748b2b44ee97aac888df0 P 443b0923c6ce405dbc603e9cdea18349: Time spent starting tablet: real 0.010s	user 0.005s	sys 0.001s
I20260709 12:16:46.925113 26162 raft_consensus.cc:740] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41aed3c61bc64062a35dd612d84e1512, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.926441 26162 consensus_queue.cc:260] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.927055 26164 tablet_bootstrap.cc:654] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:46.929277 26162 ts_tablet_manager.cc:1434] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.000s
I20260709 12:16:46.935211 26164 tablet_bootstrap.cc:492] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c: No bootstrap required, opened a new log
I20260709 12:16:46.936343 26164 ts_tablet_manager.cc:1403] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c: Time spent bootstrapping tablet: real 0.021s	user 0.012s	sys 0.004s
I20260709 12:16:46.939291 26164 raft_consensus.cc:359] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.940052 26164 raft_consensus.cc:385] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:46.940351 26164 raft_consensus.cc:740] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70728ad802d141dba54935cb3a513e3c, State: Initialized, Role: FOLLOWER
I20260709 12:16:46.940999 26164 consensus_queue.cc:260] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.943216 26164 ts_tablet_manager.cc:1434] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:46.946547 26170 raft_consensus.cc:493] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:46.947220 26170 raft_consensus.cc:515] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.949574 26170 leader_election.cc:290] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41aed3c61bc64062a35dd612d84e1512 (127.11.13.129:34807), 443b0923c6ce405dbc603e9cdea18349 (127.11.13.131:34239)
I20260709 12:16:46.949702 25973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf004ba446e748b2b44ee97aac888df0" candidate_uuid: "70728ad802d141dba54935cb3a513e3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41aed3c61bc64062a35dd612d84e1512" is_pre_election: true
I20260709 12:16:46.950502 25973 raft_consensus.cc:2468] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70728ad802d141dba54935cb3a513e3c in term 0.
I20260709 12:16:46.951073 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf004ba446e748b2b44ee97aac888df0" candidate_uuid: "70728ad802d141dba54935cb3a513e3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "443b0923c6ce405dbc603e9cdea18349" is_pre_election: true
I20260709 12:16:46.951570 26014 leader_election.cc:304] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41aed3c61bc64062a35dd612d84e1512, 70728ad802d141dba54935cb3a513e3c; no voters: 
I20260709 12:16:46.951849 26122 raft_consensus.cc:2468] T cf004ba446e748b2b44ee97aac888df0 P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70728ad802d141dba54935cb3a513e3c in term 0.
I20260709 12:16:46.952889 26170 raft_consensus.cc:2804] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:46.953269 26170 raft_consensus.cc:493] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:46.953538 26170 raft_consensus.cc:3060] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.960423 26170 raft_consensus.cc:515] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.962428 26170 leader_election.cc:290] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [CANDIDATE]: Term 1 election: Requested vote from peers 41aed3c61bc64062a35dd612d84e1512 (127.11.13.129:34807), 443b0923c6ce405dbc603e9cdea18349 (127.11.13.131:34239)
I20260709 12:16:46.963717 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf004ba446e748b2b44ee97aac888df0" candidate_uuid: "70728ad802d141dba54935cb3a513e3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "443b0923c6ce405dbc603e9cdea18349"
I20260709 12:16:46.963838 25973 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf004ba446e748b2b44ee97aac888df0" candidate_uuid: "70728ad802d141dba54935cb3a513e3c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41aed3c61bc64062a35dd612d84e1512"
I20260709 12:16:46.964532 26122 raft_consensus.cc:3060] T cf004ba446e748b2b44ee97aac888df0 P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.964689 25973 raft_consensus.cc:3060] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:46.970935 25973 raft_consensus.cc:2468] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70728ad802d141dba54935cb3a513e3c in term 1.
I20260709 12:16:46.971896 26122 raft_consensus.cc:2468] T cf004ba446e748b2b44ee97aac888df0 P 443b0923c6ce405dbc603e9cdea18349 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70728ad802d141dba54935cb3a513e3c in term 1.
I20260709 12:16:46.973054 26014 leader_election.cc:304] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41aed3c61bc64062a35dd612d84e1512, 70728ad802d141dba54935cb3a513e3c; no voters: 
I20260709 12:16:46.974364 26170 raft_consensus.cc:2804] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:46.976955 26170 raft_consensus.cc:697] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [term 1 LEADER]: Becoming Leader. State: Replica: 70728ad802d141dba54935cb3a513e3c, State: Running, Role: LEADER
I20260709 12:16:46.978511 26170 consensus_queue.cc:237] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:46.987728 25871 catalog_manager.cc:5697] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c reported cstate change: term changed from 0 to 1, leader changed from <none> to 70728ad802d141dba54935cb3a513e3c (127.11.13.130). New cstate: current_term: 1 leader_uuid: "70728ad802d141dba54935cb3a513e3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:47.091126 26169 consensus_queue.cc:1048] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:47.095757 26167 raft_consensus.cc:493] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:47.096370 26167 raft_consensus.cc:515] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:47.100183 26167 leader_election.cc:290] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 70728ad802d141dba54935cb3a513e3c (127.11.13.130:42375), 443b0923c6ce405dbc603e9cdea18349 (127.11.13.131:34239)
I20260709 12:16:47.120417 26175 consensus_queue.cc:1048] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:47.127785 26048 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f25f3dd156149cab07950fc9670b346" candidate_uuid: "41aed3c61bc64062a35dd612d84e1512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70728ad802d141dba54935cb3a513e3c" is_pre_election: true
I20260709 12:16:47.127810 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f25f3dd156149cab07950fc9670b346" candidate_uuid: "41aed3c61bc64062a35dd612d84e1512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "443b0923c6ce405dbc603e9cdea18349" is_pre_election: true
I20260709 12:16:47.129215 26048 raft_consensus.cc:2468] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41aed3c61bc64062a35dd612d84e1512 in term 0.
I20260709 12:16:47.129240 26122 raft_consensus.cc:2468] T 7f25f3dd156149cab07950fc9670b346 P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 41aed3c61bc64062a35dd612d84e1512 in term 0.
I20260709 12:16:47.131778 25936 leader_election.cc:304] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41aed3c61bc64062a35dd612d84e1512, 443b0923c6ce405dbc603e9cdea18349; no voters: 
I20260709 12:16:47.133225 26167 raft_consensus.cc:2804] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:47.133684 26167 raft_consensus.cc:493] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:47.134055 26167 raft_consensus.cc:3060] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.144174 26167 raft_consensus.cc:515] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:47.148175 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f25f3dd156149cab07950fc9670b346" candidate_uuid: "41aed3c61bc64062a35dd612d84e1512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "70728ad802d141dba54935cb3a513e3c"
I20260709 12:16:47.148207 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f25f3dd156149cab07950fc9670b346" candidate_uuid: "41aed3c61bc64062a35dd612d84e1512" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "443b0923c6ce405dbc603e9cdea18349"
I20260709 12:16:47.149188 26122 raft_consensus.cc:3060] T 7f25f3dd156149cab07950fc9670b346 P 443b0923c6ce405dbc603e9cdea18349 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.147564 26167 leader_election.cc:290] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [CANDIDATE]: Term 1 election: Requested vote from peers 70728ad802d141dba54935cb3a513e3c (127.11.13.130:42375), 443b0923c6ce405dbc603e9cdea18349 (127.11.13.131:34239)
I20260709 12:16:47.149183 26047 raft_consensus.cc:3060] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:47.159031 26122 raft_consensus.cc:2468] T 7f25f3dd156149cab07950fc9670b346 P 443b0923c6ce405dbc603e9cdea18349 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41aed3c61bc64062a35dd612d84e1512 in term 1.
I20260709 12:16:47.160422 26047 raft_consensus.cc:2468] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41aed3c61bc64062a35dd612d84e1512 in term 1.
I20260709 12:16:47.160490 25936 leader_election.cc:304] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41aed3c61bc64062a35dd612d84e1512, 443b0923c6ce405dbc603e9cdea18349; no voters: 
I20260709 12:16:47.161626 26167 raft_consensus.cc:2804] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:47.163206 26167 raft_consensus.cc:697] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [term 1 LEADER]: Becoming Leader. State: Replica: 41aed3c61bc64062a35dd612d84e1512, State: Running, Role: LEADER
I20260709 12:16:47.164041 26167 consensus_queue.cc:237] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:47.172169 25871 catalog_manager.cc:5697] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 reported cstate change: term changed from 0 to 1, leader changed from <none> to 41aed3c61bc64062a35dd612d84e1512 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "41aed3c61bc64062a35dd612d84e1512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:47.211172 26122 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2a23fa39c3c64ae5a40bc5367291b968"
dest_uuid: "443b0923c6ce405dbc603e9cdea18349"
mode: GRACEFUL
new_leader_uuid: "41aed3c61bc64062a35dd612d84e1512"
 from {username='slave'} at 127.0.0.1:38492
I20260709 12:16:47.211745 26122 raft_consensus.cc:606] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [term 1 LEADER]: Received request to transfer leadership to TS 41aed3c61bc64062a35dd612d84e1512
I20260709 12:16:47.230922 26175 consensus_queue.cc:1048] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:47.239256 26047 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "cf004ba446e748b2b44ee97aac888df0"
dest_uuid: "70728ad802d141dba54935cb3a513e3c"
mode: GRACEFUL
new_leader_uuid: "41aed3c61bc64062a35dd612d84e1512"
 from {username='slave'} at 127.0.0.1:44398
I20260709 12:16:47.239647 26197 raft_consensus.cc:993] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349: : Instructing follower 41aed3c61bc64062a35dd612d84e1512 to start an election
I20260709 12:16:47.240334 26047 raft_consensus.cc:606] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [term 1 LEADER]: Received request to transfer leadership to TS 41aed3c61bc64062a35dd612d84e1512
I20260709 12:16:47.240559 26197 raft_consensus.cc:1081] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [term 1 LEADER]: Signalling peer 41aed3c61bc64062a35dd612d84e1512 to start an election
I20260709 12:16:47.246613 25973 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2a23fa39c3c64ae5a40bc5367291b968"
dest_uuid: "41aed3c61bc64062a35dd612d84e1512"
 from {username='slave'} at 127.0.0.1:47000
I20260709 12:16:47.247403 25973 raft_consensus.cc:493] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:47.247907 25973 raft_consensus.cc:3060] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:47.252064 26198 consensus_queue.cc:1048] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:47.257097 25973 raft_consensus.cc:515] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } } peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } }
I20260709 12:16:47.260891 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a23fa39c3c64ae5a40bc5367291b968" candidate_uuid: "41aed3c61bc64062a35dd612d84e1512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "443b0923c6ce405dbc603e9cdea18349"
I20260709 12:16:47.261677 26122 raft_consensus.cc:3055] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:47.262068 26122 raft_consensus.cc:740] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 443b0923c6ce405dbc603e9cdea18349, State: Running, Role: LEADER
I20260709 12:16:47.263046 26122 consensus_queue.cc:260] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } } peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } }
I20260709 12:16:47.264565 26122 raft_consensus.cc:3060] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:47.270800 25973 leader_election.cc:290] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 [CANDIDATE]: Term 2 election: Requested vote from peers 443b0923c6ce405dbc603e9cdea18349 (127.11.13.131:34239), 70728ad802d141dba54935cb3a513e3c (127.11.13.130:42375)
I20260709 12:16:47.272384 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2a23fa39c3c64ae5a40bc5367291b968" candidate_uuid: "41aed3c61bc64062a35dd612d84e1512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "70728ad802d141dba54935cb3a513e3c"
I20260709 12:16:47.273124 26047 raft_consensus.cc:3060] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:47.275786 26122 raft_consensus.cc:2468] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41aed3c61bc64062a35dd612d84e1512 in term 2.
I20260709 12:16:47.277315 25936 leader_election.cc:304] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41aed3c61bc64062a35dd612d84e1512, 443b0923c6ce405dbc603e9cdea18349; no voters: 
I20260709 12:16:47.278347 26167 raft_consensus.cc:2804] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:47.278983 26167 raft_consensus.cc:697] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 [term 2 LEADER]: Becoming Leader. State: Replica: 41aed3c61bc64062a35dd612d84e1512, State: Running, Role: LEADER
I20260709 12:16:47.280036 26167 consensus_queue.cc:237] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } } peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } }
I20260709 12:16:47.282841 26047 raft_consensus.cc:2468] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41aed3c61bc64062a35dd612d84e1512 in term 2.
I20260709 12:16:47.290302 25869 catalog_manager.cc:5697] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 reported cstate change: term changed from 1 to 2, leader changed from 443b0923c6ce405dbc603e9cdea18349 (127.11.13.131) to 41aed3c61bc64062a35dd612d84e1512 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "41aed3c61bc64062a35dd612d84e1512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:47.408843 26170 consensus_queue.cc:1048] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:47.417371 26190 raft_consensus.cc:993] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c: : Instructing follower 41aed3c61bc64062a35dd612d84e1512 to start an election
I20260709 12:16:47.418617 26170 raft_consensus.cc:1081] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [term 1 LEADER]: Signalling peer 41aed3c61bc64062a35dd612d84e1512 to start an election
I20260709 12:16:47.424242 25972 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "cf004ba446e748b2b44ee97aac888df0"
dest_uuid: "41aed3c61bc64062a35dd612d84e1512"
 from {username='slave'} at 127.0.0.1:47016
I20260709 12:16:47.424973 25972 raft_consensus.cc:493] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:47.425411 25972 raft_consensus.cc:3060] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:47.426334 26208 consensus_queue.cc:1048] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [LEADER]: Connected to new peer: Peer: permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:47.436132 25972 raft_consensus.cc:515] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:47.439502 26047 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf004ba446e748b2b44ee97aac888df0" candidate_uuid: "41aed3c61bc64062a35dd612d84e1512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "70728ad802d141dba54935cb3a513e3c"
I20260709 12:16:47.440152 26047 raft_consensus.cc:3055] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:47.440501 26047 raft_consensus.cc:740] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 70728ad802d141dba54935cb3a513e3c, State: Running, Role: LEADER
I20260709 12:16:47.441397 26047 consensus_queue.cc:260] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:47.442710 26047 raft_consensus.cc:3060] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:47.443502 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "cf004ba446e748b2b44ee97aac888df0" candidate_uuid: "41aed3c61bc64062a35dd612d84e1512" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "443b0923c6ce405dbc603e9cdea18349"
I20260709 12:16:47.444216 26122 raft_consensus.cc:3060] T cf004ba446e748b2b44ee97aac888df0 P 443b0923c6ce405dbc603e9cdea18349 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:47.447554 25972 leader_election.cc:290] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512 [CANDIDATE]: Term 2 election: Requested vote from peers 70728ad802d141dba54935cb3a513e3c (127.11.13.130:42375), 443b0923c6ce405dbc603e9cdea18349 (127.11.13.131:34239)
I20260709 12:16:47.452026 26047 raft_consensus.cc:2468] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41aed3c61bc64062a35dd612d84e1512 in term 2.
I20260709 12:16:47.452630 26122 raft_consensus.cc:2468] T cf004ba446e748b2b44ee97aac888df0 P 443b0923c6ce405dbc603e9cdea18349 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 41aed3c61bc64062a35dd612d84e1512 in term 2.
I20260709 12:16:47.453228 25937 leader_election.cc:304] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41aed3c61bc64062a35dd612d84e1512, 70728ad802d141dba54935cb3a513e3c; no voters: 
I20260709 12:16:47.454154 26167 raft_consensus.cc:2804] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:47.454669 26167 raft_consensus.cc:697] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512 [term 2 LEADER]: Becoming Leader. State: Replica: 41aed3c61bc64062a35dd612d84e1512, State: Running, Role: LEADER
I20260709 12:16:47.455539 26167 consensus_queue.cc:237] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:47.463028 25870 catalog_manager.cc:5697] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512 reported cstate change: term changed from 1 to 2, leader changed from 70728ad802d141dba54935cb3a513e3c (127.11.13.130) to 41aed3c61bc64062a35dd612d84e1512 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "41aed3c61bc64062a35dd612d84e1512" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:47.518946 11318 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table1
I20260709 12:16:47.520663 11318 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table1, leader rebalance finish, leader transfer count: 0
I20260709 12:16:47.521212 11318 auto_leader_rebalancer.cc:141] leader rebalance for table multi_table_leader_balance_table2
I20260709 12:16:47.535789 25972 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "7f25f3dd156149cab07950fc9670b346"
dest_uuid: "41aed3c61bc64062a35dd612d84e1512"
mode: GRACEFUL
new_leader_uuid: "70728ad802d141dba54935cb3a513e3c"
 from {username='slave'} at 127.0.0.1:47030
I20260709 12:16:47.536505 25972 raft_consensus.cc:606] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [term 1 LEADER]: Received request to transfer leadership to TS 70728ad802d141dba54935cb3a513e3c
I20260709 12:16:47.538383 11318 auto_leader_rebalancer.cc:440] table: multi_table_leader_balance_table2, leader rebalance finish, leader transfer count: 1
I20260709 12:16:47.717423 26167 consensus_queue.cc:1048] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:47.731920 26167 consensus_queue.cc:1048] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:47.741731 26213 raft_consensus.cc:993] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512: : Instructing follower 70728ad802d141dba54935cb3a513e3c to start an election
I20260709 12:16:47.742317 26213 raft_consensus.cc:1081] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [term 1 LEADER]: Signalling peer 70728ad802d141dba54935cb3a513e3c to start an election
I20260709 12:16:47.744447 26047 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "7f25f3dd156149cab07950fc9670b346"
dest_uuid: "70728ad802d141dba54935cb3a513e3c"
 from {username='slave'} at 127.0.0.1:44390
I20260709 12:16:47.745098 26047 raft_consensus.cc:493] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:47.745426 26047 raft_consensus.cc:3060] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:47.752666 26047 raft_consensus.cc:515] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:47.754700 26047 leader_election.cc:290] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c [CANDIDATE]: Term 2 election: Requested vote from peers 41aed3c61bc64062a35dd612d84e1512 (127.11.13.129:34807), 443b0923c6ce405dbc603e9cdea18349 (127.11.13.131:34239)
I20260709 12:16:47.755617 25972 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f25f3dd156149cab07950fc9670b346" candidate_uuid: "70728ad802d141dba54935cb3a513e3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "41aed3c61bc64062a35dd612d84e1512"
I20260709 12:16:47.755909 26122 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "7f25f3dd156149cab07950fc9670b346" candidate_uuid: "70728ad802d141dba54935cb3a513e3c" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "443b0923c6ce405dbc603e9cdea18349"
I20260709 12:16:47.756282 25972 raft_consensus.cc:3055] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:47.756587 26122 raft_consensus.cc:3060] T 7f25f3dd156149cab07950fc9670b346 P 443b0923c6ce405dbc603e9cdea18349 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:47.756664 25972 raft_consensus.cc:740] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 41aed3c61bc64062a35dd612d84e1512, State: Running, Role: LEADER
I20260709 12:16:47.757655 25972 consensus_queue.cc:260] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:47.758790 25972 raft_consensus.cc:3060] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:47.764353 26122 raft_consensus.cc:2468] T 7f25f3dd156149cab07950fc9670b346 P 443b0923c6ce405dbc603e9cdea18349 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70728ad802d141dba54935cb3a513e3c in term 2.
I20260709 12:16:47.766052 26011 leader_election.cc:304] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 443b0923c6ce405dbc603e9cdea18349, 70728ad802d141dba54935cb3a513e3c; no voters: 
I20260709 12:16:47.766752 25972 raft_consensus.cc:2468] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70728ad802d141dba54935cb3a513e3c in term 2.
I20260709 12:16:47.766922 26208 raft_consensus.cc:2804] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:47.767580 26208 raft_consensus.cc:697] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c [term 2 LEADER]: Becoming Leader. State: Replica: 70728ad802d141dba54935cb3a513e3c, State: Running, Role: LEADER
I20260709 12:16:47.768409 26208 consensus_queue.cc:237] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } }
I20260709 12:16:47.776508 25869 catalog_manager.cc:5697] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c reported cstate change: term changed from 1 to 2, leader changed from 41aed3c61bc64062a35dd612d84e1512 (127.11.13.129) to 70728ad802d141dba54935cb3a513e3c (127.11.13.130). New cstate: current_term: 2 leader_uuid: "70728ad802d141dba54935cb3a513e3c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41aed3c61bc64062a35dd612d84e1512" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 34807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:47.851946 26047 raft_consensus.cc:1275] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [term 2 FOLLOWER]: Refusing update from remote peer 41aed3c61bc64062a35dd612d84e1512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:47.854367 26213 consensus_queue.cc:1048] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:47.864028 26122 raft_consensus.cc:1275] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [term 2 FOLLOWER]: Refusing update from remote peer 41aed3c61bc64062a35dd612d84e1512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:47.866686 26213 consensus_queue.cc:1048] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:47.876289 26121 raft_consensus.cc:1275] T cf004ba446e748b2b44ee97aac888df0 P 443b0923c6ce405dbc603e9cdea18349 [term 2 FOLLOWER]: Refusing update from remote peer 41aed3c61bc64062a35dd612d84e1512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:47.878340 26213 consensus_queue.cc:1048] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "443b0923c6ce405dbc603e9cdea18349" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 34239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:47.887887 26047 raft_consensus.cc:1275] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [term 2 FOLLOWER]: Refusing update from remote peer 41aed3c61bc64062a35dd612d84e1512: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:47.889801 26202 consensus_queue.cc:1048] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512 [LEADER]: Connected to new peer: Peer: permanent_uuid: "70728ad802d141dba54935cb3a513e3c" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 42375 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:47.919411 11318 tablet_server.cc:179] TabletServer@127.11.13.129:0 shutting down...
I20260709 12:16:47.948707 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:47.949690 11318 tablet_replica.cc:333] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512: stopping tablet replica
I20260709 12:16:47.950544 11318 raft_consensus.cc:2243] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.951597 11318 raft_consensus.cc:2272] T 2a23fa39c3c64ae5a40bc5367291b968 P 41aed3c61bc64062a35dd612d84e1512 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.954480 11318 tablet_replica.cc:333] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512: stopping tablet replica
I20260709 12:16:47.955045 11318 raft_consensus.cc:2243] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:47.955837 11318 raft_consensus.cc:2272] T cf004ba446e748b2b44ee97aac888df0 P 41aed3c61bc64062a35dd612d84e1512 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.958636 11318 tablet_replica.cc:333] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512: stopping tablet replica
I20260709 12:16:47.959136 11318 raft_consensus.cc:2243] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:47.959528 11318 raft_consensus.cc:2272] T 7f25f3dd156149cab07950fc9670b346 P 41aed3c61bc64062a35dd612d84e1512 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.961817 11318 tablet_replica.cc:333] T eaa515c878e14aeead8aba1f03969a7c P 41aed3c61bc64062a35dd612d84e1512: stopping tablet replica
I20260709 12:16:47.962411 11318 raft_consensus.cc:2243] T eaa515c878e14aeead8aba1f03969a7c P 41aed3c61bc64062a35dd612d84e1512 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:47.962802 11318 raft_consensus.cc:2272] T eaa515c878e14aeead8aba1f03969a7c P 41aed3c61bc64062a35dd612d84e1512 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:47.978991 11318 tablet_server.cc:196] TabletServer@127.11.13.129:0 shutdown complete.
I20260709 12:16:47.992851 11318 tablet_server.cc:179] TabletServer@127.11.13.130:0 shutting down...
I20260709 12:16:48.016726 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:48.017695 11318 tablet_replica.cc:333] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c: stopping tablet replica
I20260709 12:16:48.018536 11318 raft_consensus.cc:2243] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:16:48.019304 11318 pending_rounds.cc:70] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c: Trying to abort 1 pending ops.
I20260709 12:16:48.019498 11318 pending_rounds.cc:77] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c: Aborting op as it isn't in flight: id { term: 2 index: 2 } timestamp: 7305623174229835776 op_type: NO_OP noop_request { }
I20260709 12:16:48.019830 11318 raft_consensus.cc:2889] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c [term 2 LEADER]: NO_OP replication failed: Aborted: Op aborted
I20260709 12:16:48.020172 11318 raft_consensus.cc:2272] T 7f25f3dd156149cab07950fc9670b346 P 70728ad802d141dba54935cb3a513e3c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:48.022876 11318 tablet_replica.cc:333] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c: stopping tablet replica
I20260709 12:16:48.023415 11318 raft_consensus.cc:2243] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:48.023855 11318 raft_consensus.cc:2272] T cf004ba446e748b2b44ee97aac888df0 P 70728ad802d141dba54935cb3a513e3c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:48.026108 11318 tablet_replica.cc:333] T eaa515c878e14aeead8aba1f03969a7c P 70728ad802d141dba54935cb3a513e3c: stopping tablet replica
I20260709 12:16:48.026834 11318 raft_consensus.cc:2243] T eaa515c878e14aeead8aba1f03969a7c P 70728ad802d141dba54935cb3a513e3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:48.027233 11318 raft_consensus.cc:2272] T eaa515c878e14aeead8aba1f03969a7c P 70728ad802d141dba54935cb3a513e3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:48.029732 11318 tablet_replica.cc:333] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c: stopping tablet replica
I20260709 12:16:48.030385 11318 raft_consensus.cc:2243] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:48.030833 11318 raft_consensus.cc:2272] T 2a23fa39c3c64ae5a40bc5367291b968 P 70728ad802d141dba54935cb3a513e3c [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:48.056833 11318 tablet_server.cc:196] TabletServer@127.11.13.130:0 shutdown complete.
I20260709 12:16:48.071511 11318 tablet_server.cc:179] TabletServer@127.11.13.131:0 shutting down...
I20260709 12:16:48.096717 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:16:48.097440 11318 tablet_replica.cc:333] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349: stopping tablet replica
I20260709 12:16:48.098060 11318 raft_consensus.cc:2243] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:48.098716 11318 raft_consensus.cc:2272] T 2a23fa39c3c64ae5a40bc5367291b968 P 443b0923c6ce405dbc603e9cdea18349 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:48.101588 11318 tablet_replica.cc:333] T cf004ba446e748b2b44ee97aac888df0 P 443b0923c6ce405dbc603e9cdea18349: stopping tablet replica
I20260709 12:16:48.102350 11318 raft_consensus.cc:2243] T cf004ba446e748b2b44ee97aac888df0 P 443b0923c6ce405dbc603e9cdea18349 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:48.102885 11318 raft_consensus.cc:2272] T cf004ba446e748b2b44ee97aac888df0 P 443b0923c6ce405dbc603e9cdea18349 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:48.105934 11318 tablet_replica.cc:333] T 7f25f3dd156149cab07950fc9670b346 P 443b0923c6ce405dbc603e9cdea18349: stopping tablet replica
W20260709 12:16:48.105887 26088 consensus_peers.cc:597] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 -> Peer 41aed3c61bc64062a35dd612d84e1512 (127.11.13.129:34807): Couldn't send request to peer 41aed3c61bc64062a35dd612d84e1512. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:34807: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:16:48.106845 11318 raft_consensus.cc:2243] T 7f25f3dd156149cab07950fc9670b346 P 443b0923c6ce405dbc603e9cdea18349 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:16:48.107374 11318 raft_consensus.cc:2272] T 7f25f3dd156149cab07950fc9670b346 P 443b0923c6ce405dbc603e9cdea18349 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:48.110044 11318 tablet_replica.cc:333] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349: stopping tablet replica
I20260709 12:16:48.111069 11318 raft_consensus.cc:2243] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:48.112291 11318 raft_consensus.cc:2272] T eaa515c878e14aeead8aba1f03969a7c P 443b0923c6ce405dbc603e9cdea18349 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:48.138628 11318 tablet_server.cc:196] TabletServer@127.11.13.131:0 shutdown complete.
I20260709 12:16:48.152979 11318 master.cc:562] Master@127.11.13.190:44883 shutting down...
I20260709 12:16:48.177538 11318 raft_consensus.cc:2243] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:16:48.178381 11318 raft_consensus.cc:2272] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:16:48.178773 11318 tablet_replica.cc:333] T 00000000000000000000000000000000 P b8f42886d4d2406d917b306b45331bb0: stopping tablet replica
I20260709 12:16:48.193431 11318 master.cc:584] Master@127.11.13.190:44883 shutdown complete.
[       OK ] LeaderRebalancerTest.MultiTableLeaderBalance (2538 ms)
[----------] 6 tests from LeaderRebalancerTest (515994 ms total)

[----------] 2 tests from FilterSoftDeletedTableTest
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0
I20260709 12:16:48.230850 11318 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.11.13.190:45359
I20260709 12:16:48.232185 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:48.239275 26217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:48.240033 26218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:48.241202 26220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:48.242798 11318 server_base.cc:1061] running on GCE node
I20260709 12:16:48.243722 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:48.243916 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:48.244129 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599408244113 us; error 0 us; skew 500 ppm
I20260709 12:16:48.244789 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:48.248060 11318 webserver.cc:533] Webserver started at http://127.11.13.190:38651/ using document root <none> and password file <none>
I20260709 12:16:48.248617 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:48.248821 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:48.249135 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:48.250864 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598892056905-11318-0/minicluster-data/master-0-root/instance:
uuid: "8de7d5a07fb447aaa68b7f3acce9665f"
format_stamp: "Formatted at 2026-07-09 12:16:48 on dist-test-slave-zqnl"
I20260709 12:16:48.256932 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.005s	sys 0.000s
I20260709 12:16:48.261022 26225 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:48.262244 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:16:48.262717 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598892056905-11318-0/minicluster-data/master-0-root
uuid: "8de7d5a07fb447aaa68b7f3acce9665f"
format_stamp: "Formatted at 2026-07-09 12:16:48 on dist-test-slave-zqnl"
I20260709 12:16:48.263062 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598892056905-11318-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598892056905-11318-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598892056905-11318-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:48.281157 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:48.282490 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:48.326994 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.190:45359
I20260709 12:16:48.327183 26276 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.190:45359 every 8 connection(s)
I20260709 12:16:48.331488 26277 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:48.344338 26277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f: Bootstrap starting.
I20260709 12:16:48.349566 26277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:48.354525 26277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f: No bootstrap required, opened a new log
I20260709 12:16:48.357108 26277 raft_consensus.cc:359] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de7d5a07fb447aaa68b7f3acce9665f" member_type: VOTER }
I20260709 12:16:48.357564 26277 raft_consensus.cc:385] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:48.357890 26277 raft_consensus.cc:740] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8de7d5a07fb447aaa68b7f3acce9665f, State: Initialized, Role: FOLLOWER
I20260709 12:16:48.358624 26277 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de7d5a07fb447aaa68b7f3acce9665f" member_type: VOTER }
I20260709 12:16:48.359162 26277 raft_consensus.cc:399] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:16:48.359392 26277 raft_consensus.cc:493] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:16:48.359719 26277 raft_consensus.cc:3060] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:48.365345 26277 raft_consensus.cc:515] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de7d5a07fb447aaa68b7f3acce9665f" member_type: VOTER }
I20260709 12:16:48.365993 26277 leader_election.cc:304] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8de7d5a07fb447aaa68b7f3acce9665f; no voters: 
I20260709 12:16:48.367311 26277 leader_election.cc:290] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:16:48.367705 26280 raft_consensus.cc:2804] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:48.369139 26280 raft_consensus.cc:697] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f [term 1 LEADER]: Becoming Leader. State: Replica: 8de7d5a07fb447aaa68b7f3acce9665f, State: Running, Role: LEADER
I20260709 12:16:48.370311 26277 sys_catalog.cc:565] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:16:48.370373 26280 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de7d5a07fb447aaa68b7f3acce9665f" member_type: VOTER }
I20260709 12:16:48.375993 26282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8de7d5a07fb447aaa68b7f3acce9665f. Latest consensus state: current_term: 1 leader_uuid: "8de7d5a07fb447aaa68b7f3acce9665f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de7d5a07fb447aaa68b7f3acce9665f" member_type: VOTER } }
I20260709 12:16:48.376036 26281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8de7d5a07fb447aaa68b7f3acce9665f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8de7d5a07fb447aaa68b7f3acce9665f" member_type: VOTER } }
I20260709 12:16:48.376689 26282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:48.376904 26281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f [sys.catalog]: This master's current role is: LEADER
I20260709 12:16:48.380079 26288 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:16:48.388916 26288 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:16:48.390311 11318 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260709 12:16:48.400418 26288 catalog_manager.cc:1383] Generated new cluster ID: 18c856b6f7a143ef8dcbba486cf54038
I20260709 12:16:48.400887 26288 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:16:48.435366 26288 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:16:48.437055 26288 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:16:48.452945 26288 catalog_manager.cc:6070] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f: Generated new TSK 0
I20260709 12:16:48.453801 26288 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:16:48.522428 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:48.528754 26298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:48.529920 26299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:48.532047 26301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:48.532544 11318 server_base.cc:1061] running on GCE node
I20260709 12:16:48.533600 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:48.533802 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:48.534024 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599408534007 us; error 0 us; skew 500 ppm
I20260709 12:16:48.534762 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:48.537894 11318 webserver.cc:533] Webserver started at http://127.11.13.129:43379/ using document root <none> and password file <none>
I20260709 12:16:48.538551 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:48.538730 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:48.539093 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:48.540290 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598892056905-11318-0/minicluster-data/ts-0-root/instance:
uuid: "f1991907f700454badc2971ecf14911d"
format_stamp: "Formatted at 2026-07-09 12:16:48 on dist-test-slave-zqnl"
I20260709 12:16:48.545372 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.006s	sys 0.000s
I20260709 12:16:48.549182 26306 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:48.550697 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:16:48.551259 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598892056905-11318-0/minicluster-data/ts-0-root
uuid: "f1991907f700454badc2971ecf14911d"
format_stamp: "Formatted at 2026-07-09 12:16:48 on dist-test-slave-zqnl"
I20260709 12:16:48.551824 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598892056905-11318-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598892056905-11318-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598892056905-11318-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:48.569983 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:48.571311 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:48.573295 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:48.577477 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:48.577739 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:48.578032 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:48.578271 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.001s	sys 0.000s
I20260709 12:16:48.626631 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.129:37117
I20260709 12:16:48.626833 26368 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.129:37117 every 8 connection(s)
I20260709 12:16:48.632210 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:48.641121 26373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:48.641872 26374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:48.647327 11318 server_base.cc:1061] running on GCE node
W20260709 12:16:48.647303 26376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:48.648393 26369 heartbeater.cc:344] Connected to a master server at 127.11.13.190:45359
I20260709 12:16:48.648929 26369 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:48.649962 26369 heartbeater.cc:507] Master 127.11.13.190:45359 requested a full tablet report, sending...
I20260709 12:16:48.649986 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:16:48.650434 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:48.650662 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599408650644 us; error 0 us; skew 500 ppm
I20260709 12:16:48.651667 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:48.652896 26242 ts_manager.cc:194] Registered new tserver with Master: f1991907f700454badc2971ecf14911d (127.11.13.129:37117)
I20260709 12:16:48.655040 11318 webserver.cc:533] Webserver started at http://127.11.13.130:35357/ using document root <none> and password file <none>
I20260709 12:16:48.655754 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:48.655752 26242 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33694
I20260709 12:16:48.656582 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:48.657078 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:48.659461 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598892056905-11318-0/minicluster-data/ts-1-root/instance:
uuid: "cc1c2a51ec344ca28260030593d6563e"
format_stamp: "Formatted at 2026-07-09 12:16:48 on dist-test-slave-zqnl"
I20260709 12:16:48.664317 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.004s	user 0.005s	sys 0.000s
I20260709 12:16:48.668381 26381 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:48.669261 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:16:48.669710 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598892056905-11318-0/minicluster-data/ts-1-root
uuid: "cc1c2a51ec344ca28260030593d6563e"
format_stamp: "Formatted at 2026-07-09 12:16:48 on dist-test-slave-zqnl"
I20260709 12:16:48.669991 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598892056905-11318-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598892056905-11318-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598892056905-11318-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:48.695063 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:48.696290 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:48.697817 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:48.700527 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:48.700806 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:48.701089 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:48.701282 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:48.749001 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.130:38281
I20260709 12:16:48.749359 26443 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.130:38281 every 8 connection(s)
I20260709 12:16:48.754506 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:16:48.762656 26447 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:48.764600 26448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:16:48.766800 26450 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:16:48.767900 11318 server_base.cc:1061] running on GCE node
I20260709 12:16:48.769122 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260709 12:16:48.769286 26444 heartbeater.cc:344] Connected to a master server at 127.11.13.190:45359
W20260709 12:16:48.769474 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:16:48.769729 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599408769712 us; error 0 us; skew 500 ppm
I20260709 12:16:48.769815 26444 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:48.770825 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:16:48.771234 26444 heartbeater.cc:507] Master 127.11.13.190:45359 requested a full tablet report, sending...
I20260709 12:16:48.774652 26242 ts_manager.cc:194] Registered new tserver with Master: cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:48.774672 11318 webserver.cc:533] Webserver started at http://127.11.13.131:35693/ using document root <none> and password file <none>
I20260709 12:16:48.775743 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:16:48.776024 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:16:48.776329 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:16:48.776686 26242 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33710
I20260709 12:16:48.777635 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598892056905-11318-0/minicluster-data/ts-2-root/instance:
uuid: "a9c038faa7aa4de680db447a77ed8632"
format_stamp: "Formatted at 2026-07-09 12:16:48 on dist-test-slave-zqnl"
I20260709 12:16:48.783407 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.004s	sys 0.000s
I20260709 12:16:48.787264 26455 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:48.788237 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.004s	sys 0.000s
I20260709 12:16:48.788578 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598892056905-11318-0/minicluster-data/ts-2-root
uuid: "a9c038faa7aa4de680db447a77ed8632"
format_stamp: "Formatted at 2026-07-09 12:16:48 on dist-test-slave-zqnl"
I20260709 12:16:48.788867 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598892056905-11318-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598892056905-11318-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_0.1783598892056905-11318-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:16:48.809780 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:16:48.811051 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:16:48.812829 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:16:48.815412 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:16:48.815675 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:48.815951 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:16:48.816141 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:16:48.862866 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.131:38463
I20260709 12:16:48.863011 26517 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.131:38463 every 8 connection(s)
I20260709 12:16:48.879845 26518 heartbeater.cc:344] Connected to a master server at 127.11.13.190:45359
I20260709 12:16:48.880402 26518 heartbeater.cc:461] Registering TS with master...
I20260709 12:16:48.881276 26518 heartbeater.cc:507] Master 127.11.13.190:45359 requested a full tablet report, sending...
I20260709 12:16:48.883826 26242 ts_manager.cc:194] Registered new tserver with Master: a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:48.884542 11318 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.018018425s
I20260709 12:16:48.885733 26242 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33726
I20260709 12:16:48.916973 26242 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33738:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:16:48.920166 26242 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:16:48.997731 26483 tablet_service.cc:1511] Processing CreateTablet for tablet 15f1cf7b68d6463fbd60efc8c4576fc8 (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:16:48.999331 26483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15f1cf7b68d6463fbd60efc8c4576fc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.002395 26481 tablet_service.cc:1511] Processing CreateTablet for tablet 3f981b08993f4a4aa6778078403e1004 (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:16:49.003356 26482 tablet_service.cc:1511] Processing CreateTablet for tablet 0859d4d4116a463f8e3a00a7737862ea (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:16:49.004122 26481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f981b08993f4a4aa6778078403e1004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.004805 26482 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0859d4d4116a463f8e3a00a7737862ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.007275 26409 tablet_service.cc:1511] Processing CreateTablet for tablet 15f1cf7b68d6463fbd60efc8c4576fc8 (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:16:49.008793 26409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15f1cf7b68d6463fbd60efc8c4576fc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.012419 26480 tablet_service.cc:1511] Processing CreateTablet for tablet 0f82565aabb44f9985dfd97ae33d8f3f (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:16:49.012722 26406 tablet_service.cc:1511] Processing CreateTablet for tablet 0f82565aabb44f9985dfd97ae33d8f3f (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:16:49.014029 26480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f82565aabb44f9985dfd97ae33d8f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.015230 26407 tablet_service.cc:1511] Processing CreateTablet for tablet 3f981b08993f4a4aa6778078403e1004 (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:16:49.015980 26405 tablet_service.cc:1511] Processing CreateTablet for tablet 6956538aee40492683999ec2274993a4 (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:16:49.017688 26408 tablet_service.cc:1511] Processing CreateTablet for tablet 0859d4d4116a463f8e3a00a7737862ea (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:16:49.014029 26406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f82565aabb44f9985dfd97ae33d8f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.019775 26405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6956538aee40492683999ec2274993a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.022583 26407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f981b08993f4a4aa6778078403e1004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.022257 26479 tablet_service.cc:1511] Processing CreateTablet for tablet 6956538aee40492683999ec2274993a4 (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:16:49.024196 26479 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6956538aee40492683999ec2274993a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.024227 26408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0859d4d4116a463f8e3a00a7737862ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.027549 26404 tablet_service.cc:1511] Processing CreateTablet for tablet 3b5f70527bbc4070880acc6ef899ddb4 (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:16:49.028998 26404 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b5f70527bbc4070880acc6ef899ddb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.030236 26478 tablet_service.cc:1511] Processing CreateTablet for tablet 3b5f70527bbc4070880acc6ef899ddb4 (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:16:49.031615 26478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b5f70527bbc4070880acc6ef899ddb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.036875 26403 tablet_service.cc:1511] Processing CreateTablet for tablet 21d01b3e50754bb38a24d1ea383a61f9 (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:16:49.039310 26403 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21d01b3e50754bb38a24d1ea383a61f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.039459 26477 tablet_service.cc:1511] Processing CreateTablet for tablet 21d01b3e50754bb38a24d1ea383a61f9 (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:16:49.040927 26477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21d01b3e50754bb38a24d1ea383a61f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.042688 26402 tablet_service.cc:1511] Processing CreateTablet for tablet b3488dcfa84d4a699bcc7a4a3c827577 (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:16:49.044116 26402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3488dcfa84d4a699bcc7a4a3c827577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.045413 26476 tablet_service.cc:1511] Processing CreateTablet for tablet b3488dcfa84d4a699bcc7a4a3c827577 (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:16:49.046854 26476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3488dcfa84d4a699bcc7a4a3c827577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.060621 26334 tablet_service.cc:1511] Processing CreateTablet for tablet 15f1cf7b68d6463fbd60efc8c4576fc8 (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:16:49.062299 26334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 15f1cf7b68d6463fbd60efc8c4576fc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.072196 26333 tablet_service.cc:1511] Processing CreateTablet for tablet 0859d4d4116a463f8e3a00a7737862ea (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:16:49.073868 26333 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0859d4d4116a463f8e3a00a7737862ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.080129 26332 tablet_service.cc:1511] Processing CreateTablet for tablet 3f981b08993f4a4aa6778078403e1004 (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:16:49.082327 26332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3f981b08993f4a4aa6778078403e1004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.087226 26534 tablet_bootstrap.cc:492] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e: Bootstrap starting.
I20260709 12:16:49.088011 26331 tablet_service.cc:1511] Processing CreateTablet for tablet 0f82565aabb44f9985dfd97ae33d8f3f (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:16:49.090457 26330 tablet_service.cc:1511] Processing CreateTablet for tablet 6956538aee40492683999ec2274993a4 (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:16:49.092054 26331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0f82565aabb44f9985dfd97ae33d8f3f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.093071 26329 tablet_service.cc:1511] Processing CreateTablet for tablet 3b5f70527bbc4070880acc6ef899ddb4 (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:16:49.094617 26329 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 3b5f70527bbc4070880acc6ef899ddb4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.095335 26402 tablet_service.cc:1511] Processing CreateTablet for tablet 9648db75c5cb45a681230d22c473e29f (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:16:49.096333 26330 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 6956538aee40492683999ec2274993a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.096853 26402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9648db75c5cb45a681230d22c473e29f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.122812 26482 tablet_service.cc:1511] Processing CreateTablet for tablet 9648db75c5cb45a681230d22c473e29f (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:16:49.124361 26482 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9648db75c5cb45a681230d22c473e29f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.131361 26328 tablet_service.cc:1511] Processing CreateTablet for tablet 21d01b3e50754bb38a24d1ea383a61f9 (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:16:49.132902 26328 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 21d01b3e50754bb38a24d1ea383a61f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.139989 26327 tablet_service.cc:1511] Processing CreateTablet for tablet b3488dcfa84d4a699bcc7a4a3c827577 (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:16:49.142922 26327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b3488dcfa84d4a699bcc7a4a3c827577. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.148069 26326 tablet_service.cc:1511] Processing CreateTablet for tablet 9648db75c5cb45a681230d22c473e29f (DEFAULT_TABLE table=test-workload [id=d186e90961dd4ed68b14780760ca3a67]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:16:49.149672 26326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 9648db75c5cb45a681230d22c473e29f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:49.176571 26533 tablet_bootstrap.cc:492] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632: Bootstrap starting.
I20260709 12:16:49.181223 26535 tablet_bootstrap.cc:492] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d: Bootstrap starting.
I20260709 12:16:49.185930 26533 tablet_bootstrap.cc:654] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.185976 26534 tablet_bootstrap.cc:654] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.193579 26535 tablet_bootstrap.cc:654] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.204767 26533 tablet_bootstrap.cc:492] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632: No bootstrap required, opened a new log
I20260709 12:16:49.205687 26533 ts_tablet_manager.cc:1403] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.007s
I20260709 12:16:49.214478 26533 raft_consensus.cc:359] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } } peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } }
I20260709 12:16:49.215687 26533 raft_consensus.cc:385] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.216169 26533 raft_consensus.cc:740] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.224766 26533 consensus_queue.cc:260] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } } peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } }
I20260709 12:16:49.230047 26534 tablet_bootstrap.cc:492] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e: No bootstrap required, opened a new log
I20260709 12:16:49.230755 26534 ts_tablet_manager.cc:1403] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e: Time spent bootstrapping tablet: real 0.144s	user 0.025s	sys 0.007s
I20260709 12:16:49.230993 26518 heartbeater.cc:499] Master 127.11.13.190:45359 was elected leader, sending a full tablet report...
I20260709 12:16:49.231561 26533 ts_tablet_manager.cc:1434] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632: Time spent starting tablet: real 0.025s	user 0.008s	sys 0.003s
I20260709 12:16:49.231614 26535 tablet_bootstrap.cc:492] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d: No bootstrap required, opened a new log
I20260709 12:16:49.232393 26535 ts_tablet_manager.cc:1403] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d: Time spent bootstrapping tablet: real 0.052s	user 0.012s	sys 0.011s
I20260709 12:16:49.233011 26533 tablet_bootstrap.cc:492] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632: Bootstrap starting.
I20260709 12:16:49.234068 26534 raft_consensus.cc:359] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.234899 26534 raft_consensus.cc:385] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.235199 26534 raft_consensus.cc:740] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.235540 26535 raft_consensus.cc:359] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.236024 26534 consensus_queue.cc:260] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.236512 26535 raft_consensus.cc:385] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.236866 26535 raft_consensus.cc:740] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.237982 26535 consensus_queue.cc:260] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.239737 26533 tablet_bootstrap.cc:654] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.260846 26534 ts_tablet_manager.cc:1434] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e: Time spent starting tablet: real 0.030s	user 0.011s	sys 0.017s
I20260709 12:16:49.261781 26535 ts_tablet_manager.cc:1434] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d: Time spent starting tablet: real 0.029s	user 0.007s	sys 0.020s
I20260709 12:16:49.262077 26369 heartbeater.cc:499] Master 127.11.13.190:45359 was elected leader, sending a full tablet report...
I20260709 12:16:49.263562 26444 heartbeater.cc:499] Master 127.11.13.190:45359 was elected leader, sending a full tablet report...
I20260709 12:16:49.263736 26533 tablet_bootstrap.cc:492] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632: No bootstrap required, opened a new log
I20260709 12:16:49.264333 26534 tablet_bootstrap.cc:492] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e: Bootstrap starting.
I20260709 12:16:49.264528 26533 ts_tablet_manager.cc:1403] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632: Time spent bootstrapping tablet: real 0.032s	user 0.014s	sys 0.017s
I20260709 12:16:49.268050 26533 raft_consensus.cc:359] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.269016 26533 raft_consensus.cc:385] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.263542 26535 tablet_bootstrap.cc:492] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d: Bootstrap starting.
I20260709 12:16:49.269637 26533 raft_consensus.cc:740] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.270635 26533 consensus_queue.cc:260] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.271345 26534 tablet_bootstrap.cc:654] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.273257 26533 ts_tablet_manager.cc:1434] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:49.274607 26533 tablet_bootstrap.cc:492] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632: Bootstrap starting.
I20260709 12:16:49.277212 26535 tablet_bootstrap.cc:654] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.281375 26533 tablet_bootstrap.cc:654] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.287915 26534 tablet_bootstrap.cc:492] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e: No bootstrap required, opened a new log
I20260709 12:16:49.288672 26534 ts_tablet_manager.cc:1403] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e: Time spent bootstrapping tablet: real 0.025s	user 0.014s	sys 0.007s
I20260709 12:16:49.288951 26535 tablet_bootstrap.cc:492] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d: No bootstrap required, opened a new log
I20260709 12:16:49.290418 26535 ts_tablet_manager.cc:1403] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d: Time spent bootstrapping tablet: real 0.027s	user 0.014s	sys 0.006s
I20260709 12:16:49.292289 26534 raft_consensus.cc:359] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.293255 26534 raft_consensus.cc:385] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.293665 26534 raft_consensus.cc:740] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.293627 26535 raft_consensus.cc:359] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.289353 26533 tablet_bootstrap.cc:492] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632: No bootstrap required, opened a new log
I20260709 12:16:49.294775 26535 raft_consensus.cc:385] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.295492 26533 ts_tablet_manager.cc:1403] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632: Time spent bootstrapping tablet: real 0.021s	user 0.008s	sys 0.004s
I20260709 12:16:49.295305 26535 raft_consensus.cc:740] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.295080 26534 consensus_queue.cc:260] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.296691 26535 consensus_queue.cc:260] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.298601 26534 ts_tablet_manager.cc:1434] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 12:16:49.299089 26535 ts_tablet_manager.cc:1434] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.004s
I20260709 12:16:49.299249 26533 raft_consensus.cc:359] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.300031 26534 tablet_bootstrap.cc:492] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e: Bootstrap starting.
I20260709 12:16:49.300220 26535 tablet_bootstrap.cc:492] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d: Bootstrap starting.
I20260709 12:16:49.300033 26533 raft_consensus.cc:385] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.306371 26534 tablet_bootstrap.cc:654] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.306425 26533 raft_consensus.cc:740] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.307816 26533 consensus_queue.cc:260] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.308513 26535 tablet_bootstrap.cc:654] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.318168 26533 ts_tablet_manager.cc:1434] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 12:16:49.318614 26534 tablet_bootstrap.cc:492] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e: No bootstrap required, opened a new log
I20260709 12:16:49.319267 26534 ts_tablet_manager.cc:1403] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e: Time spent bootstrapping tablet: real 0.020s	user 0.015s	sys 0.003s
I20260709 12:16:49.319342 26533 tablet_bootstrap.cc:492] T 3b5f70527bbc4070880acc6ef899ddb4 P a9c038faa7aa4de680db447a77ed8632: Bootstrap starting.
I20260709 12:16:49.324872 26535 tablet_bootstrap.cc:492] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d: No bootstrap required, opened a new log
I20260709 12:16:49.325613 26535 ts_tablet_manager.cc:1403] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d: Time spent bootstrapping tablet: real 0.026s	user 0.016s	sys 0.009s
I20260709 12:16:49.331100 26533 tablet_bootstrap.cc:654] T 3b5f70527bbc4070880acc6ef899ddb4 P a9c038faa7aa4de680db447a77ed8632: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.330742 26535 raft_consensus.cc:359] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } } peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } }
I20260709 12:16:49.331701 26535 raft_consensus.cc:385] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.332052 26535 raft_consensus.cc:740] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.330574 26534 raft_consensus.cc:359] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.332953 26534 raft_consensus.cc:385] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.333251 26534 raft_consensus.cc:740] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.333966 26534 consensus_queue.cc:260] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.334472 26535 consensus_queue.cc:260] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } } peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } }
I20260709 12:16:49.336585 26534 ts_tablet_manager.cc:1434] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e: Time spent starting tablet: real 0.017s	user 0.003s	sys 0.003s
I20260709 12:16:49.337704 26534 tablet_bootstrap.cc:492] T 9648db75c5cb45a681230d22c473e29f P cc1c2a51ec344ca28260030593d6563e: Bootstrap starting.
I20260709 12:16:49.342638 26539 raft_consensus.cc:493] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.342923 26535 ts_tablet_manager.cc:1434] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 12:16:49.343349 26539 raft_consensus.cc:515] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } } peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } }
I20260709 12:16:49.344233 26535 tablet_bootstrap.cc:492] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d: Bootstrap starting.
I20260709 12:16:49.344744 26534 tablet_bootstrap.cc:654] T 9648db75c5cb45a681230d22c473e29f P cc1c2a51ec344ca28260030593d6563e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.350493 26539 leader_election.cc:290] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281), f1991907f700454badc2971ecf14911d (127.11.13.129:37117)
I20260709 12:16:49.352805 26535 tablet_bootstrap.cc:654] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.362856 26535 tablet_bootstrap.cc:492] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d: No bootstrap required, opened a new log
I20260709 12:16:49.363543 26535 ts_tablet_manager.cc:1403] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d: Time spent bootstrapping tablet: real 0.020s	user 0.012s	sys 0.005s
I20260709 12:16:49.365381 26534 tablet_bootstrap.cc:492] T 9648db75c5cb45a681230d22c473e29f P cc1c2a51ec344ca28260030593d6563e: No bootstrap required, opened a new log
I20260709 12:16:49.366209 26534 ts_tablet_manager.cc:1403] T 9648db75c5cb45a681230d22c473e29f P cc1c2a51ec344ca28260030593d6563e: Time spent bootstrapping tablet: real 0.029s	user 0.012s	sys 0.003s
I20260709 12:16:49.367472 26535 raft_consensus.cc:359] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.368319 26535 raft_consensus.cc:385] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.368881 26535 raft_consensus.cc:740] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.370203 26535 consensus_queue.cc:260] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.371312 26419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15f1cf7b68d6463fbd60efc8c4576fc8" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e" is_pre_election: true
I20260709 12:16:49.372017 26534 raft_consensus.cc:359] T 9648db75c5cb45a681230d22c473e29f P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.372797 26534 raft_consensus.cc:385] T 9648db75c5cb45a681230d22c473e29f P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.373095 26534 raft_consensus.cc:740] T 9648db75c5cb45a681230d22c473e29f P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.373858 26534 consensus_queue.cc:260] T 9648db75c5cb45a681230d22c473e29f P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.374979 26535 ts_tablet_manager.cc:1434] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 12:16:49.376184 26535 tablet_bootstrap.cc:492] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d: Bootstrap starting.
I20260709 12:16:49.379469 26533 tablet_bootstrap.cc:492] T 3b5f70527bbc4070880acc6ef899ddb4 P a9c038faa7aa4de680db447a77ed8632: No bootstrap required, opened a new log
I20260709 12:16:49.380069 26533 ts_tablet_manager.cc:1403] T 3b5f70527bbc4070880acc6ef899ddb4 P a9c038faa7aa4de680db447a77ed8632: Time spent bootstrapping tablet: real 0.061s	user 0.014s	sys 0.004s
W20260709 12:16:49.383487 26290 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
W20260709 12:16:49.384311 26459 leader_election.cc:343] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:49.383543 26533 raft_consensus.cc:359] T 3b5f70527bbc4070880acc6ef899ddb4 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.384999 26533 raft_consensus.cc:385] T 3b5f70527bbc4070880acc6ef899ddb4 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.385404 26533 raft_consensus.cc:740] T 3b5f70527bbc4070880acc6ef899ddb4 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.385993 26534 ts_tablet_manager.cc:1434] T 9648db75c5cb45a681230d22c473e29f P cc1c2a51ec344ca28260030593d6563e: Time spent starting tablet: real 0.019s	user 0.008s	sys 0.000s
I20260709 12:16:49.386211 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15f1cf7b68d6463fbd60efc8c4576fc8" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:49.387024 26344 raft_consensus.cc:2468] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 0.
I20260709 12:16:49.386525 26533 consensus_queue.cc:260] T 3b5f70527bbc4070880acc6ef899ddb4 P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.388242 26535 tablet_bootstrap.cc:654] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.387720 26534 tablet_bootstrap.cc:492] T 15f1cf7b68d6463fbd60efc8c4576fc8 P cc1c2a51ec344ca28260030593d6563e: Bootstrap starting.
I20260709 12:16:49.396589 26534 tablet_bootstrap.cc:654] T 15f1cf7b68d6463fbd60efc8c4576fc8 P cc1c2a51ec344ca28260030593d6563e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.397168 26459 leader_election.cc:304] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, f1991907f700454badc2971ecf14911d; no voters: cc1c2a51ec344ca28260030593d6563e
I20260709 12:16:49.398100 26533 ts_tablet_manager.cc:1434] T 3b5f70527bbc4070880acc6ef899ddb4 P a9c038faa7aa4de680db447a77ed8632: Time spent starting tablet: real 0.018s	user 0.011s	sys 0.000s
I20260709 12:16:49.398806 26539 raft_consensus.cc:2804] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:49.399304 26539 raft_consensus.cc:493] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:49.399688 26539 raft_consensus.cc:3060] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.400162 26533 tablet_bootstrap.cc:492] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632: Bootstrap starting.
I20260709 12:16:49.400800 26535 tablet_bootstrap.cc:492] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d: No bootstrap required, opened a new log
I20260709 12:16:49.401436 26535 ts_tablet_manager.cc:1403] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d: Time spent bootstrapping tablet: real 0.025s	user 0.006s	sys 0.011s
I20260709 12:16:49.404929 26535 raft_consensus.cc:359] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.406708 26535 raft_consensus.cc:385] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.407045 26535 raft_consensus.cc:740] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.407948 26535 consensus_queue.cc:260] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.409185 26539 raft_consensus.cc:515] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } } peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } }
I20260709 12:16:49.411789 26533 tablet_bootstrap.cc:654] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.415261 26535 ts_tablet_manager.cc:1434] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d: Time spent starting tablet: real 0.013s	user 0.009s	sys 0.000s
I20260709 12:16:49.416421 26535 tablet_bootstrap.cc:492] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d: Bootstrap starting.
I20260709 12:16:49.422224 26419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15f1cf7b68d6463fbd60efc8c4576fc8" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:49.423523 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15f1cf7b68d6463fbd60efc8c4576fc8" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:49.424240 26344 raft_consensus.cc:3060] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:16:49.425697 26459 leader_election.cc:343] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:49.430275 26539 leader_election.cc:290] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 election: Requested vote from peers cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281), f1991907f700454badc2971ecf14911d (127.11.13.129:37117)
I20260709 12:16:49.430961 26535 tablet_bootstrap.cc:654] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.433331 26344 raft_consensus.cc:2468] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 1.
I20260709 12:16:49.435154 26459 leader_election.cc:304] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, f1991907f700454badc2971ecf14911d; no voters: cc1c2a51ec344ca28260030593d6563e
I20260709 12:16:49.436941 26539 raft_consensus.cc:2804] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:49.438467 26534 tablet_bootstrap.cc:492] T 15f1cf7b68d6463fbd60efc8c4576fc8 P cc1c2a51ec344ca28260030593d6563e: No bootstrap required, opened a new log
I20260709 12:16:49.439318 26534 ts_tablet_manager.cc:1403] T 15f1cf7b68d6463fbd60efc8c4576fc8 P cc1c2a51ec344ca28260030593d6563e: Time spent bootstrapping tablet: real 0.052s	user 0.025s	sys 0.022s
I20260709 12:16:49.444000 26534 raft_consensus.cc:359] T 15f1cf7b68d6463fbd60efc8c4576fc8 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } } peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } }
I20260709 12:16:49.445143 26534 raft_consensus.cc:385] T 15f1cf7b68d6463fbd60efc8c4576fc8 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.445564 26534 raft_consensus.cc:740] T 15f1cf7b68d6463fbd60efc8c4576fc8 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.446614 26534 consensus_queue.cc:260] T 15f1cf7b68d6463fbd60efc8c4576fc8 P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } } peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } }
I20260709 12:16:49.450157 26539 raft_consensus.cc:697] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Becoming Leader. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Running, Role: LEADER
I20260709 12:16:49.450947 26535 tablet_bootstrap.cc:492] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d: No bootstrap required, opened a new log
I20260709 12:16:49.451509 26535 ts_tablet_manager.cc:1403] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d: Time spent bootstrapping tablet: real 0.035s	user 0.014s	sys 0.012s
I20260709 12:16:49.451778 26533 tablet_bootstrap.cc:492] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632: No bootstrap required, opened a new log
I20260709 12:16:49.451367 26539 consensus_queue.cc:237] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } } peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } }
I20260709 12:16:49.453179 26541 raft_consensus.cc:493] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.453858 26533 ts_tablet_manager.cc:1403] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632: Time spent bootstrapping tablet: real 0.054s	user 0.031s	sys 0.015s
I20260709 12:16:49.453784 26541 raft_consensus.cc:515] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.455756 26534 ts_tablet_manager.cc:1434] T 15f1cf7b68d6463fbd60efc8c4576fc8 P cc1c2a51ec344ca28260030593d6563e: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.004s
I20260709 12:16:49.456735 26541 leader_election.cc:290] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:49.457015 26534 tablet_bootstrap.cc:492] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e: Bootstrap starting.
I20260709 12:16:49.457330 26533 raft_consensus.cc:359] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.458238 26533 raft_consensus.cc:385] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.459380 26533 raft_consensus.cc:740] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.462193 26533 consensus_queue.cc:260] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.470207 26533 ts_tablet_manager.cc:1434] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.000s
I20260709 12:16:49.475751 26533 tablet_bootstrap.cc:492] T 3f981b08993f4a4aa6778078403e1004 P a9c038faa7aa4de680db447a77ed8632: Bootstrap starting.
I20260709 12:16:49.483222 26535 raft_consensus.cc:359] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.484754 26535 raft_consensus.cc:385] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.485152 26535 raft_consensus.cc:740] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.487042 26541 raft_consensus.cc:493] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.487675 26541 raft_consensus.cc:515] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.486343 26535 consensus_queue.cc:260] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.493052 26240 catalog_manager.cc:5697] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9c038faa7aa4de680db447a77ed8632 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "a9c038faa7aa4de680db447a77ed8632" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:49.494830 26541 leader_election.cc:290] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:49.498639 26533 tablet_bootstrap.cc:654] T 3f981b08993f4a4aa6778078403e1004 P a9c038faa7aa4de680db447a77ed8632: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.495615 26534 tablet_bootstrap.cc:654] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.505918 26539 raft_consensus.cc:493] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.506778 26539 raft_consensus.cc:515] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.510672 26419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0859d4d4116a463f8e3a00a7737862ea" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e" is_pre_election: true
I20260709 12:16:49.514650 26418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b5f70527bbc4070880acc6ef899ddb4" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e" is_pre_election: true
I20260709 12:16:49.514843 26539 leader_election.cc:290] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:49.515425 26418 raft_consensus.cc:2468] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 0.
I20260709 12:16:49.515684 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0859d4d4116a463f8e3a00a7737862ea" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:49.516050 26419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0859d4d4116a463f8e3a00a7737862ea" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e" is_pre_election: true
I20260709 12:16:49.516419 26344 raft_consensus.cc:2468] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 0.
W20260709 12:16:49.517352 26459 leader_election.cc:343] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:49.517361 26310 leader_election.cc:343] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:49.518280 26459 leader_election.cc:304] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, f1991907f700454badc2971ecf14911d; no voters: cc1c2a51ec344ca28260030593d6563e
I20260709 12:16:49.519207 26310 leader_election.cc:304] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:49.519572 26539 raft_consensus.cc:2804] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:49.520038 26539 raft_consensus.cc:493] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:49.520151 26560 raft_consensus.cc:2804] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:49.520908 26539 raft_consensus.cc:3060] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.522029 26560 raft_consensus.cc:493] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:49.528848 26533 tablet_bootstrap.cc:492] T 3f981b08993f4a4aa6778078403e1004 P a9c038faa7aa4de680db447a77ed8632: No bootstrap required, opened a new log
I20260709 12:16:49.530778 26533 ts_tablet_manager.cc:1403] T 3f981b08993f4a4aa6778078403e1004 P a9c038faa7aa4de680db447a77ed8632: Time spent bootstrapping tablet: real 0.055s	user 0.014s	sys 0.009s
I20260709 12:16:49.532505 26535 ts_tablet_manager.cc:1434] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d: Time spent starting tablet: real 0.081s	user 0.015s	sys 0.019s
I20260709 12:16:49.534725 26533 raft_consensus.cc:359] T 3f981b08993f4a4aa6778078403e1004 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.535601 26535 tablet_bootstrap.cc:492] T 9648db75c5cb45a681230d22c473e29f P f1991907f700454badc2971ecf14911d: Bootstrap starting.
I20260709 12:16:49.535778 26533 raft_consensus.cc:385] T 3f981b08993f4a4aa6778078403e1004 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.536170 26533 raft_consensus.cc:740] T 3f981b08993f4a4aa6778078403e1004 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.537093 26534 tablet_bootstrap.cc:492] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e: No bootstrap required, opened a new log
I20260709 12:16:49.537806 26534 ts_tablet_manager.cc:1403] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e: Time spent bootstrapping tablet: real 0.081s	user 0.014s	sys 0.007s
I20260709 12:16:49.537165 26533 consensus_queue.cc:260] T 3f981b08993f4a4aa6778078403e1004 P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.533252 26539 raft_consensus.cc:515] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.541085 26534 raft_consensus.cc:359] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.541889 26534 raft_consensus.cc:385] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.530076 26560 raft_consensus.cc:3060] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.542228 26534 raft_consensus.cc:740] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.543507 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0859d4d4116a463f8e3a00a7737862ea" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:49.543205 26534 consensus_queue.cc:260] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.544191 26344 raft_consensus.cc:3060] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.545975 26533 ts_tablet_manager.cc:1434] T 3f981b08993f4a4aa6778078403e1004 P a9c038faa7aa4de680db447a77ed8632: Time spent starting tablet: real 0.015s	user 0.005s	sys 0.003s
I20260709 12:16:49.547657 26533 tablet_bootstrap.cc:492] T b3488dcfa84d4a699bcc7a4a3c827577 P a9c038faa7aa4de680db447a77ed8632: Bootstrap starting.
I20260709 12:16:49.548060 26493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0859d4d4116a463f8e3a00a7737862ea" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632" is_pre_election: true
I20260709 12:16:49.549118 26493 raft_consensus.cc:2393] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f1991907f700454badc2971ecf14911d in current term 1: Already voted for candidate a9c038faa7aa4de680db447a77ed8632 in this term.
I20260709 12:16:49.550536 26308 leader_election.cc:304] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1991907f700454badc2971ecf14911d; no voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e
I20260709 12:16:49.550800 26539 leader_election.cc:290] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:49.553138 26419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0859d4d4116a463f8e3a00a7737862ea" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:49.554250 26492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b5f70527bbc4070880acc6ef899ddb4" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632" is_pre_election: true
W20260709 12:16:49.554991 26459 leader_election.cc:343] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:49.555111 26492 raft_consensus.cc:2468] T 3b5f70527bbc4070880acc6ef899ddb4 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 0.
I20260709 12:16:49.557363 26560 raft_consensus.cc:515] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.559825 26560 leader_election.cc:290] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 election: Requested vote from peers cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:49.559919 26534 ts_tablet_manager.cc:1434] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e: Time spent starting tablet: real 0.022s	user 0.006s	sys 0.001s
I20260709 12:16:49.560385 26535 tablet_bootstrap.cc:654] T 9648db75c5cb45a681230d22c473e29f P f1991907f700454badc2971ecf14911d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.561041 26492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b5f70527bbc4070880acc6ef899ddb4" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:49.561693 26492 raft_consensus.cc:3060] T 3b5f70527bbc4070880acc6ef899ddb4 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.561573 26419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b5f70527bbc4070880acc6ef899ddb4" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:49.562954 26419 raft_consensus.cc:3060] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.565857 26534 tablet_bootstrap.cc:492] T 0859d4d4116a463f8e3a00a7737862ea P cc1c2a51ec344ca28260030593d6563e: Bootstrap starting.
I20260709 12:16:49.570621 26533 tablet_bootstrap.cc:654] T b3488dcfa84d4a699bcc7a4a3c827577 P a9c038faa7aa4de680db447a77ed8632: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.576794 26534 tablet_bootstrap.cc:654] T 0859d4d4116a463f8e3a00a7737862ea P cc1c2a51ec344ca28260030593d6563e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.580375 26492 raft_consensus.cc:2468] T 3b5f70527bbc4070880acc6ef899ddb4 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 1.
I20260709 12:16:49.582255 26308 leader_election.cc:304] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:49.584067 26560 raft_consensus.cc:2804] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:49.586014 26539 raft_consensus.cc:493] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.587342 26539 raft_consensus.cc:515] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.582366 26344 raft_consensus.cc:2468] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 1.
I20260709 12:16:49.591823 26541 raft_consensus.cc:2749] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:49.593116 26539 leader_election.cc:290] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:49.594548 26418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9648db75c5cb45a681230d22c473e29f" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e" is_pre_election: true
I20260709 12:16:49.595734 26418 raft_consensus.cc:2468] T 9648db75c5cb45a681230d22c473e29f P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 0.
I20260709 12:16:49.598994 26419 raft_consensus.cc:2468] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 1.
I20260709 12:16:49.599519 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9648db75c5cb45a681230d22c473e29f" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:49.600749 26459 leader_election.cc:304] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, f1991907f700454badc2971ecf14911d; no voters: cc1c2a51ec344ca28260030593d6563e
I20260709 12:16:49.601593 26560 raft_consensus.cc:697] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [term 1 LEADER]: Becoming Leader. State: Replica: f1991907f700454badc2971ecf14911d, State: Running, Role: LEADER
I20260709 12:16:49.606396 26539 raft_consensus.cc:2804] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:49.608820 26539 raft_consensus.cc:697] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Becoming Leader. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Running, Role: LEADER
I20260709 12:16:49.606551 26459 leader_election.cc:304] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e; no voters: 
I20260709 12:16:49.610534 26534 tablet_bootstrap.cc:492] T 0859d4d4116a463f8e3a00a7737862ea P cc1c2a51ec344ca28260030593d6563e: No bootstrap required, opened a new log
I20260709 12:16:49.611438 26534 ts_tablet_manager.cc:1403] T 0859d4d4116a463f8e3a00a7737862ea P cc1c2a51ec344ca28260030593d6563e: Time spent bootstrapping tablet: real 0.046s	user 0.031s	sys 0.009s
I20260709 12:16:49.611501 26556 raft_consensus.cc:2804] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:49.612262 26556 raft_consensus.cc:493] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:49.612699 26556 raft_consensus.cc:3060] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.612407 26560 consensus_queue.cc:237] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.610059 26539 consensus_queue.cc:237] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.616488 26534 raft_consensus.cc:359] T 0859d4d4116a463f8e3a00a7737862ea P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
W20260709 12:16:49.611622 26459 leader_election.cc:343] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f1991907f700454badc2971ecf14911d (127.11.13.129:37117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:49.624152 26556 raft_consensus.cc:515] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.617525 26534 raft_consensus.cc:385] T 0859d4d4116a463f8e3a00a7737862ea P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.625887 26535 tablet_bootstrap.cc:492] T 9648db75c5cb45a681230d22c473e29f P f1991907f700454badc2971ecf14911d: No bootstrap required, opened a new log
I20260709 12:16:49.626279 26534 raft_consensus.cc:740] T 0859d4d4116a463f8e3a00a7737862ea P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.626849 26535 ts_tablet_manager.cc:1403] T 9648db75c5cb45a681230d22c473e29f P f1991907f700454badc2971ecf14911d: Time spent bootstrapping tablet: real 0.092s	user 0.023s	sys 0.008s
I20260709 12:16:49.627442 26534 consensus_queue.cc:260] T 0859d4d4116a463f8e3a00a7737862ea P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.635918 26535 raft_consensus.cc:359] T 9648db75c5cb45a681230d22c473e29f P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.636948 26535 raft_consensus.cc:385] T 9648db75c5cb45a681230d22c473e29f P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.637349 26535 raft_consensus.cc:740] T 9648db75c5cb45a681230d22c473e29f P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.638320 26535 consensus_queue.cc:260] T 9648db75c5cb45a681230d22c473e29f P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.640419 26534 ts_tablet_manager.cc:1434] T 0859d4d4116a463f8e3a00a7737862ea P cc1c2a51ec344ca28260030593d6563e: Time spent starting tablet: real 0.028s	user 0.010s	sys 0.000s
I20260709 12:16:49.641089 26535 ts_tablet_manager.cc:1434] T 9648db75c5cb45a681230d22c473e29f P f1991907f700454badc2971ecf14911d: Time spent starting tablet: real 0.014s	user 0.010s	sys 0.000s
I20260709 12:16:49.642400 26535 tablet_bootstrap.cc:492] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d: Bootstrap starting.
I20260709 12:16:49.642870 26534 tablet_bootstrap.cc:492] T 0f82565aabb44f9985dfd97ae33d8f3f P cc1c2a51ec344ca28260030593d6563e: Bootstrap starting.
I20260709 12:16:49.643369 26533 tablet_bootstrap.cc:492] T b3488dcfa84d4a699bcc7a4a3c827577 P a9c038faa7aa4de680db447a77ed8632: No bootstrap required, opened a new log
I20260709 12:16:49.643980 26533 ts_tablet_manager.cc:1403] T b3488dcfa84d4a699bcc7a4a3c827577 P a9c038faa7aa4de680db447a77ed8632: Time spent bootstrapping tablet: real 0.097s	user 0.016s	sys 0.030s
I20260709 12:16:49.646422 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9648db75c5cb45a681230d22c473e29f" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:49.647153 26343 raft_consensus.cc:3060] T 9648db75c5cb45a681230d22c473e29f P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.647361 26533 raft_consensus.cc:359] T b3488dcfa84d4a699bcc7a4a3c827577 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.648236 26533 raft_consensus.cc:385] T b3488dcfa84d4a699bcc7a4a3c827577 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.648551 26533 raft_consensus.cc:740] T b3488dcfa84d4a699bcc7a4a3c827577 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.650326 26534 tablet_bootstrap.cc:654] T 0f82565aabb44f9985dfd97ae33d8f3f P cc1c2a51ec344ca28260030593d6563e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.652024 26535 tablet_bootstrap.cc:654] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.653131 26533 consensus_queue.cc:260] T b3488dcfa84d4a699bcc7a4a3c827577 P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.655989 26533 ts_tablet_manager.cc:1434] T b3488dcfa84d4a699bcc7a4a3c827577 P a9c038faa7aa4de680db447a77ed8632: Time spent starting tablet: real 0.012s	user 0.007s	sys 0.000s
I20260709 12:16:49.657051 26533 tablet_bootstrap.cc:492] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632: Bootstrap starting.
I20260709 12:16:49.658816 26241 catalog_manager.cc:5697] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d reported cstate change: term changed from 0 to 1, leader changed from <none> to f1991907f700454badc2971ecf14911d (127.11.13.129). New cstate: current_term: 1 leader_uuid: "f1991907f700454badc2971ecf14911d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:49.663579 26556 leader_election.cc:290] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:49.665376 26419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "9648db75c5cb45a681230d22c473e29f" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:49.666092 26419 raft_consensus.cc:3060] T 9648db75c5cb45a681230d22c473e29f P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.666581 26533 tablet_bootstrap.cc:654] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.665895 26343 raft_consensus.cc:2468] T 9648db75c5cb45a681230d22c473e29f P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 1.
I20260709 12:16:49.669255 26459 leader_election.cc:304] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:49.670079 26539 raft_consensus.cc:493] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.670575 26539 raft_consensus.cc:515] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.671694 26556 raft_consensus.cc:2804] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:49.673866 26560 raft_consensus.cc:493] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.674912 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6956538aee40492683999ec2274993a4" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:49.674654 26560 raft_consensus.cc:515] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.676296 26418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6956538aee40492683999ec2274993a4" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e" is_pre_election: true
I20260709 12:16:49.677013 26418 raft_consensus.cc:2468] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 0.
I20260709 12:16:49.677964 26560 leader_election.cc:290] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:49.679275 26418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3488dcfa84d4a699bcc7a4a3c827577" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e" is_pre_election: true
I20260709 12:16:49.680197 26418 raft_consensus.cc:2468] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 0.
I20260709 12:16:49.682471 26540 raft_consensus.cc:493] T 0859d4d4116a463f8e3a00a7737862ea P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.683068 26540 raft_consensus.cc:515] T 0859d4d4116a463f8e3a00a7737862ea P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.683682 26419 raft_consensus.cc:2468] T 9648db75c5cb45a681230d22c473e29f P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 1.
I20260709 12:16:49.681733 26310 leader_election.cc:304] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:49.690608 26560 raft_consensus.cc:2804] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:49.691252 26560 raft_consensus.cc:493] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:49.691776 26560 raft_consensus.cc:3060] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.692983 26540 leader_election.cc:290] T 0859d4d4116a463f8e3a00a7737862ea P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:49.679428 26459 leader_election.cc:304] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e; no voters: 
I20260709 12:16:49.692878 26568 raft_consensus.cc:493] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.694862 26568 raft_consensus.cc:515] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.698663 26568 leader_election.cc:290] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:49.700199 26540 raft_consensus.cc:493] T 15f1cf7b68d6463fbd60efc8c4576fc8 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.701026 26540 raft_consensus.cc:515] T 15f1cf7b68d6463fbd60efc8c4576fc8 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } } peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } }
I20260709 12:16:49.703895 26540 leader_election.cc:290] T 15f1cf7b68d6463fbd60efc8c4576fc8 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463), f1991907f700454badc2971ecf14911d (127.11.13.129:37117)
I20260709 12:16:49.705818 26560 raft_consensus.cc:515] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.715734 26560 leader_election.cc:290] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 election: Requested vote from peers cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:49.715255 26539 leader_election.cc:290] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:49.717154 26539 raft_consensus.cc:2804] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:49.717643 26539 raft_consensus.cc:493] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:49.718102 26539 raft_consensus.cc:3060] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.759545 26560 raft_consensus.cc:493] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.760473 26560 raft_consensus.cc:515] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.764144 26419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d01b3e50754bb38a24d1ea383a61f9" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e" is_pre_election: true
I20260709 12:16:49.764410 26418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3488dcfa84d4a699bcc7a4a3c827577" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:49.764961 26418 raft_consensus.cc:3060] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.766366 26560 leader_election.cc:290] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
W20260709 12:16:49.774961 26310 leader_election.cc:343] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:49.777858 26419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f82565aabb44f9985dfd97ae33d8f3f" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e" is_pre_election: true
W20260709 12:16:49.778874 26459 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:38281, user_credentials={real_user=slave}} blocked reactor thread for 99662.9us
I20260709 12:16:49.780745 26492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3488dcfa84d4a699bcc7a4a3c827577" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632" is_pre_election: true
I20260709 12:16:49.781121 26493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d01b3e50754bb38a24d1ea383a61f9" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632" is_pre_election: true
I20260709 12:16:49.781924 26492 raft_consensus.cc:2468] T b3488dcfa84d4a699bcc7a4a3c827577 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 0.
W20260709 12:16:49.781929 26310 leader_election.cc:343] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:49.783746 26493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3488dcfa84d4a699bcc7a4a3c827577" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:49.784672 26493 raft_consensus.cc:3060] T b3488dcfa84d4a699bcc7a4a3c827577 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.802307 26556 raft_consensus.cc:697] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Becoming Leader. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Running, Role: LEADER
I20260709 12:16:49.803903 26556 consensus_queue.cc:237] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.806733 26492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f82565aabb44f9985dfd97ae33d8f3f" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632" is_pre_election: true
W20260709 12:16:49.808519 26459 leader_election.cc:343] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f1991907f700454badc2971ecf14911d (127.11.13.129:37117): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:16:49.810348 26308 leader_election.cc:343] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:49.810981 26308 leader_election.cc:304] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1991907f700454badc2971ecf14911d; no voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e
I20260709 12:16:49.812204 26560 raft_consensus.cc:2749] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260709 12:16:49.814644 26308 leader_election.cc:343] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:49.815340 26308 leader_election.cc:304] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1991907f700454badc2971ecf14911d; no voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e
I20260709 12:16:49.816440 26560 raft_consensus.cc:2749] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:49.829989 26540 raft_consensus.cc:493] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:49.830739 26540 raft_consensus.cc:515] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.832691 26534 tablet_bootstrap.cc:492] T 0f82565aabb44f9985dfd97ae33d8f3f P cc1c2a51ec344ca28260030593d6563e: No bootstrap required, opened a new log
I20260709 12:16:49.833498 26534 ts_tablet_manager.cc:1403] T 0f82565aabb44f9985dfd97ae33d8f3f P cc1c2a51ec344ca28260030593d6563e: Time spent bootstrapping tablet: real 0.192s	user 0.070s	sys 0.073s
I20260709 12:16:49.833530 26540 leader_election.cc:290] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:49.835829 26535 tablet_bootstrap.cc:492] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d: No bootstrap required, opened a new log
I20260709 12:16:49.836427 26535 ts_tablet_manager.cc:1403] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d: Time spent bootstrapping tablet: real 0.194s	user 0.061s	sys 0.062s
I20260709 12:16:49.838742 26534 raft_consensus.cc:359] T 0f82565aabb44f9985dfd97ae33d8f3f P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.839483 26535 raft_consensus.cc:359] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.841538 26534 raft_consensus.cc:385] T 0f82565aabb44f9985dfd97ae33d8f3f P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.845551 26534 raft_consensus.cc:740] T 0f82565aabb44f9985dfd97ae33d8f3f P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.822512 26493 raft_consensus.cc:2468] T b3488dcfa84d4a699bcc7a4a3c827577 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 1.
I20260709 12:16:49.841581 26535 raft_consensus.cc:385] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.848063 26535 raft_consensus.cc:740] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.848834 26308 leader_election.cc:304] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:49.848963 26535 consensus_queue.cc:260] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.843140 26539 raft_consensus.cc:515] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.849763 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0859d4d4116a463f8e3a00a7737862ea" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:49.850941 26343 raft_consensus.cc:2393] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cc1c2a51ec344ca28260030593d6563e in current term 1: Already voted for candidate a9c038faa7aa4de680db447a77ed8632 in this term.
I20260709 12:16:49.844357 26418 raft_consensus.cc:2468] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 1.
I20260709 12:16:49.847604 26534 consensus_queue.cc:260] T 0f82565aabb44f9985dfd97ae33d8f3f P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.851908 26539 leader_election.cc:290] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:49.852200 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15f1cf7b68d6463fbd60efc8c4576fc8" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:49.856695 26344 raft_consensus.cc:2393] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cc1c2a51ec344ca28260030593d6563e in current term 1: Already voted for candidate a9c038faa7aa4de680db447a77ed8632 in this term.
I20260709 12:16:49.855294 26533 tablet_bootstrap.cc:492] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632: No bootstrap required, opened a new log
I20260709 12:16:49.853120 26492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15f1cf7b68d6463fbd60efc8c4576fc8" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632" is_pre_election: true
I20260709 12:16:49.857636 26533 ts_tablet_manager.cc:1403] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632: Time spent bootstrapping tablet: real 0.201s	user 0.084s	sys 0.098s
I20260709 12:16:49.852798 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f981b08993f4a4aa6778078403e1004" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:49.858656 26342 raft_consensus.cc:2468] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 0.
I20260709 12:16:49.853672 26491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f981b08993f4a4aa6778078403e1004" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632" is_pre_election: true
I20260709 12:16:49.860916 26491 raft_consensus.cc:2468] T 3f981b08993f4a4aa6778078403e1004 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 0.
I20260709 12:16:49.861234 26533 raft_consensus.cc:359] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.856710 26534 ts_tablet_manager.cc:1434] T 0f82565aabb44f9985dfd97ae33d8f3f P cc1c2a51ec344ca28260030593d6563e: Time spent starting tablet: real 0.022s	user 0.005s	sys 0.004s
I20260709 12:16:49.862155 26533 raft_consensus.cc:385] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.862721 26533 raft_consensus.cc:740] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.850253 26560 raft_consensus.cc:2804] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:49.864375 26534 tablet_bootstrap.cc:492] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e: Bootstrap starting.
I20260709 12:16:49.864339 26560 raft_consensus.cc:697] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [term 1 LEADER]: Becoming Leader. State: Replica: f1991907f700454badc2971ecf14911d, State: Running, Role: LEADER
I20260709 12:16:49.862900 26383 leader_election.cc:304] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e; no voters: 
I20260709 12:16:49.864379 26533 consensus_queue.cc:260] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.855337 26535 ts_tablet_manager.cc:1434] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d: Time spent starting tablet: real 0.019s	user 0.009s	sys 0.000s
I20260709 12:16:49.866155 26540 raft_consensus.cc:2804] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:49.866776 26540 raft_consensus.cc:493] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:49.867157 26540 raft_consensus.cc:3060] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.867367 26535 tablet_bootstrap.cc:492] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d: Bootstrap starting.
I20260709 12:16:49.867444 26533 ts_tablet_manager.cc:1434] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:16:49.868912 26533 tablet_bootstrap.cc:492] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632: Bootstrap starting.
I20260709 12:16:49.865545 26560 consensus_queue.cc:237] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.875906 26535 tablet_bootstrap.cc:654] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.876854 26534 tablet_bootstrap.cc:654] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.877795 26533 tablet_bootstrap.cc:654] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:49.862646 26385 leader_election.cc:304] T 15f1cf7b68d6463fbd60efc8c4576fc8 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e; no voters: a9c038faa7aa4de680db447a77ed8632, f1991907f700454badc2971ecf14911d
I20260709 12:16:49.895087 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6956538aee40492683999ec2274993a4" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d"
W20260709 12:16:49.897305 26459 leader_election.cc:343] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer f1991907f700454badc2971ecf14911d (127.11.13.129:37117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:49.898914 26418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6956538aee40492683999ec2274993a4" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:49.899964 26418 raft_consensus.cc:3060] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.901768 26493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0859d4d4116a463f8e3a00a7737862ea" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632" is_pre_election: true
I20260709 12:16:49.908315 26540 raft_consensus.cc:515] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.912377 26540 leader_election.cc:290] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:49.912281 26491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f981b08993f4a4aa6778078403e1004" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:49.913070 26540 raft_consensus.cc:3060] T 15f1cf7b68d6463fbd60efc8c4576fc8 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.913451 26491 raft_consensus.cc:3060] T 3f981b08993f4a4aa6778078403e1004 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.906028 26240 catalog_manager.cc:5697] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d reported cstate change: term changed from 0 to 1, leader changed from <none> to f1991907f700454badc2971ecf14911d (127.11.13.129). New cstate: current_term: 1 leader_uuid: "f1991907f700454badc2971ecf14911d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:49.912271 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f981b08993f4a4aa6778078403e1004" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:49.918565 26342 raft_consensus.cc:3060] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.919864 26383 leader_election.cc:304] T 0859d4d4116a463f8e3a00a7737862ea P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e; no voters: a9c038faa7aa4de680db447a77ed8632, f1991907f700454badc2971ecf14911d
I20260709 12:16:49.922942 26418 raft_consensus.cc:2468] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 1.
I20260709 12:16:49.924225 26540 raft_consensus.cc:2749] T 15f1cf7b68d6463fbd60efc8c4576fc8 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:49.925020 26459 leader_election.cc:304] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e; no voters: f1991907f700454badc2971ecf14911d
I20260709 12:16:49.926385 26556 raft_consensus.cc:2804] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:49.927076 26556 raft_consensus.cc:697] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Becoming Leader. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Running, Role: LEADER
I20260709 12:16:49.928323 26556 consensus_queue.cc:237] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.932621 26582 raft_consensus.cc:3060] T 0859d4d4116a463f8e3a00a7737862ea P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:49.934163 26242 catalog_manager.cc:5697] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9c038faa7aa4de680db447a77ed8632 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "a9c038faa7aa4de680db447a77ed8632" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:49.936213 26491 raft_consensus.cc:2468] T 3f981b08993f4a4aa6778078403e1004 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 1.
I20260709 12:16:49.937851 26383 leader_election.cc:304] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e; no voters: 
I20260709 12:16:49.940172 26534 tablet_bootstrap.cc:492] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e: No bootstrap required, opened a new log
I20260709 12:16:49.940716 26534 ts_tablet_manager.cc:1403] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e: Time spent bootstrapping tablet: real 0.077s	user 0.013s	sys 0.003s
I20260709 12:16:49.942337 26540 raft_consensus.cc:2804] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:49.944778 26534 raft_consensus.cc:359] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.945730 26534 raft_consensus.cc:385] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.945684 26342 raft_consensus.cc:2468] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 1.
I20260709 12:16:49.944370 26242 catalog_manager.cc:5697] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9c038faa7aa4de680db447a77ed8632 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "a9c038faa7aa4de680db447a77ed8632" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:49.946117 26534 raft_consensus.cc:740] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.950636 26535 tablet_bootstrap.cc:492] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d: No bootstrap required, opened a new log
I20260709 12:16:49.951022 26534 consensus_queue.cc:260] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.951594 26535 ts_tablet_manager.cc:1403] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d: Time spent bootstrapping tablet: real 0.084s	user 0.028s	sys 0.008s
I20260709 12:16:49.952755 26582 raft_consensus.cc:2749] T 0859d4d4116a463f8e3a00a7737862ea P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:49.955904 26535 raft_consensus.cc:359] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.957046 26535 raft_consensus.cc:385] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.957535 26535 raft_consensus.cc:740] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.958765 26535 consensus_queue.cc:260] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.960764 26535 ts_tablet_manager.cc:1434] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.000s
I20260709 12:16:49.962443 26533 tablet_bootstrap.cc:492] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632: No bootstrap required, opened a new log
I20260709 12:16:49.963038 26533 ts_tablet_manager.cc:1403] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632: Time spent bootstrapping tablet: real 0.094s	user 0.018s	sys 0.041s
I20260709 12:16:49.966715 26533 raft_consensus.cc:359] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.967656 26533 raft_consensus.cc:385] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:49.967989 26533 raft_consensus.cc:740] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Initialized, Role: FOLLOWER
I20260709 12:16:49.969009 26533 consensus_queue.cc:260] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.970665 26534 ts_tablet_manager.cc:1434] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e: Time spent starting tablet: real 0.030s	user 0.009s	sys 0.004s
I20260709 12:16:49.971133 26533 ts_tablet_manager.cc:1434] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:16:49.972194 26540 raft_consensus.cc:697] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Becoming Leader. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Running, Role: LEADER
I20260709 12:16:49.973178 26540 consensus_queue.cc:237] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:49.975796 26556 consensus_queue.cc:1048] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:49.987061 26239 catalog_manager.cc:5697] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e reported cstate change: term changed from 0 to 1, leader changed from <none> to cc1c2a51ec344ca28260030593d6563e (127.11.13.130). New cstate: current_term: 1 leader_uuid: "cc1c2a51ec344ca28260030593d6563e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:49.990185 26241 catalog_manager.cc:5697] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9c038faa7aa4de680db447a77ed8632 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "a9c038faa7aa4de680db447a77ed8632" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:49.995579 26539 consensus_queue.cc:1048] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:50.012513 26560 consensus_queue.cc:1048] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:50.028927 26541 consensus_queue.cc:1048] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:50.079299 26568 raft_consensus.cc:493] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:50.079798 26568 raft_consensus.cc:515] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.081632 26568 leader_election.cc:290] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:50.082739 26418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6956538aee40492683999ec2274993a4" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e" is_pre_election: true
I20260709 12:16:50.082818 26491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6956538aee40492683999ec2274993a4" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632" is_pre_election: true
I20260709 12:16:50.083856 26418 raft_consensus.cc:2393] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f1991907f700454badc2971ecf14911d in current term 1: Already voted for candidate a9c038faa7aa4de680db447a77ed8632 in this term.
I20260709 12:16:50.084913 26310 leader_election.cc:304] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1991907f700454badc2971ecf14911d; no voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e
I20260709 12:16:50.085704 26568 raft_consensus.cc:3060] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:50.092281 26568 raft_consensus.cc:2749] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:50.105343 26539 raft_consensus.cc:493] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:50.106225 26539 raft_consensus.cc:515] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.108448 26539 leader_election.cc:290] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:50.109642 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d01b3e50754bb38a24d1ea383a61f9" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:50.109884 26418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d01b3e50754bb38a24d1ea383a61f9" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e" is_pre_election: true
I20260709 12:16:50.110494 26342 raft_consensus.cc:2468] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 0.
I20260709 12:16:50.110627 26418 raft_consensus.cc:2468] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 0.
I20260709 12:16:50.112067 26459 leader_election.cc:304] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:50.112928 26539 raft_consensus.cc:2804] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:50.113337 26539 raft_consensus.cc:493] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:50.113675 26539 raft_consensus.cc:3060] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:50.119632 26539 raft_consensus.cc:515] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.121546 26539 leader_election.cc:290] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:50.122581 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d01b3e50754bb38a24d1ea383a61f9" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:50.122864 26418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d01b3e50754bb38a24d1ea383a61f9" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:50.123175 26342 raft_consensus.cc:3060] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:50.123528 26418 raft_consensus.cc:3060] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:50.124316 26539 raft_consensus.cc:493] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:50.124855 26539 raft_consensus.cc:515] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.127092 26539 leader_election.cc:290] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:50.129052 26419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f82565aabb44f9985dfd97ae33d8f3f" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e" is_pre_election: true
I20260709 12:16:50.129805 26419 raft_consensus.cc:2468] T 0f82565aabb44f9985dfd97ae33d8f3f P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 0.
I20260709 12:16:50.131323 26459 leader_election.cc:304] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e; no voters: 
I20260709 12:16:50.131541 26342 raft_consensus.cc:2468] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 1.
I20260709 12:16:50.131963 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f82565aabb44f9985dfd97ae33d8f3f" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:50.132543 26418 raft_consensus.cc:2468] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 1.
I20260709 12:16:50.132756 26344 raft_consensus.cc:2468] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 0.
I20260709 12:16:50.132370 26539 raft_consensus.cc:2804] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:50.134207 26459 leader_election.cc:304] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e; no voters: 
I20260709 12:16:50.134218 26539 raft_consensus.cc:493] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:50.135049 26539 raft_consensus.cc:3060] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:50.135249 26556 raft_consensus.cc:2804] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:50.135829 26556 raft_consensus.cc:697] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Becoming Leader. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Running, Role: LEADER
I20260709 12:16:50.136781 26556 consensus_queue.cc:237] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.142944 26539 raft_consensus.cc:515] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.145884 26539 leader_election.cc:290] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:50.147583 26418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f82565aabb44f9985dfd97ae33d8f3f" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:50.148623 26418 raft_consensus.cc:3060] T 0f82565aabb44f9985dfd97ae33d8f3f P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:50.148761 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f82565aabb44f9985dfd97ae33d8f3f" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:50.149544 26342 raft_consensus.cc:3060] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:50.150017 26239 catalog_manager.cc:5697] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9c038faa7aa4de680db447a77ed8632 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "a9c038faa7aa4de680db447a77ed8632" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:50.152834 26539 consensus_queue.cc:1048] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:50.157618 26342 raft_consensus.cc:2468] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 1.
I20260709 12:16:50.159098 26459 leader_election.cc:304] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:50.159078 26418 raft_consensus.cc:2468] T 0f82565aabb44f9985dfd97ae33d8f3f P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 1.
I20260709 12:16:50.161154 26539 raft_consensus.cc:2804] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:50.161913 26539 raft_consensus.cc:697] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Becoming Leader. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Running, Role: LEADER
I20260709 12:16:50.163189 26539 consensus_queue.cc:237] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.174635 26239 catalog_manager.cc:5697] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9c038faa7aa4de680db447a77ed8632 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "a9c038faa7aa4de680db447a77ed8632" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:50.175215 26575 consensus_queue.cc:1048] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:50.355746 26575 consensus_queue.cc:1048] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:50.358222 26568 consensus_queue.cc:1048] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:50.370973 26556 consensus_queue.cc:1048] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:50.377805 26541 consensus_queue.cc:1048] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:50.406576 26539 consensus_queue.cc:1048] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:50.433409 26578 consensus_queue.cc:1048] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:50.457875 26491 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "15f1cf7b68d6463fbd60efc8c4576fc8"
dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
mode: GRACEFUL
new_leader_uuid: "f1991907f700454badc2971ecf14911d"
 from {username='slave'} at 127.0.0.1:48082
I20260709 12:16:50.458896 26491 raft_consensus.cc:606] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Received request to transfer leadership to TS f1991907f700454badc2971ecf14911d
I20260709 12:16:50.467105 26491 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0859d4d4116a463f8e3a00a7737862ea"
dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
mode: GRACEFUL
new_leader_uuid: "f1991907f700454badc2971ecf14911d"
 from {username='slave'} at 127.0.0.1:48082
I20260709 12:16:50.467659 26491 raft_consensus.cc:606] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Received request to transfer leadership to TS f1991907f700454badc2971ecf14911d
I20260709 12:16:50.483412 26419 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3f981b08993f4a4aa6778078403e1004"
dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
mode: GRACEFUL
new_leader_uuid: "f1991907f700454badc2971ecf14911d"
 from {username='slave'} at 127.0.0.1:55504
I20260709 12:16:50.483946 26419 raft_consensus.cc:606] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Received request to transfer leadership to TS f1991907f700454badc2971ecf14911d
I20260709 12:16:50.490283 26491 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f82565aabb44f9985dfd97ae33d8f3f"
dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
mode: GRACEFUL
new_leader_uuid: "f1991907f700454badc2971ecf14911d"
 from {username='slave'} at 127.0.0.1:48082
I20260709 12:16:50.490830 26491 raft_consensus.cc:606] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Received request to transfer leadership to TS f1991907f700454badc2971ecf14911d
I20260709 12:16:50.497251 26491 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6956538aee40492683999ec2274993a4"
dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
mode: GRACEFUL
new_leader_uuid: "cc1c2a51ec344ca28260030593d6563e"
 from {username='slave'} at 127.0.0.1:48082
I20260709 12:16:50.497761 26491 raft_consensus.cc:606] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Received request to transfer leadership to TS cc1c2a51ec344ca28260030593d6563e
I20260709 12:16:50.513260 26342 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "3b5f70527bbc4070880acc6ef899ddb4"
dest_uuid: "f1991907f700454badc2971ecf14911d"
mode: GRACEFUL
new_leader_uuid: "cc1c2a51ec344ca28260030593d6563e"
 from {username='slave'} at 127.0.0.1:60322
I20260709 12:16:50.514210 26342 raft_consensus.cc:606] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [term 1 LEADER]: Received request to transfer leadership to TS cc1c2a51ec344ca28260030593d6563e
I20260709 12:16:50.518368 26540 consensus_queue.cc:1048] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:50.522609 26493 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "21d01b3e50754bb38a24d1ea383a61f9"
dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
mode: GRACEFUL
new_leader_uuid: "cc1c2a51ec344ca28260030593d6563e"
 from {username='slave'} at 127.0.0.1:48082
I20260709 12:16:50.523295 26493 raft_consensus.cc:606] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Received request to transfer leadership to TS cc1c2a51ec344ca28260030593d6563e
I20260709 12:16:50.523367 26541 raft_consensus.cc:993] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d: : Instructing follower cc1c2a51ec344ca28260030593d6563e to start an election
I20260709 12:16:50.523998 26541 raft_consensus.cc:1081] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [term 1 LEADER]: Signalling peer cc1c2a51ec344ca28260030593d6563e to start an election
I20260709 12:16:50.526305 26418 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3b5f70527bbc4070880acc6ef899ddb4"
dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
 from {username='slave'} at 127.0.0.1:55498
I20260709 12:16:50.527011 26418 raft_consensus.cc:493] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:50.527431 26418 raft_consensus.cc:3060] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.534423 26342 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "b3488dcfa84d4a699bcc7a4a3c827577"
dest_uuid: "f1991907f700454badc2971ecf14911d"
mode: GRACEFUL
new_leader_uuid: "cc1c2a51ec344ca28260030593d6563e"
 from {username='slave'} at 127.0.0.1:60322
I20260709 12:16:50.535199 26342 raft_consensus.cc:606] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [term 1 LEADER]: Received request to transfer leadership to TS cc1c2a51ec344ca28260030593d6563e
I20260709 12:16:50.540203 26418 raft_consensus.cc:515] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.542752 26418 leader_election.cc:290] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 2 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:50.543682 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b5f70527bbc4070880acc6ef899ddb4" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:50.544437 26344 raft_consensus.cc:3055] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:50.544286 26611 consensus_queue.cc:1048] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:50.544792 26344 raft_consensus.cc:740] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Running, Role: LEADER
I20260709 12:16:50.546646 26344 consensus_queue.cc:260] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.548382 26344 raft_consensus.cc:3060] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.554420 26612 raft_consensus.cc:993] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e: : Instructing follower f1991907f700454badc2971ecf14911d to start an election
I20260709 12:16:50.554615 26491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3b5f70527bbc4070880acc6ef899ddb4" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:50.555027 26611 raft_consensus.cc:1081] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Signalling peer f1991907f700454badc2971ecf14911d to start an election
I20260709 12:16:50.555375 26491 raft_consensus.cc:3060] T 3b5f70527bbc4070880acc6ef899ddb4 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.557536 26342 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "3f981b08993f4a4aa6778078403e1004"
dest_uuid: "f1991907f700454badc2971ecf14911d"
 from {username='slave'} at 127.0.0.1:60306
I20260709 12:16:50.558413 26342 raft_consensus.cc:493] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:50.558846 26342 raft_consensus.cc:3060] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.564220 26491 raft_consensus.cc:2468] T 3b5f70527bbc4070880acc6ef899ddb4 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 2.
I20260709 12:16:50.564741 26344 raft_consensus.cc:2468] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 2.
I20260709 12:16:50.566164 26383 leader_election.cc:304] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e; no voters: 
I20260709 12:16:50.567543 26613 raft_consensus.cc:2804] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:50.567585 26342 raft_consensus.cc:515] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.568259 26613 raft_consensus.cc:697] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e [term 2 LEADER]: Becoming Leader. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Running, Role: LEADER
I20260709 12:16:50.569249 26613 consensus_queue.cc:237] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.570715 26342 leader_election.cc:290] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 2 election: Requested vote from peers cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:50.572611 26418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f981b08993f4a4aa6778078403e1004" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:50.573271 26418 raft_consensus.cc:3055] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:50.573560 26418 raft_consensus.cc:740] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Running, Role: LEADER
I20260709 12:16:50.574501 26418 consensus_queue.cc:260] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.575580 26418 raft_consensus.cc:3060] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.572561 26491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "3f981b08993f4a4aa6778078403e1004" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:50.578895 26491 raft_consensus.cc:3060] T 3f981b08993f4a4aa6778078403e1004 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.579267 26242 catalog_manager.cc:5697] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e reported cstate change: term changed from 1 to 2, leader changed from f1991907f700454badc2971ecf14911d (127.11.13.129) to cc1c2a51ec344ca28260030593d6563e (127.11.13.130). New cstate: current_term: 2 leader_uuid: "cc1c2a51ec344ca28260030593d6563e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:50.583778 26418 raft_consensus.cc:2468] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 2.
I20260709 12:16:50.585210 26310 leader_election.cc:304] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:50.586382 26560 raft_consensus.cc:2804] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:50.586899 26560 raft_consensus.cc:697] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Becoming Leader. State: Replica: f1991907f700454badc2971ecf14911d, State: Running, Role: LEADER
I20260709 12:16:50.587910 26491 raft_consensus.cc:2468] T 3f981b08993f4a4aa6778078403e1004 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 2.
I20260709 12:16:50.587836 26560 consensus_queue.cc:237] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.595501 26242 catalog_manager.cc:5697] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d reported cstate change: term changed from 1 to 2, leader changed from cc1c2a51ec344ca28260030593d6563e (127.11.13.130) to f1991907f700454badc2971ecf14911d (127.11.13.129). New cstate: current_term: 2 leader_uuid: "f1991907f700454badc2971ecf14911d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:50.674557 26539 raft_consensus.cc:993] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632: : Instructing follower f1991907f700454badc2971ecf14911d to start an election
I20260709 12:16:50.675221 26578 raft_consensus.cc:1081] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Signalling peer f1991907f700454badc2971ecf14911d to start an election
I20260709 12:16:50.677742 26344 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0859d4d4116a463f8e3a00a7737862ea"
dest_uuid: "f1991907f700454badc2971ecf14911d"
 from {username='slave'} at 127.0.0.1:60294
I20260709 12:16:50.678448 26344 raft_consensus.cc:493] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:50.678894 26344 raft_consensus.cc:3060] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.680579 26578 consensus_queue.cc:1048] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:50.682370 26578 consensus_queue.cc:1048] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:50.691465 26344 raft_consensus.cc:515] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.692301 26578 raft_consensus.cc:993] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632: : Instructing follower cc1c2a51ec344ca28260030593d6563e to start an election
I20260709 12:16:50.692785 26578 raft_consensus.cc:1081] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Signalling peer cc1c2a51ec344ca28260030593d6563e to start an election
I20260709 12:16:50.693953 26344 leader_election.cc:290] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 2 election: Requested vote from peers cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:50.694891 26418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0859d4d4116a463f8e3a00a7737862ea" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:50.695569 26418 raft_consensus.cc:3060] T 0859d4d4116a463f8e3a00a7737862ea P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.697335 26417 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "21d01b3e50754bb38a24d1ea383a61f9"
dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
 from {username='slave'} at 127.0.0.1:55488
I20260709 12:16:50.697856 26491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0859d4d4116a463f8e3a00a7737862ea" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:50.698063 26417 raft_consensus.cc:493] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:50.698558 26491 raft_consensus.cc:3055] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:50.699009 26491 raft_consensus.cc:740] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Running, Role: LEADER
I20260709 12:16:50.699959 26491 consensus_queue.cc:260] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.698558 26417 raft_consensus.cc:3060] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.701444 26491 raft_consensus.cc:3060] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.705058 26418 raft_consensus.cc:2468] T 0859d4d4116a463f8e3a00a7737862ea P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 2.
I20260709 12:16:50.707012 26310 leader_election.cc:304] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:50.707352 26578 consensus_queue.cc:1048] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:50.708124 26560 raft_consensus.cc:2804] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:50.708684 26560 raft_consensus.cc:697] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Becoming Leader. State: Replica: f1991907f700454badc2971ecf14911d, State: Running, Role: LEADER
I20260709 12:16:50.709777 26491 raft_consensus.cc:2468] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 2.
I20260709 12:16:50.709596 26560 consensus_queue.cc:237] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.713768 26417 raft_consensus.cc:515] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.718070 26419 raft_consensus.cc:1240] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Rejecting Update request from peer a9c038faa7aa4de680db447a77ed8632 for earlier term 1. Current term is 2. Ops: []
I20260709 12:16:50.723657 26417 leader_election.cc:290] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 2 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:50.723742 26539 consensus_queue.cc:1059] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:50.725076 26578 raft_consensus.cc:3055] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:50.725123 26491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d01b3e50754bb38a24d1ea383a61f9" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:50.725560 26578 raft_consensus.cc:740] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Running, Role: LEADER
I20260709 12:16:50.725978 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "21d01b3e50754bb38a24d1ea383a61f9" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:50.726708 26342 raft_consensus.cc:3060] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.726626 26578 consensus_queue.cc:260] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.726900 26242 catalog_manager.cc:5697] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d reported cstate change: term changed from 1 to 2, leader changed from a9c038faa7aa4de680db447a77ed8632 (127.11.13.131) to f1991907f700454badc2971ecf14911d (127.11.13.129). New cstate: current_term: 2 leader_uuid: "f1991907f700454badc2971ecf14911d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:50.728236 26578 raft_consensus.cc:3060] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.729579 26575 consensus_queue.cc:1048] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:50.736982 26342 raft_consensus.cc:2468] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 2.
I20260709 12:16:50.738529 26385 leader_election.cc:304] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:50.739593 26613 raft_consensus.cc:2804] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:50.740196 26613 raft_consensus.cc:697] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e [term 2 LEADER]: Becoming Leader. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Running, Role: LEADER
I20260709 12:16:50.741163 26613 consensus_queue.cc:237] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.745776 26578 raft_consensus.cc:993] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632: : Instructing follower f1991907f700454badc2971ecf14911d to start an election
I20260709 12:16:50.746426 26578 raft_consensus.cc:1081] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Signalling peer f1991907f700454badc2971ecf14911d to start an election
I20260709 12:16:50.748974 26491 raft_consensus.cc:2468] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 2.
I20260709 12:16:50.749569 26342 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0f82565aabb44f9985dfd97ae33d8f3f"
dest_uuid: "f1991907f700454badc2971ecf14911d"
 from {username='slave'} at 127.0.0.1:60294
I20260709 12:16:50.750543 26342 raft_consensus.cc:493] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:50.751168 26342 raft_consensus.cc:3060] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.759069 26239 catalog_manager.cc:5697] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e reported cstate change: term changed from 1 to 2, leader changed from a9c038faa7aa4de680db447a77ed8632 (127.11.13.131) to cc1c2a51ec344ca28260030593d6563e (127.11.13.130). New cstate: current_term: 2 leader_uuid: "cc1c2a51ec344ca28260030593d6563e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:50.761409 26342 raft_consensus.cc:515] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.764658 26342 leader_election.cc:290] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 2 election: Requested vote from peers cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:50.765486 26419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f82565aabb44f9985dfd97ae33d8f3f" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:50.766299 26419 raft_consensus.cc:3060] T 0f82565aabb44f9985dfd97ae33d8f3f P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.766923 26491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f82565aabb44f9985dfd97ae33d8f3f" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:50.767642 26491 raft_consensus.cc:3055] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:50.767999 26491 raft_consensus.cc:740] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Running, Role: LEADER
I20260709 12:16:50.768915 26491 consensus_queue.cc:260] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.770793 26491 raft_consensus.cc:3060] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.775377 26419 raft_consensus.cc:2468] T 0f82565aabb44f9985dfd97ae33d8f3f P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 2.
I20260709 12:16:50.776875 26310 leader_election.cc:304] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:50.777781 26560 raft_consensus.cc:2804] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:50.778333 26560 raft_consensus.cc:697] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Becoming Leader. State: Replica: f1991907f700454badc2971ecf14911d, State: Running, Role: LEADER
I20260709 12:16:50.779309 26560 consensus_queue.cc:237] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.780336 26491 raft_consensus.cc:2468] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 2.
I20260709 12:16:50.787504 26242 catalog_manager.cc:5697] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d reported cstate change: term changed from 1 to 2, leader changed from a9c038faa7aa4de680db447a77ed8632 (127.11.13.131) to f1991907f700454badc2971ecf14911d (127.11.13.129). New cstate: current_term: 2 leader_uuid: "f1991907f700454badc2971ecf14911d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:50.792824 26578 raft_consensus.cc:993] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632: : Instructing follower cc1c2a51ec344ca28260030593d6563e to start an election
I20260709 12:16:50.793581 26575 raft_consensus.cc:1081] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Signalling peer cc1c2a51ec344ca28260030593d6563e to start an election
I20260709 12:16:50.795702 26419 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6956538aee40492683999ec2274993a4"
dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
 from {username='slave'} at 127.0.0.1:55488
I20260709 12:16:50.796288 26419 raft_consensus.cc:493] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:50.796700 26419 raft_consensus.cc:3060] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.802871 26419 raft_consensus.cc:515] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.804759 26419 leader_election.cc:290] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 2 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:50.805884 26491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6956538aee40492683999ec2274993a4" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:50.805884 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6956538aee40492683999ec2274993a4" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:50.806943 26491 raft_consensus.cc:3055] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:50.806969 26342 raft_consensus.cc:3060] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.807544 26491 raft_consensus.cc:740] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Running, Role: LEADER
I20260709 12:16:50.808607 26491 consensus_queue.cc:260] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.810752 26491 raft_consensus.cc:3060] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.813592 26541 raft_consensus.cc:993] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d: : Instructing follower cc1c2a51ec344ca28260030593d6563e to start an election
I20260709 12:16:50.814522 26541 raft_consensus.cc:1081] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [term 1 LEADER]: Signalling peer cc1c2a51ec344ca28260030593d6563e to start an election
I20260709 12:16:50.815779 26342 raft_consensus.cc:2468] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 2.
I20260709 12:16:50.817206 26419 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "b3488dcfa84d4a699bcc7a4a3c827577"
dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
 from {username='slave'} at 127.0.0.1:55498
I20260709 12:16:50.817265 26385 leader_election.cc:304] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:50.818571 26613 raft_consensus.cc:2804] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:50.818675 26419 raft_consensus.cc:493] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:50.819267 26419 raft_consensus.cc:3060] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.819296 26613 raft_consensus.cc:697] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [term 2 LEADER]: Becoming Leader. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Running, Role: LEADER
I20260709 12:16:50.820505 26613 consensus_queue.cc:237] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.821512 26491 raft_consensus.cc:2468] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 2.
I20260709 12:16:50.826318 26419 raft_consensus.cc:515] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.828872 26419 leader_election.cc:290] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 2 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:50.829124 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3488dcfa84d4a699bcc7a4a3c827577" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:50.828086 26242 catalog_manager.cc:5697] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e reported cstate change: term changed from 1 to 2, leader changed from a9c038faa7aa4de680db447a77ed8632 (127.11.13.131) to cc1c2a51ec344ca28260030593d6563e (127.11.13.130). New cstate: current_term: 2 leader_uuid: "cc1c2a51ec344ca28260030593d6563e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:50.829825 26342 raft_consensus.cc:3055] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:50.830217 26342 raft_consensus.cc:740] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Running, Role: LEADER
I20260709 12:16:50.831071 26342 consensus_queue.cc:260] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.832242 26342 raft_consensus.cc:3060] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.832222 26491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b3488dcfa84d4a699bcc7a4a3c827577" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:50.833057 26491 raft_consensus.cc:3060] T b3488dcfa84d4a699bcc7a4a3c827577 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.835824 26575 raft_consensus.cc:993] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632: : Instructing follower f1991907f700454badc2971ecf14911d to start an election
I20260709 12:16:50.836516 26575 raft_consensus.cc:1081] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Signalling peer f1991907f700454badc2971ecf14911d to start an election
I20260709 12:16:50.838783 26344 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "15f1cf7b68d6463fbd60efc8c4576fc8"
dest_uuid: "f1991907f700454badc2971ecf14911d"
 from {username='slave'} at 127.0.0.1:60294
I20260709 12:16:50.839423 26344 raft_consensus.cc:493] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:50.839888 26344 raft_consensus.cc:3060] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.840755 26491 raft_consensus.cc:2468] T b3488dcfa84d4a699bcc7a4a3c827577 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 2.
I20260709 12:16:50.841604 26493 raft_consensus.cc:1240] T b3488dcfa84d4a699bcc7a4a3c827577 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Rejecting Update request from peer f1991907f700454badc2971ecf14911d for earlier term 1. Current term is 2. Ops: []
I20260709 12:16:50.842216 26383 leader_election.cc:304] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e; no voters: 
I20260709 12:16:50.843503 26613 raft_consensus.cc:2804] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:50.843801 26342 raft_consensus.cc:2468] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 2.
I20260709 12:16:50.844058 26613 raft_consensus.cc:697] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e [term 2 LEADER]: Becoming Leader. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Running, Role: LEADER
I20260709 12:16:50.845106 26613 consensus_queue.cc:237] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:50.848045 26344 raft_consensus.cc:515] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } } peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } }
I20260709 12:16:50.850927 26344 leader_election.cc:290] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 2 election: Requested vote from peers cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:50.851190 26419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15f1cf7b68d6463fbd60efc8c4576fc8" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:50.852317 26419 raft_consensus.cc:3060] T 15f1cf7b68d6463fbd60efc8c4576fc8 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.853739 26242 catalog_manager.cc:5697] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e reported cstate change: term changed from 1 to 2, leader changed from f1991907f700454badc2971ecf14911d (127.11.13.129) to cc1c2a51ec344ca28260030593d6563e (127.11.13.130). New cstate: current_term: 2 leader_uuid: "cc1c2a51ec344ca28260030593d6563e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:50.851562 26493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "15f1cf7b68d6463fbd60efc8c4576fc8" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:50.859150 26493 raft_consensus.cc:3055] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:50.859572 26493 raft_consensus.cc:740] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Running, Role: LEADER
I20260709 12:16:50.860468 26493 consensus_queue.cc:260] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } } peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } }
I20260709 12:16:50.861303 26419 raft_consensus.cc:2468] T 15f1cf7b68d6463fbd60efc8c4576fc8 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 2.
I20260709 12:16:50.862097 26493 raft_consensus.cc:3060] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:50.864171 26310 leader_election.cc:304] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:50.865087 26560 raft_consensus.cc:2804] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:50.865610 26560 raft_consensus.cc:697] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Becoming Leader. State: Replica: f1991907f700454badc2971ecf14911d, State: Running, Role: LEADER
I20260709 12:16:50.866513 26560 consensus_queue.cc:237] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } } peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } }
I20260709 12:16:50.872694 26493 raft_consensus.cc:2468] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 2.
I20260709 12:16:50.874564 26242 catalog_manager.cc:5697] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d reported cstate change: term changed from 1 to 2, leader changed from a9c038faa7aa4de680db447a77ed8632 (127.11.13.131) to f1991907f700454badc2971ecf14911d (127.11.13.129). New cstate: current_term: 2 leader_uuid: "f1991907f700454badc2971ecf14911d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:50.987479 26419 raft_consensus.cc:1275] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Refusing update from remote peer f1991907f700454badc2971ecf14911d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:50.989107 26560 consensus_queue.cc:1048] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:50.998024 26493 raft_consensus.cc:1275] T 3f981b08993f4a4aa6778078403e1004 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Refusing update from remote peer f1991907f700454badc2971ecf14911d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:51.001353 26568 consensus_queue.cc:1048] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:51.034724 26344 raft_consensus.cc:1275] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Refusing update from remote peer cc1c2a51ec344ca28260030593d6563e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:51.036442 26613 consensus_queue.cc:1048] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.048027 26493 raft_consensus.cc:1275] T 3b5f70527bbc4070880acc6ef899ddb4 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Refusing update from remote peer cc1c2a51ec344ca28260030593d6563e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:51.050153 26612 consensus_queue.cc:1048] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.170725 26493 raft_consensus.cc:1275] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Refusing update from remote peer cc1c2a51ec344ca28260030593d6563e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:51.172288 26612 consensus_queue.cc:1048] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.181497 26344 raft_consensus.cc:1275] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Refusing update from remote peer cc1c2a51ec344ca28260030593d6563e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:51.183531 26611 consensus_queue.cc:1048] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.188410 26493 raft_consensus.cc:1275] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Refusing update from remote peer f1991907f700454badc2971ecf14911d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:51.190450 26568 consensus_queue.cc:1048] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.202749 26419 raft_consensus.cc:1275] T 0f82565aabb44f9985dfd97ae33d8f3f P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Refusing update from remote peer f1991907f700454badc2971ecf14911d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:51.204317 26568 consensus_queue.cc:1048] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.273897 26419 raft_consensus.cc:1275] T 0859d4d4116a463f8e3a00a7737862ea P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Refusing update from remote peer f1991907f700454badc2971ecf14911d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:51.275611 26493 raft_consensus.cc:1275] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Refusing update from remote peer f1991907f700454badc2971ecf14911d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:51.275557 26568 consensus_queue.cc:1048] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.277612 26568 consensus_queue.cc:1048] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.348284 26493 raft_consensus.cc:1275] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Refusing update from remote peer cc1c2a51ec344ca28260030593d6563e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:51.350108 26611 consensus_queue.cc:1048] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.352458 26419 raft_consensus.cc:1275] T 15f1cf7b68d6463fbd60efc8c4576fc8 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Refusing update from remote peer f1991907f700454badc2971ecf14911d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:51.354409 26560 consensus_queue.cc:1048] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:51.360314 26344 raft_consensus.cc:1275] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Refusing update from remote peer cc1c2a51ec344ca28260030593d6563e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:51.362452 26613 consensus_queue.cc:1048] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.363631 26491 raft_consensus.cc:1275] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Refusing update from remote peer f1991907f700454badc2971ecf14911d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:51.366817 26560 consensus_queue.cc:1048] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:51.376183 26491 raft_consensus.cc:1275] T b3488dcfa84d4a699bcc7a4a3c827577 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Refusing update from remote peer cc1c2a51ec344ca28260030593d6563e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:51.378824 26613 consensus_queue.cc:1048] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:51.390280 26344 raft_consensus.cc:1275] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Refusing update from remote peer cc1c2a51ec344ca28260030593d6563e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:51.392171 26611 consensus_queue.cc:1048] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:53.578836 26239 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40376:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:16:53.582065 26239 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:16:53.644553 26332 tablet_service.cc:1511] Processing CreateTablet for tablet bcb7c3cc69b5458db189e1449a4196c8 (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:16:53.645301 26403 tablet_service.cc:1511] Processing CreateTablet for tablet bcb7c3cc69b5458db189e1449a4196c8 (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:16:53.647725 26480 tablet_service.cc:1511] Processing CreateTablet for tablet bcb7c3cc69b5458db189e1449a4196c8 (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:16:53.647755 26332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcb7c3cc69b5458db189e1449a4196c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.647789 26403 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcb7c3cc69b5458db189e1449a4196c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.650179 26480 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bcb7c3cc69b5458db189e1449a4196c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.651180 26330 tablet_service.cc:1511] Processing CreateTablet for tablet 18ad382f11764922a0010dd3e13478d1 (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:16:53.652772 26330 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18ad382f11764922a0010dd3e13478d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.655140 26477 tablet_service.cc:1511] Processing CreateTablet for tablet 18ad382f11764922a0010dd3e13478d1 (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:16:53.656424 26406 tablet_service.cc:1511] Processing CreateTablet for tablet 18ad382f11764922a0010dd3e13478d1 (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:16:53.656980 26477 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18ad382f11764922a0010dd3e13478d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.657966 26406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 18ad382f11764922a0010dd3e13478d1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.661234 26476 tablet_service.cc:1511] Processing CreateTablet for tablet ec84a53b3f294643b5da4d7e464931a9 (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:16:53.663100 26476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec84a53b3f294643b5da4d7e464931a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.662995 26408 tablet_service.cc:1511] Processing CreateTablet for tablet ec84a53b3f294643b5da4d7e464931a9 (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:16:53.667228 26481 tablet_service.cc:1511] Processing CreateTablet for tablet f11300f991be4590bf37eec3e516d05c (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:16:53.667681 26326 tablet_service.cc:1511] Processing CreateTablet for tablet ec84a53b3f294643b5da4d7e464931a9 (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:16:53.669071 26481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f11300f991be4590bf37eec3e516d05c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.664705 26408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec84a53b3f294643b5da4d7e464931a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.675287 26407 tablet_service.cc:1511] Processing CreateTablet for tablet f11300f991be4590bf37eec3e516d05c (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:16:53.677183 26407 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f11300f991be4590bf37eec3e516d05c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.679575 26327 tablet_service.cc:1511] Processing CreateTablet for tablet f11300f991be4590bf37eec3e516d05c (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:16:53.681535 26326 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet ec84a53b3f294643b5da4d7e464931a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.682742 26327 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f11300f991be4590bf37eec3e516d05c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.687814 26402 tablet_service.cc:1511] Processing CreateTablet for tablet d3328229dda9476c97777ff8dc69c435 (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:16:53.689380 26402 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3328229dda9476c97777ff8dc69c435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.690361 26483 tablet_service.cc:1511] Processing CreateTablet for tablet d3328229dda9476c97777ff8dc69c435 (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:16:53.692260 26483 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3328229dda9476c97777ff8dc69c435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.695839 26328 tablet_service.cc:1511] Processing CreateTablet for tablet d3328229dda9476c97777ff8dc69c435 (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:16:53.700491 26328 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d3328229dda9476c97777ff8dc69c435. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.729744 26331 tablet_service.cc:1511] Processing CreateTablet for tablet 224667a8ff894a64a8a95dac8edeeff6 (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:16:53.731385 26331 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 224667a8ff894a64a8a95dac8edeeff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.734112 26482 tablet_service.cc:1511] Processing CreateTablet for tablet 224667a8ff894a64a8a95dac8edeeff6 (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:16:53.736126 26482 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 224667a8ff894a64a8a95dac8edeeff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.737934 26334 tablet_service.cc:1511] Processing CreateTablet for tablet b97814c8781b43a08eb809dd0441fbae (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:16:53.739673 26334 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b97814c8781b43a08eb809dd0441fbae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.736223 26409 tablet_service.cc:1511] Processing CreateTablet for tablet 224667a8ff894a64a8a95dac8edeeff6 (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:16:53.743635 26409 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 224667a8ff894a64a8a95dac8edeeff6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.766142 26405 tablet_service.cc:1511] Processing CreateTablet for tablet b97814c8781b43a08eb809dd0441fbae (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:16:53.767832 26405 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b97814c8781b43a08eb809dd0441fbae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.782805 26478 tablet_service.cc:1511] Processing CreateTablet for tablet b97814c8781b43a08eb809dd0441fbae (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:16:53.784417 26478 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet b97814c8781b43a08eb809dd0441fbae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.789121 26640 tablet_bootstrap.cc:492] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632: Bootstrap starting.
I20260709 12:16:53.798188 26333 tablet_service.cc:1511] Processing CreateTablet for tablet 0cee69f616244c02b6c01f0f7cb1465b (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:16:53.800038 26333 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cee69f616244c02b6c01f0f7cb1465b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.818197 26640 tablet_bootstrap.cc:654] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632: Neither blocks nor log segments found. Creating new log.
W20260709 12:16:53.854355 26227 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.11.13.131:38463, user_credentials={real_user=slave}} blocked reactor thread for 51215.2us
I20260709 12:16:53.860913 26639 tablet_bootstrap.cc:492] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d: Bootstrap starting.
I20260709 12:16:53.871788 26481 tablet_service.cc:1511] Processing CreateTablet for tablet 0cee69f616244c02b6c01f0f7cb1465b (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:16:53.873499 26481 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cee69f616244c02b6c01f0f7cb1465b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:16:53.875373 26229 outbound_call.cc:321] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.11.13.129:37117, user_credentials={real_user=slave}} blocked reactor thread for 53049.4us
I20260709 12:16:53.875418 26476 tablet_service.cc:1511] Processing CreateTablet for tablet 8ebf90a9375a4a2798dad2c4de791d92 (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:16:53.878239 26476 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ebf90a9375a4a2798dad2c4de791d92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.883216 26406 tablet_service.cc:1511] Processing CreateTablet for tablet 8ebf90a9375a4a2798dad2c4de791d92 (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:16:53.891764 26406 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ebf90a9375a4a2798dad2c4de791d92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.886850 26332 tablet_service.cc:1511] Processing CreateTablet for tablet 8ebf90a9375a4a2798dad2c4de791d92 (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:16:53.898476 26332 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8ebf90a9375a4a2798dad2c4de791d92. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.869947 26640 tablet_bootstrap.cc:492] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632: No bootstrap required, opened a new log
I20260709 12:16:53.906781 26640 ts_tablet_manager.cc:1403] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632: Time spent bootstrapping tablet: real 0.118s	user 0.006s	sys 0.016s
I20260709 12:16:53.907387 26639 tablet_bootstrap.cc:654] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.895732 26642 tablet_bootstrap.cc:492] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e: Bootstrap starting.
I20260709 12:16:53.919276 26640 raft_consensus.cc:359] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:53.920398 26640 raft_consensus.cc:385] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.920717 26640 raft_consensus.cc:740] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.885820 26408 tablet_service.cc:1511] Processing CreateTablet for tablet 0cee69f616244c02b6c01f0f7cb1465b (DEFAULT_TABLE table=soft_deleted_table [id=309356b3042343b797534aa2fc120450]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:16:53.922953 26408 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 0cee69f616244c02b6c01f0f7cb1465b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:16:53.930951 26639 tablet_bootstrap.cc:492] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d: No bootstrap required, opened a new log
I20260709 12:16:53.933988 26642 tablet_bootstrap.cc:654] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.934592 26639 ts_tablet_manager.cc:1403] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d: Time spent bootstrapping tablet: real 0.074s	user 0.013s	sys 0.005s
I20260709 12:16:53.938138 26639 raft_consensus.cc:359] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:53.942757 26639 raft_consensus.cc:385] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.943235 26639 raft_consensus.cc:740] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.921581 26640 consensus_queue.cc:260] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:53.944350 26639 consensus_queue.cc:260] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:53.958467 26639 ts_tablet_manager.cc:1434] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d: Time spent starting tablet: real 0.023s	user 0.007s	sys 0.002s
I20260709 12:16:53.959815 26640 ts_tablet_manager.cc:1434] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632: Time spent starting tablet: real 0.053s	user 0.005s	sys 0.003s
I20260709 12:16:53.960968 26640 tablet_bootstrap.cc:492] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632: Bootstrap starting.
I20260709 12:16:53.963142 26639 tablet_bootstrap.cc:492] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d: Bootstrap starting.
I20260709 12:16:53.972981 26639 tablet_bootstrap.cc:654] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.978789 26640 tablet_bootstrap.cc:654] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:53.982251 26642 tablet_bootstrap.cc:492] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e: No bootstrap required, opened a new log
I20260709 12:16:53.982981 26642 ts_tablet_manager.cc:1403] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e: Time spent bootstrapping tablet: real 0.088s	user 0.019s	sys 0.000s
I20260709 12:16:53.986693 26642 raft_consensus.cc:359] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:53.987574 26642 raft_consensus.cc:385] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.987913 26642 raft_consensus.cc:740] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.988754 26642 consensus_queue.cc:260] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:53.990016 26639 tablet_bootstrap.cc:492] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d: No bootstrap required, opened a new log
I20260709 12:16:53.991006 26639 ts_tablet_manager.cc:1403] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d: Time spent bootstrapping tablet: real 0.028s	user 0.022s	sys 0.002s
I20260709 12:16:53.991863 26642 ts_tablet_manager.cc:1434] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:16:53.993016 26642 tablet_bootstrap.cc:492] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e: Bootstrap starting.
I20260709 12:16:53.996323 26639 raft_consensus.cc:359] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:53.998143 26639 raft_consensus.cc:385] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:53.998509 26639 raft_consensus.cc:740] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Initialized, Role: FOLLOWER
I20260709 12:16:53.999359 26639 consensus_queue.cc:260] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.000921 26642 tablet_bootstrap.cc:654] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.002043 26639 ts_tablet_manager.cc:1434] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:16:54.003638 26639 tablet_bootstrap.cc:492] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d: Bootstrap starting.
I20260709 12:16:54.008474 26640 tablet_bootstrap.cc:492] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632: No bootstrap required, opened a new log
I20260709 12:16:54.009642 26640 ts_tablet_manager.cc:1403] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632: Time spent bootstrapping tablet: real 0.049s	user 0.018s	sys 0.014s
I20260709 12:16:54.012413 26612 raft_consensus.cc:493] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.013240 26639 tablet_bootstrap.cc:654] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.013247 26612 raft_consensus.cc:515] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.022197 26642 tablet_bootstrap.cc:492] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e: No bootstrap required, opened a new log
I20260709 12:16:54.023053 26642 ts_tablet_manager.cc:1403] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e: Time spent bootstrapping tablet: real 0.030s	user 0.016s	sys 0.005s
I20260709 12:16:54.023803 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb7c3cc69b5458db189e1449a4196c8" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:54.024942 26342 raft_consensus.cc:2468] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 0.
I20260709 12:16:54.028039 26385 leader_election.cc:304] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:54.030532 26640 raft_consensus.cc:359] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.036574 26640 raft_consensus.cc:385] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.037084 26640 raft_consensus.cc:740] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.030859 26649 raft_consensus.cc:2804] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:54.027678 26642 raft_consensus.cc:359] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.038475 26640 consensus_queue.cc:260] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.039824 26642 raft_consensus.cc:385] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.040212 26642 raft_consensus.cc:740] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.041141 26642 consensus_queue.cc:260] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.032307 26612 leader_election.cc:290] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:54.035159 26491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb7c3cc69b5458db189e1449a4196c8" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632" is_pre_election: true
I20260709 12:16:54.042470 26640 ts_tablet_manager.cc:1434] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632: Time spent starting tablet: real 0.032s	user 0.010s	sys 0.000s
I20260709 12:16:54.043938 26640 tablet_bootstrap.cc:492] T d3328229dda9476c97777ff8dc69c435 P a9c038faa7aa4de680db447a77ed8632: Bootstrap starting.
W20260709 12:16:54.043998 26383 leader_election.cc:343] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.046394 26649 raft_consensus.cc:493] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.046873 26649 raft_consensus.cc:3060] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.051798 26539 raft_consensus.cc:493] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.052505 26539 raft_consensus.cc:515] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.053856 26640 tablet_bootstrap.cc:654] T d3328229dda9476c97777ff8dc69c435 P a9c038faa7aa4de680db447a77ed8632: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.054828 26642 ts_tablet_manager.cc:1434] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e: Time spent starting tablet: real 0.031s	user 0.009s	sys 0.002s
I20260709 12:16:54.056088 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec84a53b3f294643b5da4d7e464931a9" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:54.056933 26539 leader_election.cc:290] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
W20260709 12:16:54.058622 26459 leader_election.cc:343] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f1991907f700454badc2971ecf14911d (127.11.13.129:37117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.058916 26417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec84a53b3f294643b5da4d7e464931a9" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e" is_pre_election: true
I20260709 12:16:54.056119 26642 tablet_bootstrap.cc:492] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e: Bootstrap starting.
I20260709 12:16:54.059216 26649 raft_consensus.cc:515] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
W20260709 12:16:54.060765 26459 leader_election.cc:343] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.061328 26459 leader_election.cc:304] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632; no voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d
I20260709 12:16:54.062716 26539 raft_consensus.cc:2749] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:54.062875 26649 leader_election.cc:290] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:54.065256 26491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb7c3cc69b5458db189e1449a4196c8" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:54.065763 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb7c3cc69b5458db189e1449a4196c8" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:54.066205 26639 tablet_bootstrap.cc:492] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d: No bootstrap required, opened a new log
I20260709 12:16:54.066468 26342 raft_consensus.cc:3060] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.068804 26639 ts_tablet_manager.cc:1403] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d: Time spent bootstrapping tablet: real 0.065s	user 0.017s	sys 0.003s
W20260709 12:16:54.072424 26383 leader_election.cc:343] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.074033 26639 raft_consensus.cc:359] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.075261 26639 raft_consensus.cc:385] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.075686 26639 raft_consensus.cc:740] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.076862 26639 consensus_queue.cc:260] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.079893 26639 ts_tablet_manager.cc:1434] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.003s
I20260709 12:16:54.080605 26642 tablet_bootstrap.cc:654] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.081482 26639 tablet_bootstrap.cc:492] T 224667a8ff894a64a8a95dac8edeeff6 P f1991907f700454badc2971ecf14911d: Bootstrap starting.
I20260709 12:16:54.090754 26639 tablet_bootstrap.cc:654] T 224667a8ff894a64a8a95dac8edeeff6 P f1991907f700454badc2971ecf14911d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.092337 26342 raft_consensus.cc:2468] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 1.
I20260709 12:16:54.095116 26385 leader_election.cc:304] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d; no voters: a9c038faa7aa4de680db447a77ed8632
I20260709 12:16:54.097412 26649 raft_consensus.cc:2804] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:54.098253 26649 raft_consensus.cc:697] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Becoming Leader. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Running, Role: LEADER
I20260709 12:16:54.099826 26649 consensus_queue.cc:237] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.105540 26642 tablet_bootstrap.cc:492] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e: No bootstrap required, opened a new log
I20260709 12:16:54.105641 26639 tablet_bootstrap.cc:492] T 224667a8ff894a64a8a95dac8edeeff6 P f1991907f700454badc2971ecf14911d: No bootstrap required, opened a new log
I20260709 12:16:54.106343 26642 ts_tablet_manager.cc:1403] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e: Time spent bootstrapping tablet: real 0.050s	user 0.011s	sys 0.019s
I20260709 12:16:54.106591 26639 ts_tablet_manager.cc:1403] T 224667a8ff894a64a8a95dac8edeeff6 P f1991907f700454badc2971ecf14911d: Time spent bootstrapping tablet: real 0.025s	user 0.019s	sys 0.004s
I20260709 12:16:54.108637 26640 tablet_bootstrap.cc:492] T d3328229dda9476c97777ff8dc69c435 P a9c038faa7aa4de680db447a77ed8632: No bootstrap required, opened a new log
I20260709 12:16:54.109237 26640 ts_tablet_manager.cc:1403] T d3328229dda9476c97777ff8dc69c435 P a9c038faa7aa4de680db447a77ed8632: Time spent bootstrapping tablet: real 0.066s	user 0.032s	sys 0.002s
I20260709 12:16:54.110252 26642 raft_consensus.cc:359] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.111231 26642 raft_consensus.cc:385] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.111673 26642 raft_consensus.cc:740] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.112015 26639 raft_consensus.cc:359] T 224667a8ff894a64a8a95dac8edeeff6 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.112757 26640 raft_consensus.cc:359] T d3328229dda9476c97777ff8dc69c435 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.112742 26642 consensus_queue.cc:260] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.113456 26639 raft_consensus.cc:385] T 224667a8ff894a64a8a95dac8edeeff6 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.113775 26640 raft_consensus.cc:385] T d3328229dda9476c97777ff8dc69c435 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.114003 26639 raft_consensus.cc:740] T 224667a8ff894a64a8a95dac8edeeff6 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.114351 26640 raft_consensus.cc:740] T d3328229dda9476c97777ff8dc69c435 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.115447 26640 consensus_queue.cc:260] T d3328229dda9476c97777ff8dc69c435 P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.116544 26639 consensus_queue.cc:260] T 224667a8ff894a64a8a95dac8edeeff6 P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.119215 26642 ts_tablet_manager.cc:1434] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.001s
I20260709 12:16:54.120481 26642 tablet_bootstrap.cc:492] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e: Bootstrap starting.
I20260709 12:16:54.124758 26639 ts_tablet_manager.cc:1434] T 224667a8ff894a64a8a95dac8edeeff6 P f1991907f700454badc2971ecf14911d: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.000s
I20260709 12:16:54.125068 26640 ts_tablet_manager.cc:1434] T d3328229dda9476c97777ff8dc69c435 P a9c038faa7aa4de680db447a77ed8632: Time spent starting tablet: real 0.015s	user 0.010s	sys 0.000s
I20260709 12:16:54.126305 26640 tablet_bootstrap.cc:492] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632: Bootstrap starting.
I20260709 12:16:54.129824 26642 tablet_bootstrap.cc:654] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.126305 26639 tablet_bootstrap.cc:492] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d: Bootstrap starting.
I20260709 12:16:54.134393 26240 catalog_manager.cc:5697] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e reported cstate change: term changed from 0 to 1, leader changed from <none> to cc1c2a51ec344ca28260030593d6563e (127.11.13.130). New cstate: current_term: 1 leader_uuid: "cc1c2a51ec344ca28260030593d6563e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:54.141239 26640 tablet_bootstrap.cc:654] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.144474 26639 tablet_bootstrap.cc:654] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.147704 26539 raft_consensus.cc:493] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.148694 26539 raft_consensus.cc:515] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.157645 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11300f991be4590bf37eec3e516d05c" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:54.158504 26417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11300f991be4590bf37eec3e516d05c" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e" is_pre_election: true
W20260709 12:16:54.159621 26459 leader_election.cc:343] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f1991907f700454badc2971ecf14911d (127.11.13.129:37117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.160511 26539 leader_election.cc:290] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
W20260709 12:16:54.160622 26459 leader_election.cc:343] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.161293 26459 leader_election.cc:304] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632; no voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d
I20260709 12:16:54.169010 26539 raft_consensus.cc:2749] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:54.186383 26639 tablet_bootstrap.cc:492] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d: No bootstrap required, opened a new log
I20260709 12:16:54.187095 26639 ts_tablet_manager.cc:1403] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d: Time spent bootstrapping tablet: real 0.061s	user 0.014s	sys 0.034s
I20260709 12:16:54.189465 26642 tablet_bootstrap.cc:492] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e: No bootstrap required, opened a new log
I20260709 12:16:54.189674 26640 tablet_bootstrap.cc:492] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632: No bootstrap required, opened a new log
I20260709 12:16:54.190490 26640 ts_tablet_manager.cc:1403] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632: Time spent bootstrapping tablet: real 0.064s	user 0.019s	sys 0.035s
I20260709 12:16:54.190407 26642 ts_tablet_manager.cc:1403] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e: Time spent bootstrapping tablet: real 0.070s	user 0.026s	sys 0.022s
I20260709 12:16:54.193691 26639 raft_consensus.cc:359] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.194242 26642 raft_consensus.cc:359] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.194588 26640 raft_consensus.cc:359] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.194639 26639 raft_consensus.cc:385] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.195850 26640 raft_consensus.cc:385] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.196071 26639 raft_consensus.cc:740] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.196298 26640 raft_consensus.cc:740] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.195264 26642 raft_consensus.cc:385] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.198519 26642 raft_consensus.cc:740] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.198277 26639 consensus_queue.cc:260] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.199543 26642 consensus_queue.cc:260] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.201388 26639 ts_tablet_manager.cc:1434] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.000s
I20260709 12:16:54.202559 26642 ts_tablet_manager.cc:1434] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.001s
I20260709 12:16:54.202898 26639 tablet_bootstrap.cc:492] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d: Bootstrap starting.
I20260709 12:16:54.198680 26640 consensus_queue.cc:260] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.211041 26642 tablet_bootstrap.cc:492] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e: Bootstrap starting.
I20260709 12:16:54.212245 26640 ts_tablet_manager.cc:1434] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632: Time spent starting tablet: real 0.021s	user 0.008s	sys 0.001s
I20260709 12:16:54.213965 26640 tablet_bootstrap.cc:492] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632: Bootstrap starting.
I20260709 12:16:54.217829 26639 tablet_bootstrap.cc:654] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.220265 26642 tablet_bootstrap.cc:654] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.220883 26649 raft_consensus.cc:493] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.221491 26649 raft_consensus.cc:515] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.224021 26649 leader_election.cc:290] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:54.229137 26491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad382f11764922a0010dd3e13478d1" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632" is_pre_election: true
W20260709 12:16:54.231349 26383 leader_election.cc:343] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.237854 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad382f11764922a0010dd3e13478d1" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:54.239035 26342 raft_consensus.cc:2468] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 0.
I20260709 12:16:54.244864 26385 leader_election.cc:304] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d; no voters: a9c038faa7aa4de680db447a77ed8632
I20260709 12:16:54.246253 26649 raft_consensus.cc:2804] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:54.246668 26649 raft_consensus.cc:493] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.247088 26649 raft_consensus.cc:3060] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.248378 26642 tablet_bootstrap.cc:492] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e: No bootstrap required, opened a new log
I20260709 12:16:54.248725 26640 tablet_bootstrap.cc:654] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.249022 26642 ts_tablet_manager.cc:1403] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e: Time spent bootstrapping tablet: real 0.038s	user 0.015s	sys 0.018s
I20260709 12:16:54.252913 26642 raft_consensus.cc:359] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.253736 26642 raft_consensus.cc:385] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.254053 26642 raft_consensus.cc:740] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.254951 26642 consensus_queue.cc:260] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.255580 26640 tablet_bootstrap.cc:492] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632: No bootstrap required, opened a new log
I20260709 12:16:54.255555 26639 tablet_bootstrap.cc:492] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d: No bootstrap required, opened a new log
I20260709 12:16:54.256665 26640 ts_tablet_manager.cc:1403] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632: Time spent bootstrapping tablet: real 0.043s	user 0.015s	sys 0.001s
I20260709 12:16:54.256861 26639 ts_tablet_manager.cc:1403] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d: Time spent bootstrapping tablet: real 0.054s	user 0.022s	sys 0.017s
I20260709 12:16:54.257645 26642 ts_tablet_manager.cc:1434] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.003s
I20260709 12:16:54.259559 26642 tablet_bootstrap.cc:492] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e: Bootstrap starting.
I20260709 12:16:54.260318 26640 raft_consensus.cc:359] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.260684 26639 raft_consensus.cc:359] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.261250 26640 raft_consensus.cc:385] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.261709 26639 raft_consensus.cc:385] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.261994 26640 raft_consensus.cc:740] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.262790 26639 raft_consensus.cc:740] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.263650 26640 consensus_queue.cc:260] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.266251 26649 raft_consensus.cc:515] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.269412 26640 ts_tablet_manager.cc:1434] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632: Time spent starting tablet: real 0.012s	user 0.008s	sys 0.003s
I20260709 12:16:54.269717 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad382f11764922a0010dd3e13478d1" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:54.270428 26342 raft_consensus.cc:3060] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.270772 26640 tablet_bootstrap.cc:492] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632: Bootstrap starting.
I20260709 12:16:54.271086 26642 tablet_bootstrap.cc:654] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.279836 26640 tablet_bootstrap.cc:654] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.284416 26642 tablet_bootstrap.cc:492] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e: No bootstrap required, opened a new log
I20260709 12:16:54.264024 26639 consensus_queue.cc:260] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.285629 26642 ts_tablet_manager.cc:1403] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e: Time spent bootstrapping tablet: real 0.026s	user 0.013s	sys 0.008s
I20260709 12:16:54.287971 26639 ts_tablet_manager.cc:1434] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d: Time spent starting tablet: real 0.030s	user 0.007s	sys 0.000s
I20260709 12:16:54.289248 26639 tablet_bootstrap.cc:492] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d: Bootstrap starting.
I20260709 12:16:54.290500 26642 raft_consensus.cc:359] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.291400 26649 leader_election.cc:290] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:54.291638 26642 raft_consensus.cc:385] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.292099 26642 raft_consensus.cc:740] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.292888 26491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad382f11764922a0010dd3e13478d1" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:54.293529 26342 raft_consensus.cc:2468] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 1.
I20260709 12:16:54.293169 26642 consensus_queue.cc:260] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
W20260709 12:16:54.294713 26383 leader_election.cc:343] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.295383 26385 leader_election.cc:304] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d; no voters: a9c038faa7aa4de680db447a77ed8632
I20260709 12:16:54.295909 26642 ts_tablet_manager.cc:1434] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e: Time spent starting tablet: real 0.010s	user 0.008s	sys 0.000s
I20260709 12:16:54.296934 26612 raft_consensus.cc:2804] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:54.297034 26642 tablet_bootstrap.cc:492] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e: Bootstrap starting.
I20260709 12:16:54.297458 26612 raft_consensus.cc:697] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Becoming Leader. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Running, Role: LEADER
I20260709 12:16:54.298714 26612 consensus_queue.cc:237] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.304486 26639 tablet_bootstrap.cc:654] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.308950 26640 tablet_bootstrap.cc:492] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632: No bootstrap required, opened a new log
I20260709 12:16:54.309237 26642 tablet_bootstrap.cc:654] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.309549 26640 ts_tablet_manager.cc:1403] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632: Time spent bootstrapping tablet: real 0.039s	user 0.011s	sys 0.009s
I20260709 12:16:54.313400 26640 raft_consensus.cc:359] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.314520 26640 raft_consensus.cc:385] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.314930 26640 raft_consensus.cc:740] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.316676 26628 raft_consensus.cc:493] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.317548 26628 raft_consensus.cc:515] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.321098 26417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3328229dda9476c97777ff8dc69c435" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e" is_pre_election: true
I20260709 12:16:54.315953 26640 consensus_queue.cc:260] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.321832 26417 raft_consensus.cc:2468] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 0.
I20260709 12:16:54.322064 26642 tablet_bootstrap.cc:492] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e: No bootstrap required, opened a new log
I20260709 12:16:54.323045 26642 ts_tablet_manager.cc:1403] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e: Time spent bootstrapping tablet: real 0.026s	user 0.018s	sys 0.003s
I20260709 12:16:54.323201 26310 leader_election.cc:304] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:54.322909 26491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3328229dda9476c97777ff8dc69c435" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632" is_pre_election: true
I20260709 12:16:54.324079 26491 raft_consensus.cc:2468] T d3328229dda9476c97777ff8dc69c435 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 0.
I20260709 12:16:54.323501 26628 leader_election.cc:290] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:54.325733 26640 ts_tablet_manager.cc:1434] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632: Time spent starting tablet: real 0.016s	user 0.010s	sys 0.001s
I20260709 12:16:54.327490 26642 raft_consensus.cc:359] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.328791 26642 raft_consensus.cc:385] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.329114 26642 raft_consensus.cc:740] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.329468 26628 raft_consensus.cc:2804] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:54.330062 26642 consensus_queue.cc:260] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.336858 26628 raft_consensus.cc:493] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.338099 26628 raft_consensus.cc:3060] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.338889 26642 ts_tablet_manager.cc:1434] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e: Time spent starting tablet: real 0.015s	user 0.009s	sys 0.001s
I20260709 12:16:54.340299 26642 tablet_bootstrap.cc:492] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e: Bootstrap starting.
I20260709 12:16:54.330281 26640 tablet_bootstrap.cc:492] T b97814c8781b43a08eb809dd0441fbae P a9c038faa7aa4de680db447a77ed8632: Bootstrap starting.
I20260709 12:16:54.342216 26240 catalog_manager.cc:5697] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e reported cstate change: term changed from 0 to 1, leader changed from <none> to cc1c2a51ec344ca28260030593d6563e (127.11.13.130). New cstate: current_term: 1 leader_uuid: "cc1c2a51ec344ca28260030593d6563e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:54.348294 26642 tablet_bootstrap.cc:654] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.351089 26639 tablet_bootstrap.cc:492] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d: No bootstrap required, opened a new log
I20260709 12:16:54.351763 26639 ts_tablet_manager.cc:1403] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d: Time spent bootstrapping tablet: real 0.063s	user 0.008s	sys 0.017s
I20260709 12:16:54.352856 26640 tablet_bootstrap.cc:654] T b97814c8781b43a08eb809dd0441fbae P a9c038faa7aa4de680db447a77ed8632: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.356153 26639 raft_consensus.cc:359] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.357303 26639 raft_consensus.cc:385] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.357677 26639 raft_consensus.cc:740] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.359153 26639 consensus_queue.cc:260] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.360440 26642 tablet_bootstrap.cc:492] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e: No bootstrap required, opened a new log
I20260709 12:16:54.361040 26642 ts_tablet_manager.cc:1403] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e: Time spent bootstrapping tablet: real 0.021s	user 0.018s	sys 0.000s
I20260709 12:16:54.361720 26639 ts_tablet_manager.cc:1434] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d: Time spent starting tablet: real 0.010s	user 0.002s	sys 0.006s
I20260709 12:16:54.365089 26642 raft_consensus.cc:359] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.365512 26628 raft_consensus.cc:515] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.366266 26642 raft_consensus.cc:385] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.366734 26642 raft_consensus.cc:740] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.367918 26642 consensus_queue.cc:260] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.369066 26417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3328229dda9476c97777ff8dc69c435" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:54.369727 26417 raft_consensus.cc:3060] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.372177 26642 ts_tablet_manager.cc:1434] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e: Time spent starting tablet: real 0.011s	user 0.013s	sys 0.000s
I20260709 12:16:54.374182 26642 tablet_bootstrap.cc:492] T 8ebf90a9375a4a2798dad2c4de791d92 P cc1c2a51ec344ca28260030593d6563e: Bootstrap starting.
I20260709 12:16:54.383698 26642 tablet_bootstrap.cc:654] T 8ebf90a9375a4a2798dad2c4de791d92 P cc1c2a51ec344ca28260030593d6563e: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.388697 26639 tablet_bootstrap.cc:492] T 0cee69f616244c02b6c01f0f7cb1465b P f1991907f700454badc2971ecf14911d: Bootstrap starting.
I20260709 12:16:54.388593 26672 raft_consensus.cc:493] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.389482 26672 raft_consensus.cc:515] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.392234 26672 leader_election.cc:290] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:54.395426 26491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97814c8781b43a08eb809dd0441fbae" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632" is_pre_election: true
I20260709 12:16:54.397101 26642 tablet_bootstrap.cc:492] T 8ebf90a9375a4a2798dad2c4de791d92 P cc1c2a51ec344ca28260030593d6563e: No bootstrap required, opened a new log
I20260709 12:16:54.397133 26640 tablet_bootstrap.cc:492] T b97814c8781b43a08eb809dd0441fbae P a9c038faa7aa4de680db447a77ed8632: No bootstrap required, opened a new log
I20260709 12:16:54.397946 26642 ts_tablet_manager.cc:1403] T 8ebf90a9375a4a2798dad2c4de791d92 P cc1c2a51ec344ca28260030593d6563e: Time spent bootstrapping tablet: real 0.024s	user 0.014s	sys 0.007s
I20260709 12:16:54.397946 26640 ts_tablet_manager.cc:1403] T b97814c8781b43a08eb809dd0441fbae P a9c038faa7aa4de680db447a77ed8632: Time spent bootstrapping tablet: real 0.068s	user 0.025s	sys 0.021s
W20260709 12:16:54.397334 26308 leader_election.cc:343] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.400504 26612 raft_consensus.cc:493] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.401067 26612 raft_consensus.cc:515] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.401677 26642 raft_consensus.cc:359] T 8ebf90a9375a4a2798dad2c4de791d92 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.401522 26640 raft_consensus.cc:359] T b97814c8781b43a08eb809dd0441fbae P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.402459 26642 raft_consensus.cc:385] T 8ebf90a9375a4a2798dad2c4de791d92 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.402573 26640 raft_consensus.cc:385] T b97814c8781b43a08eb809dd0441fbae P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.402858 26642 raft_consensus.cc:740] T 8ebf90a9375a4a2798dad2c4de791d92 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.402952 26640 raft_consensus.cc:740] T b97814c8781b43a08eb809dd0441fbae P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.403730 26642 consensus_queue.cc:260] T 8ebf90a9375a4a2798dad2c4de791d92 P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.404000 26640 consensus_queue.cc:260] T b97814c8781b43a08eb809dd0441fbae P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.404429 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97814c8781b43a08eb809dd0441fbae" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:54.405247 26342 raft_consensus.cc:2468] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 0.
I20260709 12:16:54.405957 26642 ts_tablet_manager.cc:1434] T 8ebf90a9375a4a2798dad2c4de791d92 P cc1c2a51ec344ca28260030593d6563e: Time spent starting tablet: real 0.007s	user 0.006s	sys 0.000s
I20260709 12:16:54.406870 26385 leader_election.cc:304] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:54.407831 26649 raft_consensus.cc:2804] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:54.408241 26649 raft_consensus.cc:493] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.408600 26649 raft_consensus.cc:3060] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.409231 26640 ts_tablet_manager.cc:1434] T b97814c8781b43a08eb809dd0441fbae P a9c038faa7aa4de680db447a77ed8632: Time spent starting tablet: real 0.011s	user 0.006s	sys 0.000s
I20260709 12:16:54.410374 26640 tablet_bootstrap.cc:492] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632: Bootstrap starting.
I20260709 12:16:54.411001 26612 leader_election.cc:290] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:54.412591 26419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97814c8781b43a08eb809dd0441fbae" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e" is_pre_election: true
I20260709 12:16:54.413178 26491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97814c8781b43a08eb809dd0441fbae" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632" is_pre_election: true
I20260709 12:16:54.413817 26491 raft_consensus.cc:2468] T b97814c8781b43a08eb809dd0441fbae P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 0.
I20260709 12:16:54.414937 26417 raft_consensus.cc:2468] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 1.
I20260709 12:16:54.416565 26628 leader_election.cc:290] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 election: Requested vote from peers cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:54.418474 26640 tablet_bootstrap.cc:654] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.419569 26639 tablet_bootstrap.cc:654] T 0cee69f616244c02b6c01f0f7cb1465b P f1991907f700454badc2971ecf14911d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.421176 26491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3328229dda9476c97777ff8dc69c435" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:54.422036 26491 raft_consensus.cc:3060] T d3328229dda9476c97777ff8dc69c435 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.426829 26310 leader_election.cc:304] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:54.427983 26628 raft_consensus.cc:2804] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:54.428519 26628 raft_consensus.cc:697] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [term 1 LEADER]: Becoming Leader. State: Replica: f1991907f700454badc2971ecf14911d, State: Running, Role: LEADER
I20260709 12:16:54.429489 26628 consensus_queue.cc:237] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.435048 26539 raft_consensus.cc:493] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.435678 26612 raft_consensus.cc:493] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.435750 26539 raft_consensus.cc:515] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.436355 26612 raft_consensus.cc:515] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.438184 26539 leader_election.cc:290] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:54.439142 26612 leader_election.cc:290] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:54.439409 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec84a53b3f294643b5da4d7e464931a9" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:54.440364 26342 raft_consensus.cc:2468] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 0.
I20260709 12:16:54.440598 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb7c3cc69b5458db189e1449a4196c8" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:54.441920 26344 raft_consensus.cc:2393] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a9c038faa7aa4de680db447a77ed8632 in current term 1: Already voted for candidate cc1c2a51ec344ca28260030593d6563e in this term.
I20260709 12:16:54.443771 26385 leader_election.cc:304] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:54.419766 26649 raft_consensus.cc:515] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.444896 26493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec84a53b3f294643b5da4d7e464931a9" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632" is_pre_election: true
I20260709 12:16:54.445706 26612 raft_consensus.cc:2804] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:54.446842 26612 raft_consensus.cc:493] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.447510 26612 raft_consensus.cc:3060] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.445703 26493 raft_consensus.cc:2468] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 0.
I20260709 12:16:54.450749 26417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb7c3cc69b5458db189e1449a4196c8" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e" is_pre_election: true
I20260709 12:16:54.452958 26419 raft_consensus.cc:2393] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate f1991907f700454badc2971ecf14911d in current term 1: Already voted for candidate cc1c2a51ec344ca28260030593d6563e in this term.
I20260709 12:16:54.459069 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97814c8781b43a08eb809dd0441fbae" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:54.459889 26344 raft_consensus.cc:3060] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.460043 26239 catalog_manager.cc:5697] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d reported cstate change: term changed from 0 to 1, leader changed from <none> to f1991907f700454badc2971ecf14911d (127.11.13.129). New cstate: current_term: 1 leader_uuid: "f1991907f700454badc2971ecf14911d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:54.459664 26459 leader_election.cc:304] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632; no voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d
I20260709 12:16:54.463733 26644 raft_consensus.cc:3060] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.473042 26491 raft_consensus.cc:2468] T d3328229dda9476c97777ff8dc69c435 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 1.
I20260709 12:16:54.473234 26644 raft_consensus.cc:2749] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:54.475768 26644 raft_consensus.cc:493] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.480067 26310 leader_election.cc:304] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f1991907f700454badc2971ecf14911d; no voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e
I20260709 12:16:54.480257 26644 raft_consensus.cc:515] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.459156 26649 leader_election.cc:290] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:54.481884 26639 tablet_bootstrap.cc:492] T 0cee69f616244c02b6c01f0f7cb1465b P f1991907f700454badc2971ecf14911d: No bootstrap required, opened a new log
I20260709 12:16:54.483180 26639 ts_tablet_manager.cc:1403] T 0cee69f616244c02b6c01f0f7cb1465b P f1991907f700454badc2971ecf14911d: Time spent bootstrapping tablet: real 0.096s	user 0.025s	sys 0.026s
I20260709 12:16:54.484594 26644 leader_election.cc:290] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:54.488072 26344 raft_consensus.cc:2468] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 1.
I20260709 12:16:54.488690 26628 raft_consensus.cc:2749] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:54.490110 26640 tablet_bootstrap.cc:492] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632: No bootstrap required, opened a new log
I20260709 12:16:54.491199 26640 ts_tablet_manager.cc:1403] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632: Time spent bootstrapping tablet: real 0.081s	user 0.014s	sys 0.002s
I20260709 12:16:54.490690 26639 raft_consensus.cc:359] T 0cee69f616244c02b6c01f0f7cb1465b P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.491863 26493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "b97814c8781b43a08eb809dd0441fbae" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:54.492127 26639 raft_consensus.cc:385] T 0cee69f616244c02b6c01f0f7cb1465b P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.492650 26493 raft_consensus.cc:3060] T b97814c8781b43a08eb809dd0441fbae P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.492668 26639 raft_consensus.cc:740] T 0cee69f616244c02b6c01f0f7cb1465b P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.493803 26639 consensus_queue.cc:260] T 0cee69f616244c02b6c01f0f7cb1465b P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.491196 26417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "224667a8ff894a64a8a95dac8edeeff6" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e" is_pre_election: true
I20260709 12:16:54.495194 26417 raft_consensus.cc:2468] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 0.
I20260709 12:16:54.495539 26385 leader_election.cc:304] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:54.495432 26640 raft_consensus.cc:359] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.496873 26640 raft_consensus.cc:385] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.496989 26649 raft_consensus.cc:2804] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:54.497309 26640 raft_consensus.cc:740] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Initialized, Role: FOLLOWER
W20260709 12:16:54.490319 26290 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:16:54.499429 26639 ts_tablet_manager.cc:1434] T 0cee69f616244c02b6c01f0f7cb1465b P f1991907f700454badc2971ecf14911d: Time spent starting tablet: real 0.016s	user 0.007s	sys 0.000s
I20260709 12:16:54.500556 26639 tablet_bootstrap.cc:492] T 8ebf90a9375a4a2798dad2c4de791d92 P f1991907f700454badc2971ecf14911d: Bootstrap starting.
I20260709 12:16:54.498695 26649 raft_consensus.cc:697] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Becoming Leader. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Running, Role: LEADER
I20260709 12:16:54.501564 26649 consensus_queue.cc:237] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.490732 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "224667a8ff894a64a8a95dac8edeeff6" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:54.503111 26342 raft_consensus.cc:2468] T 224667a8ff894a64a8a95dac8edeeff6 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 0.
I20260709 12:16:54.505569 26459 leader_election.cc:304] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:54.506490 26539 raft_consensus.cc:2804] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:54.506924 26539 raft_consensus.cc:493] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.507251 26539 raft_consensus.cc:3060] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.507870 26639 tablet_bootstrap.cc:654] T 8ebf90a9375a4a2798dad2c4de791d92 P f1991907f700454badc2971ecf14911d: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.513720 26640 consensus_queue.cc:260] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.517454 26539 raft_consensus.cc:515] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.518615 26493 raft_consensus.cc:2468] T b97814c8781b43a08eb809dd0441fbae P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 1.
I20260709 12:16:54.520893 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "224667a8ff894a64a8a95dac8edeeff6" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:54.521548 26342 raft_consensus.cc:3060] T 224667a8ff894a64a8a95dac8edeeff6 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.523206 26539 leader_election.cc:290] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:54.527451 26539 raft_consensus.cc:493] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.528034 26651 consensus_queue.cc:1048] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:54.528194 26539 raft_consensus.cc:515] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.529731 26640 ts_tablet_manager.cc:1434] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632: Time spent starting tablet: real 0.038s	user 0.007s	sys 0.000s
I20260709 12:16:54.532261 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ebf90a9375a4a2798dad2c4de791d92" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:54.533392 26417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "224667a8ff894a64a8a95dac8edeeff6" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:54.534008 26651 raft_consensus.cc:493] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.534786 26651 raft_consensus.cc:515] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.538447 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11300f991be4590bf37eec3e516d05c" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:54.539440 26343 raft_consensus.cc:2468] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 0.
I20260709 12:16:54.541133 26385 leader_election.cc:304] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:54.542343 26612 raft_consensus.cc:515] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.538466 26640 tablet_bootstrap.cc:492] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632: Bootstrap starting.
I20260709 12:16:54.546880 26651 leader_election.cc:290] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:54.549329 26492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec84a53b3f294643b5da4d7e464931a9" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:54.534025 26417 raft_consensus.cc:3060] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.549772 26651 raft_consensus.cc:2804] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:54.550060 26492 raft_consensus.cc:3060] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.552557 26490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11300f991be4590bf37eec3e516d05c" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632" is_pre_election: true
I20260709 12:16:54.558773 26490 raft_consensus.cc:2468] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 0.
I20260709 12:16:54.544734 26612 leader_election.cc:290] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
W20260709 12:16:54.550551 26459 leader_election.cc:343] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer f1991907f700454badc2971ecf14911d (127.11.13.129:37117): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.555593 26651 raft_consensus.cc:493] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.566524 26651 raft_consensus.cc:3060] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.564743 26342 raft_consensus.cc:2468] T 224667a8ff894a64a8a95dac8edeeff6 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 1.
I20260709 12:16:54.570309 26539 leader_election.cc:290] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:54.573169 26459 leader_election.cc:304] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:54.574357 26640 tablet_bootstrap.cc:654] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632: Neither blocks nor log segments found. Creating new log.
I20260709 12:16:54.574479 26539 raft_consensus.cc:2804] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:54.580161 26644 raft_consensus.cc:493] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.580770 26644 raft_consensus.cc:515] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.586040 26644 leader_election.cc:290] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:54.584091 26239 catalog_manager.cc:5697] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e reported cstate change: term changed from 0 to 1, leader changed from <none> to cc1c2a51ec344ca28260030593d6563e (127.11.13.130). New cstate: current_term: 1 leader_uuid: "cc1c2a51ec344ca28260030593d6563e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:54.581041 26539 raft_consensus.cc:697] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Becoming Leader. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Running, Role: LEADER
I20260709 12:16:54.595335 26539 consensus_queue.cc:237] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.597443 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad382f11764922a0010dd3e13478d1" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:54.598279 26342 raft_consensus.cc:2393] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a9c038faa7aa4de680db447a77ed8632 in current term 1: Already voted for candidate cc1c2a51ec344ca28260030593d6563e in this term.
I20260709 12:16:54.600553 26417 raft_consensus.cc:2468] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 1.
W20260709 12:16:54.610567 26385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.129:37117, user_credentials={real_user=slave}} blocked reactor thread for 69543.5us
I20260709 12:16:54.612766 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec84a53b3f294643b5da4d7e464931a9" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:54.613351 26342 raft_consensus.cc:3060] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.615258 26417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ebf90a9375a4a2798dad2c4de791d92" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e" is_pre_election: true
I20260709 12:16:54.616304 26417 raft_consensus.cc:2468] T 8ebf90a9375a4a2798dad2c4de791d92 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 0.
I20260709 12:16:54.617460 26419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad382f11764922a0010dd3e13478d1" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e" is_pre_election: true
I20260709 12:16:54.619076 26459 leader_election.cc:304] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e; no voters: f1991907f700454badc2971ecf14911d
I20260709 12:16:54.620399 26644 raft_consensus.cc:2804] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:54.620815 26644 raft_consensus.cc:493] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.620689 26651 raft_consensus.cc:515] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.621968 26644 raft_consensus.cc:3060] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.624620 26639 tablet_bootstrap.cc:492] T 8ebf90a9375a4a2798dad2c4de791d92 P f1991907f700454badc2971ecf14911d: No bootstrap required, opened a new log
I20260709 12:16:54.625223 26639 ts_tablet_manager.cc:1403] T 8ebf90a9375a4a2798dad2c4de791d92 P f1991907f700454badc2971ecf14911d: Time spent bootstrapping tablet: real 0.125s	user 0.060s	sys 0.042s
I20260709 12:16:54.631275 26342 raft_consensus.cc:2468] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 1.
I20260709 12:16:54.637913 26649 consensus_queue.cc:1048] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:54.629936 26492 raft_consensus.cc:2468] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 1.
I20260709 12:16:54.632050 26459 leader_election.cc:304] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632; no voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d
I20260709 12:16:54.641438 26383 leader_election.cc:304] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e; no voters: 
I20260709 12:16:54.642498 26612 raft_consensus.cc:2804] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:54.631389 26644 raft_consensus.cc:515] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.645298 26644 leader_election.cc:290] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:54.646407 26644 raft_consensus.cc:3060] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.646899 26239 catalog_manager.cc:5697] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9c038faa7aa4de680db447a77ed8632 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "a9c038faa7aa4de680db447a77ed8632" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:54.637223 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11300f991be4590bf37eec3e516d05c" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:54.655016 26344 raft_consensus.cc:3060] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.657058 26644 raft_consensus.cc:2749] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:54.637313 26493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11300f991be4590bf37eec3e516d05c" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:54.658923 26493 raft_consensus.cc:3060] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.637457 26682 raft_consensus.cc:493] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.662771 26682 raft_consensus.cc:515] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.642501 26639 raft_consensus.cc:359] T 8ebf90a9375a4a2798dad2c4de791d92 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.666491 26343 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cee69f616244c02b6c01f0f7cb1465b" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:54.667354 26492 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cee69f616244c02b6c01f0f7cb1465b" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632" is_pre_election: true
I20260709 12:16:54.667469 26343 raft_consensus.cc:2468] T 0cee69f616244c02b6c01f0f7cb1465b P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 0.
I20260709 12:16:54.668431 26682 leader_election.cc:290] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:54.666518 26639 raft_consensus.cc:385] T 8ebf90a9375a4a2798dad2c4de791d92 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.669363 26639 raft_consensus.cc:740] T 8ebf90a9375a4a2798dad2c4de791d92 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Initialized, Role: FOLLOWER
W20260709 12:16:54.669803 26383 leader_election.cc:343] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.670166 26639 consensus_queue.cc:260] T 8ebf90a9375a4a2798dad2c4de791d92 P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.672747 26344 raft_consensus.cc:2468] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 1.
I20260709 12:16:54.674062 26639 ts_tablet_manager.cc:1434] T 8ebf90a9375a4a2798dad2c4de791d92 P f1991907f700454badc2971ecf14911d: Time spent starting tablet: real 0.049s	user 0.010s	sys 0.000s
I20260709 12:16:54.675895 26493 raft_consensus.cc:2468] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 1.
I20260709 12:16:54.675849 26385 leader_election.cc:304] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d; no voters: a9c038faa7aa4de680db447a77ed8632
I20260709 12:16:54.677600 26383 leader_election.cc:304] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e; no voters: 
I20260709 12:16:54.678498 26682 raft_consensus.cc:2804] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:16:54.678892 26682 raft_consensus.cc:493] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:16:54.679071 26640 tablet_bootstrap.cc:492] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632: No bootstrap required, opened a new log
I20260709 12:16:54.679255 26682 raft_consensus.cc:3060] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.679623 26640 ts_tablet_manager.cc:1403] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632: Time spent bootstrapping tablet: real 0.142s	user 0.015s	sys 0.020s
I20260709 12:16:54.680835 26690 raft_consensus.cc:2804] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:54.681561 26690 raft_consensus.cc:697] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Becoming Leader. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Running, Role: LEADER
I20260709 12:16:54.681877 26612 raft_consensus.cc:697] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Becoming Leader. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Running, Role: LEADER
I20260709 12:16:54.683022 26612 consensus_queue.cc:237] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.636193 26651 leader_election.cc:290] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:54.691926 26690 consensus_queue.cc:237] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
W20260709 12:16:54.694331 26459 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.11.13.130:38281, user_credentials={real_user=slave}} blocked reactor thread for 62363.1us
I20260709 12:16:54.694571 26682 raft_consensus.cc:515] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.696517 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ebf90a9375a4a2798dad2c4de791d92" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:54.697259 26344 raft_consensus.cc:3060] T 8ebf90a9375a4a2798dad2c4de791d92 P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.698594 26493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cee69f616244c02b6c01f0f7cb1465b" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
W20260709 12:16:54.700204 26383 leader_election.cc:343] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:16:54.704764 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cee69f616244c02b6c01f0f7cb1465b" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:54.704970 26640 raft_consensus.cc:359] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.705576 26419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8ebf90a9375a4a2798dad2c4de791d92" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:54.706068 26640 raft_consensus.cc:385] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:16:54.706362 26419 raft_consensus.cc:3060] T 8ebf90a9375a4a2798dad2c4de791d92 P cc1c2a51ec344ca28260030593d6563e [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.706493 26640 raft_consensus.cc:740] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Initialized, Role: FOLLOWER
I20260709 12:16:54.705724 26342 raft_consensus.cc:3060] T 0cee69f616244c02b6c01f0f7cb1465b P f1991907f700454badc2971ecf14911d [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.710788 26640 consensus_queue.cc:260] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.714236 26344 raft_consensus.cc:2468] T 8ebf90a9375a4a2798dad2c4de791d92 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 1.
I20260709 12:16:54.715566 26419 raft_consensus.cc:2468] T 8ebf90a9375a4a2798dad2c4de791d92 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 1.
I20260709 12:16:54.716403 26342 raft_consensus.cc:2468] T 0cee69f616244c02b6c01f0f7cb1465b P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 1.
I20260709 12:16:54.717761 26459 leader_election.cc:304] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e; no voters: 
I20260709 12:16:54.718689 26385 leader_election.cc:304] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d; no voters: a9c038faa7aa4de680db447a77ed8632
I20260709 12:16:54.719724 26651 raft_consensus.cc:2804] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:54.720141 26687 raft_consensus.cc:2804] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:16:54.720453 26651 raft_consensus.cc:697] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Becoming Leader. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Running, Role: LEADER
I20260709 12:16:54.720783 26687 raft_consensus.cc:697] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Becoming Leader. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Running, Role: LEADER
I20260709 12:16:54.721943 26651 consensus_queue.cc:237] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.721943 26687 consensus_queue.cc:237] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.701122 26682 leader_election.cc:290] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 1 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:54.737917 26238 catalog_manager.cc:5697] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e reported cstate change: term changed from 0 to 1, leader changed from <none> to cc1c2a51ec344ca28260030593d6563e (127.11.13.130). New cstate: current_term: 1 leader_uuid: "cc1c2a51ec344ca28260030593d6563e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:54.741623 26640 ts_tablet_manager.cc:1434] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632: Time spent starting tablet: real 0.062s	user 0.010s	sys 0.000s
I20260709 12:16:54.742020 26239 catalog_manager.cc:5697] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 reported cstate change: term changed from 0 to 1, leader changed from <none> to a9c038faa7aa4de680db447a77ed8632 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "a9c038faa7aa4de680db447a77ed8632" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:54.755939 26239 catalog_manager.cc:5697] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e reported cstate change: term changed from 0 to 1, leader changed from <none> to cc1c2a51ec344ca28260030593d6563e (127.11.13.130). New cstate: current_term: 1 leader_uuid: "cc1c2a51ec344ca28260030593d6563e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:54.757542 26239 catalog_manager.cc:5697] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e reported cstate change: term changed from 0 to 1, leader changed from <none> to cc1c2a51ec344ca28260030593d6563e (127.11.13.130). New cstate: current_term: 1 leader_uuid: "cc1c2a51ec344ca28260030593d6563e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:54.789608 26612 consensus_queue.cc:1048] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:54.802042 26651 consensus_queue.cc:1048] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:54.828332 26687 raft_consensus.cc:493] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:16:54.829351 26687 raft_consensus.cc:515] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.831365 26687 leader_election.cc:290] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:54.832587 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cee69f616244c02b6c01f0f7cb1465b" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1991907f700454badc2971ecf14911d" is_pre_election: true
I20260709 12:16:54.833231 26419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cee69f616244c02b6c01f0f7cb1465b" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc1c2a51ec344ca28260030593d6563e" is_pre_election: true
I20260709 12:16:54.833472 26344 raft_consensus.cc:2393] T 0cee69f616244c02b6c01f0f7cb1465b P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a9c038faa7aa4de680db447a77ed8632 in current term 1: Already voted for candidate cc1c2a51ec344ca28260030593d6563e in this term.
I20260709 12:16:54.834921 26459 leader_election.cc:304] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632; no voters: cc1c2a51ec344ca28260030593d6563e, f1991907f700454badc2971ecf14911d
I20260709 12:16:54.835613 26687 raft_consensus.cc:3060] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:16:54.843052 26687 raft_consensus.cc:2749] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:16:54.848861 26419 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "bcb7c3cc69b5458db189e1449a4196c8"
dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
mode: GRACEFUL
new_leader_uuid: "f1991907f700454badc2971ecf14911d"
 from {username='slave'} at 127.0.0.1:55504
I20260709 12:16:54.849835 26419 raft_consensus.cc:606] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Received request to transfer leadership to TS f1991907f700454badc2971ecf14911d
I20260709 12:16:54.856585 26419 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "18ad382f11764922a0010dd3e13478d1"
dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
mode: GRACEFUL
new_leader_uuid: "f1991907f700454badc2971ecf14911d"
 from {username='slave'} at 127.0.0.1:55504
I20260709 12:16:54.857254 26419 raft_consensus.cc:606] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Received request to transfer leadership to TS f1991907f700454badc2971ecf14911d
I20260709 12:16:54.858597 26628 consensus_queue.cc:1048] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:54.864986 26417 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec84a53b3f294643b5da4d7e464931a9"
dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
mode: GRACEFUL
new_leader_uuid: "f1991907f700454badc2971ecf14911d"
 from {username='slave'} at 127.0.0.1:55504
I20260709 12:16:54.865703 26417 raft_consensus.cc:606] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Received request to transfer leadership to TS f1991907f700454badc2971ecf14911d
I20260709 12:16:54.872315 26419 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f11300f991be4590bf37eec3e516d05c"
dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
mode: GRACEFUL
new_leader_uuid: "f1991907f700454badc2971ecf14911d"
 from {username='slave'} at 127.0.0.1:55504
I20260709 12:16:54.872996 26419 raft_consensus.cc:606] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Received request to transfer leadership to TS f1991907f700454badc2971ecf14911d
I20260709 12:16:54.872992 26672 consensus_queue.cc:1048] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:54.881114 26344 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "d3328229dda9476c97777ff8dc69c435"
dest_uuid: "f1991907f700454badc2971ecf14911d"
mode: GRACEFUL
new_leader_uuid: "cc1c2a51ec344ca28260030593d6563e"
 from {username='slave'} at 127.0.0.1:60322
I20260709 12:16:54.881821 26344 raft_consensus.cc:606] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [term 1 LEADER]: Received request to transfer leadership to TS cc1c2a51ec344ca28260030593d6563e
I20260709 12:16:54.887638 26628 raft_consensus.cc:993] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d: : Instructing follower cc1c2a51ec344ca28260030593d6563e to start an election
I20260709 12:16:54.888300 26671 raft_consensus.cc:1081] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [term 1 LEADER]: Signalling peer cc1c2a51ec344ca28260030593d6563e to start an election
I20260709 12:16:54.890738 26419 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "d3328229dda9476c97777ff8dc69c435"
dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
 from {username='slave'} at 127.0.0.1:55498
I20260709 12:16:54.891417 26419 raft_consensus.cc:493] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:54.891637 26491 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "224667a8ff894a64a8a95dac8edeeff6"
dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
mode: GRACEFUL
new_leader_uuid: "cc1c2a51ec344ca28260030593d6563e"
 from {username='slave'} at 127.0.0.1:48082
I20260709 12:16:54.891901 26419 raft_consensus.cc:3060] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:54.892391 26491 raft_consensus.cc:606] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Received request to transfer leadership to TS cc1c2a51ec344ca28260030593d6563e
I20260709 12:16:54.900687 26419 raft_consensus.cc:515] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.903281 26419 leader_election.cc:290] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 2 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:54.903990 26344 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3328229dda9476c97777ff8dc69c435" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:54.904745 26344 raft_consensus.cc:3055] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:54.905103 26344 raft_consensus.cc:740] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Running, Role: LEADER
I20260709 12:16:54.905234 26491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d3328229dda9476c97777ff8dc69c435" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:54.905870 26491 raft_consensus.cc:3060] T d3328229dda9476c97777ff8dc69c435 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:54.906173 26344 consensus_queue.cc:260] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.907718 26344 raft_consensus.cc:3060] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:54.913610 26491 raft_consensus.cc:2468] T d3328229dda9476c97777ff8dc69c435 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 2.
I20260709 12:16:54.915028 26344 raft_consensus.cc:2468] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 2.
I20260709 12:16:54.915288 26383 leader_election.cc:304] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e; no voters: 
I20260709 12:16:54.916430 26651 raft_consensus.cc:2804] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:54.916901 26651 raft_consensus.cc:697] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e [term 2 LEADER]: Becoming Leader. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Running, Role: LEADER
I20260709 12:16:54.917654 26651 consensus_queue.cc:237] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:54.925737 26239 catalog_manager.cc:5697] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e reported cstate change: term changed from 1 to 2, leader changed from f1991907f700454badc2971ecf14911d (127.11.13.129) to cc1c2a51ec344ca28260030593d6563e (127.11.13.130). New cstate: current_term: 2 leader_uuid: "cc1c2a51ec344ca28260030593d6563e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:55.005870 26651 consensus_queue.cc:1048] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:55.017894 26682 consensus_queue.cc:1048] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:55.049968 26682 raft_consensus.cc:993] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e: : Instructing follower f1991907f700454badc2971ecf14911d to start an election
I20260709 12:16:55.050593 26651 raft_consensus.cc:1081] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Signalling peer f1991907f700454badc2971ecf14911d to start an election
I20260709 12:16:55.053757 26344 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "bcb7c3cc69b5458db189e1449a4196c8"
dest_uuid: "f1991907f700454badc2971ecf14911d"
 from {username='slave'} at 127.0.0.1:60306
I20260709 12:16:55.054670 26344 raft_consensus.cc:493] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:55.055915 26344 raft_consensus.cc:3060] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:55.061112 26344 raft_consensus.cc:515] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:55.063186 26344 leader_election.cc:290] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 2 election: Requested vote from peers cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:55.064168 26419 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb7c3cc69b5458db189e1449a4196c8" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:55.064355 26491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bcb7c3cc69b5458db189e1449a4196c8" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:55.064812 26419 raft_consensus.cc:3055] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:55.065053 26491 raft_consensus.cc:3060] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:55.065192 26419 raft_consensus.cc:740] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Running, Role: LEADER
I20260709 12:16:55.066284 26419 consensus_queue.cc:260] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:55.067281 26419 raft_consensus.cc:3060] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:55.070621 26491 raft_consensus.cc:2468] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 2.
I20260709 12:16:55.072170 26308 leader_election.cc:304] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:55.072980 26628 raft_consensus.cc:2804] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:55.073462 26628 raft_consensus.cc:697] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Becoming Leader. State: Replica: f1991907f700454badc2971ecf14911d, State: Running, Role: LEADER
I20260709 12:16:55.073560 26419 raft_consensus.cc:2468] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 2.
I20260709 12:16:55.074409 26628 consensus_queue.cc:237] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:55.082149 26687 consensus_queue.cc:1048] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:55.083359 26240 catalog_manager.cc:5697] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d reported cstate change: term changed from 1 to 2, leader changed from cc1c2a51ec344ca28260030593d6563e (127.11.13.130) to f1991907f700454badc2971ecf14911d (127.11.13.129). New cstate: current_term: 2 leader_uuid: "f1991907f700454badc2971ecf14911d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:55.091048 26688 raft_consensus.cc:993] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632: : Instructing follower cc1c2a51ec344ca28260030593d6563e to start an election
I20260709 12:16:55.091667 26539 raft_consensus.cc:1081] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Signalling peer cc1c2a51ec344ca28260030593d6563e to start an election
I20260709 12:16:55.094357 26417 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "224667a8ff894a64a8a95dac8edeeff6"
dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
 from {username='slave'} at 127.0.0.1:55488
I20260709 12:16:55.095053 26417 raft_consensus.cc:493] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:55.095464 26417 raft_consensus.cc:3060] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:55.100744 26687 consensus_queue.cc:1048] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:55.105059 26417 raft_consensus.cc:515] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:55.106325 26419 raft_consensus.cc:1240] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Rejecting Update request from peer a9c038faa7aa4de680db447a77ed8632 for earlier term 1. Current term is 2. Ops: []
I20260709 12:16:55.108812 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "224667a8ff894a64a8a95dac8edeeff6" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:55.109527 26687 consensus_queue.cc:1059] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:55.110548 26417 leader_election.cc:290] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 2 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:55.111078 26539 raft_consensus.cc:3055] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:55.111660 26539 raft_consensus.cc:740] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Running, Role: LEADER
I20260709 12:16:55.112499 26539 consensus_queue.cc:260] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:55.113134 26491 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "224667a8ff894a64a8a95dac8edeeff6" candidate_uuid: "cc1c2a51ec344ca28260030593d6563e" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:55.116448 26539 raft_consensus.cc:3060] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:55.126403 26491 raft_consensus.cc:2468] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc1c2a51ec344ca28260030593d6563e in term 2.
I20260709 12:16:55.128001 26383 leader_election.cc:304] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e; no voters: f1991907f700454badc2971ecf14911d
I20260709 12:16:55.128697 26612 raft_consensus.cc:2804] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:55.129164 26612 raft_consensus.cc:697] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e [term 2 LEADER]: Becoming Leader. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Running, Role: LEADER
I20260709 12:16:55.129822 26612 consensus_queue.cc:237] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:55.139096 26239 catalog_manager.cc:5697] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e reported cstate change: term changed from 1 to 2, leader changed from a9c038faa7aa4de680db447a77ed8632 (127.11.13.131) to cc1c2a51ec344ca28260030593d6563e (127.11.13.130). New cstate: current_term: 2 leader_uuid: "cc1c2a51ec344ca28260030593d6563e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:55.183706 26612 consensus_queue.cc:1048] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:55.184185 26539 consensus_queue.cc:1048] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:55.198375 26682 consensus_queue.cc:1048] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:16:55.205382 26688 consensus_queue.cc:1048] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:55.219107 26612 consensus_queue.cc:1048] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:55.221414 26612 consensus_queue.cc:1048] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:55.240899 26682 consensus_queue.cc:1048] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:55.251407 26649 raft_consensus.cc:993] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e: : Instructing follower f1991907f700454badc2971ecf14911d to start an election
I20260709 12:16:55.251900 26649 raft_consensus.cc:1081] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Signalling peer f1991907f700454badc2971ecf14911d to start an election
I20260709 12:16:55.255540 26343 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "18ad382f11764922a0010dd3e13478d1"
dest_uuid: "f1991907f700454badc2971ecf14911d"
 from {username='slave'} at 127.0.0.1:60306
I20260709 12:16:55.256222 26343 raft_consensus.cc:493] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:55.256639 26343 raft_consensus.cc:3060] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:55.264371 26649 consensus_queue.cc:1048] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:55.266309 26649 raft_consensus.cc:993] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e: : Instructing follower f1991907f700454badc2971ecf14911d to start an election
I20260709 12:16:55.267786 26690 raft_consensus.cc:1081] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Signalling peer f1991907f700454badc2971ecf14911d to start an election
I20260709 12:16:55.271662 26341 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f11300f991be4590bf37eec3e516d05c"
dest_uuid: "f1991907f700454badc2971ecf14911d"
 from {username='slave'} at 127.0.0.1:60306
I20260709 12:16:55.272321 26341 raft_consensus.cc:493] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:55.272675 26341 raft_consensus.cc:3060] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:55.277333 26343 raft_consensus.cc:515] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:55.281509 26417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad382f11764922a0010dd3e13478d1" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:55.282598 26417 raft_consensus.cc:3055] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:55.283089 26417 raft_consensus.cc:740] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Running, Role: LEADER
I20260709 12:16:55.284263 26417 consensus_queue.cc:260] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:55.285626 26417 raft_consensus.cc:3060] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:55.286422 26343 leader_election.cc:290] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 2 election: Requested vote from peers cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:55.287719 26493 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "18ad382f11764922a0010dd3e13478d1" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:55.288435 26493 raft_consensus.cc:3060] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:55.295414 26341 raft_consensus.cc:515] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:55.296887 26342 raft_consensus.cc:1240] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Rejecting Update request from peer cc1c2a51ec344ca28260030593d6563e for earlier term 1. Current term is 2. Ops: []
I20260709 12:16:55.297363 26417 raft_consensus.cc:2468] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 2.
I20260709 12:16:55.298676 26493 raft_consensus.cc:2468] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 2.
I20260709 12:16:55.300292 26308 leader_election.cc:304] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:55.302526 26682 consensus_queue.cc:1059] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:16:55.303556 26628 raft_consensus.cc:2804] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:55.303941 26682 raft_consensus.cc:3055] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:55.304159 26628 raft_consensus.cc:697] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Becoming Leader. State: Replica: f1991907f700454badc2971ecf14911d, State: Running, Role: LEADER
I20260709 12:16:55.304407 26682 raft_consensus.cc:740] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Running, Role: LEADER
I20260709 12:16:55.304589 26417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11300f991be4590bf37eec3e516d05c" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:55.302491 26341 leader_election.cc:290] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 2 election: Requested vote from peers cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:55.306288 26682 consensus_queue.cc:260] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:55.308037 26490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f11300f991be4590bf37eec3e516d05c" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:55.308795 26490 raft_consensus.cc:3060] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:55.309126 26682 raft_consensus.cc:3060] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:55.305650 26628 consensus_queue.cc:237] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:55.312047 26649 raft_consensus.cc:993] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e: : Instructing follower f1991907f700454badc2971ecf14911d to start an election
I20260709 12:16:55.312565 26612 raft_consensus.cc:1081] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Signalling peer f1991907f700454badc2971ecf14911d to start an election
I20260709 12:16:55.317986 26344 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ec84a53b3f294643b5da4d7e464931a9"
dest_uuid: "f1991907f700454badc2971ecf14911d"
 from {username='slave'} at 127.0.0.1:60306
I20260709 12:16:55.318658 26344 raft_consensus.cc:493] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:55.319048 26344 raft_consensus.cc:3060] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:55.322975 26490 raft_consensus.cc:2468] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 2.
I20260709 12:16:55.324575 26308 leader_election.cc:304] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:55.325698 26628 raft_consensus.cc:2804] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:55.326099 26417 raft_consensus.cc:2468] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 2.
I20260709 12:16:55.326303 26628 raft_consensus.cc:697] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Becoming Leader. State: Replica: f1991907f700454badc2971ecf14911d, State: Running, Role: LEADER
I20260709 12:16:55.326912 26344 raft_consensus.cc:515] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:55.327281 26628 consensus_queue.cc:237] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:55.329000 26344 leader_election.cc:290] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 2 election: Requested vote from peers cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281), a9c038faa7aa4de680db447a77ed8632 (127.11.13.131:38463)
I20260709 12:16:55.329854 26417 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec84a53b3f294643b5da4d7e464931a9" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:55.330578 26417 raft_consensus.cc:3055] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:55.330968 26417 raft_consensus.cc:740] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Running, Role: LEADER
I20260709 12:16:55.331585 26490 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec84a53b3f294643b5da4d7e464931a9" candidate_uuid: "f1991907f700454badc2971ecf14911d" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
I20260709 12:16:55.332223 26490 raft_consensus.cc:3060] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:55.331875 26417 consensus_queue.cc:260] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:55.334657 26417 raft_consensus.cc:3060] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:55.334903 26493 raft_consensus.cc:1275] T d3328229dda9476c97777ff8dc69c435 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Refusing update from remote peer cc1c2a51ec344ca28260030593d6563e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:55.336122 26240 catalog_manager.cc:5697] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d reported cstate change: term changed from 1 to 2, leader changed from cc1c2a51ec344ca28260030593d6563e (127.11.13.130) to f1991907f700454badc2971ecf14911d (127.11.13.129). New cstate: current_term: 2 leader_uuid: "f1991907f700454badc2971ecf14911d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:55.336850 26682 consensus_queue.cc:1048] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:55.340143 26490 raft_consensus.cc:2468] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 2.
I20260709 12:16:55.342161 26308 leader_election.cc:304] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:55.344229 26672 raft_consensus.cc:2804] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:55.349797 26344 raft_consensus.cc:1275] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Refusing update from remote peer cc1c2a51ec344ca28260030593d6563e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:55.353890 26649 consensus_queue.cc:1048] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:55.353819 26417 raft_consensus.cc:2468] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1991907f700454badc2971ecf14911d in term 2.
I20260709 12:16:55.354506 26672 raft_consensus.cc:697] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Becoming Leader. State: Replica: f1991907f700454badc2971ecf14911d, State: Running, Role: LEADER
I20260709 12:16:55.355976 26672 consensus_queue.cc:237] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:55.369324 26239 catalog_manager.cc:5697] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d reported cstate change: term changed from 1 to 2, leader changed from cc1c2a51ec344ca28260030593d6563e (127.11.13.130) to f1991907f700454badc2971ecf14911d (127.11.13.129). New cstate: current_term: 2 leader_uuid: "f1991907f700454badc2971ecf14911d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:55.370899 26239 catalog_manager.cc:5697] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d reported cstate change: term changed from 1 to 2, leader changed from cc1c2a51ec344ca28260030593d6563e (127.11.13.130) to f1991907f700454badc2971ecf14911d (127.11.13.129). New cstate: current_term: 2 leader_uuid: "f1991907f700454badc2971ecf14911d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: UNKNOWN } } }
I20260709 12:16:55.542861 26417 raft_consensus.cc:1275] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Refusing update from remote peer f1991907f700454badc2971ecf14911d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:55.544831 26671 consensus_queue.cc:1048] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:55.556792 26490 raft_consensus.cc:1275] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Refusing update from remote peer f1991907f700454badc2971ecf14911d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:55.558557 26672 consensus_queue.cc:1048] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:55.620615 26344 raft_consensus.cc:3060] T 224667a8ff894a64a8a95dac8edeeff6 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:55.626235 26344 raft_consensus.cc:1275] T 224667a8ff894a64a8a95dac8edeeff6 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Refusing update from remote peer cc1c2a51ec344ca28260030593d6563e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:55.628022 26649 consensus_queue.cc:1048] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:55.639971 26490 raft_consensus.cc:1275] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Refusing update from remote peer cc1c2a51ec344ca28260030593d6563e: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:55.641619 26649 consensus_queue.cc:1048] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:55.692804 26419 raft_consensus.cc:1275] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Refusing update from remote peer f1991907f700454badc2971ecf14911d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:55.694250 26671 consensus_queue.cc:1048] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:55.702817 26490 raft_consensus.cc:1275] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Refusing update from remote peer f1991907f700454badc2971ecf14911d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:55.704604 26713 consensus_queue.cc:1048] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:55.736552 26490 raft_consensus.cc:1275] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Refusing update from remote peer f1991907f700454badc2971ecf14911d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:55.738632 26713 consensus_queue.cc:1048] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:55.752123 26419 raft_consensus.cc:1275] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Refusing update from remote peer f1991907f700454badc2971ecf14911d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:55.753762 26672 consensus_queue.cc:1048] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:55.928891 26419 raft_consensus.cc:1275] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Refusing update from remote peer f1991907f700454badc2971ecf14911d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:55.930284 26672 consensus_queue.cc:1048] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:16:55.938485 26490 raft_consensus.cc:1275] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Refusing update from remote peer f1991907f700454badc2971ecf14911d: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:16:55.940722 26713 consensus_queue.cc:1048] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:16:57.913092 26240 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40376:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:16:57.936774 26240 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:40376:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:16:57.982228 11318 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
W20260709 12:16:58.187669 26739 log.cc:927] Time spent T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e: Append to log took a long time: real 0.060s	user 0.001s	sys 0.000s
W20260709 12:16:58.220328 26385 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:37117, user_credentials={real_user=slave}} blocked reactor thread for 62199.4us
I20260709 12:16:58.256007 26767 tablet.cc:1724] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.237035 26766 tablet.cc:1724] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:16:58.260557 26445 tablet_mm_ops.cc:139] Rowset compaction is disabled (check --enable_rowset_compaction and disable_compaction in extra_config for tablet:d3328229dda9476c97777ff8dc69c435)
I20260709 12:16:58.312039 26771 tablet.cc:1724] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.367246 26415 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0cee69f616244c02b6c01f0f7cb1465b"
dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
mode: GRACEFUL
new_leader_uuid: "a9c038faa7aa4de680db447a77ed8632"
 from {username='slave'} at 127.0.0.1:56244
I20260709 12:16:58.371403 26415 raft_consensus.cc:606] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Received request to transfer leadership to TS a9c038faa7aa4de680db447a77ed8632
W20260709 12:16:58.388895 26370 tablet_mm_ops.cc:304] Major delta compaction is disabled (check --enable_major_delta_compaction and disable_compaction in extra_config for tablet:bcb7c3cc69b5458db189e1449a4196c8)
W20260709 12:16:58.410305 26370 tablet_mm_ops.cc:220] Minor delta compaction is disabled (check --enable_minor_delta_compaction and disable_compaction in extra_config for tablet:bcb7c3cc69b5458db189e1449a4196c8)
I20260709 12:16:58.388689 26772 tablet.cc:1724] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.447979 26782 tablet.cc:1724] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.396639 26773 tablet.cc:1724] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.444391 26777 tablet.cc:1724] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.481714 26783 tablet.cc:1724] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.495190 26784 tablet.cc:1724] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.518016 26730 raft_consensus.cc:993] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e: : Instructing follower a9c038faa7aa4de680db447a77ed8632 to start an election
I20260709 12:16:58.518739 26730 raft_consensus.cc:1081] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Signalling peer a9c038faa7aa4de680db447a77ed8632 to start an election
I20260709 12:16:58.521075 26491 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0cee69f616244c02b6c01f0f7cb1465b"
dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
 from {username='slave'} at 127.0.0.1:48078
I20260709 12:16:58.521821 26491 raft_consensus.cc:493] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:58.522334 26491 raft_consensus.cc:3060] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:58.530627 26778 tablet.cc:1724] T d3328229dda9476c97777ff8dc69c435 P a9c038faa7aa4de680db447a77ed8632: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.515558 26773 tablet.cc:1724] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.542645 26787 tablet.cc:1724] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:16:58.545894 26779 log.cc:927] Time spent T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d: Append to log took a long time: real 0.113s	user 0.001s	sys 0.000s
I20260709 12:16:58.546002 26767 tablet.cc:1724] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.548353 26771 tablet.cc:1724] T 0cee69f616244c02b6c01f0f7cb1465b P f1991907f700454badc2971ecf14911d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.486009 26766 tablet.cc:1724] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.561328 26771 tablet.cc:1724] T 224667a8ff894a64a8a95dac8edeeff6 P f1991907f700454badc2971ecf14911d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.568049 26783 tablet.cc:1724] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.565611 26788 tablet.cc:1724] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.573508 26772 tablet.cc:1724] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.589787 26777 tablet.cc:1724] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.598791 26766 tablet.cc:1724] T 8ebf90a9375a4a2798dad2c4de791d92 P cc1c2a51ec344ca28260030593d6563e: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.606802 26782 tablet.cc:1724] T 8ebf90a9375a4a2798dad2c4de791d92 P f1991907f700454badc2971ecf14911d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.607668 26342 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "ec84a53b3f294643b5da4d7e464931a9"
dest_uuid: "f1991907f700454badc2971ecf14911d"
mode: GRACEFUL
new_leader_uuid: "a9c038faa7aa4de680db447a77ed8632"
 from {username='slave'} at 127.0.0.1:39622
I20260709 12:16:58.621764 26342 raft_consensus.cc:606] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Received request to transfer leadership to TS a9c038faa7aa4de680db447a77ed8632
I20260709 12:16:58.603557 26784 tablet.cc:1724] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.610493 26491 raft_consensus.cc:515] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:58.624845 11318 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 2
I20260709 12:16:58.625696 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:58.625469 26787 tablet.cc:1724] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.651041 26342 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "0f82565aabb44f9985dfd97ae33d8f3f"
dest_uuid: "f1991907f700454badc2971ecf14911d"
mode: GRACEFUL
new_leader_uuid: "a9c038faa7aa4de680db447a77ed8632"
 from {username='slave'} at 127.0.0.1:39622
I20260709 12:16:58.651943 26342 raft_consensus.cc:606] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Received request to transfer leadership to TS a9c038faa7aa4de680db447a77ed8632
I20260709 12:16:58.653944 26773 tablet.cc:1724] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.654933 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cee69f616244c02b6c01f0f7cb1465b" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:58.655690 26342 raft_consensus.cc:3060] T 0cee69f616244c02b6c01f0f7cb1465b P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:58.660141 26778 tablet.cc:1724] T b97814c8781b43a08eb809dd0441fbae P a9c038faa7aa4de680db447a77ed8632: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.664299 26418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0cee69f616244c02b6c01f0f7cb1465b" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: true dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:58.669657 26418 raft_consensus.cc:3055] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:16:58.679028 26418 raft_consensus.cc:740] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Running, Role: LEADER
I20260709 12:16:58.680007 26418 consensus_queue.cc:260] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:58.681396 26418 raft_consensus.cc:3060] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:16:58.670743 26415 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "6956538aee40492683999ec2274993a4"
dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
mode: GRACEFUL
new_leader_uuid: "a9c038faa7aa4de680db447a77ed8632"
 from {username='slave'} at 127.0.0.1:56244
I20260709 12:16:58.686957 26415 raft_consensus.cc:606] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [term 2 LEADER]: Received request to transfer leadership to TS a9c038faa7aa4de680db447a77ed8632
I20260709 12:16:58.689361 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:16:58.672472 26491 leader_election.cc:290] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 2 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:58.690047 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:58.675959 26342 raft_consensus.cc:2468] T 0cee69f616244c02b6c01f0f7cb1465b P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 2.
I20260709 12:16:58.693317 26459 leader_election.cc:304] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:58.694396 26687 raft_consensus.cc:2804] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:16:58.694914 26687 raft_consensus.cc:697] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [term 2 LEADER]: Becoming Leader. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Running, Role: LEADER
I20260709 12:16:58.695820 26687 consensus_queue.cc:237] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:58.669602 26788 tablet.cc:1724] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:16:58.711002 26418 raft_consensus.cc:2468] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 2.
I20260709 12:16:58.723233 26241 catalog_manager.cc:6242] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f: soft_deleted_table [id=309356b3042343b797534aa2fc120450] alter complete (version 1)
I20260709 12:16:58.734907 26242 catalog_manager.cc:5697] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 reported cstate change: term changed from 1 to 2, leader changed from cc1c2a51ec344ca28260030593d6563e (127.11.13.130) to a9c038faa7aa4de680db447a77ed8632 (127.11.13.131). New cstate: current_term: 2 leader_uuid: "a9c038faa7aa4de680db447a77ed8632" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:58.807595 26716 raft_consensus.cc:993] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d: : Instructing follower a9c038faa7aa4de680db447a77ed8632 to start an election
I20260709 12:16:58.808440 26744 raft_consensus.cc:1081] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Signalling peer a9c038faa7aa4de680db447a77ed8632 to start an election
I20260709 12:16:58.810691 26491 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "ec84a53b3f294643b5da4d7e464931a9"
dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
 from {username='slave'} at 127.0.0.1:48072
I20260709 12:16:58.811312 26491 raft_consensus.cc:493] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:58.811661 26491 raft_consensus.cc:3060] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:58.817948 26491 raft_consensus.cc:515] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:58.819808 26491 leader_election.cc:290] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 3 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:58.820984 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec84a53b3f294643b5da4d7e464931a9" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:58.821658 26342 raft_consensus.cc:3055] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:58.821936 26418 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "ec84a53b3f294643b5da4d7e464931a9" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 3 candidate_status { last_received { term: 2 index: 3 } } ignore_live_leader: true dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:58.821992 26342 raft_consensus.cc:740] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Running, Role: LEADER
I20260709 12:16:58.822610 26418 raft_consensus.cc:3060] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:58.823249 26342 consensus_queue.cc:260] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:58.824625 26342 raft_consensus.cc:3060] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:58.829614 26418 raft_consensus.cc:2468] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 3.
I20260709 12:16:58.831182 26459 leader_election.cc:304] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e; no voters: 
I20260709 12:16:58.832304 26687 raft_consensus.cc:2804] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:58.832785 26687 raft_consensus.cc:697] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [term 3 LEADER]: Becoming Leader. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Running, Role: LEADER
I20260709 12:16:58.833456 26342 raft_consensus.cc:2468] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 3.
I20260709 12:16:58.833663 26687 consensus_queue.cc:237] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:58.842027 26241 catalog_manager.cc:5697] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 reported cstate change: term changed from 2 to 3, leader changed from f1991907f700454badc2971ecf14911d (127.11.13.129) to a9c038faa7aa4de680db447a77ed8632 (127.11.13.131). New cstate: current_term: 3 leader_uuid: "a9c038faa7aa4de680db447a77ed8632" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:58.874850 26730 raft_consensus.cc:993] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e: : Instructing follower a9c038faa7aa4de680db447a77ed8632 to start an election
I20260709 12:16:58.875480 26749 raft_consensus.cc:1081] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [term 2 LEADER]: Signalling peer a9c038faa7aa4de680db447a77ed8632 to start an election
I20260709 12:16:58.877207 26491 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "6956538aee40492683999ec2274993a4"
dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
 from {username='slave'} at 127.0.0.1:48078
I20260709 12:16:58.877838 26491 raft_consensus.cc:493] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:58.878252 26491 raft_consensus.cc:3060] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:58.887001 26491 raft_consensus.cc:515] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:58.889357 26491 leader_election.cc:290] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 3 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:58.890712 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6956538aee40492683999ec2274993a4" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:58.891376 26342 raft_consensus.cc:3060] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:58.891321 26416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "6956538aee40492683999ec2274993a4" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:58.892321 26416 raft_consensus.cc:3055] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:58.892678 26416 raft_consensus.cc:740] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [term 2 LEADER]: Becoming Follower/Learner. State: Replica: cc1c2a51ec344ca28260030593d6563e, State: Running, Role: LEADER
I20260709 12:16:58.893671 26416 consensus_queue.cc:260] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:58.895262 26416 raft_consensus.cc:3060] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:58.897428 26342 raft_consensus.cc:2468] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 3.
I20260709 12:16:58.898661 26459 leader_election.cc:304] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, f1991907f700454badc2971ecf14911d; no voters: 
I20260709 12:16:58.899370 26687 raft_consensus.cc:2804] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:58.899847 26687 raft_consensus.cc:697] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 3 LEADER]: Becoming Leader. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Running, Role: LEADER
I20260709 12:16:58.900735 26687 consensus_queue.cc:237] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:58.904734 26416 raft_consensus.cc:2468] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 3.
I20260709 12:16:58.917657 26241 catalog_manager.cc:5697] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 reported cstate change: term changed from 2 to 3, leader changed from cc1c2a51ec344ca28260030593d6563e (127.11.13.130) to a9c038faa7aa4de680db447a77ed8632 (127.11.13.131). New cstate: current_term: 3 leader_uuid: "a9c038faa7aa4de680db447a77ed8632" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:59.102480 26342 raft_consensus.cc:1275] T 0cee69f616244c02b6c01f0f7cb1465b P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Refusing update from remote peer a9c038faa7aa4de680db447a77ed8632: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:16:59.104382 26687 consensus_queue.cc:1048] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:59.123682 26416 raft_consensus.cc:1275] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Refusing update from remote peer a9c038faa7aa4de680db447a77ed8632: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260709 12:16:59.132157 26687 consensus_queue.cc:1048] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:59.230712 26716 raft_consensus.cc:993] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d: : Instructing follower a9c038faa7aa4de680db447a77ed8632 to start an election
I20260709 12:16:59.231323 26716 raft_consensus.cc:1081] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Signalling peer a9c038faa7aa4de680db447a77ed8632 to start an election
I20260709 12:16:59.233821 26491 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "0f82565aabb44f9985dfd97ae33d8f3f"
dest_uuid: "a9c038faa7aa4de680db447a77ed8632"
 from {username='slave'} at 127.0.0.1:48072
I20260709 12:16:59.234715 26491 raft_consensus.cc:493] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:16:59.235137 26491 raft_consensus.cc:3060] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:59.241266 26491 raft_consensus.cc:515] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:59.243460 26491 leader_election.cc:290] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 3 election: Requested vote from peers f1991907f700454badc2971ecf14911d (127.11.13.129:37117), cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281)
I20260709 12:16:59.245005 26416 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f82565aabb44f9985dfd97ae33d8f3f" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "cc1c2a51ec344ca28260030593d6563e"
I20260709 12:16:59.245005 26342 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "0f82565aabb44f9985dfd97ae33d8f3f" candidate_uuid: "a9c038faa7aa4de680db447a77ed8632" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "f1991907f700454badc2971ecf14911d"
I20260709 12:16:59.246232 26342 raft_consensus.cc:3055] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:16:59.246150 26416 raft_consensus.cc:3060] T 0f82565aabb44f9985dfd97ae33d8f3f P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:59.246711 26342 raft_consensus.cc:740] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Becoming Follower/Learner. State: Replica: f1991907f700454badc2971ecf14911d, State: Running, Role: LEADER
I20260709 12:16:59.247887 26342 consensus_queue.cc:260] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:59.249487 26342 raft_consensus.cc:3060] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:16:59.255067 26416 raft_consensus.cc:2468] T 0f82565aabb44f9985dfd97ae33d8f3f P cc1c2a51ec344ca28260030593d6563e [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 3.
I20260709 12:16:59.256698 26459 leader_election.cc:304] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9c038faa7aa4de680db447a77ed8632, cc1c2a51ec344ca28260030593d6563e; no voters: 
I20260709 12:16:59.256826 26342 raft_consensus.cc:2468] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a9c038faa7aa4de680db447a77ed8632 in term 3.
I20260709 12:16:59.257797 26797 raft_consensus.cc:2804] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:16:59.258383 26797 raft_consensus.cc:697] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 3 LEADER]: Becoming Leader. State: Replica: a9c038faa7aa4de680db447a77ed8632, State: Running, Role: LEADER
I20260709 12:16:59.259474 26797 consensus_queue.cc:237] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } }
I20260709 12:16:59.265537 26342 raft_consensus.cc:1275] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [term 3 FOLLOWER]: Refusing update from remote peer a9c038faa7aa4de680db447a77ed8632: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:16:59.267967 26797 consensus_queue.cc:1048] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260709 12:16:59.270524 26242 catalog_manager.cc:5697] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 reported cstate change: term changed from 2 to 3, leader changed from f1991907f700454badc2971ecf14911d (127.11.13.129) to a9c038faa7aa4de680db447a77ed8632 (127.11.13.131). New cstate: current_term: 3 leader_uuid: "a9c038faa7aa4de680db447a77ed8632" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9c038faa7aa4de680db447a77ed8632" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 38463 } health_report { overall_health: HEALTHY } } }
I20260709 12:16:59.279301 26416 raft_consensus.cc:1275] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 3 FOLLOWER]: Refusing update from remote peer a9c038faa7aa4de680db447a77ed8632: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260709 12:16:59.281570 26798 consensus_queue.cc:1048] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260709 12:16:59.340765 26416 raft_consensus.cc:1275] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [term 3 FOLLOWER]: Refusing update from remote peer a9c038faa7aa4de680db447a77ed8632: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:59.342341 26797 consensus_queue.cc:1048] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:16:59.355439 26342 raft_consensus.cc:1275] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d [term 3 FOLLOWER]: Refusing update from remote peer a9c038faa7aa4de680db447a77ed8632: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:59.357107 26797 consensus_queue.cc:1048] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:59.691417 11318 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:16:59.698671 11318 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:16:59.699270 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:16:59.708325 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:16:59.708992 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:16:59.710600 26418 raft_consensus.cc:1275] T 0f82565aabb44f9985dfd97ae33d8f3f P cc1c2a51ec344ca28260030593d6563e [term 3 FOLLOWER]: Refusing update from remote peer a9c038faa7aa4de680db447a77ed8632: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:59.712450 26797 consensus_queue.cc:1048] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc1c2a51ec344ca28260030593d6563e" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 38281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:16:59.713117 26342 raft_consensus.cc:1275] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 3 FOLLOWER]: Refusing update from remote peer a9c038faa7aa4de680db447a77ed8632: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:16:59.714769 26797 consensus_queue.cc:1048] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1991907f700454badc2971ecf14911d" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 37117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:00.710199 11318 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:17:00.716536 11318 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:17:00.717149 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:00.723071 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:00.723611 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:01.724903 11318 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:17:01.730358 11318 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:17:01.730862 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:01.736065 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:01.736553 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:02.738433 11318 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:17:02.749857 11318 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:17:02.750447 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:02.757627 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:02.758150 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:03.759467 11318 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:17:03.765197 11318 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:17:03.765751 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:03.772697 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:03.773198 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:04.774513 11318 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:17:04.780676 11318 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:17:04.781339 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:04.788751 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:04.789357 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:05.791144 11318 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:17:05.797176 11318 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:17:05.797734 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:05.805076 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:05.805605 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:06.806842 11318 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:17:06.812029 11318 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:17:06.812466 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:06.818702 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:06.819298 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:07.820760 11318 auto_leader_rebalancer.cc:141] leader rebalance for table soft_deleted_table
I20260709 12:17:07.828473 11318 auto_leader_rebalancer.cc:440] table: soft_deleted_table, leader rebalance finish, leader transfer count: 0
I20260709 12:17:07.828955 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:07.835196 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:07.835609 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:08.908126 11318 tablet_server.cc:179] TabletServer@127.11.13.129:0 shutting down...
W20260709 12:17:08.919366 26385 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 8 similar messages]
W20260709 12:17:08.927322 26385 consensus_peers.cc:597] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e -> Peer f1991907f700454badc2971ecf14911d (127.11.13.129:37117): Couldn't send request to peer f1991907f700454badc2971ecf14911d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:08.961680 26385 consensus_peers.cc:597] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e -> Peer f1991907f700454badc2971ecf14911d (127.11.13.129:37117): Couldn't send request to peer f1991907f700454badc2971ecf14911d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:08.968740 26459 consensus_peers.cc:597] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 -> Peer f1991907f700454badc2971ecf14911d (127.11.13.129:37117): Couldn't send request to peer f1991907f700454badc2971ecf14911d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:08.972895 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:08.973868 11318 tablet_replica.cc:333] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d: stopping tablet replica
I20260709 12:17:08.975437 11318 raft_consensus.cc:2243] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:08.977145 11318 raft_consensus.cc:2272] T f11300f991be4590bf37eec3e516d05c P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:08.980098 11318 tablet_replica.cc:333] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d: stopping tablet replica
I20260709 12:17:08.980829 11318 raft_consensus.cc:2243] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:08.981485 11318 raft_consensus.cc:2272] T d3328229dda9476c97777ff8dc69c435 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:08.984494 11318 tablet_replica.cc:333] T 224667a8ff894a64a8a95dac8edeeff6 P f1991907f700454badc2971ecf14911d: stopping tablet replica
I20260709 12:17:08.985386 11318 raft_consensus.cc:2243] T 224667a8ff894a64a8a95dac8edeeff6 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:08.986037 11318 raft_consensus.cc:2272] T 224667a8ff894a64a8a95dac8edeeff6 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:08.988777 11318 tablet_replica.cc:333] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d: stopping tablet replica
I20260709 12:17:08.989465 11318 raft_consensus.cc:2243] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:08.990290 11318 raft_consensus.cc:2272] T ec84a53b3f294643b5da4d7e464931a9 P f1991907f700454badc2971ecf14911d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:08.993672 11318 tablet_replica.cc:333] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d: stopping tablet replica
I20260709 12:17:08.994541 11318 raft_consensus.cc:2243] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:08.995848 11318 raft_consensus.cc:2272] T bcb7c3cc69b5458db189e1449a4196c8 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:08.997561 26385 consensus_peers.cc:597] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e -> Peer f1991907f700454badc2971ecf14911d (127.11.13.129:37117): Couldn't send request to peer f1991907f700454badc2971ecf14911d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:08.999105 11318 tablet_replica.cc:333] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d: stopping tablet replica
I20260709 12:17:09.000068 11318 raft_consensus.cc:2243] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.000806 11318 raft_consensus.cc:2272] T 0f82565aabb44f9985dfd97ae33d8f3f P f1991907f700454badc2971ecf14911d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.004781 11318 tablet_replica.cc:333] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d: stopping tablet replica
I20260709 12:17:09.005542 11318 raft_consensus.cc:2243] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.006177 11318 raft_consensus.cc:2272] T b3488dcfa84d4a699bcc7a4a3c827577 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.010598 11318 tablet_replica.cc:333] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d: stopping tablet replica
I20260709 12:17:09.011554 11318 raft_consensus.cc:2243] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:09.013062 11318 raft_consensus.cc:2272] T 3f981b08993f4a4aa6778078403e1004 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.016037 11318 tablet_replica.cc:333] T 9648db75c5cb45a681230d22c473e29f P f1991907f700454badc2971ecf14911d: stopping tablet replica
I20260709 12:17:09.016674 11318 raft_consensus.cc:2243] T 9648db75c5cb45a681230d22c473e29f P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.017164 11318 raft_consensus.cc:2272] T 9648db75c5cb45a681230d22c473e29f P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.019964 11318 tablet_replica.cc:333] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d: stopping tablet replica
I20260709 12:17:09.020573 11318 raft_consensus.cc:2243] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:09.021692 11318 raft_consensus.cc:2272] T 18ad382f11764922a0010dd3e13478d1 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.025642 11318 tablet_replica.cc:333] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d: stopping tablet replica
I20260709 12:17:09.026324 11318 raft_consensus.cc:2243] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.026973 11318 raft_consensus.cc:2272] T 6956538aee40492683999ec2274993a4 P f1991907f700454badc2971ecf14911d [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.029641 11318 tablet_replica.cc:333] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d: stopping tablet replica
I20260709 12:17:09.030295 11318 raft_consensus.cc:2243] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.030895 11318 raft_consensus.cc:2272] T 21d01b3e50754bb38a24d1ea383a61f9 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.034019 11318 tablet_replica.cc:333] T 0cee69f616244c02b6c01f0f7cb1465b P f1991907f700454badc2971ecf14911d: stopping tablet replica
I20260709 12:17:09.035028 11318 raft_consensus.cc:2243] T 0cee69f616244c02b6c01f0f7cb1465b P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.035794 11318 raft_consensus.cc:2272] T 0cee69f616244c02b6c01f0f7cb1465b P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.039003 11318 tablet_replica.cc:333] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d: stopping tablet replica
I20260709 12:17:09.039721 11318 raft_consensus.cc:2243] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.040387 11318 raft_consensus.cc:2272] T b97814c8781b43a08eb809dd0441fbae P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.043562 11318 tablet_replica.cc:333] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d: stopping tablet replica
I20260709 12:17:09.044579 11318 raft_consensus.cc:2243] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:09.045792 11318 raft_consensus.cc:2272] T 0859d4d4116a463f8e3a00a7737862ea P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:09.047998 26459 consensus_peers.cc:597] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 -> Peer f1991907f700454badc2971ecf14911d (127.11.13.129:37117): Couldn't send request to peer f1991907f700454badc2971ecf14911d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:09.049468 11318 tablet_replica.cc:333] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d: stopping tablet replica
I20260709 12:17:09.050081 11318 raft_consensus.cc:2243] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:09.051321 11318 raft_consensus.cc:2272] T 15f1cf7b68d6463fbd60efc8c4576fc8 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.054766 11318 tablet_replica.cc:333] T 8ebf90a9375a4a2798dad2c4de791d92 P f1991907f700454badc2971ecf14911d: stopping tablet replica
I20260709 12:17:09.055353 11318 raft_consensus.cc:2243] T 8ebf90a9375a4a2798dad2c4de791d92 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.056058 11318 raft_consensus.cc:2272] T 8ebf90a9375a4a2798dad2c4de791d92 P f1991907f700454badc2971ecf14911d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.059007 11318 tablet_replica.cc:333] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d: stopping tablet replica
I20260709 12:17:09.059531 11318 raft_consensus.cc:2243] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.060065 11318 raft_consensus.cc:2272] T 3b5f70527bbc4070880acc6ef899ddb4 P f1991907f700454badc2971ecf14911d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.117139 11318 tablet_server.cc:196] TabletServer@127.11.13.129:0 shutdown complete.
I20260709 12:17:09.172881 11318 tablet_server.cc:179] TabletServer@127.11.13.130:0 shutting down...
W20260709 12:17:09.203696 26459 consensus_peers.cc:597] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 -> Peer cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281): Couldn't send request to peer cc1c2a51ec344ca28260030593d6563e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:09.215953 26385 consensus_peers.cc:597] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e -> Peer f1991907f700454badc2971ecf14911d (127.11.13.129:37117): Couldn't send request to peer f1991907f700454badc2971ecf14911d. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:37117: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:09.219648 26459 consensus_peers.cc:597] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 -> Peer f1991907f700454badc2971ecf14911d (127.11.13.129:37117): Couldn't send request to peer f1991907f700454badc2971ecf14911d. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:37117: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:09.230769 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:09.232113 11318 tablet_replica.cc:333] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e: stopping tablet replica
I20260709 12:17:09.233198 11318 raft_consensus.cc:2243] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.234102 11318 raft_consensus.cc:2272] T f11300f991be4590bf37eec3e516d05c P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.237016 11318 tablet_replica.cc:333] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e: stopping tablet replica
I20260709 12:17:09.237730 11318 raft_consensus.cc:2243] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:09.238780 11318 raft_consensus.cc:2272] T d3328229dda9476c97777ff8dc69c435 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.241441 11318 tablet_replica.cc:333] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e: stopping tablet replica
I20260709 12:17:09.242281 11318 raft_consensus.cc:2243] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:09.243314 11318 raft_consensus.cc:2272] T 224667a8ff894a64a8a95dac8edeeff6 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.246013 11318 tablet_replica.cc:333] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e: stopping tablet replica
I20260709 12:17:09.246809 11318 raft_consensus.cc:2243] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.247490 11318 raft_consensus.cc:2272] T ec84a53b3f294643b5da4d7e464931a9 P cc1c2a51ec344ca28260030593d6563e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.250077 11318 tablet_replica.cc:333] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e: stopping tablet replica
I20260709 12:17:09.250850 11318 raft_consensus.cc:2243] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.251500 11318 raft_consensus.cc:2272] T bcb7c3cc69b5458db189e1449a4196c8 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.254097 11318 tablet_replica.cc:333] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e: stopping tablet replica
I20260709 12:17:09.254793 11318 raft_consensus.cc:2243] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:09.255769 11318 raft_consensus.cc:2272] T 21d01b3e50754bb38a24d1ea383a61f9 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.258617 11318 tablet_replica.cc:333] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e: stopping tablet replica
I20260709 12:17:09.259512 11318 raft_consensus.cc:2243] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:09.260528 11318 raft_consensus.cc:2272] T b3488dcfa84d4a699bcc7a4a3c827577 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.263880 11318 tablet_replica.cc:333] T 0f82565aabb44f9985dfd97ae33d8f3f P cc1c2a51ec344ca28260030593d6563e: stopping tablet replica
I20260709 12:17:09.264596 11318 raft_consensus.cc:2243] T 0f82565aabb44f9985dfd97ae33d8f3f P cc1c2a51ec344ca28260030593d6563e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.265406 11318 raft_consensus.cc:2272] T 0f82565aabb44f9985dfd97ae33d8f3f P cc1c2a51ec344ca28260030593d6563e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.268862 11318 tablet_replica.cc:333] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e: stopping tablet replica
I20260709 12:17:09.269747 11318 raft_consensus.cc:2243] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.270615 11318 raft_consensus.cc:2272] T 18ad382f11764922a0010dd3e13478d1 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.274859 11318 tablet_replica.cc:333] T 9648db75c5cb45a681230d22c473e29f P cc1c2a51ec344ca28260030593d6563e: stopping tablet replica
I20260709 12:17:09.275617 11318 raft_consensus.cc:2243] T 9648db75c5cb45a681230d22c473e29f P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Raft consensus shutting down.
W20260709 12:17:09.275879 26459 consensus_peers.cc:597] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 -> Peer cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281): Couldn't send request to peer cc1c2a51ec344ca28260030593d6563e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:09.276229 11318 raft_consensus.cc:2272] T 9648db75c5cb45a681230d22c473e29f P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.280262 11318 tablet_replica.cc:333] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e: stopping tablet replica
I20260709 12:17:09.281154 11318 raft_consensus.cc:2243] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.281917 11318 raft_consensus.cc:2272] T 3f981b08993f4a4aa6778078403e1004 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.285200 11318 tablet_replica.cc:333] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e: stopping tablet replica
I20260709 12:17:09.285976 11318 raft_consensus.cc:2243] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.286590 11318 raft_consensus.cc:2272] T 0cee69f616244c02b6c01f0f7cb1465b P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.289093 11318 tablet_replica.cc:333] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e: stopping tablet replica
I20260709 12:17:09.289870 11318 raft_consensus.cc:2243] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:09.290974 11318 raft_consensus.cc:2272] T b97814c8781b43a08eb809dd0441fbae P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.293514 11318 tablet_replica.cc:333] T 0859d4d4116a463f8e3a00a7737862ea P cc1c2a51ec344ca28260030593d6563e: stopping tablet replica
I20260709 12:17:09.294478 11318 raft_consensus.cc:2243] T 0859d4d4116a463f8e3a00a7737862ea P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.295022 11318 raft_consensus.cc:2272] T 0859d4d4116a463f8e3a00a7737862ea P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.297430 11318 tablet_replica.cc:333] T 15f1cf7b68d6463fbd60efc8c4576fc8 P cc1c2a51ec344ca28260030593d6563e: stopping tablet replica
I20260709 12:17:09.297966 11318 raft_consensus.cc:2243] T 15f1cf7b68d6463fbd60efc8c4576fc8 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.298540 11318 raft_consensus.cc:2272] T 15f1cf7b68d6463fbd60efc8c4576fc8 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.300874 11318 tablet_replica.cc:333] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e: stopping tablet replica
I20260709 12:17:09.301394 11318 raft_consensus.cc:2243] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.301908 11318 raft_consensus.cc:2272] T 6956538aee40492683999ec2274993a4 P cc1c2a51ec344ca28260030593d6563e [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.304613 11318 tablet_replica.cc:333] T 8ebf90a9375a4a2798dad2c4de791d92 P cc1c2a51ec344ca28260030593d6563e: stopping tablet replica
I20260709 12:17:09.305392 11318 raft_consensus.cc:2243] T 8ebf90a9375a4a2798dad2c4de791d92 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.306315 11318 raft_consensus.cc:2272] T 8ebf90a9375a4a2798dad2c4de791d92 P cc1c2a51ec344ca28260030593d6563e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.309739 11318 tablet_replica.cc:333] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e: stopping tablet replica
W20260709 12:17:09.310111 26459 consensus_peers.cc:597] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 -> Peer f1991907f700454badc2971ecf14911d (127.11.13.129:37117): Couldn't send request to peer f1991907f700454badc2971ecf14911d. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:37117: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:09.310494 11318 raft_consensus.cc:2243] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:09.311563 11318 raft_consensus.cc:2272] T 3b5f70527bbc4070880acc6ef899ddb4 P cc1c2a51ec344ca28260030593d6563e [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:09.329988 26459 consensus_peers.cc:597] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 -> Peer f1991907f700454badc2971ecf14911d (127.11.13.129:37117): Couldn't send request to peer f1991907f700454badc2971ecf14911d. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:37117: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:09.336436 26459 consensus_peers.cc:597] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 -> Peer cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281): Couldn't send request to peer cc1c2a51ec344ca28260030593d6563e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:09.365342 11318 tablet_server.cc:196] TabletServer@127.11.13.130:0 shutdown complete.
I20260709 12:17:09.409962 11318 tablet_server.cc:179] TabletServer@127.11.13.131:0 shutting down...
W20260709 12:17:09.418717 26459 consensus_peers.cc:597] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 -> Peer cc1c2a51ec344ca28260030593d6563e (127.11.13.130:38281): Couldn't send request to peer cc1c2a51ec344ca28260030593d6563e. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.130:38281: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:09.434818 26459 consensus_peers.cc:597] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 -> Peer f1991907f700454badc2971ecf14911d (127.11.13.129:37117): Couldn't send request to peer f1991907f700454badc2971ecf14911d. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:37117: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:09.452286 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:09.453306 11318 tablet_replica.cc:333] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632: stopping tablet replica
I20260709 12:17:09.454519 11318 raft_consensus.cc:2243] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.455870 11318 raft_consensus.cc:2272] T bcb7c3cc69b5458db189e1449a4196c8 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.458724 11318 tablet_replica.cc:333] T d3328229dda9476c97777ff8dc69c435 P a9c038faa7aa4de680db447a77ed8632: stopping tablet replica
I20260709 12:17:09.459362 11318 raft_consensus.cc:2243] T d3328229dda9476c97777ff8dc69c435 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.459867 11318 raft_consensus.cc:2272] T d3328229dda9476c97777ff8dc69c435 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.463318 11318 tablet_replica.cc:333] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632: stopping tablet replica
I20260709 12:17:09.463903 11318 raft_consensus.cc:2243] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.464433 11318 raft_consensus.cc:2272] T f11300f991be4590bf37eec3e516d05c P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.467188 11318 tablet_replica.cc:333] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632: stopping tablet replica
I20260709 12:17:09.467821 11318 raft_consensus.cc:2243] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.468330 11318 raft_consensus.cc:2272] T 224667a8ff894a64a8a95dac8edeeff6 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.470937 11318 tablet_replica.cc:333] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632: stopping tablet replica
I20260709 12:17:09.471534 11318 raft_consensus.cc:2243] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:09.472563 11318 raft_consensus.cc:2272] T ec84a53b3f294643b5da4d7e464931a9 P a9c038faa7aa4de680db447a77ed8632 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.475365 11318 tablet_replica.cc:333] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632: stopping tablet replica
I20260709 12:17:09.476670 11318 raft_consensus.cc:2243] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.477442 11318 raft_consensus.cc:2272] T 21d01b3e50754bb38a24d1ea383a61f9 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.479949 11318 tablet_replica.cc:333] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632: stopping tablet replica
I20260709 12:17:09.480585 11318 raft_consensus.cc:2243] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.481137 11318 raft_consensus.cc:2272] T 18ad382f11764922a0010dd3e13478d1 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.483964 11318 tablet_replica.cc:333] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632: stopping tablet replica
I20260709 12:17:09.484615 11318 raft_consensus.cc:2243] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:09.485637 11318 raft_consensus.cc:2272] T 9648db75c5cb45a681230d22c473e29f P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.488035 11318 tablet_replica.cc:333] T 3f981b08993f4a4aa6778078403e1004 P a9c038faa7aa4de680db447a77ed8632: stopping tablet replica
I20260709 12:17:09.488675 11318 raft_consensus.cc:2243] T 3f981b08993f4a4aa6778078403e1004 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.489187 11318 raft_consensus.cc:2272] T 3f981b08993f4a4aa6778078403e1004 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.491631 11318 tablet_replica.cc:333] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632: stopping tablet replica
I20260709 12:17:09.492234 11318 raft_consensus.cc:2243] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:09.493156 11318 raft_consensus.cc:2272] T 0f82565aabb44f9985dfd97ae33d8f3f P a9c038faa7aa4de680db447a77ed8632 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.495401 11318 tablet_replica.cc:333] T b3488dcfa84d4a699bcc7a4a3c827577 P a9c038faa7aa4de680db447a77ed8632: stopping tablet replica
I20260709 12:17:09.496001 11318 raft_consensus.cc:2243] T b3488dcfa84d4a699bcc7a4a3c827577 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.496562 11318 raft_consensus.cc:2272] T b3488dcfa84d4a699bcc7a4a3c827577 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.498919 11318 tablet_replica.cc:333] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632: stopping tablet replica
I20260709 12:17:09.499472 11318 raft_consensus.cc:2243] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:09.500546 11318 raft_consensus.cc:2272] T 8ebf90a9375a4a2798dad2c4de791d92 P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.504891 11318 tablet_replica.cc:333] T 3b5f70527bbc4070880acc6ef899ddb4 P a9c038faa7aa4de680db447a77ed8632: stopping tablet replica
I20260709 12:17:09.505798 11318 raft_consensus.cc:2243] T 3b5f70527bbc4070880acc6ef899ddb4 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.506690 11318 raft_consensus.cc:2272] T 3b5f70527bbc4070880acc6ef899ddb4 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.509688 11318 tablet_replica.cc:333] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632: stopping tablet replica
I20260709 12:17:09.510334 11318 raft_consensus.cc:2243] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:09.511304 11318 raft_consensus.cc:2272] T 6956538aee40492683999ec2274993a4 P a9c038faa7aa4de680db447a77ed8632 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.514022 11318 tablet_replica.cc:333] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632: stopping tablet replica
I20260709 12:17:09.514703 11318 raft_consensus.cc:2243] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.515206 11318 raft_consensus.cc:2272] T 15f1cf7b68d6463fbd60efc8c4576fc8 P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.518000 11318 tablet_replica.cc:333] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632: stopping tablet replica
I20260709 12:17:09.518626 11318 raft_consensus.cc:2243] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:09.519558 11318 raft_consensus.cc:2272] T 0cee69f616244c02b6c01f0f7cb1465b P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.521981 11318 tablet_replica.cc:333] T b97814c8781b43a08eb809dd0441fbae P a9c038faa7aa4de680db447a77ed8632: stopping tablet replica
I20260709 12:17:09.522576 11318 raft_consensus.cc:2243] T b97814c8781b43a08eb809dd0441fbae P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.523095 11318 raft_consensus.cc:2272] T b97814c8781b43a08eb809dd0441fbae P a9c038faa7aa4de680db447a77ed8632 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.525588 11318 tablet_replica.cc:333] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632: stopping tablet replica
I20260709 12:17:09.526212 11318 raft_consensus.cc:2243] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:09.526727 11318 raft_consensus.cc:2272] T 0859d4d4116a463f8e3a00a7737862ea P a9c038faa7aa4de680db447a77ed8632 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.577265 11318 tablet_server.cc:196] TabletServer@127.11.13.131:0 shutdown complete.
I20260709 12:17:09.622628 11318 master.cc:562] Master@127.11.13.190:45359 shutting down...
I20260709 12:17:09.655068 11318 raft_consensus.cc:2243] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:09.655752 11318 raft_consensus.cc:2272] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:09.656102 11318 tablet_replica.cc:333] T 00000000000000000000000000000000 P 8de7d5a07fb447aaa68b7f3acce9665f: stopping tablet replica
I20260709 12:17:09.680348 11318 master.cc:584] Master@127.11.13.190:45359 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/0 (21511 ms)
[ RUN      ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1
I20260709 12:17:09.742991 11318 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.11.13.190:40633
I20260709 12:17:09.744306 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:09.751453 26832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:09.751567 26833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:09.751685 26835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:09.753674 11318 server_base.cc:1061] running on GCE node
I20260709 12:17:09.754846 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:09.755038 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:09.755163 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599429755152 us; error 0 us; skew 500 ppm
I20260709 12:17:09.755631 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:09.759171 11318 webserver.cc:533] Webserver started at http://127.11.13.190:44339/ using document root <none> and password file <none>
I20260709 12:17:09.759735 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:09.759925 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:09.760171 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:09.761523 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598892056905-11318-0/minicluster-data/master-0-root/instance:
uuid: "d8f564118dbf464586d3dad1c5b403b2"
format_stamp: "Formatted at 2026-07-09 12:17:09 on dist-test-slave-zqnl"
I20260709 12:17:09.767429 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.001s	sys 0.004s
I20260709 12:17:09.772025 26840 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:09.773167 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.001s
I20260709 12:17:09.773607 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598892056905-11318-0/minicluster-data/master-0-root
uuid: "d8f564118dbf464586d3dad1c5b403b2"
format_stamp: "Formatted at 2026-07-09 12:17:09 on dist-test-slave-zqnl"
I20260709 12:17:09.773937 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598892056905-11318-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598892056905-11318-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598892056905-11318-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:09.803145 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:09.804807 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:09.853210 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.190:40633
I20260709 12:17:09.853350 26891 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.190:40633 every 8 connection(s)
I20260709 12:17:09.857684 26892 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:09.872802 26892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2: Bootstrap starting.
I20260709 12:17:09.878659 26892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:09.883442 26892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2: No bootstrap required, opened a new log
I20260709 12:17:09.885874 26892 raft_consensus.cc:359] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8f564118dbf464586d3dad1c5b403b2" member_type: VOTER }
I20260709 12:17:09.886401 26892 raft_consensus.cc:385] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:09.886628 26892 raft_consensus.cc:740] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8f564118dbf464586d3dad1c5b403b2, State: Initialized, Role: FOLLOWER
I20260709 12:17:09.887367 26892 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8f564118dbf464586d3dad1c5b403b2" member_type: VOTER }
I20260709 12:17:09.887857 26892 raft_consensus.cc:399] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260709 12:17:09.888110 26892 raft_consensus.cc:493] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260709 12:17:09.888389 26892 raft_consensus.cc:3060] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:09.893802 26892 raft_consensus.cc:515] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8f564118dbf464586d3dad1c5b403b2" member_type: VOTER }
I20260709 12:17:09.894485 26892 leader_election.cc:304] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d8f564118dbf464586d3dad1c5b403b2; no voters: 
I20260709 12:17:09.895915 26892 leader_election.cc:290] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2 [CANDIDATE]: Term 1 election: Requested vote from peers 
I20260709 12:17:09.896705 26895 raft_consensus.cc:2804] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:09.898478 26895 raft_consensus.cc:697] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2 [term 1 LEADER]: Becoming Leader. State: Replica: d8f564118dbf464586d3dad1c5b403b2, State: Running, Role: LEADER
I20260709 12:17:09.899379 26895 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8f564118dbf464586d3dad1c5b403b2" member_type: VOTER }
I20260709 12:17:09.899821 26892 sys_catalog.cc:565] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260709 12:17:09.906193 26897 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8f564118dbf464586d3dad1c5b403b2. Latest consensus state: current_term: 1 leader_uuid: "d8f564118dbf464586d3dad1c5b403b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8f564118dbf464586d3dad1c5b403b2" member_type: VOTER } }
I20260709 12:17:09.906706 26896 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d8f564118dbf464586d3dad1c5b403b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d8f564118dbf464586d3dad1c5b403b2" member_type: VOTER } }
I20260709 12:17:09.907881 26897 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2 [sys.catalog]: This master's current role is: LEADER
I20260709 12:17:09.908010 26896 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2 [sys.catalog]: This master's current role is: LEADER
I20260709 12:17:09.920681 11318 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
W20260709 12:17:09.924149 26911 catalog_manager.cc:1594] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260709 12:17:09.924494 26911 catalog_manager.cc:883] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260709 12:17:09.924980 26903 catalog_manager.cc:1511] Loading table and tablet metadata into memory...
I20260709 12:17:09.930017 26903 catalog_manager.cc:1520] Initializing Kudu cluster ID...
I20260709 12:17:09.939833 26903 catalog_manager.cc:1383] Generated new cluster ID: 9b7b744d12c84a6e886df9fcbba4af90
I20260709 12:17:09.940203 26903 catalog_manager.cc:1531] Initializing Kudu internal certificate authority...
I20260709 12:17:09.953894 26903 catalog_manager.cc:1406] Generated new certificate authority record
I20260709 12:17:09.955478 26903 catalog_manager.cc:1540] Loading token signing keys...
I20260709 12:17:09.974426 26903 catalog_manager.cc:6070] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2: Generated new TSK 0
I20260709 12:17:09.975132 26903 catalog_manager.cc:1550] Initializing in-progress tserver states...
I20260709 12:17:09.988379 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:09.995287 26913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:09.996572 26914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:10.000947 11318 server_base.cc:1061] running on GCE node
W20260709 12:17:10.001421 26916 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:10.005488 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:10.005743 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:10.006016 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599430005992 us; error 0 us; skew 500 ppm
I20260709 12:17:10.006711 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:10.009599 11318 webserver.cc:533] Webserver started at http://127.11.13.129:46599/ using document root <none> and password file <none>
I20260709 12:17:10.010203 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:10.010402 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:10.010699 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:10.011854 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598892056905-11318-0/minicluster-data/ts-0-root/instance:
uuid: "38218df822dc435eac030105d4f51140"
format_stamp: "Formatted at 2026-07-09 12:17:10 on dist-test-slave-zqnl"
I20260709 12:17:10.017031 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.005s	user 0.006s	sys 0.000s
I20260709 12:17:10.021286 26921 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:10.022418 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.002s	sys 0.000s
I20260709 12:17:10.022811 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598892056905-11318-0/minicluster-data/ts-0-root
uuid: "38218df822dc435eac030105d4f51140"
format_stamp: "Formatted at 2026-07-09 12:17:10 on dist-test-slave-zqnl"
I20260709 12:17:10.023144 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598892056905-11318-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598892056905-11318-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598892056905-11318-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:10.033486 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:10.034862 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:10.036504 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:10.039117 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:10.039362 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:10.039660 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:10.039865 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:10.092245 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.129:36041
I20260709 12:17:10.092372 26983 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.129:36041 every 8 connection(s)
I20260709 12:17:10.097797 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:10.106583 26988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:10.107865 26989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:10.112325 11318 server_base.cc:1061] running on GCE node
W20260709 12:17:10.112668 26991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:10.113801 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:10.114168 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:10.114387 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599430114366 us; error 0 us; skew 500 ppm
I20260709 12:17:10.114916 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:10.118098 11318 webserver.cc:533] Webserver started at http://127.11.13.130:39711/ using document root <none> and password file <none>
I20260709 12:17:10.118726 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:10.118991 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:10.119244 26984 heartbeater.cc:344] Connected to a master server at 127.11.13.190:40633
I20260709 12:17:10.119391 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:10.119874 26984 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:10.120666 26984 heartbeater.cc:507] Master 127.11.13.190:40633 requested a full tablet report, sending...
I20260709 12:17:10.121081 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598892056905-11318-0/minicluster-data/ts-1-root/instance:
uuid: "5604f5b9fef245bca9665c54cdd0d967"
format_stamp: "Formatted at 2026-07-09 12:17:10 on dist-test-slave-zqnl"
I20260709 12:17:10.123363 26857 ts_manager.cc:194] Registered new tserver with Master: 38218df822dc435eac030105d4f51140 (127.11.13.129:36041)
I20260709 12:17:10.126415 26857 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36962
I20260709 12:17:10.128769 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.007s	user 0.007s	sys 0.000s
I20260709 12:17:10.133502 26996 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:10.134671 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.000s	sys 0.002s
I20260709 12:17:10.135015 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598892056905-11318-0/minicluster-data/ts-1-root
uuid: "5604f5b9fef245bca9665c54cdd0d967"
format_stamp: "Formatted at 2026-07-09 12:17:10 on dist-test-slave-zqnl"
I20260709 12:17:10.135272 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598892056905-11318-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598892056905-11318-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598892056905-11318-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:10.151850 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:10.153147 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:10.155350 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:10.158301 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:10.158521 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:10.158849 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:10.159014 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:10.208983 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.130:37771
I20260709 12:17:10.209102 27058 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.130:37771 every 8 connection(s)
I20260709 12:17:10.214553 11318 file_cache.cc:504] Constructed file cache file cache with capacity 419430
W20260709 12:17:10.222265 27062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:10.225049 27063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260709 12:17:10.229914 27065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260709 12:17:10.231096 11318 server_base.cc:1061] running on GCE node
I20260709 12:17:10.232272 11318 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260709 12:17:10.232534 11318 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260709 12:17:10.232739 11318 hybrid_clock.cc:648] HybridClock initialized: now 1783599430232722 us; error 0 us; skew 500 ppm
I20260709 12:17:10.233237 27059 heartbeater.cc:344] Connected to a master server at 127.11.13.190:40633
I20260709 12:17:10.233467 11318 server_base.cc:861] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260709 12:17:10.233767 27059 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:10.234918 27059 heartbeater.cc:507] Master 127.11.13.190:40633 requested a full tablet report, sending...
I20260709 12:17:10.236622 11318 webserver.cc:533] Webserver started at http://127.11.13.131:35707/ using document root <none> and password file <none>
I20260709 12:17:10.237397 11318 fs_manager.cc:362] Metadata directory not provided
I20260709 12:17:10.237684 11318 fs_manager.cc:368] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260709 12:17:10.237470 26857 ts_manager.cc:194] Registered new tserver with Master: 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:10.238090 11318 server_base.cc:909] This appears to be a new deployment of Kudu; creating new FS layout
I20260709 12:17:10.239354 26857 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36974
I20260709 12:17:10.239493 11318 fs_manager.cc:1068] Generated new instance metadata in path /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598892056905-11318-0/minicluster-data/ts-2-root/instance:
uuid: "508ff162d05e44159e225081ede82882"
format_stamp: "Formatted at 2026-07-09 12:17:10 on dist-test-slave-zqnl"
I20260709 12:17:10.245882 11318 fs_manager.cc:696] Time spent creating directory manager: real 0.006s	user 0.002s	sys 0.004s
I20260709 12:17:10.250054 27070 log_block_manager.cc:4127] Time spent loading block containers with low live blocks: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:10.251276 11318 fs_manager.cc:730] Time spent opening block manager: real 0.003s	user 0.003s	sys 0.000s
I20260709 12:17:10.251668 11318 fs_manager.cc:647] Opened local filesystem: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598892056905-11318-0/minicluster-data/ts-2-root
uuid: "508ff162d05e44159e225081ede82882"
format_stamp: "Formatted at 2026-07-09 12:17:10 on dist-test-slave-zqnl"
I20260709 12:17:10.252017 11318 fs_report.cc:389] FS layout report
--------------------
wal directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598892056905-11318-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598892056905-11318-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskUH2_Yx/test-tmp/auto_leader_rebalancer-test.0.FilterSoftDeletedTableTest.TestFilterSofteDeletedTable_1.1783598892056905-11318-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260709 12:17:10.326936 11318 rpc_server.cc:225] running with OpenSSL 1.1.1  11 Sep 2018
I20260709 12:17:10.328333 11318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260709 12:17:10.329850 11318 txn_system_client.cc:432] TxnSystemClient initialization is disabled...
I20260709 12:17:10.332540 11318 ts_tablet_manager.cc:585] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260709 12:17:10.332787 11318 ts_tablet_manager.cc:531] Time spent load tablet metadata: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:10.333068 11318 ts_tablet_manager.cc:616] Registered 0 tablets
I20260709 12:17:10.333259 11318 ts_tablet_manager.cc:595] Time spent register tablets: real 0.000s	user 0.000s	sys 0.000s
I20260709 12:17:10.389014 11318 rpc_server.cc:307] RPC server started. Bound to: 127.11.13.131:40411
I20260709 12:17:10.389375 27132 acceptor_pool.cc:272] collecting diagnostics on the listening RPC socket 127.11.13.131:40411 every 8 connection(s)
I20260709 12:17:10.407367 27133 heartbeater.cc:344] Connected to a master server at 127.11.13.190:40633
I20260709 12:17:10.407882 27133 heartbeater.cc:461] Registering TS with master...
I20260709 12:17:10.408739 27133 heartbeater.cc:507] Master 127.11.13.190:40633 requested a full tablet report, sending...
I20260709 12:17:10.411343 26857 ts_manager.cc:194] Registered new tserver with Master: 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:10.412554 11318 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.019801126s
I20260709 12:17:10.412899 26857 master_service.cc:502] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:36980
I20260709 12:17:10.456878 26857 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36992:
name: "test-workload"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:17:10.460130 26857 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:17:10.539213 27098 tablet_service.cc:1511] Processing CreateTablet for tablet e2c4d9a0aa474d0aafdb7abba0ceadfd (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:17:10.541419 27098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2c4d9a0aa474d0aafdb7abba0ceadfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.543005 27097 tablet_service.cc:1511] Processing CreateTablet for tablet f6a2f90e85c04cf28ba0397d0c0a5b23 (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:17:10.544714 27097 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6a2f90e85c04cf28ba0397d0c0a5b23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.547370 27024 tablet_service.cc:1511] Processing CreateTablet for tablet e2c4d9a0aa474d0aafdb7abba0ceadfd (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:17:10.549068 27024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2c4d9a0aa474d0aafdb7abba0ceadfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.555197 27023 tablet_service.cc:1511] Processing CreateTablet for tablet f6a2f90e85c04cf28ba0397d0c0a5b23 (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:17:10.556841 27023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6a2f90e85c04cf28ba0397d0c0a5b23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.579762 27149 tablet_bootstrap.cc:492] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967: Bootstrap starting.
I20260709 12:17:10.581331 26949 tablet_service.cc:1511] Processing CreateTablet for tablet e2c4d9a0aa474d0aafdb7abba0ceadfd (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:17:10.577950 27148 tablet_bootstrap.cc:492] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882: Bootstrap starting.
I20260709 12:17:10.582988 26949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e2c4d9a0aa474d0aafdb7abba0ceadfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.583356 27097 tablet_service.cc:1511] Processing CreateTablet for tablet c6dcf02f879841949d494b2232d540ee (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:17:10.585083 27097 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6dcf02f879841949d494b2232d540ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.586180 26948 tablet_service.cc:1511] Processing CreateTablet for tablet f6a2f90e85c04cf28ba0397d0c0a5b23 (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:17:10.588102 26948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet f6a2f90e85c04cf28ba0397d0c0a5b23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.591794 26946 tablet_service.cc:1511] Processing CreateTablet for tablet caf9d48542094084a3a015e242077f30 (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:17:10.595753 26947 tablet_service.cc:1511] Processing CreateTablet for tablet c6dcf02f879841949d494b2232d540ee (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:17:10.597512 26946 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caf9d48542094084a3a015e242077f30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.598603 26947 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6dcf02f879841949d494b2232d540ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.602304 27024 tablet_service.cc:1511] Processing CreateTablet for tablet c6dcf02f879841949d494b2232d540ee (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:17:10.604068 27024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c6dcf02f879841949d494b2232d540ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.607141 27149 tablet_bootstrap.cc:654] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.611325 27148 tablet_bootstrap.cc:654] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.615077 27098 tablet_service.cc:1511] Processing CreateTablet for tablet caf9d48542094084a3a015e242077f30 (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:17:10.623091 26945 tablet_service.cc:1511] Processing CreateTablet for tablet 1baf73a2b0be4194bf53c8731fdb7da2 (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:17:10.625720 27098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caf9d48542094084a3a015e242077f30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.624709 26945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1baf73a2b0be4194bf53c8731fdb7da2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.660248 27149 tablet_bootstrap.cc:492] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967: No bootstrap required, opened a new log
I20260709 12:17:10.660880 27149 ts_tablet_manager.cc:1403] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967: Time spent bootstrapping tablet: real 0.082s	user 0.014s	sys 0.014s
I20260709 12:17:10.663424 27023 tablet_service.cc:1511] Processing CreateTablet for tablet caf9d48542094084a3a015e242077f30 (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:17:10.664083 27149 raft_consensus.cc:359] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } }
I20260709 12:17:10.664940 27149 raft_consensus.cc:385] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.665112 27023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet caf9d48542094084a3a015e242077f30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.665243 27149 raft_consensus.cc:740] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.666936 27149 consensus_queue.cc:260] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } }
I20260709 12:17:10.675662 27022 tablet_service.cc:1511] Processing CreateTablet for tablet 1baf73a2b0be4194bf53c8731fdb7da2 (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:17:10.677220 27022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1baf73a2b0be4194bf53c8731fdb7da2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.678972 27097 tablet_service.cc:1511] Processing CreateTablet for tablet 1baf73a2b0be4194bf53c8731fdb7da2 (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:17:10.680496 27097 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 1baf73a2b0be4194bf53c8731fdb7da2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.685276 27021 tablet_service.cc:1511] Processing CreateTablet for tablet 8b6087005dd541ffa46a97375e5ba4a1 (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:17:10.687417 27021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b6087005dd541ffa46a97375e5ba4a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.697225 27096 tablet_service.cc:1511] Processing CreateTablet for tablet 8b6087005dd541ffa46a97375e5ba4a1 (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:17:10.698940 27152 tablet_bootstrap.cc:492] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140: Bootstrap starting.
I20260709 12:17:10.700342 27096 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b6087005dd541ffa46a97375e5ba4a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.707647 27059 heartbeater.cc:499] Master 127.11.13.190:40633 was elected leader, sending a full tablet report...
I20260709 12:17:10.711279 27149 ts_tablet_manager.cc:1434] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967: Time spent starting tablet: real 0.050s	user 0.007s	sys 0.003s
I20260709 12:17:10.712505 27149 tablet_bootstrap.cc:492] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967: Bootstrap starting.
I20260709 12:17:10.712277 26945 tablet_service.cc:1511] Processing CreateTablet for tablet afc42e2e5dab4df88c1c00bf9df243e0 (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
W20260709 12:17:10.713798 27060 tablet.cc:2406] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260709 12:17:10.714833 26945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afc42e2e5dab4df88c1c00bf9df243e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.721418 27149 tablet_bootstrap.cc:654] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.726898 27148 tablet_bootstrap.cc:492] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882: No bootstrap required, opened a new log
I20260709 12:17:10.732386 27148 ts_tablet_manager.cc:1403] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882: Time spent bootstrapping tablet: real 0.155s	user 0.046s	sys 0.044s
I20260709 12:17:10.724275 26947 tablet_service.cc:1511] Processing CreateTablet for tablet 8b6087005dd541ffa46a97375e5ba4a1 (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:17:10.731494 27152 tablet_bootstrap.cc:654] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.735446 26947 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 8b6087005dd541ffa46a97375e5ba4a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.736066 27024 tablet_service.cc:1511] Processing CreateTablet for tablet afc42e2e5dab4df88c1c00bf9df243e0 (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:17:10.739974 27095 tablet_service.cc:1511] Processing CreateTablet for tablet afc42e2e5dab4df88c1c00bf9df243e0 (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:17:10.741513 27095 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afc42e2e5dab4df88c1c00bf9df243e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.744197 27153 raft_consensus.cc:493] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.745132 27153 raft_consensus.cc:515] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } }
I20260709 12:17:10.749611 27153 leader_election.cc:290] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 508ff162d05e44159e225081ede82882 (127.11.13.131:40411), 38218df822dc435eac030105d4f51140 (127.11.13.129:36041)
I20260709 12:17:10.745419 27148 raft_consensus.cc:359] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } }
I20260709 12:17:10.750314 27148 raft_consensus.cc:385] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.750667 27148 raft_consensus.cc:740] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.751729 27148 consensus_queue.cc:260] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } }
I20260709 12:17:10.772739 26949 tablet_service.cc:1511] Processing CreateTablet for tablet e85c5e2f2a6b4d31bd83715d05eda1c1 (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:17:10.773867 27149 tablet_bootstrap.cc:492] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967: No bootstrap required, opened a new log
I20260709 12:17:10.774817 26949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e85c5e2f2a6b4d31bd83715d05eda1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.776023 27149 ts_tablet_manager.cc:1403] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967: Time spent bootstrapping tablet: real 0.064s	user 0.012s	sys 0.004s
I20260709 12:17:10.784931 27133 heartbeater.cc:499] Master 127.11.13.190:40633 was elected leader, sending a full tablet report...
I20260709 12:17:10.787788 27021 tablet_service.cc:1511] Processing CreateTablet for tablet e85c5e2f2a6b4d31bd83715d05eda1c1 (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:17:10.785162 27024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet afc42e2e5dab4df88c1c00bf9df243e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.790820 26946 tablet_service.cc:1511] Processing CreateTablet for tablet 44117a0328e8455d8562a4dbe8a56eb3 (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:17:10.804422 26946 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44117a0328e8455d8562a4dbe8a56eb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.805660 27022 tablet_service.cc:1511] Processing CreateTablet for tablet 44117a0328e8455d8562a4dbe8a56eb3 (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:17:10.806941 27098 tablet_service.cc:1511] Processing CreateTablet for tablet 44117a0328e8455d8562a4dbe8a56eb3 (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:17:10.808260 27098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44117a0328e8455d8562a4dbe8a56eb3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.789789 27097 tablet_service.cc:1511] Processing CreateTablet for tablet e85c5e2f2a6b4d31bd83715d05eda1c1 (DEFAULT_TABLE table=test-workload [id=65e87e05882d4e18ab5b31d6da0df865]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:17:10.815184 27097 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e85c5e2f2a6b4d31bd83715d05eda1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.814066 27152 tablet_bootstrap.cc:492] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140: No bootstrap required, opened a new log
I20260709 12:17:10.825543 27152 ts_tablet_manager.cc:1403] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140: Time spent bootstrapping tablet: real 0.127s	user 0.014s	sys 0.006s
I20260709 12:17:10.801757 27021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet e85c5e2f2a6b4d31bd83715d05eda1c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:10.825944 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c4d9a0aa474d0aafdb7abba0ceadfd" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:10.826399 27022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 44117a0328e8455d8562a4dbe8a56eb3. 1 dirs total, 0 dirs full, 0 dirs failed
W20260709 12:17:10.827919 26998 leader_election.cc:343] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38218df822dc435eac030105d4f51140 (127.11.13.129:36041): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:10.797318 27149 raft_consensus.cc:359] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:10.828926 27149 raft_consensus.cc:385] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.829296 27149 raft_consensus.cc:740] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.830541 27149 consensus_queue.cc:260] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:10.792827 27148 ts_tablet_manager.cc:1434] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882: Time spent starting tablet: real 0.060s	user 0.014s	sys 0.000s
I20260709 12:17:10.837890 27148 tablet_bootstrap.cc:492] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882: Bootstrap starting.
I20260709 12:17:10.838479 27149 ts_tablet_manager.cc:1434] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967: Time spent starting tablet: real 0.062s	user 0.004s	sys 0.004s
I20260709 12:17:10.840900 27149 tablet_bootstrap.cc:492] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967: Bootstrap starting.
I20260709 12:17:10.848076 27149 tablet_bootstrap.cc:654] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.853150 27148 tablet_bootstrap.cc:654] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.858206 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c4d9a0aa474d0aafdb7abba0ceadfd" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:10.867738 27108 raft_consensus.cc:2468] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 0.
I20260709 12:17:10.869526 27000 leader_election.cc:304] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967; no voters: 38218df822dc435eac030105d4f51140
I20260709 12:17:10.870906 27153 raft_consensus.cc:2804] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:10.871291 27153 raft_consensus.cc:493] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:10.871641 27153 raft_consensus.cc:3060] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.860339 27152 raft_consensus.cc:359] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:10.874539 27152 raft_consensus.cc:385] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.875027 27152 raft_consensus.cc:740] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.876191 27152 consensus_queue.cc:260] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:10.879338 27153 raft_consensus.cc:515] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } }
I20260709 12:17:10.882820 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c4d9a0aa474d0aafdb7abba0ceadfd" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:10.883466 27108 raft_consensus.cc:3060] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.884720 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c4d9a0aa474d0aafdb7abba0ceadfd" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:10.883471 27153 leader_election.cc:290] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 election: Requested vote from peers 508ff162d05e44159e225081ede82882 (127.11.13.131:40411), 38218df822dc435eac030105d4f51140 (127.11.13.129:36041)
I20260709 12:17:10.884692 27149 tablet_bootstrap.cc:492] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967: No bootstrap required, opened a new log
W20260709 12:17:10.886426 26998 leader_election.cc:343] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38218df822dc435eac030105d4f51140 (127.11.13.129:36041): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:10.887004 27149 ts_tablet_manager.cc:1403] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967: Time spent bootstrapping tablet: real 0.048s	user 0.026s	sys 0.016s
I20260709 12:17:10.885891 26984 heartbeater.cc:499] Master 127.11.13.190:40633 was elected leader, sending a full tablet report...
I20260709 12:17:10.887394 27148 tablet_bootstrap.cc:492] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882: No bootstrap required, opened a new log
I20260709 12:17:10.888007 27148 ts_tablet_manager.cc:1403] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882: Time spent bootstrapping tablet: real 0.050s	user 0.010s	sys 0.003s
I20260709 12:17:10.888895 27152 ts_tablet_manager.cc:1434] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140: Time spent starting tablet: real 0.043s	user 0.004s	sys 0.009s
I20260709 12:17:10.890424 27152 tablet_bootstrap.cc:492] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140: Bootstrap starting.
I20260709 12:17:10.891422 27149 raft_consensus.cc:359] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:10.892412 27149 raft_consensus.cc:385] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.892835 27149 raft_consensus.cc:740] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.894775 27149 consensus_queue.cc:260] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:10.897614 27149 ts_tablet_manager.cc:1434] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967: Time spent starting tablet: real 0.010s	user 0.006s	sys 0.001s
I20260709 12:17:10.898820 27149 tablet_bootstrap.cc:492] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967: Bootstrap starting.
I20260709 12:17:10.899297 27108 raft_consensus.cc:2468] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 1.
I20260709 12:17:10.892036 27148 raft_consensus.cc:359] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:10.900398 27148 raft_consensus.cc:385] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.900604 27152 tablet_bootstrap.cc:654] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.900822 27148 raft_consensus.cc:740] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.900894 27000 leader_election.cc:304] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967; no voters: 38218df822dc435eac030105d4f51140
I20260709 12:17:10.902283 27153 raft_consensus.cc:2804] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:10.902110 27148 consensus_queue.cc:260] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:10.903715 27160 raft_consensus.cc:493] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.904258 27160 raft_consensus.cc:515] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:10.906947 27148 ts_tablet_manager.cc:1434] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882: Time spent starting tablet: real 0.018s	user 0.010s	sys 0.000s
I20260709 12:17:10.907027 27149 tablet_bootstrap.cc:654] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.908696 27148 tablet_bootstrap.cc:492] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882: Bootstrap starting.
W20260709 12:17:10.915154 26905 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:10.916143 27160 leader_election.cc:290] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:10.920688 27148 tablet_bootstrap.cc:654] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:10.937136 27153 raft_consensus.cc:697] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 1 LEADER]: Becoming Leader. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Running, Role: LEADER
I20260709 12:17:10.938352 27153 consensus_queue.cc:237] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } }
I20260709 12:17:10.941680 27149 tablet_bootstrap.cc:492] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967: No bootstrap required, opened a new log
I20260709 12:17:10.942579 27149 ts_tablet_manager.cc:1403] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967: Time spent bootstrapping tablet: real 0.044s	user 0.020s	sys 0.020s
I20260709 12:17:10.949247 27162 raft_consensus.cc:493] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:10.950150 27162 raft_consensus.cc:515] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:10.953526 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a2f90e85c04cf28ba0397d0c0a5b23" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:10.953745 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a2f90e85c04cf28ba0397d0c0a5b23" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:10.954237 26959 raft_consensus.cc:2468] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 0.
I20260709 12:17:10.954561 27108 raft_consensus.cc:2468] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 0.
I20260709 12:17:10.954309 27149 raft_consensus.cc:359] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:10.955247 27149 raft_consensus.cc:385] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.955612 27149 raft_consensus.cc:740] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.956007 26998 leader_election.cc:304] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:10.956238 26925 leader_election.cc:304] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882; no voters: 
I20260709 12:17:10.956418 27149 consensus_queue.cc:260] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:10.957281 27160 raft_consensus.cc:2804] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:10.957667 27160 raft_consensus.cc:493] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:10.958173 27160 raft_consensus.cc:3060] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.960258 27148 tablet_bootstrap.cc:492] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882: No bootstrap required, opened a new log
I20260709 12:17:10.960835 27148 ts_tablet_manager.cc:1403] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882: Time spent bootstrapping tablet: real 0.052s	user 0.011s	sys 0.002s
I20260709 12:17:10.960748 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a2f90e85c04cf28ba0397d0c0a5b23" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:10.961506 27108 raft_consensus.cc:2468] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 0.
I20260709 12:17:10.963827 27162 leader_election.cc:290] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:10.965049 27148 raft_consensus.cc:359] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:10.965369 27162 raft_consensus.cc:2804] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:10.968108 27162 raft_consensus.cc:493] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:10.968509 27162 raft_consensus.cc:3060] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.966017 27148 raft_consensus.cc:385] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.970888 27148 raft_consensus.cc:740] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.972110 27148 consensus_queue.cc:260] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:10.978845 27034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a2f90e85c04cf28ba0397d0c0a5b23" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:10.980227 27149 ts_tablet_manager.cc:1434] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967: Time spent starting tablet: real 0.037s	user 0.007s	sys 0.022s
I20260709 12:17:10.981536 27149 tablet_bootstrap.cc:492] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967: Bootstrap starting.
I20260709 12:17:10.981570 27148 ts_tablet_manager.cc:1434] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.007s
I20260709 12:17:10.981838 26856 catalog_manager.cc:5697] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130). New cstate: current_term: 1 leader_uuid: "5604f5b9fef245bca9665c54cdd0d967" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:10.983093 27148 tablet_bootstrap.cc:492] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882: Bootstrap starting.
I20260709 12:17:10.989183 27162 raft_consensus.cc:515] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:10.972002 27160 raft_consensus.cc:515] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:10.993002 27152 tablet_bootstrap.cc:492] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140: No bootstrap required, opened a new log
I20260709 12:17:10.993810 27152 ts_tablet_manager.cc:1403] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140: Time spent bootstrapping tablet: real 0.104s	user 0.021s	sys 0.016s
I20260709 12:17:10.994395 27033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a2f90e85c04cf28ba0397d0c0a5b23" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:10.994953 27162 leader_election.cc:290] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:10.996136 27160 leader_election.cc:290] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 election: Requested vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:10.997072 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a2f90e85c04cf28ba0397d0c0a5b23" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:10.997499 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a2f90e85c04cf28ba0397d0c0a5b23" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:10.997951 26959 raft_consensus.cc:2393] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5604f5b9fef245bca9665c54cdd0d967 in current term 1: Already voted for candidate 38218df822dc435eac030105d4f51140 in this term.
I20260709 12:17:10.998230 27107 raft_consensus.cc:3060] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:10.998174 27152 raft_consensus.cc:359] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } }
I20260709 12:17:10.999198 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a2f90e85c04cf28ba0397d0c0a5b23" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:10.990846 27034 raft_consensus.cc:2393] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38218df822dc435eac030105d4f51140 in current term 1: Already voted for candidate 5604f5b9fef245bca9665c54cdd0d967 in this term.
I20260709 12:17:11.004326 26998 leader_election.cc:304] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5604f5b9fef245bca9665c54cdd0d967; no voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882
I20260709 12:17:10.999226 27152 raft_consensus.cc:385] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:10.990772 27148 tablet_bootstrap.cc:654] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.005856 27152 raft_consensus.cc:740] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Initialized, Role: FOLLOWER
I20260709 12:17:10.993150 27149 tablet_bootstrap.cc:654] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.006909 27152 consensus_queue.cc:260] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } }
I20260709 12:17:11.009734 27152 ts_tablet_manager.cc:1434] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140: Time spent starting tablet: real 0.015s	user 0.006s	sys 0.002s
I20260709 12:17:11.014106 27153 raft_consensus.cc:2749] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:17:11.016526 27162 raft_consensus.cc:493] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.019194 27148 tablet_bootstrap.cc:492] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882: No bootstrap required, opened a new log
I20260709 12:17:11.019408 27149 tablet_bootstrap.cc:492] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967: No bootstrap required, opened a new log
I20260709 12:17:11.019815 27152 tablet_bootstrap.cc:492] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140: Bootstrap starting.
I20260709 12:17:11.019834 27148 ts_tablet_manager.cc:1403] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882: Time spent bootstrapping tablet: real 0.037s	user 0.009s	sys 0.011s
I20260709 12:17:11.020138 27162 raft_consensus.cc:515] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.023440 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf9d48542094084a3a015e242077f30" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:11.023447 27148 raft_consensus.cc:359] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.024307 27162 leader_election.cc:290] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:11.024433 27148 raft_consensus.cc:385] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.019949 27149 ts_tablet_manager.cc:1403] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967: Time spent bootstrapping tablet: real 0.039s	user 0.010s	sys 0.004s
W20260709 12:17:11.025602 26998 leader_election.cc:343] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38218df822dc435eac030105d4f51140 (127.11.13.129:36041): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.023799 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf9d48542094084a3a015e242077f30" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:11.027204 27152 tablet_bootstrap.cc:654] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:11.027578 27000 leader_election.cc:343] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 508ff162d05e44159e225081ede82882 (127.11.13.131:40411): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:11.028069 27000 leader_election.cc:304] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5604f5b9fef245bca9665c54cdd0d967; no voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882
I20260709 12:17:11.024799 27148 raft_consensus.cc:740] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.028889 27149 raft_consensus.cc:359] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.029814 27149 raft_consensus.cc:385] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.030241 27149 raft_consensus.cc:740] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.030033 27148 consensus_queue.cc:260] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.032636 27148 ts_tablet_manager.cc:1434] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.000s
I20260709 12:17:11.033835 27148 tablet_bootstrap.cc:492] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882: Bootstrap starting.
I20260709 12:17:11.017452 27107 raft_consensus.cc:2468] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 1.
I20260709 12:17:11.035643 27162 raft_consensus.cc:2749] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:11.035768 26925 leader_election.cc:304] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882; no voters: 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:11.036839 27160 raft_consensus.cc:2804] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:11.040469 27160 raft_consensus.cc:697] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Becoming Leader. State: Replica: 38218df822dc435eac030105d4f51140, State: Running, Role: LEADER
I20260709 12:17:11.041024 27148 tablet_bootstrap.cc:654] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.041545 27160 consensus_queue.cc:237] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.042663 27152 tablet_bootstrap.cc:492] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140: No bootstrap required, opened a new log
I20260709 12:17:11.043238 27152 ts_tablet_manager.cc:1403] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140: Time spent bootstrapping tablet: real 0.024s	user 0.012s	sys 0.007s
I20260709 12:17:11.046833 27152 raft_consensus.cc:359] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.047205 27149 consensus_queue.cc:260] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.050050 27152 raft_consensus.cc:385] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.050604 27152 raft_consensus.cc:740] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.051537 27152 consensus_queue.cc:260] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.052184 27148 tablet_bootstrap.cc:492] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882: No bootstrap required, opened a new log
I20260709 12:17:11.053081 27148 ts_tablet_manager.cc:1403] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.007s
I20260709 12:17:11.054996 27152 ts_tablet_manager.cc:1434] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140: Time spent starting tablet: real 0.011s	user 0.008s	sys 0.000s
I20260709 12:17:11.056638 27152 tablet_bootstrap.cc:492] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140: Bootstrap starting.
I20260709 12:17:11.052188 27149 ts_tablet_manager.cc:1434] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967: Time spent starting tablet: real 0.027s	user 0.008s	sys 0.000s
I20260709 12:17:11.056318 27148 raft_consensus.cc:359] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.058318 27148 raft_consensus.cc:385] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.058636 27149 tablet_bootstrap.cc:492] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967: Bootstrap starting.
I20260709 12:17:11.066056 27152 tablet_bootstrap.cc:654] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.058652 27148 raft_consensus.cc:740] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.067490 27148 consensus_queue.cc:260] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.067565 26855 catalog_manager.cc:5697] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38218df822dc435eac030105d4f51140 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "38218df822dc435eac030105d4f51140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:11.069922 27148 ts_tablet_manager.cc:1434] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882: Time spent starting tablet: real 0.016s	user 0.000s	sys 0.004s
I20260709 12:17:11.071427 27148 tablet_bootstrap.cc:492] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882: Bootstrap starting.
I20260709 12:17:11.072048 27156 raft_consensus.cc:493] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.072590 27156 raft_consensus.cc:515] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.083593 27149 tablet_bootstrap.cc:654] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.076821 27156 leader_election.cc:290] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:11.095131 27148 tablet_bootstrap.cc:654] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.099483 27034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:11.100647 27034 raft_consensus.cc:2468] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 0.
I20260709 12:17:11.102720 27152 tablet_bootstrap.cc:492] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140: No bootstrap required, opened a new log
I20260709 12:17:11.103348 27152 ts_tablet_manager.cc:1403] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140: Time spent bootstrapping tablet: real 0.047s	user 0.016s	sys 0.011s
I20260709 12:17:11.102928 27071 leader_election.cc:304] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:11.104395 27156 raft_consensus.cc:2804] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.104882 27156 raft_consensus.cc:493] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.105325 27156 raft_consensus.cc:3060] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.107719 27152 raft_consensus.cc:359] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.108800 27152 raft_consensus.cc:385] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.109208 27152 raft_consensus.cc:740] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.109715 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:11.110284 27152 consensus_queue.cc:260] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.110752 26959 raft_consensus.cc:2468] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 0.
I20260709 12:17:11.115124 27152 ts_tablet_manager.cc:1434] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140: Time spent starting tablet: real 0.011s	user 0.010s	sys 0.000s
I20260709 12:17:11.116317 27152 tablet_bootstrap.cc:492] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140: Bootstrap starting.
I20260709 12:17:11.118664 27156 raft_consensus.cc:515] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.120539 27148 tablet_bootstrap.cc:492] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882: No bootstrap required, opened a new log
I20260709 12:17:11.121637 27148 ts_tablet_manager.cc:1403] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.007s
I20260709 12:17:11.122893 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:11.123870 26959 raft_consensus.cc:3060] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.127346 27152 tablet_bootstrap.cc:654] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.127947 27148 raft_consensus.cc:359] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.129112 27148 raft_consensus.cc:385] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.129670 27148 raft_consensus.cc:740] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.132403 27149 tablet_bootstrap.cc:492] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967: No bootstrap required, opened a new log
I20260709 12:17:11.132079 27148 consensus_queue.cc:260] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.133078 27149 ts_tablet_manager.cc:1403] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967: Time spent bootstrapping tablet: real 0.075s	user 0.012s	sys 0.014s
I20260709 12:17:11.132879 27034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:11.133737 27034 raft_consensus.cc:3060] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.133809 27152 tablet_bootstrap.cc:492] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140: No bootstrap required, opened a new log
I20260709 12:17:11.134810 27152 ts_tablet_manager.cc:1403] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140: Time spent bootstrapping tablet: real 0.019s	user 0.011s	sys 0.003s
I20260709 12:17:11.137156 27149 raft_consensus.cc:359] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.138262 27149 raft_consensus.cc:385] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.138666 27149 raft_consensus.cc:740] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.139664 27149 consensus_queue.cc:260] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.144448 27152 raft_consensus.cc:359] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.145493 27152 raft_consensus.cc:385] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.145874 27152 raft_consensus.cc:740] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.147234 27152 consensus_queue.cc:260] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.149181 26959 raft_consensus.cc:2468] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 1.
I20260709 12:17:11.152302 27149 ts_tablet_manager.cc:1434] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.000s
I20260709 12:17:11.153326 27072 leader_election.cc:304] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882; no voters: 
I20260709 12:17:11.155591 27148 ts_tablet_manager.cc:1434] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882: Time spent starting tablet: real 0.034s	user 0.010s	sys 0.014s
I20260709 12:17:11.156834 27148 tablet_bootstrap.cc:492] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882: Bootstrap starting.
I20260709 12:17:11.150717 27152 ts_tablet_manager.cc:1434] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140: Time spent starting tablet: real 0.015s	user 0.007s	sys 0.003s
I20260709 12:17:11.151191 27034 raft_consensus.cc:2468] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 1.
I20260709 12:17:11.151722 27156 leader_election.cc:290] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:11.160476 27152 tablet_bootstrap.cc:492] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140: Bootstrap starting.
I20260709 12:17:11.157617 27183 raft_consensus.cc:2804] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:11.164391 27183 raft_consensus.cc:697] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Becoming Leader. State: Replica: 508ff162d05e44159e225081ede82882, State: Running, Role: LEADER
I20260709 12:17:11.165531 27148 tablet_bootstrap.cc:654] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.165836 27183 consensus_queue.cc:237] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.168654 27152 tablet_bootstrap.cc:654] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.182274 27152 tablet_bootstrap.cc:492] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140: No bootstrap required, opened a new log
I20260709 12:17:11.182994 27152 ts_tablet_manager.cc:1403] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140: Time spent bootstrapping tablet: real 0.023s	user 0.009s	sys 0.011s
I20260709 12:17:11.184800 27149 tablet_bootstrap.cc:492] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967: Bootstrap starting.
I20260709 12:17:11.186487 27152 raft_consensus.cc:359] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.187449 27152 raft_consensus.cc:385] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.187839 27152 raft_consensus.cc:740] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.188850 27152 consensus_queue.cc:260] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.194872 27152 ts_tablet_manager.cc:1434] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140: Time spent starting tablet: real 0.011s	user 0.007s	sys 0.001s
I20260709 12:17:11.196394 27152 tablet_bootstrap.cc:492] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140: Bootstrap starting.
I20260709 12:17:11.197472 27149 tablet_bootstrap.cc:654] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.200174 27148 tablet_bootstrap.cc:492] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882: No bootstrap required, opened a new log
I20260709 12:17:11.200912 27148 ts_tablet_manager.cc:1403] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882: Time spent bootstrapping tablet: real 0.044s	user 0.014s	sys 0.025s
I20260709 12:17:11.204135 27152 tablet_bootstrap.cc:654] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.205407 27148 raft_consensus.cc:359] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.207850 27148 raft_consensus.cc:385] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.208509 27148 raft_consensus.cc:740] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.209918 27148 consensus_queue.cc:260] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.193775 26857 catalog_manager.cc:5697] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 reported cstate change: term changed from 0 to 1, leader changed from <none> to 508ff162d05e44159e225081ede82882 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "508ff162d05e44159e225081ede82882" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:11.213544 27148 ts_tablet_manager.cc:1434] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882: Time spent starting tablet: real 0.012s	user 0.009s	sys 0.000s
I20260709 12:17:11.215474 27148 tablet_bootstrap.cc:492] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882: Bootstrap starting.
I20260709 12:17:11.218276 27152 tablet_bootstrap.cc:492] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140: No bootstrap required, opened a new log
I20260709 12:17:11.219030 27152 ts_tablet_manager.cc:1403] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140: Time spent bootstrapping tablet: real 0.023s	user 0.013s	sys 0.007s
I20260709 12:17:11.222512 27152 raft_consensus.cc:359] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.223984 27152 raft_consensus.cc:385] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.224370 27152 raft_consensus.cc:740] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.225061 27148 tablet_bootstrap.cc:654] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.225714 27152 consensus_queue.cc:260] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.231657 27152 ts_tablet_manager.cc:1434] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140: Time spent starting tablet: real 0.012s	user 0.006s	sys 0.001s
I20260709 12:17:11.233286 27152 tablet_bootstrap.cc:492] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140: Bootstrap starting.
I20260709 12:17:11.241057 27152 tablet_bootstrap.cc:654] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.250636 27160 raft_consensus.cc:493] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.251665 27160 raft_consensus.cc:515] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.255447 27034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1baf73a2b0be4194bf53c8731fdb7da2" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:11.256253 27034 raft_consensus.cc:2468] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 0.
I20260709 12:17:11.257004 27160 leader_election.cc:290] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:11.257804 27149 tablet_bootstrap.cc:492] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967: No bootstrap required, opened a new log
I20260709 12:17:11.258438 27149 ts_tablet_manager.cc:1403] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967: Time spent bootstrapping tablet: real 0.074s	user 0.021s	sys 0.044s
I20260709 12:17:11.258381 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1baf73a2b0be4194bf53c8731fdb7da2" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:11.259243 27107 raft_consensus.cc:2468] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 0.
I20260709 12:17:11.259624 26922 leader_election.cc:304] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:11.260176 27152 tablet_bootstrap.cc:492] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140: No bootstrap required, opened a new log
I20260709 12:17:11.260727 27152 ts_tablet_manager.cc:1403] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140: Time spent bootstrapping tablet: real 0.028s	user 0.017s	sys 0.000s
I20260709 12:17:11.260644 27160 raft_consensus.cc:2804] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.258515 27148 tablet_bootstrap.cc:492] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882: No bootstrap required, opened a new log
I20260709 12:17:11.261292 27160 raft_consensus.cc:493] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.261696 27148 ts_tablet_manager.cc:1403] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882: Time spent bootstrapping tablet: real 0.047s	user 0.026s	sys 0.016s
I20260709 12:17:11.261746 27160 raft_consensus.cc:3060] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.261984 27149 raft_consensus.cc:359] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.263242 27149 raft_consensus.cc:385] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.263564 27149 raft_consensus.cc:740] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.264426 27149 consensus_queue.cc:260] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.265141 27148 raft_consensus.cc:359] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.266109 27148 raft_consensus.cc:385] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.266474 27148 raft_consensus.cc:740] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.267504 27149 ts_tablet_manager.cc:1434] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:17:11.267318 27148 consensus_queue.cc:260] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.269004 27149 tablet_bootstrap.cc:492] T 44117a0328e8455d8562a4dbe8a56eb3 P 5604f5b9fef245bca9665c54cdd0d967: Bootstrap starting.
I20260709 12:17:11.270263 27148 ts_tablet_manager.cc:1434] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882: Time spent starting tablet: real 0.008s	user 0.006s	sys 0.003s
I20260709 12:17:11.271662 27148 tablet_bootstrap.cc:492] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882: Bootstrap starting.
I20260709 12:17:11.273581 27152 raft_consensus.cc:359] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.275060 27152 raft_consensus.cc:385] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.275453 27152 raft_consensus.cc:740] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.276290 27152 consensus_queue.cc:260] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.278338 27149 tablet_bootstrap.cc:654] T 44117a0328e8455d8562a4dbe8a56eb3 P 5604f5b9fef245bca9665c54cdd0d967: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.283272 27152 ts_tablet_manager.cc:1434] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140: Time spent starting tablet: real 0.013s	user 0.006s	sys 0.001s
I20260709 12:17:11.280707 27148 tablet_bootstrap.cc:654] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.280856 27160 raft_consensus.cc:515] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.288211 27174 raft_consensus.cc:493] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.288861 27174 raft_consensus.cc:515] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.289749 27160 leader_election.cc:290] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 election: Requested vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:11.291448 27174 leader_election.cc:290] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:11.292567 27033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf9d48542094084a3a015e242077f30" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:11.294317 27152 tablet_bootstrap.cc:492] T 44117a0328e8455d8562a4dbe8a56eb3 P 38218df822dc435eac030105d4f51140: Bootstrap starting.
I20260709 12:17:11.293938 27033 raft_consensus.cc:2468] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 0.
I20260709 12:17:11.294385 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf9d48542094084a3a015e242077f30" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:11.295697 27108 raft_consensus.cc:2468] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 0.
I20260709 12:17:11.297125 27034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1baf73a2b0be4194bf53c8731fdb7da2" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:11.297778 27034 raft_consensus.cc:3060] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.301633 26925 leader_election.cc:304] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882; no voters: 
I20260709 12:17:11.302706 27174 raft_consensus.cc:2804] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.303143 27174 raft_consensus.cc:493] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.303483 27174 raft_consensus.cc:3060] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.304371 27148 tablet_bootstrap.cc:492] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882: No bootstrap required, opened a new log
I20260709 12:17:11.304980 27148 ts_tablet_manager.cc:1403] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882: Time spent bootstrapping tablet: real 0.034s	user 0.011s	sys 0.001s
I20260709 12:17:11.306754 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1baf73a2b0be4194bf53c8731fdb7da2" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:11.307454 27107 raft_consensus.cc:3060] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.308192 27148 raft_consensus.cc:359] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.308960 27148 raft_consensus.cc:385] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.309239 27148 raft_consensus.cc:740] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.310777 27152 tablet_bootstrap.cc:654] T 44117a0328e8455d8562a4dbe8a56eb3 P 38218df822dc435eac030105d4f51140: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.317740 27149 tablet_bootstrap.cc:492] T 44117a0328e8455d8562a4dbe8a56eb3 P 5604f5b9fef245bca9665c54cdd0d967: No bootstrap required, opened a new log
I20260709 12:17:11.318181 27184 raft_consensus.cc:493] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.318548 27149 ts_tablet_manager.cc:1403] T 44117a0328e8455d8562a4dbe8a56eb3 P 5604f5b9fef245bca9665c54cdd0d967: Time spent bootstrapping tablet: real 0.050s	user 0.019s	sys 0.021s
I20260709 12:17:11.318897 27184 raft_consensus.cc:515] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.320760 27174 raft_consensus.cc:515] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.321875 27149 raft_consensus.cc:359] T 44117a0328e8455d8562a4dbe8a56eb3 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.322669 27149 raft_consensus.cc:385] T 44117a0328e8455d8562a4dbe8a56eb3 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.322768 27033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:11.323012 27149 raft_consensus.cc:740] T 44117a0328e8455d8562a4dbe8a56eb3 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.323689 27033 raft_consensus.cc:2468] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 0.
I20260709 12:17:11.324209 27032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf9d48542094084a3a015e242077f30" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:11.324251 27149 consensus_queue.cc:260] T 44117a0328e8455d8562a4dbe8a56eb3 P 5604f5b9fef245bca9665c54cdd0d967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.325129 27032 raft_consensus.cc:3060] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.325405 27071 leader_election.cc:304] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:11.327016 27149 ts_tablet_manager.cc:1434] T 44117a0328e8455d8562a4dbe8a56eb3 P 5604f5b9fef245bca9665c54cdd0d967: Time spent starting tablet: real 0.008s	user 0.007s	sys 0.000s
I20260709 12:17:11.328054 27183 raft_consensus.cc:2804] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.328538 27183 raft_consensus.cc:493] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.328868 27183 raft_consensus.cc:3060] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.330413 27149 tablet_bootstrap.cc:492] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967: Bootstrap starting.
I20260709 12:17:11.309988 27148 consensus_queue.cc:260] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.337852 27032 raft_consensus.cc:2468] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 1.
I20260709 12:17:11.336264 27184 leader_election.cc:290] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:11.339736 26922 leader_election.cc:304] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:11.341150 27149 tablet_bootstrap.cc:654] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:11.341337 27160 raft_consensus.cc:2804] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:11.342697 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:11.343525 26959 raft_consensus.cc:2468] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 0.
I20260709 12:17:11.344977 27148 ts_tablet_manager.cc:1434] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882: Time spent starting tablet: real 0.040s	user 0.010s	sys 0.000s
I20260709 12:17:11.347175 27174 leader_election.cc:290] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 election: Requested vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:11.348515 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf9d48542094084a3a015e242077f30" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:11.349225 27108 raft_consensus.cc:3060] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.350651 27107 raft_consensus.cc:2468] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 1.
I20260709 12:17:11.351302 27034 raft_consensus.cc:2468] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 1.
I20260709 12:17:11.353183 26922 leader_election.cc:304] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:11.354571 27183 raft_consensus.cc:515] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.356153 27174 raft_consensus.cc:2804] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:11.356719 27174 raft_consensus.cc:697] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Becoming Leader. State: Replica: 38218df822dc435eac030105d4f51140, State: Running, Role: LEADER
I20260709 12:17:11.357986 27174 consensus_queue.cc:237] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.363775 27160 raft_consensus.cc:697] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Becoming Leader. State: Replica: 38218df822dc435eac030105d4f51140, State: Running, Role: LEADER
I20260709 12:17:11.366271 27108 raft_consensus.cc:2468] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 1.
I20260709 12:17:11.366761 27160 consensus_queue.cc:237] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.368721 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:11.372025 26959 raft_consensus.cc:3060] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.369903 27152 tablet_bootstrap.cc:492] T 44117a0328e8455d8562a4dbe8a56eb3 P 38218df822dc435eac030105d4f51140: No bootstrap required, opened a new log
I20260709 12:17:11.369585 27034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:11.375149 27034 raft_consensus.cc:3060] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.378685 27195 raft_consensus.cc:493] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.375840 27152 ts_tablet_manager.cc:1403] T 44117a0328e8455d8562a4dbe8a56eb3 P 38218df822dc435eac030105d4f51140: Time spent bootstrapping tablet: real 0.082s	user 0.014s	sys 0.005s
I20260709 12:17:11.379261 27195 raft_consensus.cc:515] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } }
I20260709 12:17:11.380527 26958 raft_consensus.cc:3060] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.382516 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c4d9a0aa474d0aafdb7abba0ceadfd" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:11.382867 27032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c4d9a0aa474d0aafdb7abba0ceadfd" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:11.383504 27107 raft_consensus.cc:2393] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38218df822dc435eac030105d4f51140 in current term 1: Already voted for candidate 5604f5b9fef245bca9665c54cdd0d967 in this term.
I20260709 12:17:11.385030 26922 leader_election.cc:304] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38218df822dc435eac030105d4f51140; no voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:11.393126 27195 leader_election.cc:290] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 508ff162d05e44159e225081ede82882 (127.11.13.131:40411), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:11.370362 27183 leader_election.cc:290] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:11.374497 27149 tablet_bootstrap.cc:492] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967: No bootstrap required, opened a new log
I20260709 12:17:11.396425 27149 ts_tablet_manager.cc:1403] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967: Time spent bootstrapping tablet: real 0.066s	user 0.021s	sys 0.002s
I20260709 12:17:11.398952 27152 raft_consensus.cc:359] T 44117a0328e8455d8562a4dbe8a56eb3 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.400154 27152 raft_consensus.cc:385] T 44117a0328e8455d8562a4dbe8a56eb3 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.400620 27152 raft_consensus.cc:740] T 44117a0328e8455d8562a4dbe8a56eb3 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.401839 27152 consensus_queue.cc:260] T 44117a0328e8455d8562a4dbe8a56eb3 P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.405340 27152 ts_tablet_manager.cc:1434] T 44117a0328e8455d8562a4dbe8a56eb3 P 38218df822dc435eac030105d4f51140: Time spent starting tablet: real 0.026s	user 0.008s	sys 0.000s
I20260709 12:17:11.406390 27034 raft_consensus.cc:2468] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 1.
I20260709 12:17:11.409494 27071 leader_election.cc:304] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:11.410887 27183 raft_consensus.cc:2804] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:11.413542 27183 raft_consensus.cc:697] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Becoming Leader. State: Replica: 508ff162d05e44159e225081ede82882, State: Running, Role: LEADER
I20260709 12:17:11.415364 27183 consensus_queue.cc:237] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.409507 27149 raft_consensus.cc:359] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.419186 27149 raft_consensus.cc:385] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:11.419570 27149 raft_consensus.cc:740] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Initialized, Role: FOLLOWER
I20260709 12:17:11.422210 27149 consensus_queue.cc:260] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.417528 26959 raft_consensus.cc:2468] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 1.
I20260709 12:17:11.426788 27195 raft_consensus.cc:2749] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:11.429015 27162 consensus_queue.cc:1048] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:11.434052 27162 consensus_queue.cc:1048] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:11.434878 27149 ts_tablet_manager.cc:1434] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967: Time spent starting tablet: real 0.037s	user 0.005s	sys 0.003s
I20260709 12:17:11.447027 26856 catalog_manager.cc:5697] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 reported cstate change: term changed from 0 to 1, leader changed from <none> to 508ff162d05e44159e225081ede82882 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "508ff162d05e44159e225081ede82882" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:11.449874 26857 catalog_manager.cc:5697] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38218df822dc435eac030105d4f51140 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "38218df822dc435eac030105d4f51140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:11.454089 27174 consensus_queue.cc:1048] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:11.467154 27183 raft_consensus.cc:493] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.468025 27183 raft_consensus.cc:515] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.471014 27183 leader_election.cc:290] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:11.472337 26958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afc42e2e5dab4df88c1c00bf9df243e0" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:11.473955 26958 raft_consensus.cc:2468] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 0.
I20260709 12:17:11.475716 27072 leader_election.cc:304] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882; no voters: 
I20260709 12:17:11.476266 27032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afc42e2e5dab4df88c1c00bf9df243e0" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:11.477072 27032 raft_consensus.cc:2468] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 0.
I20260709 12:17:11.486472 27184 raft_consensus.cc:493] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.487205 27184 raft_consensus.cc:515] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.492022 27174 raft_consensus.cc:493] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:11.492681 27174 raft_consensus.cc:515] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.494588 27032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44117a0328e8455d8562a4dbe8a56eb3" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:11.495361 27032 raft_consensus.cc:2468] T 44117a0328e8455d8562a4dbe8a56eb3 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 0.
I20260709 12:17:11.496011 27174 leader_election.cc:290] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:11.496866 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44117a0328e8455d8562a4dbe8a56eb3" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:11.497848 26959 raft_consensus.cc:2468] T 44117a0328e8455d8562a4dbe8a56eb3 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 0.
I20260709 12:17:11.498221 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85c5e2f2a6b4d31bd83715d05eda1c1" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:11.496780 27071 leader_election.cc:304] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:11.498900 27107 raft_consensus.cc:2468] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 0.
I20260709 12:17:11.493791 27184 leader_election.cc:290] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:11.499809 27156 raft_consensus.cc:2804] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.500334 27156 raft_consensus.cc:493] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.500789 27156 raft_consensus.cc:3060] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.503280 26925 leader_election.cc:304] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882; no voters: 
I20260709 12:17:11.503934 27183 raft_consensus.cc:2804] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.504372 27183 raft_consensus.cc:493] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.504750 27183 raft_consensus.cc:3060] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.514916 27198 raft_consensus.cc:2804] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:11.515725 27032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85c5e2f2a6b4d31bd83715d05eda1c1" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:11.515753 27198 raft_consensus.cc:493] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:11.516587 27032 raft_consensus.cc:2468] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 0.
I20260709 12:17:11.516942 27198 raft_consensus.cc:3060] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.523190 27195 consensus_queue.cc:1048] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:11.529489 27198 raft_consensus.cc:515] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.532564 27198 leader_election.cc:290] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 election: Requested vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:11.508996 26856 catalog_manager.cc:5697] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38218df822dc435eac030105d4f51140 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "38218df822dc435eac030105d4f51140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:11.534574 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85c5e2f2a6b4d31bd83715d05eda1c1" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:11.535322 27108 raft_consensus.cc:3060] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.540781 27156 raft_consensus.cc:515] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.542784 27032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85c5e2f2a6b4d31bd83715d05eda1c1" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:11.544227 27032 raft_consensus.cc:3060] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.547822 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44117a0328e8455d8562a4dbe8a56eb3" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:11.549986 26959 raft_consensus.cc:3060] T 44117a0328e8455d8562a4dbe8a56eb3 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.551754 27033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44117a0328e8455d8562a4dbe8a56eb3" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:11.552577 27033 raft_consensus.cc:3060] T 44117a0328e8455d8562a4dbe8a56eb3 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.556771 27183 raft_consensus.cc:515] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.560828 27033 raft_consensus.cc:2468] T 44117a0328e8455d8562a4dbe8a56eb3 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 1.
I20260709 12:17:11.562482 27071 leader_election.cc:304] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:11.564179 27184 raft_consensus.cc:2804] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:11.565083 27183 leader_election.cc:290] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:11.567114 27031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afc42e2e5dab4df88c1c00bf9df243e0" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:11.568184 27031 raft_consensus.cc:3060] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.568701 26959 raft_consensus.cc:2468] T 44117a0328e8455d8562a4dbe8a56eb3 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 1.
I20260709 12:17:11.570092 27108 raft_consensus.cc:2468] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 1.
I20260709 12:17:11.572647 26925 leader_election.cc:304] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882; no voters: 
I20260709 12:17:11.573673 27198 raft_consensus.cc:2804] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:11.574306 27198 raft_consensus.cc:697] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Becoming Leader. State: Replica: 38218df822dc435eac030105d4f51140, State: Running, Role: LEADER
I20260709 12:17:11.574396 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afc42e2e5dab4df88c1c00bf9df243e0" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:11.578344 26959 raft_consensus.cc:3060] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:11.578604 27198 consensus_queue.cc:237] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.580755 27183 consensus_queue.cc:1048] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:11.582630 27031 raft_consensus.cc:2468] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 1.
I20260709 12:17:11.584264 27071 leader_election.cc:304] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:11.586643 26959 raft_consensus.cc:2468] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 1.
I20260709 12:17:11.577018 27032 raft_consensus.cc:2468] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 1.
I20260709 12:17:11.587350 27183 raft_consensus.cc:2804] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:11.589633 27184 raft_consensus.cc:697] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Becoming Leader. State: Replica: 508ff162d05e44159e225081ede82882, State: Running, Role: LEADER
I20260709 12:17:11.591004 27184 consensus_queue.cc:237] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.589758 27156 leader_election.cc:290] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:11.597724 27183 raft_consensus.cc:697] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Becoming Leader. State: Replica: 508ff162d05e44159e225081ede82882, State: Running, Role: LEADER
I20260709 12:17:11.612442 26856 catalog_manager.cc:5697] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38218df822dc435eac030105d4f51140 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "38218df822dc435eac030105d4f51140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:11.624651 27183 consensus_queue.cc:237] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.626770 27184 consensus_queue.cc:1048] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:11.650646 26857 catalog_manager.cc:5697] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 reported cstate change: term changed from 0 to 1, leader changed from <none> to 508ff162d05e44159e225081ede82882 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "508ff162d05e44159e225081ede82882" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:11.652197 26857 catalog_manager.cc:5697] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 reported cstate change: term changed from 0 to 1, leader changed from <none> to 508ff162d05e44159e225081ede82882 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "508ff162d05e44159e225081ede82882" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:11.694722 27034 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e2c4d9a0aa474d0aafdb7abba0ceadfd"
dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
mode: GRACEFUL
new_leader_uuid: "38218df822dc435eac030105d4f51140"
 from {username='slave'} at 127.0.0.1:57596
I20260709 12:17:11.695411 27034 raft_consensus.cc:606] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 1 LEADER]: Received request to transfer leadership to TS 38218df822dc435eac030105d4f51140
I20260709 12:17:11.716871 27107 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee"
dest_uuid: "508ff162d05e44159e225081ede82882"
mode: GRACEFUL
new_leader_uuid: "38218df822dc435eac030105d4f51140"
 from {username='slave'} at 127.0.0.1:44468
I20260709 12:17:11.717483 27107 raft_consensus.cc:606] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Received request to transfer leadership to TS 38218df822dc435eac030105d4f51140
I20260709 12:17:11.738958 26959 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "1baf73a2b0be4194bf53c8731fdb7da2"
dest_uuid: "38218df822dc435eac030105d4f51140"
mode: GRACEFUL
new_leader_uuid: "5604f5b9fef245bca9665c54cdd0d967"
 from {username='slave'} at 127.0.0.1:44740
I20260709 12:17:11.739494 26959 raft_consensus.cc:606] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Received request to transfer leadership to TS 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:11.745985 27107 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1"
dest_uuid: "508ff162d05e44159e225081ede82882"
mode: GRACEFUL
new_leader_uuid: "5604f5b9fef245bca9665c54cdd0d967"
 from {username='slave'} at 127.0.0.1:44468
I20260709 12:17:11.746637 27107 raft_consensus.cc:606] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Received request to transfer leadership to TS 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:11.753506 27107 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "afc42e2e5dab4df88c1c00bf9df243e0"
dest_uuid: "508ff162d05e44159e225081ede82882"
mode: GRACEFUL
new_leader_uuid: "5604f5b9fef245bca9665c54cdd0d967"
 from {username='slave'} at 127.0.0.1:44468
I20260709 12:17:11.754192 27107 raft_consensus.cc:606] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Received request to transfer leadership to TS 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:11.760289 26959 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e85c5e2f2a6b4d31bd83715d05eda1c1"
dest_uuid: "38218df822dc435eac030105d4f51140"
mode: GRACEFUL
new_leader_uuid: "5604f5b9fef245bca9665c54cdd0d967"
 from {username='slave'} at 127.0.0.1:44740
I20260709 12:17:11.760979 26959 raft_consensus.cc:606] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Received request to transfer leadership to TS 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:11.848479 27198 consensus_queue.cc:1048] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:11.857071 27183 consensus_queue.cc:1048] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:11.862500 27160 consensus_queue.cc:1048] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:11.880659 27174 raft_consensus.cc:993] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140: : Instructing follower 5604f5b9fef245bca9665c54cdd0d967 to start an election
I20260709 12:17:11.881206 27174 raft_consensus.cc:1081] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Signalling peer 5604f5b9fef245bca9665c54cdd0d967 to start an election
I20260709 12:17:11.881484 27214 consensus_queue.cc:1048] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:11.883323 27032 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "1baf73a2b0be4194bf53c8731fdb7da2"
dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
 from {username='slave'} at 127.0.0.1:44406
I20260709 12:17:11.884101 27032 raft_consensus.cc:493] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:11.884538 27032 raft_consensus.cc:3060] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.897341 27183 raft_consensus.cc:993] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882: : Instructing follower 5604f5b9fef245bca9665c54cdd0d967 to start an election
I20260709 12:17:11.897365 27032 raft_consensus.cc:515] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.898336 27214 raft_consensus.cc:1081] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Signalling peer 5604f5b9fef245bca9665c54cdd0d967 to start an election
I20260709 12:17:11.900803 27034 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1"
dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
 from {username='slave'} at 127.0.0.1:44418
I20260709 12:17:11.901124 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1baf73a2b0be4194bf53c8731fdb7da2" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:11.901379 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1baf73a2b0be4194bf53c8731fdb7da2" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:11.901613 27034 raft_consensus.cc:493] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:11.902112 26959 raft_consensus.cc:3055] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:11.902534 26959 raft_consensus.cc:740] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Running, Role: LEADER
I20260709 12:17:11.902256 27034 raft_consensus.cc:3060] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.902230 27107 raft_consensus.cc:3060] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.903502 26959 consensus_queue.cc:260] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.905776 26959 raft_consensus.cc:3060] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.912585 27032 leader_election.cc:290] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 2 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:11.912951 27034 raft_consensus.cc:515] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.916718 26959 raft_consensus.cc:2468] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 2.
I20260709 12:17:11.916900 27107 raft_consensus.cc:2468] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 2.
I20260709 12:17:11.918793 27160 consensus_queue.cc:1048] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:11.919078 26998 leader_election.cc:304] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:11.920153 27204 raft_consensus.cc:2804] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:11.920676 27204 raft_consensus.cc:697] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 LEADER]: Becoming Leader. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Running, Role: LEADER
I20260709 12:17:11.921602 27204 consensus_queue.cc:237] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.926980 27034 leader_election.cc:290] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 2 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:11.927296 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:11.928251 27108 raft_consensus.cc:3055] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:11.928632 27108 raft_consensus.cc:740] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Running, Role: LEADER
I20260709 12:17:11.926836 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:11.930459 26959 raft_consensus.cc:3060] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.929798 27108 consensus_queue.cc:260] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.932056 27108 raft_consensus.cc:3060] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.938786 26959 raft_consensus.cc:2468] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 2.
I20260709 12:17:11.940637 26998 leader_election.cc:304] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:11.940645 27108 raft_consensus.cc:2468] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 2.
I20260709 12:17:11.941843 27204 raft_consensus.cc:2804] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:11.942448 27204 raft_consensus.cc:697] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 LEADER]: Becoming Leader. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Running, Role: LEADER
I20260709 12:17:11.941377 26856 catalog_manager.cc:5697] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967 reported cstate change: term changed from 1 to 2, leader changed from 38218df822dc435eac030105d4f51140 (127.11.13.129) to 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130). New cstate: current_term: 2 leader_uuid: "5604f5b9fef245bca9665c54cdd0d967" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:11.943323 27204 consensus_queue.cc:237] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:11.954308 27174 consensus_queue.cc:1048] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:11.955987 27153 raft_consensus.cc:993] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967: : Instructing follower 38218df822dc435eac030105d4f51140 to start an election
I20260709 12:17:11.956559 27153 raft_consensus.cc:1081] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 1 LEADER]: Signalling peer 38218df822dc435eac030105d4f51140 to start an election
I20260709 12:17:11.959062 26959 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e2c4d9a0aa474d0aafdb7abba0ceadfd"
dest_uuid: "38218df822dc435eac030105d4f51140"
 from {username='slave'} at 127.0.0.1:48136
I20260709 12:17:11.959681 26959 raft_consensus.cc:493] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:11.960088 26959 raft_consensus.cc:3060] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.964087 26855 catalog_manager.cc:5697] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 reported cstate change: term changed from 1 to 2, leader changed from 508ff162d05e44159e225081ede82882 (127.11.13.131) to 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130). New cstate: current_term: 2 leader_uuid: "5604f5b9fef245bca9665c54cdd0d967" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:11.970769 26959 raft_consensus.cc:515] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } }
I20260709 12:17:11.974159 26959 leader_election.cc:290] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 2 election: Requested vote from peers 508ff162d05e44159e225081ede82882 (127.11.13.131:40411), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:11.974790 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c4d9a0aa474d0aafdb7abba0ceadfd" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:11.975580 27107 raft_consensus.cc:3060] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.974912 27034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c4d9a0aa474d0aafdb7abba0ceadfd" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:11.976557 27034 raft_consensus.cc:3055] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:11.976914 27034 raft_consensus.cc:740] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Running, Role: LEADER
I20260709 12:17:11.977802 27034 consensus_queue.cc:260] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } }
I20260709 12:17:11.979391 27034 raft_consensus.cc:3060] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:11.983742 27107 raft_consensus.cc:2468] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 2.
I20260709 12:17:11.985389 26925 leader_election.cc:304] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882; no voters: 
I20260709 12:17:11.986274 27174 raft_consensus.cc:2804] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:11.986780 27174 raft_consensus.cc:697] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [term 2 LEADER]: Becoming Leader. State: Replica: 38218df822dc435eac030105d4f51140, State: Running, Role: LEADER
I20260709 12:17:11.987670 27174 consensus_queue.cc:237] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } }
I20260709 12:17:11.987926 27034 raft_consensus.cc:2468] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 2.
I20260709 12:17:11.995575 26855 catalog_manager.cc:5697] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 reported cstate change: term changed from 1 to 2, leader changed from 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130) to 38218df822dc435eac030105d4f51140 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "38218df822dc435eac030105d4f51140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:12.059177 27183 raft_consensus.cc:993] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882: : Instructing follower 38218df822dc435eac030105d4f51140 to start an election
I20260709 12:17:12.059810 27214 raft_consensus.cc:1081] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Signalling peer 38218df822dc435eac030105d4f51140 to start an election
I20260709 12:17:12.061759 26958 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee"
dest_uuid: "38218df822dc435eac030105d4f51140"
 from {username='slave'} at 127.0.0.1:48144
I20260709 12:17:12.061784 27183 consensus_queue.cc:1048] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:12.062954 26958 raft_consensus.cc:493] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:12.063424 26958 raft_consensus.cc:3060] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.071753 26958 raft_consensus.cc:515] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:12.075114 27032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:12.075819 27032 raft_consensus.cc:3060] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.078568 26958 leader_election.cc:290] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 2 election: Requested vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:12.078372 27156 consensus_queue.cc:1048] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:12.080242 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:12.080888 27107 raft_consensus.cc:3055] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:12.081238 27107 raft_consensus.cc:740] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Running, Role: LEADER
I20260709 12:17:12.082329 27107 consensus_queue.cc:260] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:12.083928 27107 raft_consensus.cc:3060] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.085153 27032 raft_consensus.cc:2468] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 2.
I20260709 12:17:12.086455 26922 leader_election.cc:304] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:12.087397 27195 raft_consensus.cc:2804] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:12.087931 27195 raft_consensus.cc:697] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 2 LEADER]: Becoming Leader. State: Replica: 38218df822dc435eac030105d4f51140, State: Running, Role: LEADER
I20260709 12:17:12.088827 27195 consensus_queue.cc:237] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:12.092610 27214 consensus_queue.cc:1048] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:12.096621 27107 raft_consensus.cc:2468] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 2.
I20260709 12:17:12.104745 26854 catalog_manager.cc:5697] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 reported cstate change: term changed from 1 to 2, leader changed from 508ff162d05e44159e225081ede82882 (127.11.13.131) to 38218df822dc435eac030105d4f51140 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "38218df822dc435eac030105d4f51140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:12.117975 27214 consensus_queue.cc:1048] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:12.127530 27160 consensus_queue.cc:1048] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:12.129045 27160 consensus_queue.cc:1048] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:12.131108 27156 raft_consensus.cc:993] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882: : Instructing follower 5604f5b9fef245bca9665c54cdd0d967 to start an election
I20260709 12:17:12.131917 27214 raft_consensus.cc:1081] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Signalling peer 5604f5b9fef245bca9665c54cdd0d967 to start an election
I20260709 12:17:12.133534 27032 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "afc42e2e5dab4df88c1c00bf9df243e0"
dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
 from {username='slave'} at 127.0.0.1:44418
I20260709 12:17:12.134217 27032 raft_consensus.cc:493] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:12.134737 27032 raft_consensus.cc:3060] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.147637 27195 raft_consensus.cc:993] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140: : Instructing follower 5604f5b9fef245bca9665c54cdd0d967 to start an election
I20260709 12:17:12.148298 27195 raft_consensus.cc:1081] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Signalling peer 5604f5b9fef245bca9665c54cdd0d967 to start an election
I20260709 12:17:12.153059 27031 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e85c5e2f2a6b4d31bd83715d05eda1c1"
dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
 from {username='slave'} at 127.0.0.1:44406
I20260709 12:17:12.153735 27031 raft_consensus.cc:493] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:12.154157 27031 raft_consensus.cc:3060] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.151515 27032 raft_consensus.cc:515] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:12.156637 27032 leader_election.cc:290] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 2 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:12.157495 26958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afc42e2e5dab4df88c1c00bf9df243e0" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:12.158216 26958 raft_consensus.cc:3060] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.167512 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afc42e2e5dab4df88c1c00bf9df243e0" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:12.168473 27107 raft_consensus.cc:3055] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:12.168946 27107 raft_consensus.cc:740] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Running, Role: LEADER
I20260709 12:17:12.168087 27031 raft_consensus.cc:515] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:12.172683 26958 raft_consensus.cc:2468] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 2.
I20260709 12:17:12.174296 27031 leader_election.cc:290] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 2 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:12.174821 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85c5e2f2a6b4d31bd83715d05eda1c1" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:12.175594 26959 raft_consensus.cc:3055] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:12.175953 26959 raft_consensus.cc:740] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Running, Role: LEADER
I20260709 12:17:12.177019 26998 leader_election.cc:304] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:12.176823 26959 consensus_queue.cc:260] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:12.178032 27204 raft_consensus.cc:2804] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:12.178637 27204 raft_consensus.cc:697] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 LEADER]: Becoming Leader. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Running, Role: LEADER
I20260709 12:17:12.179224 26959 raft_consensus.cc:3060] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.179710 27204 consensus_queue.cc:237] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:12.182708 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85c5e2f2a6b4d31bd83715d05eda1c1" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:12.182884 27107 consensus_queue.cc:260] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:12.183430 27108 raft_consensus.cc:3060] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.184504 27107 raft_consensus.cc:3060] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:12.187630 26959 raft_consensus.cc:2468] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 2.
I20260709 12:17:12.189247 26998 leader_election.cc:304] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:12.190490 27204 raft_consensus.cc:2804] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:12.191311 27204 raft_consensus.cc:697] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 LEADER]: Becoming Leader. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Running, Role: LEADER
I20260709 12:17:12.191200 26854 catalog_manager.cc:5697] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967 reported cstate change: term changed from 1 to 2, leader changed from 508ff162d05e44159e225081ede82882 (127.11.13.131) to 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130). New cstate: current_term: 2 leader_uuid: "5604f5b9fef245bca9665c54cdd0d967" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:12.192902 27204 consensus_queue.cc:237] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:12.194481 27108 raft_consensus.cc:2468] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 2.
I20260709 12:17:12.200409 27107 raft_consensus.cc:2468] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 2.
I20260709 12:17:12.211597 26854 catalog_manager.cc:5697] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 reported cstate change: term changed from 1 to 2, leader changed from 38218df822dc435eac030105d4f51140 (127.11.13.129) to 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130). New cstate: current_term: 2 leader_uuid: "5604f5b9fef245bca9665c54cdd0d967" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:12.356436 27107 raft_consensus.cc:1275] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Refusing update from remote peer 5604f5b9fef245bca9665c54cdd0d967: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:12.356438 26958 raft_consensus.cc:1275] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Refusing update from remote peer 5604f5b9fef245bca9665c54cdd0d967: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:12.358246 27162 consensus_queue.cc:1048] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:12.358407 27204 consensus_queue.cc:1048] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:12.386195 27107 raft_consensus.cc:1275] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Refusing update from remote peer 5604f5b9fef245bca9665c54cdd0d967: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:12.387310 26958 raft_consensus.cc:1275] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Refusing update from remote peer 5604f5b9fef245bca9665c54cdd0d967: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:12.388836 27238 consensus_queue.cc:1048] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:12.389690 27204 consensus_queue.cc:1048] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:12.400378 27108 raft_consensus.cc:1275] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Refusing update from remote peer 38218df822dc435eac030105d4f51140: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:12.402361 27195 consensus_queue.cc:1048] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:12.406615 27031 raft_consensus.cc:1275] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Refusing update from remote peer 38218df822dc435eac030105d4f51140: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:12.408967 27195 consensus_queue.cc:1048] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:12.513257 27031 raft_consensus.cc:1275] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Refusing update from remote peer 38218df822dc435eac030105d4f51140: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:12.515470 27195 consensus_queue.cc:1048] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:12.533772 27107 raft_consensus.cc:1275] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Refusing update from remote peer 38218df822dc435eac030105d4f51140: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:12.536288 27160 consensus_queue.cc:1048] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:12.608152 26958 raft_consensus.cc:1275] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Refusing update from remote peer 5604f5b9fef245bca9665c54cdd0d967: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:12.611150 27238 consensus_queue.cc:1048] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:12.623785 27107 raft_consensus.cc:1275] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Refusing update from remote peer 5604f5b9fef245bca9665c54cdd0d967: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:12.626338 27204 consensus_queue.cc:1048] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:12.631703 26958 raft_consensus.cc:1275] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Refusing update from remote peer 5604f5b9fef245bca9665c54cdd0d967: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:12.637054 27238 consensus_queue.cc:1048] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:12.655642 27107 raft_consensus.cc:1275] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Refusing update from remote peer 5604f5b9fef245bca9665c54cdd0d967: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:12.657789 27204 consensus_queue.cc:1048] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:15.192358 27243 raft_consensus.cc:493] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38218df822dc435eac030105d4f51140)
I20260709 12:17:15.193142 27243 raft_consensus.cc:515] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:15.442631 27242 raft_consensus.cc:493] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38218df822dc435eac030105d4f51140)
I20260709 12:17:15.443706 27242 raft_consensus.cc:515] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:15.447395 27242 leader_election.cc:290] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:15.469223 27244 raft_consensus.cc:493] T 44117a0328e8455d8562a4dbe8a56eb3 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 508ff162d05e44159e225081ede82882)
I20260709 12:17:15.469991 27243 leader_election.cc:290] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:15.470227 27244 raft_consensus.cc:515] T 44117a0328e8455d8562a4dbe8a56eb3 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:15.472093 27246 raft_consensus.cc:493] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5604f5b9fef245bca9665c54cdd0d967)
I20260709 12:17:15.473130 27246 raft_consensus.cc:515] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:15.473860 27244 leader_election.cc:290] T 44117a0328e8455d8562a4dbe8a56eb3 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:15.476770 27246 leader_election.cc:290] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:15.483531 27244 raft_consensus.cc:493] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5604f5b9fef245bca9665c54cdd0d967)
I20260709 12:17:15.484308 27244 raft_consensus.cc:515] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:15.486800 27244 leader_election.cc:290] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:15.488912 27244 raft_consensus.cc:493] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5604f5b9fef245bca9665c54cdd0d967)
I20260709 12:17:15.489521 27244 raft_consensus.cc:515] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:15.492516 27244 leader_election.cc:290] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:15.532866 27245 raft_consensus.cc:493] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5604f5b9fef245bca9665c54cdd0d967)
I20260709 12:17:15.533562 27245 raft_consensus.cc:515] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:15.536900 27245 leader_election.cc:290] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:15.538256 27243 raft_consensus.cc:493] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38218df822dc435eac030105d4f51140)
I20260709 12:17:15.538959 27248 raft_consensus.cc:493] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5604f5b9fef245bca9665c54cdd0d967)
I20260709 12:17:15.539245 27243 raft_consensus.cc:515] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } }
I20260709 12:17:15.539690 27248 raft_consensus.cc:515] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:15.542438 27248 leader_election.cc:290] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:15.544727 27031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:15.545994 27034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44117a0328e8455d8562a4dbe8a56eb3" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:15.548180 27033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1baf73a2b0be4194bf53c8731fdb7da2" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:15.550038 27249 raft_consensus.cc:493] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38218df822dc435eac030105d4f51140)
I20260709 12:17:15.550869 27249 raft_consensus.cc:515] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:15.553179 27249 leader_election.cc:290] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:15.553761 27249 raft_consensus.cc:493] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5604f5b9fef245bca9665c54cdd0d967)
I20260709 12:17:15.554395 27249 raft_consensus.cc:515] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:15.555667 27033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afc42e2e5dab4df88c1c00bf9df243e0" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:15.556779 27249 leader_election.cc:290] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:15.558313 27251 raft_consensus.cc:493] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38218df822dc435eac030105d4f51140)
I20260709 12:17:15.558533 27034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85c5e2f2a6b4d31bd83715d05eda1c1" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:15.558880 27249 raft_consensus.cc:493] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38218df822dc435eac030105d4f51140)
I20260709 12:17:15.559103 27251 raft_consensus.cc:515] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:15.541797 27243 leader_election.cc:290] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 508ff162d05e44159e225081ede82882 (127.11.13.131:40411), 38218df822dc435eac030105d4f51140 (127.11.13.129:36041)
I20260709 12:17:15.559556 27249 raft_consensus.cc:515] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:15.561901 27251 leader_election.cc:290] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:15.562072 27249 leader_election.cc:290] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:15.562791 27251 raft_consensus.cc:493] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5604f5b9fef245bca9665c54cdd0d967)
I20260709 12:17:15.562956 27249 raft_consensus.cc:493] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38218df822dc435eac030105d4f51140)
I20260709 12:17:15.563881 27249 raft_consensus.cc:515] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } }
I20260709 12:17:15.567332 27249 leader_election.cc:290] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 38218df822dc435eac030105d4f51140 (127.11.13.129:36041)
I20260709 12:17:15.568015 27249 raft_consensus.cc:493] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5604f5b9fef245bca9665c54cdd0d967)
I20260709 12:17:15.563683 27251 raft_consensus.cc:515] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:15.568576 27249 raft_consensus.cc:515] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:15.570889 27251 leader_election.cc:290] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:15.571009 27249 leader_election.cc:290] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:15.573323 27031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a2f90e85c04cf28ba0397d0c0a5b23" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:15.574883 27030 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:15.575698 27034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1baf73a2b0be4194bf53c8731fdb7da2" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:15.575837 27032 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:15.576712 27029 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c4d9a0aa474d0aafdb7abba0ceadfd" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:15.576845 27028 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afc42e2e5dab4df88c1c00bf9df243e0" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:15.577096 27033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf9d48542094084a3a015e242077f30" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:15.576581 27031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85c5e2f2a6b4d31bd83715d05eda1c1" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:15.584993 26958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:15.588389 26958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf9d48542094084a3a015e242077f30" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
W20260709 12:17:15.588493 27072 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.129:36041, user_credentials={real_user=slave}} blocked reactor thread for 56185.2us
I20260709 12:17:15.590085 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "44117a0328e8455d8562a4dbe8a56eb3" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:15.593788 26958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c4d9a0aa474d0aafdb7abba0ceadfd" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:15.594205 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:15.594918 27108 raft_consensus.cc:2468] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 2.
I20260709 12:17:15.598731 27106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:15.599395 27105 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1baf73a2b0be4194bf53c8731fdb7da2" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:15.600418 27104 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85c5e2f2a6b4d31bd83715d05eda1c1" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:15.600956 26958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:15.602161 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afc42e2e5dab4df88c1c00bf9df243e0" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:15.603660 26958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a2f90e85c04cf28ba0397d0c0a5b23" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:15.603865 27106 raft_consensus.cc:2468] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 2.
I20260709 12:17:15.604379 26957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "1baf73a2b0be4194bf53c8731fdb7da2" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:15.605967 26958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf9d48542094084a3a015e242077f30" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:15.606696 26957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:15.608270 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c4d9a0aa474d0aafdb7abba0ceadfd" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:15.608614 26925 leader_election.cc:304] T 44117a0328e8455d8562a4dbe8a56eb3 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38218df822dc435eac030105d4f51140; no voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:15.609750 26957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85c5e2f2a6b4d31bd83715d05eda1c1" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:15.609805 27072 leader_election.cc:304] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 508ff162d05e44159e225081ede82882; no voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:15.610335 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "caf9d48542094084a3a015e242077f30" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:15.611058 27245 raft_consensus.cc:2749] T 44117a0328e8455d8562a4dbe8a56eb3 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:15.611284 27248 raft_consensus.cc:2749] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:15.611775 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e2c4d9a0aa474d0aafdb7abba0ceadfd" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:15.612010 27072 leader_election.cc:304] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 508ff162d05e44159e225081ede82882; no voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:15.613231 27248 raft_consensus.cc:2749] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:15.613795 27072 leader_election.cc:304] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 508ff162d05e44159e225081ede82882; no voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:15.614815 27248 raft_consensus.cc:2749] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:15.615325 27072 leader_election.cc:304] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 508ff162d05e44159e225081ede82882; no voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:15.614517 26925 leader_election.cc:304] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882; no voters: 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:15.616284 27248 raft_consensus.cc:2749] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:15.616505 27245 raft_consensus.cc:2804] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:15.616770 27072 leader_election.cc:304] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 508ff162d05e44159e225081ede82882; no voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:15.616999 27245 raft_consensus.cc:493] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5604f5b9fef245bca9665c54cdd0d967)
I20260709 12:17:15.617851 27245 raft_consensus.cc:3060] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:15.618006 27248 raft_consensus.cc:2749] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:15.616895 27000 leader_election.cc:304] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967; no voters: 38218df822dc435eac030105d4f51140
I20260709 12:17:15.619719 27252 raft_consensus.cc:2804] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:15.619804 26925 leader_election.cc:304] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38218df822dc435eac030105d4f51140; no voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:15.620196 27252 raft_consensus.cc:493] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 38218df822dc435eac030105d4f51140)
I20260709 12:17:15.620957 27252 raft_consensus.cc:3060] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:15.621381 27247 raft_consensus.cc:2749] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:15.621541 26925 leader_election.cc:304] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38218df822dc435eac030105d4f51140; no voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:15.622437 27247 raft_consensus.cc:2749] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:15.622922 26925 leader_election.cc:304] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38218df822dc435eac030105d4f51140; no voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:15.623838 27246 raft_consensus.cc:2749] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:15.624897 27000 leader_election.cc:304] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5604f5b9fef245bca9665c54cdd0d967; no voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882
I20260709 12:17:15.625692 27072 leader_election.cc:304] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 508ff162d05e44159e225081ede82882; no voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:15.625825 27243 raft_consensus.cc:2749] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:15.626335 27000 leader_election.cc:304] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5604f5b9fef245bca9665c54cdd0d967; no voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882
I20260709 12:17:15.627123 27072 leader_election.cc:304] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 508ff162d05e44159e225081ede82882; no voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:15.627318 27243 raft_consensus.cc:2749] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:15.628657 27254 raft_consensus.cc:2749] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:15.629424 27254 raft_consensus.cc:2749] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:15.632189 27252 raft_consensus.cc:515] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:15.633363 26957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "afc42e2e5dab4df88c1c00bf9df243e0" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:15.634635 27252 leader_election.cc:290] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 3 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:15.635433 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:15.635963 27072 leader_election.cc:304] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 508ff162d05e44159e225081ede82882; no voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:15.636463 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:15.636893 27248 raft_consensus.cc:2749] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:15.638412 27000 leader_election.cc:304] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5604f5b9fef245bca9665c54cdd0d967; no voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882
I20260709 12:17:15.639318 27242 raft_consensus.cc:2749] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:15.639029 27245 raft_consensus.cc:515] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:15.641484 27245 leader_election.cc:290] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 3 election: Requested vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:15.642596 27034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:15.642709 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:15.645001 26922 leader_election.cc:304] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38218df822dc435eac030105d4f51140; no voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:15.645831 27245 raft_consensus.cc:2749] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260709 12:17:15.906306 26957 raft_consensus.cc:1240] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 3 FOLLOWER]: Rejecting Update request from peer 5604f5b9fef245bca9665c54cdd0d967 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:15.908576 27242 consensus_queue.cc:1059] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:15.909811 27242 raft_consensus.cc:3055] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:15.910269 27242 raft_consensus.cc:740] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Running, Role: LEADER
I20260709 12:17:15.911329 27242 consensus_queue.cc:260] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:15.915020 27242 raft_consensus.cc:3060] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:15.916536 27034 raft_consensus.cc:1240] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 3 FOLLOWER]: Rejecting Update request from peer 38218df822dc435eac030105d4f51140 for earlier term 2. Current term is 3. Ops: []
I20260709 12:17:15.918200 27245 consensus_queue.cc:1059] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:15.919432 27244 raft_consensus.cc:3055] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:15.919836 27244 raft_consensus.cc:740] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Running, Role: LEADER
I20260709 12:17:15.920913 27244 consensus_queue.cc:260] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:15.922438 27244 raft_consensus.cc:3060] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:15.925462 26854 catalog_manager.cc:2283] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44114:
name: "soft_deleted_table"
schema {
  columns {
    name: "key"
    type: INT32
    is_key: true
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "int_val"
    type: INT32
    is_key: false
    is_nullable: false
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
  columns {
    name: "string_val"
    type: STRING
    is_key: false
    is_nullable: true
    encoding: AUTO_ENCODING
    compression: DEFAULT_COMPRESSION
    cfile_block_size: 0
    immutable: false
  }
}
num_replicas: 3
split_rows_range_bounds {
  rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
  indirect_data: """"
}
partition_schema {
  range_schema {
    columns {
      name: "key"
    }
  }
}
W20260709 12:17:15.930192 26854 catalog_manager.cc:7077] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table soft_deleted_table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260709 12:17:15.988751 26946 tablet_service.cc:1511] Processing CreateTablet for tablet c8c93363b18b455696ca407b4dbde553 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:17:15.990675 26946 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8c93363b18b455696ca407b4dbde553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:15.991946 27021 tablet_service.cc:1511] Processing CreateTablet for tablet c8c93363b18b455696ca407b4dbde553 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:17:15.992259 27097 tablet_service.cc:1511] Processing CreateTablet for tablet c8c93363b18b455696ca407b4dbde553 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260709 12:17:15.993666 27021 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8c93363b18b455696ca407b4dbde553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:15.994271 27097 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet c8c93363b18b455696ca407b4dbde553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:15.996877 26949 tablet_service.cc:1511] Processing CreateTablet for tablet 59af62b8772640f79014b776aa0a2906 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:17:15.998366 26949 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59af62b8772640f79014b776aa0a2906. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:15.999976 27022 tablet_service.cc:1511] Processing CreateTablet for tablet 59af62b8772640f79014b776aa0a2906 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:17:16.000429 27095 tablet_service.cc:1511] Processing CreateTablet for tablet 59af62b8772640f79014b776aa0a2906 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260709 12:17:16.001478 27022 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59af62b8772640f79014b776aa0a2906. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.001888 27095 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 59af62b8772640f79014b776aa0a2906. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.006456 27098 tablet_service.cc:1511] Processing CreateTablet for tablet aab7d9b4c30c466fa1e5451e42820d75 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:17:16.008324 27098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aab7d9b4c30c466fa1e5451e42820d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.009307 26945 tablet_service.cc:1511] Processing CreateTablet for tablet aab7d9b4c30c466fa1e5451e42820d75 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:17:16.012972 27096 tablet_service.cc:1511] Processing CreateTablet for tablet fd773572db7849348131c09a3a18184c (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:17:16.014741 27096 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd773572db7849348131c09a3a18184c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.015177 26945 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aab7d9b4c30c466fa1e5451e42820d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.020013 27094 tablet_service.cc:1511] Processing CreateTablet for tablet d5cc002263e547a8a5da289d103ea955 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:17:16.021808 27094 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5cc002263e547a8a5da289d103ea955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.027643 27024 tablet_service.cc:1511] Processing CreateTablet for tablet aab7d9b4c30c466fa1e5451e42820d75 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260709 12:17:16.029217 27024 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet aab7d9b4c30c466fa1e5451e42820d75. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.025435 26947 tablet_service.cc:1511] Processing CreateTablet for tablet fd773572db7849348131c09a3a18184c (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:17:16.030674 27023 tablet_service.cc:1511] Processing CreateTablet for tablet fd773572db7849348131c09a3a18184c (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260709 12:17:16.032264 27023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd773572db7849348131c09a3a18184c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.033598 26947 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet fd773572db7849348131c09a3a18184c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.035516 26948 tablet_service.cc:1511] Processing CreateTablet for tablet d5cc002263e547a8a5da289d103ea955 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:17:16.037238 26948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5cc002263e547a8a5da289d103ea955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.039276 27093 tablet_service.cc:1511] Processing CreateTablet for tablet 58ea17416b1642b2820a501561e9dd68 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:17:16.041163 27093 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58ea17416b1642b2820a501561e9dd68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.045810 26944 tablet_service.cc:1511] Processing CreateTablet for tablet 58ea17416b1642b2820a501561e9dd68 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:17:16.047644 27020 tablet_service.cc:1511] Processing CreateTablet for tablet d5cc002263e547a8a5da289d103ea955 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260709 12:17:16.049587 27020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet d5cc002263e547a8a5da289d103ea955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.047802 26944 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58ea17416b1642b2820a501561e9dd68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.070181 27019 tablet_service.cc:1511] Processing CreateTablet for tablet 58ea17416b1642b2820a501561e9dd68 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260709 12:17:16.071272 26943 tablet_service.cc:1511] Processing CreateTablet for tablet 30203b8c002948178d4966017ba69c86 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:17:16.075496 26943 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30203b8c002948178d4966017ba69c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.084872 27019 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 58ea17416b1642b2820a501561e9dd68. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.144436 27265 tablet_bootstrap.cc:492] T aab7d9b4c30c466fa1e5451e42820d75 P 5604f5b9fef245bca9665c54cdd0d967: Bootstrap starting.
I20260709 12:17:16.144126 27092 tablet_service.cc:1511] Processing CreateTablet for tablet 30203b8c002948178d4966017ba69c86 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:17:16.144847 27018 tablet_service.cc:1511] Processing CreateTablet for tablet 30203b8c002948178d4966017ba69c86 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260709 12:17:16.145867 27092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30203b8c002948178d4966017ba69c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.146414 27018 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 30203b8c002948178d4966017ba69c86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.146037 26942 tablet_service.cc:1511] Processing CreateTablet for tablet bfc3afe51dc249278d04002e83162998 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:17:16.147589 26942 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfc3afe51dc249278d04002e83162998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.143074 27266 tablet_bootstrap.cc:492] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140: Bootstrap starting.
I20260709 12:17:16.176196 27267 tablet_bootstrap.cc:492] T d5cc002263e547a8a5da289d103ea955 P 508ff162d05e44159e225081ede82882: Bootstrap starting.
I20260709 12:17:16.187549 26948 tablet_service.cc:1511] Processing CreateTablet for tablet 2bccc8c7fe754ec49743359441654476 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:17:16.197260 26948 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bccc8c7fe754ec49743359441654476. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.199088 27265 tablet_bootstrap.cc:654] T aab7d9b4c30c466fa1e5451e42820d75 P 5604f5b9fef245bca9665c54cdd0d967: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.203264 27266 tablet_bootstrap.cc:654] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.179512 27020 tablet_service.cc:1511] Processing CreateTablet for tablet bfc3afe51dc249278d04002e83162998 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:17:16.207940 27020 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfc3afe51dc249278d04002e83162998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.214200 27267 tablet_bootstrap.cc:654] T d5cc002263e547a8a5da289d103ea955 P 508ff162d05e44159e225081ede82882: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.214486 27023 tablet_service.cc:1511] Processing CreateTablet for tablet 2bccc8c7fe754ec49743359441654476 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:17:16.188916 27092 tablet_service.cc:1511] Processing CreateTablet for tablet bfc3afe51dc249278d04002e83162998 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260709 12:17:16.220269 27092 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet bfc3afe51dc249278d04002e83162998. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.247795 27098 tablet_service.cc:1511] Processing CreateTablet for tablet 2bccc8c7fe754ec49743359441654476 (DEFAULT_TABLE table=soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260709 12:17:16.249521 27098 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bccc8c7fe754ec49743359441654476. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.253350 27023 data_dirs.cc:400] Could only allocate 1 dirs of requested 3 for tablet 2bccc8c7fe754ec49743359441654476. 1 dirs total, 0 dirs full, 0 dirs failed
I20260709 12:17:16.254081 27265 tablet_bootstrap.cc:492] T aab7d9b4c30c466fa1e5451e42820d75 P 5604f5b9fef245bca9665c54cdd0d967: No bootstrap required, opened a new log
I20260709 12:17:16.254804 27265 ts_tablet_manager.cc:1403] T aab7d9b4c30c466fa1e5451e42820d75 P 5604f5b9fef245bca9665c54cdd0d967: Time spent bootstrapping tablet: real 0.111s	user 0.015s	sys 0.018s
I20260709 12:17:16.262753 27265 raft_consensus.cc:359] T aab7d9b4c30c466fa1e5451e42820d75 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.263576 27265 raft_consensus.cc:385] T aab7d9b4c30c466fa1e5451e42820d75 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.263913 27265 raft_consensus.cc:740] T aab7d9b4c30c466fa1e5451e42820d75 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.264915 27265 consensus_queue.cc:260] T aab7d9b4c30c466fa1e5451e42820d75 P 5604f5b9fef245bca9665c54cdd0d967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.270236 27266 tablet_bootstrap.cc:492] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140: No bootstrap required, opened a new log
I20260709 12:17:16.270546 27265 ts_tablet_manager.cc:1434] T aab7d9b4c30c466fa1e5451e42820d75 P 5604f5b9fef245bca9665c54cdd0d967: Time spent starting tablet: real 0.015s	user 0.008s	sys 0.002s
I20260709 12:17:16.271000 27266 ts_tablet_manager.cc:1403] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140: Time spent bootstrapping tablet: real 0.141s	user 0.018s	sys 0.010s
I20260709 12:17:16.271391 27267 tablet_bootstrap.cc:492] T d5cc002263e547a8a5da289d103ea955 P 508ff162d05e44159e225081ede82882: No bootstrap required, opened a new log
I20260709 12:17:16.272083 27267 ts_tablet_manager.cc:1403] T d5cc002263e547a8a5da289d103ea955 P 508ff162d05e44159e225081ede82882: Time spent bootstrapping tablet: real 0.109s	user 0.012s	sys 0.011s
I20260709 12:17:16.272279 27265 tablet_bootstrap.cc:492] T 59af62b8772640f79014b776aa0a2906 P 5604f5b9fef245bca9665c54cdd0d967: Bootstrap starting.
I20260709 12:17:16.274874 27266 raft_consensus.cc:359] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.275734 27266 raft_consensus.cc:385] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.276049 27266 raft_consensus.cc:740] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.276889 27266 consensus_queue.cc:260] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.276589 27267 raft_consensus.cc:359] T d5cc002263e547a8a5da289d103ea955 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.281970 27267 raft_consensus.cc:385] T d5cc002263e547a8a5da289d103ea955 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.282413 27267 raft_consensus.cc:740] T d5cc002263e547a8a5da289d103ea955 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.283370 27267 consensus_queue.cc:260] T d5cc002263e547a8a5da289d103ea955 P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.283463 27265 tablet_bootstrap.cc:654] T 59af62b8772640f79014b776aa0a2906 P 5604f5b9fef245bca9665c54cdd0d967: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.295850 27265 tablet_bootstrap.cc:492] T 59af62b8772640f79014b776aa0a2906 P 5604f5b9fef245bca9665c54cdd0d967: No bootstrap required, opened a new log
I20260709 12:17:16.296373 27267 ts_tablet_manager.cc:1434] T d5cc002263e547a8a5da289d103ea955 P 508ff162d05e44159e225081ede82882: Time spent starting tablet: real 0.024s	user 0.008s	sys 0.000s
I20260709 12:17:16.296492 27265 ts_tablet_manager.cc:1403] T 59af62b8772640f79014b776aa0a2906 P 5604f5b9fef245bca9665c54cdd0d967: Time spent bootstrapping tablet: real 0.024s	user 0.010s	sys 0.005s
I20260709 12:17:16.297940 27267 tablet_bootstrap.cc:492] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882: Bootstrap starting.
I20260709 12:17:16.301076 27265 raft_consensus.cc:359] T 59af62b8772640f79014b776aa0a2906 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.301999 27265 raft_consensus.cc:385] T 59af62b8772640f79014b776aa0a2906 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.302476 27265 raft_consensus.cc:740] T 59af62b8772640f79014b776aa0a2906 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.304680 27266 ts_tablet_manager.cc:1434] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140: Time spent starting tablet: real 0.033s	user 0.008s	sys 0.000s
I20260709 12:17:16.311987 27265 consensus_queue.cc:260] T 59af62b8772640f79014b776aa0a2906 P 5604f5b9fef245bca9665c54cdd0d967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.309122 27267 tablet_bootstrap.cc:654] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.315418 27265 ts_tablet_manager.cc:1434] T 59af62b8772640f79014b776aa0a2906 P 5604f5b9fef245bca9665c54cdd0d967: Time spent starting tablet: real 0.018s	user 0.009s	sys 0.000s
I20260709 12:17:16.316669 27265 tablet_bootstrap.cc:492] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967: Bootstrap starting.
I20260709 12:17:16.322175 27266 tablet_bootstrap.cc:492] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140: Bootstrap starting.
I20260709 12:17:16.325551 27267 tablet_bootstrap.cc:492] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882: No bootstrap required, opened a new log
I20260709 12:17:16.325994 27265 tablet_bootstrap.cc:654] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.326264 27267 ts_tablet_manager.cc:1403] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882: Time spent bootstrapping tablet: real 0.029s	user 0.018s	sys 0.002s
I20260709 12:17:16.330138 27267 raft_consensus.cc:359] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.330865 27267 raft_consensus.cc:385] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.331168 27267 raft_consensus.cc:740] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.332125 27267 consensus_queue.cc:260] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.334465 27266 tablet_bootstrap.cc:654] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.335281 27267 ts_tablet_manager.cc:1434] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882: Time spent starting tablet: real 0.008s	user 0.008s	sys 0.000s
I20260709 12:17:16.338677 27267 tablet_bootstrap.cc:492] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882: Bootstrap starting.
I20260709 12:17:16.340279 27265 tablet_bootstrap.cc:492] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967: No bootstrap required, opened a new log
I20260709 12:17:16.341012 27265 ts_tablet_manager.cc:1403] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967: Time spent bootstrapping tablet: real 0.025s	user 0.015s	sys 0.006s
I20260709 12:17:16.345127 27265 raft_consensus.cc:359] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.346041 27265 raft_consensus.cc:385] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.346451 27265 raft_consensus.cc:740] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.347343 27265 consensus_queue.cc:260] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.348929 27267 tablet_bootstrap.cc:654] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.350659 27265 ts_tablet_manager.cc:1434] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:17:16.351955 27265 tablet_bootstrap.cc:492] T c8c93363b18b455696ca407b4dbde553 P 5604f5b9fef245bca9665c54cdd0d967: Bootstrap starting.
I20260709 12:17:16.358731 27267 tablet_bootstrap.cc:492] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882: No bootstrap required, opened a new log
I20260709 12:17:16.359400 27267 ts_tablet_manager.cc:1403] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882: Time spent bootstrapping tablet: real 0.021s	user 0.016s	sys 0.003s
I20260709 12:17:16.359429 27265 tablet_bootstrap.cc:654] T c8c93363b18b455696ca407b4dbde553 P 5604f5b9fef245bca9665c54cdd0d967: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.362701 27266 tablet_bootstrap.cc:492] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140: No bootstrap required, opened a new log
I20260709 12:17:16.363396 27266 ts_tablet_manager.cc:1403] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140: Time spent bootstrapping tablet: real 0.042s	user 0.017s	sys 0.013s
I20260709 12:17:16.363341 27267 raft_consensus.cc:359] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.364362 27267 raft_consensus.cc:385] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.364852 27267 raft_consensus.cc:740] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.365850 27267 consensus_queue.cc:260] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.367275 27266 raft_consensus.cc:359] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.368273 27266 raft_consensus.cc:385] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.368417 27267 ts_tablet_manager.cc:1434] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882: Time spent starting tablet: real 0.009s	user 0.008s	sys 0.000s
I20260709 12:17:16.368847 27266 raft_consensus.cc:740] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.369858 27267 tablet_bootstrap.cc:492] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882: Bootstrap starting.
I20260709 12:17:16.369853 27266 consensus_queue.cc:260] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.379424 27267 tablet_bootstrap.cc:654] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.384811 27266 ts_tablet_manager.cc:1434] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140: Time spent starting tablet: real 0.021s	user 0.009s	sys 0.000s
I20260709 12:17:16.391140 27266 tablet_bootstrap.cc:492] T d5cc002263e547a8a5da289d103ea955 P 38218df822dc435eac030105d4f51140: Bootstrap starting.
I20260709 12:17:16.397753 27265 tablet_bootstrap.cc:492] T c8c93363b18b455696ca407b4dbde553 P 5604f5b9fef245bca9665c54cdd0d967: No bootstrap required, opened a new log
I20260709 12:17:16.398370 27265 ts_tablet_manager.cc:1403] T c8c93363b18b455696ca407b4dbde553 P 5604f5b9fef245bca9665c54cdd0d967: Time spent bootstrapping tablet: real 0.047s	user 0.021s	sys 0.023s
I20260709 12:17:16.400965 27265 raft_consensus.cc:359] T c8c93363b18b455696ca407b4dbde553 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.401655 27265 raft_consensus.cc:385] T c8c93363b18b455696ca407b4dbde553 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.402012 27265 raft_consensus.cc:740] T c8c93363b18b455696ca407b4dbde553 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.402631 27266 tablet_bootstrap.cc:654] T d5cc002263e547a8a5da289d103ea955 P 38218df822dc435eac030105d4f51140: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.403241 27265 consensus_queue.cc:260] T c8c93363b18b455696ca407b4dbde553 P 5604f5b9fef245bca9665c54cdd0d967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.405143 27267 tablet_bootstrap.cc:492] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882: No bootstrap required, opened a new log
I20260709 12:17:16.405815 27265 ts_tablet_manager.cc:1434] T c8c93363b18b455696ca407b4dbde553 P 5604f5b9fef245bca9665c54cdd0d967: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.000s
I20260709 12:17:16.406302 27267 ts_tablet_manager.cc:1403] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882: Time spent bootstrapping tablet: real 0.037s	user 0.019s	sys 0.016s
I20260709 12:17:16.407387 27265 tablet_bootstrap.cc:492] T fd773572db7849348131c09a3a18184c P 5604f5b9fef245bca9665c54cdd0d967: Bootstrap starting.
I20260709 12:17:16.411604 27267 raft_consensus.cc:359] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.413699 27267 raft_consensus.cc:385] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.414521 27267 raft_consensus.cc:740] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.415899 27267 consensus_queue.cc:260] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.433858 27267 ts_tablet_manager.cc:1434] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882: Time spent starting tablet: real 0.027s	user 0.010s	sys 0.000s
I20260709 12:17:16.435464 27267 tablet_bootstrap.cc:492] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882: Bootstrap starting.
I20260709 12:17:16.436082 27266 tablet_bootstrap.cc:492] T d5cc002263e547a8a5da289d103ea955 P 38218df822dc435eac030105d4f51140: No bootstrap required, opened a new log
I20260709 12:17:16.436947 27266 ts_tablet_manager.cc:1403] T d5cc002263e547a8a5da289d103ea955 P 38218df822dc435eac030105d4f51140: Time spent bootstrapping tablet: real 0.046s	user 0.021s	sys 0.000s
I20260709 12:17:16.441721 27265 tablet_bootstrap.cc:654] T fd773572db7849348131c09a3a18184c P 5604f5b9fef245bca9665c54cdd0d967: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.443864 27266 raft_consensus.cc:359] T d5cc002263e547a8a5da289d103ea955 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.444595 27266 raft_consensus.cc:385] T d5cc002263e547a8a5da289d103ea955 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.444811 27266 raft_consensus.cc:740] T d5cc002263e547a8a5da289d103ea955 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.445617 27266 consensus_queue.cc:260] T d5cc002263e547a8a5da289d103ea955 P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.449018 27266 ts_tablet_manager.cc:1434] T d5cc002263e547a8a5da289d103ea955 P 38218df822dc435eac030105d4f51140: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.004s
I20260709 12:17:16.450244 27267 tablet_bootstrap.cc:654] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.450757 27266 tablet_bootstrap.cc:492] T 30203b8c002948178d4966017ba69c86 P 38218df822dc435eac030105d4f51140: Bootstrap starting.
I20260709 12:17:16.457276 27265 tablet_bootstrap.cc:492] T fd773572db7849348131c09a3a18184c P 5604f5b9fef245bca9665c54cdd0d967: No bootstrap required, opened a new log
I20260709 12:17:16.457970 27265 ts_tablet_manager.cc:1403] T fd773572db7849348131c09a3a18184c P 5604f5b9fef245bca9665c54cdd0d967: Time spent bootstrapping tablet: real 0.051s	user 0.015s	sys 0.004s
I20260709 12:17:16.462412 27266 tablet_bootstrap.cc:654] T 30203b8c002948178d4966017ba69c86 P 38218df822dc435eac030105d4f51140: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.462718 27265 raft_consensus.cc:359] T fd773572db7849348131c09a3a18184c P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.463572 27265 raft_consensus.cc:385] T fd773572db7849348131c09a3a18184c P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.463908 27265 raft_consensus.cc:740] T fd773572db7849348131c09a3a18184c P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.464819 27265 consensus_queue.cc:260] T fd773572db7849348131c09a3a18184c P 5604f5b9fef245bca9665c54cdd0d967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.467859 27265 ts_tablet_manager.cc:1434] T fd773572db7849348131c09a3a18184c P 5604f5b9fef245bca9665c54cdd0d967: Time spent starting tablet: real 0.009s	user 0.006s	sys 0.002s
I20260709 12:17:16.469537 27265 tablet_bootstrap.cc:492] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967: Bootstrap starting.
I20260709 12:17:16.473830 27267 tablet_bootstrap.cc:492] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882: No bootstrap required, opened a new log
I20260709 12:17:16.475029 27267 ts_tablet_manager.cc:1403] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882: Time spent bootstrapping tablet: real 0.040s	user 0.017s	sys 0.011s
I20260709 12:17:16.478021 27265 tablet_bootstrap.cc:654] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.480834 27242 raft_consensus.cc:493] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.480309 27267 raft_consensus.cc:359] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.481582 27242 raft_consensus.cc:515] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.481738 27267 raft_consensus.cc:385] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.483182 27267 raft_consensus.cc:740] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.484820 27267 consensus_queue.cc:260] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.487730 27266 tablet_bootstrap.cc:492] T 30203b8c002948178d4966017ba69c86 P 38218df822dc435eac030105d4f51140: No bootstrap required, opened a new log
I20260709 12:17:16.488274 27267 ts_tablet_manager.cc:1434] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882: Time spent starting tablet: real 0.013s	user 0.008s	sys 0.005s
I20260709 12:17:16.488358 27266 ts_tablet_manager.cc:1403] T 30203b8c002948178d4966017ba69c86 P 38218df822dc435eac030105d4f51140: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.014s
I20260709 12:17:16.490491 26957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5cc002263e547a8a5da289d103ea955" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:16.495282 26957 raft_consensus.cc:2468] T d5cc002263e547a8a5da289d103ea955 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 0.
I20260709 12:17:16.492184 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5cc002263e547a8a5da289d103ea955" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:16.496896 27108 raft_consensus.cc:2468] T d5cc002263e547a8a5da289d103ea955 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 0.
I20260709 12:17:16.494284 27267 tablet_bootstrap.cc:492] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882: Bootstrap starting.
I20260709 12:17:16.498622 26998 leader_election.cc:304] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:16.499768 27265 tablet_bootstrap.cc:492] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967: No bootstrap required, opened a new log
I20260709 12:17:16.492302 27242 leader_election.cc:290] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:16.499703 27252 raft_consensus.cc:2804] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:16.500475 27265 ts_tablet_manager.cc:1403] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967: Time spent bootstrapping tablet: real 0.031s	user 0.010s	sys 0.007s
I20260709 12:17:16.500710 27252 raft_consensus.cc:493] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.501171 27252 raft_consensus.cc:3060] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.504452 27265 raft_consensus.cc:359] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.505496 27265 raft_consensus.cc:385] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.505815 27265 raft_consensus.cc:740] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.506680 27265 consensus_queue.cc:260] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.508488 27267 tablet_bootstrap.cc:654] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.509714 27265 ts_tablet_manager.cc:1434] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967: Time spent starting tablet: real 0.009s	user 0.007s	sys 0.000s
I20260709 12:17:16.504522 27266 raft_consensus.cc:359] T 30203b8c002948178d4966017ba69c86 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.510919 27266 raft_consensus.cc:385] T 30203b8c002948178d4966017ba69c86 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.511250 27266 raft_consensus.cc:740] T 30203b8c002948178d4966017ba69c86 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.511317 27252 raft_consensus.cc:515] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.512181 27266 consensus_queue.cc:260] T 30203b8c002948178d4966017ba69c86 P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.513842 27245 raft_consensus.cc:493] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.514840 27266 ts_tablet_manager.cc:1434] T 30203b8c002948178d4966017ba69c86 P 38218df822dc435eac030105d4f51140: Time spent starting tablet: real 0.020s	user 0.004s	sys 0.004s
I20260709 12:17:16.514602 27245 raft_consensus.cc:515] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.516077 27266 tablet_bootstrap.cc:492] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140: Bootstrap starting.
I20260709 12:17:16.517587 26957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5cc002263e547a8a5da289d103ea955" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:16.518316 26957 raft_consensus.cc:3060] T d5cc002263e547a8a5da289d103ea955 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Advancing to term 1
W20260709 12:17:16.523007 27129 debug-util.cc:398] Leaking SignalData structure 0x7b080033efe0 after lost signal to thread 11321
I20260709 12:17:16.523411 27265 tablet_bootstrap.cc:492] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967: Bootstrap starting.
I20260709 12:17:16.528506 27266 tablet_bootstrap.cc:654] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:16.528684 27129 debug-util.cc:398] Leaking SignalData structure 0x7b080012e060 after lost signal to thread 26983
I20260709 12:17:16.532812 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aab7d9b4c30c466fa1e5451e42820d75" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:16.533880 27108 raft_consensus.cc:2468] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 0.
I20260709 12:17:16.534742 26957 raft_consensus.cc:2468] T d5cc002263e547a8a5da289d103ea955 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 1.
I20260709 12:17:16.535835 27267 tablet_bootstrap.cc:492] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882: No bootstrap required, opened a new log
I20260709 12:17:16.535583 26925 leader_election.cc:304] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882; no voters: 
I20260709 12:17:16.536199 26998 leader_election.cc:304] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:16.536499 27267 ts_tablet_manager.cc:1403] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882: Time spent bootstrapping tablet: real 0.043s	user 0.016s	sys 0.005s
I20260709 12:17:16.537279 27242 raft_consensus.cc:2804] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:16.537670 27248 raft_consensus.cc:493] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.538285 27265 tablet_bootstrap.cc:654] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.538280 27248 raft_consensus.cc:515] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
W20260709 12:17:16.540357 26905 auto_rebalancer.cc:288] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260709 12:17:16.541618 27034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aab7d9b4c30c466fa1e5451e42820d75" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:16.542426 27034 raft_consensus.cc:2468] T aab7d9b4c30c466fa1e5451e42820d75 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 0.
I20260709 12:17:16.542932 27248 leader_election.cc:290] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:16.543442 27031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af62b8772640f79014b776aa0a2906" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:16.544422 27031 raft_consensus.cc:2468] T 59af62b8772640f79014b776aa0a2906 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 0.
I20260709 12:17:16.546550 27071 leader_election.cc:304] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:16.548780 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "d5cc002263e547a8a5da289d103ea955" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:16.548655 27267 raft_consensus.cc:359] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.549180 27252 leader_election.cc:290] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:16.550359 27285 raft_consensus.cc:2804] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:16.550843 27285 raft_consensus.cc:493] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.551221 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af62b8772640f79014b776aa0a2906" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
W20260709 12:17:16.552891 27072 leader_election.cc:343] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38218df822dc435eac030105d4f51140 (127.11.13.129:36041): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:16.553092 27242 raft_consensus.cc:697] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 LEADER]: Becoming Leader. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Running, Role: LEADER
I20260709 12:17:16.553984 27248 raft_consensus.cc:2804] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:16.554651 27248 raft_consensus.cc:493] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.554219 27242 consensus_queue.cc:237] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.555140 27248 raft_consensus.cc:3060] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.555347 27265 tablet_bootstrap.cc:492] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967: No bootstrap required, opened a new log
I20260709 12:17:16.556223 27265 ts_tablet_manager.cc:1403] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967: Time spent bootstrapping tablet: real 0.033s	user 0.009s	sys 0.010s
I20260709 12:17:16.556684 27245 leader_election.cc:290] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:16.551219 27285 raft_consensus.cc:3060] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.560734 27265 raft_consensus.cc:359] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.561949 27265 raft_consensus.cc:385] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.562435 27265 raft_consensus.cc:740] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.549633 27267 raft_consensus.cc:385] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.564406 27267 raft_consensus.cc:740] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.563726 27265 consensus_queue.cc:260] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
W20260709 12:17:16.566622 27129 debug-util.cc:398] Leaking SignalData structure 0x7b08000f8360 after lost signal to thread 27058
I20260709 12:17:16.566341 27267 consensus_queue.cc:260] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.566975 27265 ts_tablet_manager.cc:1434] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967: Time spent starting tablet: real 0.010s	user 0.011s	sys 0.000s
I20260709 12:17:16.568228 27266 tablet_bootstrap.cc:492] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140: No bootstrap required, opened a new log
I20260709 12:17:16.568746 27265 tablet_bootstrap.cc:492] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967: Bootstrap starting.
I20260709 12:17:16.568809 27266 ts_tablet_manager.cc:1403] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140: Time spent bootstrapping tablet: real 0.053s	user 0.023s	sys 0.009s
I20260709 12:17:16.549574 27108 raft_consensus.cc:3060] T d5cc002263e547a8a5da289d103ea955 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.572822 27266 raft_consensus.cc:359] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.573686 27266 raft_consensus.cc:385] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.574030 27266 raft_consensus.cc:740] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.575996 27266 consensus_queue.cc:260] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.576165 27285 raft_consensus.cc:515] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.577147 27265 tablet_bootstrap.cc:654] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.579566 27031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aab7d9b4c30c466fa1e5451e42820d75" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:16.580340 27031 raft_consensus.cc:3060] T aab7d9b4c30c466fa1e5451e42820d75 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.585426 27266 ts_tablet_manager.cc:1434] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140: Time spent starting tablet: real 0.016s	user 0.006s	sys 0.008s
I20260709 12:17:16.586690 27266 tablet_bootstrap.cc:492] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140: Bootstrap starting.
W20260709 12:17:16.590977 27129 debug-util.cc:398] Leaking SignalData structure 0x7b08001db000 after lost signal to thread 27132
I20260709 12:17:16.593226 27031 raft_consensus.cc:2468] T aab7d9b4c30c466fa1e5451e42820d75 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 1.
I20260709 12:17:16.595820 27285 leader_election.cc:290] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 election: Requested vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:16.596271 27265 tablet_bootstrap.cc:492] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967: No bootstrap required, opened a new log
I20260709 12:17:16.595832 26922 leader_election.cc:304] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:16.596870 27265 ts_tablet_manager.cc:1403] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967: Time spent bootstrapping tablet: real 0.029s	user 0.021s	sys 0.004s
I20260709 12:17:16.597007 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "aab7d9b4c30c466fa1e5451e42820d75" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:16.597126 27285 raft_consensus.cc:2804] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:16.597396 27267 ts_tablet_manager.cc:1434] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882: Time spent starting tablet: real 0.060s	user 0.009s	sys 0.001s
I20260709 12:17:16.597781 27107 raft_consensus.cc:3060] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.597903 27285 raft_consensus.cc:697] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Becoming Leader. State: Replica: 38218df822dc435eac030105d4f51140, State: Running, Role: LEADER
I20260709 12:17:16.599145 27267 tablet_bootstrap.cc:492] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882: Bootstrap starting.
I20260709 12:17:16.599087 27285 consensus_queue.cc:237] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.600951 27265 raft_consensus.cc:359] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.601737 27265 raft_consensus.cc:385] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.602056 27265 raft_consensus.cc:740] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.602955 27266 tablet_bootstrap.cc:654] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.604704 27265 consensus_queue.cc:260] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.607170 27265 ts_tablet_manager.cc:1434] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967: Time spent starting tablet: real 0.010s	user 0.010s	sys 0.000s
I20260709 12:17:16.612229 27265 tablet_bootstrap.cc:492] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967: Bootstrap starting.
I20260709 12:17:16.617738 27107 raft_consensus.cc:2468] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 1.
I20260709 12:17:16.624184 27108 raft_consensus.cc:2468] T d5cc002263e547a8a5da289d103ea955 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 1.
I20260709 12:17:16.638173 26856 catalog_manager.cc:5697] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130). New cstate: current_term: 1 leader_uuid: "5604f5b9fef245bca9665c54cdd0d967" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:16.640681 27267 tablet_bootstrap.cc:654] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.644797 27265 tablet_bootstrap.cc:654] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.637827 27248 raft_consensus.cc:515] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.650981 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af62b8772640f79014b776aa0a2906" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:16.651961 26959 raft_consensus.cc:3060] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.656450 27248 leader_election.cc:290] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:16.656078 27031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af62b8772640f79014b776aa0a2906" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:16.657765 27031 raft_consensus.cc:3060] T 59af62b8772640f79014b776aa0a2906 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.669438 27031 raft_consensus.cc:2468] T 59af62b8772640f79014b776aa0a2906 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 1.
I20260709 12:17:16.681919 26959 raft_consensus.cc:2468] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 1.
I20260709 12:17:16.683511 27071 leader_election.cc:304] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:16.685848 27248 raft_consensus.cc:2804] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:16.686609 27248 raft_consensus.cc:697] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Becoming Leader. State: Replica: 508ff162d05e44159e225081ede82882, State: Running, Role: LEADER
I20260709 12:17:16.689400 27266 tablet_bootstrap.cc:492] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140: No bootstrap required, opened a new log
I20260709 12:17:16.690001 27266 ts_tablet_manager.cc:1403] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140: Time spent bootstrapping tablet: real 0.104s	user 0.015s	sys 0.026s
I20260709 12:17:16.694413 27267 tablet_bootstrap.cc:492] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882: No bootstrap required, opened a new log
I20260709 12:17:16.687970 27248 consensus_queue.cc:237] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.695060 27267 ts_tablet_manager.cc:1403] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882: Time spent bootstrapping tablet: real 0.096s	user 0.018s	sys 0.002s
I20260709 12:17:16.695791 27266 raft_consensus.cc:359] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.697036 27266 raft_consensus.cc:385] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.697714 27266 raft_consensus.cc:740] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.699363 27267 raft_consensus.cc:359] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.699244 27266 consensus_queue.cc:260] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.701447 27285 raft_consensus.cc:493] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.702224 27285 raft_consensus.cc:515] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.703955 27289 raft_consensus.cc:493] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.704783 27242 raft_consensus.cc:493] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.705374 27242 raft_consensus.cc:515] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.704834 27289 raft_consensus.cc:515] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.709103 27031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8c93363b18b455696ca407b4dbde553" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:16.710073 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30203b8c002948178d4966017ba69c86" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:16.710906 26959 raft_consensus.cc:2468] T 30203b8c002948178d4966017ba69c86 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 0.
I20260709 12:17:16.711028 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8c93363b18b455696ca407b4dbde553" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:16.711418 27242 leader_election.cc:290] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:16.711984 27107 raft_consensus.cc:2468] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 0.
I20260709 12:17:16.712756 27072 leader_election.cc:304] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882; no voters: 
I20260709 12:17:16.710261 27031 raft_consensus.cc:2468] T c8c93363b18b455696ca407b4dbde553 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 0.
I20260709 12:17:16.700405 27267 raft_consensus.cc:385] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.718750 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30203b8c002948178d4966017ba69c86" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:16.719977 27267 raft_consensus.cc:740] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.720687 27107 raft_consensus.cc:2468] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 0.
I20260709 12:17:16.719352 26922 leader_election.cc:304] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:16.721753 27289 leader_election.cc:290] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:16.719661 27265 tablet_bootstrap.cc:492] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967: No bootstrap required, opened a new log
I20260709 12:17:16.725898 27265 ts_tablet_manager.cc:1403] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967: Time spent bootstrapping tablet: real 0.118s	user 0.017s	sys 0.018s
I20260709 12:17:16.727360 27293 raft_consensus.cc:2804] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:16.727886 27293 raft_consensus.cc:493] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.728341 27293 raft_consensus.cc:3060] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.728528 27000 leader_election.cc:304] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:16.728555 27031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30203b8c002948178d4966017ba69c86" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:16.729534 27031 raft_consensus.cc:2468] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 0.
I20260709 12:17:16.719035 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30203b8c002948178d4966017ba69c86" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:16.730098 27242 raft_consensus.cc:2804] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:16.731951 26959 raft_consensus.cc:2468] T 30203b8c002948178d4966017ba69c86 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 0.
I20260709 12:17:16.732028 27242 raft_consensus.cc:493] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.730225 27289 raft_consensus.cc:2804] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:16.735608 27289 raft_consensus.cc:493] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.735994 27289 raft_consensus.cc:3060] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.736166 27265 raft_consensus.cc:359] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.737337 27265 raft_consensus.cc:385] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.723867 27267 consensus_queue.cc:260] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.703539 27266 ts_tablet_manager.cc:1434] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140: Time spent starting tablet: real 0.013s	user 0.010s	sys 0.000s
I20260709 12:17:16.714874 27285 leader_election.cc:290] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:16.735137 27242 raft_consensus.cc:3060] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.737731 27265 raft_consensus.cc:740] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.741189 27265 consensus_queue.cc:260] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.743754 27265 ts_tablet_manager.cc:1434] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967: Time spent starting tablet: real 0.017s	user 0.009s	sys 0.000s
I20260709 12:17:16.747550 27293 raft_consensus.cc:515] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.749553 27242 raft_consensus.cc:515] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.751345 27031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8c93363b18b455696ca407b4dbde553" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:16.752043 27031 raft_consensus.cc:3060] T c8c93363b18b455696ca407b4dbde553 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.752220 27267 ts_tablet_manager.cc:1434] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882: Time spent starting tablet: real 0.057s	user 0.010s	sys 0.000s
I20260709 12:17:16.752828 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30203b8c002948178d4966017ba69c86" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:16.753423 26959 raft_consensus.cc:3060] T 30203b8c002948178d4966017ba69c86 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.755781 27266 tablet_bootstrap.cc:492] T bfc3afe51dc249278d04002e83162998 P 38218df822dc435eac030105d4f51140: Bootstrap starting.
I20260709 12:17:16.753450 27267 tablet_bootstrap.cc:492] T bfc3afe51dc249278d04002e83162998 P 508ff162d05e44159e225081ede82882: Bootstrap starting.
I20260709 12:17:16.761951 26855 catalog_manager.cc:5697] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 reported cstate change: term changed from 0 to 1, leader changed from <none> to 508ff162d05e44159e225081ede82882 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "508ff162d05e44159e225081ede82882" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:16.765471 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30203b8c002948178d4966017ba69c86" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:16.765347 27242 leader_election.cc:290] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:16.771368 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c8c93363b18b455696ca407b4dbde553" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:16.772161 27108 raft_consensus.cc:3060] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.781200 27031 raft_consensus.cc:2468] T c8c93363b18b455696ca407b4dbde553 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 1.
I20260709 12:17:16.782781 26922 leader_election.cc:304] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:16.783900 27285 raft_consensus.cc:2804] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:16.784442 27285 raft_consensus.cc:697] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Becoming Leader. State: Replica: 38218df822dc435eac030105d4f51140, State: Running, Role: LEADER
I20260709 12:17:16.773506 27289 raft_consensus.cc:515] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.786191 27107 raft_consensus.cc:2393] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5604f5b9fef245bca9665c54cdd0d967 in current term 1: Already voted for candidate 508ff162d05e44159e225081ede82882 in this term.
I20260709 12:17:16.785686 27285 consensus_queue.cc:237] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.792136 27289 leader_election.cc:290] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:16.793056 26957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30203b8c002948178d4966017ba69c86" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:16.795043 27293 leader_election.cc:290] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 election: Requested vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:16.794605 26854 catalog_manager.cc:5697] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38218df822dc435eac030105d4f51140 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "38218df822dc435eac030105d4f51140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:16.800256 27289 raft_consensus.cc:493] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.802611 27031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "30203b8c002948178d4966017ba69c86" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:16.808614 26959 raft_consensus.cc:2468] T 30203b8c002948178d4966017ba69c86 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 1.
I20260709 12:17:16.813411 27031 raft_consensus.cc:2393] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 508ff162d05e44159e225081ede82882 in current term 1: Already voted for candidate 5604f5b9fef245bca9665c54cdd0d967 in this term.
I20260709 12:17:16.821519 27300 raft_consensus.cc:493] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.822922 27300 raft_consensus.cc:515] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.813369 27289 raft_consensus.cc:515] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.823715 27071 leader_election.cc:304] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 508ff162d05e44159e225081ede82882; no voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:16.823271 27242 raft_consensus.cc:493] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.825519 27289 leader_election.cc:290] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:16.826522 27300 leader_election.cc:290] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:16.827028 27248 raft_consensus.cc:2749] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260709 12:17:16.827450 27242 raft_consensus.cc:515] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.825342 26998 leader_election.cc:304] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 508ff162d05e44159e225081ede82882
I20260709 12:17:16.829967 27286 raft_consensus.cc:2804] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:16.830410 27031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ea17416b1642b2820a501561e9dd68" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:16.830855 27286 raft_consensus.cc:697] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 LEADER]: Becoming Leader. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Running, Role: LEADER
I20260709 12:17:16.831292 27031 raft_consensus.cc:2468] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 0.
I20260709 12:17:16.831323 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd773572db7849348131c09a3a18184c" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:16.806329 27267 tablet_bootstrap.cc:654] T bfc3afe51dc249278d04002e83162998 P 508ff162d05e44159e225081ede82882: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.829952 27034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd773572db7849348131c09a3a18184c" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:16.831921 27286 consensus_queue.cc:237] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.832862 26959 raft_consensus.cc:2468] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 0.
I20260709 12:17:16.834230 27034 raft_consensus.cc:2468] T fd773572db7849348131c09a3a18184c P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 0.
I20260709 12:17:16.834652 26958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ea17416b1642b2820a501561e9dd68" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:16.835405 27071 leader_election.cc:304] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:16.806811 27108 raft_consensus.cc:2468] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 1.
I20260709 12:17:16.836520 27071 leader_election.cc:304] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:16.836683 27252 raft_consensus.cc:493] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.837421 27248 raft_consensus.cc:2804] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:16.837245 27252 raft_consensus.cc:515] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.837904 27248 raft_consensus.cc:493] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.838378 27248 raft_consensus.cc:3060] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.839900 27252 leader_election.cc:290] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:16.841519 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ea17416b1642b2820a501561e9dd68" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:16.833407 26957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ea17416b1642b2820a501561e9dd68" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:16.842818 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bccc8c7fe754ec49743359441654476" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:16.830440 27242 leader_election.cc:290] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:16.844995 26957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bccc8c7fe754ec49743359441654476" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
W20260709 12:17:16.845027 27000 leader_election.cc:343] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 508ff162d05e44159e225081ede82882 (127.11.13.131:40411): Illegal state: must be running to vote when last-logged opid is not known
W20260709 12:17:16.844849 26998 leader_election.cc:343] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38218df822dc435eac030105d4f51140 (127.11.13.129:36041): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:16.847520 27300 raft_consensus.cc:2804] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:16.848023 27300 raft_consensus.cc:493] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260709 12:17:16.848165 26998 leader_election.cc:343] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38218df822dc435eac030105d4f51140 (127.11.13.129:36041): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:16.848459 27300 raft_consensus.cc:3060] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.848727 26998 leader_election.cc:304] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5604f5b9fef245bca9665c54cdd0d967; no voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882
I20260709 12:17:16.850507 27242 raft_consensus.cc:2749] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:16.832242 27266 tablet_bootstrap.cc:654] T bfc3afe51dc249278d04002e83162998 P 38218df822dc435eac030105d4f51140: Neither blocks nor log segments found. Creating new log.
W20260709 12:17:16.855949 27072 leader_election.cc:343] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 38218df822dc435eac030105d4f51140 (127.11.13.129:36041): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:16.856611 26854 catalog_manager.cc:5697] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38218df822dc435eac030105d4f51140 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "38218df822dc435eac030105d4f51140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:16.865203 27248 raft_consensus.cc:515] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.866914 27108 raft_consensus.cc:2393] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5604f5b9fef245bca9665c54cdd0d967 in current term 1: Already voted for candidate 508ff162d05e44159e225081ede82882 in this term.
I20260709 12:17:16.868600 26957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ea17416b1642b2820a501561e9dd68" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:16.870550 27000 leader_election.cc:304] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5604f5b9fef245bca9665c54cdd0d967; no voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882
W20260709 12:17:16.872251 27072 leader_election.cc:343] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 38218df822dc435eac030105d4f51140 (127.11.13.129:36041): Illegal state: must be running to vote when last-logged opid is not known
I20260709 12:17:16.875017 27248 leader_election.cc:290] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:16.876827 27300 raft_consensus.cc:515] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.877069 27034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ea17416b1642b2820a501561e9dd68" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:16.878083 27286 raft_consensus.cc:3060] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.881322 27300 leader_election.cc:290] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:16.882869 27031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd773572db7849348131c09a3a18184c" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:16.883646 27031 raft_consensus.cc:3060] T fd773572db7849348131c09a3a18184c P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.881448 26957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd773572db7849348131c09a3a18184c" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:16.887252 26957 raft_consensus.cc:3060] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.888381 26854 catalog_manager.cc:5697] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130). New cstate: current_term: 1 leader_uuid: "5604f5b9fef245bca9665c54cdd0d967" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:16.894086 27286 raft_consensus.cc:2749] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:16.899910 26957 raft_consensus.cc:2468] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 1.
I20260709 12:17:16.901712 27072 leader_election.cc:304] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882; no voters: 
I20260709 12:17:16.900306 27267 tablet_bootstrap.cc:492] T bfc3afe51dc249278d04002e83162998 P 508ff162d05e44159e225081ede82882: No bootstrap required, opened a new log
I20260709 12:17:16.903843 27267 ts_tablet_manager.cc:1403] T bfc3afe51dc249278d04002e83162998 P 508ff162d05e44159e225081ede82882: Time spent bootstrapping tablet: real 0.151s	user 0.007s	sys 0.020s
I20260709 12:17:16.903317 27300 raft_consensus.cc:2804] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:16.905166 27300 raft_consensus.cc:697] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Becoming Leader. State: Replica: 508ff162d05e44159e225081ede82882, State: Running, Role: LEADER
I20260709 12:17:16.906374 27266 tablet_bootstrap.cc:492] T bfc3afe51dc249278d04002e83162998 P 38218df822dc435eac030105d4f51140: No bootstrap required, opened a new log
I20260709 12:17:16.907425 27266 ts_tablet_manager.cc:1403] T bfc3afe51dc249278d04002e83162998 P 38218df822dc435eac030105d4f51140: Time spent bootstrapping tablet: real 0.152s	user 0.020s	sys 0.020s
I20260709 12:17:16.906524 27300 consensus_queue.cc:237] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.908811 27267 raft_consensus.cc:359] T bfc3afe51dc249278d04002e83162998 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.910534 27267 raft_consensus.cc:385] T bfc3afe51dc249278d04002e83162998 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.910801 27267 raft_consensus.cc:740] T bfc3afe51dc249278d04002e83162998 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.913058 27267 consensus_queue.cc:260] T bfc3afe51dc249278d04002e83162998 P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.914193 27266 raft_consensus.cc:359] T bfc3afe51dc249278d04002e83162998 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.915117 27266 raft_consensus.cc:385] T bfc3afe51dc249278d04002e83162998 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.915478 27266 raft_consensus.cc:740] T bfc3afe51dc249278d04002e83162998 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.916692 27266 consensus_queue.cc:260] T bfc3afe51dc249278d04002e83162998 P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.917191 27031 raft_consensus.cc:2468] T fd773572db7849348131c09a3a18184c P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 1.
I20260709 12:17:16.919950 27266 ts_tablet_manager.cc:1434] T bfc3afe51dc249278d04002e83162998 P 38218df822dc435eac030105d4f51140: Time spent starting tablet: real 0.012s	user 0.005s	sys 0.002s
I20260709 12:17:16.920809 27034 raft_consensus.cc:2468] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 1.
I20260709 12:17:16.921182 27266 tablet_bootstrap.cc:492] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140: Bootstrap starting.
I20260709 12:17:16.922727 27267 ts_tablet_manager.cc:1434] T bfc3afe51dc249278d04002e83162998 P 508ff162d05e44159e225081ede82882: Time spent starting tablet: real 0.019s	user 0.007s	sys 0.001s
I20260709 12:17:16.926921 26855 catalog_manager.cc:5697] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 reported cstate change: term changed from 0 to 1, leader changed from <none> to 508ff162d05e44159e225081ede82882 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "508ff162d05e44159e225081ede82882" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:16.932339 27267 tablet_bootstrap.cc:492] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882: Bootstrap starting.
I20260709 12:17:16.936777 27071 leader_election.cc:304] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967; no voters: 38218df822dc435eac030105d4f51140
I20260709 12:17:16.939232 27266 tablet_bootstrap.cc:654] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.940306 27300 raft_consensus.cc:2804] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:16.941143 27300 raft_consensus.cc:697] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Becoming Leader. State: Replica: 508ff162d05e44159e225081ede82882, State: Running, Role: LEADER
I20260709 12:17:16.942757 27267 tablet_bootstrap.cc:654] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:16.943722 27286 raft_consensus.cc:493] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:16.942826 27300 consensus_queue.cc:237] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.946007 27286 raft_consensus.cc:515] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.948684 27286 leader_election.cc:290] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:16.949971 26957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc3afe51dc249278d04002e83162998" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:16.950909 26957 raft_consensus.cc:2468] T bfc3afe51dc249278d04002e83162998 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 0.
I20260709 12:17:16.951105 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc3afe51dc249278d04002e83162998" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:16.952106 27108 raft_consensus.cc:2468] T bfc3afe51dc249278d04002e83162998 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 0.
I20260709 12:17:16.952484 26998 leader_election.cc:304] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:16.954560 27286 raft_consensus.cc:2804] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:16.955094 27286 raft_consensus.cc:493] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:16.955596 27286 raft_consensus.cc:3060] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.958221 27266 tablet_bootstrap.cc:492] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140: No bootstrap required, opened a new log
I20260709 12:17:16.958916 27266 ts_tablet_manager.cc:1403] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140: Time spent bootstrapping tablet: real 0.038s	user 0.012s	sys 0.008s
I20260709 12:17:16.962291 27266 raft_consensus.cc:359] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.963285 27266 raft_consensus.cc:385] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.963670 27266 raft_consensus.cc:740] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.964605 27266 consensus_queue.cc:260] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.967509 27286 raft_consensus.cc:515] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.968822 27267 tablet_bootstrap.cc:492] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882: No bootstrap required, opened a new log
I20260709 12:17:16.969630 27267 ts_tablet_manager.cc:1403] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882: Time spent bootstrapping tablet: real 0.038s	user 0.018s	sys 0.012s
I20260709 12:17:16.970789 26957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc3afe51dc249278d04002e83162998" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:16.971491 26957 raft_consensus.cc:3060] T bfc3afe51dc249278d04002e83162998 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.973341 27267 raft_consensus.cc:359] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.974278 27267 raft_consensus.cc:385] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:16.974660 27267 raft_consensus.cc:740] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Initialized, Role: FOLLOWER
I20260709 12:17:16.975534 27267 consensus_queue.cc:260] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.977401 27266 ts_tablet_manager.cc:1434] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140: Time spent starting tablet: real 0.018s	user 0.007s	sys 0.000s
I20260709 12:17:16.977767 27267 ts_tablet_manager.cc:1434] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882: Time spent starting tablet: real 0.008s	user 0.004s	sys 0.000s
I20260709 12:17:16.980311 26957 raft_consensus.cc:2468] T bfc3afe51dc249278d04002e83162998 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 1.
I20260709 12:17:16.982646 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "bfc3afe51dc249278d04002e83162998" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:16.983318 27286 leader_election.cc:290] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:16.984337 27108 raft_consensus.cc:3060] T bfc3afe51dc249278d04002e83162998 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:16.983917 26855 catalog_manager.cc:5697] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 reported cstate change: term changed from 0 to 1, leader changed from <none> to 508ff162d05e44159e225081ede82882 (127.11.13.131). New cstate: current_term: 1 leader_uuid: "508ff162d05e44159e225081ede82882" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:16.985386 26998 leader_election.cc:304] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:16.986608 27286 raft_consensus.cc:2804] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:16.988452 27286 raft_consensus.cc:697] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 LEADER]: Becoming Leader. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Running, Role: LEADER
I20260709 12:17:16.989468 27286 consensus_queue.cc:237] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:16.993573 27108 raft_consensus.cc:2468] T bfc3afe51dc249278d04002e83162998 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 1.
I20260709 12:17:16.983754 27266 tablet_bootstrap.cc:492] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140: Bootstrap starting.
I20260709 12:17:17.002107 27266 tablet_bootstrap.cc:654] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140: Neither blocks nor log segments found. Creating new log.
I20260709 12:17:17.004957 26856 catalog_manager.cc:5697] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130). New cstate: current_term: 1 leader_uuid: "5604f5b9fef245bca9665c54cdd0d967" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:17.012710 27266 tablet_bootstrap.cc:492] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140: No bootstrap required, opened a new log
I20260709 12:17:17.013473 27266 ts_tablet_manager.cc:1403] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140: Time spent bootstrapping tablet: real 0.030s	user 0.015s	sys 0.000s
I20260709 12:17:17.016755 27266 raft_consensus.cc:359] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.017304 27266 raft_consensus.cc:385] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260709 12:17:17.017482 27266 raft_consensus.cc:740] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Initialized, Role: FOLLOWER
I20260709 12:17:17.018276 27266 consensus_queue.cc:260] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.020313 27266 ts_tablet_manager.cc:1434] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140: Time spent starting tablet: real 0.007s	user 0.005s	sys 0.002s
I20260709 12:17:17.039709 27304 consensus_queue.cc:1048] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:17.064839 27286 consensus_queue.cc:1048] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:17.072790 27285 raft_consensus.cc:493] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.073513 27285 raft_consensus.cc:515] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.076709 27285 leader_election.cc:290] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:17.078481 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bccc8c7fe754ec49743359441654476" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:17.079416 27108 raft_consensus.cc:2468] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 0.
I20260709 12:17:17.080456 27293 consensus_queue.cc:1048] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:17.082255 27034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bccc8c7fe754ec49743359441654476" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:17.083447 27034 raft_consensus.cc:2468] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 0.
I20260709 12:17:17.083729 26925 leader_election.cc:304] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882; no voters: 
I20260709 12:17:17.086485 27293 raft_consensus.cc:2804] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260709 12:17:17.091446 27293 raft_consensus.cc:493] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.091918 27293 raft_consensus.cc:3060] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.104319 27285 consensus_queue.cc:1048] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:17.116428 27293 raft_consensus.cc:515] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.121336 27031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bccc8c7fe754ec49743359441654476" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:17.122269 27031 raft_consensus.cc:3060] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.140736 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bccc8c7fe754ec49743359441654476" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:17.141680 27108 raft_consensus.cc:3060] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.154275 27293 leader_election.cc:290] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 election: Requested vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:17.157341 27031 raft_consensus.cc:2468] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 1.
I20260709 12:17:17.159288 26922 leader_election.cc:304] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:17.160403 27293 raft_consensus.cc:2804] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Leader election won for term 1
I20260709 12:17:17.161180 27293 raft_consensus.cc:697] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Becoming Leader. State: Replica: 38218df822dc435eac030105d4f51140, State: Running, Role: LEADER
I20260709 12:17:17.162427 27293 consensus_queue.cc:237] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.164757 27108 raft_consensus.cc:2468] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 1.
I20260709 12:17:17.181880 26855 catalog_manager.cc:5697] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38218df822dc435eac030105d4f51140 (127.11.13.129). New cstate: current_term: 1 leader_uuid: "38218df822dc435eac030105d4f51140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:17.196765 27293 consensus_queue.cc:1048] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:17.211923 27290 consensus_queue.cc:1048] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:17.272233 27300 consensus_queue.cc:1048] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:17.291267 27300 consensus_queue.cc:1048] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:17.298846 27304 consensus_queue.cc:1048] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:17.300591 27300 raft_consensus.cc:493] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5604f5b9fef245bca9665c54cdd0d967)
I20260709 12:17:17.301266 27300 raft_consensus.cc:515] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.305416 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:17.306416 26959 raft_consensus.cc:2393] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 508ff162d05e44159e225081ede82882 in current term 3: Already voted for candidate 38218df822dc435eac030105d4f51140 in this term.
I20260709 12:17:17.303787 27300 leader_election.cc:290] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:17.318864 27031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:17.322464 27031 raft_consensus.cc:2468] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 3.
I20260709 12:17:17.323714 27108 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "59af62b8772640f79014b776aa0a2906"
dest_uuid: "508ff162d05e44159e225081ede82882"
mode: GRACEFUL
new_leader_uuid: "38218df822dc435eac030105d4f51140"
 from {username='slave'} at 127.0.0.1:44468
I20260709 12:17:17.324234 27071 leader_election.cc:304] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967; no voters: 38218df822dc435eac030105d4f51140
I20260709 12:17:17.324712 27108 raft_consensus.cc:606] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Received request to transfer leadership to TS 38218df822dc435eac030105d4f51140
I20260709 12:17:17.325407 27300 raft_consensus.cc:2804] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.325917 27300 raft_consensus.cc:493] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5604f5b9fef245bca9665c54cdd0d967)
I20260709 12:17:17.326332 27300 raft_consensus.cc:3060] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.336208 27300 raft_consensus.cc:515] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.338714 27108 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "fd773572db7849348131c09a3a18184c"
dest_uuid: "508ff162d05e44159e225081ede82882"
mode: GRACEFUL
new_leader_uuid: "38218df822dc435eac030105d4f51140"
 from {username='slave'} at 127.0.0.1:44468
I20260709 12:17:17.339327 26957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:17.339497 27108 raft_consensus.cc:606] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Received request to transfer leadership to TS 38218df822dc435eac030105d4f51140
I20260709 12:17:17.341336 26957 raft_consensus.cc:2393] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 508ff162d05e44159e225081ede82882 in current term 3: Already voted for candidate 38218df822dc435eac030105d4f51140 in this term.
I20260709 12:17:17.341490 27300 leader_election.cc:290] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:17.343472 27031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:17.350895 27286 consensus_queue.cc:1048] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260709 12:17:17.352761 27108 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "58ea17416b1642b2820a501561e9dd68"
dest_uuid: "508ff162d05e44159e225081ede82882"
mode: GRACEFUL
new_leader_uuid: "5604f5b9fef245bca9665c54cdd0d967"
 from {username='slave'} at 127.0.0.1:44468
I20260709 12:17:17.353578 27108 raft_consensus.cc:606] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Received request to transfer leadership to TS 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:17.353853 27031 raft_consensus.cc:2468] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 3.
I20260709 12:17:17.354890 27290 raft_consensus.cc:493] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.355420 27290 raft_consensus.cc:515] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.357839 27290 leader_election.cc:290] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:17.360462 27034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ea17416b1642b2820a501561e9dd68" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:17.362665 27034 raft_consensus.cc:2393] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38218df822dc435eac030105d4f51140 in current term 1: Already voted for candidate 508ff162d05e44159e225081ede82882 in this term.
I20260709 12:17:17.362680 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ea17416b1642b2820a501561e9dd68" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:17.364830 26925 leader_election.cc:304] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38218df822dc435eac030105d4f51140; no voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:17.365814 27290 raft_consensus.cc:3060] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140 [term 0 FOLLOWER]: Advancing to term 1
I20260709 12:17:17.370878 27071 leader_election.cc:304] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 508ff162d05e44159e225081ede82882, 5604f5b9fef245bca9665c54cdd0d967; no voters: 38218df822dc435eac030105d4f51140
I20260709 12:17:17.372048 27300 raft_consensus.cc:2804] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:17.372834 27300 raft_consensus.cc:697] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 3 LEADER]: Becoming Leader. State: Replica: 508ff162d05e44159e225081ede82882, State: Running, Role: LEADER
I20260709 12:17:17.373786 27300 consensus_queue.cc:237] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.385432 26957 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "2bccc8c7fe754ec49743359441654476"
dest_uuid: "38218df822dc435eac030105d4f51140"
mode: GRACEFUL
new_leader_uuid: "508ff162d05e44159e225081ede82882"
 from {username='slave'} at 127.0.0.1:44740
I20260709 12:17:17.386322 26957 raft_consensus.cc:606] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Received request to transfer leadership to TS 508ff162d05e44159e225081ede82882
I20260709 12:17:17.392305 27289 consensus_queue.cc:1048] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:17.400929 27290 raft_consensus.cc:2749] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260709 12:17:17.402136 27248 raft_consensus.cc:993] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882: : Instructing follower 38218df822dc435eac030105d4f51140 to start an election
I20260709 12:17:17.404124 27289 raft_consensus.cc:1081] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Signalling peer 38218df822dc435eac030105d4f51140 to start an election
I20260709 12:17:17.409613 27289 raft_consensus.cc:493] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38218df822dc435eac030105d4f51140)
I20260709 12:17:17.426704 26959 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "fd773572db7849348131c09a3a18184c"
dest_uuid: "38218df822dc435eac030105d4f51140"
 from {username='slave'} at 127.0.0.1:48144
I20260709 12:17:17.427731 27326 raft_consensus.cc:493] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.427874 26959 raft_consensus.cc:493] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:17.426688 27289 raft_consensus.cc:515] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.430346 27290 raft_consensus.cc:493] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260709 12:17:17.435889 26959 raft_consensus.cc:3060] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.440400 27289 leader_election.cc:290] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:17.440717 26957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:17.441087 27031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:17.441604 26957 raft_consensus.cc:2468] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 3.
I20260709 12:17:17.436919 27290 raft_consensus.cc:515] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.444156 27072 leader_election.cc:304] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882; no voters: 
I20260709 12:17:17.445667 27290 leader_election.cc:290] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:17.446540 27034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967" is_pre_election: true
I20260709 12:17:17.447746 27300 raft_consensus.cc:2804] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260709 12:17:17.448386 27300 raft_consensus.cc:493] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 38218df822dc435eac030105d4f51140)
I20260709 12:17:17.448348 27034 raft_consensus.cc:2468] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 3.
I20260709 12:17:17.449288 27300 raft_consensus.cc:3060] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:17.450536 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:17.450838 26922 leader_election.cc:304] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:17.436612 27326 raft_consensus.cc:515] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.454231 27031 raft_consensus.cc:2393] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 508ff162d05e44159e225081ede82882 in current term 3: Already voted for candidate 5604f5b9fef245bca9665c54cdd0d967 in this term.
I20260709 12:17:17.457051 26957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140" is_pre_election: true
I20260709 12:17:17.457708 27330 consensus_queue.cc:1048] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:17.459105 26957 raft_consensus.cc:2468] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 3.
I20260709 12:17:17.460827 26998 leader_election.cc:304] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:17.453336 27290 raft_consensus.cc:2804] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:17.461894 27242 raft_consensus.cc:2804] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260709 12:17:17.465497 27242 raft_consensus.cc:493] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.466043 27242 raft_consensus.cc:3060] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:17.467411 27300 raft_consensus.cc:515] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.470275 27290 raft_consensus.cc:493] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260709 12:17:17.470768 27290 raft_consensus.cc:3060] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:17.471166 27300 leader_election.cc:290] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:17.455344 27326 leader_election.cc:290] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:17.471812 26957 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:17.456238 26959 raft_consensus.cc:515] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.473986 26959 leader_election.cc:290] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 2 election: Requested vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:17.479154 27031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:17.480232 27107 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd773572db7849348131c09a3a18184c" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:17.480845 27107 raft_consensus.cc:3055] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:17.481194 27107 raft_consensus.cc:740] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Running, Role: LEADER
I20260709 12:17:17.484464 27242 raft_consensus.cc:515] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.485147 26957 raft_consensus.cc:2468] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 3.
I20260709 12:17:17.485241 27290 raft_consensus.cc:515] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.500497 27031 raft_consensus.cc:2368] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 508ff162d05e44159e225081ede82882 for earlier term 3. Current term is 4.
I20260709 12:17:17.501355 27072 leader_election.cc:304] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882; no voters: 
I20260709 12:17:17.502951 27071 leader_election.cc:400] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 election: Vote denied by peer 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771) with higher term. Message: Invalid argument: T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 508ff162d05e44159e225081ede82882 for earlier term 3. Current term is 4.
I20260709 12:17:17.487213 27107 consensus_queue.cc:260] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.490374 27034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "fd773572db7849348131c09a3a18184c" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:17.503896 27034 raft_consensus.cc:3060] T fd773572db7849348131c09a3a18184c P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.504568 27107 raft_consensus.cc:3060] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.505076 27290 leader_election.cc:290] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 4 election: Requested vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:17.506603 27242 leader_election.cc:290] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 4 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:17.505203 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:17.505528 27031 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:17.512688 27031 raft_consensus.cc:3060] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:17.511119 27106 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "8b6087005dd541ffa46a97375e5ba4a1" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:17.511430 27300 raft_consensus.cc:2804] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:17.514456 27330 consensus_queue.cc:1048] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:17.515013 27300 raft_consensus.cc:697] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 3 LEADER]: Becoming Leader. State: Replica: 508ff162d05e44159e225081ede82882, State: Running, Role: LEADER
I20260709 12:17:17.516343 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882" is_pre_election: true
I20260709 12:17:17.516113 27300 consensus_queue.cc:237] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.519752 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "c6dcf02f879841949d494b2232d540ee" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:17.508080 26959 raft_consensus.cc:3060] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:17.523008 27242 consensus_queue.cc:1048] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260709 12:17:17.543439 26959 raft_consensus.cc:2468] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 4.
I20260709 12:17:17.545706 26998 leader_election.cc:304] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 508ff162d05e44159e225081ede82882
I20260709 12:17:17.547832 27330 consensus_queue.cc:1048] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:17.552557 27326 raft_consensus.cc:2804] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:17.553141 27326 raft_consensus.cc:697] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 4 LEADER]: Becoming Leader. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Running, Role: LEADER
I20260709 12:17:17.554243 27107 raft_consensus.cc:2468] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 2.
I20260709 12:17:17.554195 27326 consensus_queue.cc:237] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.556196 27034 raft_consensus.cc:2468] T fd773572db7849348131c09a3a18184c P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 2.
I20260709 12:17:17.557655 26925 leader_election.cc:304] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882; no voters: 
I20260709 12:17:17.538214 27031 raft_consensus.cc:2468] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 4.
I20260709 12:17:17.579299 27245 raft_consensus.cc:2804] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:17.579933 27245 raft_consensus.cc:697] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140 [term 2 LEADER]: Becoming Leader. State: Replica: 38218df822dc435eac030105d4f51140, State: Running, Role: LEADER
I20260709 12:17:17.580880 27245 consensus_queue.cc:237] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.585462 26855 catalog_manager.cc:5697] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 reported cstate change: term changed from 2 to 3, leader changed from 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130) to 508ff162d05e44159e225081ede82882 (127.11.13.131). New cstate: current_term: 3 leader_uuid: "508ff162d05e44159e225081ede82882" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:17.596446 26855 catalog_manager.cc:5697] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 reported cstate change: term changed from 2 to 3, leader changed from 38218df822dc435eac030105d4f51140 (127.11.13.129) to 508ff162d05e44159e225081ede82882 (127.11.13.131). New cstate: current_term: 3 leader_uuid: "508ff162d05e44159e225081ede82882" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:17.602526 26922 leader_election.cc:304] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 508ff162d05e44159e225081ede82882
I20260709 12:17:17.604879 27285 raft_consensus.cc:2804] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 4 FOLLOWER]: Leader election won for term 4
I20260709 12:17:17.605556 27285 raft_consensus.cc:697] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 4 LEADER]: Becoming Leader. State: Replica: 38218df822dc435eac030105d4f51140, State: Running, Role: LEADER
I20260709 12:17:17.606580 27285 consensus_queue.cc:237] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.617985 27293 consensus_queue.cc:1048] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:17.625797 26856 catalog_manager.cc:5697] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140 reported cstate change: term changed from 1 to 2, leader changed from 508ff162d05e44159e225081ede82882 (127.11.13.131) to 38218df822dc435eac030105d4f51140 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "38218df822dc435eac030105d4f51140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:17.628214 27330 raft_consensus.cc:993] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882: : Instructing follower 5604f5b9fef245bca9665c54cdd0d967 to start an election
I20260709 12:17:17.628957 27330 raft_consensus.cc:1081] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Signalling peer 5604f5b9fef245bca9665c54cdd0d967 to start an election
I20260709 12:17:17.609771 27286 consensus_queue.cc:1048] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.016s
I20260709 12:17:17.631798 26854 catalog_manager.cc:5697] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 reported cstate change: term changed from 3 to 4, leader changed from 508ff162d05e44159e225081ede82882 (127.11.13.131) to 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130). New cstate: current_term: 4 leader_uuid: "5604f5b9fef245bca9665c54cdd0d967" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:17.631814 27033 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "58ea17416b1642b2820a501561e9dd68"
dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
 from {username='slave'} at 127.0.0.1:44418
I20260709 12:17:17.639230 27033 raft_consensus.cc:493] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:17.639832 27033 raft_consensus.cc:3060] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.669306 27293 consensus_queue.cc:1048] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260709 12:17:17.697576 27290 raft_consensus.cc:993] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140: : Instructing follower 508ff162d05e44159e225081ede82882 to start an election
I20260709 12:17:17.698314 27285 raft_consensus.cc:1081] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Signalling peer 508ff162d05e44159e225081ede82882 to start an election
I20260709 12:17:17.702406 27107 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "2bccc8c7fe754ec49743359441654476"
dest_uuid: "508ff162d05e44159e225081ede82882"
 from {username='slave'} at 127.0.0.1:39480
I20260709 12:17:17.703052 27107 raft_consensus.cc:493] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:17.703497 27107 raft_consensus.cc:3060] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.701855 27033 raft_consensus.cc:515] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.710810 27031 raft_consensus.cc:1240] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Rejecting Update request from peer 508ff162d05e44159e225081ede82882 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:17.711025 26856 catalog_manager.cc:5697] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 reported cstate change: term changed from 3 to 4, leader changed from 508ff162d05e44159e225081ede82882 (127.11.13.131) to 38218df822dc435eac030105d4f51140 (127.11.13.129). New cstate: current_term: 4 leader_uuid: "38218df822dc435eac030105d4f51140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:17.714340 27107 raft_consensus.cc:515] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.716001 27106 raft_consensus.cc:1240] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Rejecting Update request from peer 38218df822dc435eac030105d4f51140 for earlier term 1. Current term is 2. Ops: []
I20260709 12:17:17.717178 27107 leader_election.cc:290] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:17.718175 26959 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bccc8c7fe754ec49743359441654476" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:17.718987 26959 raft_consensus.cc:3055] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:17.719306 26959 raft_consensus.cc:740] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Running, Role: LEADER
I20260709 12:17:17.719601 27245 consensus_queue.cc:1059] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260709 12:17:17.720578 27034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "2bccc8c7fe754ec49743359441654476" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:17.722280 26959 consensus_queue.cc:260] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.723286 26959 raft_consensus.cc:3060] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.714550 27300 consensus_queue.cc:1059] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.010s
I20260709 12:17:17.725236 27330 raft_consensus.cc:3055] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:17.725644 27330 raft_consensus.cc:740] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Running, Role: LEADER
I20260709 12:17:17.726624 27330 consensus_queue.cc:260] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.728370 27330 raft_consensus.cc:3060] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.733271 26958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ea17416b1642b2820a501561e9dd68" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:17.734148 26958 raft_consensus.cc:3060] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.740162 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "58ea17416b1642b2820a501561e9dd68" candidate_uuid: "5604f5b9fef245bca9665c54cdd0d967" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:17.740573 26959 raft_consensus.cc:2468] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 2.
I20260709 12:17:17.743332 26958 raft_consensus.cc:2468] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 2.
I20260709 12:17:17.744247 27033 leader_election.cc:290] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 2 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
W20260709 12:17:17.745199 27245 raft_consensus.cc:936] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher.
I20260709 12:17:17.744972 26998 leader_election.cc:304] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967; no voters: 
I20260709 12:17:17.746517 27072 leader_election.cc:304] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882; no voters: 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:17.747852 27326 raft_consensus.cc:2804] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:17.748325 27326 raft_consensus.cc:697] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 LEADER]: Becoming Leader. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Running, Role: LEADER
I20260709 12:17:17.749004 27326 consensus_queue.cc:237] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.750864 27330 raft_consensus.cc:2804] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:17.753526 27330 raft_consensus.cc:697] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 [term 2 LEADER]: Becoming Leader. State: Replica: 508ff162d05e44159e225081ede82882, State: Running, Role: LEADER
I20260709 12:17:17.754559 27330 consensus_queue.cc:237] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.762025 27108 raft_consensus.cc:2468] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5604f5b9fef245bca9665c54cdd0d967 in term 2.
I20260709 12:17:17.763834 26856 catalog_manager.cc:5697] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 reported cstate change: term changed from 1 to 2, leader changed from 38218df822dc435eac030105d4f51140 (127.11.13.129) to 508ff162d05e44159e225081ede82882 (127.11.13.131). New cstate: current_term: 2 leader_uuid: "508ff162d05e44159e225081ede82882" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:17.767282 26855 catalog_manager.cc:5697] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 reported cstate change: term changed from 1 to 2, leader changed from 508ff162d05e44159e225081ede82882 (127.11.13.131) to 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130). New cstate: current_term: 2 leader_uuid: "5604f5b9fef245bca9665c54cdd0d967" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:17.771653 27330 raft_consensus.cc:993] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882: : Instructing follower 38218df822dc435eac030105d4f51140 to start an election
I20260709 12:17:17.772266 27330 raft_consensus.cc:1081] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Signalling peer 38218df822dc435eac030105d4f51140 to start an election
I20260709 12:17:17.774457 26959 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "59af62b8772640f79014b776aa0a2906"
dest_uuid: "38218df822dc435eac030105d4f51140"
 from {username='slave'} at 127.0.0.1:48144
I20260709 12:17:17.775063 26959 raft_consensus.cc:493] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:17.775429 26959 raft_consensus.cc:3060] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.784345 26959 raft_consensus.cc:515] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.787694 26959 leader_election.cc:290] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 2 election: Requested vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 508ff162d05e44159e225081ede82882 (127.11.13.131:40411)
I20260709 12:17:17.788156 27033 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af62b8772640f79014b776aa0a2906" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:17.788978 27108 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "59af62b8772640f79014b776aa0a2906" candidate_uuid: "38218df822dc435eac030105d4f51140" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "508ff162d05e44159e225081ede82882"
I20260709 12:17:17.789304 27033 raft_consensus.cc:3060] T 59af62b8772640f79014b776aa0a2906 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.789871 27108 raft_consensus.cc:3055] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:17.790354 27108 raft_consensus.cc:740] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Running, Role: LEADER
I20260709 12:17:17.791389 27108 consensus_queue.cc:260] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.793315 27108 raft_consensus.cc:3060] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:17.802191 27108 raft_consensus.cc:2468] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 2.
I20260709 12:17:17.805004 26925 leader_election.cc:304] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882; no voters: 
I20260709 12:17:17.806284 27245 raft_consensus.cc:2804] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:17.806872 27245 raft_consensus.cc:697] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140 [term 2 LEADER]: Becoming Leader. State: Replica: 38218df822dc435eac030105d4f51140, State: Running, Role: LEADER
I20260709 12:17:17.807818 27245 consensus_queue.cc:237] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.809005 27033 raft_consensus.cc:2468] T 59af62b8772640f79014b776aa0a2906 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38218df822dc435eac030105d4f51140 in term 2.
I20260709 12:17:17.818037 26855 catalog_manager.cc:5697] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140 reported cstate change: term changed from 1 to 2, leader changed from 508ff162d05e44159e225081ede82882 (127.11.13.131) to 38218df822dc435eac030105d4f51140 (127.11.13.129). New cstate: current_term: 2 leader_uuid: "38218df822dc435eac030105d4f51140" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: UNKNOWN } } }
I20260709 12:17:17.886072 26959 raft_consensus.cc:1240] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 4 LEADER]: Rejecting Update request from peer 508ff162d05e44159e225081ede82882 for earlier term 3. Current term is 4. Ops: []
I20260709 12:17:17.887914 27248 consensus_queue.cc:1059] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:17.888600 27248 consensus_queue.cc:1243] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [LEADER]: Peer 38218df822dc435eac030105d4f51140 log is divergent from this leader: its last log entry 4.3 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260709 12:17:17.889094 27300 raft_consensus.cc:3055] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:17.889425 27300 raft_consensus.cc:740] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Running, Role: LEADER
I20260709 12:17:17.890415 27300 consensus_queue.cc:260] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.891502 27300 raft_consensus.cc:3060] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:17.986274 26959 raft_consensus.cc:1275] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 4 FOLLOWER]: Refusing update from remote peer 5604f5b9fef245bca9665c54cdd0d967: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:17.988060 27286 consensus_queue.cc:1048] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:17.991199 27108 raft_consensus.cc:3055] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 3 LEADER]: Stepping down as leader of term 3
I20260709 12:17:17.991690 27108 raft_consensus.cc:740] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 508ff162d05e44159e225081ede82882, State: Running, Role: LEADER
I20260709 12:17:17.992621 27108 consensus_queue.cc:260] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:17.996264 27108 raft_consensus.cc:3060] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 3 FOLLOWER]: Advancing to term 4
I20260709 12:17:18.006330 27108 raft_consensus.cc:1275] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 4 FOLLOWER]: Refusing update from remote peer 5604f5b9fef245bca9665c54cdd0d967: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:17:18.006737 27108 pending_rounds.cc:85] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882: Aborting all ops after (but not including) 2
I20260709 12:17:18.006965 27108 pending_rounds.cc:107] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:17:18.007300 27108 raft_consensus.cc:2889] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:18.010775 27242 consensus_queue.cc:1048] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260709 12:17:18.011104 27033 raft_consensus.cc:1275] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 4 FOLLOWER]: Refusing update from remote peer 38218df822dc435eac030105d4f51140: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260709 12:17:18.015441 27290 consensus_queue.cc:1048] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:18.017817 27106 raft_consensus.cc:1275] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 4 FOLLOWER]: Refusing update from remote peer 38218df822dc435eac030105d4f51140: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260709 12:17:18.018396 27106 pending_rounds.cc:85] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882: Aborting all ops after (but not including) 2
I20260709 12:17:18.018667 27106 pending_rounds.cc:107] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260709 12:17:18.018932 27106 raft_consensus.cc:2889] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260709 12:17:18.022455 27290 consensus_queue.cc:1048] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:18.065647 27033 raft_consensus.cc:1275] T fd773572db7849348131c09a3a18184c P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Refusing update from remote peer 38218df822dc435eac030105d4f51140: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:18.067505 27293 consensus_queue.cc:1048] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:18.082007 27108 raft_consensus.cc:1275] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Refusing update from remote peer 38218df822dc435eac030105d4f51140: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:18.084587 27245 consensus_queue.cc:1048] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:18.159333 27108 raft_consensus.cc:1275] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Refusing update from remote peer 5604f5b9fef245bca9665c54cdd0d967: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:18.160756 27326 consensus_queue.cc:1048] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:18.170353 26958 raft_consensus.cc:1275] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Refusing update from remote peer 5604f5b9fef245bca9665c54cdd0d967: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:18.172585 27326 consensus_queue.cc:1048] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:18.195617 27108 raft_consensus.cc:1275] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Refusing update from remote peer 38218df822dc435eac030105d4f51140: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:18.197198 27290 consensus_queue.cc:1048] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:18.208871 27033 raft_consensus.cc:1275] T 59af62b8772640f79014b776aa0a2906 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Refusing update from remote peer 38218df822dc435eac030105d4f51140: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:18.210814 27285 consensus_queue.cc:1048] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:18.274222 26958 raft_consensus.cc:1275] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Refusing update from remote peer 508ff162d05e44159e225081ede82882: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:18.274807 27033 raft_consensus.cc:3060] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:18.276093 27330 consensus_queue.cc:1048] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:18.283640 27033 raft_consensus.cc:1275] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Refusing update from remote peer 508ff162d05e44159e225081ede82882: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:18.289710 27248 consensus_queue.cc:1048] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260709 12:17:20.391362 26854 catalog_manager.cc:2533] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44114:
table { table_name: "soft_deleted_table" } modify_external_catalogs: true reserve_seconds: 3600
I20260709 12:17:20.421155 26854 catalog_manager.cc:3570] Servicing AlterTable request from {username='slave'} at 127.0.0.1:44114:
table { table_name: "soft_deleted_table" } new_extra_configs { key: "kudu.table.disable_compaction" value: "true" }
I20260709 12:17:20.572260 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
W20260709 12:17:20.577234 26925 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.131:40411, user_credentials={real_user=slave}} blocked reactor thread for 44511.3us
W20260709 12:17:20.627465 27367 log.cc:927] Time spent T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140: Append to log took a long time: real 0.057s	user 0.001s	sys 0.000s
W20260709 12:17:20.627722 27370 log.cc:927] Time spent T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882: Append to log took a long time: real 0.053s	user 0.000s	sys 0.001s
W20260709 12:17:20.690183 26922 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:37771, user_credentials={real_user=slave}} blocked reactor thread for 110454us
I20260709 12:17:20.764757 27398 tablet.cc:1724] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:17:20.811378 27390 log.cc:927] Time spent T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882: Append to log took a long time: real 0.051s	user 0.001s	sys 0.000s
I20260709 12:17:20.820416 27395 tablet.cc:1724] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:20.784705 27391 tablet.cc:1724] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:20.821070 27396 tablet.cc:1724] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:20.843935 27407 tablet.cc:1724] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
W20260709 12:17:20.948228 27071 outbound_call.cc:321] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.11.13.130:37771, user_credentials={real_user=slave}} blocked reactor thread for 91173.1us
I20260709 12:17:20.989837 27391 tablet.cc:1724] T d5cc002263e547a8a5da289d103ea955 P 38218df822dc435eac030105d4f51140: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.011101 26854 catalog_manager.cc:5789] TS 38218df822dc435eac030105d4f51140 (127.11.13.129:36041) does not have the latest schema for tablet fd773572db7849348131c09a3a18184c (table soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]). Expected version 1 got 0
I20260709 12:17:20.974864 27410 tablet.cc:1724] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.024158 26854 catalog_manager.cc:5789] TS 38218df822dc435eac030105d4f51140 (127.11.13.129:36041) does not have the latest schema for tablet aab7d9b4c30c466fa1e5451e42820d75 (table soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6]). Expected version 1 got 0
I20260709 12:17:20.960137 27411 tablet.cc:1724] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.029971 27033 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "e85c5e2f2a6b4d31bd83715d05eda1c1"
dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
mode: GRACEFUL
new_leader_uuid: "508ff162d05e44159e225081ede82882"
 from {username='slave'} at 127.0.0.1:57610
I20260709 12:17:20.953110 27414 tablet.cc:1724] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.066805 27413 tablet.cc:1724] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.067646 27033 raft_consensus.cc:606] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 LEADER]: Received request to transfer leadership to TS 508ff162d05e44159e225081ede82882
I20260709 12:17:21.039161 27396 tablet.cc:1724] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.049760 27395 tablet.cc:1724] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.088359 27418 tablet.cc:1724] T c8c93363b18b455696ca407b4dbde553 P 5604f5b9fef245bca9665c54cdd0d967: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.099004 27419 tablet.cc:1724] T aab7d9b4c30c466fa1e5451e42820d75 P 5604f5b9fef245bca9665c54cdd0d967: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.106398 27391 tablet.cc:1724] T 30203b8c002948178d4966017ba69c86 P 38218df822dc435eac030105d4f51140: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.101166 27413 tablet.cc:1724] T d5cc002263e547a8a5da289d103ea955 P 508ff162d05e44159e225081ede82882: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.093928 27417 tablet.cc:1724] T 59af62b8772640f79014b776aa0a2906 P 5604f5b9fef245bca9665c54cdd0d967: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.132407 27398 tablet.cc:1724] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.139400 27395 tablet.cc:1724] T bfc3afe51dc249278d04002e83162998 P 38218df822dc435eac030105d4f51140: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.141757 27414 tablet.cc:1724] T bfc3afe51dc249278d04002e83162998 P 508ff162d05e44159e225081ede82882: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.144405 27418 tablet.cc:1724] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.161049 27413 tablet.cc:1724] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.172927 27411 tablet.cc:1724] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.181563 27419 tablet.cc:1724] T fd773572db7849348131c09a3a18184c P 5604f5b9fef245bca9665c54cdd0d967: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.207472 27391 tablet.cc:1724] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.216679 27417 tablet.cc:1724] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.231122 27418 tablet.cc:1724] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967: Alter schema from (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 0 to (
    10:key INT32 NOT NULL,
    11:int_val INT32 NOT NULL,
    12:string_val STRING NULLABLE,
    PRIMARY KEY (key)
) version 1
I20260709 12:17:21.241412 26958 tablet_service.cc:2072] Received LeaderStepDown RPC: tablet_id: "f6a2f90e85c04cf28ba0397d0c0a5b23"
dest_uuid: "38218df822dc435eac030105d4f51140"
mode: GRACEFUL
new_leader_uuid: "508ff162d05e44159e225081ede82882"
 from {username='slave'} at 127.0.0.1:44754
I20260709 12:17:21.245293 26958 raft_consensus.cc:606] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Received request to transfer leadership to TS 508ff162d05e44159e225081ede82882
I20260709 12:17:21.247635 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260709 12:17:21.248441 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:21.250036 27286 raft_consensus.cc:993] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967: : Instructing follower 508ff162d05e44159e225081ede82882 to start an election
I20260709 12:17:21.250728 27286 raft_consensus.cc:1081] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 LEADER]: Signalling peer 508ff162d05e44159e225081ede82882 to start an election
I20260709 12:17:21.253154 27103 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "e85c5e2f2a6b4d31bd83715d05eda1c1"
dest_uuid: "508ff162d05e44159e225081ede82882"
 from {username='slave'} at 127.0.0.1:39474
I20260709 12:17:21.253815 27103 raft_consensus.cc:493] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:21.254288 27103 raft_consensus.cc:3060] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.272529 27103 raft_consensus.cc:515] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:21.275882 26958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85c5e2f2a6b4d31bd83715d05eda1c1" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:21.276672 26958 raft_consensus.cc:3060] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.280055 26855 catalog_manager.cc:6242] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2: soft_deleted_table [id=533c4cc1a91e4df686a96a1d79d247b6] alter complete (version 1)
I20260709 12:17:21.291940 27103 leader_election.cc:290] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:21.292368 27034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "e85c5e2f2a6b4d31bd83715d05eda1c1" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:21.293095 27034 raft_consensus.cc:3055] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 LEADER]: Stepping down as leader of term 2
I20260709 12:17:21.293170 26958 raft_consensus.cc:2468] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 3.
I20260709 12:17:21.293470 27034 raft_consensus.cc:740] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5604f5b9fef245bca9665c54cdd0d967, State: Running, Role: LEADER
I20260709 12:17:21.294458 27034 consensus_queue.cc:260] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:21.294776 27072 leader_election.cc:304] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882; no voters: 
I20260709 12:17:21.295732 27034 raft_consensus.cc:3060] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Advancing to term 3
I20260709 12:17:21.295884 27409 raft_consensus.cc:2804] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [term 3 FOLLOWER]: Leader election won for term 3
I20260709 12:17:21.296442 27409 raft_consensus.cc:697] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [term 3 LEADER]: Becoming Leader. State: Replica: 508ff162d05e44159e225081ede82882, State: Running, Role: LEADER
I20260709 12:17:21.297286 27409 consensus_queue.cc:237] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:21.308779 27034 raft_consensus.cc:2468] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 3.
I20260709 12:17:21.325898 26854 catalog_manager.cc:5697] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 reported cstate change: term changed from 2 to 3, leader changed from 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130) to 508ff162d05e44159e225081ede82882 (127.11.13.131). New cstate: current_term: 3 leader_uuid: "508ff162d05e44159e225081ede82882" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:21.431396 27371 raft_consensus.cc:993] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140: : Instructing follower 508ff162d05e44159e225081ede82882 to start an election
I20260709 12:17:21.432024 27371 raft_consensus.cc:1081] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Signalling peer 508ff162d05e44159e225081ede82882 to start an election
I20260709 12:17:21.433924 27103 tablet_service.cc:2044] Received Run Leader Election RPC: tablet_id: "f6a2f90e85c04cf28ba0397d0c0a5b23"
dest_uuid: "508ff162d05e44159e225081ede82882"
 from {username='slave'} at 127.0.0.1:39480
I20260709 12:17:21.434650 27103 raft_consensus.cc:493] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260709 12:17:21.435078 27103 raft_consensus.cc:3060] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.444284 27103 raft_consensus.cc:515] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:21.447162 27103 leader_election.cc:290] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 election: Requested vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:21.448052 26958 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a2f90e85c04cf28ba0397d0c0a5b23" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "38218df822dc435eac030105d4f51140"
I20260709 12:17:21.448335 27034 tablet_service.cc:1917] Received RequestConsensusVote() RPC: tablet_id: "f6a2f90e85c04cf28ba0397d0c0a5b23" candidate_uuid: "508ff162d05e44159e225081ede82882" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "5604f5b9fef245bca9665c54cdd0d967"
I20260709 12:17:21.448786 26958 raft_consensus.cc:3055] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Stepping down as leader of term 1
I20260709 12:17:21.449352 26958 raft_consensus.cc:740] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 38218df822dc435eac030105d4f51140, State: Running, Role: LEADER
I20260709 12:17:21.449388 27034 raft_consensus.cc:3060] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.450438 26958 consensus_queue.cc:260] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:21.452170 26958 raft_consensus.cc:3060] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Advancing to term 2
I20260709 12:17:21.460728 26958 raft_consensus.cc:2468] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 2.
I20260709 12:17:21.461305 27034 raft_consensus.cc:2468] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 508ff162d05e44159e225081ede82882 in term 2.
I20260709 12:17:21.462160 27072 leader_election.cc:304] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38218df822dc435eac030105d4f51140, 508ff162d05e44159e225081ede82882; no voters: 
I20260709 12:17:21.463459 27409 raft_consensus.cc:2804] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader election won for term 2
I20260709 12:17:21.464092 27409 raft_consensus.cc:697] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [term 2 LEADER]: Becoming Leader. State: Replica: 508ff162d05e44159e225081ede82882, State: Running, Role: LEADER
I20260709 12:17:21.465469 27409 consensus_queue.cc:237] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:21.476015 26854 catalog_manager.cc:5697] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 reported cstate change: term changed from 1 to 2, leader changed from 38218df822dc435eac030105d4f51140 (127.11.13.129) to 508ff162d05e44159e225081ede82882 (127.11.13.131). New cstate: current_term: 2 leader_uuid: "508ff162d05e44159e225081ede82882" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } health_report { overall_health: HEALTHY } } }
I20260709 12:17:21.855171 27034 raft_consensus.cc:1275] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [term 3 FOLLOWER]: Refusing update from remote peer 508ff162d05e44159e225081ede82882: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:21.856899 27409 consensus_queue.cc:1048] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:21.872012 26958 raft_consensus.cc:1275] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 3 FOLLOWER]: Refusing update from remote peer 508ff162d05e44159e225081ede82882: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260709 12:17:21.873844 27409 consensus_queue.cc:1048] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260709 12:17:21.976469 27034 raft_consensus.cc:1275] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Refusing update from remote peer 508ff162d05e44159e225081ede82882: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:21.978467 27409 consensus_queue.cc:1048] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:21.995235 26959 raft_consensus.cc:1275] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Refusing update from remote peer 508ff162d05e44159e225081ede82882: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260709 12:17:21.996855 27412 consensus_queue.cc:1048] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260709 12:17:22.250370 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:22.261247 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:22.261832 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:23.263753 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:23.270473 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:23.270947 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:24.272370 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:24.279421 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:24.280043 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:25.281381 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:25.286877 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:25.287379 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:26.288618 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:26.294093 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:26.294695 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:27.296075 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:27.302415 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:27.302932 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:28.304188 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:28.309504 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:28.310050 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:29.311281 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:29.318163 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:29.318696 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:30.320214 11318 auto_leader_rebalancer.cc:141] leader rebalance for table test-workload
I20260709 12:17:30.328369 11318 auto_leader_rebalancer.cc:440] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260709 12:17:30.328992 11318 auto_leader_rebalancer.cc:515] All tables' leader rebalancing finished this round
I20260709 12:17:31.406314 11318 tablet_server.cc:179] TabletServer@127.11.13.129:0 shutting down...
W20260709 12:17:31.413558 27072 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 14 similar messages]
W20260709 12:17:31.420647 27072 consensus_peers.cc:597] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 -> Peer 38218df822dc435eac030105d4f51140 (127.11.13.129:36041): Couldn't send request to peer 38218df822dc435eac030105d4f51140. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:31.423620 26998 consensus_peers.cc:597] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 -> Peer 38218df822dc435eac030105d4f51140 (127.11.13.129:36041): Couldn't send request to peer 38218df822dc435eac030105d4f51140. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:31.432729 26998 consensus_peers.cc:597] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967 -> Peer 38218df822dc435eac030105d4f51140 (127.11.13.129:36041): Couldn't send request to peer 38218df822dc435eac030105d4f51140. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:31.446317 27072 consensus_peers.cc:597] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 -> Peer 38218df822dc435eac030105d4f51140 (127.11.13.129:36041): Couldn't send request to peer 38218df822dc435eac030105d4f51140. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:31.464042 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:31.464946 11318 tablet_replica.cc:333] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140: stopping tablet replica
I20260709 12:17:31.466346 11318 raft_consensus.cc:2243] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.467015 11318 raft_consensus.cc:2272] T 58ea17416b1642b2820a501561e9dd68 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.470115 11318 tablet_replica.cc:333] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140: stopping tablet replica
I20260709 12:17:31.470810 11318 raft_consensus.cc:2243] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:31.471868 11318 raft_consensus.cc:2272] T fd773572db7849348131c09a3a18184c P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.474839 11318 tablet_replica.cc:333] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140: stopping tablet replica
I20260709 12:17:31.475652 11318 raft_consensus.cc:2243] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:31.476953 11318 raft_consensus.cc:2272] T 59af62b8772640f79014b776aa0a2906 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.480230 11318 tablet_replica.cc:333] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140: stopping tablet replica
I20260709 12:17:31.480962 11318 raft_consensus.cc:2243] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.481621 11318 raft_consensus.cc:2272] T 2bccc8c7fe754ec49743359441654476 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.484339 11318 tablet_replica.cc:333] T d5cc002263e547a8a5da289d103ea955 P 38218df822dc435eac030105d4f51140: stopping tablet replica
I20260709 12:17:31.485090 11318 raft_consensus.cc:2243] T d5cc002263e547a8a5da289d103ea955 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.486061 11318 raft_consensus.cc:2272] T d5cc002263e547a8a5da289d103ea955 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.490784 11318 tablet_replica.cc:333] T 30203b8c002948178d4966017ba69c86 P 38218df822dc435eac030105d4f51140: stopping tablet replica
I20260709 12:17:31.491621 11318 raft_consensus.cc:2243] T 30203b8c002948178d4966017ba69c86 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.492506 11318 raft_consensus.cc:2272] T 30203b8c002948178d4966017ba69c86 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.496407 11318 tablet_replica.cc:333] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140: stopping tablet replica
I20260709 12:17:31.497558 11318 raft_consensus.cc:2243] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.498261 11318 raft_consensus.cc:2272] T 1baf73a2b0be4194bf53c8731fdb7da2 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.501071 11318 tablet_replica.cc:333] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140: stopping tablet replica
I20260709 12:17:31.501784 11318 raft_consensus.cc:2243] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:31.502815 11318 raft_consensus.cc:2272] T aab7d9b4c30c466fa1e5451e42820d75 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.505436 11318 tablet_replica.cc:333] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140: stopping tablet replica
I20260709 12:17:31.506034 11318 raft_consensus.cc:2243] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.506626 11318 raft_consensus.cc:2272] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 38218df822dc435eac030105d4f51140 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.509303 11318 tablet_replica.cc:333] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140: stopping tablet replica
I20260709 12:17:31.510226 11318 raft_consensus.cc:2243] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.511000 11318 raft_consensus.cc:2272] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.514914 11318 tablet_replica.cc:333] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140: stopping tablet replica
I20260709 12:17:31.515836 11318 raft_consensus.cc:2243] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:17:31.516994 11318 raft_consensus.cc:2272] T 8b6087005dd541ffa46a97375e5ba4a1 P 38218df822dc435eac030105d4f51140 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.520185 11318 tablet_replica.cc:333] T 44117a0328e8455d8562a4dbe8a56eb3 P 38218df822dc435eac030105d4f51140: stopping tablet replica
I20260709 12:17:31.520830 11318 raft_consensus.cc:2243] T 44117a0328e8455d8562a4dbe8a56eb3 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.521322 11318 raft_consensus.cc:2272] T 44117a0328e8455d8562a4dbe8a56eb3 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.524088 11318 tablet_replica.cc:333] T bfc3afe51dc249278d04002e83162998 P 38218df822dc435eac030105d4f51140: stopping tablet replica
I20260709 12:17:31.524742 11318 raft_consensus.cc:2243] T bfc3afe51dc249278d04002e83162998 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.525255 11318 raft_consensus.cc:2272] T bfc3afe51dc249278d04002e83162998 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.527808 11318 tablet_replica.cc:333] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140: stopping tablet replica
I20260709 12:17:31.528367 11318 raft_consensus.cc:2243] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.528817 11318 raft_consensus.cc:2272] T afc42e2e5dab4df88c1c00bf9df243e0 P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.531528 11318 tablet_replica.cc:333] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140: stopping tablet replica
I20260709 12:17:31.532101 11318 raft_consensus.cc:2243] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:31.533077 11318 raft_consensus.cc:2272] T caf9d48542094084a3a015e242077f30 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.535943 11318 tablet_replica.cc:333] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140: stopping tablet replica
I20260709 12:17:31.536535 11318 raft_consensus.cc:2243] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:31.537525 11318 raft_consensus.cc:2272] T c8c93363b18b455696ca407b4dbde553 P 38218df822dc435eac030105d4f51140 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.540540 11318 tablet_replica.cc:333] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140: stopping tablet replica
I20260709 12:17:31.541109 11318 raft_consensus.cc:2243] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.541636 11318 raft_consensus.cc:2272] T c6dcf02f879841949d494b2232d540ee P 38218df822dc435eac030105d4f51140 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.544214 11318 tablet_replica.cc:333] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140: stopping tablet replica
I20260709 12:17:31.544730 11318 raft_consensus.cc:2243] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:31.545692 11318 raft_consensus.cc:2272] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 38218df822dc435eac030105d4f51140 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.599990 11318 tablet_server.cc:196] TabletServer@127.11.13.129:0 shutdown complete.
W20260709 12:17:31.664208 27072 consensus_peers.cc:597] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 -> Peer 38218df822dc435eac030105d4f51140 (127.11.13.129:36041): Couldn't send request to peer 38218df822dc435eac030105d4f51140. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:36041: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:31.666860 11318 tablet_server.cc:179] TabletServer@127.11.13.130:0 shutting down...
I20260709 12:17:31.717226 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:31.718089 11318 tablet_replica.cc:333] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967: stopping tablet replica
I20260709 12:17:31.719345 11318 raft_consensus.cc:2243] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:31.720484 11318 raft_consensus.cc:2272] T 58ea17416b1642b2820a501561e9dd68 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.723790 11318 tablet_replica.cc:333] T fd773572db7849348131c09a3a18184c P 5604f5b9fef245bca9665c54cdd0d967: stopping tablet replica
I20260709 12:17:31.724817 11318 raft_consensus.cc:2243] T fd773572db7849348131c09a3a18184c P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.725474 11318 raft_consensus.cc:2272] T fd773572db7849348131c09a3a18184c P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.728132 11318 tablet_replica.cc:333] T c8c93363b18b455696ca407b4dbde553 P 5604f5b9fef245bca9665c54cdd0d967: stopping tablet replica
I20260709 12:17:31.728794 11318 raft_consensus.cc:2243] T c8c93363b18b455696ca407b4dbde553 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.729470 11318 raft_consensus.cc:2272] T c8c93363b18b455696ca407b4dbde553 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.732494 11318 tablet_replica.cc:333] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967: stopping tablet replica
I20260709 12:17:31.733198 11318 raft_consensus.cc:2243] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.733843 11318 raft_consensus.cc:2272] T 2bccc8c7fe754ec49743359441654476 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.736414 11318 tablet_replica.cc:333] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967: stopping tablet replica
I20260709 12:17:31.737085 11318 raft_consensus.cc:2243] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:31.738262 11318 raft_consensus.cc:2272] T d5cc002263e547a8a5da289d103ea955 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.741017 11318 tablet_replica.cc:333] T 59af62b8772640f79014b776aa0a2906 P 5604f5b9fef245bca9665c54cdd0d967: stopping tablet replica
I20260709 12:17:31.741747 11318 raft_consensus.cc:2243] T 59af62b8772640f79014b776aa0a2906 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.742465 11318 raft_consensus.cc:2272] T 59af62b8772640f79014b776aa0a2906 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.744892 11318 tablet_replica.cc:333] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967: stopping tablet replica
I20260709 12:17:31.745524 11318 raft_consensus.cc:2243] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:31.746507 11318 raft_consensus.cc:2272] T 30203b8c002948178d4966017ba69c86 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.749895 11318 tablet_replica.cc:333] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967: stopping tablet replica
I20260709 12:17:31.750700 11318 raft_consensus.cc:2243] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 LEADER]: Raft consensus shutting down.
W20260709 12:17:31.751331 27072 consensus_peers.cc:597] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 -> Peer 38218df822dc435eac030105d4f51140 (127.11.13.129:36041): Couldn't send request to peer 38218df822dc435eac030105d4f51140. Status: Network error: Client connection negotiation failed: client connection to 127.11.13.129:36041: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260709 12:17:31.752161 11318 raft_consensus.cc:2272] T 1baf73a2b0be4194bf53c8731fdb7da2 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.754901 11318 tablet_replica.cc:333] T aab7d9b4c30c466fa1e5451e42820d75 P 5604f5b9fef245bca9665c54cdd0d967: stopping tablet replica
I20260709 12:17:31.755735 11318 raft_consensus.cc:2243] T aab7d9b4c30c466fa1e5451e42820d75 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.756333 11318 raft_consensus.cc:2272] T aab7d9b4c30c466fa1e5451e42820d75 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.759052 11318 tablet_replica.cc:333] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967: stopping tablet replica
I20260709 12:17:31.759707 11318 raft_consensus.cc:2243] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.760267 11318 raft_consensus.cc:2272] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 5604f5b9fef245bca9665c54cdd0d967 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.762972 11318 tablet_replica.cc:333] T 44117a0328e8455d8562a4dbe8a56eb3 P 5604f5b9fef245bca9665c54cdd0d967: stopping tablet replica
I20260709 12:17:31.763589 11318 raft_consensus.cc:2243] T 44117a0328e8455d8562a4dbe8a56eb3 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.764118 11318 raft_consensus.cc:2272] T 44117a0328e8455d8562a4dbe8a56eb3 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.766829 11318 tablet_replica.cc:333] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967: stopping tablet replica
I20260709 12:17:31.767441 11318 raft_consensus.cc:2243] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:31.768303 11318 raft_consensus.cc:2272] T bfc3afe51dc249278d04002e83162998 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.770884 11318 tablet_replica.cc:333] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967: stopping tablet replica
I20260709 12:17:31.771493 11318 raft_consensus.cc:2243] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:31.772331 11318 raft_consensus.cc:2272] T afc42e2e5dab4df88c1c00bf9df243e0 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.774955 11318 tablet_replica.cc:333] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967: stopping tablet replica
I20260709 12:17:31.775578 11318 raft_consensus.cc:2243] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.776074 11318 raft_consensus.cc:2272] T caf9d48542094084a3a015e242077f30 P 5604f5b9fef245bca9665c54cdd0d967 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.778560 11318 tablet_replica.cc:333] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967: stopping tablet replica
I20260709 12:17:31.779165 11318 raft_consensus.cc:2243] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.779677 11318 raft_consensus.cc:2272] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.782160 11318 tablet_replica.cc:333] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967: stopping tablet replica
I20260709 12:17:31.782829 11318 raft_consensus.cc:2243] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.783324 11318 raft_consensus.cc:2272] T 8b6087005dd541ffa46a97375e5ba4a1 P 5604f5b9fef245bca9665c54cdd0d967 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.785714 11318 tablet_replica.cc:333] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967: stopping tablet replica
I20260709 12:17:31.786329 11318 raft_consensus.cc:2243] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 4 LEADER]: Raft consensus shutting down.
I20260709 12:17:31.787420 11318 raft_consensus.cc:2272] T c6dcf02f879841949d494b2232d540ee P 5604f5b9fef245bca9665c54cdd0d967 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:31.789908 11318 tablet_replica.cc:333] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967: stopping tablet replica
I20260709 12:17:31.790483 11318 raft_consensus.cc:2243] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:31.791031 11318 raft_consensus.cc:2272] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 5604f5b9fef245bca9665c54cdd0d967 [term 2 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:31.849315 27071 consensus_peers.cc:597] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 -> Peer 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771): Couldn't send request to peer 5604f5b9fef245bca9665c54cdd0d967. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:32.025753 27000 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:32.100612 27071 consensus_peers.cc:597] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 -> Peer 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771): Couldn't send request to peer 5604f5b9fef245bca9665c54cdd0d967. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:32.122813 26999 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:32.144140 26998 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:32.202216 27071 consensus_peers.cc:597] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 -> Peer 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771): Couldn't send request to peer 5604f5b9fef245bca9665c54cdd0d967. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:32.255542 27071 consensus_peers.cc:597] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 -> Peer 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771): Couldn't send request to peer 5604f5b9fef245bca9665c54cdd0d967. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
W20260709 12:17:32.300835 26997 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:32.512996 27000 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
I20260709 12:17:32.523258 27467 raft_consensus.cc:493] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38218df822dc435eac030105d4f51140)
I20260709 12:17:32.524269 27467 raft_consensus.cc:515] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:32.526618 27467 leader_election.cc:290] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
W20260709 12:17:32.530454 27071 leader_election.cc:336] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:17:32.531490 27072 leader_election.cc:336] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 38218df822dc435eac030105d4f51140 (127.11.13.129:36041): Network error: Client connection negotiation failed: client connection to 127.11.13.129:36041: connect: Connection refused (error 111)
I20260709 12:17:32.531745 27467 raft_consensus.cc:493] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38218df822dc435eac030105d4f51140)
I20260709 12:17:32.531944 27072 leader_election.cc:304] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 508ff162d05e44159e225081ede82882; no voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:32.532531 27467 raft_consensus.cc:515] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } }
I20260709 12:17:32.533264 27458 raft_consensus.cc:2749] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:32.534744 27467 leader_election.cc:290] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771), 38218df822dc435eac030105d4f51140 (127.11.13.129:36041)
W20260709 12:17:32.538394 27071 leader_election.cc:336] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:17:32.538524 26999 consensus_peers.cc:183] SignalRequest failed: Illegal state: peer closed
W20260709 12:17:32.539964 27072 leader_election.cc:336] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 38218df822dc435eac030105d4f51140 (127.11.13.129:36041): Network error: Client connection negotiation failed: client connection to 127.11.13.129:36041: connect: Connection refused (error 111)
I20260709 12:17:32.540402 27072 leader_election.cc:304] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 508ff162d05e44159e225081ede82882; no voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:32.541164 27467 raft_consensus.cc:2749] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:32.567924 27467 raft_consensus.cc:493] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38218df822dc435eac030105d4f51140)
I20260709 12:17:32.568445 27467 raft_consensus.cc:515] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:32.570374 27467 leader_election.cc:290] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
W20260709 12:17:32.573278 27071 leader_election.cc:336] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771): Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260709 12:17:32.574108 27072 leader_election.cc:336] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 38218df822dc435eac030105d4f51140 (127.11.13.129:36041): Network error: Client connection negotiation failed: client connection to 127.11.13.129:36041: connect: Connection refused (error 111)
I20260709 12:17:32.574494 27072 leader_election.cc:304] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 508ff162d05e44159e225081ede82882; no voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:32.575239 27467 raft_consensus.cc:2749] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:32.629040 27467 raft_consensus.cc:493] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 38218df822dc435eac030105d4f51140)
I20260709 12:17:32.631155 27467 raft_consensus.cc:515] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:32.637633 27467 leader_election.cc:290] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
W20260709 12:17:32.649044 27071 leader_election.cc:336] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771): Network error: Client connection negotiation failed: client connection to 127.11.13.130:37771: connect: Connection refused (error 111)
W20260709 12:17:32.650225 27072 leader_election.cc:336] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 38218df822dc435eac030105d4f51140 (127.11.13.129:36041): Network error: Client connection negotiation failed: client connection to 127.11.13.129:36041: connect: Connection refused (error 111)
I20260709 12:17:32.650774 27072 leader_election.cc:304] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 508ff162d05e44159e225081ede82882; no voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:32.651693 27467 raft_consensus.cc:2749] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:32.657681 11318 tablet_server.cc:196] TabletServer@127.11.13.130:0 shutdown complete.
I20260709 12:17:32.740285 11318 tablet_server.cc:179] TabletServer@127.11.13.131:0 shutting down...
I20260709 12:17:32.793102 27467 raft_consensus.cc:493] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 38218df822dc435eac030105d4f51140)
I20260709 12:17:32.794202 27467 raft_consensus.cc:515] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:32.796420 27467 leader_election.cc:290] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:32.799635 11318 ts_tablet_manager.cc:1507] Shutting down tablet manager...
I20260709 12:17:32.801194 11318 tablet_replica.cc:333] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882: stopping tablet replica
W20260709 12:17:32.802469 27071 leader_election.cc:336] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771): Network error: Client connection negotiation failed: client connection to 127.11.13.130:37771: connect: Connection refused (error 111)
I20260709 12:17:32.803192 11318 raft_consensus.cc:2243] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Raft consensus shutting down.
W20260709 12:17:32.803551 27072 leader_election.cc:336] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 38218df822dc435eac030105d4f51140 (127.11.13.129:36041): Network error: Client connection negotiation failed: client connection to 127.11.13.129:36041: connect: Connection refused (error 111)
I20260709 12:17:32.804122 27072 leader_election.cc:304] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 508ff162d05e44159e225081ede82882; no voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:32.804414 11318 raft_consensus.cc:2272] T 58ea17416b1642b2820a501561e9dd68 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:32.805544 27467 raft_consensus.cc:2749] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260709 12:17:32.808350 11318 tablet_replica.cc:333] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882: stopping tablet replica
I20260709 12:17:32.809209 11318 raft_consensus.cc:2243] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:32.810302 11318 raft_consensus.cc:2272] T c8c93363b18b455696ca407b4dbde553 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:32.813668 11318 tablet_replica.cc:333] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882: stopping tablet replica
I20260709 12:17:32.814512 11318 raft_consensus.cc:2243] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:32.815500 11318 raft_consensus.cc:2272] T fd773572db7849348131c09a3a18184c P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:32.818940 11318 tablet_replica.cc:333] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882: stopping tablet replica
I20260709 12:17:32.819581 11318 raft_consensus.cc:2243] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:32.820590 11318 raft_consensus.cc:2272] T 59af62b8772640f79014b776aa0a2906 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:32.823868 11318 tablet_replica.cc:333] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882: stopping tablet replica
I20260709 12:17:32.824756 11318 raft_consensus.cc:2243] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:32.826514 11318 raft_consensus.cc:2272] T 2bccc8c7fe754ec49743359441654476 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:32.829305 27467 raft_consensus.cc:493] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5604f5b9fef245bca9665c54cdd0d967)
I20260709 12:17:32.830354 27467 raft_consensus.cc:515] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "38218df822dc435eac030105d4f51140" member_type: VOTER last_known_addr { host: "127.11.13.129" port: 36041 } } peers { permanent_uuid: "5604f5b9fef245bca9665c54cdd0d967" member_type: VOTER last_known_addr { host: "127.11.13.130" port: 37771 } } peers { permanent_uuid: "508ff162d05e44159e225081ede82882" member_type: VOTER last_known_addr { host: "127.11.13.131" port: 40411 } }
I20260709 12:17:32.831701 11318 tablet_replica.cc:333] T d5cc002263e547a8a5da289d103ea955 P 508ff162d05e44159e225081ede82882: stopping tablet replica
I20260709 12:17:32.833117 11318 raft_consensus.cc:2243] T d5cc002263e547a8a5da289d103ea955 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:32.833906 27467 leader_election.cc:290] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 38218df822dc435eac030105d4f51140 (127.11.13.129:36041), 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771)
I20260709 12:17:32.834479 11318 raft_consensus.cc:2272] T d5cc002263e547a8a5da289d103ea955 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260709 12:17:32.838261 27072 leader_election.cc:336] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 38218df822dc435eac030105d4f51140 (127.11.13.129:36041): Network error: Client connection negotiation failed: client connection to 127.11.13.129:36041: connect: Connection refused (error 111)
I20260709 12:17:32.839767 11318 tablet_replica.cc:333] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882: stopping tablet replica
W20260709 12:17:32.839556 27071 leader_election.cc:336] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5604f5b9fef245bca9665c54cdd0d967 (127.11.13.130:37771): Network error: Client connection negotiation failed: client connection to 127.11.13.130:37771: connect: Connection refused (error 111)
I20260709 12:17:32.840574 27071 leader_election.cc:304] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 508ff162d05e44159e225081ede82882; no voters: 38218df822dc435eac030105d4f51140, 5604f5b9fef245bca9665c54cdd0d967
I20260709 12:17:32.841374 11318 raft_consensus.cc:2243] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [term 3 LEADER]: Raft consensus shutting down.
I20260709 12:17:32.841374 27467 raft_consensus.cc:2749] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260709 12:17:32.844141 11318 raft_consensus.cc:2272] T e85c5e2f2a6b4d31bd83715d05eda1c1 P 508ff162d05e44159e225081ede82882 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:32.847272 11318 tablet_replica.cc:333] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882: stopping tablet replica
I20260709 12:17:32.847941 11318 raft_consensus.cc:2243] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [term 2 LEADER]: Raft consensus shutting down.
I20260709 12:17:32.849228 11318 raft_consensus.cc:2272] T f6a2f90e85c04cf28ba0397d0c0a5b23 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:32.852653 11318 tablet_replica.cc:333] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882: stopping tablet replica
I20260709 12:17:32.853344 11318 raft_consensus.cc:2243] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:32.854490 11318 raft_consensus.cc:2272] T 8b6087005dd541ffa46a97375e5ba4a1 P 508ff162d05e44159e225081ede82882 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:32.857954 11318 tablet_replica.cc:333] T bfc3afe51dc249278d04002e83162998 P 508ff162d05e44159e225081ede82882: stopping tablet replica
I20260709 12:17:32.859066 11318 raft_consensus.cc:2243] T bfc3afe51dc249278d04002e83162998 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:32.860134 11318 raft_consensus.cc:2272] T bfc3afe51dc249278d04002e83162998 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:32.863548 11318 tablet_replica.cc:333] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882: stopping tablet replica
I20260709 12:17:32.864333 11318 raft_consensus.cc:2243] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:32.865319 11318 raft_consensus.cc:2272] T caf9d48542094084a3a015e242077f30 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:32.868963 11318 tablet_replica.cc:333] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882: stopping tablet replica
I20260709 12:17:32.869767 11318 raft_consensus.cc:2243] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:32.870821 11318 raft_consensus.cc:2272] T afc42e2e5dab4df88c1c00bf9df243e0 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:32.873755 11318 tablet_replica.cc:333] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882: stopping tablet replica
I20260709 12:17:32.874500 11318 raft_consensus.cc:2243] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:32.875578 11318 raft_consensus.cc:2272] T 44117a0328e8455d8562a4dbe8a56eb3 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:32.878242 11318 tablet_replica.cc:333] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882: stopping tablet replica
I20260709 12:17:32.878873 11318 raft_consensus.cc:2243] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:32.879567 11318 raft_consensus.cc:2272] T aab7d9b4c30c466fa1e5451e42820d75 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:32.883301 11318 tablet_replica.cc:333] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882: stopping tablet replica
I20260709 12:17:32.884063 11318 raft_consensus.cc:2243] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:32.884828 11318 raft_consensus.cc:2272] T 30203b8c002948178d4966017ba69c86 P 508ff162d05e44159e225081ede82882 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:32.888259 11318 tablet_replica.cc:333] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882: stopping tablet replica
I20260709 12:17:32.889024 11318 raft_consensus.cc:2243] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:32.889824 11318 raft_consensus.cc:2272] T 1baf73a2b0be4194bf53c8731fdb7da2 P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:32.892722 11318 tablet_replica.cc:333] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882: stopping tablet replica
I20260709 12:17:32.893337 11318 raft_consensus.cc:2243] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:32.894104 11318 raft_consensus.cc:2272] T c6dcf02f879841949d494b2232d540ee P 508ff162d05e44159e225081ede82882 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:32.897868 11318 tablet_replica.cc:333] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882: stopping tablet replica
I20260709 12:17:32.898514 11318 raft_consensus.cc:2243] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260709 12:17:32.899379 11318 raft_consensus.cc:2272] T e2c4d9a0aa474d0aafdb7abba0ceadfd P 508ff162d05e44159e225081ede82882 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:32.974916 11318 tablet_server.cc:196] TabletServer@127.11.13.131:0 shutdown complete.
I20260709 12:17:33.030944 11318 master.cc:562] Master@127.11.13.190:40633 shutting down...
I20260709 12:17:33.075397 11318 raft_consensus.cc:2243] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2 [term 1 LEADER]: Raft consensus shutting down.
I20260709 12:17:33.076334 11318 raft_consensus.cc:2272] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260709 12:17:33.076732 11318 tablet_replica.cc:333] T 00000000000000000000000000000000 P d8f564118dbf464586d3dad1c5b403b2: stopping tablet replica
I20260709 12:17:33.103271 11318 master.cc:584] Master@127.11.13.190:40633 shutdown complete.
[       OK ] FilterSoftDeletedTableTest.TestFilterSofteDeletedTable/1 (23431 ms)
[----------] 2 tests from FilterSoftDeletedTableTest (44943 ms total)

[----------] Global test environment tear-down
[==========] 8 tests from 2 test suites ran. (560938 ms total)
[  PASSED  ] 7 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] LeaderRebalancerTest.AddTserver

 1 FAILED TEST
I20260709 12:17:33.177277 11318 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:220: suppressed but not reported on 2290 messages since previous log ~34 seconds ago
I20260709 12:17:33.177538 11318 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:304: suppressed but not reported on 2290 messages since previous log ~34 seconds ago
I20260709 12:17:33.177775 11318 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet_mm_ops.cc:139: suppressed but not reported on 2286 messages since previous log ~34 seconds ago
I20260709 12:17:33.178020 11318 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:600: suppressed but not reported on 9 messages since previous log ~53 seconds ago
I20260709 12:17:33.178385 11318 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/ts_tablet_manager.cc:548: suppressed but not reported on 9 messages since previous log ~53 seconds ago
I20260709 12:17:33.178655 11318 logging.cc:436] LogThrottler Service Queue Overflow: suppressed but not reported on 8 messages since previous log ~220 seconds ago
I20260709 12:17:33.178849 11318 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/service_pool.cc:136: suppressed but not reported on 5 messages since previous log ~218 seconds ago
I20260709 12:17:33.179021 11318 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tserver/heartbeater.cc:646: suppressed but not reported on 2 messages since previous log ~178 seconds ago
I20260709 12:17:33.179267 11318 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/rpc/proxy.cc:239: suppressed but not reported on 37 messages since previous log ~1 seconds ago
I20260709 12:17:33.179548 11318 logging.cc:436] LogThrottler /apache/dev/git/kudu/src/kudu/tablet/tablet.cc:2406: suppressed but not reported on 53 messages since previous log ~22 seconds ago
